Improve Performance in Large Bulk Imports

The performance of the import of very large numbers of products can be improved with the use of an alternate plug-in.

This section applies to Open Storefront Framework (OSF).

If you are trying to import very large numbers of products, you can improve performance by using an alternate plug-in, ProductsV2. Oracle recommends using ProductsV2 during your initial load of data. You can choose between the Products and ProductsV2 plug-ins during the import process. The ProductsV2 plug-in takes advantage of multi-threaded processing. The major difference between Products and ProductsV2 is the file failedAssociationsRecordsFile, which appears in an import response when the ProductsV2 plug-in is used as the import ID. For Import Status, SuccessCount and Failure Count do not depend on Category or Add on product failures. As a result, if Category or Add On products are invalid in the passed import data, the products will still be created. After your import, even if the success count matches the product count passed, you should always check failedAssociationsRecordsFile for any association failures. If you find failure records, correct failedAssociationsRecordsFile and re-import.

How to use the ProductsV2 Plug-in

The ProductsV2 plug-in can be used in the same way as the Products plug-in. For more information, see Perform Bulk Export and Import. The following examples show how to use the ProductsV2 plug-in in Standalone and Bundle mode.

Standalone mode code sample

POST http://localhost:9080/ccadminui/v1/importProcess
Standalone mode Request:
{
    "mode": "standalone",
    "fileName": "<filename>",
    "format": "json",
    "id": "ProductsV2"
}

Note:

For CSV request formats, specify "csv" as the "format" value.

Bundle mode code sample In bundle mode, place files to be imported in zip format and place the content.json file inside the zip which describes importing files details. For example: [{ "fileName": "example.csv", "format": "csv", "id": "ProductsV2" },{ "fileName": "example2.csv", "format": "csv", "id": "ProductsV2" }]

POST http://localhost:9080/ccadminui/v1/importProcess
Bundle mode Request:
{  
  "mode": "bundle",  
  "fileName": "<filename.zip>"
}

Note:

For CSV request formats, specify "csv" as the "format" value.

Sample response

{
  "completedPercentage": 100,
  "progress": "succeeded",
  "startTime": "2019-09-24T18:31:45.214Z",
  "links": [
    {
      "rel": "meta",
      "href": "http://example.com:3021/file/v5804828446398654872/import/cEIWX7B43XJVrfwZm7ygfK4nr8_10001/importStatus_ProductsV2.json"
    },
    {
      "rel": "failedRecordsFile",
      "href": "http://example.com:3021/file/v3721096615427138463/import/cEIWX7B43XJVrfwZm7ygfK4nr8_10001/importFile.csv"
    },
    {
      "rel": "failedAssociationRecordsFile",
      "href": "http://example.com:3021/file/v3721096615427138463/import/cEIWX7B43XJVrfwZm7ygfK4nr8_10001/importFile_failedAssociations.csv"
    },
    {
      "rel": "self",
      "href": "http://example.com:3021/ccadminui/v1/importProcess/cEIWX7B43XJVrfwZm7ygfK4nr8_10001?fileName=Bulk_Import100K.zip"
    }
  ],
  "endTime": "2019-09-24T18:35:39.725Z",
  "completed": true,
  "requestStatus": 200
}