Syncing Collections Using Delta Files

Collections help organize products for commerce channels and shoppers by using category source inclusions/exclusions and attribute-based rules.

Product Catalog maintains collection assignments through two background jobs, which evaluate and update product assignment based on attribute or rule changes:

If you are subscribed to Product Catalog’s Collections webhook events, the system synchronizes product updates using delta files. The following diagram shows the sequence of tasks performed during the synchronization process:

The system runs background jobs that generate product update files. Depending on the job status, an event is triggered.

​

Synchronization Process

​

1. Job execution

The system runs one of two background jobs:

• Product Update: Collection Evaluation (re-evaluates product fitment based on attribute updates).
• Collection Update: Product Evaluation (re-evaluates products based on collection rule changes).

​

2. Event triggering

Once a job completes successfully or fails, it triggers an event. The webhook listener must be subscribed to both completed and failed events because failed jobs may still update some products.

​

3. Event delivered

The fabric event publisher delivers the event to the webhook listener.

​

4. Extract the output file ID

The Webhook Listener retrieves the first output file ID from the event payload. If multiple output files exist:

• Only the first file should be processed.
• Additional files should be reviewed to determine if they contain duplicates, sequential data, or logs.

​

Example payload with output files

 {
    "specversion": "v3",
    "type": "pim:jobs.productUpdate.collectionEvaluation:completed",
    "tenantid": "64cbda7e0cf2dcad87b14456",
    "events": [
        {
            "id": "87852cfa-0cf8-4625-8935-20f2292ea0d3",
            "time": "2024-02-07T12:09:21.623Z",
            "source": "pim:job",
            "data": [
                {
                    "id": "65c3705726f919316ba95361",
                    "tenantId": "64cbda7e0cf2dcad87b14456",
                    "name": "Product Job_02.07.2024",
                    "type": "PRODUCT_JOB",
                    "outputFiles": [
                        "65c372f1b683f0539a8c18b1"
                    ],
                    "errorFiles": [],
                    "status": "COMPLETED",
                    "createdAt": "2024-02-07T11:58:15.814Z",
                    "updatedAt": "2024-02-07T12:09:21.618Z",
                    "batchJobId": "afe320dc-fb2f-4550-89d4-3578a5c9d255",
                    "completedAt": "2024-02-07T12:09:21.615Z",
                    "statusMessage": "Job completed successfully"
               }
            ]
        }
    ]
 }

​

5. Retrieve the file download link

Make an API request to fetch the file download link using the extracted file ID.

​

Example cURL request

 curl--location
 'https://api.fabric.inc/v3/product-files/actions/download?fileId=65c372f1b683f0
 539a8c18b1' \--header 'Authorization: <token>' \--header 'x-fabric-tenant-id: 64cbda7e0cf2dcad87b14456'

​

6. Download and extract the file

Download the output file from the retrieved link. The downloaded file is a .zip archive that must be extracted before use.

Retry logic:

• If the download fails, retry with exponential backoff.
• Verify file integrity before extraction.

​

Example signed URL for file download

{
  "location": "https://pim-v3-prod01-us-east-1-files.s3.us-east-1.amazonaws.com/product-batch-job/64cbda7e0cf2dcad87b14ad7/prod01/1707307761501-65c3705726f919316ba95361-product-job-outputFile.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240207T193937Z&X-Amz-Expires=300&X-Amz-Signature=3f33806438e10764832d0a3dd5f2e98d02be31476b5bb94f7805b775744b60d0"
}

This signed URL is a temporary, pre-authenticated link that allows you to download the generated output file. The URL includes security parameters such as an expiration time X-Amz-Expires, an algorithm used for signing X-Amz-Algorithm, and a signature X-Amz-Signature.

Ensure that the webhook listener processes the file right after retrieval to avoid needing multiple API requests.

If the URL expires before the file is downloaded, request a new signed URL by calling the File API again.

​

7. Process the extracted data

If no products were updated, the file contains the message “No products updated.”

If products were updated, the file contains JSON entries listing:

• Product ID
• SKU
• Type (item, variant, or bundle)
• Status (live, draft)
• Collections the product was added to or removed from

Category attributes are inherited by default, meaning sub-collection nodes automatically adopt attribute values from their parent categories. However, users can override these values as needed, allowing for customization at different hierarchy levels.

​

Example JSON entries

{
  "id": "65b293d4bc83fd68411eb0e7",
  "sku": "GZZ19228-150-35_pim_pim_2",
  "type": "ITEM",
  "status": "LIVE",
  "collections": {
    "added": ["65c3e29b8cf6b31a1f1845fb"],
    "removed": []
  }
}

​

8. Synchronize the updated data

Process the extracted product data for further synchronization within the system to ensure that any updates correctly propagate through downstream services.