Overview

Optimized data management is at the core of every successful e-commerce operation. For fabric users managing extensive product catalogs, finely tuned data ingestion is paramount.

Adhering to fabric’s best practices will ensure the fastest processing speed, optimum resource management, and enhanced accuracy when importing your data.

This topic covers subjects such as file size restrictions, types of import actions, reconciling errors, and most importantly, the best method of data ingestion: delta updates.

File Size and Upload Guidelines

Before you upload your first file, it’s important to understand file size restrictions and how fabric handles files that exceed those restrictions.

  • No files larger than 300MB Limit the size of your uploads to 300MB.
  • Split files larger than 300MB into smaller ones Splitting large files into smaller ones before uploading them is the quickest way to import large amounts of data. For fastest processing, the ideal file size is between 80-100MB.
  • Parallel processing fabric can process multiple files in parallel. The number of parallel files depends on your package. When the limit has been reached, subsequent files will be in a “scheduled” status until moved into the queue. Reach out to your account representative to learn more.
  • Automatic file chunking is available fabric can automatically chunk files larger than 300MB into smaller files to improve performance. This feature is only available in select packages. Reach out to your account representative to learn more.

Delta Updates

A delta update involves transmitting only the changed data fields when making an update. This is in contrast to the more traditional “full feed” updates that send the entire dataset. By sending only the changed data fields, fabric can process updates without reprocessing unchanged data.

Delta updates are the preferred method for all uploads.

Delta updates vs. full feed updates

Full Feed Data UpdatesDelta Data Updates
Resource UsageRequires more resourcesRequires fewer resources
Processing TimeLonger processing timeShorter processing time
Data TransmissionTransmits entire datasetTransmits only modified data fields
Network BandwidthConsumes more network bandwidthRequires less network bandwidth
StorageRequires more storage spaceRequires less storage space
Error HandlingProne to errors during full data transmissionLess prone to errors due to focused updates
ScalabilityLess scalable for large datasetsMore scalable, especially for large datasets
Data AccuracyPotential for data redundancy and inconsistencyEnhances data accuracy by focusing on changes
Operational EfficiencyLower operational efficiency due to higher resource consumptionHigher operational efficiency due to optimized resource usage
Incremental UpdatesUpdates entire dataset each timeUpdates only modified data fields incrementally

Ways to Import Data

You can import data into fabric using the following methods:

  • CSV import via API
  • Import via RESTful APIs
  • CSV import via the Copilot interface

The import method you choose is up to you, but in each case, uploading smaller files and using the delta update method will result in quicker processing, better resource management, and a higher degree of accuracy.

Data Formatting

It’s crucial to make sure your dataset is accurate and compatible with fabric’s formatting before initiating the upload process. Validate your data to avoid errors by reviewing the file to identify any changes since the last upload and confirm that the data structure and format are correct. See the following pages for formatting guidelines:

Import Actions

The actions you use when importing items, bundles, categories, and collections tell fabric how you are modifying your data. The following actions are available:

  • UPSERT: Creates a new product if the product doesn’t exist, or it updates an existing product.
  • CREATE: Creates a new product.
  • UPDATE: Updates existing product.
  • PUBLISH: Publishes an existing product that was in draft state. The product is published to the storefront.
  • UNPUBLISH_KEEP_DRAFT: Unpublishes an existing product. If the product already has a draft version, the live version is unpublished and discarded. If product doesn’t already have a draft version, the live version is unpublished and saved as a draft.
  • UNPUBLISH_KEEP_LIVE: Unpublishes an existing product. If the product already has a draft version, the draft version is discarded.
  • DELETE: Deletes the existing product.
  • ATTACH_VARIANT: Adds variants to an existing parent product. The variant column holds the variant to be attached to the SKU.
  • DETACH_VARIANT: Detaches existing variants. The variant column holds the variant to be detached from the SKU.
  • CHANGE_CATEGORY: Updates the category of existing product.
  • ATTACH_CHANNELS: Appends listed channels to the product, allowing it to be available across various sales channels. You can specify the channels to be attached in the Channels column. After attaching channels, users should verify the attachment post-action to confirm the successful addition of channels to the product.
  • DETACH_CHANNELS: Removes listed channels from a product. Users specify the channels to be detached in the Channels column. This action is useful when a product needs to be removed from specific sales channels while remaining available on others. Fabric users should verify the detachment of channels post-action to ensure the desired channels are removed from the product.

Reconciling Errors

If there are errors during processing, download the error file and review each error to identify the problem. Correct the errors by updating the CSV file with the necessary changes and validate the corrected CSV file before re-importing.