Here's how you can perform an Incremental load / Delta merge in WakeupData Connect
Incremental load / Delta merge
Updating your product data as often as possible is really valuable to digital marketers running ecommerce campaigns but often comes with technical challenges that can be both time-consuming and expensive.
Therefore, we have now made it possible to update some of your product feed attributes without needing to generate the whole feed from scratch - you update only what is important and when is important.
Make sense? Let's take an example:
New price changes from the purchasing department can be updated through these incremental loads (updating only the pricing attribute of your feed).
This means that the rest of the product feed remains intact, increasing the speed and cutting down on overall costs of running. This is called delta merge/feeds or incremental loads.
Steps to set it up
- Upload incremental csv feeds on either your ftp server or ours.
If you wish to upload multiple csv feeds with different content. For example, if stock and prices can’t be sent in one incremental csv feed, create two folders, one for prices and one for stock and upload the files there.
We can import up to 20 columns of information from these feeds, this has to include also the identifier of the product
The incremental loaded data can be used across projects. - Map these csv feeds on Connect.wakeupdata.com
- Start by going to preferences and enable incremental loads under Projects
- Then under any given project, you will find a new tab under more options, this will lead to incremental loads, where you can create a new load
- When you have filled in the above and clicked test, you will be prompted with more fields
- When these have been filled and save, you are now done setting it up. The incremental data will now be imported whenever the projects using the data are running.
When you have saved, you will now be able to craft an url that will allow you to import the data. - This url can then be added as a feed on url to any existing projects
Notes
- The data will be loaded in when the url is triggered during the project importing process
- Generally it is a good idea to generate the incremental files on a 15 minute basis. You should not create a new file for every change made
- Once in a while it can be a good idea to send a full set of data, to “clean” all loaded data in case something wrong has happened.
- Don’t send full updates every time, or it will clog up the system and that data won't be updated
- It is often a good idea to use this for prices and stock data updates from ERP system logs
- We do not allow data in other formats than csv files
- During the import of delta data, the data is compared to the data from the last import and separated into four types:
- New: These items are new for delta
- Changed: These items have been in the feed before, but attributes have changed
- Deleted: These items are not in the feed anymore (they could be skipped, or removed completely from the platform).
- Unchanged: These items have been in the feed before and no attribute has changed