Merging data feeds from different sources

[dealotz]

Tweet

Hi,

I am newbie to affliate marketing. I am planning to start a deals website. I have applied for most of the affliates listed above in the other group. I have few fundamental questions about dealing with the feeds
1) I could see that some of the affiliate networks provide us with the dynamic data feeds with the help of API (eg: CJ.com) .. As when we get the feeds from different networks, they would have different XML formats . How can we merge them and display it in the site.

2) There are some networks which provide static data feed in csv or xml format, Do we need to have process to read them and store them in the Database.

Is it like for both the options we need to store the data in the data base and then read from it. Cant we directly hit the API and parse and display in the Website.

Sorry for being naive, Any help is much appreciated,

Thanks
Hari

[D-Mac]

My personal preference is to store the data in a database, then read from that. You can run directly from the APIs, but it means your content is:
a) dependant on the API responding properly
b) makes it more of a faff to clean the data before you display it

[dealotz]

My personal preference is to store the data in a database, then read from that. You can run directly from the APIs, but it means your content is:
a) dependant on the API responding properly
b) makes it more of a faff to clean the data before you display it

Thank you for the information. I see your point using API's.
If we store in the database we can store relevant fields and update them periodically (may be every night) with a script.

How frequently do we need to update the data? I used to have a product comparison site pulling data from google API .. there was a restriction of saving the data into the local DB . Does any of the affiliate networks have this restriction.

Thank you once agian for the help.
Hari

[D-Mac]

No affiliate networks that I know of have this restriction. Updating once a day is usually sufficient, although some merchant feeds are updated less frequently than this.