Importing from poorly generated data feeds in CSV format

Firstly I've got to say that what you guys are doing is awesome. The thing is I'm struggling to do automated imports (I've been looking into Feeds module for this) but I need to know if what I have to go on with i correct. I am trying to set up a dropshipping business with a lingerie wholesaler who supply CSV feeds. The thing is, that they supply a CSV file with a SKU field which has the same value for all of the attributes available for a particular product. For example, if they have a shoe with 5 sizes and three colours they supply 15 products all with the same SKU with the only difference being the "Attribute1" field and the "Attribute2" field.

Now I know this seems wrong and that's my major issue with them - there is no GUID or anything unique per line in the CSV file. So I've written a PHP script which basically takes their CSV output and creates a new CSV file with the following fields:

Name, Manufacturer, Catalog, Description, Image, SKU, List price, Cost, Sell price, Shippable, Attribute1, Attibute2, Sub-SKU, Active, Stock, Threshold, Sale Item

The script creates a unique sub-sku by combining the sku with a hyphen, then the value in attribute1 then (if it exists) another hyphen followed by attribute 2

The real pain in the butt is that instead of having / supplying a size field and a colour field separately, they just have "attribute1" and "attribute2" fields where attribute1 is mostly a size designation, with attribute2 being a colour (or empty) - OR and it's a big OR attribute1 is a colour field when there is no size and the size is "generic or one size fits all".

Also because of the nature of their business, new attributes could be added at anytime with exotic names. For instance, a new dress could come out with a "attribute1" (yeah I know) value of "petite" where it does not exist already as a size option or "Flaming Red" where that was also not a colour option either. Or both!

So far I have been dealing with Ubercart in Drupal 6 and I am looking into ways of automating product information / images / attributes but most importantly STOCK as this has so far been non-automatic and I have had the misfortune of selling items which have been out of stock due to the fact that updating so far has been a manual process modifying the feed in OpenOffice.

Is there a way of Ubercart dealing with their poor (but as said above able to be manipulated by me) feeds
Can Commerce with Drupal 7 help here?
Is there any good tutorials on dealing with "unknown or new" attributes?

Any help is much appreciated as always

Posted: Aug 24, 2011


pixel8 on March 1, 2012

Hi I've taken a look at the examples and all looks promising. The only thing though is the possibility of new / unknown attributes being encountered. As far as I can see, attributes / options are handled by a select field with allowed values. So all of the possible options for colour / size already have to exist for the import to work properly. Is there a solution for new / unknown options (colour / size) being encountered in a CSV file?