Fieldable Products and the Add to Cart Field

One of the most exciting changes planned for Ubercore for both site builders and developers is the near total re-implementation of products thanks to Drupal 7's Fields API. The following represents a summary of conversations, midnight brainstorming, and sprint discussion that should be refined into a solid spec. for the first round of Ubercore development.

Right now, Ubercart confuses "what a product is" with "how a product is displayed." When you start out, a product is a node. However, thanks to SKU adjustments, a node might represent multiple products. This then causes problems with reporting, stock tracking, selling some non-tangible goods / accepting donations, and importing / exporting the product database. Furthermore, it makes it difficult to define a decent API for developers working with products and product attributes.

So, the primary thing we wanted to do was to separate a product from its display on the site. This means products will no longer be tied to nodes. The core product entity will be a separate, fieldable entity. It will consist of at least a SKU and a default price and title. Additional fields describing the products may be attached to product entities to form new product types. These fields will take the place of what attributes are now, the main difference being that attributes are being assigned to individual products, not used as selectors between multiple products displayed on a single product node. I imagine we'll work out a way to do things like attach a Filefield to a product so that when that product is purchased the user has access to that file, but that's hardly a core feature.

We're still concerned with how products will be displayed for purchase, though. To that end, we'll probably still either ship with (or create on installation) a product display node type that bundles an Add to Cart field. In edit mode, an Add to Cart field will let you select from among products you have already defined and described in your product database (or add a new one) through a modal dialog. You can just add one product to sell something much like you would through a product node now, or you might add multiple products and specify how they should be combined in the add to cart form for sale. So, if you selected three t-shirt products that each had a different value in their "Size" field, you could have them be displayed on a form that lets you specify the product through a "Size" select list in the same way you would now through attributes. The key difference here is that every variation of a product is stored in the product database, giving us a clear record of what was sold. Anyone familiar with the shortcomings of the product attributes system (both its UI and API) should see the benefits here. Additionally, we figured that the Add to Cart field shouldn't just be restricted to displaying a form but should be open to links, buttons, etc. So, through the field you're selecting what products are displayed on the node (or whatever entity you attach the field to), how the products should be associated, and how they will be added to the shopping cart.

Separating the product from the display gives the site builder or developer more freedom to decide how products get in the shopping cart. For example, sites taking donations or selling non-tangible goods might not want to fool with product nodes at all! They might just want a way to drop a product in the cart and go straight to checkout. With this system, they could do so, and the default title and price on the product let us have a way to represent the product in the cart even if it's not linked to any particular node on the site.

We had some wireframes and whiteboard brainstormings related to all this posted here:

  • [node:59]
  • [node:60]
  • [node:61]
  • [node:62]
Ryan Szrama
Posted: Oct 25, 2009


Amitaibu (not verified) on October 26, 2009

I have taken a similar approach with OG7 - where I've replaced all those different checkboxes and radio buttons with field API - which is very powerful. I'm also using Rules as the event handler, so for example there are no drupal_set_message() -- it's all in the Rules integration (which is not a dependency).

Anyway, Ubercore sounds exciting.

Ryan Ryan Szrama on October 26, 2009

To be clear about how attributes operate / would be defined, I'm cross-posting this from another thread...

Basically, when I think of attributes as they are now, there are those that function as product associations (i.e. attributes that are specifying unique SKUs - the size of shirt) and product describers (i.e. attributes that give the customer a chance to specify something unique about their purchase - the name on a plaque). The description in that post replaces the need for the first by letting you define your products up front and then join them together in a product display around those particular attributes. It doesn't adequately consider product describers, but that's just because we haven't got that far. It's definitely on the radar! : )

alpritt on October 26, 2009

I don't yet have a full grasp of how Fields API works, but I had to develop a lot of bespoke code including a product module in order to deal with the problems outlined above; so I have some definite thoughts on some requirements.

Basically my project required products to be attached not only to a node content type but also a commission (a content type distinct from a node). Actually the product represented a bid for a parent commission object.

What I did was create a base product class. Then two inherited classes for each of a bid product and a node product. These each had additional data stored alongside the basic product data and slightly different methods for adding items to the cart.

Probably also related to this discussion was the requirement to purchase a product in multiple payments (notably a deposit and then payment of the remainder). I won't get into how that was implemented because it wouldn't be flexibile enough for Ubercore. It's a bit of an edge case but I want to try and think how it could be implemented on top of our solution here.

Ryan Ryan Szrama on October 26, 2009

Hmm, you indirectly bring up a good point... how will we handle specifications for things related to products and product purchasing that aren't really fields, like specifying something as paid in installments. At the very least, the existing concept of product features should be easier to implement and target SKUs, but I wonder if something like you need would simply have a separate UI and reference products using some kind of product reference field (I'm thinking autocomplete for anything over 10 products in a catalog or something).

liberatr on November 2, 2009

alpritt brought up something I think will be pretty important to Ubercart going forward, the idea that there is more than one kind of basic product.

If you think about it, certain kinds of actions are really behaviors of the product more than just conditional actions that apply to anything in the store.

The idea of commission-based is one, or an auction-style. These things may not need to be implemented by ubercore, but it should allow people to write them, and PHP object-oriented techniques would help there.

At the very least, a "features"-style system for moving around a product with all of its attributes, CA workflow, etc.

Amitaibu (not verified) on October 26, 2009

> The core product entity will be a separate, fieldable entity.

I think we don't need to define an entity at all - we just need to define the field types. Having another not-here-not-there entity IMO doesn't have any benefit. If we want to force a title, then node are perfect.
Again, like in OG7, when you set a content type to be a "product content type", UC7 will add the minimium required fields for you (e.g. SKU which is a textfield).

minor: About the FileField privacy -- probably http://drupal.org/project/field_permissions .

Ryan Ryan Szrama on October 26, 2009

Well, the whole idea is to not get into the current situation where you're forced to create a node to add a product to the site, especially in cases where the product is never intended to be displayed. Maintaining a dependency on nodes doesn't accomplish that, and it also means we can't control what happens to products as accurately as we would otherwise.

Damien Tournoud on October 26, 2009

We really need a new entity here. The identifier of that entity is the product_id. It doesn't makes a lot of sense to use nodes for that (for exactly the same reasons users are not nodes).

Amitaibu (not verified) on October 27, 2009

DamZ explained it in the IRC, and now it makes a perfect sense:

"products" in the Ubercore sense are not nodes. they are not meant to be displayed as-is to the user; they don't have an author, they don't have revisions, they are not identified by an nid.
in D7, there are absolutely no basis anymore to want everything to be nodes.
products are *displayed* by nodes, using a special "product display" field. one node can display one or more products using this field

Thanks DamZ :)

redben on October 31, 2009

I don't get it. Why would you need to display more than one product on a node ? I'd appreciate if someone could give me an example :-)
Is one meant to understand it as products package/product kit ?

Ryan Ryan Szrama on October 31, 2009

There's the case of product kits, but even then multiple products are together functioning as a single product. The idea is more clear if you think about what a customer presumes to be one product but which in your warehouse or inventory system is separate products. This might be a single style of t-shirt in several sizes or an exercise ball in various colors. You wouldn't want to have to make a separate node for each of the variations, and so Ubercart right now has attributes to allow the customer to go to one node and specify the variation they want in the add to cart form.

The problem this creates is that there is no good definition of what a product actually is. Ubercart doesn't know that that one node is representing multiple variations of a single product, and we have to do some trickery to get it to accept the fact that a single product node might represent multiple SKUs. So, by forcing products to all be created uniquely and then joining them together on display, we can achieve a good record of products on the site and flexibility on the add to cart form.

Ryan Ryan Szrama on October 31, 2009

Yeah, exactly. What's most exciting here is that all the variations will be defined in one product table, and we'll be able to use Fields on them to do it. This will make import/export a lot easier, stock control simpler, "invisible" products easier to configure, etc. I'm really looking forward to it!

Also, definitely keep the questions coming. I think we've discussed a lot but miss things when we put it in the site. The more questions we get, the more we have to define things and figure out how to explain them... every question helps! : )

Emma (not verified) on December 2, 2009

I'm currently struggling with the display of size options for my products - some have 20 or more size options which are a long and unwieldy drop down box...

However, the sizes would make sense in a grid (they are bras so each size is a combination of chest size and cup size)- would this be possible to code this with this new system.

It's also complicated as some sizes will be "instock" some "coming in x days" some "out of stock" so ideally I need an extra setting somewhere to keep track of this.


Shaun (not verified) on January 21, 2010

Haha, I also run a store with bras, and yeah, those size attributes are just a whole massive step beyond what a "normal" item might typically have.

It's what... 6x10 = 60 sizes? More?
Then add in combinations with colours.

Bra sizes deserves its own contrib module! Perhaps when D7C is released I'll try it myself.

splash112 (not verified) on November 2, 2009

So every combination off all attributes lead to a new product?

I see some restrictions in the system on how I use (and hope to use in the future) Ubercart now. 3 different select lists with 5-10 options and some checkboxes should be doable, hopefully without creating scores of products and SKU's.

The idea I am playing around with now would consists of a big product array which contains as much possible information as necessary. Attributes could change or add values to this array on multiple levels.
Think for example about a USB stick store that sells sticks with 512MB, 1GB, 2GB and 8GB. Ideally, this store should be able to add the USB cost/price and SKU at a level what would now be where one sets a default price for options (note, product would get 2 SKU's, one for the USB housing and one for the USB). For a T'shirt in a different size one would rather change the SKU at the product level (only 1 SKU) and maybe the weight at a higher level (smaller t'shirts weigh less?).

Hopefully Ubercart keeps it's strength in doing almost anything, wouldn't want to see it get trimmed down to one of the many very restricted shops available already.

Ryan Ryan Szrama on November 2, 2009

I think questions of product entry can be handled at the UI level. It may in fact be cumbersome to have to submit a form for every variation of a product you have, but that doesn't stop us from using the underlying data model. We can just create a multi-entry form, perhaps, to speed up the product entry process for any given product type. Such a form could even copy through the title information or even accommodate some SKU patterning so that [USB-*] turned * into an autoincrementing variable or something.

When it comes to preparing these products for display, you'd attach them to a node through a product entry field with an add to cart display formatter. (Using CCK terms here.) The node body can just be one blob of text about the product "line" while the add to cart form can automatically create those select lists for you based on the products you selected on the edit widget.

Being able to define new products at the point of display can still happen through a modal dialog, but it shouldn't be the primary place such a UI exists. It also shouldn't be possible to create new variations of SKUs at the display level that aren't recorded somewhere in the product database. Right now, there's a uc_products table that holds the base product data defined on the node edit form and another table that holds SKU adjustments based on arbitrary attribute option selections. The data model / underling DB architecture is quite hideous and hard to maintain and scale.

So, what we're doing is primarily adjusting the underlying data model, but that doesn't mean we still can't have simply to use UIs for entering product data.

Garrett Albright on November 12, 2009

I wonder if you might be on to something with the patterning idea. Instead of each variant having a separate SKU, the user could say that the main product has an SKU and each variant automatically adds a fragment to the SKU; so I define the MEMSTICK product with that SKU, define various sizes with SKU fragments, and define various colors with various SKU fragments, and then the system could automatically put together MEMSTICK-2GB-RED and MEMSTICK-1GB-BLUE and such.

Of course, this should all be optional, so if the user really does have to manually give each variant an arbitrary SKU, they can do so…

Tim on November 8, 2009

I have seen a need for a package builder, I think the items discussed here will make it easy, but I wanted to give an example and ask for clarification.

In this example all of the individual products are also available for sale.

Package is $45.00

Select one of these three hats
Hat A - Hat B - Hat C

Select one of these three shirts
Shirt A - Shirt B - Shirt C

Select one of these three misc items
Mug - Poster - DVD

AeM (not verified) on November 9, 2009

what about multilingual shops ?

a node is a single-language version linked to other languages with the i18n module...

such as : original node (english) - french translated node - japanese translated node - german translated node

in that example : 4 nodes but the same product, thus the same stock level, discount level and so on to use !

Michael M on November 9, 2009

This is exciting. I woke up this morning determined (after doin' a little wrasslin' with attribute deserialization) to put this on the table here, only to find that you guys are well beyond my feeble glimmerings of concept.

Besides needing to do various manipulations of attribute values for a single "thing" purchased (mine are events), but sometimes I want to kit another "thing" with that first item, but flexibly. The buyer can pick a quantity and also can buy that thing separately. Example would be dinners with a race entry. Buy em at entry time or buy em separately.

This approach will be very helpful.

Ryan Ryan Szrama on November 10, 2009

Just to keep all the information accessible through this thread, I've incorporated the feedback here in a blog post that also includes the attached image as a way to visualize the differences between the current system and how I think it should work out.

CpILL on November 20, 2009

just discovered this site. I'm glad to see the idea of separating the Node and the 'Product' is being taken up. Now where over that conceptual hurdle, execution is important.

Every product will still need a Node to represent it to the customer, but this coupling should be loose so you can have many products sharing a Node via a many-to-many relationship. The main reasons are:
- i18n i.e. Nodes work with Drupals existing translations system.
- Stock tracking
- import/export of data
These are issues I've had on nearly every UC site I've made and would seem to be in the general used case.

I would try and keep the core Product Entity to the minimum required to pass thought checkout. So just a SKU + Price I'm thinking. Shipping, file attachments, kits etc should be separate modules. You should think about the major use cases (perhaps do a questionnaire about what product configurations are more common on the community site), but keep the core functionality clean and minimal and with many, consistent hooks to make it easy to build on.

On the code level I'd make the Product Entity an Object for easy passing around in the system. Perhaps even allow developers to inherit and override functionality as needed if they want to do some custom business logic (which they always will want). This will give the product a API which is desperately needed. And Product Classes as actual Classes! (crazyness, I know). The Product Class UI could actually generate a PHP Class (like other frameworks). I think there should be more of this in Drupal.

Anyway, thats my 2cents. Looking forward to how it pans out :)

Ryan Ryan Szrama on November 20, 2009

Thanks for stopping in, CpILL! I definitely look forward to your feedback (and code? :) moving forward. I'm definitely for using objects in the Ubercore, and I like the way you're thinking regarding additional product functionality. Your experience with Ubercart's pain points is invaluable. Feel free to ping me if you want to get in at a lower level... we're definitely pushing a collaborative development process with plenty of room for sub-maintainers and extra eyes.

CpILL on November 27, 2009

Hey Ryan,

If you guys do some design doc's before you start coding (even just some Class diagrams) I'd be more than willing to review and give some feedback on em.

Shoot me an email or use the contact form at www.tsd.net.au/contact or put them up here for community review even. I've alowayd found a little bit of thinking at the beginning saves a lot of code in the end:

"Thus spake the master programmer: "A well-written program is its own heaven; a poorly-written program is its own hell."
- from the 'Tao of Programming' http://www.textfiles.com/100/taoprogram.pro