The Case for Commerce_SP

This is pretty long but if you are interested in subscriptions / recurring and are a drupal developer its worth a read and more importantly a comment / feedback please.

A while ago I needed to create a subscription based website. The user would be subscribing monthly/yearly for access to premium content, but we were not going to save cards on file and we didn't want it to be automated billing. We wanted notification emails to be sent out prior to the subscription ending telling them to buy a new one. When they did, their subscription time would be extended.

Very straight forward use case for the commerce_sp module. For those that have not used it, you basically create subscription products with a length of time. There are rules for start, stop, continue subscriptions where you can do stuff like adding/removing roles, etc. The expiration is controlled by a date "validity" field attached directly to the user entity.

This works pretty fantastic if you want to subscribe to a site and there is only one membership level. But what if there are multiple membership levels? How do we keep track of the different roles? How can that work with just a single validity field?

After a bof at Drupal Camp Atlanta recently on this topic we discussed some of the other modules out there specifically commerce_recurring and commerce_licenses.

After that bof, we came to a conclusion that for these types of subscription based sites, we might be better off using commerce_recurring so I started looking into it.

With recurring you have a subscription/recurring product type where you can define a length, recurring length, and end time. On purchase of the product, a subscription entity is created that keeps track of the recurring product, the order, etc. Then on cron, when the subscription is expired, a rule triggers that will automatically create a new order with the same product, and assuming you have cardonfile installed and configured with the rule, automatically bill. This sounds fantastic, even without the card on file, I can just email them the url to the order checkout and have them do it manually. But here is where I ran into problems with this solution:

  • If I buy a subscription to something, in my example the sites premium content, and then a day later buy another subscription, I would expect my subscription entity created the first time to just extend that time similar to how commerce_sp would extend the validity.
  • With the above in mind as is, most of the time I would be buying access to a role. If I add a role on subscription start, even when I buy a second time and I already have the role, when the first expired I would then lose the role (since I'm not doing auto pay) even though I already purchased a second subscription to extend my time.

With that in mind I am now wondering if there is still a use case for commerce_sp. I figure we have two ways to proceed:

  • I don't know of anyone that is using commerce_sp and is NOT adding a role on start of subscription. With that in mind we might as well add a role referencefield to the product type so we dynamically add/remove the role out of the box without having the user modify the rules. If it doesn't have a value, no problem, no role.
  • Instead of having a single validity date field on the user we need to either:
    1. Use field collection to keep track of product => validity relationship on the user entity so if I buy a month of membership A and a year of membership B, those dates are tracked independently of each other and start/end with the appropriate actions.
    2. Abandon the fields on user approach and use a subscription entity approach like commerce_recurring does, but if I buy the same product my validity should extend, not create a new entity. Entity revisions can help keep track of changes like this for historical data. I rather like this approach but is obviously a lot more work.
  • Currently commerce_sp uses scheduled rules to execute the components that end subscriptions. In the very first site I did I used the same platform to schedule notifications as well on start/updates. I think again I rather like the cron approach better. Still have components but let cron execute them, not scheduled rules that for some strange reason could accidentally get deleted?
  • Long term thinking: As a module maintainer I cringe at the thought of users installing a module I contribute to and immediately start modifying and overriding stuff like rules. I don't know how to maintain updates to uses with this being the case. I think one idea here is instead of having/telling users to modify the start/end/update rules, we tell them to create components for what they want to happen then they can configure which components to execute on a global and product level. Entityform does something similar with its notification components and its rather convenient.
  • Speaking of notifications: I have yet to find a good way to create, maintain, and "schedule" notifications for these subscription products. For example emails to go out 1 month, 1 week, 1 day before expiration, etc. I was thinking about creating a notification entity type that had a title, body (email content) and a interval field. Then each email would be an entity and cron could send em. Add a product reference field and you have per product notifications. Any thoughts? Maybe in a separate discussion?

There are my thoughts. I would be far more comfortable upgrading commerce_sp than I would commerce_recurring just because of my experience with the former. If enough of you say, no we can do that with recurring then sure lets dig in. I would love to hear others thoughts and experiences with either or modules.

Posted: Nov 5, 2013


dwkitchen David Kitchen on November 6, 2013

Great to read you post and get some feedback on Commerce Recurring (v2).

We (Pedro & I) put some thought in to how we could create a more universal solution for recurring 'things' and how it could work with Card on File, which I was working on. We also made it so you could recur a whole order not just a product

I implemented it for a subscription site and had some of the issues that you mentioned. Although some of this is business logic:

A subscriber could not buy a second subscription or extend outside of the normal subscription process. So someone with the 'subscriber' role can't buy the subscription product.

The subscription product had a role reference field and I used role expire and rules to give the role on payment. I added on 3 days extra from the recurring entity expiry date to allow for late payment.

I think it would be possible to do the 'top-up' subscription method with the recurring entity by changing the expiry/next due date and extending the role expiry if the user already has a recurring entity for that product.

mrconnerton on November 6, 2013

We did think about implementing the business logic of not letting the user buy a subscription they already own. The issue I had with that was the time between expiration and purchasing again would remove that role. Even if you give them extra time with the role, the user can't purchase the subscription until that role is gone.

As part of this inititive I was going to see how hard it would be to extend subscriptions with commerce_recurring if they exist instead of creating new ones.

As bojanz pointed out in irc, this particular use case isn't "recurring" since we don't want to auto charge people, and technically don't want to auto create orders. Its more of a "hey, don't forget to pay for this product again". The module is just the tool todo "stuff" on start/end/update of a subscription and keep track of that subscription data and history.

dwkitchen David Kitchen on November 6, 2013

I forgot to add one of the limitations that also needs work is with 'subscription' payment gateways.

Card on File is designed to work where it initiates the payment. However with some payment gateways you set the subscription up at the start e.g. PayPal or GoCardless for Direct Debit in the UK.

In this case the payment gateway does the subscription charge and needs to be reconciled against the order that recurring creates.

mrconnerton on November 6, 2013

After the bof we had and the understanding I got from the modules, I am in the boat of letting commerce be the manager of subscriptions and recurring paying. As far as the payment gateway is concerned you are just doing a "one time" charge for each payment via card on file.

Implementing each payment gateways subscription apis sounds messy.

rerooting on November 11, 2013

Over the last year I've built some very complex custom recurring subscription/billing functionality for a few projects, and I think I've gained some perspective on this situation (as it seems others here have).

I've found that commerce_cardonfile and commerce_dunning are key componets of any subscription system.

Furthermore, I've found that commerce_recurring 2.x is a wonderful and highly functional *example*, and if you are able to employ it with minimal customization/tinkering for your project, then you are really lucky!

Sometimes, you can simply just be keen with your use of fields on particular products and the way rules is set up. You can even expose some of the commerce_recurring fields on the add to cart form (for example, I've done this with Open CRM while storing the recurring entity as a party dataset for a simple, civicrm-esque recurring donation form where you can choose the frequency and amount, and the til date. Yay for commerce recurring!).

In other cases, you may want to take a long look at the rules actions/conditions and utility functions that commerce_recurring provides. Is the business logic really running up against what you are trying to do? In these cases, you may be able to write a simple custom module that introduces additional rules components with the proper business logic, and maybe some new field bases/instances (i.e. maybe a common set recurring date for all orders rather than intervals). This can sit alongside commerce_recurring, and you can just disable the rules conditions/actions that you need to replace, and drop in your new ones. This way, you can leverage some of the utility functions, the cron hook, and other niceties that commerce recurring provides as well as the recurring entity, and all the handy default rules, etc.

In some cases, you may be better off building your recurring order/product module itself from scratch, while still leveraging cardonfile and dunning which cover a lot of very common territory very nicely. For example, I had a very complex case that involved adding line items to existing orders, the most forgiving billing system ever, conditions where orders where managed by people on the ground via party_og access rather than through cardonfile, and such intense use of open crm-driven logic with multiple subscription types that using commerce recurring altogether was implausabile - so I instead just used fields from party's profile2 datasets that staff and membership were already editing, rather than throwing the recurring entity into the mix anyways.

All you need to do in your custom module is provide the proper rules actions/conditons/etc to trigger the functionality of the commerce dunning and commerce cardonfile modules that you need. Though they can be used with recurring, they are most certainly not married to it. Sprinkle in some custom notification messages using message where need be, and you are done!