Shipping Requirements Have Changed

Heretofore, shipping has focused on exposing a simple yet effective API for connecting rating and shipping companies to the checkout process. Many stores eschew real-time rating and instead opt to offer flat-rate shipping or free shipping on orders. This entirely removes the need for rating. For those who do use real-time rating, the existing process offers a simple set of tools that are only adequate for simple shipping needs.

However, since the original release of the Shipping module, the ecosystem and community have changed. Functionality that could be centralized has been replicated in module after module, usually as a one-off change. 3rd Party Logistics providers, who store products and ship orders on behalf of businesses, are giving their customers the ability to manage and ship orders like their bigger competitors. It’s not uncommon anymore to ship certain products from one or more of these third party logistics providers. Being able to intelligently group products into less shipments by the cheapest or closest warehouse can have a huge impact on the bottom line. Shipping things like chemicals or bulldozers underscore the need for handling complex requirements, rate quoting, and itemized reporting. The existing Commerce Shipping module fails to address these things.

So let’s talk about Shipping 3.x.

Shipping 3.x Goals

Shipping 3.x has two main goals, summarized here:

1. Reduce customization effort. Customizations currently involve rewriting parts of every carrier used for shipping.
2. Reduce code duplication. Incorporate common functionality across current shipping modules.

In other words, the goal of Shipping 3.x is not to implement a wide range of functionality. No, primarily, this release attempts to make the developer experience better and make it possible to implement complex shipping requirements with less code and better “core” support for the types of requirements we see on a daily basis.

The release of Commerce Shipment was a huge step forward. It lays the ground work for breaking up an order into actual shipments. Commerce Shipment, making a shipment a fieldable entity, simplifies solutions to many common problems:

1. Storing tracking numbers.
2. Representing multiple shipments on a single order.
3. Packing slips and pick lists.

It logically follows that Shipping 3.x would take advantage of Commerce Shipment and build upon it to create a more robust solution. At the core of that is the new rating engine. Which the rest of this post will cover.

Commerce Shipping 3.x Overview

To summarize what Shipping 3.x looks like, here is a side-by-side comparison of Shipping 2.x on the left and 3.x on the right:

Current workflow

Configuration

1. Manually configure shipping carriers and their methods in the backend.

Rating

1. Rate the entire order if Rules conditions allow to do so (how it rates is dependent on the rating engine).
2. Show the user the rates provided by the rater(s).
3. Once the user selects a rate, add a shipping line-item to the order.

While this works fine for small, single origin and destination configurations, businesses looking to differentiate themselves or better serve their customers need more options. Things like multi-warehouse fulfillment, shipping to multiple destinations, and unique packing requirements often require custom one-off customizations to the entire fulfillment workflow.

The new, proposed workflow

Configuration

1. Optional configuration of the shipping workflow (ships backwards compatible).
2. Manual configuration of raters and shipping workflow.

Shipping

1. A new (optional) pre-shipping pane that allows for user customization of the order. Think multiple shipping destinations or gift selection.
2. Splitting - orders are split up based on their shipping location and their shipping destination.
3. Packing - Orders are packed into boxes/shipments.
4. Rating - Individual shipments are rated with shippers.
5. Show the user the rates provided and, if necessary, allow them to choose the shipping methods and rates they want.
6. Add shipping information to the order.

Order Completion

1. Provide post-order support by allowing shipping engines to expose labeling and tracking numbers directly back to the shipping module.

Nuts and Bolts of Shipping 3.x

The New Rating Engine

At the heart of Shipping 3 is a new, multi-step, pluggable rating system. The multi-step rating engine allows for strategic management of shipping by allowing for plugins to only modify the steps in the rating engine that are required to achieve their goals. The new rating engine will have three specific hook points:

1. Splitting
2. Packing
3. Rating

Splitting

During this step orders can be split based on both the source and destination. Here, a multi-warehouse module might reach out to an API to determine the best sources for a given order. The shipments created in this step would will have all of their products split up. Likewise, a business which allows for multiple shipping destinations would be able to split up the orders based on the destination addresses populated in a checkout pane.

More examples:

1. Single-origin products could have an address field added to them and a plugin used to split the order by looking for that field on the products within the order.
2. Products fulfilled by 3PL would be split apart from the other products in an order.

By default, only a single shipment will be created on all orders.

Packing

During packing, a shipment will be further split into one or more boxes. Here, a plugin could allocate products by configured or automatic manipulation. Shipping costs could also be added based on packing material, cost assigned to each box, information added to generate packing and pick lists. For businesses that ship in specific quantities or have exact shipping requirements, this hook will make it easy to simply split up those products into boxes.

To accomplish this, we’ll be exploring the use of Commerce Box to provide an interface and management system for box types as well a generic packing algorithm. Ideally, each rating engine would provide their preferred boxes, but businesses could create their own box types.

As far as the actual packing is concerned, there are a couple of potential suggestions such as Commerce Packing and Packaging that both have starts at implementing packaging algoritms. All that would be necessary is to implement the proper hooks and allow them to work on the shipments instead of the order itself.

By default, only a single box will be created on all shipments and all items will go in that box.

Rating

Finally, the carriers (FedEx, UPS, USPS, etc) would need to be contacted for rates of any applicable shipping services. Or you can use a table rater. Each carrier will need to update to the latest version of the API to rate all packages and shipments in the order (as opposed to rating the order en toto).

Configuration

1. Out of the box, we can make configuration a bit more straight-forward by requiring an admin enter a store-evel origination address instead of making the owner enter the configuration for each carrier.
2. Configuring the rating engine will likely be a simple UI at first to choose which plugins will be used, and like Shipping 2.x currently works, Rules will be utilized to enable and disable pieces based on conditions.

Customer Experience

1. As mentioned earlier, a pre-splitting pane can be made available so that a store who wants to be able to provide a custom destination system could do so. You could upload a CSV or have a click-and-drag system where orders are split up based on user preference.
2. Additionally, once the shipments have been split, the shipping pane will be refactored to allow for shipping experiences give the user as much control as you’d like to have. They could choose rates based on each package (e.g. Package 1 ships Ground and Package 2 ships Next-Day Air) or a single shipping method (Ground vs. 2-day).

Backwards Compatibility

Shipping 3.x needs to (at least initially) continue the trend of simplicity. All that should be required out of the box is to install Commerce Shipping (+ Shipment) and a 3.x-ready rating engine. By default, the splitting step will just create a single shipment. Admins will be responsible for creating a origination shipping profile (using the Shipping configuration area). This will serve as the source. The destination will be the shipping profile on the order. This shipment will consist only of physical items that are marked as being shippable. Packing will be “dumb” (throwing every item into a single, rater-defined box). Rating will add the cumulative weight, and then rate the single box.

Since there will be only one box and therefore only one shipment, the existing pane which shows rates should work exactly as expected. As will the existing shipping line item.

Shipping 3.x release timeframe

It’ll ship when it’s ready!

But seriously, we’re currently aiming for a beta release at Drupalcon LA. At a minimum, this will implement the new rating engine, the final release should be “drop in ready” for new sites.

How you can help

There are 3 ways you can help:

1. Code: As code begins to be released and the API updated, carrier engines and other modules using Shipping 2.x will need to be updated. Check the issue queue as well.
2. Test: Try it out, and provide feedback.
3. Sponsor: If you think the changes in Shipping 3.x would help your organization, join us in building 3.x by sponsoring the work directly.