Commerce 2.x Stories: Taxes
"Why doesn’t Commerce/Magento/$otherSolution handle my taxes properly? That’s the most basic feature!” - many people, often.
When it comes to eCommerce, nobody likes taxes. We expect taxes to “just work”, so we can finish our projects and get on with our lives. At the same time, no other topic is as complex.
Selling online puts us at the crossroads of different (and sometimes conflicting) laws with many rules and even more exceptions. All eCommerce systems provide the basic tools (“Define your tax rates and specify when to apply them”) and make the site developer responsible for tax compliance. The developer usually passes that responsibility to the client, sometimes implicitly. The client consults an accountant, sometimes. But the buck has to stop somewhere, and it often comes back to the developer, 5 days after launch.
As taxes become more and more complex, there is a need for smarter tax handling, where the application does more and the site administrator less. In the Commerce 1.x lifecycle we’ve built the commerce_vat module to handle the more and more complex VAT taxes. For 2.x, we’re bringing this approach back into core, and releasing several libraries to share the solution with the wider PHP community.
Problems and potential solutions
What are my tax rates?
Each country defines one or more tax rates. There is usually a standard rate which applies to most products, and one or more reduced rates which apply to special products like wine or ebooks.
Most eCommerce systems provide no predefined tax rates. The site administrator / developer must research the rates that must be charged, and enter them into the system. Easy enough, right? If the store operates from Serbia, we create the Serbian tax rate(s), apply them if the customer is from Serbia, call it a day. Unfortunately, most stores operate in countries that make this more complicated.
Canada has 13 subdivisions (10 provinces, 3 territories), each with their own tax rates. If the customer is from Canada, the store must charge the tax rates defined by the customer’s home province/territory. This means that all 13 sets of tax rates must be defined for Canada.
The EU has 28 countries. A French company selling t-shirts will charge French VAT to EU customers. If the French company sells more than 35 000 EUR of t-shirts to Belgium per year, it must start charging Belgian VAT to customers from Belgium. From January 1st 2015, for all digital products and services sold inside the EU (e.g. hosting, ebooks, elearning, memberships) the customer’s VAT rates must be charged. Belgian customer - Belgian VAT. German customer - German VAT. This means that the store must know the tax rates of all 28 EU countries.
Other countries are also trying to impose charging their own taxes when digital products are sold to their residents, for example Norway, South Africa, and soon Japan. While it is currently unclear how much power they have to enforce this, trade agreements will make this a reality and add those rates to the list of ones to know and maintain.
We (the PHP eCommerce community) need to create&maintain a dataset of known tax rates.
We can do this for all countries except the US (where tax rates vary even by zip code, and are therefore impossible to predefine). Users will then be able to simply import the tax rates they need.
We also need to provide resolver classes with territory-specific (EU, Canada...) rules for selecting the right tax rates.
This allows the system to automatically choose the right strategy based on the customer and store location. More on this later (under "Resolving the applicable tax types and rates").
Tax rates can change.
France has 4 different tax rates: Standard, Intermediate, Reduced, Super Reduced. On January 1st 2014, the Standard rate changed from 19.6% to 20%. The Intermediate rate changed from 7% to 10%.
If a store needs to care about several tax rates, chances are that at least one of them will change in a given year. Since eCommerce systems don’t provide a data model to handle this, it means spending New Year’s Eve with one’s finger on the submit button, so that the tax rate percentage is changed at midnight, and all customers pay the correct tax rate.
Each TaxRate object must reference a set of TaxRateAmount objects, each containing a percentage and its start/end dates. The TaxRate should be able to return the correct amount object for a requested date.
This allows the system to reference tax rates without fearing about percentage changes, since the actual percentages are stored one level below.
We can group the TaxRate objects using TaxType objects (French VAT, Australian GST, etc), containing an admin label and data common to all rates (e.g, the rounding mode, other information needed for calculation).
Resolving the applicable tax types and rates
In order to start implementing classes for resolving tax rates, we must first define the tax type “zones”. The zone specifies the territories where the specific tax type and its rates are in use. Common sense tells us that French VAT is used in France, German VAT is used in Germany, etc.
But is French VAT used only in France? No, it is used in Monaco as well, which is a separate country. The UK VAT is also used on the Isle of Mann, a Crown dependency, but not an EU member. German VAT isn’t used on the island of Heligoland and the town of Büsingen (where Swiss VAT is used), but it is used in two Austrian towns: Jungholz and Mittelberg. Austrian VAT is then used in all of Austria except in Jungholz and Mittelberg.
So much for common sense. And these are only some of the examples.
Each TaxType must have a matching Zone specifying the territories where the TaxType is in use.
Once we know where the tax types are used, we can implement our resolvers. Each resolver is a class (tagged service in Symfony, plugin in Drupal) that takes a taxable object, customer and store information, and tries to find the applicable tax types. One resolver evaluates one set of tax types ("EU", "Canada", "All others"), implementing the expected logic, such as:
- In Canada, the store charges the tax rates defined by the customer’s home province/territory. Selling from Quebec to Ontario? Apply the Ontario HST.
- A French company selling physical products (e.g. t-shirts) will charge French VAT to EU customers.
- A French company selling digital products (e.g. ebooks) from January 1st 2015 will apply the customer's tax rates (German customer - German VAT)
- A French company providing a training in Belgium will charge Belgian VAT (explicit place of supply).
Resolvers are registered in a central service, sorted by priority and invoked individually until one returns a result.
The libraries
commerceguys/zone
Our new zone library built on top of commerceguys/addressing.
Zones are territorial groupings used for shipping or tax purposes. A zone can match other zones, countries, subdivisions (states/provinces/municipalities), postal codes. Postal codes can also be expressed using ranges or regular expressions.
commerceguys/tax
Our new tax library:
- Smart data model designed for fluctuating tax rate amounts ("19% -> 21% on January 1st")
- Predefined tax rates for EU countries and Switzerland. More to come.
- Tax resolvers with logic for all major use cases.
Visit the GitHub pages for code examples and further implementation details.
Commerce 2.x
To summarize, what does this mean for Commerce 2.x?
- Expanded data model with support for fluctuating tax rate amounts
- New concept of zones (reusable for shipping as well)
- Predefined tax rates
- Greatly expanded default logic
We’re tackling pricing next, from storage to calculation, with a special focus on the interaction between discounts and taxes.
Comments
Sales Tax in the United States
Bojan
Thank you for this comprehensive update on the sales tax architecture in Commerce 2.x - which provides an excellent update for Sales Tax - particularly as it applies to Europe.
I am the architect and author of the sales tax module - commerce_avatax - which integrates the cloud sales tax service from Avalara, Inc with Drupal Commerce and Ubercart (both 2.x and 3.x) (I mention Ubercart as there are a number of large US corporation migrating from Ubercart to Drupal Commerce, who require consistent sales tax compliance) - the latest version of the module (also does USA address validation, adding the 4 extra zip code values) - demo hosted on Platform.sh with latest version of Kickstart - http://master-ajv4gthsatep4.us.platform.sh/
Sorry Community; a shameless plug for Platform.sh. I have worked with 140 plus Integrators in the USA over the last 4 years - nearly 200 corporations, ranging from one man start ups having Nexus in one state, to Global Corporations having Nexus in all 50 states, and International sales tax needs. When it comes to sales tax compliance, we always advise our customers to get professional accounting/legal help. If you are a business with sales of > $200,000.00 - you need expert opinion on the Amazon Law, what defines Nexus, when to add Nexus, proposed congressional changes and your sales tax reporting requirements. If you are a start up with one product located in Delaware (or one of the 5 states that do not have sales tax) you need read no further. In the same way that bigger corporation require help with sales tax compliance, I believe bigger companies require help (Platform.sh is help) for hosting compliance. A "self supported" shared instances of Unix; an undocumented plan to upgrade and update modules; and a GoDaddy SS/L certificate is OK for start ups. This is not OK for US corporations. Especially considering the litigious business society (Europe you are so fortunate) we endure here. Bigger corporations have to have professional indemnity and product liability insurance. Platform.sh may have proved 4 times faster than Pantheon (we tested) - but it is the standard of business compliance which it enforces that I believe larger corporation need to seriously consider. Guys, no one paid me to say this - I am 57 - I have been in IT since I was 12 - in my opinion open source makes the wild west look safe at times - and we have to do better in the eyes of the accounting profession to be taken seriously.
Back to sales tax compliance in the USA: In Europe, the tax is known as Value Added tax (VAT) while in the USA the tax is known as Sales Tax or Use Tax. It is not possible for me to list all the legal and architectural implications of this apparent nuance in terminology. Please please can we all understand that these are completely different forms of taxation. On the surface, it is true that the both forms of taxation require the user to pay more than the store owner will keep (the tax) but in fact this is the only commonality between the two forms of taxation.
In Europe, the VAT is added to the value of a product through the manufacturing channel. Each manufacturer collects VAT as they sell up the manufacturing channel, and deducts any VAT charged to them from downstream in the channel. The end consumer has no one to sell to, and as the result bears the cost of the value added taxation. The responsibility to pay VAT (or buy VAT exempt for manufacturing using a VAT Exemption certificate) and collect VAT from consumers - is placed on all sellers and manufacturers across Europe.
This has significant technical implications to a product line item data model. VAT is a data component of each product line item. Sellers can offer products Inclusive of VAT, in which case the selling price includes the VAT that must be calculated and saved as an internal price component of the line item, when the product is added to the catalog. Or they can advertise and sell Excluding of VAT, in which case the VAT must be added to the selling price at the time of sale, but again stored as a added component of the line item. In both cases the law does require that the amount of VAT be clearly displayed to the consumer.
In the USA the tax is a consumer tax. It is required to be collected according to the sales tax legislation in each state (there are 50 plus sets of sales tax legislation) by all businesses in that state, when selling to any consumer located in that state. The first major change from Europe is that the basis of tax is not a national legal process implemented by the seller based on their location. It is a tax on the consumer based on the CONSUMERS location, where the seller may or may not (depending on the state) have an obligation to collect and pay over the tax. If the seller has an obligation to collect and pay the tax, the tax is called Sales Tax. If the seller does not have obligation to collect and pay over the tax, the tax does not go away. The tax is then called a Use Tax, and the consumer IS REQUIRED, to pay this tax at the rate applicable to the address of consumption (defined as their home address for items like cars) (defined as where the goods were purchased/eaten for foods and most consumer items) (defined as where the goods transferred ownership/or to be consumed/or billing address) for items purchased from web sites. Sales tax law in most states was written before the Internet existed, and is confusing and complex as to what location be applied for the sales tax calculation. But, it is quite clear that the tax is the responsibility of the consumer, and if the amount collected is not correct, the consumer is responsible to make good the difference in their annual State Income tax return.
This difference, has significant business and technical implications:
Business: Despite Amazon implying during the early years of the Internet that goods purchased on-line did not accrue sales tax; and that most states were ignoring the fact of their residents purchasing e items from (web sites out of state) and so evading sales tax; the facts are that that on-line businesses were effectively assisting in consumer sales tax evasion. The state of New York, closely followed by California were the first states to say "no more". They could not afford to lose the consumer taxes being evaded by on line buying of their residents. The US constitution prohibits any State from passing laws to regulate the commercial activities of a business located in another state. New York pioneered legislation allowing it to regulate the presence of any Internet business to the extent that it is has operations or presence IN the state of New York. Called the Amazon Law. And sued Amazon for it's operation in the state of new York for 4 billion dollars. The matter is still being litigated, but experts believe Amazon will settle eventually, for the cost to the state of New York of taxes they did not collect, and consumers did not declare. California followed, and this time Amazon quickly conceded and agreed to collect sales tax for shipments into the state of California. Since then Amazon is agreeing to collect taxes in states, as each demand this.
The business point: Over the next few years, the perceived marketing message of "buy without sales tax" - is going to end. It will start with larger corporations, led by Amazon who have lost this capability and are now at a business disadvantage. In parallel states are auditing their residents, and will go back 7 years if they can find that the consumer has not declared use taxes. Consumers and small business no longer want to buy on-line as a way to evade sales tax. They do not want to be exposed to an income tax audit!
Technical: Unlike Europe where the sales tax amount is stored as a component of the product line item. Sales Tax is NOT a component of the sales tax line item. It needs to be stored as it's own line item, where the amount is linked to the products on the order, but is NOT a part of the products price components. This requirement has been validated by both third party integrations (Exactor, Avalara, inc) where the sales tax is calculated using hook_order_presave - and sales tax is added to the order, the order total recalculated, as a final line item to the order as the final step in the checkout process.
In New York, there are over 400,000 potential sales tax rates. The rate includes the type of product, the value of the TOTAL order (not the line item), the address it is being shipped to (state, locality, and district taxes), the time of the year (winter vs. summer for warm weather essentials) and even a specific day of the year that can announced by the state governor at any time (sales tax holidays to stimulate the economy) - it is simply not possible to create the rules, and geo-boundaries that define district taxes, let alone update the continual stream of new boundaries, new rates, and a tax holiday.
Ever state has it's own laws for the way sales tax is calculated on shipping. As a result, the sales tax request (hook_order_presave) must only be called after shipping has been correct calculated, and the shipping line item added to the order. Some states charge no sales tax on shipping. California charges no sales tax on shipping if it is by commercial carrier and their is NO mark up on what was charged to the consumer. If there is a mark up, the total amount of shipping is taxable. In other states, only the mark up on the shipping is taxable. At the moment, the shipping modules in the USA, do not carry the concept of how the shipping line is to be taxed. Compliments to Commerce Guys, any line item can have its data model extended. The AvaTax module does extend the shipping line item so that it can be correctly taxed.
Because the sales tax rate can be updated based on the TOTAL value of the order, unlike the Europe, where the sales tax module correctly gets called by hook, each time a product line item is added to the order, this is NOT correct for USA sales tax. The tax rate, can only be determined after all the line items are on the order, always including product discounts, but not always including coupons, and then this rate applied to all line items, including the shipping line, to calculate the sales tax amount.
If the site is a marketplace, which allows users to select products from multiple stores to be added to a single order. The sales tax rate needs to be applied separately to each line item, based on which store they are from, depending on the address the goods are being shipped to and the Nexus the store has. All current hooks apply the rate across all line items. This is an architectural error for USA taxes.
Reporting: States require that sales tax be reported, including the state tax, the county tax, the district taxes (up to 5 of per single address) and remitted by legislative region. The current architecture makes no provision for the storing of any of these data elements.
Corner cases: If the company is a Multi Level marketing corporation, the value on which sales tax is applied, is not the selling price, but it is the "value based" retail price (consumption value) - to the consumer. If the company offers promotional discounts (often coupons) where the value is a "subsidy" - like an iPhone where the communications company will earn out the subsidy by their 2 year contract - the sales tax is applied to the "adjusted retail price" of the line item. Again, thank you CG architecture, that the price component of DC can be extended to include a new value called "taxable value" - and of course this be used for the sales tax calculation.
We are obviously delighted to see the tax architecture being extended, and we hope that the functional requirements outlined above can be included in the new architecture. At the same time, no platform, can support the infinite combination of US sales tax permutations for a larger USA corporation with Nexus in multiple states. Drupal Commerce is growing up, and is appealing to a larger sized corporation in the USA than before. This is creating a different demand on sales tax compliance.
I urge all Integrators to understand the issues involved here. If you are working with a start up, they are mostly not applicable, and a simple rule will suffice. If they are a major corporation, I urge the community to look beyond rules, to understand the issues and why most larger USA corporation use one of the four cloud based services. Including retail stores, who have no location based requirements, yet depend on these services for their sales tax audits, and implementation of sales tax holidays. They simply can not do the updates to their skews, in their one state, to eliminate sales tax for one day in the year when a local council decides to use sales tax as a business stimulus.
It is not technically possible for a larger US on-line store, having a 1000 plus line items in the catalog and presence (Nexus) in all 50 states to implement sales tax without a 3rd party cloud service. It is even less possible from a business compliance, audit and accounting perspective. I worry Integrators, that we are excluding ourselves from being considered by larger corporations, when we fail to understand these issues. We recently won a project from Magento, where one of the key considerations was the poor implementation Magento has of AvaTax (an accounting dictated requirement for the project) (using the older PHP library, and not the new rEST api) - as the cost implications of Magento's commercialization, and lesser capabilities for compliance forced the non-profit corporation to invest in a new site and explore an alternative platform.
As part of it's commitment to sales tax education for the Drupal Community (Integrators and End Users) Avalara have provided a Community Edition of their cloud service that can be used with the latest version of the Drupal Commerce for AvaTax Connector. It is a configurable option to the project, downloaded as a regular Drupal contributed module and enabled (development or Production) with generous licensing. This has been sized to allow any business to collect taxes correctly, in their state of Nexus, as they mature into a viable business.
Thank you for your lengthy
Thank you for your lengthy comment. It deserves to be a blog post of its own, I encourage you to publish it somewhere :)
This is the part of tax that I hinted at under "pricing", to be covered in the next blog post. I specifically didn't cover it this time because it would make the existing post it too long.
We are aware of the problems involved, and Commerce 2.x actually will switch to calculating and storing sales tax on the order level, as a separate line item.
In the blog post I said "We can do this for all countries except the US (where tax rates vary even by zip code, and are therefore impossible to predefine", to emphasize that we won't try to predefine US sales tax rates, this is truly an impossible venture, and we leave that to commercial solutions such as AvaTax and Exactor.
Zip Codes vs. Districts
I will explore a logical place to post this explanation - probably on both the web sites - http://drupalsalestax.com - and the web site - http://open4tax.com - so that we can try and make it commercially neutral. I openly disclose that I am the author of the AvaTax implementation, but the issue of a correctly designed sales tax architecture for Drupal Commerce 2.x must rise above any single commercial interest.
Delighted to hear that Commerce 2.x will define a sales tax line item. It is needed for every USA order and this way all commercial providers can leverage it correctly.
On zip codes. You are correct, and it is more complex. District taxes trend to follow zip codes, but for some reason best know to the USA - districts do not follow zip codes. It is only because the zip codes are close in proximity, that 85% of zip codes will actually be in any defined district. It is worse than this. There are different ways to define a district, and each is allowed to define and raise a new tax. For example:
i) School district - will include a geography based on # of students
ii) Hospital district - will include a geography based on medical resources in the geography
iii) Rail district - will cover all parties who gain advantage from a rail line
The only way to handle this is by defining a geo-boundary for the district - and then resolving the actual shipping address for the districts that it may fall into.
There are zip code based systems, but at 85% accurate - they are not acceptable to larger companies accounting departments.
Awesome, looking forward to a
Awesome, looking forward to a correct implementation of Australian GST as lumping it in with VAT isn't correct, and the existing commerce_australia module, while an acceptable hack, is technically not correct for several use cases.
Don't get sucked into this black hole
@adTumbler touched on some of this already, I wrote this before I noticed he is the author of avatax- good to see you on here...
This problem is absolutely massive. I think the team's efforts would be much better spent on something else. What you're talking about here is a massive feature that will require massive maintenance, in other words, a paid service. These paid services already exist. See:
http://www.avalara.com/products/avatax/
https://taxcloud.net/default.aspx
https://www.accuratetax.com/
I really question the wisdom of trying to actually bite this off as an open source project... Why not integrate with these services and be done with it? Write a module like payments, which has integrates for these. If someone comes up with a free one, integrate that. I know it really sucks. It's annoying the states/governments hold us to these standards but can't even be bothered to submit their rates in a standard format to a centralized database. That's because our representatives are to old and technically illiterate to even make sense of what I just said. Hopefully one day that won't be the case. I have this "here we go again" feeling though it will be too late and we'll already have an industry based around this with it's own lobby. Yay to forced inefficiency.
Maybe someone can setup a centralized database you said with a standardized format/api, but that seems like a project separate from what Commerce is trying to accomplish, and I think that database needs to be made successfully before basing all of Commerce's tax handling on it.
When stores are small (10k to as high as 1m yearly) compliance doesn't really matter too much, nobody is going to come after you unless you do something really dumb on your tax return. The community can do small contrib modules to deal with specific states/countries so people can still launch their stores with a reasonable amount of compliance. I wrote a Drupal module for ubercart to handle California taxes using the CSV they provide, which we still use at my company, and we do a lot of business with that simple system.
We're not forcing anyone to
We're not forcing anyone to use the predefined tax rates. You will be able to use Commerce 2.x without importing them, if that's your wish.
And again, as stated above,
And again, as stated above, we're not touching U.S. sales taxes with a ten foot pole. Not sure how that got missed. : ?
Agree with Dual Approach
Bojan has a good point!
Alaska, Delaware, Montano (d) and New Hampshire have no Sales Tax. For small businesses who only have Nexus is these states, there is nothing to be done.
Connecticut, Indiana, Maine, Maryland, Massachusetts, Michigan, New Hampshire, Oregon, Rhode Island and DC, have a single rate across the state. A sales tax table and a rule will do the job quite correctly for any business with Nexus in these states.
California, and the others (useful web site here: http://taxfoundation.org/article/state-and-local-sales-tax-rates-2014) have 100's of county and district taxes. A table will not work as it does not define the sales tax boundaries. Zip Code based tables are 85% accurate. The bigger issue is reporting the sales tax at the end of the reporting period. Most states require a report per district. New York is the most challenging. Sales tax includes type of goods, the shipping address, total value of order and time of year. Over 400,000 potential rates.
It really is a horses for courses debate. Although I strongly suggest the decision involves and accountant or tax adviser. The fines for getting it wrong are in the many 10's of thousands for larger companies.
The Connector for AvaTax offers a Community Edition with fair free use. If you are not certain, give it a try. It takes a few moments to download and install. In dev mode it reports the district taxes to the Drupal log so that you can see exactly what you have to deal with for the state (or states) your site has Nexus. Including state, county, district and shipping sales tax.
Project url: https://www.drupal.org/project/commerce_avatax