Drupal Commerce Blog

What's happening in the world of Drupal Commerce.

Commerce 2.x: Unit, Kernel, and Functional Tests Oh My!

At the end of May, I made an initiative to move all of the Drupal Commerce tests away from Simpletest and to use the available test classes built off of PHPUnit. Why? Simpletest is a test framework within Drupal and not used by the PHP community at large.

With the KernelTestBaseTNG™ issue, Drupal core officially moved to being based on top of PHPUnit for Kernel and Unit tests. Soon more test types were to follow, such as browser tests and JavaScript testing.

Death to Simpletest, Long Live PHPUnit, Mink, and PhantomJS

We now have PHPUnit as our test framework, the choice of the greater PHP community. The browser tests use the Mink browser emulator, which anyone working with Behat should be somewhat familiar. Testing JavaScript is done by pointing PhantomJS configuration to Mink. No longer are we limited to the functionalities of Simpletest and our community to develop it.

How hard is it?

Within the first few days of work, our TravisCI build was updated to run PhantomJS, and we had our first JavaScript tests running.

We have plenty of commits with examples of porting our tests from Simpletest to Kernel or BrowserTest tests. However, the EntitySelectWidgetTest is probably the most straightforward and shows how easy it is to migrate. See the commit and diff here:

https://github.com/drupalcommerce/commerce/commit/9846bd7d081d364418413a...

Thanks to

This wasn't a lone effort. Shout out to those who helped ensure we have a well tested project:

  • ThinkBean for sponsoring my Commerce 2.x, which helped get this initiative wrapped up.
  • Shawn McCabe at Acro Media for working on the product tests and the other remaining tests.

Unit, Kernel, Functional, and FunctionalJavascript what?

With Simpletest, there were two test suites (kind of): web test and unit test. However, I do not recall seeing any of the latter in the wild because it could not communicate with the database.

In Drupal 8 we now have four different test suites, each with specific purposes.

  • Unit: You are testing class functionality, and mock other required services. See DefaultStoreResolverTest as an example.
  • Kernel: Think of a Kernel test as a being a Unit test which has the ability to install schema and config to the database. You do not need to mock service objects when using a Kernel test. See ProductTest (covers product entity.) as an example
  • Functional: Functional tests are tests which require a fully bootstrapped Drupal and provide browser emulation via Mink. This test type will look familiar to those who have worked with Simpletest. See StoreTypeTest as an example.
  • FunctionalJavascript: These tests have Mink configured to talk to PhantomJS to run JavaScript. A great example is our AddToCartMultipleTest.

Tests live in the Drupal\Tests\$module\$type namespace (e.g. Drupal\Tests\commerce\Kernel) and tests/src/$type folder (e.g. tests/src/Kernel).

Note: Simpletest had the ability to render Drupal's AJAX Form API operations manually. It never actually supported JavaScript. When working with robust field widgets, such as Address' you'll need to use a FunctionalJavascript test.

Get Setup

We will cover how to setup PhpStorm to run Drupal Commerce's tests. PhpStorm provides a PHPUnit test runner and lets you stay within your workbench to run tests. I run my tests using PHP's built-in server and SQLite. This allows you to run tests with a minimal environment setup.

PhpStorm professional? Here's the tl;dr configurations to put in your project: https://gist.github.com/mglaman/f0cbe19a2fae4fb7f685c25606ab60ea

If you are new to all of this, you will also want to check out the Running PHPUnit tests handbook page on Drupal.org: https://www.drupal.org/node/2116263

Configure phpunit.xml

The first step is to copy the phpunit.xml.dist shipped with Drupal. Make a copy and name it phpunit.xml. There are three values you will want to adjust in the <php> section of the configuration file.

  • SIMPLETEST_BASE_URL: The URL the Drupal site can be viewed at.
  • SIMPLETEST_DB: If the site is not installed, or is different than the default database connection.
  • BROWSERTEST_OUTPUT_DIRECTORY: Specifying this path will save Functional and FunctionalJavascript HTML output.

You can use SQLite for testing, which is my primary use case for setting SIMPLETEST_DB. Below is a sample of what the configuration changes would look for the environment settings.

<!-- Snippet of environment settings. -->
  <php>
    <!-- Set error reporting to E_ALL. -->
    <ini name="error_reporting" value="32767"/>
    <!-- Do not limit the amount of memory tests take to run. -->
    <ini name="memory_limit" value="-1"/>
    <!-- Example SIMPLETEST_BASE_URL value: http://localhost -->
    <env name="SIMPLETEST_BASE_URL" value="http://localhost:8080"/>
    <!-- Example SIMPLETEST_DB value: mysql://username:password@localhost/databasename#table_prefix -->
    <env name="SIMPLETEST_DB" value="sqlite://localhost/sites/default/files/.ht.sqlite"/>
    <!-- Example BROWSERTEST_OUTPUT_DIRECTORY value: /path/to/webroot/sites/simpletest/browser_output -->
    <!--<env name="BROWSERTEST_OUTPUT_DIRECTORY" value="/var/platform/www/sites/simpletest/browser_output"/>-->
    <env name="BROWSERTEST_OUTPUT_DIRECTORY" value="/Users/mglaman/Drupal/sites/commerce2x/www/sites/simpletest/browser_output"/>
    <ini name="display_errors" value="On" />
    <ini name="display_startup_errors" value="On" />
  </php>

Setup PHPUnit Runner

The first step is to create a PHPUnit Runner. Visit Run -> Edit Configurations and create a new PHPUnit runner. You will want to have it configured to look for tests inside of modules/commerce (or wherever you put Drupal Commerce, e.g. modules/contrib/commerce.) Then change the configuration to be base on the one you created previously, such as core/phpunit.xml.

PHPUnit run configuration

Setup PhantomJS

You will need to setup PhantomJS to test the FunctionalJavascript test suite tests. If you are running Mac OS X and have Homebrew, you can just run brew install phantomjs. Otherwise, review http://phantomjs.org/download.html and use the appropriate steps to install PhantomJS. It's worth doing, as most visual regression tools (like Wraith) need it.

Drupal expects PhantomJS to run on a particular port with seeded configuration. The IntelliJ platform does not have a PhantomJS plugin yet, so I recommend writing a Bash script, and then a Bash runner. Note: You will need to install the BashSupport plugin in order to have this runner.

#!/usr/bin/env bash
phantomjs --ssl-protocol=any --ignore-ssl-errors=true ./vendor/jcalderonzumba/gastonjs/src/Client/main.js 8510 1024 768

The above line starts PhantomJS as the Drupal tests assume it will be running. You then just need to add a configuration to have PhpStorm execute the script.

PhantomJS runner

Note: I am currently working on an IntelliJ PhantomJS integration plugin to make this easier.

Optional: Built-in server

As stated, I run my tests using PHP's built-in server instead of booting up my Docker containers. In fact, this is how our tests run on TravisCI. Create the runner and configure the domain and port.

Built-in PHP server runner

Optional: Compound runner

I like to be lazy efficient. We just created three different items that need to be running to execute the test suite. PhpStorm has a way to group runners, so they execute at once. I recommend making Compound runner that executes all three of the runners.

Combo runner for all configurations

Matt Glaman
Posted: Jun 21, 2016

Comments

GoZ GoZ on June 21, 2016

Thanks Matt for this post.

Some adjustments :

  • Bash is available after installing end enabling BashSupport plugin.
  • If you want to use sqlite with an existing site, so a settings.php is already configured, you have to remove/rename settings.php file. Otherwise, phpunit will return errors as drupal is already installed.

cosmicdreams on June 21, 2016

It's great that you're investing in this now. In the future it will be a huge help in finding out bugs and keeping a steady pace when advancing cool stuff.

GoZ GoZ on July 27, 2016

Phpstorm Built-in Server has some issues with rewrite and urls containing dots like admin/structure/types/manage/article/fields/node.article.field_address.

To fix this, the only solution i found is to use a router for phpstorm.

create a file in web/routing.php:

<?php
if (preg_match('/\.(?:php|png|jpg|jpeg|gif|ico|css|js)\??.*$/', $_SERVER["REQUEST_URI"])) {
  return false;
} else {
  include __DIR__ . '/index.php';
}

And configure Built-in to use this router script checking "Use router script" and filling path to script.