Testing Guidelines

Jumping off some email conversations I've had with Ryan I wanted to get some feedback and possibly suggest some module structure ideas and a pattern for implementing tests for Drupal Commerce.

With regards to the structure of the module a common practice I try to follow is to distinguish between public and private APIs. All code will follow this to a certain extent, but it may make sense to make it a part of the standard for the module to have a clearly defined public API and then an API that is private and meant to be consumed only by the methods making up the PUblic API.

Now, the reason this is meaningful for testing is that you can start writing tests for the public API and by virtue of the the module's structure you are covering the various parts of the private API in doing so. This also makes your public API test suite more like integration tests since they are testing integrated behaviors rather than granular functions. Then you can drill down to the private API to write more granular unit tests.

Another thing to note is that any tests should not focus on the implementation specifics but on the behavior expected of the function being tested. Basically treat functions like black boxes. What matters is that when you plug something in you get what you expect back out, the how is not important, at least it shouldn't be. Too many function tests end up dealing with implementations which makes it impossible to use the tests as a regression suite.

Anyways, this is an area that probably needs to be talked about in detail, so lets bounce this stuff around.

Posted: Jan 25, 2010


Ryan Ryan Szrama on January 25, 2010

Mmm, thanks for starting off the discussion, James. Out of curiosity, how would you differentiate public vs. private API functions? How restrictive should the use case of a specific function be before we decide that this just simply doesn't need to be called by another module?

Also, fwiw, Drupal only allows "soft" scope limiting by using a function naming pattern. In practice, Ubercart at least didn't really follow this.

And on a total side note, didn't know if you saw the announcement on d.o or not, but it will now run tests on contributed modules somehow. I'm interested to see how that works out, especially for complex modules like we're developing, but at least it will do code review and post-commit testing, too.

See: http://drupal.org/node/689730