testo – A minimal C++ unit test framework.

For many years, I’ve been using cppunit for my unit tests. It is cross-platform and easy to use. I’ve been quite happy with the results.

Recently, I wanted a tool that would also measure regression in performance, so do some timing of tests, not only give a failed / pass result. I thought of adding that in my current cppunit tests, as some kind of extension, but decided to have a look around, at other unit test frameworks and their review. I was a bit surprise to see the lack of love for cppunit, most review complaining how verbose it is to write new tests.

Here’s a test suite with cppunit:

#include <cppunit/extensions/HelperMacros.h>

class my_test_suite : public CppUnit::TestFixture
{
  CPPUNIT_TEST_SUITE( my_test_suite );
  CPPUNIT_TEST( test_case_1 );
  CPPUNIT_TEST( test_case_2 );
  CPPUNIT_TEST_SUITE_END();

  void test_case_1()
  {
    CPPUNIT_ASSERT( some test );
  }
  void test_case_2()
  {
    CPPUNIT_ASSERT( some more test );
  }
};

CPPUNIT_TEST_SUITE_REGISTRATION( my_test_suite );

To be frank, it is verbose. You have to derive from a base class and use 4 macros just to have a test suite, then declare each test and register them with yet another macro. On the other hand, since I was serious about having unit tests, all my work environments had a test suite template ready to use, usually with a keyboard shortcut, so I rarely, if ever, typed this code, one keystroke and I had a new test suite ready to be filled.

Most of the other unit test frameworks were not as verbose, but they rely on hiding the code in even more complex macros to the point where the code does not look like C++ anymore (see boost.test or googletest for example).

I don’t like macros and I was wondering if I could do better, do a C++ unit test framework that would 1) be as minimal as possible in its syntax, 2) not rely on macros (as much as possible) and 3) record performance of some tests. It has to produce result similar to what I was getting with my simple usage of cppunit.

Introducing testo.

Here the same example as before with testo:

#include <testo/testo.h>

struct my_test_suite
{
  void test_case_1()
  {
    TEST_ASSERT( some test );
  }
  void test_case_2()
  {
    TEST_ASSERT( some more test );
  }
};

REGISTER_TESTS( my_test_suite,
            test_case_1,
            test_case_2 );


Notice: no base class to inherit from, only 1 macro to register all. That’s it! The test runner will pick this up and give exactly the same result as with cppunit.

Why still some macros ? Well, I could get rid of the REGISTER_TESTS macro, but it would make test suite registration much more verbose, something like:


REGISTER_TESTS< my_test_suite >(
         &my_test_suite::test_case_1, "test_case_1",
         &my_test_suite::test_case_2, "test_case_2" );


So I’ll live with it for now. If proposal N3951 for adding reflection to C++ is ever accepted, I will be able to get rid of it, but that’s for a distant future.

TEST_ASSERT & cie are still macros so that they can expand the __FILE__ and __LINE preprocessor value when compiling.

Other implemented features:

• the test suite can have a setup() and/or teardown() method. setup() will be called before each tests and teardown() will be called after each tests.
• a test method’s name must start with « test_ » or « timedtest_ ». The method name is the name of the test in the output. timedtest have their performance recorded.
• the name of the class is taken as the test suite name, unless the class implement a name() const method. If so, the return value of name() is used.

Hopefully, some more features coming. For instance, I’d like to record all result in a local database and also have some web UI visualisation tool. More customisable output. etc.

Anyway, it is complete enough now and has fully replace cppunit in all my unit tests.

Code on github.