Next Previous Contents

3. Tutorial: Basic unit testing

This tutorial will use the JUnit Test Infected article (see Test Infected) as a starting point. We will be creating a library to represent money, allowing conversions between different currency types. The development style will be "test a little, code a little" with unit test writing preceding coding. This constantly gives us insights into module usage, and also makes sure we are constantly thinking about how to test our code.

3.1 How to write a test

Test writing using Check is very simple. The file in which the checks are defined must include check.h as so:


#include <check.h>

The basic unit test looks as follows:


START_TEST(test_name)
{
  /* unit test code */
}
END_TEST

The START_TEST/END_TEST pair are macros that setup basic structures to permit testing. It is a mistake to leave off the END_TEST marker; doing so produces all sorts of strange errors when the check is compiled.

3.2 Setting up the money tests

Since we are creating a library to handle money, we will first create a header money.h, and a file to contain our unit tests, check_money.c (there is a pun there, but no matter...). To manage everything we'll use Autoconf/Automake for this example. (One could do something similar with ordinary makefiles, but in the author's opinion, it is generally easier to use Autoconf/Automake than bare makefiles, and it provides built-in support for running tests). Here is the Makefile.am:


TESTS=check_money
noinst_PROGRAMS=check_money
check_money_SOURCES=  money.h money.c check_money.c
check_money_INCLUDES= @CHECK_CFLAGS@
check_money_LIBS= @CHECK_LIBS@

Special mumbojumbo to use in configure.in is:


AM_PATH_CHECK()

This will ensure that things link up properly with Check by defining the appropriate compiler and linker flags as CHECK_CFLAGS and CHECK_LIBS. It also makes sure that we can only compile in an environment that has Check. The money.h header should only contain the standard #ifndef MONEY_H stuff, money.c should be empty, and check_money.c should only contain an empty main function. Run this with make -k check, after going through the setups to get autoconf and friends working. If all goes well, make should report that our tests passed. No surprise, because there aren't any tests to fail.

The AM_PATH_CHECK() macro is defined in the file check.m4 which is installed by Check. If you see warnings from automake or aclocal this most certainly means that you forgot to call aclocal or that aclocal can't find check.m4. AM_PATH_CHECK() has some optional parameters that you might find useful:


AM_PATH_CHECK([MINIMUM-VERSION,[ACTION-IF-FOUND[,ACTION-IF-NOT-FOUND]]])

3.3 Test a little, code a little

The Test Infected article starts out with a Money class, and so will we. Of course, we can't do classes with C, but we don't really need to. The Test Infected approach to writing code says that we should write the unit test before we write the code, and in this, we will be even more dogmatic and doctrinaire than the authors of Test Infected (who clearly don't really get this stuff, only being some of the originators of the Patterns approach to software development and OO design).

Here is our first unit test:


START_TEST(test_create) 
{ 
 Money *m; 
 m = money_create (5, "USD");
 fail_unless (money_amount (m) == 5, 
              "Amount not set correctly on creation");
 fail_unless (strcmp (money_currency (m), "USD") == 0, 
              "Currency not set correctly on creation"); 
 money_free (m); 
}
END_TEST

A unit test should just chug along and complete. If it exits early, or is signaled, it will fail with a generic error message. (Note: it is conceivable that you expect an early exit, or a signal. There is currently nothing in Check to specifically assert that we should expect either -- if that is valuable, it may be worth while adding to Check). If we want to get some information about what failed, we need to use the "fail_unless" function. The "fail_unless" function (actually a macro) takes a first Boolean argument, and an error message to send if the condition is not true. If the Boolean argument is too complicated to elegantly express within "fail_unless", there is an alternate function "fail", that unconditionally fails. The second test above can be rewritten as follows:


if (strcmp (money_currency (m), "USD") != 0) {
  fail  ("Currency not set correctly on creation");
}

For your convenience the "fail_unless" function also accepts NULL as the msg argument and substitutes a suitable message for you. So you could also write a test as follows:


fail_unless (money_amount (m) == 5, NULL);

This is equivalent to the line:


fail_unless (money_amount (m) == 5, "Assertion 'money_amount (m) == 5' failed");

When we try to compile and run the test suite now, we get a whole host of compilation errors. It may seem a bit strange to deliberately write code that won't compile, but notice what we are doing: in creating the unit test, we are also defining requirements for the money interface. Compilation errors are, in a way, unit test failures of their own, telling us that the implementation does not match the specification. If all we do is edit the sources so that the unit test compiles, we are actually making progress, guided by the unit tests, so that's what we will now do.

We will add the following to our header money.h:


typedef struct Money Money;
 
Money *money_create (int amount, char *currency); 
int money_amount (Money *m); 
char *money_currency (Money *m); 
void money_free (Money *m);

and our code now compiles, but fails to link, since we haven't implemented any of the functions. Let's do that now, creating stubs for all of the functions:


#include <stdlib.h> 
#include "money.h"
Money *money_create (int amount, char *currency) 
{ 
  return NULL; 
}
int money_amount (Money *m) 
{ 
  return 0; 
}
char *money_currency (Money *m) 
{ 
  return NULL; 
}
void money_free (Money *m) 
{ 
  return; 
}

Now, everything compiles, and we still pass all our tests. How can that be??? Of course -- we haven't run any of our tests yet....

3.4 Creating a suite

To run unit tests with Check, we must create some test cases, aggregate them into a suite, and run them with a suite runner. That's a bit of overhead, but it is mostly one-off. Here's the code in check_money.c. Note that we include stdlib.h to get the definitions of EXIT_SUCCESS and EXIT_FAILURE.


#include <stdlib.h>
#include <check.h>

Suite *money_suite (void) 
{ 
  Suite *s = suite_create ("Money"); 
  TCase *tc_core = tcase_create ("Core");
 
  suite_add_tcase (s, tc_core);
 
  tcase_add_test (tc_core, test_create); 
  return s; 
}
 
int main (void) 
{ 
  int nf; 
  Suite *s = money_suite (); 
  SRunner *sr = srunner_create (s); 
  srunner_run_all (sr, CK_NORMAL); 
  nf = srunner_ntests_failed (sr); 
  srunner_free (sr); 
  suite_free (s); 
  return (nf == 0) ? EXIT_SUCCESS : EXIT_FAILURE; 
}

Most of the money_suite code should be self-explanatory. We are creating a suite, creating a test case, adding the test case to the suite, and adding the unit test we created above to the test case. Why separate this off into a separate function, rather than inlining it in main? Because any new tests will get added in money_suite, but nothing will need to change in main for the rest of this example, so main will stay relatively clean and simple.

Unit tests are internally defined as static functions. This means that the code to add unit tests to test cases must be in the same compilation unit as the unit tests themselves. This provides another reason to put the creation of the test suite in a separate function: you may later want to keep one source file per suite; defining a uniquely named suite creation function allows you later to define a header file giving prototypes for all the suite creation functions, and encapsulate the details of where and how unit tests are defined behind those functions. See the test program defined for Check itself for an example of this strategy.

The code in main bears some explanation. We are creating a suite runner object from the suite we created in money_suite. We then run the suite, using the CK_NORMAL flag to specify that we should print a summary of the run, and list any failures that may have occurred. We capture the number of failures that occurred during the run, and use that to decide how to return. The check target created by Automake uses the return value to decide whether the tests passed or failed.

3.5 SRunner output

The function to run tests in an SRunner is defined as follows:


void srunner_run_all (SRunner *sr, enum print_output print_mode);

This function does two things:

  1. Runs all of the unit tests for all of the test cases defined for all of the suites in the SRunner, and collects the results in the SRunner
  2. Prints the results according to the print mode specified

For SRunners that have already been run, there is also a separate printing function defined as follows:


void srunner_print (SRunner *sr, enum print_output print_mode);

The enumeration values defined in Check to control print output are as follows:

CK_SILENT

Specifies that no output is to be generated. If you use this flag, you either need to programmatically examine the SRunner object, print separately, or use test logging (described below: Test Logging).

CK_MINIMAL

Only a summary of the test run will be printed (number run, passed, failed, errors).

CK_NORMAL

Prints the summary of the run, and prints one message per failed tests.

CK_VERBOSE

Prints the summary, and one message per test (passed or failed)

With the CK_NORMAL flag specified, let's rerun make check now. We get the following satisfying output:


Running suite(s): Money 
0%: Checks: 1, Failures: 1, Errors: 0 
check_money.c:9:F:Core: Amount not set correctly on creation

The first number in the summary line tells us that 0% of our tests passed, and the rest of the line tells us that there was one check, and one failure. The next line tells us exactly where that failure occurred, what kind of failure it was (P for pass, F for failure, E for error).

Let's implement the money_amount function, so that it will pass its tests. We first have to create a Money structure to hold the amount:


struct Money { 
  int amount; 
};

Then we will implement the money_amount function to return the correct amount:


int money_amount (Money *m) 
{ 
  return m->amount;
}

We will now rerun make check and... What's this? The output is now as follows:


Running suite(s): Money 
0%: Checks: 1, Failures: 0, Errors: 1 
check_money.c:5:E:Core: (after this point) Received signal 11

What does this mean? Note that we now have an error, rather than a failure. This means that our unit test either exited early, or was signaled. Next note that the failure message says "after this point" This means that somewhere after the point noted (check_money.c, line 5) there was a problem: signal 11 (AKA segmentation fault). The last point reached is set on entry to the unit test, and after every call to fail_unless, fail, or the special function mark_point. E.g., if we wrote some test code as follows:


stuff_that_works ();
mark_point ();
stuff_that_dies ();

then the point returned will be that marked by mark_point.

The reason our test failed so horribly is that we haven't implemented money_create to create any money. Go ahead and implement that, and money_currency, to make the unit tests pass.


Next Previous Contents