We grabbed the best features of each framework and created what we think it’s the best C++ unit-testing framework out there (for our needs anyway). We took the results and put them up in Sourceforge under a veryunrestrictive license, and that’s how UnitTest++ was born.
I had been using a modified version of CppUnitLite for quite a while, slowly fixing the parts in need of mending, and adding new functionality as it became needed. Eventually I released most of those changes as CppUnitLite2, and I thought that would be the end of that. It turns out that was just the beginning.
Shortly after that, several people started emailing me with new functionality, fixes, patches, suggestions, etc. At the same time, Charles Nicholson was starting to get quite a bit of response as well for his own testing framework, sometimes with similar fixes and suggestions.
If this was going to grow to be a real project, developing over time, supporting users’ needs, and accepting code contributions, clearly it would make sense to join forces. And that’s exactly what we did. We grabbed the best features of each framework and created what we think it’s the best C++ unit-testing framework out there (for our needs anyway).
We took the results and put them up in Sourceforge under a very unrestrictive license, and that’s how UnitTest++ was born.
Our ultimate goal is to apply test-driven development to game development. All of the existing frameworks fell short in one area or another. Specifically, the driving forces behind the design of UnitTest++ are:
- Portability. As game developers, we need to write tests for a variety of platforms, most of which are not supported by normal software packages (all the game consoles). So the ability to easily port the framework to a new platform was very important.
- Simplicity. The simpler the framework, the easier it is to add new features or adapt it to meet new needs, especially in very limited platforms.
- Development speed. Writing and running tests should be as fast and straightforward as possible. We’re going to be running many tests hundreds of times per day, so running the tests should be fast and the results well integrated with the workflow.
UnitTest++ is a fully featured testing framework, with many of the features you may expect from Xunit-style frameworks such as fixtures and reporting. Additionally, here are some of the specific features that make UnitTest++ unique:
- Minimal work required to create a new test. For TDD development, we write lots and lots of tests, so creating a new test should be as quick and painless as possible. UnitTest++ does not require explicit test registrations, and creating new tests requires minimal work.
- Good assert and crash handling. When automated tests are executed, it’s very important not to hang the process or abort the tests any time the program crashes or an assert fails. UnitTest++ will catch and report exceptions as failed tests and print a lot of information about them. It will translate signals to exceptions as well as be able to catch invalid memory accesses and other problems.
- Minimal footprint and minimal reliance on heavy libraries. When you intend to run tests on a Cell SPU with a measly 256 KB of RAM, you really want to be as lean and mean as possible. UnitTest++ is very lightweight, and it will get even lighter weight once strstream is replaced.
- No dynamic memory allocations done by the framework, which makes it much easier to track memory leaks and generally more attractive for embedded systems.
- Multiplatform support. Right now it supports Win32, Linux, and Mac OS X out of the “box,†and other platforms will probably work without any changes. The source code makes use of platform-specific functions whenever necessary, so it is easy to hook up special timers, allocators, or reporters for specific platforms. The code comes with a makefile as well as with project and solution files for Visual Studio 2003 and 2005.
- Full set of unit tests for itself. UnitTest++ was TDD’d from the ground up, including some of the ugly macros (imagine how much fun it was bending the framework to test itself that way!). It goes without saying, all the tests are included with the source code, so you can go totally wild with them 🙂
This was just the first release. UnitTest++ is ready for full production work, but we already have plans beyond this release. This is a peek at some of the features we have in mind for upcoming versions:
- Out-of-the-box support for game console platforms (Xbox 360, PS3 PPU and SPU, and Nintendo Revolution). Of course, this is dependent on us getting permission from the console manufacturers to release some code that works with their SDK.
- Ability to trade some features (such as rich check reporting) for some extra memory. In some platforms, like Cell SPUs, you really need to go barebones.
- Suites to group tests together.
- Different reporters (HTML, GUIs, etc). Don’t worry, all those modules will be optional and well separated, so they won’t affect the simplicity of the framework.
- Per-test timings. Maybe the ability to flag some tests as not exceeding a certain amount of time, or just failing any test that goes over a threshold.
- Built-in memory leak detection.
So that’s it. Download the source code and give it a try. We welcome any comments and suggestions (best channels would be through the project mailing list or comments here). We hope you find it useful and that it makes your TDDing even more productive.
UnitTest++
I read today in Games from Within that a new unit test framework have been released: UnitTest++, an improved version of CppUnitLite. I’ve been using unit tests for a long time, mostly as a regression error tester and to report bugs in the databas…
how about adding some basic documentation about how to use it?
Yeah, we’ll get some documentation together for the next release. In the meanwhile, I suggest you look at how the test project is set up. Also, look at the TestUnitTest.cpp file. That is pretty much a set of “documentation” tests showing one of each type and how they work.
how about it if you guys use TDD to make sure version 1.0 compiles?
Linking…
Performing Post-Build Event…
‘c:\Documents’ is not recognized as an internal or external command,
That’s already been reported. Move it to a directory without spaces or put ” around the $(TargetPath) in the postbuild step (I’m assuming you’re using Windows and Visual Studio).
the linux signal translator looks highly suspicious, here:
void RegisterSignals()
{
…
if (sigsetjmp( g_sigMark, 1 ) != 0)
throw (“Unhandled system exception”);
}
it’s calling setjmp and then returning (==0). the longjmp will jump into no man’s land – the stack frame is gone.
I’ve got some code that works if you want it, drop me a line.
Looking good so far. The plans for built-in memory detection and test timing sound cool, but am I right to guess that support for suites and alternative reporters comes first, considering that they were in your previous checklist of features that a good unit test framework should have?
i was just wondering if you guys use any code coverage tool in combination with UnitTest++ ?
Eugene, We’ll definitely have some alternate reporters soon (probably an XML and HTML ones). Suites might take a bit more, but they’re in the works.
Better timings are actually quite important to us, so we’ve already added them to the current Sourceforge code and they’ll be in the next release.
If you have any specific requests or high priority features in mind, jump in the unittestcpp-devel mailing list and let’s discuss it there.
Dirck, let’s discuss the signal translator on the dev mailing list: https://lists.sourceforge.net/lists/listinfo/unittest-cpp-devel
And Tobias, no, I don’t use any code coverage tool. Since we’re applying TDD, the goal isn’t to get 100% unit test coverage, but to use the tests to help the design of your code. Also, naturally, with TDD, you’ll get something very close to 100% coverage anyway.
I’ve been dabbling with CPPUnitLite for quite a while, naturally i tried to improve it as well and came up with…well…something similar as unittest++. albeit i’m not using fixtures but suites to group my tests, so tests are basically written like:
TESTSUITE(theSuiteName) {
TEST(theFirstTest){
…
}
TEST(theSecondTest){
…
}
…
}
the whole framework resides within a single header file,
tests are automatically registered as you write them, and if it wasn’t for the unknown order how translation units were handled you could run the tests outside
of main.
code is nearly platform independent, although there’s some platform specific code to redirect testresults to the output pane of visual studio (and the output format is tailored for vs, so i can click on a failed test and will jump to corresponding source file and line)
if it’s interesting for you to include some of my ideas in unittest++ then drop me a line. i’d be more than glad to share.
kind regards,
rené