Games from Within is a weblog written by Noel Llopis,

who has a pretty impressive game d…

Guess what?! I discovered a bug in my first test! I was alittle puzzled at first because I thought “it can’t be! it works fine in the game!”. I made an unconscious assumption about a container never being empty, dereferencing the iterator in a tiny hidden loop. BOOM! I stepped in with gdb, found the function that was called and repaired the sourcecode.

Am I convinced? Hell yes, it’s fun too.

The drawback I was talking about earlier is really a more general problem and not related directly to TDD. I guess it is more related to agile programming. I really like the idea of constantly refactoring code/classes/interfaces and corresponding unit-tests. I rarely get all of my class methods right the first time I create them. It is not until I start using the class do I realize the best methods I need and how they should work. Being able to refactor is crucial to getting to the right design and implementation.

But most projects I’ve been on tend to resist any changes to “public” code that could affect documents, developers, or testers. Instead developers are encouraged to work-around any deficiencies. Work-arounds can be Ok but they can also lead to hacky bloated code, duplicate functionality, and performance problems. The longer the work-arounds are in the more likely they become a permanent part of the code base. Changes to the code (and unit-tests) sometimes finally occur at the eleventh hour when ship stopping functional and performance problems become apparent.

This is really a management problem and not a TDD problem. I just brought this up because in my last project we used unit-tests to not only drive code development but to also document and specify public interfaces/classes. Later in the project during coding that actually used the public classes it was almost impossible to get a method changed or added. This in part caused severe end-game problems and a seven month slip in ship date.

Do you have a link to Röken’s presentation? I Googled for it but nothing came up. I’d love to find out more about what they’re doing.

Each unit test should really be blazingly fast. Probably in the order of a microsecond. You should be able to run 100,000 of those in one second, so that shouldn’t be an issue. Remember, these are *unit tests*, so they work directly on one class, not on the whole program.

What do you see at some of the drawbacks of TDD? There’s the extra code, but frankly, I see that more of an advantage than a disadvantage.

One major problem was that we left out the “refactor code, refactor tests” iteration of TDD. Once the tests and code were written and passing they became a kind of spec/contract and were “locked down”. This was done to meet a very tight schedule. If you wanted to find out how an interface was supposed to work then go look at the unit-tests.

The problem, of course, was that owner written unit-tests don’t always equate to how code is really used by customers. After some severe customer performance/functional problems we finally incorporated code/unit-test refactoring based on actual code use and need. Locking down the interfaces early just caused product slip as customers first tried to work around interface problems, then finally allowing the code and unit tests to be refactored to meet customer needs. There is a fine balance between process driven development and maintaining flexibility.

2.) I personaly build test/test-suites into modules (DLLs on win32) and store them in a directory hierarchy (engine/audio/oggStreaming.dll). That has in my opinion some advantages over using executeables. Memory tracking and timing is implemented in one place, tests can be run with different “views”, a command line exe for a post build step in VS, an UI that supports result history comparing …. A test DLL’s source and project files are created from a template so that only the code for the test case itself needs to be written. Multiple test can also easily be refactored into a test-suite contained in a single module.

All this from a tools, not a game perspective – but I doubt things are that different. Most of the drawbacks only hit after a while – we’ve got about 700 tests or so covering our code base, and it’s slowly having an impact.

Out of curiosity – how much code that gets written at your place is actually TDDed? How do you keep build times low? (And what do you consider low?)

There are a few other drawbacks to TDD. It remains a great practice, but like all “best practices”, you’ve got to know the constraints. Since I’m doing the complaining, I guess that means I need to sit down and write a bit about it – stay tuned

This is also by no means the last time I’m planning on writing about TDD (noticed I created a whole category for it), so I’ll be coming back to it on a regular basis.

Al: I did have a section in the outline to talk about when TDD is not appropriate. As you said, exploratory programming does not benefit from TDD (being throwaway code that you just want to write quickly and learn from it–spikes in XP terminology).

There are also times where doing TDD is more trouble than it’s worth: writing a wrapper around an existing library, dealing very close to the hardware, the GUI layer of a tool, or super high-level game code in a scripting language. But I do believe that TDD is a great way to go for 99% of the C++ code written in games.

For Ian: graphics is indeed hard to test (though not impossible), whereas network layers is a bit more approachable. I would probably use mock objects (see mockpp for a framework that is rather well supported).