The second session of the GDC 2004 roundtable “By The Books: Software Engineering in the Games Industry” concentrated on processes and methodologies. In particular, we had a good look at agile development and how it can be applied to game development.
Thursday 25th: Processes and Methodologies
From the initial discussions and show of hands it was clear the most people did not have one specific methodology that they followed during development. The majority of the participants admitted to using some sort of “code and fix” (or “hack and slash”, depending on the severity) approach. About 20% of the participants claimed to be using a more controlled iterative approach, while only about 6% were doing some form of agile development.
Agile development is a hot topic now and people seem very curious about how it can be adopted for game development since it seems to fit well with the chaotic and ever-changing nature of our industry. The rest of the session covered several aspects of agile development in detail.
SCRUM is a specific agile development process (like its more famous cousin extreme programming). It is best described in the book Agile Software Development with SCRUM by Ken Schawaber and Mike Beedle. SCRUM is not as radical a departure from traditional iterative development as extreme programming, so it might be easier to adopt for more conservative shops. SCRUM is currently being used at Sammy Studios with very positive results (but it has only been in place for slightly over a month). I’m really looking forward to hearing from them next year to see how it worked once they have more experience with it.
Perhaps the most characteristic part of SCRUM is its emphasis on 30-day iterations (called sprints). Like other agile methodologies, it requires small tasks (4-16 hours long), task estimates from the programmers, and continuous re-evaluation and prioritization. Meetings are frequent but very quick (no sitting allowed!). Applied to game development, the priorities for the tasks should come from the designers and the producers.
SCRUM allows for very good project visibility by keeping track of what tasks are completed and which ones are left for each iteration. It also allows to measure the development velocity (how many hours of real programming are done each day), which is invaluable when making estimates and schedules for the next 30-day sprint.
Test-driven development is an integral part of many agile-development methodologies. For every feature, test-driven development requires that a unit test be written first before the feature is implemented. This is applied to all features of the program, and tests are refactored mercilessly as needed. Some good books on this subject are Test Driven Development: By Example by Kent Beck, and Test Driven Development: A Practical Guide by David Astels.
A surprisingly-high 18% of the participants were doing some form of test-driven development (especially considering that last year only a handful of people were doing any sort of unit testing at all).
The unit tests generated are a great form of regression tests for the code. Whenever anything breaks that has a unit test, you’ll know it right away (assuming the unit tests are executed very frequently). A good rule is that whenever a bug is found, first a test must be written that shows the bug, and only then the bug is fixed. That ensures it won’t pop up again accidentally. Other benefits of test-driven development is that it narrows down where to look for existing bugs, it serves as a form of documentation that can never get out of date, it helps tremendously with refactoring, and it forces a more modular design.
On the down side, it requires a large amount of “extra” code. People reported ratios anywhere from 1:1 to 2:1 of test code to application code. Some systems can be very difficult to test with unit tests, especially high-level systems or non-deterministic ones. Some people were using unit tests that compared the output of the renderer pixel by pixel with previous screenshots for some very specific rendering features.
Two people in the roundtable were doing full extreme programming, but almost 20% of the people were using one or more features from extreme programming. The best introduction to extreme programming is the white book: Extreme Programming Explained: Embrace Change by Kent Beck.
Pair programming is one of the most controversial extreme programming features. A lot of people were doing it but only at some specific times (like near a milestone). One participant tried doing full pair programming for a small tool project and reported very positive results with it. Some of the advantages listed for pair programming was dissemination of knowledge, and training of junior developers. Interestingly, someone pointed out that pairing an experienced developer with a junior one was often a great combination because you would get the stability of the experienced developer but the fresh ideas and perspectives of the junior one. Just about everybody agreed that pair programming was much more useful than code reviews, which are often too late to do anything about any code problems. Also, pairing with another person could force developers to concentrate for longer periods of time without checking their email, answering the phone, or browsing the web.
Of course, not everybody likes pair programming. Some people feel they can be more productive by themselves, and especially if they are not comfortable with the other person or they think in totally different ways. The final output of two people pairing won’t be 2x the amount of code, but it’s supposed to be more than 2x as reliable (which means less maintenance and bugs down the road).
One interesting question was how to convince management to adopt extreme programming. The best answer was to make sure you don’t use the words extreme programming (and certainly don’t write it eXtreme programming or you’ll be doomed! ). Talk about what specific processes you want to put in place, or even talk about agile development instead. Another view was to start applying some of the techniques of extreme programming without direct management approval, such as test-driven development, refactoring, or pair programming. Showing management a cost-analysis study comparing extreme programming and plain code-and-fix could be another, more direct approach.
Extreme programming promotes an open-office environment (the bullpen environment). The possible noise and distractions are supposed to make up for the increase in communication. The participants were clearly divided. Some of them found such an environment very useful (to the point of putting the computers on desks with wheels to arrange things in whichever way made more sense for that particular project), but others wanted to have the peace and quiet of a single office that allows them to concentrate and be productive. Some of the people who had an open environment also had private offices where people could go sometimes when they needed to concentrate more. Finally, nobody seemed to like cubicles very much. They’re almost the worst of both worlds: noisy, but not as open to communication as a fully open environment.
Previous GDC roundtables