Today I had lunch with an old friend. Even though he’s not a programmer, he’s enough of a tech-head that our conversation eventually turned towards agile development. In the past, I’ve given presentations about agile game development to two distinct groups of people: game developers without much exposure to agile development, and agile developers who were unfamiliar with game development. This morning I realized how interesting it was to explain the goals and reasons behind agile development to someone completely outside those circles. It forced me to go back to basics and think about the fundamentals of a topic I’ve been so involved in that I’d stopped thinking about in those terms..

What exactly is the key concept behind agile development? One possible definition is “a set of techniques and procedures with minimal overhead that allow us to make sound, just-in-time decisions.” [1]

There are several important parts to that definition:

• Decisions: It’s not about the code, or the team, or the tests. It’s about decisions. The ultimate goal is to deliver a useful product, but agile development helps us to make the decisions along the way. What do we create next? How do we want this thing to look? What features do we have to ship with?
• Just-in-time: Maybe the agile community has a better term for this, but the concept strongly reminded me of the just-in-time compiling strategy of Java interpreters. Basically, agile methods will delay decisions until the last moment they have to be made. Not only does this help avoiding design paralysis and having to make too many decisions early on, but it also means that a lot of decisions will never have to be made because things changed from the original expectations.
• Sound: I originally left this one out, but I realized that an 8-ball can make just-in-time decisions with the best. It’s making sound ones that is difficult. Whenever a decision needs to be made, agile teams will draw on all the information they’ve acquired up until then.. Not only will they know more about market trends and competitors’ products, but they’ll also know about their own team strengths, technology limitations, or the shortcomings of past implementations. That’s a lot more information that they would have had available at the beginning of the project, during the traditional waterfall planning phase.
• Minimal overhead: Agile processes are, by definition, very light. The goal is to create a valuable product, not the process itself. Agile techniques introduce the minimal amount of overhead to achieve their purpose.

My friend was very curious about this approach, since it’s quite different from the normal way of working he’s used to in his company. He had a very insightful question: “Don’t you end up doing a lot of re-work?”. And the answer is, yes, absolutely. However, it’s not as wasteful as it sounds. It’s actually quite an efficient approach for a lot of projects.

Let’s visualize the progress made in a project as a path on a 2D surface. The amount of work is the length of the path. The progress is how far from the starting point you’ve gone, and the direction represents different approaches, designs, or implementations. An ideal project would look like this:

The project starts, knows exactly where it’s going, and progresses in a straight line. Euclid would agree: You can’t be more efficient than by taking the straight line between two points.

This diagram can be used to represent any scale: from the whole project spanning several years, to how a particular feature is implemented in a couple of weeks, or how a class is implemented in one morning. The same principles apply in all cases.

As a comparison, I suppose a death march project would look more like this:

You are constantly doing work, but you aren’t getting any closer to your destination. The project eventually gets canned, or development stops somewhere and it gets patched up the best it can and shipped to unsuspecting customers.

You might hope that an agile project looks more like this:

A project like that implies that some changes happened during development, but while always making steady progress. That would be nice, but in practice it often doesn’t work that way. When you change your plans or even some implementation, you often end up with some amount of rework:

You can see all the rework by the zigzags going in and out indicating varying levels of progress. I believe this rework is inevitable (and maybe even desirable) in agile projects. It happens at a low level, like changing interfaces to a class and having up update all the hundred tests you wrote that relied on that interface. But it also happens at a higher level, when you realize that a game feature isn’t as fun as you thought and you drop it in favor of another one that could be more promising.

From the above diagrams, agile development really looks inefficient. Clearly, we have traveled a much longer path to reach our destination (work done). But agile development can actually do better. How can we be more efficient than traveling in a straight line without resorting to strange relativistic warped space?

The diagram doesn’t tell the whole picture. Remember that agile development relies on just-in-time decisions. That includes the final goal: As we’re working towards that goal, we might see other, better or easier-to-reach ones. As a matter of fact, this is something that happens most of the time. In that case, the picture looks quite different:

Now the amount of work done is less than it was in the ideal first case! And we might have a better solution to boot. Who says you can’t have your cake and eat it too?

With this in mind, are there some cases in which it is better not to use agile development? Clearly. If you know things won’t change and you have a clear idea of how to reach your final goal, then go for it. Agile development won’t help much there. I have yet to see a project that worked that way. Maybe a sports game that ships year after year, or another simple port to a mobile platform. But most projects I’ve seen have such a high level of chaos and uncertainty due to all sorts of external factors that they could all greatly benefit from an agile approach.

This can be visualized very well by mapping the characteristics of a project in terms of requirements vs. technology. Projects that fit agile development well fall anywhere in the complicated to anarchy zone (a fair amount of technology uncertainty and requirements disagreement). On the other hand, projects in the lower left corner, which are very well understood and are very certain, would benefit the least from an agile approach. [2]

I don’t usually bash agile development because I truly believe it’s a great way to do development, especially for games. But how would an agile development project gone bad look?

Here there was a huge amount of rework without having any significant impact in the project. The project ended up very near the original goal (a bit further actually), but it spent most of its time thrashing around. You could argue that maybe some of that was exploring different solutions, but it’s probably not a sign of a good project anyway.

This brings up a good point: With agile development, it’s important to let go of the control-freak mindset. It’s good to have an idea of what your ultimate goal is, but it should never take precedence over what you discover along the way.

I see this frequently in people who have just learned the mechanics of test-driven development: they go through the cycle correctly, writing a test, coding, and then refactoring. But they’re still implementing the design they had in their head when they started instead of letting the tests dictate how the design should look. Often they get quite frustrated because they can’t figure out how to test things. This also applies at a larger scale. If a feature isn’t giving you good results, maybe it should be dropped or significantly changed.

This is one of the reasons I try to do my best not to talk about things as concrete as classes or functions during early discussions about some feature we will implement. I find that doing so tends to solidify those concepts too much and I tend to be much more likely to follow them instead of letting design evolve from what we learned during its implementation. So now instead, I prefer to talk about more abstract concepts like operations, modules, or data flow.

Whether you’re an agile developer or not, it’s always important to step back and question what you’re doing and how you’re doing it. Think about it in the context of your projects and needs, and re-evaluate it accordingly. Sometimes you’ll be surprised what new insights you achieve by stepping back and looking at the fundamentals of something you’re intimately familiar with. Just make sure you don’t bore your friends with too much technobabble.

1. Scott Ambler has a very good, much more detailed definition of agile development that describes more the practices rather than the goals.
2. Diagram adapted from Clinton Keith’s Agile Game Development GDC 2006 presentation

This article was first published in the 2005 Career Guide issue of Game Developer Magazine.

I lead the R&D team here, and my team’s primary responsibility is to come up with the technology that game teams use for different projects. Nowadays, that means putting a lot of middleware programs through their paces and choosing the ones that suit our needs. But it also means getting down and dirty and writing a lot of code for our engine and tools.

With that in mind, come on and follow me through a typical workday.

8:10 AM

I roll in to work on my bicycle like I do every day. Even though I’m an early bird, Jim, a programmer in my team, arrived a few minutes earlier and is already at his desk.

I quickly catch up with my email. I also notice that the PCLint pass on our codebase last night caught a couple of minor warnings, so I quickly fix those and check them in.

8:20 AM

Today is Tuesday and our two-week iteration ends on Friday. An iteration consists of a fixed period (usually two weeks in our case) during which the team commits to deliver a set of functionality described through customer stories. The customer (in our case the other internal teams in our company) creates and prioritizes a set of stories. My team then breaks down those stories into tasks and estimates how long they will take to complete (tasks vary between 1 and 8 hours).

Jim and I walk over to the “war room” (the room with all the task cards corresponding to user stories pinned to the wall) and we choose the task that reads “Blend ragdoll and animation,” which is part of the user story listed as “Throwing a rigid body at a character and seeing a hit reaction.” The task was originally estimated at 3 hours, but now that we know that the ragdoll system is a bit more complicated than we thought, we re-estimate it at 4 hours.

8:23 AM

In addition to our own personal desk areas, we have pair-programming stations in the R&D lab, with two monitors, two keyboards, two mice, two chairs, and plenty of room for two people. All production code is written by pairs of programmers. We grab a station and start working on the task.

Since we’re using test-driven development, we first write a very simple unit test for what we want to do, and only then write the code to make it pass. In this case, our first test checks that we create a blender object without inputs and that it produces no output. Then we write the blender class and make the test pass. It’s a tiny step, but it’s a step in the right direction. The whole cycle of write test, write code to make test pass, refactor, takes only 5-10 minutes, and we do it over and over.

8:39 AM

We have implemented a small amount of functionality, and all the code builds and all tests pass, so we check it in source control. This is called continuous integration, and it requires that programmers work on the latest version in source control and check in their own changes many times throughout the day.

8:50 AM

Other people from the team roll in, grab other pair programming stations, and start going at it. On their way in we have a quick chat and find out what we’re all working on.

Work hard…

9:37 AM

I overhear Joel and Gary discussing how they’re going to test something that requires updating the physics simulations. I just did that a couple of days ago so I jump in the discussion. It turns out they need to do something that is almost the same as what I already did, so I point them to what I wrote and they will modify it to suit their needs.

10:05 AM

Things are moving along very nicely. We’ve checked in four times already this morning. When a pair gets really going, they might check in as many as 20 or more times in a single day. At this rate we might be done sooner than the four hours we had estimated.

10:14 AM

We have the daily Scrum meeting at 10:15, so we head over to the war room. Scrum meetings are very short, standup meetings with the whole team (eight people plus Brian, our producer). We quickly go around the room discussing what people are doing to get everybody up to speed.

10:23 AM

During the meeting the topic of how we’re loading physics assets came up. So we return to the pair programming area and have a quick discussion with everybody involved. We draw some quick UML charts on a whiteboard, think about how the data is going to be passed around, and after ten minutes we reach an agreement and go back to work on our tasks.

11:15 AM

We got the blending working correctly. All the unit tests pass, although we haven’t implemented it in the demo yet. There’s another task card for that. We check the code in right away. The code definitely can stand to improve in a few places because we were just concentrating on getting things working, so we spend some time refactoring. We have lots of unit tests, so we’re confident our refactoring isn’t introducing any bugs.

11:56 AM

The code is now in a much better state. We check it in and wait a few minutes for the build server to report that everything built correctly and all tests passed. During that time we talk about what the next task should be.

12:05 PM

We have nice shoulder-high surf today, so I’m going out surfing at lunch with several of my teammates. If it’s not surfing, it’s a basketball game, cycling, running, or even yoga. If all else fails, there’s always a game of Guild Wars with the rest of the High Moon clan.

1:10 PM

Back at my desk I eat a quick lunch while I catch up on my email (which I haven’t read since this morning because I’ve been at the pair programming station). I read an email from a middleware developer in response to one of our queries earlier in the day and fire back a quick response.

1:25 PM

Sean stops by my desk. He’s ready to go back to work, but the programmer he was pairing with in the morning got pulled to work on some last-minute issues with Darkwatch, which is about to go gold and has priority over anything we’re doing.

Sean quickly brings me up to speed on what they had been working on that morning, which was to display the exact memory usage for our physics system. I remember them mentioning that in this morning’s Scrum meeting. I also worked on the physics system last week, so I’m pretty familiar with the code. In a couple of minutes we’re already making progress.

… rock hard

2:07 PM

After writing several unit tests and implementing some functionality, we’re ready to add the memory display to the demo program. A couple of minutes later, we have a display in the demo indicating how much memory the physics system is using, and we see it go up and down as we add and remove rigid bodies from the demo.

But something is wrong. When we remove rigid bodies, the memory doesn’t come down to the same level it was before. We exit the demo and we see a long memory leak dump. First things first, we check in our changes, and then we dive in and look for the code that is leaking memory. It’s probably not caused by the code we just wrote, but we have “collective code ownership,” which means that everybody is expected to fix anything that needs fixing anywhere.

2:12 PM

The build just broke! The build server detected a failed build and notified us through a system tray application. I bring up the latest build log and I see that one of the unit tests failed in release mode. Tyson, who is sitting at the station next to ours, says “Oh yeah, I know what that is. I’ll fix it right now.” In less than 30 seconds he makes the change, and checks it in. A few minutes later, the build system reports a passing build and everything is back to normal.

2:17 PM

We found the memory leak. It was a misuse of reference counting. We first wrote a unit test that showed it failing, and then we fixed in the physics library. We check in our code.

2:18 PM

We go to the war room and grab the next task. This one has to do with being able to expose different variables and functions on the demo to tweak them through a GUI. We sign up for the task and start working on it.

3:43 PM

We’re totally in the zone. We’ve been writing tests and code like crazy and this task is going really well. We’ve done three check-ins for this task alone in the last hour.

4:12 PM

Another pair is discussing how to handle errors for some particular case. This is an important topic and it should be done consistently across all the code, so we have a quick discussion about it involving most of the team. Five minutes later we’ve made a decision and we all resume working on what we were doing before.

5:40 PM

We do our last check in for the day. We’re almost done with the task, but not quite there yet. Even though we could stay another hour and try to finish it, we’re both quite tired by now and we’re starting to not think as clearly and make some mistakes. We can wrap this up tomorrow morning as soon as we get in. The important part is that we got to a state where we could check in.

We have a rule that says nobody can check in code and leave. You have to wait for the build server to build the code successfully and pass all the tests. We keep build times short, so that usually means hanging around an extra 4-5 minutes. If anything breaks, you need to fix it or revert what you did, but there’s no excuse to leave with a broken build.

While I’m waiting for the build server to finish, I check the pile of email that accumulated in my inbox during the day.

5:44 PM

After a few minutes we get the green go ahead from the build server. Today was a pretty productive day, and at this pace we will definitely complete all the user stories by Friday. The demo we’re putting together is also starting to look very cool.

One of the things that agile development, and especially pair programming, does is to make each day very intense. There are no little breaks to read email, check a web site, or just goof around. We get a lot accomplished in a work day, but we can’t keep that pace for a long time, so it’s important to call it quits and go home. That leaves me with time at home to read technical books, prototype different ideas, or work on side projects in addition to unwinding, spending time with my family, and enjoying other hobbies.

I hop on my bike and head home with a big smile on my face.

This talk discusses how agile development can help in all those scenarios. In particular we look at methodologies like XP and Scrum, and techniques such as test-driven development and pair programming.

Slides from the Montréal Game Summit 2004 talk (pdf format)