Fundamentally Agile

In the past, I’ve given presentations about agile game development to two distinct groups of people: game developers without much exposure to agile development, and agile developers who were unfamiliar with game development. This morning I realized how interesting it was to explain the goals and reasons behind agile development to someone completely outside those circles.

Today I had lunch with an old friend. Even though he’s not a programmer, he’s enough of a tech-head that our conversation eventually turned towards agile development. In the past, I’ve given presentations about agile game development to two distinct groups of people: game developers without much exposure to agile development, and agile developers who were unfamiliar with game development. This morning I realized how interesting it was to explain the goals and reasons behind agile development to someone completely outside those circles. It forced me to go back to basics and think about the fundamentals of a topic I’ve been so involved in that I’d stopped thinking about in those terms..

What exactly is the key concept behind agile development? One possible definition is “a set of techniques and procedures with minimal overhead that allow us to make sound, just-in-time decisions.” [1]

There are several important parts to that definition:

• Decisions: It’s not about the code, or the team, or the tests. It’s about decisions. The ultimate goal is to deliver a useful product, but agile development helps us to make the decisions along the way. What do we create next? How do we want this thing to look? What features do we have to ship with?
• Just-in-time: Maybe the agile community has a better term for this, but the concept strongly reminded me of the just-in-time compiling strategy of Java interpreters. Basically, agile methods will delay decisions until the last moment they have to be made. Not only does this help avoiding design paralysis and having to make too many decisions early on, but it also means that a lot of decisions will never have to be made because things changed from the original expectations.
• Sound: I originally left this one out, but I realized that an 8-ball can make just-in-time decisions with the best. It’s making sound ones that is difficult. Whenever a decision needs to be made, agile teams will draw on all the information they’ve acquired up until then.. Not only will they know more about market trends and competitors’ products, but they’ll also know about their own team strengths, technology limitations, or the shortcomings of past implementations. That’s a lot more information that they would have had available at the beginning of the project, during the traditional waterfall planning phase.
• Minimal overhead: Agile processes are, by definition, very light. The goal is to create a valuable product, not the process itself. Agile techniques introduce the minimal amount of overhead to achieve their purpose.

My friend was very curious about this approach, since it’s quite different from the normal way of working he’s used to in his company. He had a very insightful question: “Don’t you end up doing a lot of re-work?”. And the answer is, yes, absolutely. However, it’s not as wasteful as it sounds. It’s actually quite an efficient approach for a lot of projects.

Let’s visualize the progress made in a project as a path on a 2D surface. The amount of work is the length of the path. The progress is how far from the starting point you’ve gone, and the direction represents different approaches, designs, or implementations. An ideal project would look like this:

The project starts, knows exactly where it’s going, and progresses in a straight line. Euclid would agree: You can’t be more efficient than by taking the straight line between two points.

This diagram can be used to represent any scale: from the whole project spanning several years, to how a particular feature is implemented in a couple of weeks, or how a class is implemented in one morning. The same principles apply in all cases.

As a comparison, I suppose a death march project would look more like this:

You are constantly doing work, but you aren’t getting any closer to your destination. The project eventually gets canned, or development stops somewhere and it gets patched up the best it can and shipped to unsuspecting customers.

You might hope that an agile project looks more like this:

A project like that implies that some changes happened during development, but while always making steady progress. That would be nice, but in practice it often doesn’t work that way. When you change your plans or even some implementation, you often end up with some amount of rework:

You can see all the rework by the zigzags going in and out indicating varying levels of progress. I believe this rework is inevitable (and maybe even desirable) in agile projects. It happens at a low level, like changing interfaces to a class and having up update all the hundred tests you wrote that relied on that interface. But it also happens at a higher level, when you realize that a game feature isn’t as fun as you thought and you drop it in favor of another one that could be more promising.

From the above diagrams, agile development really looks inefficient. Clearly, we have traveled a much longer path to reach our destination (work done). But agile development can actually do better. How can we be more efficient than traveling in a straight line without resorting to strange relativistic warped space?

The diagram doesn’t tell the whole picture. Remember that agile development relies on just-in-time decisions. That includes the final goal: As we’re working towards that goal, we might see other, better or easier-to-reach ones. As a matter of fact, this is something that happens most of the time. In that case, the picture looks quite different:

Now the amount of work done is less than it was in the ideal first case! And we might have a better solution to boot. Who says you can’t have your cake and eat it too?

With this in mind, are there some cases in which it is better not to use agile development? Clearly. If you know things won’t change and you have a clear idea of how to reach your final goal, then go for it. Agile development won’t help much there. I have yet to see a project that worked that way. Maybe a sports game that ships year after year, or another simple port to a mobile platform. But most projects I’ve seen have such a high level of chaos and uncertainty due to all sorts of external factors that they could all greatly benefit from an agile approach.

This can be visualized very well by mapping the characteristics of a project in terms of requirements vs. technology. Projects that fit agile development well fall anywhere in the complicated to anarchy zone (a fair amount of technology uncertainty and requirements disagreement). On the other hand, projects in the lower left corner, which are very well understood and are very certain, would benefit the least from an agile approach. [2]

I don’t usually bash agile development because I truly believe it’s a great way to do development, especially for games. But how would an agile development project gone bad look?

Here there was a huge amount of rework without having any significant impact in the project. The project ended up very near the original goal (a bit further actually), but it spent most of its time thrashing around. You could argue that maybe some of that was exploring different solutions, but it’s probably not a sign of a good project anyway.

This brings up a good point: With agile development, it’s important to let go of the control-freak mindset. It’s good to have an idea of what your ultimate goal is, but it should never take precedence over what you discover along the way.

I see this frequently in people who have just learned the mechanics of test-driven development: they go through the cycle correctly, writing a test, coding, and then refactoring. But they’re still implementing the design they had in their head when they started instead of letting the tests dictate how the design should look. Often they get quite frustrated because they can’t figure out how to test things. This also applies at a larger scale. If a feature isn’t giving you good results, maybe it should be dropped or significantly changed.

This is one of the reasons I try to do my best not to talk about things as concrete as classes or functions during early discussions about some feature we will implement. I find that doing so tends to solidify those concepts too much and I tend to be much more likely to follow them instead of letting design evolve from what we learned during its implementation. So now instead, I prefer to talk about more abstract concepts like operations, modules, or data flow.

Whether you’re an agile developer or not, it’s always important to step back and question what you’re doing and how you’re doing it. Think about it in the context of your projects and needs, and re-evaluate it accordingly. Sometimes you’ll be surprised what new insights you achieve by stepping back and looking at the fundamentals of something you’re intimately familiar with. Just make sure you don’t bore your friends with too much technobabble.

1. Scott Ambler has a very good, much more detailed definition of agile development that describes more the practices rather than the goals.
2. Diagram adapted from Clinton Keith’s Agile Game Development GDC 2006 presentation