Exactly a year ago today, I put up the first article on Games from Within. It was a review of Tom DeMarco’s book Slack. I thought it would make for a nice, symmetrical bookend to wrap the year up with a review for another book by DeMarco: Waltzing with Bears.
As the subtitle indicates, Waltzing with Bears deals with managing risk in software development projects. Managing risk, not reducing risk, or removing risk. Do you think that low risk or even no risk is a good thing? Think again. One of the central points of the book is that a project with no risk is not worth doing. Yes, you read that correctly. Intrigued? Go and read the book right now.
I have to admit that the idea caught me by surprise as well. Maybe because I was used to the high-risk environment of game development, I always thought the best thing you could do for a project was to minimize the risk. But, as the authors point out, a project with no risk is not worth doing. It’s not going to enrich you in any way, and anybody can do it anyway. Their point is that risk and benefits are tied together. A project with a certain amount of risk will come with a certain amount of benefits.
Now that I’ve gotten quite a bit into playing poker, I see it very much like the analysis of expectation in poker. You want to bet when there’s a positive expected value, that is, when the potential benefits you’ll get from the bet outweigh the potential loss of money. As with poker, you can choose little risk and little benefit situations, or you can aim for high risk but huge potential payoffs (especially in no-limit poker). What you want to avoid are dead-end situations with high risk and very little payoff.
In particular, the authors single out death march projects as being of extremely low value.
On a death march, unflinching sacrifice from each and every project member is absolutely required. The project demands abandonment of personal life, tons of overtime, Saturdays and Sundays in the office, estrangement from family, and so on. Nothing less than total dedication to the project can be accepted.
Sounds strangely familiar, doesn’t it? The book then goes on to say:
In our experience, the one common characteristic among death-march projects is low expected value. They are projects aimed at putting out products of monumental insignificance. The only real justification for the death march is that with value so minuscule, doing the project at normal cost would clearly result in costs that are greater than benefits. Only with heroic effort can one hope to make the pig fly.
Next time you’re deep into your third 60+ hour week in a row, think about those statements again and try to look at the big picture of your project in a new light. That just adds more fuel to the fire to my arguments against extended overtime and death march projects in general.
There are some things you need to watch out for if you’re going to start directly managing risk in your organization. The biggest problem you might encounter is that to manage risk you need to be brutally honest about the current situation of the project and possible negative outcomes. In a company where risk management is not a usual activity, it may come across as being overly negative, and you might scare away developers or even upper management. It can be particularly dangerous to do in a company with a very macho, “can-do†attitude (also described in the book as testosterone-based decision making), which is unfortunately all too familiar in the games industry.
One of the tools used throughout the book to describe projects and risks is probability curves. This emphasizes how things are not black and white. You don’t necessarily answer the question “when is the project going to be completed?†with a firm date, but with a probability curve. Looking at it tells you how likely the project is of being completed by a particular date (assuming the analysis was done correctly). The more “noise†and uncertainty there is in the project, the wider the curve is going to be. Unfortunately, it seems that it’s describing the games industry to a T again, because we tend to have a lot of uncertainty in our projects with moving technology targets and a somewhat whimsical market.
Another very interesting observation brought up in Waltzing with Bears is the concept of “early delivery.†Of all the games you worked on, how many of them were delivered early? Yeah, that’s what I figured. Same thing with me. Not a single one. Early delivery is seen as somewhat of a taboo. As if you didn’t work hard enough or you could have done a better job otherwise. So clearly, companies will never deliver a project earlier than the commit date. At best, they’ll do it by the exact deadline. At worst, the project will slip. As long as the two only possible outcomes are “on time†or “late,†then projects are very likely to keep being late. That’s some food for thought.
Later in the book, in the chapter about risk mitigation strategies, the authors make one very astute observation that I’ve seen repeated over and over in the projects I’ve worked on: projects that finish late are almost always projects that started too late. Truer words were never spoken. There are often very good reasons why a project can’t get started when it should (helping out another project, lack of funding, etc), but those weeks or months lost early on would be precious towards the end.
Perhaps it’s because I have a hopeless breadth-first personality, but I hate when books try to cover every single insignificant detail from the very beginning before giving you the big picture. I would much rather get a couple of passes, each building on what I learned in the previous one. That’s exactly what Waltzing with Bears does, with the first 50 pages dealing with an introduction and motivation to risk management, and the remaining 140 pages covering the same material in more depth and giving it a more advanced treatment. More books should learn from that organization.
Just go ahead and read the book and then pass it on to your manager. It’s short and well worth the read. The important thing is that somebody in your project should be actively managing risk and balancing how much risk you take versus what the potential benefits will be. Otherwise you might run into some very ugly situations that might lead to the cancellation of the project itself.
May 2005 be everything you hope for, and may your projects be managed wisely, with just the right amount of risk. Happy holidays to everybody!
I read this article on GamaSutra today:
http://www.gamasutra.com/features/20041229/kelly_01.shtml
I haven’t read the book but it makes an interesting comparison with your review.
I disagree with you on one point. There should not simply be “somebody on your project managing risk”. That is something that management cannot deal with alone, or even fully understand. Everyone but the most junior employee should be aware of and managing their risks and those of their team and their project as a whole. I would argue that a programmer with average technical skill but a sound understanding of risk is far more valuable than a programmer with extraordinary technical skill but no awareness of risk.
It seems to me that it is impossible to consider risk in isolation. At a minimum, here is a trade-off between risk, timeliness and quality. Are these issues considered in this book?
In our experience, the one common characteristic among death-march projects is low expected value. They are projects aimed at putting out products of monumental insignificance. The only real justification for the death march is that with value so minuscule, doing the project at normal cost would clearly result in costs that are greater than benefits. Only with heroic effort can one hope to make the pig fly.
Maybe I have the wrong definition of death march but if it means 6 or more months of overtime then I can tell you 100% for sure that the above statement is pure BS in the game industry. I would be willing to bet that every nearly every hit game in the industry required heroic amounts of overtime to ship. Clearly the justification was very real.
I think that statement might be more applicable to the games industry that you may think.
Leaving aside other potential reasons for working overtime (which I’ve discussed quite a bit here before: http://www.gamesfromwithin.com/articles/0412/000054.html ), I think it can still apply to a large percentage of the projects in the games industry.
The book says “the one common characteristic among death-march projects is low expected value”. It doesn’t say anything about whether the projects were a mega hit or not. Just that not much was expected out of them.
If a company is making a game and they know it’s going to sell 5 million units, it would stand to reason that they would put whatever resources were necessary behind the project. In the grand scheme of things, spending an extra million bucks in development is nothing compared to the many more millions of profit that it will reap.
On the other hand, if they’re not sure how well the game is going to do, then they’ll try to cut every corner and minimize costs on the hopes of breaking even or not losing much money (low expected value).
The games industry is a very hit-driven industry, and we don’t often know we have a mega hit in our hands (unless our game is called Halo 2 or Half Life 2), which might explain some of the attitude from publishers.
In any case, low expected value or lack of resources are not the only reason for overtime in the games industry. Which is unfortunate because those could be relatively easy to fix compared to ingrained cultural issues.
“I would be willing to bet that every nearly every hit game in the industry required heroic amounts of overtime to ship. Clearly the justification was very real.”
I would be willing to bet that nearly every failure in the industry was accompanied by “heroic” amounts of overtime, too, but still failed. Clearly the justification is no justification at all.
To look at it from another angle… all the successful games may have been GIVEN “heroic” amounts of overtime, but who’s to say they would have failed if they hadn’t, and there had instead been a sensible, rational, realistic project management solution?
Either you think a game is a reasonable investment – or you cancel it, and shift the resources to a project you think is a better bet. Or apply Pareto’s Law, and cut high cost low value features. If you’re not doing this, you’re not managing properly.
Except… there often is no single managerial “you” in this business, because responsibility is split between publisher and developer, lead programmer, producer, CEO, marketing and business development. They’ve all got different interests and perceptions, and for many of them keeping a team and developers sane functioning isn’t a rewarded activity.
“In our experience, the one common characteristic among death-march projects is low expected value. They are projects aimed at putting out products of monumental insignificance…”
Boy this quote really speaks to me. I’ll definitely pick the book up. The last (non game) project I was on matches this perfectly. The project was to do a complete rewrite of UI and middle layers of an existing system. Management didn’t want to spend much time on this because a completely new system was in the works that would replace the old. But in the mean time the existing product was so unpopular a stop gap was needed for customers. Given the requirements and (small) dev resources we came up with a good design and a fairly aggressive implementation schedule of 10-11 months. However this was unacceptable and the product had to ship in 6 months. This was a low value project and management wasn’t willing to pay the costs in time and resources. At this point fun new macho “can-do” terminology emerged and I discovered we were in “internet time” implementing a “compressed development cycle”. What this meant was that everyone single person had to do the work of two. Crunch time started at the very first milestone and dinners were brought in for late hours. To make a long story short, the product shipped 14 months later with very questionable quality. Work on the new system was slowed and still at least two years out.
This was without doubt the worst managed project I’ve ever been on and seems fully predictable from this book. Another sad but funny anecdote is how management motivated the developers. We had a “ship party” at the head honcho’s home about 4 months before actual ship (management was still in a state of denial). This “home” was a large mansion complete with a full blown movie theater (including lobby and wet bar) built on the bottom floor. I don’t know the message management was trying to convey, unless it was “we really appreciate the unending long hours you work so we can continue to live in total decadence”.