Bad News for Scons Fans

We have been talking a lot about Scons recently at the Power of Two Games World Headquarters. MSBuild has proven to be quite a pain to work with for our asset builds and eventually left us dissatisfied (that’s material for a whole other entry). So we kept looking over to Scons as a possible solution.

On paper it looks great:

  • Built from the ground up for parallel builds
  • Uses a real programming language (Python) instead of overly-complicated XML files
  • It allows for discovery of assets to build as other assets are parsed

The big question mark hanging over it was whether it was fast enough. I had done some experiments with Scons and other build systems a few years ago, and Scons totally failed by having ridiculously long incremental build times just checking if anything had changed. That was simply not acceptable. If no data has changed, I want the build system to detect it and come back in a second or less.

Since then, I’ve been told that Scons had finally fixed its dependency checking and it was much faster now. Music to my ears!

So I resurrected the old script I wrote back then to generate a nightmare stress test, downloaded the latest stable version of Scons (1.0.1) and fired it up.

I’m sorry to report that Scons is still as unusable as it was back then. I generated a codebase like the one I used to measure all the build systems (50 libraries, 100 classes each, 15 internal includes, 5 external includes). I built everything once, then ran it again and it took 49 seconds. I have some pretty nice hardware, including a fast SATA hard drive and a 2.8GHz Core2Duo with plenty of RAM, and 49 seconds to say that nothing needs to be changed just doesn’t cut it.

Now to be fair, right now we’re looking at Scons to build our assets, not our code. Maybe the nested project configuration is not particularly realistic, so I tried something simpler: 5000 files in a single library without any dependencies among each other at all. This is a very conservative example for an asset build of a large commercial game.

  • It’s only 5000 files. A real game can easily have 10 times as many asset files.
  • All the files are tiny (cpp files in this case). Assets can get very large when dealing with textures, sounds, and animations.
  • There are zero dependencies among assets in this example. You’ll often have models depending on shaders and textures, levels on game entity definitions, etc.

Building assets doesn’t have to be the instant, no delay kind of build that I require building code. I expect most developers will be building assets locally, tweaking things, rebuilding, and trying them in the game. The faster the build is, the better, but a delay of a few seconds can be acceptable. If Scons can’t handle this build in less than 10 seconds, it’s certainly doomed in a full-scale game.

The result: 37 seconds!

For purely masochistic reasons, I decided to see how it scales, so I threw 20,000 files at it (again, without any dependencies). Incremental build time without any changes: 3 minutes and 46 seconds. That’s over 6 times longer for processing 4 times the number of files, so it doesn’t even scale linearly.

Scons is definitely too slow, and MSBuild has its share of problems. What’s left out there? Any recommendations for good parallel asset building systems? Hopefully something lightweight and easily configurable through a real language. Anything?