There were even a couple of cases during Casey’s Contraptions that we wanted a particular effect, and couldn’t get it quite right, but it would have been trivial to whip up a shader if we had been using OpenGL ES 2.0. In the end, we had to resort to texture combiners (yuck), and it wasn’t exactly what we had in mind.OpenGL ES 2.0 requires that you write shaders for any different kind of “thing” you render, whether it’s a simple 2D quad, or a fancy skinned character. I wanted to provide at least a basic 2D rendering shader in my graphics library, but the workflow was a bit awkward.

First of all, shaders on iOS are compiled at runtime [1], so we need to access the shader source and compile it. Initially I was thinking of having the shader source files as text in the graphics library, but for the game to access those files, it would have to explicitly add them to its resources so they can be copied during the build phase. That’s error prone (I would always forget to do that with every new project) and just plain ugly.

Another possibility would be to include the source text as a string in the library. That would simplify things, but it would mean that:

1. You would lose shader syntax highlighting.
2. You would need to have the annoying \ at the end of every line.

After asking on Twitter and getting some good suggestions, I decided the easiest way would be to use xxd. It’s a shell tool that comes preinstalled in all Macs (and Linux) that takes a file and converts it into a C array that can be included in a program directly.

I liked xxd even better than the old standby of bin2c because it already comes installed, and it generates the whole file, not just an array that then needs to be included from another file. Yes, it would be easy enough to modify, but why bother when you have a perfect tool for the job already?

Now, I can set up a custom rule in Xcode 4 like this:

Update: The array generated by xxd isn’t null-terminated, so we need to add a zero to the array. The easiest way is to use sed. Thanks to SixEcho Studios for the heads up. Also, I updated it to use the varaibles INPUT_FILE_DIR and INPUT_FILE_NAME thanks to Miguel’s comments.

Now, whenever the shader source changes, it generates a header file like this:

unsigned char PassThrough_fsh[] = {
  0x0a, 0x0a, 0x76, 0x61, 0x72, 0x79, 0x69, 0x6e, 0x67, 0x20, 0x6c, 0x6f,
  0x77, 0x70, 0x20, 0x76, 0x65, 0x63, 0x34, 0x20, 0x44, 0x65, 0x73, 0x74,
  0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x43, 0x6f, 0x6c, 0x6f, 0x72,
  0x3b, 0x0a, 0x0a, 0x76, 0x6f, 0x69, 0x64, 0x20, 0x6d, 0x61, 0x69, 0x6e,
  0x28, 0x29, 0x0a, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x67, 0x6c, 0x5f,
  0x46, 0x72, 0x61, 0x67, 0x43, 0x6f, 0x6c, 0x6f, 0x72, 0x20, 0x3d, 0x20,
  0x44, 0x65, 0x73, 0x74, 0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x43,
  0x6f, 0x6c, 0x6f, 0x72, 0x3b, 0x0a, 0x7d
,0x00};
unsigned int PassThrough_fsh_len = 91;

To compile that shader in the game, all I have to add is two lines:

#include 
//...
const int fs = ShaderUtils::CompileShader(PassThrough_fsh, GL_FRAGMENT_SHADER);

Done! Clean, simple, and minimal cruft.

Update: I haven’t tried it yet because I haven’t had the need for any expensive shaders, but I hear really good things about Aras Pranckevičius GLSL optimizer. In that case, I would add it before the xxd step. You would still use the shader the same way, but it would be optimized and perform faster.

[1] That’s a huge mistake and I hope Apple fixes that ASAP. Loading times for any amount of non-trivial shaders adds up really quickly. That’s the last thing you want on a mobile device! The OpenGL ES 2.0 standard provides an optional way to load binary shaders, and all that Apple has to do is provide a tool that will compile shaders for the iOS hardware. To be completely future-proof, you they can even mandate that if the binary load fails, you still need to compile from source.

Contrary to popular belief, I’m not that picky when it comes to a development environment. Or at least, it doesn’t feel that way to me. I want the basics, that’s all. Doesn’t everybody want this?

• Have 2-3 source code windows opened side by side.
• Build and see the output.
• Do everything with keyboard shortcuts.

There are lots of other things I want (project configuration, debugging, run tests), but those are the ones I’m doing 99% of the time. I never had a problem doing any of those things with past development environments.

The hit list

Having more tools and options is great. I love having the option to do advanced operations on my code, but I want to decide when to use those tools, not being bombarded by them. That, in a single sentence, is why Xcode 4, out of the box, is completely unusable for me.

Attempting to do something as simple as type code and build it is extremely frustrating. Xcode will react to me doing that by flashing all sorts of stuff on screen, bring up sidebars with messages, change which source files are displayed, draw squiggles all over my code, and flash alerts overlays. All of that, while not showing me the build output.

Working on Xcode is an extremely noisy (visually) and distracting experience. I understand it’s trying help me, but it’s failing miserably at it. I’m trying to concentrate on what the code does, and I feel I have a parrot yelling stuff from my shoulder and flapping in front of the screen in some misguided attempt to help me code better.

That might work for some people, but not for me. I’m the guy who turns off “spell check as I type” everywhere because I don’t want to be taken out of flow while I’m typing (I’ll do a spell checking pass later on, thank you). I don’t have notifications on my mail or Twitter; I check them explicitly when I want to. Now take that kind of annoyances and multiply it times 10 and you start getting an idea of what working on Xcode 4 is like. Xcode 4 feels like it’s squarely designed for the texting/ADD generation.

Fortunately, most of it can be remedied and turned into a half-way sane environment.

Here’s how to tame the noise in Xcode 4:

• Autocomplete is one of the worst offenders. Turn it off! Good autocomplete is awesome, but I’d rather ask for it (with a keystroke) than have it bombard me constantly. Not just that, but it’s so badly implemented that it will a) put shadows on top of the text I’m typing b) replace the text I’m typing on the fly so I can’t even see what I typed (see screenshot below). Off with it!



• Having the whole screen flash with squiggles on and off while I’m typing isn’t helping in any way. Xcode is trying to “highlight instances of selected symbol”, which is a really useful tool, but not while I’m typing/thinking! Not just that, but half the time it will add extra squiggles to the screen to make sure I can’t even read the symbols themselves. So off with that one too.

  

  Here’s one very frustrating thing: When I turn off the option to automatically highlight instances of the selected symbol from the settings, I lose the ability to do that at all. Yes, you heard that right: I can’t (as far as I can tell) press a key and have the IDE highlight all the instances on demand. No sir, that’s not the way it was intended.

• Continuing with the theme of noise and distractions, having a sidebar popup and start showing me errors with my code as I’m typing is also definitely NOT helping. Fortunately we can turn off “Show live issues” in the general settings.
• One thing that Xcode 4 appeared to have gotten right is the idea of showing multiple source files at the same time. Now many IDEs do that by default these days, so that was a plus. Unfortunately, it’s part of the “smart” assistant. As a general rule, I end up turning off anything with the word “smart” in it, and this is no exception. The assistant tries to present “relevant” files to the one I’m working on. No, no, no, no! *I* want to decide what to display and what to work on. Just get out of the way and let me do that easily. You can turn off the view to “Manual”, which helps a huge amount.

Just those changes make Xcode 4 into something that at least I can type into and not get sick at my stomach. We’re making progress.

Now I can finally build and… everything goes wrong. Xcode doesn’t show me the build output and it decides to flash a “build completed” screen overlay in case I looked away in the two seconds it took to build. Seriously? Maybe my attention was drawn into some of the flashing squiggles and I forgot it was building.

Fortunately for my sanity, it’s possible to fix both those things. The alert is easy: Go to behaviors, “Build succeeds” and turn off “Show bezel alert” (make sure it’s off for all the others while you’re there).

The other one is a bit trickier, but it’s certainly possible. Set up the “Build starts” behavior to open a named tab and go to the current log. Like this:

Now when you build your project, you’ll get to see something sane like this. It might initially be as a tab, but you can drag that out and make it into a separate window of its own so it doesn’t obscure the source code window:

You should be able to navigate through any issues found during the build with Cmd + ‘, or, if you like the fancy issue display, press Cmd + 4. Things are so much better when you initiate them on demand rather than when they’re forced on you!

Finally, a couple minor UI annoyances that are easily fixed:

• The toolbar is positively huge. I feel crammed in the high-res 15″ MBP (1680 x 1050), so I can’t imagine how anyone works with something smaller. In any case, hiding the toolbar helps tremendously. As a bonus, the setting is per window, so your source code windows get the extra real estate, but the build output window keeps the toolbar so you have easy access to changing the target platform.
• Line wrap is on by default. Seriously? Off.

The suck list

If that was everything, I’d be happy. It means that even though Xcode 4 ships with retarded defaults, it can be whipped into shape into something usable. Unfortunately, there are a two major things I haven’t found a workaround for:

• Can’t build a single file. Someone at Apple, in their infinite wisdom, decided that building a single file was obsolete with the awesome new feature of “show live features”. Except that “show live features” sucks and has to be turned off. Apparently you either take it their way, or the highway, because there isn’t an alternative. As with the case of highlighting symbol instances, there isn’t a command I can use to “show live features” on demand.

  The alternative is doing a full build. Most of the time that’s fine, but whenever you’re doing one of those refactorings that breaks half the codebase, then you’re really screwed. The only way I can stay focused in a case like that is by cleaning up one file at the time, but this makes it impossible as you keep getting errors from all over the codebase. And as you fix one, the first error in the next build might be in a different file. Very annoying to say the least, especially because this was working and they removed it on purpose. I never file radar bugs (long story), but I made an exception just for this.

• Build configurations and schemes. Did you notice that when you built the project earlier there was no option to choose debug vs. release or some other configuration? That’s another one of those decisions that make you wonder if Apple uses Xcode to manage their own projects (and if they do, what kind of developers or projects they have).

  They turned something simple like project configurations that nobody ever complained about, into something byzantine worthy of some of the best efforts from Redmond. This gets the award for “most complicated feature nobody needed or wanted”. It seems now we have an extra layer of indirection. So you act on a scheme, which then decides which configuration to build and what to do. That’s why there are “Build for analysis” (debug), “Build for running” (release), “Build for testing” (WTF?), “Build for archiving” (distribution?).

  After spending some time with it, I’m still confused how I would go about running the release configuration, or how to mass-edit all the schemes at once (every target seems to get a different scheme).

  The whole schemes thing leaves me completely speechless, and that vein on the side of my forehead is pulsating just from thinking about it, so let’s just move on.

The good

After you take the time to fix Xcode 4 into something usable, and (somehow) put up with the new broken features, is there something to like?

I had to dig deep, but I found a couple of things that I like better than Xcode 3:

• Much better project settings UI: Multiple panes, showing multiple values for different configurations, etc. Way better than Xcode 3.
• Better semantic analysis. During the build process, Xcode 4 was able to correctly flag some problems in my code that Xcode 3 never warned me about. Mostly to do with constness of parent-scope variable when using blocks.
• Better keyboard shortcut support for opening files in different windows (assistant pressing the Option key). I never found a way to open a file in a specific window in Xcode 3 without having to use the mouse.

And that’s pretty much the whole good list that I was able to find

Conclusion

Xcode has been progressively evolving from version 1 until version 3.2. I have no doubt that’s the reason for some of the quirks in the later versions. Xcode 4 seems to be a total re-design and re-implementation, and suffers from the classic second system effect. It’s change for the sake of change, most of the time scrapping useful features and not offering useful alternatives.

I’m sure I only scrapped the surface of the horrors of Xcode 4 in the day I spent with it. I didn’t look very closely at the debugger or the Interface Builder (saving those for another day). I keep hearing about developers with stability problems with Xcode, although I didn’t run into any crashes (but I hardly did any real work with it).

Overall, my recommendation is that, if you’re comfortably working on Xcode 3, stay as far as possible from Xcode 4. We’ll all be forced to move there sooner or later, but in the meanwhile you can continue being productive. Maybe by the time they discontinue Xcode 3, Xcode 4 will have improved a bit.

These are mostly quick notes to myself so I remember what to change when I upgrade my main machine, but I thought other developers hesitant to upgrade to Lion might find it useful as well.

Things that had to go

I found Snow Leopard (and Leopard before it) to be very palatable out of the box. I don’t remember any things I absolutely had to do. Lion, however, had some questionable defaults. Or at least, ones that I just can’t take.

After making these changes, Lion was quite usable:

• Turn off “natural” scroll direction. The fact that before you can run Lion for the first time it forces you to scroll that way should be a tip off that people aren’t ready for this one yet. Maybe in a few years it will make sense to have it as the default, but now it’s just against every single computer out there (including most Macs).

• Autocorrect. This is not an iPhone. It *has* a perfectly fine keyboard. Thank you.

• Setting up Spaces. Oh, excuse me, Mission Control (sigh). This is the one I was really afraid of. I absolutely rely on having about 8 spaces (2 monitors in each) open at all times and swapping with a keystroke. That’s something that I’ve been using for many years (first in Linux, then hacked onto Windows XP, and finally in Leopard/Snow Leopard). I just absolutely have to have that.

  Update: There are still a couple annoying things with the new setup. You can’t see the number of the desktop you’re currently on, like you did with earlier versions of OS X. Daniel Collin pointed me to an interesting thread on StackOverflow where people are discussing how to get access to desktop numbers. Give it a few more weeks. Also, when you restart the computer, Lion is smart enough to restore all the apps to their last state… except that they’re all opened in the first desktop, not in the one they were when the system restarted. How difficult would that have been? Here’s hoping it’s fixed in a patchupdate soon.

  The good news is that it’s possible to set up Lion that way. The bad news is that by default, it’s quite far from that. Bring up Mission Control and create by hand as many desktops as you want. Then go to System Settings | Keyboard | Keyboard Shortcuts and turn on one for each desktop. And for the love of god, please go to Mission Control settings and turn off “Automatically rearrange spaces based on most recent use”. Clearly Apple doesn’t think people want to go to a space directly, and instead prefer to browse them one at the time.

The next few changes just made my experience better, but they’re not absolutely musts:

• This might sound dumb, but Finder didn’t have a good way to easily browse my files. Instead, it has this weird “All My Files” view that jumbles together all your junk there. I have no idea who thought that was a good idea. Maybe Apple is trying to wean users from browsing around the file system. In any case, you can drag your home directory to the sidebar in Finder and that makes it much easier to browse. While you’re at it, turn off the “Hide extensions in files” dumb setting.

• Having used a Mac for several years, I’m much more sensitive to ugly UI elements. That’s why it’s so weird that, by default, Lion has the sidebar font in all programs just very slightly larger than the regular system font. Fortunately I wasn’t the only one bothered by that and it’s an easy fix.

That’s about it. Lion is not very nice and usable (i.e. looks like Snow Leopard). You have that weird “full screen” button on the top right of all windows, but that’s much better ignored (which funny that, was the feature Apple really pushed in their PR for Lion). I have no idea why anyone would want to use that other than the usual word processor that blocks out everything else. If I want to have crippled multitasking, I’ll just use the iPad.

Good things about Lion

So far, if all I’m going to do is disable new features so it looks like Snow Leopard, is there there any reason to upgrade to Lion? I surprised myself, but the answer is yes (although they’re all very, very minor).

• Resize a window in any corner and cursor feedback (finally, only 20 years late to the party).
• Perfect contact and calendar syncing with Google. I can’t tell you how much I appreciate that. I use Google as the backend for all my mail, calendar, and contacts. With Snow Leopard you could sync them, but you had to do some digging and mess around with the sync program. With Lion it works out of the box. Totally awesome. That alone makes it worth the upgrade.
• Much improved Mail UI: Better layout, better shortcuts, better conversation view, and, my favorite by far, finally an Archive command for Gmail. Archive doesn’t exactly do the same as archiving in Gmail, but it’s close enough. Finally I can have two different keyboard shortcuts: One for trashing an email and another one for archiving it.
• The jury is still out on how Lion restores the application state when you launch an app. I’ve been pleasantly surprised a couple of times, and annoyed a couple other times that I had to close a bunch of useless tabs. Maybe there’s a shortcut like Shift + Cmd + Q that quits and app without saving state? That’d be handy (OK, that one is not it Oh wow, it’s Option + Cmd + Q! I love it when something makes sense like that and you can guess it. OK, so that makes this feature very cool.
• Slightly improved Safari (better download management). I really don’t see a reason to use Firefox or Chrome when you can use the preinstalled Safari. But then again, I can’t stand web UIs and I use native clients for lots of things.

Must-have apps

It’s actually quite refreshing to use the bare minimum number of extra apps beyond what comes with the OS. But there are some apps I just have to have in order to do anything. The main two are 1Password and Alfred. I can’t recommend them highly enough. They completely change how I work and make me ten times as productive (you’ve probably noticed by now that I’m not a mouse person).

After those two awesome super-apps, is the second tear of very useful ones: JustNotes, Things, and TextWrangler.

Probably Dropbox should be there somewhere, but for some reason, I still haven’t felt the need to use it heavily. I know some people put almost everything on Dropbox and can use across computers. Maybe it’s because I only have one that I haven’t felt the need for it too much. 1Password and Things both sync from the iPhone so it was really handy restoring all my data. I probably could have done the same thing with Dropbox.

Tomorrow I’ll post similar quick notes about Xcode 4. Those might not be quite as positive though.

I use the URL shortener in Flower Garden to send bouquets through SMS. Space is limited in a text message, so the message just contains some text explaining what is it and the URL pointing to the bouquet image. (Yes, I would much rather send them through MMS, but Apple isn’t exposing that yet to developers).

In this case, the full URL is http://flowers.snappytouch.com/sms.php?id=949618b4b3c6f3d76e32b45446e238a0 which gets thankfully shortened to http://goo.gl/IV5cq.

Sending bouquets through SMS has been in Flower Garden for several months, but it was using bit.ly before, which is probably the most popular URL shortener out there. I like their web interface and their super-easy to use API, but unfortunately it seems that I hit some mysterious API limit during the Mother’s Day Flower Garden promotion. That limit isn’t public anywhere, and as far as I can tell, I can’t even see it myself through the web interface or through an API query.

Finding out that I reached the API limit was quite shocking, because sending bouquets through SMS isn’t a particularly popular feature. Unfortunately I don’t have good analytics hooked up to that step, but I can’t imagine there were more than a few thousand per day.

They were very nice and contacted me instead of shutting down my account since it was just a spike. They also tried to sell me their “Enterprise” account, but $995/month is a tad too expensive for me. By about $990 probably, so I had to look for other options.

After a very quick research, goo.gl was the perfect alternative. Not only is it very fast (and backed up by the giant Google no less), but they have an API limit of 1,000,000 queries/day. If I ever blow that budget, I’ll be able to afford the $995/month without a problem

All URL shorteners are very easy to use. You need an API key, and figure out the exact format of the HTTP message you send and the response you get.

goo.gl

Goo.gl has a great Getting Started Guide that tells you everything you need to know. Get your private API key from here, and you’re ready to rock.

Drop this in your app and start shortening away. You’ll notice I used a synchronous HTTP request, which is usually a big no-no. Here I felt it was justified since the user is blocked waiting for the SMS to be prepared and sent. Besides, goo.gl is very, very fast, so it’s never even noticeable.

const NSString* GooGlAPIURL = @"https://www.googleapis.com/urlshortener/v1/url?key=YOUR_API_KEY_HERE";

NSString* ShortenURLWithGooGl(NSString* longURL)
{
	NSURL* apiUrl = [NSURL URLWithString:GooGlAPIURL];

	NSMutableURLRequest* req = [[NSMutableURLRequest alloc] initWithURL:apiUrl];
	[req setHTTPMethod:@"POST"];
	[req setTimeoutInterval:Timeout];
	[req setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];

	NSString* body = [[NSString alloc] initWithFormat:@"{\"longUrl\": \"%@\"}", longURL];
	[req setHTTPBody:[body dataUsingEncoding:NSUTF8StringEncoding]];
	[body release];

	NSError* error = [[NSError alloc] init];
	NSHTTPURLResponse* urlResponse = nil;
	NSData* data = [NSURLConnection sendSynchronousRequest:req returningResponse:&urlResponse error:&error];
	[error release];
	if (data == NULL || ([urlResponse statusCode] < 200 && [urlResponse statusCode] >= 300))
		return NULL;

	NSString* response = [[NSString alloc] initWithData:data encoding:NSASCIIStringEncoding];
	NSDictionary* responseDict = [response JSONValue];
	NSString* shortURL = [[responseDict objectForKey:@"id"] retain];
	[response release];
	return shortURL;
}

I’m using a JSON framework to parse the answer, but it’s so simple I probably wouldn’t even have to. I only used it because it’s already part of the project because of Facebook Connect.

bit.ly

Even though it’s not my favorite shortener, I’m adding it here for completeness (and because I had the code already written).

The one thing that bit.ly has going for it is that it’s even easier to use than goo.gl. No JSON involved, and you don’t even need to send a body with your request. As usual, get your API key by signing up with bit.ly.

const NSString* BITLYAPIURL = @"http://api.bit.ly/v3/shorten?login=%@&apiKey=%@&format=txt&";

NSString* ShortenURLWithBitLy(NSString* longURL)
{
	NSString* urlWithoutParams = [NSString stringWithFormat:BITLYAPIURL, LoginName, APIKey];
	CFStringRef encodedParamCF = CFURLCreateStringByAddingPercentEscapes(kCFAllocatorDefault,
															 (CFStringRef) longURL,
															 nil, (CFStringRef) @"&+", kCFStringEncodingUTF8);
	NSString* encodedURL = (NSString*)encodedParamCF;
	NSString* parameters = [NSString stringWithFormat:@"longUrl=%@", [encodedURL stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding]];
	NSString* finalURL = [urlWithoutParams stringByAppendingString:parameters];

	NSURL* url = [NSURL URLWithString:finalURL];

	NSMutableURLRequest* req = [[NSMutableURLRequest alloc] initWithURL:url];
	[req setTimeoutInterval:Timeout];

	NSHTTPURLResponse* urlResponse = nil;
	NSData* data = [NSURLConnection sendSynchronousRequest:req returningResponse:&urlResponse error:NULL];
	if (data == NULL || ([urlResponse statusCode] < 200 && [urlResponse statusCode] >= 300))
		return NULL;

	NSString* response = [[NSString alloc] initWithData:data encoding:NSASCIIStringEncoding];
	return response;
}

That’s it. You should be able to drop in either one of those snippets in your project and spend your time working on the things that really matter in your games.

But that was because of the kind of games I was making, which were very light on content, with mostly procedurally generated assets (the consequence of working by myself and being much better at programming than at Photoshop).

That game that Miguel and I are working on right now is a lot heavier on assets. It has locations, and levels, and the whole shebang. And that’s where XCode starts falling short.

Explicit Resources

Before, I was adding all my game assets to the Resources folder in XCode. That adds the file to the “Copy Bundle Resources” step. And as you expect, when you do a build, it checks the date of the existing file, and only copies it if the source file is newer than the destination file.

Personally, I really like this approach. I like to be explicit about what gets included in a game, and I don’t mind at all having to add files manually to the project.

Unfortunately, it has one major flaw: It collapses all assets at the root of the application directory, ignoring the directory structure where they came from. I have no idea what the rational is for this “feature”, but someone needs to be taken out to the public town plaza and whipped for that. Actually, make that a double-whipping session if the reason was “convenience”.

The reason this becomes a big deal now is that we have per-level resources. To keep things sane, we decided to use a directory hierarchy, so Levels/Level00 contains all the files necessary for that level. Same thing with Level01, etc. The problem comes that both those levels have similarly named files: Background.jpg Layout.bin, etc.

Any guesses what happens if you add to XCode two files with the same filename in different paths? Yup. One of them overrides the other. Not a single warning either. Let’s make that a triple-serving of whipping, please.

I briefly considered prefixing all the files with the level name (Level00_Background.jpg), but if later I decide to move Level00 to Level05 that’s a lot of files to rename, so I would end up having to write scripts, or create a separate file with the level ordering, or just generally waste my time doing something that should have been taken care of by the tool.

Folder References

Even though I had read they had their share of problems, I decided to look a folder references (at Miguel’s prompting mostly).

When you add some resources to XCode, you have an option to check “Create Folder References for any added folders”. That option automatically adds any files in those folders without having to explicitly add them to XCode. So you could add the Levels folder, and then any files you create there will be copied with the game.

I’m not a big fan of assets copied automatically, but as a side effect, that step preserves the directory hierarchy each of those files was in. So any files copied this way can be accessed from within the game by using their full directory structure.

I have to ask again: Why are directory structures preserved here but not with explicit resources added to the project? The mind boggles.

But hey, at least it works, right? Not exactly. There are a couple of gotchas.

The big one I had read in multiple places, is that XCode doesn’t detect any changes to files inside the referenced folder. So you can be making all sorts of changes, building the game, and not seeing anything different. The recommended solution was to add an extra step to the build process that would start by touching the reference folder, forcing a full copy of all assets.

I tested that, I’m glad to report that at least in XCode 3.2.4, that’s not the case. If you modify any file inside a referenced folder, the file will get copied over correctly during the build process without the need of extra steps.

The bad news is that all the files in the referenced folder will be copied. Why oh why??? They clearly know which file changed, why do they feel the need to copy all of the other files? No idea. This is not a big deal early on, but as you start to accumulate dozens and hundreds of megabytes of assets, build times start increasing quite a bit, especially on the device itself.

This is what the copy command looks like for referenced folders:

CpResource build/Debug-iphonesimulator/Test.app/Levels Levels
cd /Users/noel/Development/Test/trunk/Test
setenv PATH "/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin:/Developer/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin"
/Developer/Library/PrivateFrameworks/DevToolsCore.framework/Resources/pbxcp -exclude .DS_Store -exclude CVS -exclude .svn -Testve-src-symlinks /Users/noel/Development/Test/trunk/Test/Levels /Users/noel/Development/Test/trunk/Test/build/Debug-iphonesimulator/Test.app

It’s nice touch that it automatically excludes .svn directories though. I was wondering why they use CpResource instead of plain, old cp, but I guess that’s to be able to -exclude specific files. Fair enough.

However, what CpResource apparently doesn’t do is to process any of the resources in ways that were processed before by XCode. For example, a png file would have been processed by premultiplying it and byte swapping it so loading it in the iPhone would be slightly more efficient. CpResource just does a regular copy and leaves it alone. So if you were relying on that behavior, you need to do it explicitly yourself in your asset baking step.

What I Really Want

For now, I’m using folder references for the levels, and explicit references for everything else. That way I keep the data size to a minimum but I get to have the directory hierarchy. Not ideal, but at least it works.

This is what I would really like thought:

1. Easiest: Explicit assets with paths. I really want to just add resources to XCode and have it preserve the directory structure. It’s not that hard. If XCode were open source, I would have made that change a long time ago. Can we at least have this as an option?
2. Second easiest: Folder references that only copy the changed resources. That would also be OK in my book, and I can’t believe it would be much harder to implement either.
3. Best: A remote file system hosted on the Mac during debug build. All file references go out to the host machine and get loaded on the fly. This would allow for fastest build times and loading times would not be that different from a fragmented drive on an old device probably. I know some of you already have something like this. Has anybody made one open source (preferably minimalistic and standalone)? I’d love to check it out.

Of course, all of this has probably changed already with XCode 4, but I’m deathly afraid of installing it while working on a production game. Has anybody tried it yet? Have they fixed anything, or is it even more broken?

To wrap things up, and since Miguel is spilling the beans on Twitter, I’ll share a few assets from our current game. Now back to the game because we’re submitting it to the Independent Games Festival on Monday. Next Thursday I’ll talk about the IGF. Wish us luck!

This post is part of iDevBlogADay, a group of indie iPhone development blogs featuring two posts per day. You can keep up with iDevBlogADay through the web site, RSS feed, or Twitter.

Yesterday things changed a bit. As a result of last week’s “growing” post, Manuel Freire is going to join me to help with Flower Garden development. That makes two of us banging on the same codebase, and from two different time zones, so we don’t get the benefit of being in the same closet as it was the case with Power of Two Games. Since I was in my get-things-done mindset, I figured I would just set up a new svn repository for the project, move over the Flower Garden data, give us both access to it, and move on.

But no, it couldn’t be that easy. Can you guess what the first words out of his mouth were when I asked him about version control? “Oh, I love Git!”

That was the last straw. I had to do a mini-research session on version control systems, so I spent a couple of hours looking into it. If we were going to move over to something, now would be the time to do it.

Git and Mercurial

Git and Mercurial both look great. I was debating which one to go with until I realized that they’re both two flavors of the same thing, so it comes down more to personal preferences and tastes. This is best description I found on the differences between Git and Mercurial. When it comes to computers, I’m totally a MacGyver guy (actually, that might be true when it comes to anything now that I think about it), so that made my decision easier.

The big feature everybody keeps talking about for distributed version systems is effortless branching. That’s great, but I really have no intention of branching much. I haven’t created a single branch in the last four years, and I don’t expect to start doing that now. Next.

The other big feature is working disconnected from the network. That’s something I could use, but considering I’m only offline a handful of times a year, it really isn’t enough of an incentive to switch to a whole new system.

Git sounds like a great tool for large, distributed, open-source projects with hundreds of contributors, but frankly, I couldn’t find anything else that was worth mentioning for a small project and a handful of people. I feel like someone is trying to sell me a Porsche when my beat-up Hyundai is still perfectly functional for driving to the grocery store once a week. Am I missing something obvious?

Hosting

Hosting the actual repository was part of the consideration. This is for a private project, so all open-source sites are out. Ideally I wanted to host it just like I do with Subversion in Dreamhost, but the instruction page on how to install Git and Mercurial are enough to put most people off. Clearly, there’s a steep learning curve there.

So I asked on Twitter for recommendations. I’ve learned that Git and Mercurial users are definitely very vocal and are always willing to help someone join their ranks. Within minutes I had all the suggestions listed below:

• Github (Git)
• Repositoryhosting.com (Git, Hg, SVN)
• Bitbucket (Hg)
• Klin (Hg)
• CodaSet (Git)
• Assembla (Git, SVN)
• Unfuddle (Git, SVN)
• Beanstalkapp.com (Git, SVN)
• SourceRepo (Git, Hg, SVN)
• Plain Dropbox (Git I guess)
• Self-hosting (gitosis for managing Git)

Prices varied a lot. From the $25/month/user of Kiln, to the $6/month of RepositoryHosting (gets you unlimited users and 2GB of storage). The Snappy Touch repository is already over 2GB, so it would end up being a bit more expensive than that, but not too bad.

Of those, Github was definitely the most recommended. I was starting to feel Git might not be the one for me, so I looked a bit more into RepositoryHosting because they had Subversion support. It turns out they also provide Trac, which is a great tool, although I already have that set up myself.

Wish List

Binary files

The one thing I really want in a version control system is good large binary file handling. I check in everything under version control, source code, assets, raw assets, and even built executables for each version. I want to be able to throw multiple GB psd files in the repository and have it work correctly (meaning fast, and using the least amount of space possible).

Perforce did an OK job with that. Git and Mercurial apparently are both horrible at it. So is Subversion, but at least it’s a tool I already know and I don’t have to spend time learning the ins and outs of how to optimize the Git database or how to make backups, or cull unused trees.

GUI

I love my command-line tools. I live in Terminal for a good part of the day, and having a real shell is one of the things that makes my life so much more pleasant under Mac OS than under Windows. But there are some things for which a GUI tool is a really useful addition, and version control is one of them.

On the Mac, I’ve been using Versions as a client for Subversion and it does everything I want. It’s fast, handles multiple repositories, lets me browse history, diff changes, etc. From my brief search and other people’s comments, there’s nothing quite like that for Git or Mercurial yet. That’s a pretty big, gaping hole.

Low-level access

Looking at all those hosting providers, I realized how much I want to have low-level access to the database. I want to be able to back it up myself, and run svnadmin when I want to. A lot of those hosting sites looked really pretty, but you were very limited in what you could do.

Coming To A Decision

If this were a thriller, you’d be disappointed. I’m afraid there are no plot twists and you can already guess the conclusion.

In the end, since neither Git nor Hg are built to address my biggest need (large binary files), I’ll stick with svn. It might be old, it might not be cool, but it serves my needs, I already know how to use it, I have the tools, I can admin it and fix a problem. I can concentrate on what matters instead: Writing games.

I decided to continue hosting it myself on Dreamhost. I can easily have one repository for every major project. However, by default, Dreamhost creates svn repositories using htaccess security and HTTP protocol. That’s OK, except that none of the actual data is encrypted as it would be if I were using ssh. I could use HTTPS, but then I would have to set up a certificate and pay for a fixed IP address, so instead I found an alternate way to have a secure connection.

All Subversion repositories live in a user account (svnuser). I create a new user group for every repository, and change all the files in the repository to belong to that group. Make sure you also set the SGID bit so any files created in that directory still belong to the group. Then I can create a new shell user for every collaborator, and add him to the groups of the repositories I’d like him to have access to. At that point, he’ll be able to access the repository as svh+ssh://username@hostname.com/home/svnuser/repository. All safe and secure.

Bonus SVN-Fu

Here’s something that I learned yesterday while I was moving repositories around. It’s probably common knowledge for seasoned SVN admins, but it was new to me.

I had a repository that included a bunch of my projects. What I wanted was to create a new repository that still had all the history, but only for the FlowerGarden part of the tree. I knew about svnadmin dump for transferring whole repositories, but I didn’t know there was a very simple way to only transfer part of it.

First you need to dump it as usual:

svnadmin dump repository > repos-dumpfile

Now, it turns out you can process the dump file before adding it back to another repository. So we can do:

svndumpfilter include FlowerGarden --drop-empty-revs --renumber-revs < repos-dumpfile > fg-dumpfile

Finally, you can add the resulting dump file into a fresh repository and have all the history for that project and only that project:

svnadmin create flowergarden
svnadmin load --ignore-uuid flowergarden < fg-dumpfile

Amazingly, for a repository that was over 2GB, that only took a few minutes. Go Subversion!

[1] Or reading. Or going for a walk. Heck, even sleeping would be a better use of my time than futzing around with a new tool.[2]

[2] And yes, I realize I sound like a grumpy old man. Getting there apparently. Now get off my lawn.

This post is part of iDevBlogADay, a group of indie iPhone development blogs featuring two posts per day. You can keep up with iDevBlogADay through the web site, RSS feed, or Twitter.

Development tools and hardware have changed quite a bit from the times I was writing in straight Z80 assembly and saving the programs to a tape. But I’ve also changed and learned a lot during all those years developing games, and even though I’m writing games by myself again, I’m doing things very differently from how I did them back at the start.

One thing that I’ve always done is to question everything. Why should I do things a certain way? Why is that the “accepted” way of doing something? And not surprisingly, at each step of the way, I’ve changed my development style to match my situation (often in ways that went against the “common wisdom”).

When it comes to solo development, I rely heavily on the concept of goals and iterations at multiple levels:

• Immediate (< 1 minute): Prioritizing the ideas going through my mind. Writing tests. Writing code.
• Short Range (< few hours): Tasks that move the project forward in some way.
• Mid Range (< 2 weeks): “Stories” that define a self-contained, significant part of the project.
• Long Range (full project, several months): Ship date, beta testing, etc.

It turns out, I use a different set of tools to help me manage the items at each level of the development cycle.

Immediate

These are the actions I take and complete in less than a minute or two. Most of them are writing tests (with UnitTest++, of course), writing code to make those tests pass, refactor code, and check it into Subversion. I have my Subversion repository hosted on Dreamhost and I access it through SSH so it’s secure, accessible from anywhere with an internet connection (or 3G signal and HandyLight), offsite, and easy to backup. And because Subversion works great offline (unlike Perforce), it’s not a problem to work without connection to the repository for a while.

I also need to manage my minute-to-minute thoughts, write down ideas and reprioritize them when the time comes. When I’m “in the zone”, I get way more ideas than I can execute with my fingers: This function needs to be moved to a different file, I really should be compacting that data over there, who was stupid enough to name this file this way?, that variable shouldn’t be cached, etc. If I don’t write things down, I will either forget them, or I’ll stress for hours until I finally get around to doing them. I could also do things as I think about them, but then I would be chasing a rabbit down a neverending hole and wouldn’t get any work done (I’m sure anybody who’s gotten lost browsing web pages can identify with that).

I used to do this the old-fashioned way, simply with paper and pencil (like Bob described in his blog post). However, I found that physical paper and pencil was just too limiting: I can type a lot faster than I can write, switching to writing requires moving away from the keyboard, and, most importantly, I need to bring the notes with me everywhere and it’s very difficult to rearrange, sort, or coalesce them in any way.

So instead, my tool of choice these days is JustNotes. It’s perfect for jotting down thoughts in a matter of seconds without even interrupting my train of thought. I have JustNotes bound to a global key, so in the middle of typing a line of code, I can press that key, enter whatever I’m thinking about, press the key again, and finish the line of code. All in 4-5 seconds. Don’t laugh: The fact that I can do that in just a few seconds without moving my hands away from the keyboard means I can use it any time without much penalty. It’s amazing how many things I jot down that I wouldn’t do otherwise.

Short Range

To manage tasks up to a few hours in length, I use Trac. Trac is a fantastic issue-tracking tool: It’s free, it’s fast, it’s simple, and it’s configurable. In the past I’ve used anything from spreadsheets, to Bugzilla, to publisher-owned bug databases, and nothing comes close to Trac for my needs. It also scales very well to teams of more than one person (although it might not be good enough for hundreds of people).

Just like Subversion, I have Trac hosted externally, through my web hosting company. Sometimes it’s frustrating if I need to access it and the server is down, but again, the convenience of having it off site makes it well-worth it.

Any task that requires more than 10-15 minutes goes in Trac, and then I can easily prioritize tasks depending on their importance. Usually, if an item has been on my instant queue in JustNotes for about a day and I haven’t gotten around to it, it either gets deleted or it gets moved into a full item in Trac. The only way progress happens is by ticking off Trac tasks. In the end, my projects live and die by Trac.

Medium Range

User stories (to borrow terminology from Scrum/XP) are visible, relatively self-contained features of the final project. They’re made up of several tasks, and usually take a few days to a few weeks to implement. A group of user stories make up a full iteration (sprint) of the game, which is usually between one and two weeks long. Sometimes user stories are complex enough (add a replay feature visible on the web site) that the full iteration is just the one user story.

I keep track of these stories in Trac as well. Trac is both an issue-tracking system and a wiki, so the wiki part is perfect to keep these user stories. In addition to that, I label tasks as belonging o a particular iteration. That allows me to separate what needs to be done for this iteration, from other tasks that I added for the future. At the start of each iteration, I decide on the user stories and either generate new tasks or label existing tasks as due for this iteration.

The wiki in Trac is extremely valuable for all sorts of other things: game design ideas, general brainstorming, gathering reference material, etc.

Trac ends up being the perfect mid-range vision of my project.

Long Range

User stories and tasks in Trac aren’t enough to cover a project that is potentially 3-4 months long. I need something that helps me with the longer view, otherwise I find that things creep up on me without realizing it because I’m so focused on the short and mid-range items.

The best tool I’ve found so far is very low-tech: A printable month-per-page calendar covering the full length of the project. Right now I’m shooting for a November release of my current project, so I printed August, September, October, and November and pinned them to the corkboard on my office. It’s amazing the sense of urgency that seeing your ship date gives you. You realize that you only have a handful of weeks before shipping and makes it much easier to prioritize tasks (and chop off features or save them for an update).

I realize this long-term, calendar view isn’t very useful if you don’t have a set release date and you want to continue chipping at your game until it’s ready. But even if your release date is flexible, having this long-term view can help you keep budgets in perspective and manage them accordingly.

Finally, for an extra bit of motivation (or maybe this falls in the category of excessive pressure), I just started using a countdown widget for the Mac OS Dashboard. Just in case the calendar view wasn’t enough, here’s a countdown (down to the second) of the time left until release.

Speaking of which, I think it’s time I get back to work. Only 102 days left!

One of the things you potentially give up by keeping the game runtime to a minimum is an editor built in the game itself. But that’s one of those things that sounds a lot better than it really is. From a technical point of view, having an editor in the game usually complicates the code a huge amount. All of a sudden you need to deal with objects being created and destroyed randomly (instead of through clearly defined events in the game), and you have to deal with all sorts of crazy inputs and configurations.

The worse part though, is having to implement some sort of GUI editing system in every platform. Creating the GUI to run on top of the game is not easy, requiring that you create custom GUI code or try to use some of the OpenGL/DirectX libraries available. And even then, a complex in-game GUI might not be a big deal on a PC, but wait and try to use that interface on a PS3 or iPhone. After all, making games is already complicated and time-consuming enough to waste more time reinventing the widget wheel.

Remote game editing manages to keep a minimal runtime and allow you to quickly create native GUIs that run on a PC. It’s the best of both worlds, and although it’s not quite a perfect solution, it’s the best approach I know.

Debug Server

Miguel Ángel Friginal already covered the basics of the debug server, so I’m not going to get in details there.

The idea is that you run a very simple socket server on the game, listening in a particular port. This server implements the basic telnet protocol, which pretty much means that it’s a line-based, plain-text communication.

The main difference between my debug server and Miguel’s (other than mine is written in cross-platform C/C++ instead of ObjC), is that I’m not using Lua to execute commands. Using Lua for that purpose is a pretty great idea, but goes against the philosophy of keeping the runtime as lean and mean as possible.

Instead, I register variables with the server by hand. For each variable, I specify its memory address, it’s type, any restrictions (such as minimum and maximum values), and a “pretty name” to display in the client. Sounds like a lot of work, but it’s just one line with the help of a template:

registry.Add(Tweak(&plantOptions.renderGloss, "render/gloss", "Render gloss"));
registry.Add(Tweak(&BouquetParams::FovY, "bouquet/fov", "FOV", Pi/32, Pi/3))

And yes, if I were to implement this today, I would probably get rid of the templates and make it all explicit instead (ah, the follies of youth

TweakUtils::AddBool(registry, &plantOptions.renderGloss, "render/gloss", "Render gloss");
TweakUtils::AddFloat(registry, &BouquetParams::FovY, "bouquet/fov", "FOV", Pi/32, Pi/3);

The debug server itself responds to three simple commands:

• list. Lists all the variables registered in the server.
• set varname value. Sets a value.
• print varname. Gets the value for that variable.

For example, whenever the server receives a set command, it parses the value, verifies that it’s within the acceptable range, and applies it to the variable at the given memory location.

Telnet Clients

Because we used the standard telnet protocol, we can start playing with it right away. Launch the game, telnet into the right port, and you can start typing away.

However, most telnet clients leave much to be desired for this. They all rely on the history and cursor manipulation being handled by the shell they assume you’re connected to. Here we aren’t connected to much of anything, but I’d like to be able to push up arrow and get my last command, and be able to move to the beginning of the line or the previous word like I would do in any text editor. The easiest solution I found for that was to use a telnet client prepared for that kind of thing: A MUD client! Just about any will do, but one that works well for me is Atlantis.

So far, we’ve implemented the equivalent of a FPS console, but working remotely. And because the code is fully portable, our game can be in just about any platform and we can always access it from our PC. Not just that, but we can even open multiple simultaneous connections to various development devices if you need to run them all at once.

Custom Clients

Game parameter tweaking is something that is OK through a text-based console, but really comes into its own when you add a GUI. That’s exactly what we did at Power of Two Games. We created a generic GUI tool (based on WinForms since we were on Windows at the time), that would connect to the server, ask for a list of variables, and generate a GUI on the fly to represent those variables. Since we knew type and name of each variable, it was really easy to construct the GUI elements on the fly: A slider with a text field for floats and ints, a checkbox for bools, four text fields for vectors, and even a color picker for variables of the type color.

It worked beautifully, and adjusting different values by moving sliders around was fantastic. We quickly ran into two problems through.

The first one is that we added so many different tweaks to the game, that it quickly became unmanageable to find each one we wanted to tweak. So, in the spirit of keeping things as simple as possible (and pushing the complexity onto the client), we decided that the / symbol in a name would separate group name and variable name. That way we could group all related variables together and make everything usable again.

The second problem was realizing that some variables were changing on the runtime without us knowing it on the client. That created weird situations when moving sliders around. We decided that any time a registed variable changes on the server, it should notify any connected clients. That worked fine, but, as you can imagine, it became prohibitively expensive very quickly. To get around that, we added a fourth command: monitor varname. This way clients need to explicitly register themselves to receive notifications whenever a variable changes, and the GUI client only did it for the variables currently displayed on the screen.

During this process, it was extremely useful to be able to display a log console to see what kind of traffic there was going back and forth. It helped me track down a few instances of bugs where changing a variable in the client would update it in the server, sending an update back to the client, which would send it again back to the server, getting stuck in an infinite loop.

You don’t need to stop at a totally generic tool like this either. You could create a more custom tool, like a level editor, that still communicates with the runtime through this channel.

Flower Garden Example

For Flower Garden, I knew I was going to need a lot of knobs to tweak all those plant DNA parameters, so I initially looked into more traditional GUI libraries that worked on OpenGL. The sad truth is that they all fell way short, even for development purposes. So I decided to grab what I had at hand: My trusty tweaking system from Power of Two Games.

I’m glad I did. It saved a lot of time and scaled pretty well to deal with the hundreds of parameters in an individual flower, as well as the miscellaneous tweaks for the game itself (rendering settings, infinite fertilizer, fast-forwarding time, etc).

Unfortunately, there was one very annoying thing: The tweaker GUI was written in .Net. Sure, it would take me a couple of days to re-write it in Cocoa (faster if I actually knew any Cocoa), but as an indie, I never feel I can take two days to do something tangential like that. So instead, I just launched it from VMWare Fusion running Windows XP and… it worked. Amazingly enough, I’m able to connect from the tweaker running in VMWare Fusion to the iPhone running in the simulator. Kind of mind boggling when you stop and think about it. It also connects directly to the iPhone hardware without a problem.

VMWare Fusion uses up a lot of memory, so I briefly looked into running the tweaker client in Mono. Unfortunately Mono for the Mac didn’t seem mature enough to handle it, and not only was the rendering of the GUI not refreshing correctly, but events were triggered in a slightly different order than in Windows, causing even more chaos with the variable updates.

Here’s a time-lapse video of the creation of a Flower Garden seed from the tweaker:

Drawbacks

As I mentioned earlier, I love this system and it’s better than anything else I’ve tried, but it’s not without its share of problems.

Tweaking data is great, but once you find that set of values that balances the level to perfection… then what? You write those numbers down and enter them in code or in the level data file? That gets old fast. Ideally you want a way to automatically write those values back. That’s easy if the tool itself is the editor, but if it’s just a generic tweaker, it’s a bit more difficult.

One thing that helped was adding a Save/Load feature to the tweaker GUI. It would simply write out a large text-based file with all the variables and their current values. Whenever you load one of those, it would attempt to apply those same values to the current registered variables. In the end, I ended up making the Flower Garden offline seed file format match with what the tweaker saved out, so that process went pretty smoothly.

Another problem is if you want lots of real-time (or close to real time) updates from the server. For example, you might want to monitor a bunch of data points and plot them on the client (fps, memory usage, number of collisions per frame, etc). Since those values change every frame, it can quickly overwhelm the simple text channel. For those cases, I created side binary socket channels that can simply send real-time data without any overhead.

Finally, the last drawback is that this tweaking system makes editing variables very easy, but calling functions is not quite as simple. For the most part, I’ve learned to live without function calls, but sometimes you really want to do it. You can extend the server to register function pointers and map those to buttons in the client GUI, but that will only work for functions without any parameters. What if you wanted to call any arbitrary function? At that point you might be better off integrating Lua in your server.

Future Directions

This is a topic I’ve been interested in for a long time, but the current implementation of the system I’m using was written 3-4 years ago. As you all know by now, my coding style and programming philosophy changes quite a bit over time. If I were to implement a system like this today, I would do it quite differently.

For all I said about keeping the server lean and minimal, it could be even more minimal. Right now the server is receiving text commands, parsing them, validating them, and interpreting them. Instead, I would push all that work on the client, and all the server would receive would be a memory address, and some data to blast at that location. All of that information would be sent in binary (not text) format over a socket channel, so it would be much more efficient too. The only drawback is that we would lose the ability to connect with a simple telnet client, but it would probably be worth it in the long run.

This post is part of iDevBlogADay, a group of indie iPhone development blogs featuring two posts per day. You can keep up with iDevBlogADay through the web site, RSS feed, or Twitter.

iSimulate consists of two different parts: The iPhone app, which you need to purchase through the App Store, and a static library that you link with your program. Once your app is using the library, launching the iSimulate iPhone app lets you connect your device to your app and forward accelerometer, touch, and GPS events. In other words, most of the things the simulator doesn’t do natively.

iSimulate definitely passed my picky installation requirements with flying colors. I was afraid the installation would hook up with existing framework libraries, add new configurations, and in general mess up my system. Fortunately, hooking up the iSimulate library couldn’t be easier: Copy the provided .a library anywhere you want, link with it in your app (don’t forget the -ObjC linker flag), make sure you’re using the Core framework, and you’re good to go. That simple. That’s how I want external libraries to work.

You don’t even need to initialize anything. Just launch your app in the simulator, launch iSimulate on the iPhone and it detects your program running and lets you connect to it. The iPhone app itself even looks very pretty and polished. The overall package is definitely very professional-looking.

From there, any touch on the device is forwarded to the simulator. So you can finally do off-center multi touches, or two touches that happen at different times, or even 5-touch events (the max the iPhone support). The other big one is being able to tilt or shake your device and see the game react correctly on the simulator, which is would have been very handy when writing the accelerometer code to move flowers around in Flower Garden, but would almost be necessary when you’re making heavy use of accelerometer input for games like Sky Burger. Finally, it also claims to give you location information, but I didn’t have an app I could easily test it with.

Flowers affected by gravity on the simulator

Another situation where iSimulate is a life savior is when trying to capture a video of your game. If you rely on multi touch or accelerometer features, then your only options are to either hack in something yourself, or dig out a physical video camera and record it the old-fashioned way (which is surprisingly time consuming and creates less-than-ideal results). With iSimulate you can still use a desktop video capture software on the simulator and create a top-quality in a fraction of the time.

Not everything is perfect though. I thought the forwarding of touch events would be incredibly useful, until I realize I have no idea what I’m about to tap because the program is running on the simulator, not on the device. So I’m just making a guess about where my finger is going to land. It’s not even like a mouse cursor that you see where it’s going and then you click. You’re literally tapping blind hoping you touch what you wanted. That works with a static UI, but if you have moving elements you need to touch, you’re totally out of luck. I wonder if they could extend iSimulate to capture the image from the simulator and forward it to the iPhone. Even if they only refreshed the image a few times per second, that would be enough to make touch input a lot more friendly.

There’s another big problem with touch input though. From their docs page, iSimulate works great with OpenGL apps, but doesn’t work with the following UIKit controls: Keyboard, UIScrollView (including MKMapView), UIPickerView and UITableView. Wow! Apart from the keyboard one (no big deal, we have a real keyboard attached) the other ones are pretty major. I’m sure they have a good reason why, but I can’t imagine what that might be from my knowledge of how the touch even system works (anybody from Vimov care to comment on that?). Heck, I’d even be willing to add some extra code to my app to hook directly into iSimulate input events if it means it would work with those classes. Apart from that, I found one case in Flower Garden where iSimulate failed: Touching the bouquet tag to bring up the editing mode. It’s weird, it would let me touch-drag it to move it, but it wouldn’t go into editing mode.

The accelerometer input worked flawlessly, but here iSimulate has some competition from Accelerometer Simulator, an open-source app plus library to forward accelerometer data from the device to the app. It’s not nearly as slick as iSimulate, but it’s free, so that migth be a more attractive solution for some people.

Overall, is iSimulate worth the $32 price tag (it’s $15.99 at the moment)? I would say yes if you’re developing OpenGL iPhone games with heavy use of multi touch and accelerometer input and you’re planning on making money from them. In that case, you’re better off spending an hour of your time polishing the game (or taking a break from programming) and buying iSimulate. However, if you’re using UIKit objects or you just need accelerometer input, then you might be better off just using Accelerometer Simulator. If Vimov can fix the UIKit input and display the output of the simulator on the device, iSimulate would become an indispensable tool in every programmer’s toolbox.

Clearly, those are all things that need to be done, but if they are so repetitive and mindless, couldn’t we put some of the computers around us to good use and have them do the job for us?

A build server will do all that and more, much faster and more reliably than we could, and it will free us to work on the thing that made us fall in love with this industry in the first place: the game.

Getting Off The Ground

Before we can start thinking about setting up a build server, we need to be able to build the game with a single command from the command line. No clicking around, no GUI apps, no multiple steps, no magical incantations that only work during a full moon. Just type a command and the build for the game and all its libraries starts.

This is not just a necessary step to set up a build server; it’s a very good engineering practice. So if you’re not there, spend some time on it right away and you’ll be glad you did when candidate submission time comes around.

Building the game with a single command should be fairly easy, but the specifics will depend on your environment and build system. If you’re using Visual Studio, you can put the game and all the libraries in a single solution with the correct dependencies. Then you can invoke the devenv.com command line program specifying the solution and configuration you want to build: devenv.com mygame.sln /build Debug. You can wrap that up in a single batch file buildgame.bat for extra convenience and you’re done.

If you’re using another build system, such as make or jam, you can probably already build it with a single command. If you’re using a bunch of mode-made scripts, at least wrap them all up in a single script file so they can be run with a single command.

Just building the game with a single command isn’t enough. We must have a way to automatically detect whether the build succeeded or failed. Fortunately, most build systems (including devenv. com and make) return an error code when the build fails. If you’re rolling your own build script file, make sure to capture build failure and return an error code as well.

Setting Up The Build Server

A build server should be a dedicated machine with access to version control. Whenever a new build is needed, the server syncs to the latest version in version control, starts a build, and notifies the team in case of any errors.

That’s a fine start, but we could make things much better. For example, we could include the error message in the notification email so programmers can see right there what the problem was instead of being forced to sync and build the game themselves. We could also trigger a build in different circumstances (for example, code checked-in, forced by a person, or some other event) instead of only at fixed intervals. We might want to distribute the build across multiple machines, or keep logs and make them available on a web page, or format emails better, or use more direct notification methods …

Put away those Python reference manuals because fortunately, someone has already done all the work for us: CruiseControl (and CruiseControl.Net). It’s a free, open source build server program with all the bells and whistles that you could possibly want. And did I mention it’s free?

There are three main parts to it:

1. The build server. It runs as an application or a Windows service. It’s configured through a very simple XML file that tells it when to sync, where to sync, what to build, and how to report it.
2. The web front end. CruiseControl features a pretty, web-based dashboard showing all the builds, their status, past logs, and other pertinent data.
3. The system tray notificator. This is a little app that runs in the system tray and shows the status of all the builds and notifies you of any changes right away with a message and by playing some sounds. This is my favorite way to keep up to date with the build status. You’ll be up and running in about 10 minutes. The most complicated part is probably installing a web server (if you don’t already have one) and getting the web dashboard running. You’ll spend a few more hours tinkering with it to get it “just right,” and then you’re done. The only time I have to mess with it is to upgrade to a new version every so often. Other than that, it’s virtually maintenance free.

At this point you’ll have a fully featured build server in place. It verifies that the game can be built from the latest checked-in version of the code. It notifies developers of failed and successful builds right away. It increases version numbers, keeps a build history and statistics, archives executables, and emails logs.

CruiseControl and CruiseControl.Net are the two build servers I have most experience with. There are other build servers out there, with slightly different features, integrations with different environments, and so forth. Some of them are commercial and come with full support in case you’re more comfortable with that model.

It’s important to stress that a build server is not intended to be the only machine that builds the game. Every programmer (and maybe every member of the team) should be able to build the game in his or her own machine from scratch. The build server is there to verify that all the checked-in changes build correctly on a clean machine, and to make sure that all platforms and configurations are building successfully. Any official builds should be created exclusively from the build server, though. Especially any builds distributed externally to publishers or manufacturers. This ensures that the build is clean, was created in a repeatable manner, and is free of any idiosyncrasies from a particular machine.

How Often?

Once the build server is in place and is producing successful builds reliably, the question arises of how often to make builds of the game.

It used to be considered good practice to do a weekly build. The team would start ramping things up on Thursday to try and get a build out the door by the end of the day on Friday. Anybody who has done that knows how stressful it can be and how it can easily become a bottleneck.

Why wait a week if you can do one every night? More teams started switching to the daily build, which is much less stressful because there are fewer changes in each new build. It also gives the team a chance to fix anything that was found broken in the previous day’s build. Soon, teams took it beyond the daily build and started making two builds a day, or even one every hour.

The build server has been very appropriately described as the heartbeat of the project. A “green build” is one heartbeat and one small step forward. A “red build” is done when something is wrong and needs to get fixed as soon as possible. If you have a red build several days in a row, the project is in serious trouble. The more often you make a successful build, the better. You’ll find fewer surprises and stay more on course that way.

My favorite approach is continuous integration. With continuous integration, the build server starts a new build as soon as there’s a new check-in. If multiple check-ins come in while the build is in progress, another build starts right after it’s done, with all the new changes queued during that time. When following this practice, programmers sync to the latest version often, make small changes, and check-in code frequently, rather than batching many changes. Very conveniently, Cruise Control has a setting to start builds whenever anything changes.

The main benefit of continuous integration is that you are notified as soon as a check-in breaks the build—not a day later, or even an hour later, but minutes later. It tells you, “The last build was good. This one is not.” You can look through the last couple of check-ins that happened during that short time period and quickly narrow down the problem and fix it. Imagine trying to narrow down an elusive crash bug from all the check- ins for a full day or two!

Another benefit is that all programmers are working on a version very close to the latest one. This means that there are fewer source code conflicts when checking-in code, and fewer surprises lurking in the code. The flip side of that is that working on the latest version is living in the proverbial bleeding edge. It’s not unusual for someone to check-in code that has some accidental bad side effects. As long as those bad check-ins are limited, and that whenever they happen they are fixed right away, I have found the benefits to outweigh some instability in the main branch. Some of the ways to minimize disruptions when working with continuous integration are:

• Make sure that any code compiles before checking it in (that should go without saying!)
• Execute a fast set of unit tests to verify that basic functionality is working correctly, and
• Have the build server notify everybody as soon as there’s a broken build so it can be fixed and so that nobody else syncs or checks-in any code while the build is broken.

Need For Speed

Ideally, I’d like to check in some code and see whether the build server found any problems right away. In the real world, things can be much slower. After all, the build server needs to sync to the latest code, kick off builds for multiple platforms and multiple configurations, and perform some other time-consuming steps. Even so, there is work we can do to get feedback as soon as possible. Perform incremental builds during the day, so only the affected sections of the code need to be built. It’s still a good idea to do a full build at least every night to make sure that everything can be built from scratch.

Set up each platform and configuration as separate builds. That way you get feedback as soon as one of them completes. The only downside is if an error makes it through that causes all the builds to fail, get ready for lots and lots of broken build sounds playing all over the company. Speed up build times through good physical dependencies, modularity, precompiled headers, and good use of forward declarations.

Split up different builds and configurations in different machines. The easiest way is to set up one machine per platform and configuration (or maybe do a couple of configurations per machine). Cruise Control lets you easily integrate several build servers into the same web dashboard and system tray application, so this is a very easy solution.

Don’t Skimp on Hardware

Get the beefiest computers you can afford. Throw fast CPUs, disk access, and gigabit ethernet. Get multiprocessor cores and make sure your build system takes advantage of them. Does it sound like a lot of money? Not when you take into account how few servers you’ll have and how much time you’ll save all the members of the team.

I have tried several distributed build systems, and even though they can sometimes be beneficial for some codebases, I’m still not a huge fan. I find that you can often achieve the same (or better) results by using multiple processors and good build architectures, and you avoid the complexity and overhead of a distributed build system.

One “gotcha” we ran into when we scaled our build farm beyond about 15 build servers was that each of them was hitting our version control repository every few seconds to see if anything had changed. That wasn’t a trivial operation, and so many servers doing it so frequently definitely slowed things down to a crawl.

To remedy that, instead of having the build servers poll the overtaxed source control server, we had the source control server push out a notification. Whenever there was a check-in, the source control server changed a timestamp in a file located on an internal web server. We changed the build servers to constantly monitor the internal web server for changes in that file, and whenever it changed it triggered a build, which completely eliminated the overhead on the version control server.

Beyond The Build

So far, we’ve only been talking about building the game. But the build server is a great tool that we can put to good use for many other purposes. Why restrict ourselves to just the game? All the in-house tools would also benefit from getting the same treatment. We can even take it a step further and deploy the freshly-built copies of all the tools on a network drive or web page so they’re available to the whole team.

The build server can also double up as a symbol server. That makes it much more convenient for programmers to debug an earlier version of the game and libraries and have all the debugging information available without having to rebuild everything locally.

There’s no reason to limit the build server to just building source code. One of the most useful things you can do with it is use it to build game assets as well. Building assets is usually a slow process. Having a fast asset build system that can correctly perform incremental builds is crucial to keep asset build times down.

Build servers are general enough to perform just about any task. Running both unit tests (small tests on each class or function) and functional tests (tests that exercise a larger module or even the whole game) are perfect uses for the build server. Functional tests can be pretty slow, so make sure that they’re treated as a separate build and not as the last step in building the game. Nobody wants to wait for hours for all the functional tests to complete before they can see the successful build status after a check-in. The sky is the limit with what the build server can do. We use it to run static analysis of our source code, checking for spots in the code that can lead to subtle and dangerous bugs (uninitialized variables, implicit type conversions, and the like).

Another great use is to run through the different levels of the game, recording frame rate at different points of each level, logging the results, and failing the build if it ever drops below a certain threshold. Having the performance history for specific levels can be really useful to narrow down why a particular section is chugging at 20fps but was running at a solid 60 a couple of weeks ago. For bonus points, integrate all the collected data into easy-to-visualize graphs available through the web front end.

The build server is definitely the heartbeat of a project. Keep those check-ins coming and those builds green, and you know you’re heading in the right direction.

This article was originally printed in the August 2008 issue of Game Developer.

On paper it looks great:

• Built from the ground up for parallel builds
• Uses a real programming language (Python) instead of overly-complicated XML files
• It allows for discovery of assets to build as other assets are parsed

The big question mark hanging over it was whether it was fast enough. I had done some experiments with Scons and other build systems a few years ago, and Scons totally failed by having ridiculously long incremental build times just checking if anything had changed. That was simply not acceptable. If no data has changed, I want the build system to detect it and come back in a second or less.

Since then, I’ve been told that Scons had finally fixed its dependency checking and it was much faster now. Music to my ears!

So I resurrected the old script I wrote back then to generate a nightmare stress test, downloaded the latest stable version of Scons (1.0.1) and fired it up.

I’m sorry to report that Scons is still as unusable as it was back then. I generated a codebase like the one I used to measure all the build systems (50 libraries, 100 classes each, 15 internal includes, 5 external includes). I built everything once, then ran it again and it took 49 seconds. I have some pretty nice hardware, including a fast SATA hard drive and a 2.8GHz Core2Duo with plenty of RAM, and 49 seconds to say that nothing needs to be changed just doesn’t cut it.

Now to be fair, right now we’re looking at Scons to build our assets, not our code. Maybe the nested project configuration is not particularly realistic, so I tried something simpler: 5000 files in a single library without any dependencies among each other at all. This is a very conservative example for an asset build of a large commercial game.

• It’s only 5000 files. A real game can easily have 10 times as many asset files.
• All the files are tiny (cpp files in this case). Assets can get very large when dealing with textures, sounds, and animations.
• There are zero dependencies among assets in this example. You’ll often have models depending on shaders and textures, levels on game entity definitions, etc.

Building assets doesn’t have to be the instant, no delay kind of build that I require building code. I expect most developers will be building assets locally, tweaking things, rebuilding, and trying them in the game. The faster the build is, the better, but a delay of a few seconds can be acceptable. If Scons can’t handle this build in less than 10 seconds, it’s certainly doomed in a full-scale game.

The result: 37 seconds!

For purely masochistic reasons, I decided to see how it scales, so I threw 20,000 files at it (again, without any dependencies). Incremental build time without any changes: 3 minutes and 46 seconds. That’s over 6 times longer for processing 4 times the number of files, so it doesn’t even scale linearly.

Scons is definitely too slow, and MSBuild has its share of problems. What’s left out there? Any recommendations for good parallel asset building systems? Hopefully something lightweight and easily configurable through a real language. Anything?

A couple of months ago something unusual happened: The functional test failed but I wasn’t able to reproduce the problem right away. The failure was not a crash, but an object in the world ending up in a different state than expected. That’s always tougher to track down. To make things even more fun, was object was affected changed depending on whether the game was run from the command line or the debugger. Oh, and did I mention it only happened in release mode? I’ve got a baaad feeling about this!

My first instinct was to blame it on Havok. After some investigating, I narrowed it down to one of the enemies that was in mid-air. Sometimes it would fall down, and other times it would stay hovering, as if not affected by gravity. I knew that Havok was supposed to be deterministic, but it’s every programmer’s instinct to always mistrust other people’s code. I should have known better.

I spent a whole day tracking this down. Adding more and more information to the recorded world state to detect the problem as soon as possible. Slowly circling in; silently creeping up on the unsuspecting bug. Eventually, I had my “aha!” moment. I had finally found it.

A member variable, m_alive, in the RobotLogical class was never initialized! That was causing the Havok character controller not to be updated sometimes, causing the discrepancy in object state between the recording pass and the playback pass. What made it even more difficult is that m_alive is a boolean, so any bit pattern other than zero, it’s going to make it true, which is the initial state I expected for the robots. It never failed in debug mode because unused heap memory is filled with patterns like 0xdeadbeef or (the definitely less fun, but more practical) 0xcdcdcdcd, which all make the variable come up as true. Doh!

Yes, C++ sucks because of time-wasters like that, but it didn’t prevent me from feeling really stupid for having sunk a whole day on it.

All the code Charles and I write is done through TDD so we have almost 100% unit-test coverage [2]. How come Test Driven Development didn’t catch that? TDD is not about catching bugs, it’s about designing code. This is the type of error that is only triggered once every thousand times, so running one or two unit tests that cover that path probably won’t trigger it. That’s one of the many reasons end-to-end functional tests are invaluable. Go LeChimp!

Help, LeChimp! Help!

Detecting there’s a problem is good, but if it takes a whole day to fix it, things are going to be pretty painful. The more complicated the game gets, the more difficult life will be.

A good start is to crank up the warning level of your compiler as far as it goes. We’re compiling everything with warning level 4 in VC2005, so we’re already covered in that end [3]. Unfortunately, warning level 4 doesn’t warn us about uninitialized member variables. We need something else.

Fortunately we have a secret weapon in our arsenal we had been ignoring: Lint.

Lint performs static analysis of C++ programs and spews out lots and lots of warnings. Think of it as an extremely pedantic warning level on a C++ compiler. And I mean extremely! It will warn you not only of potential errors and dodgy constructs, but of best practices, and sometimes even of recommendations in books like Effective C++.

As you can imagine, with all those things to check against, Lint is going to find lots to complain about. So taming the output into something readable is a definite challenge.

When it comes to Linting your code, there are a few options available.

Team Edition Compiler

My friend Jim Tilander brought up that there’s a Lint-like tool hidden in the bowels of the Microsoft Platform SDK. Boy, he wasn’t kidding about the hidden part! It’s a whopping 1.4 GB download for a DVD image that then needs to be extracted and installed in order to get to the small free compiler. When will Microsoft adopt a more modular approach?

Once I got past all the installation hurdles, I was pretty excited to give it a test. A free tool is a free tool. And if it has good integration with Visual Studio, so much the better. Strictly speaking, it’s not really a Lint tool, but an optimizing compiler with a special /analyze switch, which performs a lot of the same checks as Lint. But as we know, a rose by any other name…

Right off the bat, it fell flat the moment I started using it. Apparently it doesn’t know how to deal with vcproj files. To work on all the files in a project, I had to feed each cpp file separately. That in itself is only a slight annoyance, but it meant that you also need to pass it all the compiler switches set in the vcproj files: include directories, defines, etc. For a Microsoft product, I really would expect it to understand vcproj files. Oh well. It’s not the end of the world. I can always write a quick script to do that for me.

Unfortunately, the next problem was more serious. There was no global way to control what warnings I wanted reported. I can disable specific warnings in the command line, but if I only care about a few, it gets really cumbersome.

Finally the killer: It simply doesn’t report some of the most important warnings I expect out of Lint, such as uninitialized member variables or variables appearing in incorrect order in the initializer list.

In its defense, it had some decent features I would expect of good Lint tools, like the ability to suppress certain warnings from code or to give extra hits in the code itself. On the other hand, the way to control the behavior of the analyze switch is done with special keywords, which will make other compilers choke, making it harder to write cross-platform code. Somehow, I’m not terribly surprised.

LeChimp’s recommendation: Even for a free tool, it simply doesn’t stand up to any serious use. Skip it.

PC Lint

Discouraged from the few hours I spent in the Team Edition compiler fiasco, I turned to Gimpel Software PC Lint.

Don’t be put off by PC Lint’s archaic web site that seems to be snatched straight out of 1995 [4]. PC Lint has been around forever and it shows. Not just in their web site, but in the program itself. Don’t expect any fancy GUIs, or dialog boxes, or any fancy cyber-amenities developed in the last 20 years.

Still, what it lacks in sophistication and pleasing visuals it more than makes up in functionality: Three minutes, $239, and a 4MB download later, I was ready to go.

In about an hour, I had PC Lint fully integrated with our build server and checking for problems in all of our source code. In just half an hour more, all our code was passing Lint’s rigorous inspection.

PC Lint can be quite intimidating out of the (virtual) box. If you run it on a typical codebase (or even a really high-quality one), it will treat you to megabytes of warning spew. Most of it useless, I have to say. So you have to take some time and tame it. Teach it to be a bit more polite, or more relevant. Or at least not to yell so much.

My preference is to start out with no warnings by starting out the options.lnt file with -e* and then add warning messages that I find useful: uninitalized member variables, unreferenced symbols, gotchas with virtual functions, etc..

Even running Lint with a small number warnings enabled, there were a few places in our code that caused PC Lint to complain but we didn’t want to change (Vec4 really shouldn’t initialize any member variables in its default constructor, or the NonCopyable class shouldn’t have a virtual destructor). In those cases, we can add a few discrete comments directly in the code that tell Lint to be quiet. We know what we’re doing, Lint. Really.

After we got all the code passing the initial set of checks, Charles dug into the Lint options file with a gleeful glint in his eyes and became the Lint-nazi, adding a good couple dozen extra warnings. It only took a few more minutes to get all the code to pass all the new checks. Frankly, the slowest part of the process is reading the thousands of different checks that PC Lint can do and decipher them (there are a couple of them I’m still unsure what exactly they mean).

This is what our options.lnt file looks like today:

-d__debugbreak()=
-e*

+e539  // Did not expect positive indentation from Location
+e549  // Suspicious cast
+e616  // control flows into case/default
+e744  // switch statement has no default
+e750  // local macro 'Symbol' (Location) not referenced
+e751  // local typedef 'Symbol' (Location) not referenced
+e752  // local declarator 'Symbol' (Location) not referenced
+e753  // local struct, union or enum tag 'Symbol' (Location) not referenced
+e764  // switch statement does not have a case
+e766  // Include of header file FileName not used in module String
+e767  // macro 'Symbol' was defined differently in another module
+e773  // Expression-like macro 'Symbol' not parenthesized
+e777  // Testing floats for equality
+e783  // Line does not end with new-line
+e784  // Nul character truncated from string
+e789  // Assigning address of auto variable 'Symbol' to static
+e796  // Conceivable access of out-of-bounds pointer
+e797  // Conceivable creation of out-of-bounds pointer
+e801  // Use of goto is deprecated
+e802  // Conceivably passing a null pointer to function 'Symbol'
+e803  // Conceivable data overrun for function 'Symbol'
+e804  // Conceivable access beyond array for function 'Symbol'
+e806  // Small bit field is signed rather than unsigned
+e814  // useless declaration
+e815  // Arithmetic modification of unsaved pointer
+e817  // Conceivably negative subscript (Integer) in operator 'String'
+e818  // Pointer parameter 'Symbol' (Location) could be declared ptr to const
+e820  // Boolean test of a parenthesized assignment
+e825  // control flows into case/default without -fallthrough comment
+e940  // omitted braces within an initializer
+e954  // Pointer variable 'Symbol' (Location) could be declared as pointing to a const

// TURN THESE ON IF WE CAN FIGURE OUT HOW TO APPLY IT ONLY TO VALUES AND NOT POINTERS!
//+e952  // Parameter 'Symbol' (Location) could be declared const
//+e953  // Variable 'Symbol' (Location) could be declared as const

+e1401 // member symbol 'Symbol' (Location) not initialized by constructor
+e1404 // deleting an object of type 'Symbol' before type is defined
+e1411 // Member with different signature hides virtual member 'Symbol'
+e1413 // function 'Symbol' is returning a temporary via a reference

+e1506 // Call to virtual function 'Symbol' within a constructor or destructor
+e1507 // attempting to 'delete' an array (not a pointer)
+e1509 // base class destructor for class 'Name' is not virtual
+e1511 // Member hides non-virtual member 'Symbol' (Location)
+e1512 // destructor for base class 'Symbol' (Location) is not virtual
+e1516 // Data member hides inherited member 'Symbol' (Location)
+e1520 // Multiple assignment operators for class 'Symbol'
+e1521 // Multiple copy constructors for class 'Symbol'
+e1534 // static variable 'Symbol' found within inline function in header
+e1535 // Exposing low access data through member 'Symbol'
+e1537 // const function returns pointer data member 'Symbol'
+e1538 // base class 'Name' absent from initializer list for copy constructor
+e1539 // member 'Symbol' (Location) not assigned by assignment operator
+e1541 // member 'Symbol' (Location) possibly not initialized by constructor
+e1542 // member 'Symbol' (Location) possibly not initialized
+e1543 // member 'Symbol' (Location) possibly not initialized
+e1544 // value of variable 'Symbol' (Location) indeterminate (order of initialization)
+e1545 // value of variable 'Symbol' used previously to initialize variable 'Symbol' (Location)
+e1547 // Assignment of array to pointer to base class (Context)
+e1552 // Converting pointer to array-of-derived to pointer to base
+e1554 // Direct pointer copy of member 'Symbol' within copy constructor
+e1555 // Direct pointer copy of member 'Symbol' within copy assignment operator
+e1556 // 'new Type(integer)' is suspicious
+e1557 // const member 'Symbol' is not initialized
+e1561 // Reference initialization causes loss of const/volatile integrity

+e1705 // static class member may be accessed by the scoping operator
+e1706 // Declaration with scope operator is unusual within a class
+e1707 // static assumed for String
+e1710 // An implicit 'typename' was assumed
+e1711 // class 'Symbol' (Location) has a virtual function but is not inherited
+e1718 // expression within brackets ignored
+e1719 // assignment operator for class 'Symbol' has non-reference parameter
+e1720 // assignment operator for class 'Symbol' has non-const parameter
+e1724 // Argument to copy constructor for class 'Symbol' should be a const reference
+e1726 // taking address of overloaded function name 'Symbol'
+e1729 // Initializer inversion detected for member 'Symbol'
+e1734 // Had difficulty compiling template function: 'Symbol'
+e1736 // Redundant access specifier (String)

+e1931 // Constructor 'Symbol' can be used for implicit conversions

I originally only had about a dozen warnings enabled, but, as you can see, Charles really went to town with it

To integrate it into our functional test, we just set up a new target in msbuild. For each project, we call it twice: once to generate a lnt file for the project, and once to run it on the files for the project.

<PropertyGroup>
    <LintDir>..\Bin\Lint\</LintDir>
    <Lint>$(LintDir)lint-nt.exe</Lint>
</PropertyGroup>

<ItemGroup>
    <ProjectFiles Exclude="..\Engine\**\*Tests.vcproj"
                  Include="Src\SweetPea.vcproj;..\Engine\**\*.vcproj;"/>
</ItemGroup>

<Target Name="Lint" DependsOnTargets="CleanLint;RunLint"/>

<Target Name="CleanLint">
    <Delete Files="%(ProjectFiles.RelativeDir)%(ProjectFiles.Filename).lnt"/>
</Target>

<Target Name="RunLint" Inputs="@(ProjectFiles)"
        Outputs="@(ProjectFiles -> %(ProjectFiles.RelativeDir)%(ProjectFiles.Filename).lnt)" >
    <Exec Command="$(Lint) -v -os(%(ProjectFiles.RelativeDir)%(ProjectFiles.Filename).lnt) %(ProjectFiles.Identity)"/>
    <Exec WorkingDirectory="%(ProjectFiles.RelativeDir)"
          Command="$(MSBuildProjectDirectory)\$(Lint) -i$(MSBuildProjectDirectory)\$(LintDir) std.lnt %(ProjectFiles.Filename).lnt"/>
</Target>

Whenever PC Lint detects some code that violates one of its rules, it returns an error message, which fails the msbuild target and the whole build is reported as failed by Cruise Control .Net. I love it when things work exactly like you want them to without doing any extra work. To make it even better, because PC Lint can output errors in VC++ format, they’re understood and parsed correctly by msbuild and CCNet, so they show up in the CCNet log like any other error.

PC Lint runs surprisingly fast. It takes about 20 seconds to run on all of our source code. I was almost tempted to add it to the incremental build in the build server, but I really want to keep build times to a minimum there, so it runs once every hour, which seems to be good enough.

If you want to be more hands-on with PC Lint, it’s a breeze to integrate with Visual Studio. Just set up a few custom commands in External Tools and you’re good to go. The Visual Studio Lint options file even details exactly how to set up those commands.

My only minor complaint with PC Lint is the strange behavior of error 766 (Include of header file not used in module). I don’t know what algorithm PC Lint uses to determine that, but it seems not to report some unused headers. That wouldn’t be a big deal, but sometimes just rearranging headers or moving some code around will cause it to recognize a header as unused, triggering a failed build. It’s a very useful warning though, so I’m willing to live with that quirk.

Working with PC Lint is a pleasure. It’s really low level and old school, but that’s precisely what made it so easy to integrate with our code and our build system. It doesn’t need installation programs, databases, registry entries, dll deployments, or anything like that. Just a good-ol’ ini file with options and you’re good to go. We even have it checked it in version control to make it easier to deploy to our development stations and the build server.

PC Lint has the characteristics I want in a tool: Easy to buy, deploy, and integrate into the way I want to work with it.

LeChimp’s recommendation: An absolute must! Worth every penny. Two opposable thumbs up!

Other Lint Alternatives

Lint was originally a Unix utility developed in the late 70s. You would expect that lots of different versions would be available for modern compilers and platforms, but there aren’t that many. Maybe it’s not a really fun project to engage enough Open Source hackers?

I have to admit I haven’t looked at any other alternatives to PC Lint and the Team Edition Compiler that work under Windows. A quick Google search reveals there are a few tools out there. Has anybody tried them? I’d love to hear your experiences with them.

• Splint. Only works on C. Emphasis on security. Open source.
• CodeCheck. Works on C++. Commercial.
• A bunch for Linux/Unix. As much as I’d like to, the reality is that most game development happens under Windows.

Conclusion: LeChimp Rules The Day (Again)

PC Lint should be a requirement for any C++ project. The day you set up a build server (which should be the first day), go ahead and spend the extra hour setting up PC Lint as well. Be safe up front and avoid wasting any time down the line. We should have followed our own advice, but sometimes priorities slip a bit in the rush of getting a whole company off the ground.

Now I can finally sleep better knowing that LeChimp is keeping busy linting my code several times a day.

[1] This is our current broken build sound bite.

[2] There are some things we don’t bother TDDing: main functions, super-high level leaf code, etc.

[3] Of course, you need to have a clean compile in warning level 4 in order to be worth it. That means builds without any warnings or other spew. Incidentally, I do hate the setting to treat warnings as errors because it means you can’t temporarily have warnings while you’re refactoring between checkins, which slows things down.

[5] That was just a wild guess, but it seems it really wasn’t that different in 1996.

Of course, we’re far from loaded with money, so keeping expenses to a minimum is definitely a top priority. It’s not coincidence that one of our most popular lunches is sharing a gigantic $5 sub at Manhattan Giant Pizza or the $3.50 Kealani’s chicken teriyaki sandwich (fortunately, they’re actually delicious too!).

Whenever it makes sense, we’ve opted for the most inexpensive options[1]: plastic workbenches from Costco for our desks, lights from Ikea with unpronounceable names, an outdated P3 that became our file server given to us by a friend, or a free scanner/printer donated by my girlfriend.

So when it came time to set up our office tools, we also looked for the most inexpensive solution that met all our needs.

Email

Our requirements for our email solution were:

• Minimize costs. Not just the software, but the hardware needed to run any severs, support, etc.

• Minimize maintenance. We’re our own IT staff, so the less time we have to spend screwing around with severs and updates, the more time we can spend coding and designing the game.

• Reliable. This, after all, is going to be used to send company emails, contracts out to outsourcing companies, responses to publishers, etc. We don’t want to be missing any emails.

• Good search capabilities.

• Available from any location. I guess that implies a web front end.

• Fully integrated with our domain powerof2games.com.

In the past, the companies we had worked at all used Microsoft Exchange and Outlook or the god-awful Lotus Notes. Apart from having to endure the torture of using these day in and day out, they’re quite expensive, so we looked for alternatives.

We could use the email server provided by our web hosting company. It would give us the bare bones functionality that we needed, including some form of web front end. It could work, but after using Gmail for personal email for several years, it felt like a major step backwards.

Since Charles and I were already using Gmail for our personal accounts, we thought of creating a new account for Power of Two Games, but having pow2games@gmail.com in all our business emails doesn’t exactly look very professional.

That’s when we found out about Google Apps for small business. It’s a free service offered by Google that allows integration of Gmail into your domain and use it as your corporate email. That seemed too good to be true. Gmail is hands down the best email web interface I have ever used, so the thought of using it as the company email was extremely tempting.

Setting up Gmail took a bit of messing around with some DNS advanced settings. All email sent to powerof2games.com had to be routed to Google somehow, so we had to set up a custom MX record in our web host and point it to Google’s servers.

We also wanted to access our email through a nice URL like mail.powerof2games.com instead of some long (and hard to remember) default URL going through Google. For that, we had to add a CNAME record pointing the new mail subdomain to ghs.google.com.

Once all that was set up, Gmail was up and running and fully integrated with our Power of Two domain. Not only that, but it was totally independent of our personal Gmail accounts, so we can even have them both open in different tabs in the same browser. Good job, Google!

Calendar

Gmail was just the tip of the iceberg. By going the Google route, we also get Google Calendar for our domain. And Google Calendar totally rules. It’s not like we were planning on scheduling company meetings at all hours of the day (although, technically, since we’re in the same office I guess we’re in a permanent meeting). But Calendar comes in really handy to put down big milestones, remind us to pay the rent, or keep track who we’re having lunch with this week.

Calendar works out of the “box” without any extra set up. The only thing we did was to create a custom subdomain to access it through calendar.powerof2games.com, so we did the same trick as for our mail subdomain with the CNAME entry.

I can even share the Power of Two Calendar with my personal Google Calendar account and see all my events combined in a single place. My whole life in a single web page. Very handy!

Extras

But wait, there’s more. We’re not always working at the office at the same time. Sometimes we’ll work from home for a few hours in the morning before walking down to the office, or maybe I’ll do some work from E Street late in the evening with an oatmeal chocolate chip cookie and some live music. At those times, it’s very helpful to be within easy reach of communication, and since we’re using Gmail, we get Gmail chat for free.

Documents

I’m usually very hesitant of “integrated solutions”. I always prefer to be able to mix and match programs, components, or whatever instead of going the route of vendor lock-in. And the truth is that the different Google components are totally optional, but it’s curious that we’re biting, hook, line, and sinker all the way into Google’s office suite.

I was thinking we would end up using OpenOffice.org, which is a great Microsoft Office clone. But we gave a try to Google Documents and Spreadsheets and we were totally impressed.

Not only has Google Documents reached point as far as features and user interface that they’re usable for real-world situations, but they offer a lot more functionality.

• It’s a web-based app, so we can edit the documents from any computer with web access.

• We can collaborate on the same document at the same time, which came in very handy when we were both working on the Sony Developer Program application document.

• Because it’s a Google product, you can search through the contents of all your documents and spreadsheets very easily.

• Tagging and sorting. You can tag any document with any set of labels and then sort or filter by them. I’m glad the days of hierarchical trees for organizing information are coming to an end.

• Exporting to a variety of formats (MS Office, pdf, etc) comes in very handy when we need to send a document to somebody else who is not using Google.

There are only a few things we wish it did differently, like being able to upload a pdf document (even if it’s just in read-only mode for archiving and searching purposes). The other odd quirk is that we can’t make all documents and spreadsheets shared by default between the two of us. Instead, we need to explicitly share them. Oh well. I guess they have bigger companies than us in mind. I don’t blame them.

It’s very encouraging to see these applications grow and get better week after week. Just a few months ago, Google Spreadsheets couldn’t do plots, but they added that functionality since then. They’ve also improved the whole front end with the tagging and listing of documents.

Voice communication

How the times have changed. When we started Power of Two Games, we didn’t even bother with a phone. Even though we ended up contracting a DSL Internet connection and getting a voice phone line would have been extremely easy, we simply didn’t need it. We were simply using our cell phones for the few times when email communication or chat wasn’t good enough.

But as we started outsourcing art and talking to console manufacturers, we started having the need to do voice conferences and calling internationally. After a couple, very uncomfortable, conference calls using the speakerphone on our cellphone, we decided that we really had to look for a better solution.

For once, Google didn’t have the solution for us (although wait a couple of years and see). I have been using Skype to call internationally for quite a while with great results, so it was natural that we would look at it as our first option. Again, we were pleasantly surprised: not only could we make very inexpensive international calls (with great voice quality), but for a small yearly fee we can get SkypeIn and have anybody call us from a regular phone.

Conclusion

We’re extremely pleased with how Google Apps and Skype have been working for us so far. They fit our needs perfectly and the cost is right. We were able to set things up in about a morning and never have to fuss with it again. How good is that?

I really don’t see why Google Apps wouldn’t work for a larger company. But companies seem to be stuck in the Exchange/Microsoft Office (or some other heavyweight and expensive server solution) choice. I wonder if there’s a good reason for that other than inertia.

We’re certainly happy to have these services available for free and being about to spend our time and money in what really matters: Creating a kick-ass game and getting it out the door.

Notes

[1] There are some things where going rock-bottom cheap doesn’t make any sense though. The $7 desktop speakers we bought online were quite the surprise. I wasn’t expecting top quality sound, but they sounded like those two way radios that security guards use. I didn’t even know they made speakers that bad! Also, we have “splurged” on 24″ LCD monitors and comfortable chairs (and even so, always trying to find the cheapest price point for what we want), because those things make a world of difference when you’re using them many hours every day.

]]> http://gamesfromwithin.com/office-tools-for-starving-startups/feed 13 http://gamesfromwithin.com/unittest-v10-released http://gamesfromwithin.com/unittest-v10-released#comments Sun, 19 Mar 2006 02:48:16 +0000 Noel http://www.gamesfromwithin.dreamhosters.com/?p=343 Continue reading →]]> We grabbed the best features of each framework and created what we think it’s the best C++ unit-testing framework out there (for our needs anyway). We took the results and put them up in Sourceforge under a veryunrestrictive license, and that’s how UnitTest++ was born.

I had been using a modified version of CppUnitLite for quite a while, slowly fixing the parts in need of mending, and adding new functionality as it became needed. Eventually I released most of those changes as CppUnitLite2, and I thought that would be the end of that. It turns out that was just the beginning.

Shortly after that, several people started emailing me with new functionality, fixes, patches, suggestions, etc. At the same time, Charles Nicholson was starting to get quite a bit of response as well for his own testing framework, sometimes with similar fixes and suggestions.

If this was going to grow to be a real project, developing over time, supporting users’ needs, and accepting code contributions, clearly it would make sense to join forces. And that’s exactly what we did. We grabbed the best features of each framework and created what we think it’s the best C++ unit-testing framework out there (for our needs anyway).

We took the results and put them up in Sourceforge under a very unrestrictive license, and that’s how UnitTest++ was born.

Our ultimate goal is to apply test-driven development to game development. All of the existing frameworks fell short in one area or another. Specifically, the driving forces behind the design of UnitTest++ are:

• Portability. As game developers, we need to write tests for a variety of platforms, most of which are not supported by normal software packages (all the game consoles). So the ability to easily port the framework to a new platform was very important.
• Simplicity. The simpler the framework, the easier it is to add new features or adapt it to meet new needs, especially in very limited platforms.
• Development speed. Writing and running tests should be as fast and straightforward as possible. We’re going to be running many tests hundreds of times per day, so running the tests should be fast and the results well integrated with the workflow.

UnitTest++ is a fully featured testing framework, with many of the features you may expect from Xunit-style frameworks such as fixtures and reporting. Additionally, here are some of the specific features that make UnitTest++ unique:

• Minimal work required to create a new test. For TDD development, we write lots and lots of tests, so creating a new test should be as quick and painless as possible. UnitTest++ does not require explicit test registrations, and creating new tests requires minimal work.
• Good assert and crash handling. When automated tests are executed, it’s very important not to hang the process or abort the tests any time the program crashes or an assert fails. UnitTest++ will catch and report exceptions as failed tests and print a lot of information about them. It will translate signals to exceptions as well as be able to catch invalid memory accesses and other problems.
• Minimal footprint and minimal reliance on heavy libraries. When you intend to run tests on a Cell SPU with a measly 256 KB of RAM, you really want to be as lean and mean as possible. UnitTest++ is very lightweight, and it will get even lighter weight once strstream is replaced.
• No dynamic memory allocations done by the framework, which makes it much easier to track memory leaks and generally more attractive for embedded systems.
• Multiplatform support. Right now it supports Win32, Linux, and Mac OS X out of the “box,” and other platforms will probably work without any changes. The source code makes use of platform-specific functions whenever necessary, so it is easy to hook up special timers, allocators, or reporters for specific platforms. The code comes with a makefile as well as with project and solution files for Visual Studio 2003 and 2005.
• Full set of unit tests for itself. UnitTest++ was TDD’d from the ground up, including some of the ugly macros (imagine how much fun it was bending the framework to test itself that way!). It goes without saying, all the tests are included with the source code, so you can go totally wild with them

This was just the first release. UnitTest++ is ready for full production work, but we already have plans beyond this release. This is a peek at some of the features we have in mind for upcoming versions:

• Out-of-the-box support for game console platforms (Xbox 360, PS3 PPU and SPU, and Nintendo Revolution). Of course, this is dependent on us getting permission from the console manufacturers to release some code that works with their SDK.
• Ability to trade some features (such as rich check reporting) for some extra memory. In some platforms, like Cell SPUs, you really need to go barebones.
• Suites to group tests together.
• Different reporters (HTML, GUIs, etc). Don’t worry, all those modules will be optional and well separated, so they won’t affect the simplicity of the framework.
• Per-test timings. Maybe the ability to flag some tests as not exceeding a certain amount of time, or just failing any test that goes over a threshold.
• Built-in memory leak detection.

So that’s it. Download the source code and give it a try. We welcome any comments and suggestions (best channels would be through the project mailing list or comments here). We hope you find it useful and that it makes your TDDing even more productive.

CppUnitLite2_1_1.tar.gz

About a year ago, I wrote an article comparing unit test frameworks that went to become one of the most popular in this site. The article concluded with the recommendation of one of the less well known frameworks: CxxTest.

Shortly after writing that article, it came time to roll out test-driven development at work. Interestingly, once I weighted in all the requirements, I ended up not following my own advice. Instead, I decided to go with a customized version of CppUnitLite, which was listed in the article as the second choice. The idea of an extremely simple unit testing framework was too attractive to pass up in an environment where we have to support a large variety of environments and target platforms.

At this point, we have been using CppUnitLite2 for a year at High Moon Studios to do TDD on Windows, Xbox 360, and some PS3 (we even have a stripped-down version that runs on a PS3 SPU). It has been used to unit test libraries of an engine, pipeline tools, GUI applications, and production game code.

CppUnitLite2 was originally released along with my article a year ago. It was a heavily modified version of the original CppUnitLite by Michael Feathers. This version introduces the changes we made at work as different issues came up and we needed more functionality from the framework. In particular, Jim Tilander and Charles Nicholson were responsible for a lot of those changes. By now, I doubt there’s a single line left from the original code. Additionally, CppUnitLite2 itself has a set of tests using itself in an interestingly recursive way.

What’s new

Fixtures created and destroyed around each test.

I can’t believe that the original version of CppUnitLite2 didn’t have this feature. It was the biggest glaring problem with the framework (I even pointed it out in the article). Each fixture used to be created at static initialization time, which, apart from being a really bad idea, caused a lot of objects to be created and live in memory at the same time. Considering that some of those objects would initialize/shutdown some important game systems, we clearly needed to manage their lifetime a bit better.

The test macro was changed to register a temporary test with the TestRegistry, which in turn will create the fixture at execution time. Now fixtures and tests are created just before each test, and destroyed right after, just as you would expect from any sane framework.

For those masochists out there, here’s the ugly macro for a test with fixture in all its glory. Notice how the TestRegistrar registers the dummy test class that we created as part of the macro.

 #define TEST_F(fixture, test_name)                                                      \ struct fixture##test_name : public fixture {                                        \ fixture##test_name(const char * name_) : m_name(name_) {}                       \ void test_name(TestResult& result_);                                            \ const char * m_name;                                                            \ };                                                                                  \ class fixture##test_name##Test : public Test                                        \ {                                                                                   \ public:                                                                         \ fixture##test_name##Test ()                                                 \ : Test (#test_name "Test", __FILE__, __LINE__) {}                       \ protected:                                                                      \ virtual void RunTest (TestResult& result_);                                 \ } fixture##test_name##Instance;                                                     \ TestRegistrar fixture##test_name##_registrar (&fixture##test_name##Instance);       \ void fixture##test_name##Test::RunTest (TestResult& result_) {                      \ fixture##test_name mt(m_name);                                                  \ mt.test_name(result_);                                                          \ }                                                                                   \ void fixture ## test_name::test_name(TestResult& result_)

Improved check statements

Another one of the major weaknesses of CppUnitLite as it stood was the check statements. You needed to have a custom check macro for each data type you cared to check. The generic CHECK_EQUAL macro will now work on any data type as long as it can be compared with operator == and output to a stream.

This version of CppUnitLite2 also includes some special macros to check arrays with one and two dimensions. Again, this works on any data type that supports operator == and operator[]. It came it very handy for checking equality of vectors and matrices in a game engine. For example, the following code checks that two matrices are equal:

Ironically, in some heavily restricted environments (like the PS3 cell processors), we can’t use streams, so in that case we’re forced to go back to the old specialized check macros.

Invariant statements in checks

Another really annoying feature of the previous version of CppUnitLite2 was that the code in a check statement was called multiple times, which could cause strange side effects.

Consider the following code:

The way the check statement was expanded out, both those functions could be evaluated multiple times. If the functions had side effects, they would be executed several times, causing unexpected results (for example, the check could fail but the printed result could be different). The fix wasn’t trivial because we couldn’t save off the value of each statement because they could be of any type and we have no way of finding the type from within the macro.

The check macro now expands out to a templated function, which is able to evaluate both statements only once and work with the saved value to avoid any surprising side effects.

Optional fixture initialization

As we continued hammering on the unit test framework for all our development, one clear pattern appeared that wasn’t handled by the framework: Being able to create fixtures with parameters defined in the test. For example, the fixture might do ten different steps, but we want to set a particular value in one of those steps, and by the time control reaches the test code itself it’s too late.

We started getting around that by creating different fixtures, and then by inhereting from other fixtures and changing some parameters. Both of those solutions were less than ideal (lots of code duplication and added complexity). Things got out of hand when we made the fixtures a templated class on a value, and created them by specifying the value as part of the texture name. Now it was a pain to debug and things were even less clear.

So we finally implemented it natively in CppUnitLite2 as fixtures with initialization parameters. We introduced a new macro, TEST_FP (test with fixture and parameters), which takes any parameters you want to pass to the fixture constuctor.

Now you can declare tests like this:

We should prob
ably simplify that to avoid repeating the fixture name, but that works fine for now.

Include cleanup

The original version of CppUnitLite was pretty casual about what headers it included. Unfortunately, that means that all test code were automatically exposed to a variety of standard headers, whether you wanted it or not. This version is a lot more strict, and the only header that will get pulled in from using it will be iosfwd, which is even a fairly lightweight one.

Memory allocations

We have very strict memory allocation wrappers around our tests, and if any memory leaks, it is reported as a failed test. Since checking for memory allocations is very platform specific, it is left out of the framework (although it could be added pretty easily in the platform-specific sections). In any case, some of the static-initialization time allocations were confusing the memory tracker, so we removed them completely. All it means is that we use an array to register tests instead of a std::vector and we cap the maximum number of tests to a fixed amount. If you need more than the default 5000 tests, go right ahead and add a few zeros.

New license

Previous versions of CppUnitLite were not released with an explicit license. To make things clear, I explicitly added the license text for the MIT license with an added restriction. The MIT license allows you to use the code in any project, commercial or otherwise, without any restrictions. The added restriction to the license prohibits its use in any military applications or projects with any military funding.

Ratings

How does this release of the CppUnitLite2 compare with respect to my initial set of features I used to rate the different unit tests frameworks?

• Minimal amount of work needed to add new tests. Nothing has changed there. Still passes with flying colors.
• Easy to modify and port. That has always been the strong suite of this framework.
• Supports setup/teardown steps (fixtures). Much improved now by creating them correctly around each test.
• Handles exceptions and crashes well. No changes there. Still one of the best I’ve seen.
• Good assert functionality. The check macros have been much improved to match the best frameworks out there.
• Supports different outputs. Yup. Still there.
• Supports suites. It still doesn’t. It just shows that I really haven’t needed this feature yet.

Future work

So, what’s next for CppUnitLite2? It’s really quite usable right now, but I could see a few changes happening in the next year, and maybe a few others will come up and surprise me.

Test registration is based on static initialization of global variables, which is not very reliable across compliers. For example, if you try to put the tests in a static library, MSVC will strip out the registration code leaving you with no tests. To get around that, we could introduce a custom build step in a scripting language that would generate a simple file with explicit registration of tests. This could also be used to collect tests from many different libraries into a single executable.

A couple of times it would have been convenient to have parameterized tests (I think that’s the correct term for it). Basically, to have tests that could run with different data. Something like this:

 TEST_PARAM(MyCustomTest, int a, int b) { CHECK (something); CHECK(something else); CHECK(yadda yadda); }  TEST(MyTest) { CALL_TEST(MyCustomTest, 10, 5); } TEST(MyTest) { CALL_TEST(MyCustomTest, 12, 7); }

The key point is that a test fails in the parameterized test, it should report the error correctly indicating where it came from.

This is actually pretty easy to implement with a couple of macros. We simply haven’t needed it more than a couple of times, so we haven’t bothered yet. But I imagine next time we run up against this we’ll bite the bullet and implement it.

Wrap-up

Charles had so much fun working with this framework, he decided to write his own from scratch. He made it available on his web site, so go check it out.

Thanks to High Moon Studios for being open and letting us release the changes we made and share them back with everybody. If you end up using the framework and you make any changes, drop me an email and I’ll add them for the next release.