Google App Engine As Back End For iPhone Apps

As soon as a game involves servers, there’s no such a thing anymore as “ship and forget”. Flower Garden put this in evidence about a month ago when I started getting complaints from users that the Flower Shop kept going down. Sometimes they weren’t even getting the items they were purchasing! (fortunately they can always do a free re-download, but they don’t always know that).

Flower Garden was using a shared Dreamhost server to upload bouquets, host in-game news, and, most importantly, to serve the Flower Shop (which involves hosting the static shop files, redeeming promo codes, recording transactions, verifying receipts with Apple’s server, and delivering purchased content). All the Flower Shop functionality was implement in PHP and using a mySQL database.

After some stressful sorting through the logs, it looked like the server was running out of memory and killing PHP processes. Dreamhost’s technical support didn’t help much and just claimed that I was simply something that happened and they would be happy to sell me a virtual private server. I was surprised to hear it was using up a lot of memory, and certainly nothing I could see with the tools available to me showed that. I certainly hadn’t done any changes in a while and traffic was constant. Doing some Googling at the time brought up lots of other angry Dreamhost users in a similar situation, so I suspected server configuration problems.

Whatever the case, I couldn’t let it be that way and I had to fix it somehow. I briefly considered a virtual private server, or even a dedicated host in some highly recommended providers. But in the end, the simplicity, scalability, and affordability of the Google App Engine won me over.

Now that Flower Garden has been using the Google App Engine for several weeks, I still think it’s fantastic. I wish someone had whacked me on the head when I started writing server code and forced me to use it. Hopefully this post will be more pleasant than a hit to the head and will still have a similar (good) effect.

Google App Engine Overview

Even though I was pretty much completely new to the Google App Engine (I had looked into it briefly before the launch of Flower Garden but dismissed it because they couldn’t support the amount of email traffic I needed), it took me two days to port over everything. Actually flipping the switch from the old system to the new one took a while longer, but that required more courage than work. More on that later.

This is not a Google App Engine tutorial. Instead, it’s going to be an overview of what it can do and why you should consider it as a backend for your iPhone app instead of using a shared or even dedicated server.

The first shock (at least for me) when looking into the Google App Engine is that you can’t run PHP. You’re limited to Java or Python. It turns out Python is my go-to scripting language, so I was thrilled at the idea. PHP is fine, but Python is a whole class above.

Next, it took a bit of adjusting to the fact that the environment to run anything on the Google App Engine has to be carefully controlled. You need to create a configuration file indicating the entry points and their handlers. Fortunately, all of that is well covered in the extensive online documentation.

This is what the Flower Garden config file looks like, indicating both static files and several execution entry points:

application: st-flowershop
version: 1
runtime: python
api_version: 1

handlers:
- url: /catalog
  static_dir: catalog

- url: /emailassets
  static_dir: emailassets

- url: /news
  static_dir: news

- url: /moregames
  static_dir: moregames

- url: /.*
  script: purchase.py

If you hit one of the URLs listed as static_dir, you get those files directly (like the in-game news). Anything else is handled by the Python script purchase.py. Google even provides the webapp framework that easily allows you to connect different parts of the script to handle different requests and easily get to the input parameters.

Once you’re past that and you have the requisite “Hello World” up and running, then it’s all fun and games.

One of the great features of the Google App Engine is that it comes with a fully-featured, local environment. This environment is installed automatically with the Google App Engine SDK, and there’s nothing to configure. It’s certainly nothing like setting up Apache + mySQL + PHP in your dev station! That way, you can do all your work locally, and only update the servers when you’re confident everything is working.

Beyond this, the only other thing that is different from what you may be used to is the datastore. They provide a query language called GQL, which is not exactly SQL, but it’s very similar. Close enough that I had no trouble porting over my very simple queries anyway.

Adding data to a data store couldn’t be simpler. For example, to add a promo code, I need the following class definition:

class PromoCode(db.Model):
	code = db.StringProperty()
	productid = db.StringProperty()
	amount = db.IntegerProperty(default=1)
	singleuse = db.BooleanProperty(default=True)
	enabled = db.BooleanProperty(default=True)

And then I can add it this way:

	promocode = PromoCode()
	# fill it up here
	promocode.put()

How do I check if a promocode is valid? I can use the SQL-like syntax, or something even simpler:

	q = PromoCode.all()
	q.filter("code =", code)
	return q.get()

Once you know that, you can go to town with your Python programs.

Google App Engine Advantages

Scalability
This was the big reason that forced me to move away from Dreamhost. I have no way of testing how scalable it really is, but I’m going to take Google’s word for it. They know a tiny little bit about scalability. And besides, Flower Garden is nothing compared to other apps (it serves about 1.5 requests per second on average).

Since I don’t know how things are implemented under the hood in the Google App Engine, I don’t have a good feel for performance best practices. So I’m hoping that my very simple queries (add to the end of a table, or see if a record is present) are just fine. If not, I can do some tuning down the line (and I won’t have to wait for any approval to make the code go live!).

Local environment
The local environment is simply great. It makes developing a pleasure. You can add any test data you want to the local datastore through a web interface without affecting the live server. Probably my favorite feature is the console: Since the server is running locally, you can print out any debug info in the server code and you can see it live as you perform some action. That saved me hours of debugging compared to doing it on a remote server with php!

Server synching
In the past, I’ve synced my server scripts through ftp and I kept meaning to write an automated script to do that. In this case, Google provides a script (and a GUI tool) to do the actual synching with the servers, which is great. It even waits a few seconds until it can verify that the new version is live on the server.

Python
I’m not a big Java fan (although I used it a bit way back in the day when it first came out), but I do love Python. For a program with some complexity like this, it’s definitely a much better choice than PHP.

Real-time stats
The Google App Engine web console displays real-time stats of access to your site. You see it expressed in requests per second and you can visualize it in the past few hours, days, or even month. You also have access to all the logs, and you can filter by type of log message (debug, info, warning, error).

Price
This is one category where the Google App Engine definitely shines. Every day you’re given a free quota of bandwidth, disk usage, and CPU usage. If you stay under that quota you don’t pay anything. If you go over, you pay per unit as you go (at very reasonable prices). You can even set a maximum cap on daily spending so you don’t encounter any nasty surprises at the end of the month.

So far, Flower Garden has been hovering right at the edge of the outgoing bandwidth free quota (the rest of the metrics it doesn’t even come close). So even if traffic were to double, expenses would be very reasonable.

Development Tricks

Along the way, I figured out a couple of interesting tricks that helped me during development.

Bypass the App store for testing
For me, few things are more annoying than doing development in the actual device. Iteration is slow and stepping with the debugger is mostly impossible (seems to depend on the mood of the debugger that day). So I try to do the bulk of the development on the simulator and just test on the device when things are ready.

Unfortunately, the device can’t access the App Store, which makes testing all the server functionality dealing with the App Store painful. What I did was to add a define that, when present, the program bypasses the App Store but still functions as usual.

Local server
During development, you want to be using your local server. In my case, I set it up so that Debug versions of the code access the local server, but Release and Distribution ones access the live one.

There is one problem: The Google App Engine server, by default, binds itself to localhost, not to the IP address of your network card. That meant I was able to hit it from the simulator just fine accessing http://localhost:8080, but not from the device. In order to access the development server from the device, I had to explicitly tell the server to bind itself to another address like this:

dev_appserver.py --address 192.168.1.150 FlowerShop

Now I can access the server at 192.168.1.150:8080 from both the simulator and any device.

Switching servers
Porting the code took just a couple of days. Making the switch on the live server was a lot scarier though. I couldn’t wait for a new update to be released because that seems to take over a week these days, so I forwarded requests from the old server to the new one.

I did it one system every day, avoiding weekends which is when there’s the most traffic. News was easy, and so was the web view with more games. But I ran into a snag with promo codes and Flower Shop purchases.

Initially, I was just redirecting requests in .htaccess this way:

Redirect permanent /Shop/catalog http://st-flowershop.appspot.com/catalog
Redirect 301 /Shop/catalog http://st-flowershop.appspot.com/catalog

As I learned the hard way, POST variables aren’t preserved in a redirect of that kind (I guess to avoid state being changed in multiple servers). I tried a bunch of things, but in the end, I wrote a quick php script that gathered all POST variables and re-posted them to the new request. A bit ugly, but it did the trick:

<?php
function PostRequest($url, $_data) {

    $data = array();    
    while(list($n,$v) = each($_data)){
        $data[] = "$n=" . urlencode($v);
    }    
    $data = implode('&', $data);
    // format --> test1=a&test2=b etc.

    $url = parse_url($url);
    if ($url['scheme'] != 'http') { 
        die('Only HTTP request are supported !');
    }

    $host = $url['host'];
    $path = $url['path'];

    $fp = fsockopen($host, 80);

    fputs($fp, "POST $path HTTP/1.1\r\n");
    fputs($fp, "Host: $host\r\n");
    fputs($fp, "Content-type: application/x-www-form-urlencoded\r\n");
    fputs($fp, "Content-length: ". strlen($data) ."\r\n");
    fputs($fp, "Connection: close\r\n\r\n");
    fputs($fp, $data);

    $result = ''; 
    while(!feof($fp)) {
        $result .= fgets($fp, 128);
    }
    fclose($fp);

    $result = explode("\r\n\r\n", $result, 2);
    $header = isset($result[0]) ? $result[0] : '';
    $content = isset($result[1]) ? $result[1] : '';
    return $content;
}


print PostRequest("http://st-flowershop.appspot.com/purchase", $_POST);
?>

Conclusion

I hope this post is enough to at least make you interested in checking out the Google App Engine and help you get over a couple of the initial hurdles. I’ll definitely be using it in any future projects.

This post is part of iDevBlogADay, a group of indie iPhone development blogs featuring two posts per day. You can keep up with iDevBlogADay through the web site, RSS feed, or Twitter.