in Game tech

Analytics For iOS Games

Unlike a lot of console and PC games, most mobile and web games keep evolving over time [1]. It’s up to a game’s designers to ultimately decide how to change and improve the game, but the more data about players’ habits they have, the more informed a decision they’ll be able to make. Having good analytics on iOS games is simply essential these days.Recording particular events as part of the analytics is only part of it. The most important part is how that data is presented to the developer. Having tables with millions of entries does me no good, and as a busy indie developer, I can’t afford to spend hours writing scripts to analyze it. I want something that allows me to easily visualize the data and makes sense out of it at a glance.

One possible snagging point about analytics is that Apple was cracking down on some applications with analytics enabled a while back. Specifically, the iOS developer agreement states:

3.3.9 You and Your Applications may not collect user or device data without prior user consent, and then only to provide a service or function that is directly relevant to the use of the Application, or to serve advertising. You may not use analytics software in Your Application to collect and send device data to a third party.

Where does that leave us? Reading it carefully, it seems that the restrictions are limited to “user or device data”. I’m interpreting that to mean things like UDIDs and emails, not anonymous player usage data (how long did it take to reach level 5, how long are play sessions, etc), so I think we’re clear there.

The puzzling thing is that a lot (all?) of the third party analytics libraries do report device information, like what kind of hardware or iOS version is running. That is extremely important information that developers really benefit from knowing, but it seems to go against point 3.3.9. Maybe “device data” only applies to information about that specific device? (as in, the UDID that is now going away). I hope so.

Not having analytics isn’t really an option. Unless you make a game that you plan to throw on the App Store, never touch again, and hope for the best, you’re flying blind without analytics.

What are some of the options we have then?

Home brewed

If you read this blog regularly, you probably know that I’m a low-level, do-it-myself kind of guy, with a deep mistrust and suspicion of middleware. So you would think that I would want to write my own analytics package. After all, how hard can it be? Collect the data you want and ping your server with it. If you get fancy, you can even use a scalable server back end like AWS or GAE. Done.

Not so fast. To do that well, it’s a lot more involved than that. You want to batch when you send out the information, and you might want to distinguish between WiFi and 3G connection (to avoid causing extra data usage for players on a limited data plan).

That in itself is not even that bad. The real pain comes in visualizing that data, and that’s where you can easily sink in days or weeks, and you would still not have something as good as some of the other alternatives.

The other drawback is that if you have some successful applications, you may be generating Terabytes of data per day. Think about the storage and bandwidth costs for that. Yes, I know that sounds insane, but Playfish reported generating that much analytics data at a GDC 2011 talk (video for paid GDC Vault members).

Flurry

Flurry Analytics appears to be the most common analytics package out there among my indie iOS dev friends. It’s free and it’s easy to integrate. The visualization page is pretty good, and it even offers some fancy features beyond session length and events, such as flow through the application.

So what’s not to like? I was never able to make heads or tails of the application flow. When you get at that level, you start needing to spend some serious time making sense of data, which is what I don’t have as an indie. The web page to visualize the data is written in Flash, so for those of you using an iPad to check it, it’s not a good option. Update: Flurry apparently added a Flash-free web page since I last looked at it a few months back. Thanks Charilaos Kalogirou for the tip.

The killer deal for me was bloat. Adding the Flurry analytics library to an app increased the executable size by 500KB. I’m sorry, but that’s completely unacceptable for me. Memory is tight, and half a MB is very significant. I would rather add another large texture or another music track. And if you think about it, why does it need 500KB to buffer and send some events? That’s simply ridiculous.

Google Analytics

I never actually tried out Google Analytics. They have an iOS SDK and it integrates quite well into the web page Google Analytics environment. The main drawback I heard from other developers is that it’s designed more for web pages rather than for apps and games, so it wasn’t a perfect fit.

Anyone who used it want to expand on this in the comments?

Localytics

Localytics is a relative newcomer to the iOS analytics field, but it was love at first sight for me.

It has the same ease of integration of Flurry, and it provides very similar functionality. Localytics, however, is completely open source, so instead of a black box library, you get the source code and you can add it directly to your game. How much does it increase executable size? 4KB! You have to wonder what the other 496KB were for in the Flurry library.

As a bonus, their visualization web page works great on an iPad, although it can be a bit slow for very large data sets sometimes.

One of their biggest selling points is that they report the analytics in real time, but I really don’t care one way or another. Waiting 12 or 24 hours to see the analytics doesn’t bother me one bit.

Unlike Flurry, you can only add strings as parameters to events. That works fine if I have a set of discrete options. For example, when someone sends a bouquet in Flower Garden, I can send a “bouquet sent” event with a parameters that is “email”, “facebook”, or “sms”. As a result, I can see a nice pie chart with the breakdown of how people are sending bouquets. Very useful stuff!

But how about things that don’t have discrete options? For example, in Casey’s Contraptions, we wanted to see how long players take to solve each level. It turns out you can’t have a number as a parameter, but you can easily get around that by discretizing it yourself, which in the end, is easier to visualize. So when we send the LevelXXFinished event, we look at how long the player took to finish it, and we break it down into ranges: under 30s, 30s-1min, 1min-2min, etc.

This is what the report looks like for one of Casey’s Contraptions levels:

Piechart

It looks like a fairly balanced level with the majority of the people spending under 3 minutes to solve it.

Random tips

A couple random things we learned along the way about analytics:

  • Less is more. Start with just a few events and go from there. If you have tons of data, you might never have the time to look at it.
  • Use analytics during playtesting. One thing is what people tell you, and another thing is what they really do. Since most of our playtesting is done remotely and we can’t observe as people play (which is invaluable), we can at least gather some hard data about it.
  • Turn off analytics reporting in debug mode. Trust me on that one.

How about you? What’s your favorite analytics package and why?

[1] For example, Flower Garden has been on the App Store for almost two and a half years, and it has changed radically in that time!

  • wow, never occurred to me that the analytics lib could be so huge. I’ve been using Google Analytics since I already am pretty familiar with using it for websites.

    Was shocked when I checked to find that the google analytics iOS lib weighs in at a massive 934KB! What’s going on in there?

    • All I can say is…. o_O

  • Flurry *does* work on iPad as it serves a flash-free version for devices not supporting flash. I use it every day with no problem from my iPad!

    • Thanks for the heads up. They didn’t a few months ago, but it’s good to know. I updated the paragraph.

  • We’ve just started using at Apsalar.com, which is by far the most sophisticated for actionable analytics, as opposed to just vanity metrics. Their conversion funnels are excellent. And the price is great (assuming they don’t run out of cash). We still do logging to urls that we crunch offline with scripts. Super low tech, super small impact at build and run time, but very powerful as we can look at any correlation we like.

    • Very interesting, Mark. I’d love to read a full post on what you’re tracking and how it helps. (hint, hint) 🙂

  • For me – Localytics doesn’t provide enough reporting options, so I chose a middle way. I’ve used their SDK, but built my own server running PHP+MySQL which handles numbers as parameteres and a number of other cool things.

    Oh, and I discovered a nasty bug in Localytics SDK while I was working on my own server. So nasty that I still can’t believe it’s real – the SDK actually send stats only on the app start. The code that you type in at “willEnterBackground” doesn’t actually work. So – these users that install app and play it only once, get disappointed and uninstall the app? You won’t get information about them in your reports!

    The bug was there 3 months ago, I don’t know whether they fix it, but if you plan to use Localytics, double check that thing (add tracing code at the end of the upload thread and check whether it’s being actually executed – in my case iOS killed the app before it sent anything to the server).

    • I thought that was the intended behavior: Only send data at startup. That’s not that uncommon. You certainly don’t want to be doing stuff when the user exits your app, so your other alternative is doing it while the app is running, which needs to be done very carefully.

    • Hi Tomasz,

      Localytics provides an upload call which can be used at any time. The default behavior is to upload at app start (immediately after the session is started) and then again when the app is exiting. On iOS4 it is possible to perform all closing uploads in the background (guaranteeing their success) using beginBackgroundTaskWithExpirationHandler.

      Here is a helpful little code snippet that was provided:

      (void)startBackgroundTask {
      if ([[UIDevice currentDevice] isMultitaskingSupported] &&
      _backgroundRecordingID == UIBackgroundTaskInvalid) {
      _backgroundRecordingID = [[UIApplication sharedApplication] beginBackgroundTaskWithExpirationHandler:^{
      [[UIApplication sharedApplication] endBackgroundTask:_backgroundRecordingID];
      _backgroundRecordingID = UIBackgroundTaskInvalid;
      }];
      }
      }

      Calling that function right before the upload and then closing the task in [self complete] by doing:
      if (_backgroundRecordingID != UIBackgroundTaskInvalid) {
      [[UIApplication sharedApplication]
      endBackgroundTask:_backgroundRecordingID];
      _backgroundRecordingID = UIBackgroundTaskInvalid;
      }

  • Bill Nalen

    Kind of related, but are you using something like AppViz to measure app sales and downloads? Seems like a lot of data needs to be collected, sales, downloads, usage, in app purchases, etc. I wonder if anyone has an all in one solution.

    • I’m a fan of AppViz because it’s a local client. It only deals with sales, but it seems different enough from usage that I’m fine with that. AppFigures is another good option if you prefer a web app.

      If you want something to download the data automatically for you on a cro job, appdailysales is great (http://code.google.com/p/appdailysales/)

  • Another vote for Apsalar. We liked it so much we integrated it into our Moai SDK.

  • Hi, thanks for the great comparison.

    As to Google Analytics, I have been using it since my first iPhone game. You are right, it’s mostly for web. But recently Google has made a lot changes for iOS devices. Overall I think it’s pretty good. Also, I like real time data. Maybe just because I’m an impatient person. 🙂

    I wrote a blog a while ago about how I use GA, which is also part of iDevBlogADay. You can find it here: http://www.clingmarks.com/?p=473 .

  • Hendrik

    By Apple’s definition “device data” includes not only things like UDID and OS version but also the extremely general “application usage data”. In other words: all analytics.
    So by the dev agreement you have to get the user’s permission to report any kind of analytics (except for advertising).

  • Hendrik

    Actually, reading 3.3.9 and Apple’s definition of “device data” again it means that analytics are just completely verboten. Even asking users prior to data collection doesn’t make it ok according to Apple’s rules.

    Hopefully Apple will “calm down from being pissed off” soon: http://allthingsd.com/20100601/d8-video-steve-jobs-explains-his-iads-restrictions-and-blames-flurry

  • Jason Bakker

    I’ve started using Localytics and it seems pretty sweet!

    I was wondering whether Noel or anyone knew – is there any way to set up an event to “replace” for a particular user instead of “append”? It doesn’t seem like there is, which is a bit of a shame – otherwise we could be tracking things like “maximum scores” or “maximum time spent on level”.

    Hopefully they will look into integrating that feature soon 🙂

    • I don’t think most analytics packets have that kind of functionality because they’re not organized around individual users as much as around sessions. So it’s more of a fire and forget approach.

      However, you can cobble something up yourself. For example, every time they finish a level, you could send the score range, and the high score range. It won’t be exactly what you want, but it’ll give you a very rough idea of where people fall as far as high scores.

      • Jason Bakker

        Ah okay, that makes sense. Thanks Noel!

  • Ryan Maloney

    What’s your opinion on TestFlight Live and what they have to offer with regards to analytics?  I haven’t used any analytics yet because I’m just getting into iOS dev, but they seem like they have a pretty complete package.