Play a game on a device, put it down, pick up another device, and continue playing exactly where you left off. This is the future of games.
That future is a reality today for some games and apps (Netflix, Kindle), and I’m convinced that players will expect that in most games in the next year or so. So obviously, the next bit of new iOS tech I decided to try was iCloud. I would love to turn Flower Garden into that kind of seamless experience, independently of the device you use to access it.
As a quick spoiler, it turns out I won’t be able to make Flower Garden quite so seamless without a lot of extra work. But I learned a lot along the way and I should be able to take a small step in that direction.
Design considerations
iCloud promises to be the “available from everywhere” storage solution that will be a key component towards the scenario of playing on any device at any time. Unfortunately, it’s just a component of that whole scenario and still requires quite a bit of work on the part of the developer.
Since I’m just retrofitting Flower Garden to work with iCloud, I wanted a simple way to simply replicate the game state on iCloud. Then, whenever you play it from any device, you get the latest state and everything works as expected.
The big problem with that approach is that you can’t always count on having an active (or fast enough) internet connection. iPod Touches, iPads, or even iPhone in a plane or a cell tower black spot will make it so your device has no way to communicate with iCloud. So what happens if the player plays without an internet connection, then comes home, grabs another device, plays for a bit with internet connection, and finally launches the original device, this time with internet connection. You’ll end up with two conflicting game states. Not fun!
I can think of two different strategies to deal with this situation:
- Require internet connection to play. Some games do this today. It can be quite frustrating not being able to play your favorite game in the plane, but it does solve the game state conflict problem because they can always communicate with iCloud, so they usually provide seamless multi-device play.
- Detect changes on both game states and intelligently solve them. Sounds good in theory, but it’s a pain in practice. Unless you have a very simple game state (a percentage completion for example), it means not only do you have to record the state, but you have to record the events that led to that state, and be ready to examine them, compare them, and marge them in the case of a conflict. And the worst part is that even if you put lots of care into it, there will be cases where the merge is not ideal and the player will feel like he lost something in the merge.
For Flower Garden, neither solution is particularly attractive. It’s the proverbial rock and a hard place. So for the moment I’ve decided to implement just a small step in that direction: Sync things that only progress in one direction, without any danger of conflict. That includes purchased IAPs and unlocked seeds. Later on, once that’s working solidly without any problems, I can consider adding counts like Fertilizer, Color Dust, or Green Thumb points. That will be a bit trickier because there’s still chance for conflict since the count can go up and down, but it’s possible to change each amount into two different values that always go up: Earned amount, and spent amount. That way I can always grab the maximum value that I find either local or on iCloud and things should work smoothly. (Update: What I wrote there about separating them in two different values is totally wrong, and any DB programmer worth his salt would be laughing at that. Separating them in two different values won’t help at all. That’s what I get for doing brain-dump posts on the fly 🙂
Apple provides two APIs to store data in iCloud: key-value storage and file storage.
Key Value storage
Initially, this seems like a great way to go. It’s a very simple API, and it’s very familiar to most iOS programmers because it’s just like NSUserDefaults.
Don’t be fooled into thinking you need to store data in strings or one field at the time. You can stuff whatever binary chunk of data you want with NSData, so it’s quite versatile.
It does have three pretty big drawbacks thought:
- Size. You can only store up to 64KB of data. That’s definitely a big deal if you’re planning on storing large data files. Even for Flower Garden, each pot is about 30KB, so I wouldn’t be able to save the full garden state this way.
- Syncing. The Apple docs say that “Keys and values are transferred to and from iCloud at periodic intervals.” Ouch! What does that mean? Clearly this is intended for non-crucial data (like settings), so the potential delay isn’t a big deal. In my tests, I found that data was often not available when starting the app, but would become available a few seconds later.
- No way to check state. When starting the app, there’s no way to find out if the values you’re reading are up to date. Combined with the slow syncing, it makes it less than idea for important data. On the flip side, you can be notified when the data changes, but that means having to deal with changes while the game is running (which I was hoping not to do).
File storage
File storage might be a better option then since it doesn’t have any of those drawbacks: there’s no size limit, data is synced much more aggressively, and you can check if the data is up-to-date, and wait until it is otherwise.
What’s not to like about file storage then? The cumbersome and intrusive API that Apple created around it. If you read the docs, they make it sound like you need to inherit from UIDocument and load all your data through that class. I like to keep things simple and portable, so I’d really rather not introduce UIDocument into the file-loading process if I don’t have to.
It turns out that Apple’s docs on the iCloud SDK are quite sparse and lacking details. I was able to put together a demo from those docs and some experimentation, so hopefully this will be useful to other devs as well.
The device has an iCloud storage daemon running, monitoring specific directories for changes. You get the directory assigned to your app by calling URLForUbiquityContainerIdentifier.
To add a new file, the docs recommend first creating the file somewhere else, and then calling NSFileManager:setUbiquitous:itemAtURL:destinationURL:error: to move it to iCloud. Interestingly, I accidentally skipped this step and just wrote to the iCloud directory directly, and the files were stored correctly anyway. Might as well leave it just in case that behavior changes later on, or there’s some side effect I didn’t notice.
After that, every time you write to the file, the iCloud daemon will detect the changes and push them out to iCloud. This is an important point because a) You don’t have explicit control to say “start send it now” or even “This file is ready to be sent to iCloud” (that would be my choice), and b) I don’t know what it does with partial updates, so I would be very careful about writing to those files and make sure it’s an atomic operation (save somewhere else, and them move the file in one operation).
To get the latest version of the files on iCloud, you can check whether they’re fully downloaded or not by calling NSURL:getResourceValue:&isDownloaded forKey:NSURLUbiquitousItemIsDownloadedKey error:. If they are up to date, you can move on, otherwise, you can initiate a download of the files by calling NSFileManager:startDownloadingUbiquitousItemAtURL:error:.
One weird thing is that the startDownloading function doesn’t have a callback (that I can see). So I had to set up a timer to check periodically if the files are synced.
Since all the syncing happens at startup, there’s no complexity involved with data changing while the app is running. If that’s a case you need to handle, you might be better off using UIDocument since it at least detects conflicts. If you do it this way, the latest one will overwrite any past changes.
Also, working this way, it seemed easier to keep the files in Application Support (or Documents), and only move them to/from iCloud when I wanted to. That has the advantage of iCloud not changing things from under you while the game is running, and the fact that, if for some reason the files in iCloud are corrupt, you always have good local files to fall back to.
Demo
The demo iCloudTest saves data both with the key value pair, and the file storage system directly like I described above. If you run it from different devices, you’ll see that it’s amazing how quickly file data gets propagated, but key-value data takes a few more seconds.
The demo project also works in the simulator (there’s no iCloud support, so it just uses local files), and it even works with iOS4 (also using local files, and it avoids using any iOS5 symbols while detecting iCloud support).
Sharing Data Between Multiple Apps
Once you’re at this point, sharing data between multiple apps is really straightforward.
First you need to make sure the app IDs of both apps start with the seam Team ID. For some unknown reason lost in the midsts of time, Flower Garden and Flower Garden Free use different prefixes, so that option is out for me. Hopefully most people having been using the Team ID only.
Then you need to decide which type of data you want to share. If you want to share files, you need to add the app ID of the other app to your iCloud entitlement, and then as for the correct app ID (including prefix!) when you call URLForUbiquityContainerIdentifier. If you just want the app’s own iCloud data, you can pass NULL as the parameter.
As far as I can tell, each application can only have one set of key-value data. So sharing it between two apps means changing one app to use the app ID of the other app in the com.snappytouch.icloudtest field of the iCloud entitlements file. As long as both apps use the same Team ID prefix, it should work fine without having to do anything else.
Perfect timing! I was just looking for the best way to save game progress (i.e. # of missions completed) between different devices, and more importantly, between the upcoming free version of my game and the existing paid version.
“If you want to share files, you need to add the app ID of the other app to your iCloud entitlement”
Unfortunately, that doesn’t work. This will surely (hopefully?) be fixed in iOS 5.1 but as of iOS 5.0.1, apps only use the first app ID you specify in the iCloud containers list. If you want to share files between apps, just nominate one app ID to be used as your one and only iCloud container.
(Yes, I have apps in the store that do this.)
Interesting. I had only tried it with one app ID like you said. So I just went back and trying adding a second one and asking for it explicitly, and it all seems to work fine. (using iOS 5.0.1). What problems did you have?
Make sure you’re passing the *full* appID. So @”XXXXXX.com.yourcompany.appname” or something like that. Where XXXXX is your teamID and it needs to be common between the two apps. Both of them also need to have been registered in the app list in the provisioning web site and have iCloud enabled.
Yep, did you try and open a file belonging to the other app? That’s where it breaks down. I could find them using NSMetadataQuery, but I couldn’t open them. At least, I couldn’t open them using UIDocument’s openWithCompletionHandler. When I did that, I got a permission error.
I just tested that and it worked fine. I have iCloudTest2, which is just a copy of iCloudTest with new app ID and new provisioning profiles. Added both XXXXXX.com.snappytouch.icloudtest and XXXXXX.com.snappytouch.icloudtest2 to the file storage entitlements, and I’m able to request files from either one. So it sounds like what you’re describing is a bug in UIDocument. It would be so much better if Apple provided the full source code to at least high-level classes like that one!
Did you try NSMetadataQuery to observe changes in the files instead of polling yourself? I have a setup similar to yours, with the difference that I’m using NSMetadataQuery; however, even though it works perfectly in iOS I’m having trouble under Mac OS: some times the query returns no results for apparently no reason. I wonder if that’s a bug in Max OS X or I’m using it the wrong way.
No I didn’t try it because I didn’t know about that function (and it wasn’t referenced in the iCloud docs). Interesting. Thanks! I should give it a try.
Thanks for the interesting analysis! 🙂
“So what happens if the player plays without an internet connection, then
comes home, grabs another device, plays for a bit with internet
connection, and finally launches the original device, this time with
internet connection. You’ll end up with two conflicting game states. Not
fun!”
This is a pretty fundamental concern with cloud storage for games it seems. We had fully implemented Steam Cloud for our game and – at the last minute – decided to not include the feature due to this possibility, which we saw occur in testing. I’m curious: did you actually see this occur?
In general keeping your own separate “live” copies and backups seems like good practice.
Thanks again.
You’re welcome, Eddy. I haven’t actually released this yet, so I didn’t see it “in the wild”. However, when you have tons of active users, you know a bunch of people are going to run into it. I’d rather err in the side of caution and don’t provide a feature than be flooded with angry users that lost their data.
I agree it’s a very fundamental problem. I think the way to solve it is to look at what databases have been doing for ages and implement some kind of DB-like interface on the game state (not just with binary blobs, but with the actions of how you got there).ÂÂ
Totally agreed with you about erring on the side of caution! It’s not worth adding a nice-to-have feature when it’ll likely cause some players a lot of frustration.
I like your proposal of using DB-like tracking: forget sharing progress files themselves and just share a copy of a transaction file. Every time it gets updated by the cloud, compare it to your local copy and merge in any missing transactions, like level & achievement unlocks, gains and losses of expendable items, etc.
Interesting stuff! 🙂
The offline situation and synchronizing state is indeed a dilemma.  Some ideas:
a) If the app is launched with no network available, tell the user that this play session will not be posted as a result and ask if they want to continue (maybe offer to let them turn on the network by opening settings).
b) if the app launches with a network, update the server to say “a game is being played on device XYZ” and clear this flag when the app goes into the background and updates the game state.
c) if the app launches and notices a pending game being played it then checks to see if it’s running on the same device; if so, use device data as most recent, if not, can tell the user there are unsaved changes from a session on XYZ device, do you want to continue and lose those changes or launch the game on that device first to save them?
The idea here is to keep/track enough state information that you can tell the user what the situation is and ask them how they’d like to proceed.  Obviously, this only applies to data you can’t resolve yourself since we don’t want to bother the user unless we really have to.
The problem with that approach is that you’re putting the responsibility on the user’s hands. That might work if the app is aimed a geek users/developers, but not for a casual player. What would your parents/grandparents say when presented with a message like that telling them that the ‘game state might not be posted” or whatever? Most of them will just ignore it (after being intimidated by it) and then complain that their game state is out of sync. Definitely not a good solution for me. I feel it has to be easier transparent and work 100% of the time, or not be there.
Hi,
I try to download a file from iCloud to my local device with:
[fileMgr startDownloadingUbiquitousItemAtURL:fileURL error:nil];
But where is the file after download is completed ? I don’t find it on the local device
Thanks
Mirco
In regards to conflicting game state between multiple, potentially disconnected clients, it seems like this would be an ideal situation to make use of an event-oriented data model. ÂÂ
Instead of storing game state, simply store the actions that the user has taken during a particular session.  i.e.:
DateTime        ActionTaken               Context
————–        ——————               —————–
1/3/2012 2:00pm   Purchased 8 Fertilizers       iPhone (Online) sees 8 fertilizers available
1/3/2012 3:01pm   Used 1 Fertilizer on Flower A   iPad (Online, but goes offline immediately before the actual action) sees 7 available
1/3/2012 3:02pm   Used 7 Fertilizer on Flower B   iPhone (Online)  sees 1 available, as the iPad event has not yet been uploaded
1/3/2012 4:00pm   User 2 Fertilizer on Flower A   iPad (Online, has re-connected and sent these events) sees -2 available
The context data is not actually stored, but it helps when thinking about this.
When a device is able to connect, it creates a snapshot of the game state based upon the all actions taken.The user can then play, potentially going offline. ÂÂ
Next time the device can connect, it then uploads all the actions taken during the play session.During that time, another device could potentially connect and perform some actions.
This doesn’t obviate the need for state collision detection, but the upshot is that any particular game state is always the result of player actions taken. ÂÂ
In the example above, it is possible that the user could, after the last event, be at -2 fertilizers.  There are a few options for handling this.
1)  -2 is a valid count, and the user now has to buy 3 fertilizers in order to get 1.  (technically fair, but could anger users)
2)  Give the user a pass.  (makes the user happy, but open to abuse)
3)  Present the user with the 3 different event streams (iPhone only, iPad only, or both) and have them choose.  (Also fair, but could be difficult to present to the user)
4)  Only present the user with resolution options when there is a non-optimal state (i.e. Would you like to include the final event or not?)
There are other options for resolution as well, but ultimately the point is that whatever state the data is in, it is the result of the user’s actions.  The calculated snapshot at any point may be non-optimal, but it is always technically valid and leaves room for various methods of resolution that can be understood by the user.
Daniel
Great analysis if using iCloud for syncing state, Noel. And of course there are multiple pitfalls of using iCloud as well. You might want to check out something like http://cloudmine.me (disclaimer: I’m a founder) that will allow you to sync across platforms and apps very easily AND allow you to perform conflict resolution on the server-side if you wish.
I love flower garden. Is there a possibility of adding small animals and other insects. I would love to see a rabbit, chipmunk or squirrel in the future of this game.