One of the unintended side effects of my previous post on the horrible memory situation on the iPhone, was that some people pointed out it was possible to hook up a disk storage back end to the iPhone’s virtual memory system. That’s quite ironic because I think of it as compounding the already dismal situation rather than a solution or a even a stopgap measure.
As far as I’m concerned, virtual memory paging is the lazy man’s caching scheme. There, I’ve said it. Now let me qualify it a bit and justify why I feel that way.
A lot of applications, and especially games, can’t keep in memory every bit of data they need for their execution. There’s just too much data and not enough RAM to keep all the levels, all the textures, all the characters, and all the movies. Even the thought of keeping everything in memory is ridiculous. Instead, games are architected to strike a balance between memory usage and responsive interfaces. Usually that means loading levels, or parts of levels on demand, and keeping them in memory while they can be needed. Some other times it means loading levels of detail for textures, meshes, and animations depending on the player position. The point is, nobody knows as well as the game itself what needs to be in memory, what can be unloaded, and when the best time to do it is.
A different approach would be to ignore thinking about managing memory as a scarce resource and use as much of it as we need. If we ever go over the amount of available physical memory, the virtual memory system will kick in and page out memory to disk to make room for the memory we’ve requested. But there lies the problem: The virtual memory system is a lower-level system that knows absolutely nothing about our game or application. It can only make guesses about what memory is OK to evict and when it’s a good time to do it. If you’re unlucky it will choose to page out memory when you need performance the most, and it might evict a page that you’re about to use in a few milliseconds. If that happens, you can kiss your performance bye-bye.
Because of that, virtual memory paging will always do a much worse job than we could have done ourselves. But hey, it takes no effort or thinking on the part of the programmers, hence the “lazy man’s caching scheme” I was referring to earlier.
However, virtual memory paging is sometimes a necessary evil. The assumption running through the previous three paragraphs is that we, as programmers of the game, can always manage the physical memory so our game runs efficiently on it. That is true for certain platforms, usually with fixed hardware specs, and ones with minimal or no background processes running. Game consoles are perfect examples of this: We know how much RAM we have to start, we know how much the system needs, how much video memory there is, and we can deal with the rest ourselves. We’ll decide what to load and when to load it.
On something like a modern PC (whether it’s Windows, Mac, Linux, or any other flavor), we have none of those guarantees. We don’t know how much memory we’re going to encounter when we run our game, and, what’s even worse, we have no idea how that available memory is going to change during the execution of the program . In a situation like that, we can plan for some minimum memory requirements, but we’re going to need that virtual memory paging to bail us out of tricky low-memory situations. It might sound like a good idea, but that’s one of the main reasons why PC games are often choppy and with inconsistent frame rates (buggy drivers being the other main reason).
Finally coming back to the iPhone, how does virtual memory paging fit there? There’s no doubt that it would help because it would reduce the number of crashes due to programs unexpectedly running out of memory. But at the same time, it would cause most games and apps to be choppy and unresponsive, especially when it’s just launched. But the iPhone is mostly a fixed spec platform, with minimal background processes , so we can always do better than the dumb virtual memory system. Maybe that’s not a big deal if you’re writing an app that interfaces with a web site and writes data to a database, but it’s crucial to be able to write responsible games, which are considered near real-time apps.
On the iPhone there should be no need for virtual memory paging (not to mention that it would probably drain the battery a lot faster). Instead of solving the memory problem by throwing more and more complex systems at it, what we really need is a guaranteed amount of memory for our app to run and we can take care of the rest. But Apple needs to take the first step and set that memory aside.
We’re waiting, Apple.
 We could go totally hard core and allocate the memory we need and then mark it as not swappable by the virtual memory system. That wouldn’t do any good because if other processes start requesting memory and they run out, the virtual memory system will swap out other pages (and possibly cause other processes to thrash) and kill our performance as well. So things are pretty much out of our hands as soon as virtual memory paging comes into play.