I’m taking a couple of days to upgrade some of my libraries for doing prototyping both in 2D and 3D. One of the many overdue things I wanted to do, was to finally ditch OpenGL ES 1.1 and move to 2.0 exclusively. Yes, even if you’re only doing a 2D game, OpenGL ES 2.0 is way worth it.
There were even a couple of cases during Casey’s Contraptions that we wanted a particular effect, and couldn’t get it quite right, but it would have been trivial to whip up a shader if we had been using OpenGL ES 2.0. In the end, we had to resort to texture combiners (yuck), and it wasn’t exactly what we had in mind.OpenGL ES 2.0 requires that you write shaders for any different kind of “thing” you render, whether it’s a simple 2D quad, or a fancy skinned character. I wanted to provide at least a basic 2D rendering shader in my graphics library, but the workflow was a bit awkward.
First of all, shaders on iOS are compiled at runtime [1], so we need to access the shader source and compile it. Initially I was thinking of having the shader source files as text in the graphics library, but for the game to access those files, it would have to explicitly add them to its resources so they can be copied during the build phase. That’s error prone (I would always forget to do that with every new project) and just plain ugly.
Another possibility would be to include the source text as a string in the library. That would simplify things, but it would mean that:
- You would lose shader syntax highlighting.
- You would need to have the annoying \ at the end of every line.
After asking on Twitter and getting some good suggestions, I decided the easiest way would be to use xxd. It’s a shell tool that comes preinstalled in all Macs (and Linux) that takes a file and converts it into a C array that can be included in a program directly.
I liked xxd even better than the old standby of bin2c because it already comes installed, and it generates the whole file, not just an array that then needs to be included from another file. Yes, it would be easy enough to modify, but why bother when you have a perfect tool for the job already?
Now, I can set up a custom rule in Xcode 4 like this:
Update: The array generated by xxd isn’t null-terminated, so we need to add a zero to the array. The easiest way is to use sed. Thanks to SixEcho Studios for the heads up. Also, I updated it to use the varaibles INPUT_FILE_DIR and INPUT_FILE_NAME thanks to Miguel’s comments.
Now, whenever the shader source changes, it generates a header file like this:
unsigned char PassThrough_fsh[] = {
0x0a, 0x0a, 0x76, 0x61, 0x72, 0x79, 0x69, 0x6e, 0x67, 0x20, 0x6c, 0x6f,
0x77, 0x70, 0x20, 0x76, 0x65, 0x63, 0x34, 0x20, 0x44, 0x65, 0x73, 0x74,
0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x43, 0x6f, 0x6c, 0x6f, 0x72,
0x3b, 0x0a, 0x0a, 0x76, 0x6f, 0x69, 0x64, 0x20, 0x6d, 0x61, 0x69, 0x6e,
0x28, 0x29, 0x0a, 0x7b, 0x0a, 0x20, 0x20, 0x20, 0x20, 0x67, 0x6c, 0x5f,
0x46, 0x72, 0x61, 0x67, 0x43, 0x6f, 0x6c, 0x6f, 0x72, 0x20, 0x3d, 0x20,
0x44, 0x65, 0x73, 0x74, 0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x43,
0x6f, 0x6c, 0x6f, 0x72, 0x3b, 0x0a, 0x7d
,0x00};
unsigned int PassThrough_fsh_len = 91;
To compile that shader in the game, all I have to add is two lines:
#include//... const int fs = ShaderUtils::CompileShader(PassThrough_fsh, GL_FRAGMENT_SHADER);
Done! Clean, simple, and minimal cruft.
Update: I haven’t tried it yet because I haven’t had the need for any expensive shaders, but I hear really good things about Aras PranckeviÄius GLSL optimizer. In that case, I would add it before the xxd step. You would still use the shader the same way, but it would be optimized and perform faster.
[1] That’s a huge mistake and I hope Apple fixes that ASAP. Loading times for any amount of non-trivial shaders adds up really quickly. That’s the last thing you want on a mobile device! The OpenGL ES 2.0 standard provides an optional way to load binary shaders, and all that Apple has to do is provide a tool that will compile shaders for the iOS hardware. To be completely future-proof, you they can even mandate that if the binary load fails, you still need to compile from source.
I dont like very much this method, because it implies that ALL your shader’s code will be loaded in ram always, and you will not able to free that. By this way you will end with two source code copies of the shader code: the one you declared and the one the driver keeps in its memory. Sometimes it doesn’t matter, but sometimes you need all bits of extra memory you could get from the system 🙂
That’s true. This is one of those cases where I’ll choose convenience over saving a few KB. But it’s true that it can be an isssue if you have lots and lots of shaders.
It doesn’t imply that at all. Typically in a modern OS, the application binary is memory mapped as a copy-on-write area which is backed by disk. Thus it’s loaded on demand and can be dropped by the kernel at any time at essentially no cost. We’re talking about a fairly small chunk stored in the data segment, which will never be written and is thus essentially free.
I guess we use the ‘unsigned int PassThrough_fsh_len = 91;’Â to allocate the data in heap, and free whenever done
Just a quick note: you can use INPUT_FILE_DIR (no filename part), INPUT_FILE_NAME (filename with extension), or INPUT_FILE_BASE (filename without extension) in the script. Cheers! 🙂
Oh cool. Where did you find that? I looked and looked and wasn’t able to find any good docs on that.
$(something) is the result of the command “something” and ${name} is the same as $name for bash/sh variables which its replaced by the content of the variable. for example LS_RESULT=$(ls); echo $LS_RESULT, or echo ${LS_RESULT}
BTW, great tool xxd, I had a python and lua scripts to do the job to hexify static resources, now I know a much direct way 🙂
How do you handle the requirement that the shader sources are 0x00 terminated?
You’re right! I had it that way originally because I was generating a string, which was already zero-terminated. Using sed seems to be the easiest way to fix that. Updating the screenshot right now. Thanks for the heads up.
You might want to have a look at this GLSL compiler I wrote:
http://community.spinxengine.com/blogs/jarkkol/19-glsl_shader_compiler.html
It’s a Windows exe, but there’s also source code available for it.
I have the compiler integrated in MSVC as a custom build step for glsl files so it’s very convenient to use. It outputs a header file which you can #include to your code, and also properly formatted messages upon errors, so you can jump to the offending line easily by hitting F4 in MSVC (just like for c++ code). I guess there’s something like that in xcode as well. And unlike GLSL standard, it also supports #includes directives, which are pretty useful once you get more complex shaders.
Nice technique, Noel. Â Thanks for the tip!
It’s fact if you have lots of shaders then it can be definitely issue for you.
Thanks
Hi Noel,
I’m a fan of your presentation on combining OpenGL and UIKit. At the end of that document,
you mention there’s some sample code on your website.
Does said code exist? I’d love to have a look at it.
Kind regards,
-joshua
Very cool very useful, thanks for sharing
Here’s the contents of that script in an easier, copy-and-pasteable form:
cd ${INPUT_FILE_DIR}
xxd -i ${INPUT_FILE_NAME} | sed s/};/,0x00};/ > ${INPUT_FILE_PATH}.h
Also, don’t forget to add the *.vsh and *.fsh files to your project, first!
UPDATED IMPLEMENTATION (7/30):
On iOS 6.x with GLES 2.0, glShaderSource takes 4 arguments.
I’m using glShaderSource, due to this example to load the shaders into GL:
http://developer.apple.com/library/ios/#documentation/3DDrawing/Conceptual/OpenGLES_ProgrammingGuide/BestPracticesforShaders/BestPracticesforShaders.html#//apple_ref/doc/uid/TP40008793-CH7-SW3
glShaderSource requires 4 arguments with some tricky syntax.
First, it expects const chars. So, I modified the Build Script so that the strings are rendered as const chars, not unsigned chars:
cd ${INPUT_FILE_DIR}
xxd -i ${INPUT_FILE_NAME} | sed s/};/,0x00};/ | sed s/unsigned/const/g > ${INPUT_FILE_PATH}.h
(NOTE: This makes all of the arrays ‘const’ now!)
Next, I used this syntax for glShaderSource:
// foo.fsh.h (Build Script generated code)
const char foo_fsh[] = {
0xDE, 0xAD, 0xBE, 0xEF
, 0x00};
// XXMyOwn.m
const GLchar *foo_string[] = {foo_fsh};
glShaderSource(shader, 1, foo_string, NULL);
(I haven’t tested this in GL yet, but this is correct syntax according to the compiler, FWIW.)