EjectaGL - WebGL on iOS with Ejecta!

Summary: 

I am starting a project EjectaGL which adds WebGL binding to the awesome Ejecta project by Dominic Szablewski. EjectaGL extends the Ejecta framework to provide a WebGL implementation on vanilla JavascriptCore without any of the DOM overhead. EjectaGL provides a way to develop WebGL apps that can be shipped to the iOS app store(Not tested, but very possibly). It is far from being complete - So please contribute if you are interested! 

https://github.com/vikerman/EjectaGL 

EjectaGL is distributed in the same MIT license as Ejecta.

Longer story:

I had worked on Voxel Flyer for js13kgames - A WebGL based game in less than 13K compressed Javascript. I wanted to keep developing it but I sort of wished it would work on iOS too. A search for WebGL on iOS usually yields the jailbreak method for enabling WebGL on iOS which is not very useful if you want to ship your WebGL app on the App Store. It feels like Apple might open up WebGL at any time but that never seems to happen. I always wanted to look into adding WebGL binding to AppMobi's directCanvas - but that never happened due to the complexity of the directCanvas code(and not to mention that the open source version has hardly seen the updates of their recent XDK).

Meanwhile I saw that Dominic from Phobos Labs(on whose work directCanvas is actually based upon) released Ejecta, a newer version of his iOS library to provide a barebones fast implementation of Canvas and Audio without all the DOM overhead. This time I felt the code was much more friendlier and  I could actually have a real shot at implementing WebGL on top of Ejecta. 

So couple of days back I started reading my fist tutorial on Objective-C and started hacking Ejecta! The result is EjectaGL. EjectaGL replaces the onscreen 2D canvas with a WebGL enabled canvas. At the current stage it's just a proof of concept and I am discovering and fixing issues as I am porting each lesson from  http://learningwebgl.com/. I tried porting Voxel Flyer to it and discovered that there is something wrong with the way uniforms are being passed in which I'm yet to finish debugging.

So basically it's in a very nacent stage but things are looking mostly doable. I can definitely use some help from people to add more bindings, port more examples and review my code for memory leaks(I'm sure there are few there given my newness to Obejctive-C) and other issues.

Check out the screenshot of Voxel Flyer running in the emulator with buggy lighting turned off at an astounding 9 frames/second. (You can run this demo by downloading the github source, creating an App/ folder and copying the contents of examples/VoxelFlyer to it)

As for some implementation details the one major caveat is the absence of Typed arrays in the current JavascriptCore that Ejecta uses(It might be coming some time later though). I have tried to work around it by implementing a very primitive pseudo typed array in Objective-C. Basically you can instantiate a typed array but that's about it. I found that most WebGL code just create the typed Float32Array, Uint16Array just before binding to the buffer anyway and don' do much with the contents anyway. There could be issues with libraries like glMatrix that use typed arrays for fast matrix calculations. Currently I just replace the typed array with a regular array in my stripped down version of glMatrix. I see a workaround here of implementing glMatrix itself in Objective-C that doesn't have typed array in it's method interface but can use the internal representation of EjectaGL's typed array which can then be passed on to EjectaGL. A similar strategy can be adopted if we want to get libraries like three.js also running on EjectaGL(though that would be a big undertaking of it's own).

There are not lot of WebGL APIs implemented currently but I don't see any other major issues in doing that (other than the typed arrays). Also I am thinking of adding stuff like rendering offline to a 2D Canvas and using that a texture in WebGL (My Voxel Flyer originally had a mini map that was a 2D canvas on top of the WebGL canvas).

So for now I just wanted to quickly mention the project and get the ball rolling so that I can ask for help and also make sure there is no duplication of effort (I am pretty sure 100 different people were also thinking of a similar project).

I think there are lot of applications for this project - It would be nice to get all those nice shader/demos working on the mobile device, possibly even the three.js ones. I also see a possibility of using all the cool WebGL shader effects available on Construct2 to be now fully exportable to the iOS target. 

Drop me a line here or on twitter - @vikerman

Update : EjectaGL is now merged into Ejecta main! You can get it at https://github.com/phoboslab/Ejecta

GameDoc: On the fly Javascript game modding for the mobile web

I had a crazy idea and wanted to create a quick experiment to test it out. Mobile browsers are becoming good  - Javascript and Canvas implementation are becoming faster. There are certain advantages to the mobile browsers as a game distribution platform - the most important of which is that it is open and not controlled by arbitrary rules set by Apple, Google or whomever (Classic example - Phone Story). HTML5 games in general are picking up steam but game makers are ignoring the mobile browser.

There are two factors why games on mobile browsers don't work well -

1. Game developers ignore things like centering the screen on the game area, setting the right zoom level, allowing for touch input(or any alternate mechanism like accelerometer) when there is no keyboard
2. Browser makers don't provide a good way to do certain things  - locking screen orientation, a working audio API etc.

1) is definitely solvable while 2) involves letting platform creators know about the issues faced by game developers (Some influential developers that are already trying to do that - http://www.phoboslab.org/log/javascript). Maybe newer platforms like Mozilla OS will push the standards higher on the existing ones.

So the crazy idea is centered around the concept that HTML5 games being based on open standards should be modifiable by anyone (An idea floating around in Mozilla land) - In this specific case modified to work well on mobile browsers. If a game developer didn't take care to make the game work well on mobile browsers, anyone should be able to fix it.

To prove this point I created a proof of concept - which I called "GameDoc". For the experiment I took the game emptyblack.com. I really liked the game when I played it on the desktop and I wanted to play it on my phone. The game seems to run ok as far as the graphics is concerned but it had the classic issues mentioned in 1). Since there is no keyboard input in the phone the game is unplayable.

GameDoc is a native mobile App that basically wraps a browser WebView. It loads the game from the original URL but it sets the required zoom level on the WebView to get a good view of the game area and locks the screen orientation to landscape. And here is the cool part - It fixes the absence of keyboard by injecting Javascript snippet into the page that adds on-screen controls. When these controla are pressed keyboard events are simulated using the dispatchEvent mechanism. I used the WebView wrapper route to be able to do this Javascript injection on the device (due to absence of browser extensions on the mobile).

So here is the video of how the converted game works. The video shows how the game is not playable in it's original form in the mobile Chrome browser and how the fixed version works with simulated onscreen controls. It's not perfect - Sometimes keyup/keydown events seem to get missed and I think it might be due to multitouch issues. As you can see I really threw this together rather quickly with horrible looking icons for the controls - But you get the idea.

The grand idea is that there can be a crowd sourced HTML5 game portal where people can add their own customization to the game controls or the game itself and the Gamedoc App(or whatever better name I come up with) would pull that extra information for the game and does the necessary modding.

Can I avoid the need to have a separate App that wraps the WebView and just use the native browser? Maybe I can do something at the server side but that might screw up with things that are tied to the original domain like cookies or the local store. Maybe I can just do the controls with some sort of IME trickery.

Anyway as I said it's a crazy experiment. At the least I hope to push HTML5 game developers to think more about the mobile web.

Let me know of your thoughts/comments.

Update: Looks like IOS6 has better support for Audio opening up better game support! And the awesome folks who make Construct2 already support it - https://www.scirra.com/blog/98/the-web-as-the-platform

js13kgames and Voxel Shooter

I knew I had to take part in js13kgames - A competition to write Javascript games less than 13KB when zipped. I used HTML5 canvas for the last two of the last three game projects I worked on and was increasingly identifying myself as a HTML5 game programmer. However a visit back to India and the work on Mario Ball had left very little time for me to actually work on my 13KB game.

At 13KB I had decided to have some sort of procedural generation for the elements of the game. I also wanted to explore the Web Audio API for the sound. However I was having a very hard time trying to come up with a concrete idea for the game. My very first idea was a weird fighting game where the player's form and powers were determined by a random combination of 3 elements - Rectangles, Circles and Bezier curves. The player and enemies basically looked like modern art pictures. I wasn't sure whether I was going too weird with this one. Some of the denizens of this unfinished game:

Then this entry came out on js13kgames - 13th Knight. It was a fully textured 3D world in WebGL in under 13KB! And it had awesome looking trees!! I wanted to make a WebGL game now. I always wanted to learn WebGL but hadn't looked closely into it it before. I had started looking at three.js for 7dfps but never got the time to do anything beyond a sphere moving on plane. This time around three.js wasn't going to be an option because no external libraries were allowed and three.js would not fit into 13KB (I later learnt somebody else had managed to get a subset of three.js working in their 13KB game - Mindless).

So not exactly knowing what I'm going to build I started with the WebGL lessons at http://learningwebgl.com/blog/?page_id=1217 for the basic template for a WebGL program. I played around with it till I was able to draw a flat colored cube with directional lighting. There were 5 days till the competition ended and I had to come up with something quick. Then it struck me - I will just redraw the cubes a lot of times to create a Voxel engine! (Being completely ignorant of the issues before starting a project is sometimes a good thing). I was missing a good flying game these days and I thought I will do a flying game where you shoot enemies on the ground - Voxel Shooter!!

I generated the terrain using some random combination of sin curves. I copied a small subset of the glMatrix library for the matrix operations. And I happily went about drawing cubes over and over again - One drawElements call per cube. I could reach up to 50 x 20 array of cubes before the rendering became too slow. I tried removing faces from the cubes that are hidden behind other cubes - A small increase in the frame rate but nothing great. I tried some cheating like increasing the size of each cube so that I can get away with a smaller number of cubes. Nope - the map was still too small. I banged my head trying to figure out a frustum culling method but it just involved too much Math to be figured out in a small amount of time. Also the far plane was going to be too near and field of view was going to be too narrow if I can display only 1000 cubes. I just had to figure out a way to improve performance first.

Then I came across this - Google I/O 2011 : WebGL Techniques and Performance. I had basically made all the mistakes mentioned in the talk. First and foremost I was switching too much context for every vertex - Even the uniforms that were common to all vertices like lighting and camera and projection matrices. I got a substantial boost when I fixed this issue but still not going to allow the 256 x 256 map size I was aiming for. The next thing was to reduce the number of GL draw calls by combining many cubes into one giant mesh. This one I had some problem getting my head around because I hadn't understood the concept of attributes properly. Then I saw how I could augment the vertex data by putting its world position as an attribute and I could draw the entire terrain with a single drawArrays call - And BAM! Now everything was running smoothly at 19fps (yeah. Much better than 7 fps) for an entire map of 256 x 256.

Everything from here on was mostly polishing. I worked a lot on the camera - I was using the LookAt matrix but couldn't figure out how to roll and I wanted the camera to roll on turning. After scourging the internet found a simple method to just pre-multiply the LookAt matrix with a roll matrix.  I didn't like the way you could lose your bearing if you looked right up. So I added some clouds(All clouds drawn with a single GL draw call), filled the surrounding areas with water and added a wrap around if you strayed too far from the island. I added some cubes to represent the enemies (all drawn with a single call). I spent quite a bit of time overlaying a 2D canvas to represent the minimap so that you would knew where your enemies were. I tried adding bullets to shoot down the enemies with the basic physics handled right in the shader but that wouldn't work - I just had to give up on it. (Did you wonder why it was called Voxel Shooter but there is no shooting? :) I spent the whole night before the competition deadline at 7AM adding these things and then before I realized it was time to submit!

I was very dejected that I didn't complete the game and I was too near to the deadline. You just flew around doing pretty much nothing. You couldn't even crash into anything. Staying awake for the whole night was a complete waste of my time. And then out of sheer desperation I thought I will just add a simple check for crashing into the terrain and that would be the game - Avoid crashing into anything. I was testing it and I kept going through the enemies. I thought I should add collision detection with the enemies as well. I added that and then I realized instead of treating crashing into enemies as a failure, that could be the whole point of the game!! I quickly added a score for picking up the red cube (no longer the enemy!) and increase the speed of the game every time you finished a batch of them. I submitted this new version and was smugly happy with my own crisis handling. Next time I should just give myself more time to avoid such things.

The next day playing through the other entries of the js13kgames and I liked many of them. It was interesting to see the methods they used to keep the size down. The most amazing thing I learnt about was jsfxr - A simple sound effects generator without having to store audio in huge audio files. I am definitely going to use this more in my upcoming projects and in the future versions of Voxel Shooter.

Some people seemed to like Voxel Shooter and it was really cool that indiegames.com picked it - http://indiegames.com/2012/09/browser_game_pick_voxel_shoote.html. Even Cheezbruger picked it for their Lunch leisure  - http://t.co/xO9drzmZ! Some were amazed it was done within 13KB but I should mention that my game was very simple - it just does the same thing a lot of times and I didn't spend any time trying to reduce the size.

The initial reaction from people has given me encouragement to keep working on the game and I think I will make it into a full fledged game in the future. Lots of ideas flying in my mind now - Sounds, bigger worlds, flying enemies, improved shading, shadows, water, lava, gamepad support, multiplayer, levels, missions... But now I will automatically think of one more thing - How small can I keep the code and assets?

And to end it all here is a video from my awesome friend Brian who scored 148 in Voxel Shooter! Can you beat that?!!