I had a crazy idea and wanted to create a quick experiment to test it out. Mobile browsers are becoming good - Javascript and Canvas implementation are becoming faster. There are certain advantages to the mobile browsers as a game distribution platform - the most important of which is that it is open and not controlled by arbitrary rules set by Apple, Google or whomever (Classic example - Phone Story). HTML5 games in general are picking up steam but game makers are ignoring the mobile browser.
There are two factors why games on mobile browsers don't work well -
Game developers ignore things like centering the screen on the game area, setting the right zoom level, allowing for touch input(or any alternate mechanism like accelerometer) when there is no keyboard
Browser makers don't provide a good way to do certain things - locking screen orientation, a working audio API etc.
1) is definitely solvable while 2) involves letting platform creators know about the issues faced by game developers (Some influential developers that are already trying to do that - http://www.phoboslab.org/log/javascript). Maybe newer platforms like Mozilla OS will push the standards higher on the existing ones.
So the crazy idea is centered around the concept that HTML5 games being based on open standards should be modifiable by anyone (An idea floating around in Mozilla land) - In this specific case modified to work well on mobile browsers. If a game developer didn't take care to make the game work well on mobile browsers, anyone should be able to fix it.
To prove this point I created a proof of concept - which I called "GameDoc". For the experiment I took the game emptyblack.com. I really liked the game when I played it on the desktop and I wanted to play it on my phone. The game seems to run ok as far as the graphics is concerned but it had the classic issues mentioned in 1). Since there is no keyboard input in the phone the game is unplayable.
GameDoc is a native mobile App that basically wraps a browser WebView. It loads the game from the original URL but it sets the required zoom level on the WebView to get a good view of the game area and locks the screen orientation to landscape. And here is the cool part - It fixes the absence of keyboard by injecting Javascript snippet into the page that adds on-screen controls. When these controla are pressed keyboard events are simulated using the dispatchEvent mechanism. I used the WebView wrapper route to be able to do this Javascript injection on the device (due to absence of browser extensions on the mobile).
So here is the video of how the converted game works. The video shows how the game is not playable in it's original form in the mobile Chrome browser and how the fixed version works with simulated onscreen controls. It's not perfect - Sometimes keyup/keydown events seem to get missed and I think it might be due to multitouch issues. As you can see I really threw this together rather quickly with horrible looking icons for the controls - But you get the idea.
The grand idea is that there can be a crowd sourced HTML5 game portal where people can add their own customization to the game controls or the game itself and the Gamedoc App(or whatever better name I come up with) would pull that extra information for the game and does the necessary modding.
Can I avoid the need to have a separate App that wraps the WebView and just use the native browser? Maybe I can do something at the server side but that might screw up with things that are tied to the original domain like cookies or the local store. Maybe I can just do the controls with some sort of IME trickery.
Anyway as I said it's a crazy experiment. At the least I hope to push HTML5 game developers to think more about the mobile web.
Let me know of your thoughts/comments.
Update: Looks like IOS6 has better support for Audio opening up better game support! And the awesome folks who make Construct2 already support it - https://www.scirra.com/blog/98/the-web-as-the-platform
I knew I had to take part in js13kgames - A competition to write Javascript games less than 13KB when zipped. I used HTML5 canvas for the lasttwo of the last three game projects I worked on and was increasingly identifying myself as a HTML5 game programmer. However a visit back to India and the work on Mario Ball had left very little time for me to actually work on my 13KB game.
At 13KB I had decided to have some sort of procedural generation for the elements of the game. I also wanted to explore the Web Audio API for the sound. However I was having a very hard time trying to come up with a concrete idea for the game. My very first idea was a weird fighting game where the player's form and powers were determined by a random combination of 3 elements - Rectangles, Circles and Bezier curves. The player and enemies basically looked like modern art pictures. I wasn't sure whether I was going too weird with this one. Some of the denizens of this unfinished game:
Then this entry came out on js13kgames - 13th Knight. It was a fully textured 3D world in WebGL in under 13KB! And it had awesome looking trees!! I wanted to make a WebGL game now. I always wanted to learn WebGL but hadn't looked closely into it it before. I had started looking at three.js for 7dfps but never got the time to do anything beyond a sphere moving on plane. This time around three.js wasn't going to be an option because no external libraries were allowed and three.js would not fit into 13KB (I later learnt somebody else had managed to get a subset of three.js working in their 13KB game - Mindless).
So not exactly knowing what I'm going to build I started with the WebGL lessons at http://learningwebgl.com/blog/?page_id=1217 for the basic template for a WebGL program. I played around with it till I was able to draw a flat colored cube with directional lighting. There were 5 days till the competition ended and I had to come up with something quick. Then it struck me - I will just redraw the cubes a lot of times to create a Voxel engine! (Being completely ignorant of the issues before starting a project is sometimes a good thing). I was missing a good flying game these days and I thought I will do a flying game where you shoot enemies on the ground - Voxel Shooter!!
I generated the terrain using some random combination of sin curves. I copied a small subset of the glMatrix library for the matrix operations. And I happily went about drawing cubes over and over again - One drawElements call per cube. I could reach up to 50 x 20 array of cubes before the rendering became too slow. I tried removing faces from the cubes that are hidden behind other cubes - A small increase in the frame rate but nothing great. I tried some cheating like increasing the size of each cube so that I can get away with a smaller number of cubes. Nope - the map was still too small. I banged my head trying to figure out a frustum culling method but it just involved too much Math to be figured out in a small amount of time. Also the far plane was going to be too near and field of view was going to be too narrow if I can display only 1000 cubes. I just had to figure out a way to improve performance first.
Then I came across this - Google I/O 2011 : WebGL Techniques and Performance. I had basically made all the mistakes mentioned in the talk. First and foremost I was switching too much context for every vertex - Even the uniforms that were common to all vertices like lighting and camera and projection matrices. I got a substantial boost when I fixed this issue but still not going to allow the 256 x 256 map size I was aiming for. The next thing was to reduce the number of GL draw calls by combining many cubes into one giant mesh. This one I had some problem getting my head around because I hadn't understood the concept of attributes properly. Then I saw how I could augment the vertex data by putting its world position as an attribute and I could draw the entire terrain with a single drawArrays call - And BAM! Now everything was running smoothly at 19fps (yeah. Much better than 7 fps) for an entire map of 256 x 256.
Everything from here on was mostly polishing. I worked a lot on the camera - I was using the LookAt matrix but couldn't figure out how to roll and I wanted the camera to roll on turning. After scourging the internet found a simple method to just pre-multiply the LookAt matrix with a roll matrix. I didn't like the way you could lose your bearing if you looked right up. So I added some clouds(All clouds drawn with a single GL draw call), filled the surrounding areas with water and added a wrap around if you strayed too far from the island. I added some cubes to represent the enemies (all drawn with a single call). I spent quite a bit of time overlaying a 2D canvas to represent the minimap so that you would knew where your enemies were. I tried adding bullets to shoot down the enemies with the basic physics handled right in the shader but that wouldn't work - I just had to give up on it. (Did you wonder why it was called Voxel Shooter but there is no shooting? :) I spent the whole night before the competition deadline at 7AM adding these things and then before I realized it was time to submit!
I was very dejected that I didn't complete the game and I was too near to the deadline. You just flew around doing pretty much nothing. You couldn't even crash into anything. Staying awake for the whole night was a complete waste of my time. And then out of sheer desperation I thought I will just add a simple check for crashing into the terrain and that would be the game - Avoid crashing into anything. I was testing it and I kept going through the enemies. I thought I should add collision detection with the enemies as well. I added that and then I realized instead of treating crashing into enemies as a failure, that could be the whole point of the game!! I quickly added a score for picking up the red cube (no longer the enemy!) and increase the speed of the game every time you finished a batch of them. I submitted this new version and was smugly happy with my own crisis handling. Next time I should just give myself more time to avoid such things.
The next day playing through the other entries of the js13kgames and I liked manyofthem. It was interesting to see the methods they used to keep the size down. The most amazing thing I learnt about was jsfxr - A simple sound effects generator without having to store audio in huge audio files. I am definitely going to use this more in my upcoming projects and in the future versions of Voxel Shooter.
Some people seemed to like Voxel Shooter and it was really cool that indiegames.com picked it - http://indiegames.com/2012/09/browser_game_pick_voxel_shoote.html. Even Cheezbruger picked it for their Lunch leisure - http://t.co/xO9drzmZ! Some were amazed it was done within 13KB but I should mention that my game was very simple - it just does the same thing a lot of times and I didn't spend any time trying to reduce the size.
The initial reaction from people has given me encouragement to keep working on the game and I think I will make it into a full fledged game in the future. Lots of ideas flying in my mind now - Sounds, bigger worlds, flying enemies, improved shading, shadows, water, lava, gamepad support, multiplayer, levels, missions... But now I will automatically think of one more thing - How small can I keep the code and assets?
And to end it all here is a video from my awesome friend Brian who scored 148 in Voxel Shooter! Can you beat that?!!
Something very unexpected happened. The Babycastles folks were running a Summit at the Museum of Modern Art and Design(MAD) - It was centered around bringing Keita Takahashi's ideas to reality along with other things like game (not just videogames but also about things like games in physical spaces) related talks and indie music bands.
They wanted some help with the programming for one of their projects. For some inexplicable reason they thought of asking me and I jumped at the opportunity. I had always wanted to explore physical games/ alternate interfaces and wanted to do things similar to what Kaho Abe does. And now they told me I was actually going to work with her! Not to mention an opportunity to work with Keita Takahashi!!
I had an initial meeting with Keita and Kaho. The project, Mario Ball, was to build the original Mario brothers - but instead of using a normal controller the ideas was to control it using a wooden maze representing the level and Mario actually controlled by a ball. I think Keita's original drawing explains this the best.
Kaho had decided that we are going to use a magnetic ball for Mario and an array of magnetic reed switches underneath the controller surface to detect where the ball is an move the on screen Mario accordingly (It was decided not to use computer vision because that was already being used for another game). This is where I was going to help them - Actually build the game that processes the controller input to move Mario around in the game. It was also decided to build the controller as a big box so that it would require two people to work with each other to actually control Mario! We also decided that we would make Mario go up side down when the ball is actually moved up and it stays up there - something like a change in direction of gravity. I was concerned with the resolution of the magnetic reeds and asked Kaho to also add an accelerometer so that we can try and interpolate Mario's position when the ball was actually between two magnetic sensors and we had to guess which direction Mario was actually moving. Throughout the meeting I was mostly just thinking - Yay!! I am talking with Keita!!
Kaho was going to build this insane setup with 90 magnetic reed switches connected to an Arduino through a set of multiplexers. The program running on the Arduino would detect the active magnetic switch and write that value to the serial port along with the 3 axis acceleration information from the accelerometer. So now we were ready to start building the controller and the game. The kicker - We had 6 days to build this whole thing!! (And not even 6 whole days. I had to go to my day job on all the days and work in the evenings. Kaho was handling the hardware for 2 other games)
I think my experience with working with multi-tiered systems in my day job (At this small startup) helped here. We decided what the format of the controller output is going to be like so that we can initially work separately and in parallel and then put the controller and the game together when they were ready. I decided to build the game with a ball and magnetic switches that are actually simulated (Similar to mocks/stubs that we use to test in isolation without external dependency) so that I could iron out some of the issues I would face even before the actual controller was ready. In hindsight this got me to 50% of the actual solution - but that was good enough given the time crunch. The things I totally got wrong in my simulation were the area which each magnetic switch covered and the maximum speed the ball would actually be able to roll in the real world.
So I went back and started to actually build the game. I didn't spend much time thinking about it but I decided to use Processing because of its inbuilt ability to use the serial port to interface with the controller. This was a good and a bad idea but at the end of the day was probably the right decision. The good part was that it all worked out and even though I developed the whole thing on the Mac it perfectly ran on the Windows machine where we finally put it - including the serial port part. The bad things included Processing not really having an easy way to rotate and blend images at the same time (Really? I expected better) and some bugs when part of the rendered image going offscreen. The IDE itself was horrible at times and I had to actually delete one file at a time to find syntax errors. At the end of the day the positives probably overrode the negatives. But the next time around I would think a bit harder before using Processing for a game.
I spent the first few days to actually get the assets of the original Mario brothers(Thanks Syed!) in and trying to mimic the looks of the original game as much as possible. I built the game in a way which is probably not recommended - I tried to get the graphics of the game done well before the actual logic/gameplay of the game. But I found that doing this motivated me much more than just dummy boxes and circles. Looking back I think this also helped out because I would have never been able to put in the effort to get the graphics right in the crazy days that were going to follow. I also spent the initial time building the framework in Processing to read in the world layout and character animation from Tiled TMX files. This also helped me out later when Keita wanted me to add extra characters into the game and I was able to do it with relative ease. As usual I wrote custom collision detection code instead of trying to figure out how to use Box2d with Processing. In my defense I didn't have time to ramp up on anything more complicated than the simple graphics APIs and this was the first time I was using Processing.
We met halfway through the week - Both the hardware and software were lagging behind where they should have been at that point. Keita was amazing as always - He had built the frame for the controller and created an actual Mario Ball by pasting colored strips to create a Mario on the magnetic ball (Thanks Lauren for the photo!).
That night me and Kaho just worked through the whole night trying to finish as much as possible. We made really good progress but we still weren't ready to put things together. The whole time I wasn't stressing myself out and tried to stay as calm as possible. I also told myself that Babycastles always manages to pull off these things - So I had nothing to worry! (It was half true at the end :). So here is the picture of the amazing circuitry that Kaho put together that night. I think she called it a unicorn barf or something like that.
The next day we had to shift to the MAD and this is where things started going wrong. We weren't allowed to stay late the day before we had to set the whole thing up! So on the opening day(Friday) we just had a few hours to get the whole thing ready. Kaho was also getting very busy dividing her time between all the projects she had to get working. This was a very tense day and unfortunately we weren't able to get Mario Ball ready that night.
On Saturday we finally got the game and the controller talking with each other. It was a magical moment. I never thought we would ever get to this stage given the craziness that was the whole project :) It had few kinks including a bug in the Arduino code which made it print lots of useless values in between the actual useful ones. A few hours later and few critical bug fixes later we had something close to playable - There were some issues with Mario appearing to teleport when the ball moved too quickly - but the game was definitely playable. And so by Saturday afternoon Mario Ball was ready to go! We spent some time getting the actual hardware for it setup and the game was finally ready to be put up.
I haven't talked much about Keita and probably mention here just a bit. He is a person of few but precise words and I would like to keep this section the same way. He is an amazing combination of a nice guy and a person who pushes you to the limit. He wasn't there to just have fun - But actually do something great. Good enough was not good enough - perfection was the only end point. In the end I didn't do everything that he asked for - but the game definitely ended up much better than if we had just aimed for something good enough. The couple of things I took out of this whole experience was to aim high and always keep improving.
Even though we were almost a day late I was very happy to have Mario Ball running for more than half of the summit. I think most people had lots of fun playing the game. There were some who said it was too experimental - but I thought that was part of what we were trying to do. Kaho had done an amazing job with the hardware - It worked without anything breaking for most of the summit (It briefly broke down on Sunday evening - but it was just someone had yanked on the controller so hard that the USB cable connected to the Arduino had come off. The crazy wiring just kept working the whole while).
So here it is - A picture of couple of people playing Mario Ball in the jungle themed arcade section of the Summit (The jungle theme was an amazing achievement of its own. Somebody needs to write about that). This should give an idea of the size of the controller and how it would take two people to use it. There are so many details in the controller I can't capture here. Keita added the fur like exterior and window handles(which reminded Keita of his Grandmother's house?) on the side so that you can easily hold and play.
Here is a video of people playing it at the Summit. I am sort of happy that I was able to capture the experience of the two players actually co-ordinating with each other to play the game.
I have no good words to end this post. Maybe that this is just the beginning. We will keep improving. You will see a better Mario Ball in the future!!
