Here at tombola we strive to make our games as engaging as possible and one of the core aspects of our designs is audio playback and how it compliments the gaming experience. Delivering these expectations on mobile browsers has proved to be a great challenge as many of our games require fast paced, responsive and concurrent audio playback. The HTML5 audio tag has been around for some time now and is usually described as “limited”, “clunky” and “cumbersome to work with”. Furthermore, in the world of mobile browsers, it is unable to playback more than one sound at once. All in all, with the appearance of Web Audio API, it would seem like it has no place in the world of modern HTML5 games – especially if we want to match desktop/native app audio experiences.

Introducing Web Audio API

Web Audio API is a game changer for web developers as it finally allows us to harness advanced audio capabilities across multiple browsers. I won’t go into detail about all of the awesome features on offer as that is an entire blog post in it’s own right – I’ll provide some useful links at the bottom of the post for people interested in learning more. For our use case we wanted to take advantage of the responsive playback and the ability to play concurrent sounds and we were sure this would be a quick win to give our mobile HTML5 platform a huge boost in user experience.

Unfortunately we quickly ran into a big problem – memory usage.

Where Did All The Memory Go?

After loading 3 or 4 games most mobile browsers would crash due to low memory exceptions. Initially we thought that we had issues in our code where we weren’t removing references correctly and allowing garbage collection to reclaim this memory allocation. After lots of investigation this wasn’t the case. Web Audio API takes the source audio file and decodes it into Linear PCM format which starts eating up lots of memory – and quickly! For example, we were seeing a single 500kb audio file using 80-90MB of memory which on a mobile device is not to be sniffed at. For reasons (still) unknown, browsers were taking a VERY long time reclaiming the huge amounts memory being used, which resulted in memory pressure and eventual browser death. We were sure we must be doing something wrong, so we started looking around for answers, but it turns our that we weren’t the only developers finding memory issues with the Web Audio API:

http://stackoverflow.com/questions/24119684/web-audio-api-memory-leaks-on-mobile-platforms
https://github.com/CreateJS/SoundJS/issues/102
https://code.google.com/p/chromium/issues/detail?id=447580
https://code.google.com/p/chromium/issues/detail?id=400023

We investigated and experimented with different approaches, ranging from using a single sound sprite per game, to splitting all sounds into individual hits and finally even handling all Web Audio API interacting inside of an iframe in the hope that when the iframe was destroyed the browser would treat it like a page reload. However, from our research it seemed that only a full page refresh was the only way to reclaim memory, which would completely break the “native app” feel of our single page application solution. We then moved onto our final roll of the dice.

HTML5 Audio To The Rescue!

For many of our games we identified that we can split our audio into two categories: calls and sound fx. Calls account for 99% of the audio file size in any of our games, have a few seconds gap between them, never play at the same time and do not need to be as responsive as sound fx – a perfect candidate for HTML5 audio playback. On the other hand, sound fx are usually small, need to play concurrently over the top of calls and have to be responsive. Web Audio API anyone?
The results were quite astounding. Using both HTML5 and Web Audio API as a solution significantly reduced the amount of memory allocation, memory pressure over time and also gave browsers enough breathing space to garbage collect without crashing mobile browsers. There was also the added benefit of having a graceful fallback for users who were accessing games through a browser that does not support Web Audio API – I’m looking at you IE!

Conclusion

Web Audio API is an amazing tool for web developers to have in their arsenal, which is capable of delivering brilliant solutions and is clearly the technology of the future for audio on the web. However, when combining it with a single page application architecture on a memory limited mobile platform, developers should be wary, tread lightly and really assess if it is needed in all instances. HTML5 audio is still very much your friend and from our experiences clearly still has a place in the world of HTML5 games for the foreseeable future!

Useful Links

Getting Started with Web Audio API
MDN Web Audio Overview
W3C Spec