Lyrely was created to fill a gap in online music portfolio software as it is specifically aimed at video game composers. It needed to do three things well:
- Have a web player that sits on an artist’s page nicely (no iframes, mobile support)
- Stream high quality music that would support gapless looping
- Secure media files and password protected portfolios due to NDAs and the nature of game development.
Music is the heart of the platform and there are many different music file formats all with various ups and downs, OS support, etc. I was working with a friend of mine, Tommy Pedrini to help me determine how to maximize sound quality while attempting to keep the file size low. At first we started with AAC, designed to be the successor to MP3 files. AAC, while better than MP3s in terms of file size and audio quality was still not reaching the standard that we hoped to achieve.
OPUS was the answer to our audio quality problems, it supports near-lossless audio wrapped in a reasonable file size (5-8mb typically). However it is not supported on Apple devices unless wrapped as a CAF file, this added some additional complexity in how audio files are converted and served (more on this in the Tech Stack area).
The web player takes advantage of Audio Streaming, HTML Canvas and the Web Audio API to deliver quick load times, advanced visualizer features and gapless looping. The audio visualization features (dynamic waveform and particle effects) and gapless looping required use of the Web Audio API. However, the Web Audio API meant that users would have to download the entire song before any music would begin to play. This delay was extremely noticeable and made the player feed sluggish and broken.
The fix was a hybrid approach: when a user presses play on a song the player will use native HTML Audio Streaming to begin playback. In the background it begins downloading the entire song file and once that download is complete it begins playing the song again at 0% volume using the Web Audio API. Finally, it fades the audio stream out and the Web Audio player in. By fading one audio source out and the other in at the same time, the user doesn’t notice the switch. With full access to the Web Audio API stream we can use FFTs to analyze the incoming audio (to determine “energy” for the particle systems / visualizers) and setup loop points for seamless loops.