AI-generated music is a peculiar thing to think about for two reasons; 1) it is a fascinating new field, and 2) we don't really know where it is going and it feels uncanny. However, it is interesting both musically and commercially.
OpenAI has revealed new horizons on the subject by creating recreations of songs in the style of Pink Floyd, Ella Fitzgerald, 2Pac, and beyond.
Introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We're releasing a tool for everyone to explore the generated samples, as well as the model and code: https://t.co/EUq7hNZv62 pic.twitter.com/sh5yHz7qrc— OpenAI (@OpenAI) April 30, 2020
The results are rather convincing; however, they also sound like drunk karaoke versions of your favorite tunes.
Jukebox, the organization’s new music-generating system, was announced and detailed in a blog post and a new paper. According to their website, Jukebox is "a neural network that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles."
They also released the code behind the Jukebox, along with a tool to explore the generated samples. They promise you a 12-minute read and 10-day listen. It's a sweet deal.
Compressing music into codes
The science behind the AI Jukebox is super interesting. In order to make it possible, they used raw audio to train the model, which spit out raw audio in return.
In order to get it to work, the researchers first used convolutional neural networks to encode and compress raw audio.
After that, they used what they call a transformer to generate new compressed audio that was then upsampled to turn it back into raw audio. By doing so, the neural network generates songs that don't exist in the first place.
Provided with genre, artist, and lyrics as input, Jukebox outputs a new music sample produced from scratch.
For example, check out the classic "Never Gonna Give You Up" with a twist: Jukebox attempts to continuously generate more of the song instead.
This YouTube comment gets an award: "Legend has it that this originated as a tool to re-Rickroll those who had become immune to the original meme."
From Jukebox Sample Explorer, you can explore nearly 8000 generated tracks from numerous artists, complete with instrumental and voices with lyrics. If you've run out of songs to listen by your favorite artist, this is a possible gold mine for you.
Future steps for the project
The team is continuing to work on generating audio samples conditioned on different kinds of priming information, such as MIDI files and stem files. This way, they think that they'll be able to give musicians more control over the generations. Moreover, they are hiring!
This is such a concept that could change music culture forever. Listen for yourself...