Meta's AI 'MusicGen' creates rock lullaby in just 341 seconds with text prompts

MusicGen has been trained on 20,000 hours of music.
Sejal Sharma
Representational image.
Representational image.

Menno van Dijk/iStock 

Meta has unveiled MusicGen, an artificial intelligence(AI) music-generating system which can be conditioned using text prompts or melodies. It’s similar to Google’s MusicLM, which can build on existing melodies, whether they are whistled, hummed, sung, or played on an instrument.

Generating music is a challenging task as it contains harmonies and melodies from different instruments, which create complex structures. Meta’s model was trained on 20,000 hours of music, reported Tech Xplore. Meta released a demo of MusicGen on Hugging Face, and Interesting Engineering decided to have a go at it.

The language model was asked to generate a ‘rock lullaby’. The system took 341 seconds to come up with a 15-second audio clip. The short excerpt was a combination of tunes from the piano and an electric guitar and sounded like the start of an old classic rock track. 

We are not saying that the melody produced by MusicGen is comparable to Led Zeppelin's ‘Stairway to Heaven,’ but it was impressive nevertheless.

Making music from text prompts

Meta and their team of researchers have uploaded a sample set. They have compared different AI music-generating models like MusicGen, MusicLM, Riffusion, and Mousai, by giving these pre-trained models the same textual prompts.

Staying true to its open-source approach towards technology, Meta revealed MusicGen’s code via Github on Saturday. The Zuckerberg-led tech giant also revealed open-source AI models like LLaMA in February. Google’s MusicLM went against making the system publicly available due to ethical issues, said the company in January.

Felix Kreuk, one of the co-authors of the study took to Twitter to share how the platform works. 

Kreuk explained that MusicGen is built on a single-stage transformerLM which eliminates the need for cascading several models. Speaking on the limitations and ethical aspects of MusicGen, the researchers mentioned in their study that they ensured that all the data MusicGen was trained on was covered by legal agreements with music libraries like Shutterstock. 

“A second aspect is the potential lack of diversity in the dataset we used, which contains a larger proportion of western-style music,” acknowledged the researchers.

Touching upon how generative AI has set off waves in the music industry, Meta said, “Generative models can represent an unfair competition for artists, which is an open problem.” This comes at a time when music streaming services like Spotify, Apple Music, and YouTube are still grappling with what AI could mean for copyrighted music.

“Open research can ensure that all actors have equal access to these models. Through the development of more advanced controls, such as the melody conditioning we introduced, we hope that such models can become useful both to music amateurs and professionals,” Meta said further.

Meanwhile, Twitter users are uploading music generated by MusicGen, using their demo website.

Study Abstract:

We tackle the task of conditional music generation. We introduce MUSICGEN, a single Language Model (LM) that operates over several streams of compressed discrete music representation, i.e., tokens. Unlike prior work, MUSICGEN is comprised of a single-stage transformer LM together with efficient token interleaving patterns, which eliminates the need for cascading several models, e.g., hierarchically or upsampling. Following this approach, we demonstrate how MUSICGEN can generate high-quality samples, while being conditioned on textual description or melodic features, allowing better controls over the generated output. We conduct extensive empirical evaluation, considering both automatic and human studies, showing the proposed approach is superior to the evaluated baselines on a standard text-to-music benchmark. Through ablation studies, we shed light over the importance of each of the components comprising MUSICGEN.

Add Interesting Engineering to your Google News feed.
Add Interesting Engineering to your Google News feed.
message circleSHOW COMMENT (1)chevron
Job Board