Meta's AI 'MusicGen' creates rock lullaby in just 341 seconds with text prompts
%2Favatars%2FEwuSxftyWaVt.png&w=96&q=75)
%2F2023%2F06%2F14%2Fimage%2Fjpeg%2FibutVArdmg5t1py3BFbuG1jj2rCbAUZEJvLR0GRK.jpg&w=3840&q=75)
Meta has unveiled MusicGen, an artificial intelligence(AI) music-generating system which can be conditioned using text prompts or melodies. It’s similar to Google’s MusicLM, which can build on existing melodies, whether they are whistled, hummed, sung, or played on an instrument.
Generating music is a challenging task as it contains harmonies and melodies from different instruments, which create complex structures. Meta’s model was trained on 20,000 hours of music, reported Tech Xplore. Meta released a demo of MusicGen on Hugging Face, and Interesting Engineering decided to have a go at it.
The language model was asked to generate a ‘rock lullaby’. The system took 341 seconds to come up with a 15-second audio clip. The short excerpt was a combination of tunes from the piano and an electric guitar and sounded like the start of an old classic rock track.
We are not saying that the melody produced by MusicGen is comparable to Led Zeppelin's ‘Stairway to Heaven,’ but it was impressive nevertheless.
Making music from text prompts
Meta and their team of researchers have uploaded a sample set. They have compared different AI music-generating models like MusicGen, MusicLM, Riffusion, and Mousai, by giving these pre-trained models the same textual prompts.
Staying true to its open-source approach towards technology, Meta revealed MusicGen’s code via Github on Saturday. The Zuckerberg-led tech giant also revealed open-source AI models like LLaMA in February. Google’s MusicLM went against making the system publicly available due to ethical issues, said the company in January.
Felix Kreuk, one of the co-authors of the study took to Twitter to share how the platform works.
We present MusicGen: A simple and controllable music generation model. MusicGen can be prompted by both text and melody.
— Felix Kreuk (@FelixKreuk) June 9, 2023
We release code (MIT) and models (CC-BY NC) for open research, reproducibility, and for the music community: https://t.co/OkYjL4xDN7 pic.twitter.com/h1l4LGzYgf
Kreuk explained that MusicGen is built on a single-stage transformerLM which eliminates the need for cascading several models. Speaking on the limitations and ethical aspects of MusicGen, the researchers mentioned in their study that they ensured that all the data MusicGen was trained on was covered by legal agreements with music libraries like Shutterstock.
“A second aspect is the potential lack of diversity in the dataset we used, which contains a larger proportion of western-style music,” acknowledged the researchers.
Touching upon how generative AI has set off waves in the music industry, Meta said, “Generative models can represent an unfair competition for artists, which is an open problem.” This comes at a time when music streaming services like Spotify, Apple Music, and YouTube are still grappling with what AI could mean for copyrighted music.
“Open research can ensure that all actors have equal access to these models. Through the development of more advanced controls, such as the melody conditioning we introduced, we hope that such models can become useful both to music amateurs and professionals,” Meta said further.
Meanwhile, Twitter users are uploading music generated by MusicGen, using their demo website.
So fun trying to generate Bollywood style music using the MusicGen demo from my colleagues at FAIR! 🇮🇳🎶
— Nikhila Ravi (@nikhilaravi) June 11, 2023
"Bollywood music track, traditional Indian sounds of melodic sitar and tabla, blended with modern pop sounds, catchy melody, and infectious rhythms" https://t.co/nXNJEnaJkY pic.twitter.com/WlgEb5ZMob
Study Abstract:
We tackle the task of conditional music generation. We introduce MUSICGEN, a single Language Model (LM) that operates over several streams of compressed discrete music representation, i.e., tokens. Unlike prior work, MUSICGEN is comprised of a single-stage transformer LM together with efficient token interleaving patterns, which eliminates the need for cascading several models, e.g., hierarchically or upsampling. Following this approach, we demonstrate how MUSICGEN can generate high-quality samples, while being conditioned on textual description or melodic features, allowing better controls over the generated output. We conduct extensive empirical evaluation, considering both automatic and human studies, showing the proposed approach is superior to the evaluated baselines on a standard text-to-music benchmark. Through ablation studies, we shed light over the importance of each of the components comprising MUSICGEN.
SHOW COMMENT (1)
/2023/10/04/image/jpeg/edyTH7wBVGRCw06zSkmj84JbL9Z3QWGytZ93rAtk.jpg "Homo sapiens used another route to migrate out of Africa")
/2023/09/22/image/jpeg/N09BSEVNiNkMLnA50kD8o3yiMP8ye1mAEFwtWa4G.jpg "Is carbon capture the missing piece in the net zero puzzle?")
/2023/10/04/image/png/uZDSs7vy0tg9xGYNPTpebaWHAJ1IgCMFlz4zUiwn.png "CEOs say 47% of workforce unprepared for the future of work")
/2023/09/24/image/jpeg/LbEWVi9Dqey9YK0OQQHLWcm6Fwci9Kgmhp4SL6OG.jpg "10 ancient civil engineering projects still in use")
/2023/10/04/image/jpeg/CxUBYC3GrWAJN1xNxxWfeFrQnabHYIYkt4enfNEM.jpg "Droneliner promises 70% less cost for air freight transfers")
/2023/08/30/image/jpeg/79rLCSNuXkygYT4n4NHbsaeEzi0PBuvPGTEd9Hl5.jpg "Are smart phones destroying our mental health?")
/2023/07/26/image/jpeg/9Dkowuh7yI1YIB0hFtRiZ5DWxel88daCnm1Eld3Y.jpg "Can AI accurately identify patients with respiratory symptoms?")
/2023/10/04/image/jpeg/zP7nT9CM4OigYk7QAtUcsZ0dXW5MOVagfHc0yOAP.jpg "No mirrors: In a first, lasers deflected using only air")
/2023/10/04/image/jpeg/mxK2HCx3zNJKKQBocAvIQTYC7a5xxRYsau3xRRg6.jpg "FDA-approved drug, Amiloride reverses symptoms of ELAs")
/2023/09/25/image/jpeg/PLsjlszLj5qkBsD2EnRN71RHCyq1XFTxyJEU0FTD.jpg "Space startup aims to raise Hubble's orbit using water — here's how")