VALL-E: Microsoft's AI can translate your voice into foreign languages
Researchers at Microsoft continue to unveil new artificial intelligence (AI)- based developments, even as products from its collaboration with Open AI are being rolled out to users across the globe. This time, the Redmond-based tech giant's upcoming AI technology will let users speak in a foreign language in their own voice.
A sample of the technology is available in the tweet below,
Speaking foreign languages with your own voice + accent + emotion will soon be a reality.— Lior⚡ (@AlphaSignalAI) March 8, 2023
Microsoft researchers recently extended VALL-E and trained a multi-lingual conditional codec language model to predict acoustic token sequences.
Sound ON 🔊 pic.twitter.com/1mhoChJzcC
Early this year, Microsoft co-founder Bill Gates had said that AI is the next big thing in the tech industry and will bring about the biggest shifts in the coming years. Last November, OpenAI debuted its conversational chatbot and the rest is history. ChatGPT is now part of Microsoft's existing products. The large language model is being scaled to reach millions of users every month.
Microsoft's own voice-based AI
Even as Microsoft piggybacks on OpenAI's success, which it has financially funded, the company researchers aren't taking a back seat when it comes to AI research. In January this year, the company revealed the progress it had made on its text-to-speech AI, VALL-E.
Moving away from conventional approaches like manipulating waveforms to generate speech, Microsoft's researchers gave the AI an ability to generate discrete audio codecs from the text and audio prompts. The team was also able to reduce the audio prompt length to just three seconds and still get an accurate speech matching the user's voice.
With the recent development, researchers have definitely taken AI further allowing users to communicate in foreign languages using their own voice, even if they do not speak them. This could go a long way in helping people communicate more effectively across borders, reducing hurdles in communication. As seen in the clip above, the AI is also capable of correctly depicting multiple emotions, making it even more realistic.
Microsoft is also well aware of the drawbacks and potential misuse of this technology and has therefore not released it to the public yet. Interestingly, VALL-E has been trained on the LibriLight dataset, which was created by Mark Zuckerberg's Meta, which is betting big on the metaverse as the future of the internet.
Interesting Engineering recently reported that search engine giant Google is also working on its ambitious AI that could help users translate more than 1,000 languages spoken all over the globe.
Interesting Engineering highlights various Martian geological features, including those discovered by NASA's Curiosity rover over its 10-year voyage.