ImageBind: Meta pushes AI boundaries, new tool may enable machines to sense like humans
%2Favatars%2FEwuSxftyWaVt.png&w=96&q=75)
%2F2023%2F05%2F10%2Fimage%2Fjpeg%2FNiFaxW7GJykqF64BnQohK4BNxC2MStjLBHSe9XMT.jpg&w=3840&q=75)
Meta has introduced a new open-sourced AI tool called ImageBind, which combines six kinds of data, namely images, text, audio, depth, thermal, and IMU data, to create multisensory content.
Thermal and IMU mean that the model works by calculating motion and position as well.
The goal of the research team was to create a single joint embedding space for multiple streams of data using images to bind them together. However, it does not need datasets where all modalities co-occur with each other.
The company hasn’t discontinued its strategy of keeping its technology open-source. It had developed in February a "state-of-the-art" AI large language model called LLaMA, which was also open-source.
ImageBind mimics human perception
Meta, the parent company of Facebook, Instagram, and Whatsapp, says that the new machine learning tool brings us one step closer to training an artificial intelligence in how humans learn from their environments through their senses.
ImageBind equips machines with a holistic understanding that connects objects in a photo with how they will sound, their 3D shape, how warm or cold they are, and how they move, said Meta in a statement.
The AI model works by detecting objects in a photo and giving information about the same. For example, ImageBind will provide information on how hot or cold an object in an image will be, what sound it will generate, what its shape will be, and how it will move.
It will also allow content creators to make videos using sounds and movement based on only one or two kinds of data like text, image, or audio. For example, a creator could couple an image with an alarm clock and a rooster crowing, and use a crowing audio prompt to segment the rooster or the sound of an alarm to segment the clock and animate both into a video sequence, said the company in a statement.
Meta also announced that they will soon introduce more streams of data that link as many senses as possible, like touch, speech, smell, and brain fMRI signals. “This will enable richer human-centric AI models,” said the company.
The AI model is a research prototype at this point, and the research team says it cannot be readily used for real-world applications just yet.
Companies ranging from Microsoft to Google are working on developing new AI models and tools to woo their users. Meanwhile, Meta’s chatbot Blenderbot 3 failed to impress as a rival to OpenAI’s ChatGPT, Microsoft’s Bing, and Google’s Bard AI.
SHOW COMMENT (1)
/2023/02/11/image/jpeg/aSPT6SrX97dY3X0XZv9oG3R4skxYmqc0Kf7fc7pL.jpg "Paradise lost: How phosphate mining devastated island Nauru")
/2022/09/13/image/jpeg/XESipDgzo626YevyD1EBRRN1fWfmMkGrbPgIJuoF.jpg "Disruptive innovation: how the likes of Apple and Microsoft excel")
/2023/06/03/image/jpeg/MwGIp61b2PM577qboU4pfCmoiPEju8IbV9fCHbx8.jpg "Hawking was right: All large objects will eventually evaporate")
/img/iea/9lwjqq916E/untitled-1-12.jpg "A study on 10,000 kinds of Earth's minerals could help us discover extraterrestrial life")
/img/iea/3gG948kkOV/istock-639533814.jpg "Researchers discover people are using the internet for sex")
/2023/05/26/image/webp/6Co80ZJJXboUOYLXIuCvkafLsBWpH8MROKX60PUF.webp "Hyper-luxury Bugatti tower will have its residents drive up to their floor")
/2023/05/27/image/jpeg/hUmy6fVQvimxfN8HBZ2IjrGKW1f8AVmrkTi3wPtJ.jpg "Stanford professor says he is 100% sure that aliens are already on Earth")
/2023/06/03/image/jpeg/LJsTJ6HYfeZoh6Hlc4JLiP05g8ykKNa1WXClqVwX.jpg "Scientists detect the breath between atoms")
/2023/05/03/image/jpeg/94IU5ecY33CpPOCYgomPHSve8e3xhDiaK3Xo4LA3.jpg "First-of-its-kind inflatable drone perches like a bird for search and rescue")
/2023/05/26/image/jpeg/smTFO3AG7ESAzCi8L0kVnrUEHbVmcjiZ2KhVW0QE.jpg "Potential cure for diabetes found in groundbreaking stem cell study")