Researchers used AI and MRI scans to decode thoughts - and they were mostly accurate

The subject does not require surgical implants

Led by Jerry Tang, a doctoral student in computer science, and Alex Huth, an assistant professor of neuroscience and computer science at UT Austin, the work relies to an extent on a transformer model similar to the ones that power Open AI’s ChatGPT and Google’s Bard, the release said.

The technology also does not require the person to have surgical implants, unlike other language decoding systems. But mind you; it doesn't mean anyone can read thoughts involuntarily; the subject has to be actively cooperating with the scientist.

"For a noninvasive method, this is a real leap forward compared to what’s been done before, which is typically single words or short sentences," Huth said in a statement. "We’re getting the model to decode continuous language for extended periods of time with complicated ideas."

How does this work?

Subjects were asked to listen to hours of podcasts in a functional MRI scanner. Listening to these podcasts triggered brain activity, which was then decoded by the machine, thereby generating text.

As aforementioned, the decoder produces text that matches the intended meanings of the original words, not the actual text itself. For example, if a participant listening to a speaker said, "I don’t have my driver’s license yet", the machine decoded it as "She has not even started to learn to drive yet."

The subjects were also asked to watch four short, silent videos while in the scanner. Using their brain activity, the semantic decoder described certain events from the videos.

How practical is this system?

Currently, it relies on an fMRI machine so that it can be used only within the limits of a lab. According to the researchers, this work could be transferred to others, such as more portable brain-imaging systems, such as functional near-infrared spectroscopy (fNIRS).

"fNIRS measures where there’s more or less blood flow in the brain at different points in time, which, it turns out, is exactly the same kind of signal that fMRI is measuring," Huth said. "So, our exact kind of approach should translate to fNIRS," although, he noted, the resolution with fNIRS would be lower.

But they are aware that bad actors could misuse the tech.

"We take very seriously the concerns that it could be used for bad purposes and have worked to avoid that," Tang said. "We want to make sure people only use these types of technologies when they want to and that it helps them."

Study Abstract:

A brain–computer interface that decodes continuous language from non-invasive recordings would have many scientific and practical applications. Currently, however, non-invasive language decoders can only identify stimuli from among a small set of words or phrases. Here we introduce a non-invasive decoder that reconstructs continuous language from cortical semantic representations recorded using functional magnetic resonance imaging (fMRI). Given novel brain recordings, this decoder generates intelligible word sequences that recover the meaning of perceived speech, imagined speech and even silent videos, demonstrating that a single decoder can be applied to a range of tasks. We tested the decoder across cortex and found that continuous language can be separately decoded from multiple regions. As brain–computer interfaces should respect mental privacy, we tested whether successful decoding requires subject cooperation and found that subject cooperation is required both to train and to apply the decoder. Our findings demonstrate the viability of non-invasive language brain–computer interfaces.