New vocal cords on a chip can let mute people speak

It achieves this by using an artificial intelligence model to interpret captured sounds and bodily vibrations to generate vocalizations.

"Its feasible [voice] fabrication process, stable performance, resistance to noise, and integrated vocalization make the AT a promising tool for next-generation speech recognition and interaction systems," Tianling said.

He further explained that the new chip’s graphene sensors are ideally suited for the detection of tiny vibrations on skin surfaces. The device, he said, "can sense muscle motions and audio vibrations transmitted to the surface of the skin" and "convert recognizable mechanical information into speech."

Noise interference is not a problem

Better yet, interference from noisy, aggressive environments such as highways, fire disasters, and airplane cockpits do not affect the device. It continues to function at optimum levels regardless of the environment it is in.

"The speaker's health status, such as neurological diseases, cancer, trauma, and the surrounding environment, noise interference, and transmission medium, often affect the transmission and recognition of sound," Tianling said.

These circumstances do not affect the chip which has the unique quality of being able to function despite any external interference. More studies are required to bring more expressiveness to vocalizations but the researcher believes the device is practical and efficient enough to become commonplace in the future.

"Our intelligent AT provides a new paradigm for speech recognition, and is expected to pave the way for applications of mechanical sensors to intelligent home health-monitoring systems, wearable electronics, and even cryptographic security," he said.

The study was published in Nature Machine Intelligence on February 23.

Study abstract:

Researchers have recently been pursuing technologies for universal speech recognition and interaction that can work well with subtle sounds or noisy environments. Multichannel acoustic sensors can improve the accuracy of recognition of sound but lead to large devices that cannot be worn. To solve this problem, we propose a graphene-based intelligent, wearable artificial throat (AT) that is sensitive to human speech and vocalization-related motions. Its perception of the mixed modalities of acoustic signals and mechanical motions enables the AT to acquire signals with a low fundamental frequency while remaining noise resistant. The experimental results showed that the mixed-modality AT can detect basic speech elements (phonemes, tones and words) with an average accuracy of 99.05%. We further demonstrated its interactive applications for speech recognition and voice reproduction for the vocally disabled. It was able to recognize everyday words vaguely spoken by a patient with laryngectomy with an accuracy of over 90% through an ensemble AI model. The recognized content was synthesized into speech and played on the AT to rehabilitate the capability of the patient for vocalization. Its feasible fabrication process, stable performance, resistance to noise and integrated vocalization make the AT a promising tool for next-generation speech recognition and interaction systems.