AI-powered robotic glove helps stroke patients play the piano again

Smart hand exoskeleton glove has tactile sensors, soft actuators, and artificial intelligence to help neurotrauma patients relearn manual tasks.
Shubhangi Dua
Representational image of a robot playing the piano

Possessed Photography / Unsplash 

Strokes are a leading cause of serious long-term disability in the US, reducing mobility in more than half of stroke survivors ages 65 and older.

Patients suffering from the disabilities caused by a stroke often require intense rehabilitation in order for them to be able to walk, talk, or perform everyday tasks again.

For many, the idea of being able to play a musical instrument must feel like a distant dream.

However, thanks to a new AI-powered soft robotic glove, people trained in music who have suffered a stroke may now be able to play again, and perform other tasks that require dexterity and coordination.

Watch the video of the glove playing independently and with a hand inside below:


Dr Maohua Lin, Professor at the Department of Ocean & Mechanical Engineering at Florida Atlantic University said: "Here we show that our smart exoskeleton glove, with its integrated tactile sensors, soft actuators, and artificial intelligence, can effectively aid in the relearning of manual tasks after neurotrauma."

The AI-powered glove weighs just 191g and was designed in the shape of a multi-layered flexible 3D-printed robo-glove.

The glove's feel is meant to be flexible and feel soft on the skin of the entire palm and wrist area, and can be customized to fit the patient's hand.

Author of the study Dr Erik Engeberg, also a professor at Florida Atlantic University’s Department of Ocean & Mechanical Engineering says that the glove is designed to assist and enhance natural hand movements.

"It allows them to control the flexion and extension of their fingers. The glove supplies hand guidance, providing support and amplifying dexterity,” he said.

A statement by the researchers said, "besides physical and occupational therapy, music therapy can help stroke patients to recover language and motor function". 

Sensing device

The research emphasizes that the glove is modeled with soft pneumatic actuators located in the fingertips that generate motion and exert force. Thereby, allowing the patient to mimic natural hand movements.

The wearable devices' fingertips are also engineered with "an array of 16 flexible sensors or 'taxels'" to provide tactile sensations to the wearer's hand during interaction with objects or surfaces. 

The robot device's actuators and sensors have been placed through a single molding process.

Furthermore, the researchers used machine learning and taught the glove to feel in a way that it can differentiate between playing the correct and incorrect versions of a beginner’s song on the piano.

The authors experimented with the nursery rhyme ‘Mary had a little lamb’, which requires four fingers to play.

Engeberg said, “We found that the glove can learn to distinguish between correct and incorrect piano play. This means it could be a valuable tool for personalized rehabilitation of people who wish to relearn to play music."

The study was published on 29 June in Frontiers in Robotics and AI.


Individuals who have suffered neurotrauma like a stroke or brachial plexus injury often experience reduced limb functionality. Soft robotic exoskeletons have been successful in assisting rehabilitative treatment and improving activities of daily life but restoring dexterity for tasks such as playing musical instruments has proven challenging. This research presents a soft robotic hand exoskeleton coupled with machine learning algorithms to aid in relearning how to play the piano by ‘feeling’ the difference between correct and incorrect versions of the same song. The exoskeleton features piezoresistive sensor arrays with 16 taxels integrated into each fingertip. The hand exoskeleton was created as a single unit, with polyvinyl acid (PVA) used as a stent and later dissolved to construct the internal pressure chambers for the five individually actuated digits. Ten variations of a song were produced, one that was correct and nine containing rhythmic errors. To classify these song variations, Random Forest (RF), K-Nearest Neighbor (KNN), and Artificial Neural Network (ANN) algorithms were trained with data from the 80 taxels combined from the tactile sensors in the fingertips. Feeling the differences between correct and incorrect versions of the song was done with the exoskeleton independently and while the exoskeleton was worn by a person. Results demonstrated that the ANN algorithm had the highest classification accuracy of 97.13% ± 2.00% with the human subject and 94.60% ± 1.26% without. These findings highlight the potential of the smart exoskeleton to aid disabled individuals in relearning dexterous tasks like playing musical instruments.

Add Interesting Engineering to your Google News feed.
Add Interesting Engineering to your Google News feed.
message circleSHOW COMMENT (1)chevron
Job Board