3D Hand-Sensing Wristband Could Revolutionize Wearables

A wearable wristband device that could fully grasp our hand movements has the potential to revolutionize the way we interact with technology; hand signals could be used to control our devices and sign language could be effectively transcribed by computers using such a device.

In a new development, a joint team of researchers at Cornell Unversity and the University of Wisconsin-Madison has developed a device they call the FingerTrak, which could allow future wearables to employ incredibly nuanced hand detection by continuously tracking the wearer's hand in 3D.

RELATED: THIS WRISTBAND LETS YOU CONTROL ROBOTS, DRONES, AND TOYS

Reading the contours of the wrist

By using three or four miniature, low-resolution thermal cameras that read contours on the wrist, the bracelet device, FingerTrak, can sense and translate into 3D the human hand's 20 finger joint positions.

"This was a major discovery by our team — that by looking at your wrist contours, the technology could reconstruct in 3D, with keen accuracy, where your fingers are," Cheng Zhang, assistant professor of information science and director of Cornell's new SciFi Lab, said in a press release. "It's the first system to reconstruct your full hand posture based on the contours of the wrist."

Previous iterations that used thermal cameras were considered too bulky to be employed for everyday use. Now, thanks to the research team behind FingerTrak, we might see such a device used in "sign language translation, virtual reality, mobile health, human-robot interaction, and other areas," the researchers explained.

Turning 'silhouette' images into a 3D hand

Impressively, FingerTrak doesn't use its cameras to constantly track the user's full hand movements. Instead, FingerTrak uses a combination of thermal imaging and machine learning to virtually reconstruct the hand.

The bracelet's four miniature, thermal cameras — each of which is approximately the size of a pea — compile multiple "silhouette" images that form an outline of the wearer's hand. A deep neural network is then used to stitch these silhouette images together and reconstruct a virtual hand in 3D that mirrors the exact movements of the wearer.

"How we move our hands and fingers often tells about our health condition," said Yin Li, assistant professor of biostatistics and medical informatics at the University of Wisconsin, Madison School of Medicine and Public Health, who contributed to the software behind FingerTrak.

"A device like this might be used to better understand how the elderly use their hands in daily life, helping to detect early signs of diseases like Parkinson's and Alzheimer's."

The researchers say that, aside from such impressive health tech applications, this device could really push sign language translation into new areas, as it allows incredibly accurate hand tracking on an easy-to-use device.