New Soft Robotics Tech Uses Shadows to Sense Human Touch

The technique allows soft robots to detect pats, punches, and even hugs without relying on touch.
Chris Young

Though robots are able to carry out a large list of vital actions, and their importance has increased during the pandemic, human-robot interaction is still in its infancy.

With this issue in mind, scientists from Cornell University developed a method for soft robots to identify human touch by analyzing a user's shadows via a camera.

The system, called ShadowSense, utilizes an off-the-shelf USB camera to capture shadow movements produced by hand gestures on a robot's skin.

Algorithms then analyze the movements to distinguish between different types of physical interactions, including punches, pats, and hugs.

Guy Hoffman, study lead and author of a paper detailing the technology, explains that the method allows for interaction with robots, without the need for large, costly sensor arrays.

As full-body touch for robots would typically require a huge amount of sensors, roboticists have thus far deemed it an impractical goal. The new research from Hoffman and his team provides an efficient alternative.

In order to test their system, the researchers used an inflatable robot with a camera under its skin. They trained their physical interaction classification algorithms using shadow images of six separate gestures: palm touching, punching, hugging, touching with two hands, pointing, and not touching.

The soft robot was able to successfully distinguish between the different gestures with 87.5 - 96 percent accuracy, though lighting conditions did have an effect on the outcome.

Most Popular

Potential for privacy-friendly soft robots for the home

Hoffman also says the method of interaction via shadows could also allow for privacy-friendly soft robots and droids for the home.

"If the robot can only see you in the form of your shadow, it can detect what you’re doing without taking high fidelity images of your appearance," Hoffman said in a press release. "That gives you a physical filter and protection, and provides psychological comfort."

The researchers say their technology could be used by mobile guide robots that could respond to taps and pokes from nearby humans, as well as home droids that provide company to the elderly or carry out a set of predetermined tasks.

With digital privacy issues arising worldwide, it's encouraging to see technology with an in-built function that prevents firms from collecting third party data from our homes.

message circleSHOW COMMENT (1)chevron