Augmented Reality Helps the Researchers to Understand the Active Sensing in Fish

A new study using augmented reality is a stepping stone to showing how active sensing helps fish understand their surroundings.
Joseph Wolkin

Animals have an amazing sense of where they are. It makes you wonder sometimes just how incredible their inherent abilities truly are.

That's exactly what researchers from NJIT and Johns Hopkins were thinking when they wanted to test how the brain controls active sensing.


What is most exciting is that this study has allowed us to explore feedback in ways that we have been dreaming about for over 10 years,” Eric Fortune, associate professor of biology, who led the study, said.

“This is perhaps the first study where augmented reality has been used to probe, in real time, this fundamental process of movement-based active sensing, which nearly all animals use to perceive the environment around them.”

The research, published in the Current Biology journal, show there are indeed small movements of active sensing in the glass knifefish. They believe it uses sensory feedback control to improve the fish's sensory information.

This means that this specific fish, and likely many other animals, utilize a dual-control system to create active sensing movements. 

What Does This Mean for Animals?

“We’ve known for a long time that these fish will follow the position of their refuge, but more recently we discovered that they generate small movements that reminded us of the tiny movements that are seen in human eyes,” Fortune said.

“That led us to devise our augmented reality system and see if we could experimentally perturb the relationship between the sensory and motor systems of these fish without completely unlinking them. Until now, this was very hard to do.”

The augmented reality system Fortune is talking about is one widely used in research. Researchers placed artificial intelligence with the electric fish in a tank. 

Most Popular

From there, the researchers tracked how the fish moved. The fish that would move the most to get sensory information were doing so because they geared up the augmented reality to do so.

It's likely that the pair of control loops the researchers found in the fish are common in other animals -- in the sea and on the Earth.

One of the loops controls the information flowing in and out of active sensing movements. The other one captures the data and informs the motor function of what to do next.

In the future, this could lead to finding specific neurons that take over the control loops in not only this fish, but all animals. 

Eventually, this research can be applied to finding out information about the active sensing behavior in people.

message circleSHOW COMMENT (1)chevron