Mantis Shrimp-Inspired Camera Lets Self-driving Cars See in Dangerous Conditions

By mimicking the vision of a mantis shrimp, a new bio-inspired camera can spot hazards even in the most challenging road conditions.
Kashyap Vyas

Bio-inspired cameras are a new type of camera developed by researchers at the University of Illinois. According to them, it can immensely improve how self-driving cars function at the moment.


It can easily empower the cars to detect other cars, people or even detect hazards that are three times farther as compared to the current color cameras that are used in the cars. Moreover, these cameras can be mass-produced at the cost as low as $10 per piece.

It can be a revolutionary step in the next-generation of sensors. Mantis Shrimps completely inspired these cameras. “Our new bioinspired camera shows that nature has a lot of interesting solutions that we can take advantage of for designing next-generation sensors,” said Missael Garcia, first author of the paper.

In any case, these cameras can immediately help self-driven cars in making them more reliable and less prone to accidents.

According to the team leader of this project, Viktor Gruev of the University of Illinois at Urbana-Champaign USA, “In a recent crash involving a self-driving car, the car failed to detect a semi-truck because of its color and light intensity blended with that of the sky in the background. Our camera can solve this problem because its high dynamic range makes it easier to detect objects that are similar to the background and the polarization of a truck is different than that of the sky.”

The visual system of a Mantis Shrimp                       

The eyes of Mantis Shrimps allow them to distinguish between wavelengths of about 1-5 nanometers. They have a logarithmic response to the intensity of light.

Inspired by this, the researchers of the bio-inspired camera tweaked the way the photodiodes of the camera converts light into an electrical current. They used the forward bias mode which made the output to have a logarithmic response like the shrimp.

Moreover, the researchers also mimicked the way the shrimps integrate the process of polarized light detection into their photoreceptors through the course of depositing nanomaterials into their imaging chip. 

“These nanomaterials essentially act as polarization filters at the pixel level to detect polarization in the same way that the mantis shrimp sees polarization,” Gruev added.

Usage of the camera in self-driven cars

The researchers tested the camera with various light intensities, polarization, and colors in the lab and it gave impressive results. So, they went on to test it in the field to see its functions in bright light or shadow.

A member of the research team, Tyler Davis, said that “We used the camera under different driving lighting conditions such as tunnels or foggy conditions. The camera handled these challenging imaging conditions without any problems.” Hence, it passed all the tests, and now the researchers are trying to see if they can detect objects well in advance to prevent any possible accidents or link it to the airbags so that they can be deployed a few milliseconds earlier and ensure complete safety.