This New 4D Camera Could Soon Boost Robot Vision to the Next Level

Stanford has created a 4D camera that can capture 140 degrees of information. The new technology would be the perfect addition to robots and autonomous vehicles. The 4D camera relies on light field photography which allows it to gather such a wide degree of information.

Light field camera, or standard plenoptic camera, works by capturing information about the light field emanating from the scene. It measures the intensity of the light in the scene and also the direction that the light rays travel. Traditional photography only captures the light intensity.  

[Image Source: L.A. Cicero/Stanford]

The researchers proudly call their design to be the "first-ever single-lens, wide field of view, light field camera." The camera uses the information it has gathered about the light at the scene in combination with the 2D image to create the 4D image.  

This means the photo can be refocused after the image has been captured. The researchers cleverly use the analogy of the difference between looking out a window and through a peephole to describe the difference between the traditional photography and the new technology. They say, “"A 2D photo is like a peephole because you can't move your head around to gain more information about depth, translucency or light scattering'. Looking through a window, you can move and, as a result, identify features like shape, transparency and shininess."

[Image Source: DonaldGDansereau]

The 4D camera’s unique qualities make it perfect for use with robots.  For instance, images captured by a search and rescue robot could be zoomed in and refocused to provide important information to base control. The imagery produced by the 4D camera could also have application in augmented reality as the information rich images could help with better quality rendering.  

[Image Source: DonaldGDansereau]

This is the first time a camera has been designed for specific use with robot technology, rather than applying standard cameras to robots.  

The super detailed and wide lens imagery is gained by using a spherical lens. The problem in past with these types of lenses was using it in combination with a flat sensor. The Stanford scientists took a new approach to this problem and formed a collaborative team that combines optics and fabrication knowledge from the University of California San Diego with the algorithmic expertise from the Stanford Computational Imaging Lab to solve the problem.

[Image Source: DonaldGDansereau]

The 4D camera is still at a  proof-of-concept stage, and too big for any of the future possible applications. But now the technology is at a working stage, smaller and lighter versions can be developed. The researchers explain the motivation for creating a camera specifically for robots. Donald Dansereau, a postdoctoral fellow in electrical engineering explains, “We want to consider what would be the right camera for a robot that drives or delivers packages by air. We’re great at making cameras for humans but do robots need to see the way humans do? Probably not."

The research will be presented at the computer vision conference, CVPR 2017 on July 23.

Sources: Stanford Computational Imaging Lab, Engadget, DonaldGDansereau,  Stanford News Service

SEE ALSO: This Adorable Robotic Camera Is the Latest Member of the International Space Station