Discover the Streets of Paris Through the 'Eyes' of Tesla Autopilot

This video from a Tesla hacker gives us the best views yet of how the company's Autopilot visualizes its surroundings.
Shelby Rogers

How Tesla's Autopilot system registers the world around it has been under scrutiny, particularly when it makes a mistake. However, this video gives viewers an in-depth look at the radar and ultrasonic sensor technology used in conjunction with the cameras throughout the car. 

This could be the best view yet at how the Autopilot sees with this drive through Parisian streets. 

Uploader 'verygreen' and TMC user DamianXVI collaborated on the upload. The two created the video by downloading a Autopilot Hardware 2.5 computer on eBay and hacked into a fully-unlocked development version.

verygreen explained on a Tesla subreddit the importance of the video, particularly for those fascinated by Tesla's ever-changing Autopilot functionality. 

"So keep in mind our visualizations are not what Tesla devs see out of their car footage and we do not fully understand all the values either (though we have decent visibility into the system now as you can see)," they explained. "Since we don’t know anybody inside Tesla development, we don’t even know what sort of visual output their tools have."

If you're wondering what those colors mean, verygreen also breaks down a simple explanation. Each color represent types of objects, but the uploaders have to make estimated guesses as Tesla hasn't explained the nuances associated with each color to the public. 

“The green fill at the bottom represents “possible driving space”, lines denote various detected lane and road boundaries (colors represent different types, actual meaning is unknown for now)," verygreen said. "Various objects detected are enumerated by type and have coordinates in 3D space and depth information (also 2D bounding box, but we have not identified enough data for a 3D one), correlated radar data (if present) and various other properties.”

It's important to note this particular video features a human driver. What the Autopilot registers is largely based on the research done by verygreen and Damian. 

Discover the Streets of Paris Through the 'Eyes' of Tesla Autopilot 

Add Interesting Engineering to your Google News feed.
Add Interesting Engineering to your Google News feed.
message circleSHOW COMMENT (1)chevron
Job Board