Tesla Autopilot system has been under scrutiny many times to see how it sees the world especially when it makes a mistake. A recent video released shows an in-depth look at the radar and ultrasonic sensor technology which is used with cameras installed throughout the car. This is so far the best view which has been achieved to see how the Autopilot sees when passing through the Parisian streets. Verygreen and DamianXVI have uploaded the video. They created the video by downloading an Autopilot Hardware 2.5 computer on eBay and hacked into a fully-unlocked development version. Vverygreen explained the importance of video in a Tesla subreddit saying that it is useful especially for those who are fascinated by Tesla’s ever-changing Autopilot functionality.
The host of the video explained, “So keep in mind our visualizations are not what Tesla devs see out of their car footage, and we do not fully understand all the values either (though we have decent visibility into the system now as you can see). Since we don’t know anybody inside Tesla development, we don’t even know what sort of visual output their tools have.”
Verygreen has also given a simple explanation to tell about the different colors. Each color represents a type of object. However since Tesla hasn’t explained anything about it, the uploaders have made guesses of what the colors might mean. Verygreen said, “The green fill at the bottom represents “possible driving space,” lines denote various detected lane and road boundaries (colors represent different types, the actual meaning is unknown for now). Various objects detected are enumerated by type and have coordinates in 3D space and depth information (also 2D bounding box, but we have not identified enough data for a 3D one), correlated radar data (if present) and various other properties.” An important thing to note in the video is that it features a human driver. Whatever the Autopilot is registering is mostly based on the research done by Verygreen and Damian.
