Tesla Autopilot, an advanced driver-assistance system (ADAS) for Tesla vehicles, uses a suite of sensors and an onboard computer. It has undergone several hardware changes and versions since 2014, most notably moving to an all-camera-based system by 2023, in contrast with ADAS from other companies, which generally include radar and/or lidar sensors. Initially, the ADAS used a combination of cameras capturing the visual spectrum, forward-facing radar, ultrasonic proximity sensors, and a Mobileye EyeQ3 computer as Hardware 1, fitted to Model S vehicles starting in October 2014. After Mobileye ended its partnership with Tesla in 2016, Tesla began shipping cars equipped with an Nvidia Drive PX 2 computer and an increased number of cameras as Hardware 2. In 2019, Tesla shifted to a computer using a custom "FSD Chip" designed by Tesla, branded as Hardware 3. Starting in 2021, Tesla stopped installing the radar sensor in new vehicles, and the ADAS was updated to drop radar support. In 2022, Tesla announced it also would drop support for the ultrasonic sensors, moving the ADAS to an all-visual system. The most recent sensor and computer implementation is Hardware 4, which began shipping in January 2023.
Hardware versions
Hardware 1
Vehicles manufactured after late September 2014 are equipped with a single camera mounted at the top of the windshield, forward looking radar in the lower grille, and 12 ultrasonic acoustic location sensors in the front and rear bumpers that provide a 360-degree view around the car. The computer is the Mobileye EyeQ3; as implemented, this chip is built on a 40 nm process with a TDP of 2.5 W and a clock speed of 500 MHz. This equipment allows suitably equipped Tesla Model S and Model X vehicles to detect lane markings, obstacles, and other vehicles, enabling advanced driver-assistance functions branded Autosteer (automatic lane-keeping), Auto lane change, Autopark (parallel parking robot), and Side-collision warning. The EyeQ3 used a neural network approach, relying primarily on inputs from the camera to recognize and label objects to determine which potential areas in the camera's field of view are unoccupied and possible for the vehicle to travel through. Auto lane change can be initiated by the driver turning on the lane changing signal when safe (due to the ultrasonic sensors' 5-metre limited range capability), and then the system completes the lane change. In 2016 the system did not detect pedestrians or cyclists, and while Autopilot detects motorcycles, there have been two instances of HW1 cars rear-ending motorcycles. Tesla released a new version of Autopilot in September 2016 that changed the object detection algorithm to more fully use the radar sensor. Previously, primary obstacle detection responsibilities fell to the cameras and the radar was used in a secondary role to confirm their presence, but was not given the authority to initiate emergency braking alone. After the update, the radar data was given an equal role in object detection and made capable of identifying "dense obstacles," including "other vehicles, moose, or even alien spaceships," according to Musk. He added that Tesla "believes it would have" prevented the fatal May 2016 underride crash in Williston, Florida, in which Autopilot failed to detect a white trailer against the sky. Mobileye ended its partnership with Tesla in 2016, stating that Tesla was "going to hurt the interests of [Mobileye] and hurt the interests of an entire industry, if a company of our reputation will continue to be associated with this type of pushing the envelope in terms of safety". Tesla responded that Mobileye backed away after learning Tesla was developing its own vision-based sensor system. Speculation immediately following the announcement included partnering with Nvidia and potentially designing its own ADAS computer. After Hardware 2 was released, an upgrade from Hardware 1 to Hardware 2 was not offered as it would have required substantial work and cost.
Hardware 2
HW2, included in vehicles manufactured after October 2016, includes an Nvidia Drive PX 2 computer. Tesla claimed that the hardware was capable of processing 200 frames per second. Elon Musk called HW2 "basically a supercomputer in a car", referring to its ability to perform up to 12 trillion floating point operations per second (TFlops), a performance similar to a $700 GTX 1080 Ti consumer graphics card at the time. The Autopilot computer hardware, housed just above the glovebox, is replaceable to allow for future upgrades. Tesla claimed the HW2 suite of sensors and computation provided the necessary equipment to allow FSD at SAE level 5.
The hardware includes eight cameras covering an aggregate view of 360° around the car and 12 ultrasonic sensors, in addition to forward-facing radar with enhanced processing capabilities. The radar is able to observe beneath and ahead of the vehicle in front of the Tesla; the radar can see vehicles through heavy rain, fog or dust. The eight cameras are mounted in various locations around the vehicle: three forward-facing, next to the central rearview mirror mounted on the windshield; two front/side cameras, one each mounted in the left and right B-pillars; two rear/side cameras, mounted in the left and right front fender turn-signal repeaters; and one rear camera, above the license plate. When "Enhanced Autopilot" was enabled in February 2017 by the v8.0 (17.5.36) software update, testing showed the system was limited to using one of the eight onboard cameras—the main forward-facing camera. The v8.1 software update released a month later enabled a second camera, the narrow-angle forward-facing camera. With all eight cameras enabled, data extracted from Autopilot in debugging mode showed the cameras provide a black-and-white feed to the computer, possibly to improve image processing speed. The Tesla Model 3, introduced in 2017, and related Model Y, introduced in 2019, are equipped with an additional driver-facing in-cabin camera. This was disabled at launch and was intended to monitor the cabin remotely while the owner was operating the vehicle as an autonomous robotaxi, but was activated in May 2021 to monitor driver attentiveness while using Autopilot in vehicles without radar sensors.
Hardware 2.5
… excerpt ends here. Continue reading the full article.
