Time-of-flight camera

A time-of-flight camera (ToF camera), also known as time-of-flight sensor (ToF sensor), is a range imaging camera system for measuring distances between the camera and the subject for each point of the image based on time-of-flight, the round trip time of an artificial light signal, as provided by a laser or an LED. Laser-based time-of-flight cameras are part of a broader class of scannerless LIDAR, in which the entire scene is captured with each laser pulse, as opposed to point-by-point with a laser beam such as in scanning LIDAR systems. Time-of-flight camera products for civil applications began to emerge around 2000, as the semiconductor processes allowed the production of components fast enough for such devices. The systems cover ranges of a few centimeters up to several kilometers.

Types of devices Several different technologies for time-of-flight cameras have been developed.

RF-modulated light sources with phase detectors Photonic Mixer Devices (PMD), the Swiss Ranger, and CanestaVision work by modulating the outgoing beam with an RF carrier, then measuring the phase shift of that carrier on the receiver side. This approach has a modular error challenge: measured ranges are modulo the RF carrier wavelength. The Swiss Ranger is a compact, short-range device, with ranges of 5 or 10 meters and a resolution of 176 x 144 pixels. With phase unwrapping algorithms, the maximum uniqueness range can be increased. The PMD can provide ranges up to 60 m. Illumination is pulsed LEDs rather than a laser. More recent CW-ToF camera systems illuminate the scene with high-frequency modulated LED light and analyze the phase shift of the returning signal at each pixel to compute depth. For example, in traffic enforcement applications, retroreflective surfaces such as license plates and vehicle reflectors generate strong return signals that are used to construct depth images over time. These images allow tracking of vehicle positions in 3D space and calculation of speed by applying regression analysis to the position-time data. Unlike conventional RADAR, this method measures speed along the vehicle's true direction of travel and is independent of the vehicle’s distance and angle relative to the camera. In some continuous-wave ToF systems, depth images captured over successive time intervals are used to estimate the 3D positions of moving objects, such as vehicles. The system tracks multiple retroreflective points across consecutive frames and reconstructs the object’s trajectory through 3D space. By applying regression analysis to the change in position over time, the system accurately determines the object's speed along its path of travel. Unlike conventional RADAR, this approach minimizes errors associated with distance and angle to the target. CanestaVision developer Canesta was purchased by Microsoft in 2010. The Kinect2 for Xbox One was based on ToF technology from Canesta.

Range gated imagers These devices have a built-in shutter in the image sensor that opens and closes at the same rate as the light pulses are sent out. Most time-of-flight 3D sensors are based on this principle invented by Medina. Because part of every returning pulse is blocked by the shutter according to its time of arrival, the amount of light received relates to the distance the pulse has traveled. The distance can be calculated using the equation, z = R (S2 − S1) / 2(S1 + S2) + R / 2 for an ideal camera. R is the camera range, determined by the round trip of the light pulse, S1 the amount of the light pulse that is received, and S2 the amount of the light pulse that is blocked. The ZCam by 3DV Systems is a range-gated system. Microsoft purchased 3DV in 2009. Microsoft's second-generation Kinect sensor was developed using knowledge gained from Canesta and 3DV Systems. Similar principles are used in the ToF camera line developed by the Fraunhofer Institute of Microelectronic Circuits and Systems and TriDiCam. These cameras employ photodetectors with a fast electronic shutter. The depth resolution of ToF cameras can be improved with ultra-fast gating intensified CCD cameras. These cameras provide gating times down to 200ps and enable ToF setup with sub-millimeter depth resolution. Range gated imagers can also be used in 2D imaging to suppress anything outside a specified distance range, such as to see through fog. A pulsed laser provides illumination, and an optical gate allows light to reach the imager only during the desired time period.

Direct Time-of-Flight imagers These devices measure the direct time-of-flight required for a single laser pulse to leave the camera and reflect back onto the focal plane array. Also known as "trigger mode", the 3D images captured using this methodology image complete spatial and temporal data, recording full 3D scenes with single laser pulse. This allows rapid acquisition and rapid real-time processing of scene information. For time-sensitive autonomous operations, this approach has been demonstrated for autonomous space testing and operation such as used on the OSIRIS-REx Bennu asteroid sample and return mission and autonomous helicopter landing. Advanced Scientific Concepts, Inc. provides application specific (e.g. aerial, automotive, space) Direct TOF vision systems known as 3D Flash LIDAR cameras. Their approach utilizes InGaAs Avalanche Photo Diode (APD) or PIN photodetector arrays capable of imaging laser pulse in the 980 nm to 1600 nm wavelengths.

Components A time-of-flight camera consists of the following components:

… excerpt ends here. Continue reading the full article.