Fourier Theory of CW Time-of-Flight Imaging

Time-of-flight (ToF) based 3D imaging in scenes with multipath interference (interreflections) and scattering (e.g., smoke and fog). Frequency domain analysis of transient light transport. Applications in autonomous transportation, 3D mapping and robotic automation.

Time-of-flight (ToF) cameras are fast becoming the method of choice for depth sensing in a wide range of applications. In correlation-based time-of-flight (C-ToF) imaging systems, light sources with temporally varying intensities illuminate the scene. Due to global illumination (multi path interference), the temporally varying radiance received at the sensor is a combination of light received along multiple paths. Recovering scene properties (e.g., scene depths) from the received radiance requires separating these contributions, which is challenging due to the complexity of global illumination and the additional temporal dimension of the radiance.

We propose phasor imaging, a framework for performing fast inverse light transport analysis using C-ToF sensors. Phasor imaging is based on the idea that by representing light transport quantities as phasors and light transport events as phasor transformations, light transport analysis can be greatly simplified in the temporal frequency domain. We study the dependence of light transport on the illumination frequency, and show that for a broad range of scenes, global radiance vanishes for frequencies higher than a scene-dependent threshold. We use this observation for developing two novel scene recovery techniques. First, we present Micro ToF imaging, a ToF based shape recovery technique that is robust to errors due to global illumination (multi-path interference). Second, we present a technique for separating the direct and global components of radiance. Both techniques require capturing as few as 3-4 images and minimal computations. We demonstrate the validity of the presented techniques via simulations and experiments performed with our hardware prototype.

Publications

Phasor Imaging: A Generalization of Correlation ToF Imaging

Mohit Gupta, Shree Nayar, Matthias Hullin, Jaime Martin

ACM TOG, presented at SIGGRAPH, 2015

Phasor Light Transport

Using phasors, all the light transport events in C-ToF imaging can be represented by linear operations on complex numbers. Propagation of a light ray changes the phase of the phasor radiance. Local reflection and scattering events change only the amplitude of the radiance. Multiple rays at the same point in space traveling in the same direction can be represented by a single ray whose radiance is the complex sum of the radiance of individual rays.

Frequency Dependence of Global Illumination

If the light source emits high frequency illumination, the individual indirect phasor radiances span the complete circle, and the resultant (total global radiance) is zero.

Transport Robust Shape Recovery

The proposed Micro ToF imaging technique consists of illuminating the scene sequentially with multiple high frequency sinusoids, and computing phases corresponding to each of them. If all the frequencies are sufficiently high, global illumination does not introduce errors in the phases. The individual phases have depth ambiguities. Unambiguous depth is recovered by unwrapping the phases, which can be done either analytically or by building a look-up table.

Direct Global Separation

If the scene is illuminated with high-frequency illumination, the direct and global components can be computed by measuring the offset and the amplitude at every pixel. This requires taking as few as three measurements.

Hardware Prototype

Our hardware prototype is based on the PMDTechnologies CamBoard Nano, a correlation-based time-of-flight (C-ToF) sensor. An array of 650 nm laser diodes acts as the light source.

Experimental Results of Depth Estimation

We show depth estimation results on a scene consisting of a fixed wall and a movable wall arranged at an angle so that they form a v-groove. The apex angle can be changed by moving the right wall.