Handling Multi-ToF-Camera Interference

Novel approach for mitigating multi-ToF-Camera interference by reducing both AC and DC interference

Multi-camera interference (MCI) is an important challenge faced by continuous-wave time-of-flight (C-ToF) cameras. In the presence of other cameras, a C-ToF camera may receive light from other cameras’ sources, resulting in potentially large depth errors. We propose stochastic exposure coding (SEC), a novel approach to mitigate MCI. In SEC, the camera integration time is divided into multiple time slots. Each camera is turned on during a slot with an optimal probability to avoid interference while maintaining high signal-to-noise ratio (SNR). The proposed approach has the following benefits. First, SEC can filter out both the AC and DC components of interfering signals effectively, which simultaneously achieves high SNR and mitigates depth errors. Second, time-slotting in SEC enables 3D imaging without saturation in the high photon flux regime. Third, the energy savings due to camera turning on during only a fraction of integration time can be utilized to amplify the source peak power, which increases the robustness of SEC to ambient light. Lastly, SEC can be implemented without modifying the C-ToF camera’s coding functions, and thus, can be used with a wide range of cameras with minimal changes. We demonstrate the performance benefits of SEC with thorough theoretical analysis, simulations and real experiments, across a wide range of imaging scenarios.

Publications

Mitigating AC and DC Interference in Multi-ToF-Camera Environments

Jongho Lee, Mohit Gupta

IEEE Trans. on PAMI 2023

pdf
@article{Lee_PAMI23,
title = “Mitigating AC and DC Interference in Multi-ToF-Camera Environments”,
author = “Jongho Lee and Mohit Gupta”,
journal = “Transactions on Pattern Analysis and Machine Intelligence (PAMI)”,
year = “2023”,
publisher = “IEEE”
}

When Two Cameras Are a Crowd (Understanding and Handling Interference Across Multiple Active Cameras)

Jongho Lee, Mohit Gupta, Bhuvana Krishnaswamy, Suman Banerjee

Communications of the ACM 2023

Stochastic Exposure Coding for Handling Multi-ToF-Camera Interference

Jongho Lee, Mohit Gupta

Proc. ICCV 2019

oral presentation

pdf
@inproceedings{lee2019stochastic,
title={Stochastic Exposure Coding for Handling Multi-ToF-Camera Interference},
author={Lee, Jongho and Gupta, Mohit},
booktitle={Proceedings of the IEEE International Conference on Computer Vision},
pages={7880–7888},
year={2019}
}

Smart Time-Multiplexing of Quads Solves the Multicamera Interference Problem

T. Pribanic, T. Petkovic, David Bojanic, K. Bartol, Mohit Gupta

Proc. International Conf. on 3D Vision (3DV) 2020

pdf
@INPROCEEDINGS{Pribanic_3DV2020,
author={Tomislav Pribanic and Tomislav Petkovic and David Bojanic and Kristijan Bartol and Mohit Gupta},
booktitle={Proc. 3DV},
title={Smart Time-Multiplexing of Quads Solves the Multicamera Interference Problem},
year={2020},
month={},}

Multi-Camera interference in C-ToF imaging

When several C-ToF cameras capture the same scene concurrently, each sensor may receive light from the sources of other cameras. This interfering signal prevents correct depth estimation, resulting in potentially large, structured errors.

Multi-camera interference and interference reduction in C-ToF imaging

(a) In C-ToF imaging, depths are recovered from the phases of the measured waveforms. (b) If there are multiple cameras, interfering sources corrupt the measured waveforms, resulting in systematic depth errors. (c) Conventional MCI reduction approaches (ACO) reduce systematic errors by removing AC interference, but DC interference remains, resulting in lower SNR and random depth errors due to higher photon noise. (d) Our approaches (SEC and CMB) mitigate both AC and DC interference, thus reducing both systematic and random depth errors.

Concept of stochastic exposure coding (SEC)

A frame, the most basic unit to estimate
the depth, is divided into M number of slots. Each slot is activated with a probability p. A depth value is estimated from non-clashed ON (activated) slots.

Layered C-ToF coding

The proposed approach operates in the exposure coding layer, where the camera and the source are modulated at micro/millisecond time scales. In contrast, existing MCI reduction approaches operate in the lower depth coding layer, where modulation is performed at nanosecond time scales.

3-D model reconstruction over different number of interfering cameras

Our approaches achieve better performance in both subjective and objective quality over different number of interfering cameras N. The RMSE values (in mm) are shown.

Hardware prototype

Front and top views of our setup to implement ACO, SEC, and CMB. The setup consists of four C-ToF cameras and four microcontrollers to generate random binary sequences to activate the cameras by given slot ON probabilities.

Depth estimation comparison over different energy consumption

Our approaches show better performance at lower energy consumption than the conventional approach. The % of inliers (non-black pixels) and RMSE values (in m) at the inliers are represented for comparison between approaches.