Single-Photon Structured Light

The enabling technologies behind Single-Photon SL are single-photon cameras (e.g. SPAD arrays) and digital micromirror devices (DMD). We operate both devices at speeds as high as 20,000 Hz.

Such rapidly acquired binary frames are sufficient for a large class of SL techniques, facilitating high precision at low acquisition times.

Notably, when operating at such extreme speeds, read noise becomes an important consideration. Even the read noise floor of state-of-the-art high-speed cameras result in imperceptible signals.

Unlike existing high-speed cameras, single-photon cameras such as SPADs directly detect the arrival of photons and are not corrupted by read noise.

Since SPAD sensors have no read-noise, the predominant source of noise is the random arrival of photons, called photon noise.

This can lead to errors, or bit-flips, such as when a bright pattern is projected but the SPAD detects no photons and outputs a '0'. Bit-flips can also occur when a dark pattern is projected but the SPAD outputs a '1'.

A naïve strategy to mitigate photon noise is to simply repeat projected patterns and perform a majority vote.

A more robust and principled strategy is to append parity patterns which can be used to correct errors that arise from bit-flips. Such parity patterns are derived using binary error-correcting codes such as the BCH codes.

Unfortunately, a majority of the parity patterns feature high-spatial frequency. These can be easily distorted by non-idealities that commonly occur in practical SL systems such as projector and camera defocus, and dramatic resolution mismatch between the two devices.

To provide robustness to these 'short-range effects', we devise novel hierarchical codes using a combination of error correction and binary shifting.

Our lab prototype of Single-Photon SL uses the SwissSPAD2 array, a 512 x 256 pixel sensor. For a projector, we employ the DLP6500 DMD which can be operated at 20,000 Hz and a resolution of 1024 x 768 pixels.

We validate the effectiveness of the proposed coding schemes by first considering static scenes, for which groundtruth measurements can be obtained.

The proposed hybrid strategy, robust to both photon noise and short-range effects, is considerably more consistent and accurate than both naïve and BCH strategies.

Given a fixed acquisition time and power requirement, low-albedo objects are often a challenge for structured light systems as they return only a small fraction of incident photons.

However, the low-light imaging capabilities of SPAD arrays coupled with the error tolerant coding schemes proposed allows Single-Photon SL recover reconstructions of commonplace dark objects. Here, we show the 3D scan of a car tire with its tread pattern clearly visible.

As an extreme example, we also show that Single-Photon SL can recover the exact 3D geometry of objects covered with very low-albedo materials.

Shown here is an inverted V-groove covered by Acktar Metal velvet, which returns just 0.1% of incident illumination. Evidently, the highly absorptive nature of this material makes it difficult to perceive depth from the reference image alone.

Capturing a dynamic non-rigid object, such as a white cloth rapidly waved in front of the camera, requires high-speed acquisition. Otherwise, excessive motion blur is induced.

As the depth video and mesh reconstructions demonstrate, Single-Photon SL can recover detailed 3D geometry of non-rigid objects.

Another scene with extremely fast motion--a jack-in-the-box toy which springs up in less than 40 milliseconds.

A conventional depth sensor such as the Kinect operates at 30 FPS and is too slow to recover depth at intermediate instants.

However, Single-Photon SL can run at a significantly faster 2000 FPS and recovers finer-details of the toy as it springs up, such as its ears.