High Flux Passive Imaging with Single-Photon Sensors
Proc. CVPR 2019
Photon-counting achieves several orders of magnitude higher dynamic range than conventional CCD or CMOS pixels
Image sensors capable of counting individual photons are conventionally used in specialized low photon flux applications such as LiDAR and fluorescence lifetime imaging microscopy. Here we present a new imaging modality that enables single-photon avalanche diode (SPAD) sensors to operate at extremely high photon flux levels and hence provide unprecedented dynamic range. We call this technique passive free-running SPAD or PF-SPAD. Our scene brightness estimator uses the number of photons detected by a dead time-limited SPAD sensor over a fixed exposure time. Our key insight is that due to the random nature of photon arrivals, a PF-SPAD does not suffer from a hard saturation limit, even for extremely bright scene points. Coupled with high sensitivity at low flux, this enables a PF-SPAD to achieve extreme dynamic range. We present theoretical analysis, simulation results and experimental demonstration of an improvement of over 2 orders of magnitude over conventional sensors by imaging scenes spanning a dynamic range of 1,000,000:1. We envision that this will expand the applicability of SPADs as general-purpose passive image sensors for a broad range of applications including consumer photography, computer vision and microscopy.
Proc. CVPR 2019
Proc. CVPR 2021
Proc. SIGGRAPH 2020 (ACM Trans. on Graphics)
SIGGRAPH technical papers highlights
A tunnel scene with a dynamic range of over 1,000,000:1. A conventional CMOS camera cannot simultaneously capture the bright traffic sign outside the tunnel and darker speed limit sign inside the tunnel. Two different exposure times are required to capture these extreme illumination conditions. Our PF-SPAD prototype can captures both extremes in a single exposure.
A tabletop scene. The bright filament appears saturated when imaged with a conventional CMOS camera with a long exposure time. Lowering the exposure time makes the darker text on the clock grainy and illegible. In a single exposure the PF-SPAD can simultaneously capture the bright bulb filament and the dark text on the alarm clock.
Single-pixel PF-SPAD hardware prototype. The SPAD is mounted on two translation stages to raster-scan the image plane. There is no active light source. The PF-SPAD passively measures photon flux for each scene point. Photon counts are captured using a single-photon counting module operated without a synchronization signal.