PF-SPAD vs. Conventional CCD or CMOS Pixel

From One Photon to a Billion: High Flux Passive Imaging with Single-Photon Sensors

Photon-counting achieves several orders of magnitude higher dynamic range than conventional CCD or CMOS pixels

Image sensors capable of counting individual photons are conventionally used in specialized low photon flux applications such as LiDAR and fluorescence lifetime imaging microscopy. Here we present a new imaging modality that enables single-photon avalanche diode (SPAD) sensors to operate at extremely high photon flux levels and hence provide unprecedented dynamic range. We call this technique passive free-running SPAD or PF-SPAD. Our scene brightness estimator uses the number of photons detected by a dead time-limited SPAD sensor over a fixed exposure time. Our key insight is that due to the random nature of photon arrivals, a PF-SPAD does not suffer from a hard saturation limit, even for extremely bright scene points. Coupled with high sensitivity at low flux, this enables a PF-SPAD to achieve extreme dynamic range. We present theoretical analysis, simulation results and experimental demonstration of an improvement of over 2 orders of magnitude over conventional sensors by imaging scenes spanning a dynamic range of 1,000,000:1. We envision that this will expand the applicability of SPADs as general-purpose passive image sensors for a broad range of applications including consumer photography, computer vision and microscopy.

Publications

High Flux Passive Imaging with Single-Photon Sensors

Atul Ingle, A Velten, Mohit Gupta

Proc. CVPR 2019

oral presentation

pdf
@INPROCEEDINGS{Ingle_CVPR2019,
author={Atul Ingle and Andreas Velten and Mohit Gupta},
booktitle={Proc. CVPR},
title={High Flux Passive Imaging with Single Photon Sensors},
year={2019},
month={June},}

Passive Inter-Photon Imaging

Atul Ingle, Trevor Seets, Mauro Buttafava, Shantanu Gupta, Alberto Tosi, Mohit Gupta*, Andreas Velten*

Proc. CVPR 2021

oral presentation

pdf
@inproceedings{Ingle_CVPR_21,
title={Passive Inter-Photon Imaging},
author={Ingle, Atul and Seets, Trevor and Buttafava, M and Gupta, Shantanu and Tosi, Alberto and Velten, Andreas and Gupta, Mohit},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
pages={},
year={2021}
}

Quanta Burst Photography

Sizhuo Ma, S Gupta, Arin C. Ulku, Claudio Bruschini, Edoardo Charbon, Mohit Gupta

Proc. SIGGRAPH 2020

SIGGRAPH technical papers highlights

pdf
@article{Ma_SIGGRAPH20,
title = “Quanta Burst Photography”,
author = “Ma, Sizhuo and Gupta, Shantanu and Ulku, Arin C. and Brushini, Claudio and Charbon, Edoardo and Gupta, Mohit”,
journal = “ACM Transactions on Graphics (TOG)”,
doi = “10.1145/3386569.3392470”,
volume = “39”,
number = “4”,
year = “2020”,
month = “7”
publisher = “ACM”
}

Video of Method and Results

Experimental Result - Tunnel Scene

A tunnel scene with a dynamic range of over 1,000,000:1. A conventional CMOS camera cannot simultaneously capture the bright traffic sign outside the tunnel and darker speed limit sign inside the tunnel. Two different exposure times are required to capture these extreme illumination conditions. Our PF-SPAD prototype can captures both extremes in a single exposure.

Experimental Result - Tabletop Scene

A tabletop scene. The bright filament appears saturated when imaged with a conventional CMOS camera with a long exposure time. Lowering the exposure time makes the darker text on the clock grainy and illegible. In a single exposure the PF-SPAD can simultaneously capture the bright bulb filament and the dark text on the alarm clock.

Hardware Prototype

Single-pixel PF-SPAD hardware prototype. The SPAD is mounted on two translation stages to raster-scan the image plane. There is no active light source. The PF-SPAD passively measures photon flux for each scene point. Photon counts are captured using a single-photon counting module operated without a synchronization signal.