Eulerian Single-Photon Vision

Single-photon sensors measure light signals at the finest possible resolution — individual photons. These sensors introduce two major challenges in the form of strong Poisson noise and extremely large data acquisition rates, which are also inherited by downstream computer vision tasks. Previous work has largely focused on solving the image reconstruction problem first and then using off-the-shelf methods for downstream tasks, but the most general solutions that account for motion are costly and not scalable to large data volumes produced by single-photon sensors.

This work forgoes the image reconstruction problem. Instead, we demonstrate computationally light-weight phase-based algorithms for the tasks of edge detection and motion estimation. These methods directly process the raw single-photon data as a 3D volume with a bank of velocity-tuned filters, achieving speed-ups of more than two orders of magnitude compared to explicit reconstruction-based methods.

Code available on this page and on GitHub: https://github.com/shantanu-gupta/ESPV. Also includes 8 real SwissSPAD2 sequences used for the figures in the paper (after some temporal averaging).

Publications

Eulerian Single-Photon Vision

Shantanu Gupta, Mohit Gupta

Proc. ICCV 2023

Quanta Burst Photography

Sizhuo Ma, S Gupta, Arin C. Ulku, Claudio Bruschini, Edoardo Charbon, Mohit Gupta

Proc. SIGGRAPH 2020 (ACM Trans. on Graphics)

SIGGRAPH technical papers highlights

University news coverage

Project Overview

Short presentation and video clips corresponding to result figures in paper.

Resources

Presentation Slides

Source Code

Video

Comments

Share This Article