Publications – Wision Lab Web

2023

CASPI: Collaborative Photon Processing for Single-Photon Imaging

Technology- and application-agnostic, training-free photon processing for high-resolution single-photon cameras

Jongho Lee, Atul Ingle, J V Chacko, K W Eliceiri, Mohit Gupta

Nature Communications 2023

SoDaCam: Software-defined Cameras via Single-Photon Imaging

The data acquired by single-photon sensors can be used to emulate a wide variety of imaging modalities.

Varun Sundar, Andrei Ardelean, Tristan Swedish, C Bruschini, Edoardo Charbon, Mohit Gupta

Proc. ICCV 2023

oral presentation

web
pdf
@inproceedings{Sundar:2023:SoDaCam,
author = {Varun Sundar and Andrei Ardelean and Tristan Swedish and Claudio Bruschini and Edoardo Charbon and Mohit Gupta},
year = {2023},
title = {SoDaCam: Software-defined Cameras via Single-Photon Imaging},
booktitle = {Proceedings of the {IEEE} {International} {Conference} on {Computer} {Vision} ({ICCV})},
}

Eventful Transformers: Leveraging Temporal Redundancy in Vision Transformers

Reducing the cost of vision Transformers for video inference, by exploiting temporal redundancy between subsequent inputs
GitHub: PyTorch

Matthew Dutson, Yin Li, Mohit Gupta

Proc. ICCV 2023

web
pdf
@inproceedings{Dutson_ICCV_2023,
title = {Eventful {Transformers}, Leveraging Temporal Redundancy in Vision {Transformers}},
author = {Dutson, Matthew and Li, Yin and Gupta, Mohit},
booktitle = {Proceedings of the IEEE International Conference on Computer Vision (ICCV)},
year = {2023},
}

Computational 3D Imaging with Position Sensors

Low-bandwidth computational imaging and 3D sensing with a novel class of position sensors

Jeremy Klotz, Mohit Gupta, Aswin Sankaranarayanan

Proc. ICCV 2023

oral presentation

web
pdf
@inproceedings{Klotz:2023:Position3D,
author = {Jeremy Klotz and Mohit Gupta and Aswin Sankaranarayanan},
year = {2023},
title = {Computational 3D Imaging with Position Sensors},
booktitle = {Proceedings of the {IEEE} {International} {Conference} on {Computer} {Vision} ({ICCV})},
}

Learned Compressive Representations for Single-Photon 3D Imaging

A framework to compress the spatio-temporal information of photon timestamp data into a compressive histogram, and a method to integrate the compressive histogram with existing SPAD data denoising models based on 3D CNNs.

Project Page

Felipe Gutierrez-Barragan, Fangzhou Mu, Andrei Ardelean, Atul Ingle, Claudio Bruschini, Edoardo Charbon, Yin Li, Mohit Gupta, Andreas Velten

Proc. ICCV 2023

web
pdf
@inproceedings{Gutierrez:2023:CompressiveSPAD,
author = {Felipe Gutierrez-Barragan and Fangzhou Mu and Andrei Ardelean and Atul N Ingle and Claudio Bruschini and Edoardo Charbon and Yin Li and Mohit Gupta and Andreas Velten},
year = {2023},
title = {Learned Compressive Representations for Single-Photon 3D Imaging},
booktitle = {Proceedings of the {IEEE} {International} {Conference} on {Computer} {Vision} ({ICCV})},
}

Eulerian Single-Photon Vision

Fast low-level vision in low light with single-photon cameras

Shantanu Gupta, Mohit Gupta

Proc. ICCV 2023

web
pdf
@inproceedings{Gupta:2023:Eulerian,
author = {Gupta, Shantanu and Gupta, Mohit},
year = {2023},
title = {Eulerian {Single}-{Photon} {Vision}},
booktitle = {Proceedings of the {IEEE} {International} {Conference} on {Computer} {Vision} ({ICCV})},
}

Panoramas from Photons

High-quality single photon panoramas under fast motion and low-light

Sacha Jungerman, Atul Ingle, Mohit Gupta

Proc. ICCV 2023

web
pdf
@InProceedings{Jungerman_2023_ICCV,
author = {Jungerman, Sacha and Ingle, Atul and Gupta, Mohit},
title = {Panoramas from Photons},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2023},
}

Mitigating AC and DC Interference in Multi-ToF-Camera Environments

Novel approach for mitigating multi-ToF-Camera interference by reducing both AC and DC interference

Jongho Lee, Mohit Gupta

IEEE Trans. on PAMI 2023

Seeing Photons in Color

Color filter and algorithm design for single-photon color imaging in low light

Sizhuo Ma, Varun Sundar, Paul Mos, Claudio Bruschini, Edoardo Charbon, Mohit Gupta

Proc. SIGGRAPH 2023 (ACM Trans. on Graphics)

web
pdf
@article{Ma_SIGGRAPH23,
title = “Seeing Photons in Color”,
author = “Ma, Sizhuo and Sundar, Varun and Mos, Paul and Brushini, Claudio and Charbon, Edoardo and Gupta, Mohit”,
journal = “ACM Transactions on Graphics (TOG)”,
year = “2023”,
publisher = “ACM”
}

Light Codes for Fast Two-Way Human-Centric Visual Communication

A human-centric communication modality that enables fast two-way exchange of visual code-like information among users

Mohit Gupta, Jian Wang, Karl Bayer, Shree Nayar

ACM Trans. on Graphics, presented at SIGGRAPH 2024

web
pdf
@article{Gupta_TOG23,
title = “Light Codes for Fast Two-Way Human-Centric Visual Communication”,
author = “Mohit Gupta and Jian Wang and Karl Bayer and Shree Nayar”,
journal = “ACM Transactions on Graphics (TOG)”,
year = “2023”,
publisher = “ACM”
}

Unlocking the Performance of Proximity Sensors by Transient Histograms

Utilizing Ultra Low-Cost Single-Photon Proximity Sensors for Robotics Manipulation Tasks by Utilizing Transient Histograms

Project Page

Carter Sifferman, Yeping Wang, Mohit Gupta, Michael Gleicher

IEEE Robotics and Automation Letters (RAL)

Proc. IEEE ICRA 2024

When Two Cameras Are a Crowd (Understanding and Handling Interference Across Multiple Active Cameras)

Novel approach for mitigating multi-ToF-Camera interference by reducing both AC and DC interference

Jongho Lee, Mohit Gupta, Bhuvana Krishnaswamy, Suman Banerjee

Communications of the ACM 2023

QfaR: Location-Guided Scanning of Visual Codes from Long Distances

Location-Guided Scanning of Visual Codes from Long Distances

Project Page

Sizhuo Ma, Jian Wang, Wenzheng Chen, Suman Banerjee, Mohit Gupta, Shree Nayar

Proc. ACM MobiCom 2023

web
pdf
@article{Ma_MobiCom23,
title = “QfaR: Location-Guided Scanning of Visual Codes from Long Distances”,
author = “Ma, Sizhuo and Wang, Jian and Chen, Wenzheng and Banerjee, Suman and Gupta, Mohit and Nayar, Shree”,
journal = “ACM MobiCom”,
year = “2023”,
publisher = “ACM”
}

Spike-Based Anytime Perception

Moving beyond frame-based visual recognition, targeting applications with tight latency and power constraints
GitHub: TensorFlow

Matthew Dutson, Yin Li, Mohit Gupta

Proc. WACV 2023

web
pdf
@inproceedings{Dutson_WACV_2023,
title = {Spike-Based Anytime Perception},
author = {Dutson, Matthew and and Li, Yin and Gupta, Mohit},
booktitle = {Proceedings of IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
year = {2023},
}

Burst Vision Using Single-Photon Cameras

Exploring the capabilities of SPAD sensors for a wide gamut of real-world computer vision tasks including object detection, pose estimation, SLAM, text recognition and so on.

Sizhuo Ma, Paul Mos, E Charbon, Mohit Gupta

Proc. WACV 2023

web
pdf
@inproceedings{Ma_WACV_23,
title={Burst Vision Using Single-Photon Cameras},
author={Ma, Sizhuo and Mos, Paul and Charbon, E and Gupta, Mohit},
booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
pages={},
year={2023}
}

2022

Event Neural Networks

Leveraging temporal redundancy for efficient visual recognition on video and other time-series data
GitHub: PyTorch, TensorFlow

Matthew Dutson, Yin Li, Mohit Gupta

Proc. ECCV 2022

web
pdf
@inproceedings{Dutson_ECCV_2022,
title = {Event Neural Networks},
author = {Dutson, Matthew and and Li, Yin and Gupta, Mohit},
booktitle = {Proceedings of the European Conference on Computer Vision},
year = {2022},
}

Fig. 1. Transient histogram: Measurement and examples. (a) Measuring a transient histogram involves illuminating the scene with a pulsed illumination source such as a pulsed laser which is diffused uniformly over the field of view of a time-resolved image sensor pixel such as a single-photon avalanche diode. (b) Transient histograms were captured using a hardware prototype for some basic 3D shapes. Observe that these have unique features which can help distinguish these shapes.

3D Scene Inference from Transient Histograms

Time-critical and resource-constrained 3D scene inference using transient histograms as a primitive scene representation.

Sacha Jungerman, Atul Ingle, Yin Li, Mohit Gupta

Proc. ECCV 2022

web
pdf
@inproceedings{Jungerman_ECCV_22,
title={3D Scene Inference from Transient Histograms},
author={Jungerman, Sacha and Ingle, Atul and Li, Yin and Gupta, Mohit},
booktitle={Proceedings of the European Conference on Computer Vision},
pages={},
year={2022}
}

Robust Scene Inference under Noise-Blur Dual Corruptions

Scene Understanding under Low Light and Motion

Bhavya Goyal, JF Lalonde, Yin Li, Mohit Gupta

Proc. ICCP 2022

Geometric Calibration of Single-Pixel Distance Sensors

Pose estimation for ultra-low cost single-pixel distance sensors for robotics applications

Project Page

Carter Sifferman, Dev Mehrotra, Mohit Gupta, Michael Gleicher

IEEE Robotics and Automation Letters (RAL)

Proc. IEEE IROS 2022

Single-Photon Structured Light

Structured light 3D imaging enabled at extreme speeds and challenging scenarios using single-photon cameras and digital micro-mirror devices.

Varun Sundar, Sizhuo Ma, Aswin Sankaranarayanan, Mohit Gupta

Proc. CVPR 2022

web
pdf
@InProceedings{Sundar_2022_CVPR, author = {Sundar, Varun and Ma, Sizhuo and Sankaranarayanan, Aswin C. and Gupta, Mohit}, title = {Single-Photon Structured Light}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2022}, pages = {17865-17875} }

Compressive Single-Photon 3D Cameras

An online compression framework for SPAD-based 3D cameras based on the coded projection of photon timestamp data.

Project Page

Felipe Gutierrez-Barragan, Atul Ingle, Trevor Seets, Mohit Gupta, Andreas Velten

Proc. CVPR 2022

web
pdf
@InProceedings{Gutierrez_2022_CVPR, author = {Felipe Gutierrez-Barragan and Atul Ingle and Trevor Seets and Mohit Gupta and Andreas Velten}, title = {Compressive Single-Photon 3D Cameras}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2022}, pages = {} }

Single-Photon Camera Guided Extreme Dynamic Range Imaging

Hybrid single-photon and conventional imaging for high-resolution and high dynamic range imaging capabilities

Project Page

Yuhao Liu, Felipe Gutierrez-Barragan, Atul Ingle, Mohit Gupta, Andreas Velten

IEEE WACV 2022

Resolving Motion with Single-Photon Cameras

Single-photon cameras can measure motion robustly in low-light, high-speed scenarios, which can be used to resolve the blur-noise trade-off and reconstruct high-quality images not only for human viewing, but also for subsequent computer vision tasks.

Sizhuo Ma

PhD Thesis

Outstanding Graduate Student Research Award

web
pdf
@phdthesis{10.5555/AAI28964917,
author = {Ma, Sizhuo},
title = {Resolving Motion with Single-Photon Cameras},
year = {2022},
isbn = {9798780614869},
publisher = {The University of Wisconsin – Madison},
}

2021

Photon-Starved Scene Inference using Single Photon Cameras

Scene understanding under low-light conditions

[Code] [Arxiv]

Bhavya Goyal, Mohit Gupta

Proc. ICCV 2021

web
pdf
@InProceedings{Goyal_2021_ICCV,
author = {Goyal, Bhavya and Gupta, Mohit},
title = {Photon-Starved Scene Inference Using Single Photon Cameras},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021},
pages = {2512-2521}
}

Blocks-World Cameras

Novel approach to recover piece-wise planar scenes accurately at low computational cost without creating 3D point clouds

Jongho Lee, Mohit Gupta

Proc. CVPR 2021

oral presentation

web
pdf
@inproceedings{lee2021blocks,
title={Blocks-World Cameras},
author={Lee, Jongho and Gupta, Mohit},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={11412–11422},
year={2021}
}

Passive Inter-Photon Imaging

Inter-photon timing measurements captured by a passive single-photon sensitive camera enable unprecedented dynamic range

Atul Ingle, Trevor Seets, Mauro Buttafava, Shantanu Gupta, Alberto Tosi, Mohit Gupta*, Andreas Velten*

Proc. CVPR 2021

oral presentation

web
pdf
@inproceedings{Ingle_CVPR_21,
title={Passive Inter-Photon Imaging},
author={Ingle, Atul and Seets, Trevor and Buttafava, M and Gupta, Shantanu and Tosi, Alberto and Velten, Andreas and Gupta, Mohit},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
pages={},
year={2021}
}

Invisible Perturbations: Physical Adversarial Examples Exploiting the Rolling Shutter Effect

Physical adversarial perturbations using the rolling shutter effect of cameras

Athena Sayles, Ashish Hooda, Mohit Gupta, Rahul Chatterjee, Earlence Fernandes

Proc. CVPR 2021

web
pdf
@inproceedings{Sayles_CVPR_21,
title={Invisible Perturbations: Physical Adversarial Examples Exploiting the Rolling Shutter Effect},
author={Sayles, Athena and Hooda, Ashish and Gupta, Mohit and Chatterjee, Rahul and Fernandes, Earlence},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
pages={},
year={2021}
}

iToF2dToF: A Robust and Flexible Representation for Data-driven Time-of-Flight Imaging

Project Page

Felipe Gutierrez, Huajin Chen, Mohit Gupta, Andreas Velten, Jinwei Gu

IEEE Transactions on Computational Imaging

2020

Quanta Burst Photography

A computational imaging technique with single-photon cameras enables ultra-low light photography

Sizhuo Ma, S Gupta, Arin C. Ulku, Claudio Bruschini, Edoardo Charbon, Mohit Gupta

Proc. SIGGRAPH 2020 (ACM Trans. on Graphics)

SIGGRAPH technical papers highlights

University news coverage

web
pdf
@article{Ma_SIGGRAPH20,
title = “Quanta Burst Photography”,
author = “Ma, Sizhuo and Gupta, Shantanu and Ulku, Arin C. and Brushini, Claudio and Charbon, Edoardo and Gupta, Mohit”,
journal = “ACM Transactions on Graphics (TOG)”,
doi = “10.1145/3386569.3392470”,
volume = “39”,
number = “4”,
year = “2020”,
month = “7”
publisher = “ACM”
}

Inertial Safety from Structured Light

A novel scene representation that enables fast detection of obstacles in scenarios involving camera or scene motion using single-shot structured light.

Sizhuo Ma, Mohit Gupta

Proc. ECCV 2020

web
pdf
@inproceedings{Ma_ECCV_20,
title={Inertial Safety from Structured Light},
author={Ma, Sizhuo and Gupta, Mohit},
booktitle={Proceedings of the European Conference on Computer Vision},
pages={744-761},
year={2020}
}

Smart Time-Multiplexing of Quads Solves the Multicamera Interference Problem

Novel approach for mitigating multi-ToF-Camera interference by reducing both AC and DC interference

T. Pribanic, T. Petkovic, David Bojanic, K. Bartol, Mohit Gupta

Proc. International Conf. on 3D Vision (3DV) 2020

web
pdf
@INPROCEEDINGS{Pribanic_3DV2020,
author={Tomislav Pribanic and Tomislav Petkovic and David Bojanic and Kristijan Bartol and Mohit Gupta},
booktitle={Proc. 3DV},
title={Smart Time-Multiplexing of Quads Solves the Multicamera Interference Problem},
year={2020},
month={},}

2019

Asynchronous Single-Photon 3D Imaging

Optimal acquisition strategies for single-photon 3D cameras: Towards long-range laser-scan quality 3D imaging

Anant Gupta*, Atul Ingle*, Mohit Gupta

Proc. ICCV 2019

Marr Prize (best paper) honorable mention

oral presentation

web
pdf
@INPROCEEDINGS{Gupta_ICCV2019,
author={Anant Gupta and Atul Ingle and Mohit Gupta},
booktitle={Proc. ICCV},
title={Asynchronous Single-Photon 3D Imaging},
year={2019},
month={October},}

Stochastic Exposure Coding for Handling Multi-ToF-Camera Interference

Novel approach for mitigating multi-ToF-Camera interference by reducing both AC and DC interference

Jongho Lee, Mohit Gupta

Proc. ICCV 2019

oral presentation

web
pdf
@inproceedings{lee2019stochastic,
title={Stochastic Exposure Coding for Handling Multi-ToF-Camera Interference},
author={Lee, Jongho and Gupta, Mohit},
booktitle={Proceedings of the IEEE International Conference on Computer Vision},
pages={7880–7888},
year={2019}
}

Micro-Baseline Structured Light

Low-cost compact structured light 3D imaging for small-sized devices

Viswanath Saradagam, J Wang, Mohit Gupta, Shree Nayar

Proc. ICCV 2019

web
pdf
@INPROCEEDINGS{Venkata_ICCV2019,
author={Vishwanath Saragadam Raja Venkata and Jian Wang and Shree Nayar and Mohit Gupta},
booktitle={Proc. ICCV},
title={Micro-Baseline Structured Light},
year={2019},
month={October},}

Differential Scene Flow from Light-Field Gradients

Novel techniques that recover high-precision dense scene flow from light fields

Sizhuo Ma, Brandon M. Smith, Mohit Gupta

International Journal of Computer Vision

Special Issue on ‘Best Papers of ECCV’ (Invited Paper)

web
pdf
@article{Ma_IJCV2019,
author={Sizhuo Ma and Brandon M. Smith and Mohit Gupta},
booktitle={International Journal on Computer Vision},
title={Differential Scene Flow from Light-Field Gradients},
volume={128},
number={3},
pages={679-697},
year={2020},
}

Coding Optimization for Fast Fluorescence Lifetime Imaging

Novel and high-performance coding schemes for fast fluorescence lifetime imaging that achieve up to an order of magnitude shorter acquisition time as compared to existing approaches

Jongho Lee, JV Chacko, B Dai, SA Reza, AK Sagar, KW Eliceiri, A Velten, Mohit Gupta

ACM TOG, presented at SIGGRAPH 2019

web
pdf
@article{Lee:2019:CSO:3322934.3325136,
author = {Lee, Jongho and Chacko, Jenu Varghese and Dai, Bing and Reza, Syed Azer and Sagar, Abdul Kader and Eliceiri, Kevin W. and Velten, Andreas and Gupta, Mohit},
title = {Coding Scheme Optimization for Fast Fluorescence Lifetime Imaging},
journal = {ACM Trans. Graph.},
issue_date = {June 2019},
volume = {38},
number = {3},
month = jun,
year = {2019},
issn = {0730-0301},
pages = {26:1–26:16},
articleno = {26},
numpages = {16},
url = {http://doi.acm.org/10.1145/3325136},
doi = {10.1145/3325136},
acmid = {3325136},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {Fluorescence lifetime, coding optimization, time-of-flight, waveform optimization},
}

Practical Coding Function Design for Time-of-Flight Imaging

ToF coding theory and novel time-of-flight camera designs that achieve extreme depth resolution at long distances

Felipe Gutierrez, SA Reza, A Velten, Mohit Gupta

Proc. CVPR 2019

web
pdf
@INPROCEEDINGS{Gutierrez_CVPR2019,
author={Felipe Gutierrez-Barragan and S Reza and Andreas Velten and Mohit Gupta},
booktitle={Proc. CVPR},
title={Practical Coding Function Design for Time-of-Flight Imaging},
year={2019},
month={June},}

PF-SPAD vs. Conventional CCD or CMOS Pixel

High Flux Passive Imaging with Single-Photon Sensors

Photon-counting achieves several orders of magnitude higher dynamic range than conventional CCD or CMOS pixels

Atul Ingle, A Velten, Mohit Gupta

Proc. CVPR 2019

oral presentation

web
pdf
@INPROCEEDINGS{Ingle_CVPR2019,
author={Atul Ingle and Andreas Velten and Mohit Gupta},
booktitle={Proc. CVPR},
title={High Flux Passive Imaging with Single Photon Sensors},
year={2019},
month={June},}

Photon-Flooded Single-Photon 3D Cameras

Optimal acquisition strategies for single-photon 3D cameras: Towards long-range laser-scan quality 3D imaging

Anant Gupta, Atul Ingle, A Velten, Mohit Gupta

Proc. CVPR 2019

oral presentation

web
pdf
@INPROCEEDINGS{Gupta_CVPR2019,
author={Anant Gupta and Atul Ingle and Andreas Velten and Mohit Gupta},
booktitle={Proc. CVPR},
title={Photon-Flooded Single-Photon 3D Cameras},
year={2019},
month={June},}

2018

What Are Optimal Coding Functions for Time-of-Flight Imaging?

ToF coding theory and novel time-of-flight camera designs that achieve extreme depth resolution at long distances

Mohit Gupta, S Nayar, A Velten, Eric Breitbach

ACM TOG, presented at SIGGRAPH 2018

Finalists in the Most Influential Research category AutoSens Awards, 2019

web
pdf
@article{GuptaToG_2018,
author = {Gupta, Mohit and Velten, Andreas and Nayar, Shree K. and Breitbach, Eric},
title = {What Are Optimal Coding Functions for Time-of-Flight Imaging?},
journal = {ACM Trans. Graph.},
issue_date = {July 2018},
volume = {37},
number = {2},
month = feb,
year = {2018},
issn = {0730-0301},
pages = {13:1–13:18},
articleno = {13},
numpages = {18},
url = {http://doi.acm.org/10.1145/3152155},
doi = {10.1145/3152155},
acmid = {3152155},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {Computational time-of-flight imaging, Hamiltonian cycle, gray codes, low-power 3D cameras},
}

Micro Resolution Time-of-Flight Imaging

A technique to generate modulation frequencies as high as 1 THz using optical super heterodyne interferometry for high resolution time-of-flight depth imaging

Fengqiang Li, Florian Willomitzer, P Rangarajan, Mohit Gupta, Andreas Velten, Oliver Cossairt

Proc. ICCP 2018

web
pdf
@INPROCEEDINGS{Li_ICCP_2018,
author={F. Li and F. Willomitzer and P. Rangarajan and M. Gupta and A. Velten and O. Cossairt},
booktitle={2018 IEEE International Conference on Computational Photography (ICCP)},
title={SH-ToF: Micro resolution time-of-flight imaging with superheterodyne interferometry},
year={2018},
volume={},
number={},
pages={1-10},
keywords={image resolution;image sensors;light interferometry;SH-ToF;ToF cameras;ToF sensors;academia;depth resolution;industry;modulation frequencies;optical superheterodyne interferometry;three dimensional imaging techniques;time-of-flight sensors;Adaptive optics;Frequency modulation;Laser beams;Optical imaging;Optical interferometry;Optical sensors},
doi={10.1109/ICCPHOT.2018.8368473},
ISSN={},
month={May},}

A Geometric Perspective on Structured Light Coding

ToF coding theory and novel time-of-flight camera designs that achieve extreme depth resolution at long distances

Mohit Gupta, Nikhil Nakhate

Proc. ECCV 2018

web
pdf
@INPROCEEDINGS{Gupta_ECCV2018,
author={Mohit Gupta and Nikhil Nakhate},
booktitle={Proc. ECCV 2018},
title={A Geometric Perspective on Structured Light Coding},
year={2018},
month={September},}

3D Scene Flow from 4D Light Field Gradients

Novel techniques that recover high-precision dense scene flow from light fields

Sizhuo Ma, Brandon Smith, Mohit Gupta

Proc. ECCV 2018

oral presentation

selected for IJCV Special Issue on `Best of ECCV’

web
pdf
@INPROCEEDINGS{Ma_ECCV2018,
author={Sizhuo Ma and Brandon Smith and Mohit Gupta},
booktitle={Proc. ECCV},
title={3D Scene Flow from 4D Light Field Gradients},
pages={666–681},
year={2018},
month={September},}

Trapping Light for Time of Flight

A novel imaging method for near-complete, surround, 3D reconstruction of geometrically complex objects, in a single scan

Ruilin Xu, Mohit Gupta, Shree Nayar

Proc. CVPR 2018

oral presentation

web
pdf
@INPROCEEDINGS{Xu_CVPR2018,
author={Ruilin Xu and Mohit Gupta and Shree Nayar},
booktitle={2018 CVF and IEEE Conference on Computer Vision and Pattern Recognition},
title={Trapping Light for Time-of-Flight},
year={2018},
month={June},}

Tracking Objects Outside the Line of Sight using Speckle Imaging

Techniques for tracking micro-motion (<10 microns) of multiple objects, around the corner, with only low-cost components

Brandon Smith, Matthew O'Toole, Mohit Gupta

Proc. CVPR 2018

spotlight oral

web
pdf
@INPROCEEDINGS{SmithOtooleGuptaCVPR2018,
author={Brandon M. Smith and Matthew O’Toole and Mohit Gupta},
booktitle={2018 CVF and IEEE Conference on Computer Vision and Pattern Recognition},
title={Tracking Multiple Objects Outside the Line of Sight using Speckle Imaging},
year={2018},
month={June},}

2017

CoLux: 3D Micro-Motion Analysis Using Speckle Imaging

Novel low-cost speckle imaging techniques that can measure micro motions of multiple, independently moving objects

Brandon Smith, P Desai, V Agarwal, Mohit Gupta

Proc. SIGGRAPH 2017

Finalist, WARF Innovation Awards

web
pdf
@article{Smith:SIGGRAPH2017,
author = {Brandon M. Smith and Pratham Desai and Vishal Agarwal and Mohit Gupta},
title = {CoLux: Multi-Object 3D Micro-Motion Analysis Using Speckle Imaging},
journal = {ACM Trans. Graph.},
issue_date = {July 2017},
volume = {36},
number = {4},
month = jul,
year = {2017},
articleno = {34},
numpages = {12},
url = {http://doi.acm.org/10.1145/3072959.3073607},
doi = {10.1145/3072959.3073607},
publisher = {ACM},
address = {New York, NY, USA},
}

High-depth-resolution Range Imaging with Multiple-wavelength Superheterodyne Interferometry using 1550-nm Lasers

A technique to generate modulation frequencies as high as 1 THz using optical super heterodyne interferometry for high resolution time-of-flight depth imaging

Fengqiang Li, Joshua Yablon, Andreas Velten, Mohit Gupta, Oliver Cossairt

Applied Optics 2017

web
pdf
@article{Li:17,

author = {Fengqiang Li and Joshua Yablon and Andreas Velten and Mohit Gupta and Oliver Cossairt},

journal = {Appl. Opt.},

keywords = {Imaging systems; Lasers and laser optics; Interferometric imaging ; Diode lasers; Imaging systems; Imaging techniques; Laser beam combining; Phase modulation; Range imaging},

number = {31},

pages = {H51–H56},

publisher = {OSA},

title = {High-depth-resolution range imaging with multiple-wavelength superheterodyne interferometry using 1550-nm lasers},

volume = {56},

month = {Nov},

year = {2017},

url = {http://ao.osa.org/abstract.cfm?URI=ao-56-31-H51},

doi = {10.1364/AO.56.000H51},

abstract = {Lasers and laser diodes are widely used as illumination sources for optical imaging techniques. Time-of-flight (ToF) cameras with laser diodes and range imaging based on optical interferometry systems using lasers are among these techniques, with various applications in fields such as metrology and machine vision. ToF cameras can have imaging ranges of several meters, but offer only centimeter-level depth resolution. On the other hand, range imaging based on optical interferometry has depth resolution on the micrometer and even nanometer scale, but offers very limited (sub-millimeter) imaging ranges. In this paper, we propose a range imaging system based on multi-wavelength superheterodyne interferometry to simultaneously provide sub-millimeter depth resolution and an imaging range of tens to hundreds of millimeters. The proposed setup uses two tunable III-V semiconductor lasers and offers leverage between imaging range and resolution. The system is composed entirely of fiber connections except the scanning head, which enables it to be made into a portable device. We believe our proposed system has the potential to tremendously benefit many fields, such as metrology and computer vision.},

}

2016

Dual Structured Light 3D using a 1D Sensor

a novel structured light approach that uses only a 1D sensor with simple optics and no moving parts to reconstruct scenes with the same acquisition speed as a traditional 2D sensor

Jian Wang, Aswin Sankaranarayanan, Mohit Gupta, Srinivasa Narasimhan

Proc. ECCV 2016

oral presentation

DisCo: Display-Camera Communication Using Rolling Shutter Sensors

We present DisCo, a novel display-camera communication system. DisCo enables displays and cameras to communicate with each other, while also displaying and capturing images for human consumption.

Kensei Jo, Mohit Gupta, Shree Nayar

ACM TOG, presented at SIGGRAPH 2016

2015

SpeDo: 6 DOF Ego-Motion Sensor Using Speckle Defocus Imaging

Sensors that measure their motion with respect to the surrounding environment (ego-motion sensors) can be broadly classified into two categories. The first is inertial sensors, such as accelerometers.

Kensei Jo, Mohit Gupta, Shree Nayar

Proc. ICCV 2015

LiSens— A Scalable Architecture for Video Compressive Sensing

The measurement rate of cameras that take spatially multiplexed measurements by using spatial light modulators (SLM) is often limited by the switching speed of the SLMs.

Jian Wang, Mohit Gupta, Aswin Sankaranarayanan

Proc. ICCP 2015

MC3D: Motion Contrast 3D Scanning

We present a novel structured light technique called Motion Contrast 3D scanning (MC3D) that maximizes bandwidth and light source power to avoid performance trade-offs.

Nathan Matsuda, Oliver Cossairt, Mohit Gupta

Proc. ICCP 2015

Phasor Imaging: A Generalization of Correlation ToF Imaging

Time-of-flight (ToF) based 3D imaging in scenes with multipath interference (interreflections) and scattering (e.g., smoke and fog). Frequency domain analysis of transient light transport. Applications in autonomous transportation, 3D mapping and robotic automation.

Mohit Gupta, Shree Nayar, Matthias Hullin, Jaime Martin

ACM TOG, presented at SIGGRAPH, 2015

2014

Recovering Scene Geometry under Wavy Fluid via Distortion and Defocus Analysis

undistorting scenes that are immersed in transparent refractive media with a dynamic surface

M Zhang, Xing Lin, Mohit Gupta, Jinli Suo, Q Dai

Proc. ECCV 2014

Digital Refocusing with Incoherent Holography

Scene refocusing using 2D coherent holograms

Oliver Cossairt, Nathan Matsuda, Mohit Gupta

Proc. ICCP 2014

best paper (honorable mention)

Efficient Space-Time Sampling with Pixel-wise Coded Exposure for High Speed Imaging

High speed compressive imaging (up to 1000 fps) using a conventional 30 fps camera, an LCoS (liquid crystal on silicon) light modulator and large scale dictionary learning. Applications in consumer and scientific imaging.

Dengyu Liu, Jinwei Gu, Y Hitomi, Mohit Gupta, Tomoo Mitsunaga, Shree Nayar

IEEE Trans. PAMI

2013

Fibonacci Exposure Bracketing for High Dynamic Range Imaging

High dynamic range imaging for moving scenes and cameras by capturing images with Fibonacci sequence exposures. Applications in consumer imaging.

Mohit Gupta, Daisuke Iso, Shree Nayar

Proc. ICCV 2013

Structured Light in Sunlight

Outdoor 3D scanning in strong ambient illumination with a limited power-budget. Applications in autonomous transportation, outdoor robotics and urban mapping

Mohit Gupta, Qi Yin, Shree Nayar

Proc. ICCV 2013

2012

A Practical Approach to 3D Scanning in the Presence of Interreflections, Subsurface Scattering and Defocus

The goal of this work is to build an end-to-end system for structured light 3D scanning under a broad range of global illumination effects, such as inter-reflections, diffusion and sub-surface scattering.

Mohit Gupta, Amit Agrawal, Ashok Veeraraghavan, Srinivasa Narasimhan

International Journal of Computer Vision

When Does Computational Imaging Improve Performance?

Performance analysis of computational imaging techniques such as defocus and motion deblurring and light field capture. Practical guidelines for imaging system design.

Oliver Cossairt, Mohit Gupta, Shree Nayar

IEEE Trans. on Image Processing

A Combined Theory of Defocused Illumination and Global Light Transport

The goal is to recover scene properties in the presence of global illumination.

Mohit Gupta, Yuandong Tian, Srinivasa Narasimhan, Li Zhang

International Journal of Computer Vision

Micro Phase Shifting

3D scanning of optically challenging scenes, such as with translucent materials (e.g., marble, human skin) and concave objects. Applications in digital cultural heritage and performance capture, factory automation and indoor mapping

Mohit Gupta, Shree Nayar

Proc. CVPR 2012

oral presentation

Diffuse Structured Light

High resolution 3D scanning for specular materials (e.g., metallic machine parts), intricate geometries (e.g., finger-prints) and objects with sharp depth discontinuities.

Shree Nayar, Mohit Gupta

Proc. ICCP 2012

2011

Multiplexed Illumination for Scene Recovery in the Presence of Global Illumination

Direct-global separation and scene recovery using multiple light sources

J Gu, T Kobayashi, Mohit Gupta, Shree Nayar

Proc. ICCV 2011

oral presentation

Video from a Single Coded Exposure Photograph using a Learned Over-Complete Dictionary

Y Hitomi, J Gu, Mohit Gupta, Tomoo Mitsunaga, Shree Nayar

Proc. ICCV 2011

Structured Light 3D Scanning in the Presence of Global Illumination

Mohit Gupta, Amit Agrawal, Ashok Veeraraghavan, Srinivasa Narasimhan

Proc. CVPR 2011

2010

Flexible Voxels for Motion-Aware Videography

Post capture control of spatial resolution and frame-rate of videos, depending on the scene content. Applications in consumer imaging, scientific imaging and microscopy.

Mohit Gupta, Amit Agrawal, Ashok Veeraraghavan, Srinivasa Narasimhan

Proc. ECCV 2010

Optimal Coded Sampling for Temporal Super-Resolution

We propose coded sampling that address these issues: using N cameras it allows N times temporal super-resolution while allowing N/2 times more light compared to an equivalent high speed camera.

Amit Agrawal, Mohit Gupta, Ashok Veeraraghavan, Srinivasa Narasimhan

Proc. CVPR 2010

2009

(De) Focusing on Global Light Transport for Active Scene Recovery

The goal is to recover scene properties in the presence of global illumination.

Mohit Gupta, Y Tian, S Narasimhan, Li Zhang

Proc. CVPR 2009

oral presentation

2008

On Controlling Light Transport in Poor Visibility Environments

Seeing clearer and farther in scattering media using active illumination and polarization. Applications in underwater imaging, autonomous transportation and medical imaging.

Mohit Gupta, S Narasimhan, Yoav Schechner

Proc. CVPR 2008

High Resolution Tracking of Non-Rigid 3D Motion Using Harmonic Maps

High resolution tracking of facial expressions using harmonic maps

Yang Wang, Mohit Gupta, Song Zhang, Sen Wang, X Gu, D Samaras, Peisen Huang

International Journal of Computer Vision

2007

Legendre Fluids: A Unified Framework for Analytic Reduced Space Modeling and Rendering of Participating Media

Reduced space framework for fast simulation and rendering of scattering media such as smoke and fog. Applications in gaming, special effects and virtual reality.

Mohit Gupta, Srinivasa Narasimhan

ACM Symposium on Computer Animation 2007

Legendre polynomials Triple Product Integral and Chebyshev Polynomials

Reduced space framework for fast simulation and rendering of scattering media such as smoke and fog. Applications in gaming, special effects and virtual reality.

Mohit Gupta, Srinivasa Narasimhan

Technical Report, Robotics Institute, CMU

2006

Acquiring Scattering Properties of Participating Media by Dilution

Imaging and illumination setup for measuring scattering properties of liquids. Applications in computer graphics rendering and ocean monitoring.

Srinivasa Narasimhan, Mohit Gupta, Craig Donner, R Ramamoorthi, S Nayar, H Jensen

Proc. SIGGRAPH 2006

2005

High Resolution Tracking of Non-Rigid 3D Motion of Densely Sampled Data Using Harmonic Maps

High resolution tracking of facial expressions using harmonic maps

Yang Wang, Mohit Gupta, Song Zhang, Sen Wang, X Gu, D Samaras, Peisen Huang

Proc. ICCV 2005

oral presentation

2003

pdf