Robust Scene Inference under Noise-Blur Dual Corruptions

Scene inference under low-light is a challenging problem due to severe noise in the captured images. One way to reduce noise is to use longer exposure during the capture. However, in the presence of motion (scene or camera motion), longer exposures lead to motion blur, resulting in loss of image information. This creates a trade-off between these two kinds of image degradations: motion blur (due to long exposure) vs. noise (due to short exposure), also referred as a dual image corruption pair in this paper. With the rise of cameras capable of capturing multiple exposures of the same scene simultaneously, it is possible to overcome this trade-off.

Our key observation is that although the amount and nature of degradation varies for these different image captures, the semantic content remains the same across all images. To this end, we propose a method to leverage these multi exposure captures for robust inference under low-light and motion. Our method builds on a feature consistency loss to encourage similar results from these individual captures, and uses the ensemble of their final predictions for robust visual recognition. We demonstrate the effectiveness of our approach on simulated images as well as real captures with multiple exposures, and across the tasks of object detection and image classification.

Publications

Robust Scene Inference under Noise-Blur Dual Corruptions

Bhavya Goyal, JF Lalonde, Yin Li, Mohit Gupta

Proc. ICCP 2022

Noise Blur Trade-off

Under low light and motion conditions, there exists a trade-off between noise and blur, which we call Noise-Blur Dual. By varying exposure of the image capture, we can trade-off these two image degradations, hence selecting optimal exposure is important.

Complimentary Information in Varying Exposures

Different exposures have complimentary information useful for inference. Long exposure perform well for stationary objects and short exposure performs well on fast moving objects.

Multi-Exposure Ensemble

We train model with multiple exposures to get more robust inference. During inference, we use output prediction from all exposures and ensemble output is our final prediction.

Robust Object Detection

Our model can detect fast moving object under low-light with multi-exposure ensemble, which shows our approach is more robust to both noise and blur.

Resources

Presentation Slides

Comments

Share This Article