When Does Computational Imaging Improve Performance?

All CI techniques discussed in this paper can be modeled using a linear image formation model. In order to recover the desired image, these techniques require an additional decoding step, which amplifies noise. Impulse imaging techniques measure the signal directly without requiring any decoding. A stopped down aperture can be used to avoid defocus blur, a shorter exposure can be used to avoid motion blur, and a pin-hole mask can be placed near the sensor to directly measure the light field.

We analyze the performance of a variety of CI techniques and derive a bound on their performance in terms of SNR. We show that CI techniques provide a significant performance advantage only if the average signal level is significantly lower than the sensor read noise variance. In this we simulate the performance of several defocus deblurring cameras[1][2][3][4][5], motion deblurring cameras[6][7], and light field multiplexing cameras[2][8]. All techniques perform at or below the performance bound derived in the paper.

We provide guidelines for when to use CI given an imaging scenario. The scenarios are defined in terms of the application (e.g., motion deblurring, defocus deblurring), real-world lighting (e.g., moonlit night or cloudy day, indoor or outdoor), scene properties (albedo, object velocities, depth range) and sensor characteristics. These figures show contour plot of the SNR gain bound for motion and defocus deblurring cameras. For both cameras, the SNR gain is always negligible when the illuminance is greater than 125 lux (typical indoor lighting).

We use simulations to compare the performance of flutter-shutter[6] and impulse cameras (i.e. a camera with a short exposure). The top row of this figure shows an image blurred by the flutter sequence given in[6]. The second and fourth rows show the results after deblurring with linear inversion and the BM3D algorithm[14], respectively. The third row shows the results from the impulse camera. The last row shows the results for denoising the images in the third row with the BM3D algorithm. The flutter shutter camera has higher SNR when the illumination is less than 100 lux.

We use simulations to compare the performance of focal sweep[4][5] and impulse cameras (i.e. a camera with a stopped-down aperture). The top row shows an image blurred by a focal sweep PSF. The second and fourth rows show the results after deblurring with linear inversion and the BM3D algorithm[14], respectively. The third row shows the results from the impulse camera. The last row shows the results for denoising the images in the third row with the BM3D algorithm. The focal sweep camera always has a higher SNR than impulse imaging, but the improvement becomes negligible when illumination is greater than 100 lux.

We also provide empirical results using perceptually motivated metrics and regularized deblurring algorithms. Here we show performance for the MSE, SSIM[9], VIF[10], and UQI[11] metrics. The top row shows performance for the focal sweep camera, and the bottom row shows performance for the flutter shutter camera. For each plot, the performance gain is plotted on a log scale. The black line corresponds to our derived performance bound. The magenta lines correspond to performance gain using direct linear inversion. The red, green, and blue curves correspond to reconstructions using Gaussian[12][13], TV[14], and BM3D[15] priors. The bound derived in the paper is emperically found to be an upper bound for performance across all metrics and priors.