DisCo: Display-Camera Communication Using Rolling Shutter Sensors
ACM TOG, presented at SIGGRAPH 2016
We present DisCo, a novel display-camera communication system. DisCo enables displays and cameras to communicate with each other, while also displaying and capturing images for human consumption.
We present DisCo, a novel display-camera communication system. DisCo enables displays and cameras to communicate with each other, while also displaying and capturing images for human consumption. Messages are transmitted by temporally modulating the display brightness at high frequencies so that they are imperceptible to humans. Messages are received by a rolling shutter camera which converts the temporally modulated incident light into a spatial flicker pattern. In the captured image, the flicker pattern is superimposed on the pattern shown on the display. The flicker and the display pattern are separated by capturing two images with different exposures. The proposed system performs robustly in challenging real-world situations such as occlusion, variable display size, defocus blur, perspective distortion and camera rotation. Unlike several existing visible light communication methods, DisCo works with off-the-shelf image sensors. It is compatible with a variety of sources (including displays, single LEDs), as well as reflective surfaces illuminated with light sources. We have built hardware prototypes that demonstrate DisCo’s performance in several scenarios. Because of its robustness, speed, ease of use and generality, DisCo can be widely deployed in several CHI applications, such as advertising, pairing of displays with cell-phones, tagging objects in stores and museums, and indoor navigation.
DisCo can work with a broad range of devices. (a) For transmitters, DisCo can use display monitors, light sources or objects illuminated by light as the transmitter (Display). (b) The display sends information to the receiver, which can be implemented using rolling shutter cameras. (c) The display also shows images for human consumption, while simultaneously communicating with the camera.
(a) Exposure timing of an SDE rolling shutter sensor with two different exposures. (b) An example captured image (input) with a rolling shutter SDE sensor. The display was modulated with a 500 Hz. temporal sinusoid. (c-d) Images of different exposures are extracted from (b), from which the signal (e) and the flicker-free image (f) are recovered (output).
We have extensively tested the proposed system in a variety of real-world imaging situations. (a) Display smaller than cameraÂfs FOV due to large display-camera distance. (b) Display larger than cameraÂfs FOV due to large zoom/small display-camera distance. (c) Display blurred due to camera defocus. (d-e) Occluding objects between camera and display. In these examples, the signal (URL https://www.google.com/) was transmitted and received by the camera. The system adapts to challenging conditions by capturing multiple frames.
Share This Article