Communication with Ambient Light using Digital Micromirror Devices Roy Blokker Talia Xu Marco A. Zúñiga Zamalloa Delft University of Technology Delft University of Technology Delft University of Technology Delft, Netherlands Delft, Netherlands Delft, Netherlands Figure 1: Phone reflecting light as a mirror. This symbolises the main idea of our research. We utilize micro mirrors to transmit information via sunlight reflections and use the smartphone’s camera as a receiver to decode those reflections. ABSTRACT to easily interact with the system and our evaluation shows that Passive visible light communication (VLC) takes advantage of the the link can achieve a data rate of 1bps at a distance of 30cm. pervasive nature of ambient light in our environment for wireless transmissions. The design of transmitters in passive VLC predom- CCS CONCEPTS inately uses liquid crystal displays (LCDs). While LCDs are an • Hardware → Wireless devices; • Computer systems organi- economical choice with low power consumption, they lack some zation → Embedded systems. key properties that are desirable for passive VLC. For example, LCDs absorb more than half of the incident light, leaving only a KEYWORDS small portion to be used for communication. In addition, since the Visible Light Communication, Passive Communication, Digital Mi- direction of ambient can change over time, the relative positions of cromirror Device (DMD) the LCDs and receivers have to be changed constantly to maintain the correct alignment. To overcome these shortcomings, we propose the use of a novel 1 INTRODUCTION transmitter with integrated optical fibres and digital micro-mirror Visible Light Communication (VLC) is an emerging technology devices (DMDs). DMDs are able to reflect up to 97% of the incident for wireless communication that has gained traction from both light, while the accompanying optical fibres aim to capture ambient academia and industry in recent years. Compared to traditional light from various angles and guide them to the DMDs in a fixed radio frequency wireless communication, VLC has several advan- direction. This design is a first step towards the goal of decoupling tages such as an unregulated wide bandwidth and high security. the direction of ambient light from the direction of the optical link, VLC can be further divided into two main areas: active and passive. while achieving the same communication characteristics as LCDs In both areas, the intensity of the light is modulated at a speed that with a much smaller device. We also design an App to allow users is invisible to the human eye, but can be received and decoded by optical receivers. In active VLC, the driver circuitry is modified to modulate message signals by varying the driving currents of Copyright 2021 for this paper by its authors. Use permitted under Creative Commons an LED. In passive VLC, an external surface is used to modulate License Attribution 4.0 International (CC BY 4.0). message signals by changing the characteristics of the light passing through or reflecting from the surface. Compared to active VLC, passive VLC has the advantage of exploiting the ambient light in our environment, without having the need to directly control the light source. CHIIoT 1, February 17, 2021, Delft, The Netherlands Blokker, et al. Figure 2: Light energy lost using an LCD: 50% Figure 4: The sun moves during the day, therefore the re- flected light changes direction Figure 3: Light energy lost using a DMD: 3% The majority of passive VLC systems proposed in the literature rely on the use of liquid crystal displays (LCDs) to modulate light [2][6][7][8][9][10], as shown in Figure 2. A LCD modulates the incoming light on its surface with two states: in the opaque state, Figure 5: By using a lens to capture light, the movement of the LCD surface absorbs the incident light (logic zero), and in the the sun doesn’t influence the direction of the reflection clear state, the LCD surface allows the incident light to pass through (logic one). However, as LCDs are only able to realize these two states in combination with polarizers, less than half of the incident light can typically pass through the surface in the clear state due to polarization mismatch. One device that is able to overcome this disadvantage is the digital micromirror device (DMD). A DMD is a small chip containing thousands of small mirrors with the size of less than 10 microns. DMDs are widely used in video projection technology (beamers), where every mirror represents a pixel. The mirrors can be switched to two fixed angles with respect to the surface normal, allowing two binary states to be sent by reflecting light (or not) towards the intended receiver. In our work, we are trying to implement a communication link between a DMD and a smartphone. Smartphones have been used before as VLC receivers but mainly using active lights sources as transmitters (LEDs) [1][3][4][5]. In our design, the DMD reflects the modulated ambient light towards the camera of a smartphone, which is then decoded by an App and displayed on the screen. 2 OPTICAL AND MECHANICAL STRUCTURE As the reflection off a DMD is primarily specular, the light source Figure 6: 3D printed light collector and the receiver have to be precisely aligned to establish an opera- tional link. When sunlight is used as the light source in passive VLC, as the position of the sun changes in the sky throughout the day, its direction with respect to the DMD also changes. This causes the multiple optical fibers. In this manner, the collected light is guided reflected light to be misaligned to the receiver, as shown in Figure 4, through the optical fibers and emitted directly onto the DMD. This where the signal-to-noise ratio (SNR) can significantly deteriorate. allows the incident angle of the light on the DMD to remain the To overcome this problem, we propose the integration of optical same regardless of the location of the sun. The design is shown in fibers and lenses into a passive VLC system, as shown in Figure 5. Figure 6, and the other end of the optical fibers, illuminating the The sunlight is "collected" using a convex plano lens connected to DMD at a fixed angle, is shown in Figure 7. Communication with Ambient Light using Digital Micromirror Devices CHIIoT 1, February 17, 2021, Delft, The Netherlands Figure 9: [Packet format Figure 7: DMD reflecting light coming from optic fibers Figure 10: (1): Raw image. (2): Image tracking follows the DMD and draws a region of interest. (3): Only the ROI re- mains. (4): ROI is sent to image processing. (5): Image is con- verted to greyscale to determine the average pixel value. 4 RECEIVER Images from the smartphone camera are captured and processed using an Android App. After selecting a region of interest, every captured frame is sent to an image processing pipeline, as shown Figure 8: DLP 2000 by Texas Instruments in Figure 10. The region of interest makes the processing easier, as only a small part of the captured image needs to be used. An OpenCV tracker is used to track the DMD, so when the user moves the hand a little, the region of interest will still be on the DMD chip. 3 TRANSMITTER To describe the decoding process, let us denote R𝑖 as the region of interest at time 𝑖 (i.e. frame 𝑖). We calculate the average pixel value For a proof of concept design, we choose the DLP2000 DMD from Texas Instrument. The DLP2000 DMD has an aperture size of of each region of interest 𝑖, denoted as R c𝑖 . During the preamble 4.84mm by 3.26mm, as shown in figure Figure 8. In our design, transmission, when the DMD is "on", the DMD will be reflecting all pixels of the DMD have the same state, and the entire DMD light and the average reaches its maximum value R̂𝑚𝑎𝑥 . When the device acts as a single pixel with on and off states. A simple pulse DMD is "off", we obtain the lowest average value R̂𝑚𝑖𝑛 . Later, when width modulation (PWM) was chosen to transmit the data. When a the ASCII characters are transmitted, if | R̂𝑖 − R̂𝑚𝑎𝑥 | < | R̂𝑖 − R̂𝑚𝑖𝑛 |, logic one is sent, the light is reflected into the receiver for a certain the DMD is decoded as "on", else as "off". Every time the state of time period and then not reflected for the same amount of time. the DMD changes, from "on" to "off" or vice-versa, we calculate When a logic zero is sent, the time that the light is not reflected the number of frames that the DMD spent in that state. If the "on" into the receiver is doubled. This modulation scheme allows ones state has approximately the same length as the subsequent "off" and zeros to be easily distinguish, in the expense of unequal trans- state, a one is decoded, otherwise it is a zero. As soon as the bits are mit times for different symbols. To demonstrate our design, ASCII decoded, they are converted back to ASCII characters and displayed texts are sent over the optical link. The non-extended ASCII table on the screen. contains characters that are all 8-bit long and start with a zero. Because there isn’t a character of all ones a preamble is chosen 5 EVALUATION containing eight ones and then a zero. Note that using mostly ones Currently a bit rate of 1bps can be achieved. One of the main in the preamble is faster because zeros require more transmission reasons for the low bit rate is the tracker used in the app. There is time. The preamble is sent multiple times to make sure the receiver significant room for improvement. For example, putting the tracker will be able to see it. After the preambles, the ASCII characters are in a different thread, so the rest of the program doesn’t have to wait sent. There is no limit to the number of characters that can be sent. for the tracker to finish. A communication distance of up to 30cm is The format of the data frame is shown in Figure 9. possible with our prototype. There are also some opportunities to CHIIoT 1, February 17, 2021, Delft, The Netherlands Blokker, et al. increase the range. The image recognition step can be improved to Netherlands) (BuildSys ’17). Association for Computing Machinery, New York, look for smaller regions of interest. If we can capture only the DMD, NY, USA, Article 38, 2 pages. https://doi.org/10.1145/3137133.3141435 [5] Ye-Sheng Kuo, Pat Pannuto, Ko-Jen Hsiao, and Prabal Dutta. 2014. Luxapose: the noise introduced by the surrounding areas will be eliminated Indoor Positioning with Mobile Phones and Visible Light. In Proceedings of the and the range could increase significantly. At the moment the region 20th Annual International Conference on Mobile Computing and Networking (Maui, Hawaii, USA) (MobiCom ’14). Association for Computing Machinery, New York, of interest is set to a fixed size. NY, USA, 447–458. https://doi.org/10.1145/2639108.2639109 [6] Jiangtao Li, Angli Liu, Guobin Shen, Liqun Li, Chao Sun, and Feng Zhao. 2015. 6 ACKNOWLEDGEMENTS Retro-VLC: Enabling Battery-Free Duplex Visible Light Communication for Mo- bile and IoT Applications. In Proceedings of the 16th International Workshop on This work has been funded in part by the European Union’s Hori- Mobile Computing Systems and Applications (Santa Fe, New Mexico, USA) (Hot- zon 2020 research and innovation programme under the Marie Mobile ’15). Association for Computing Machinery, New York, NY, USA, 21–26. https://doi.org/10.1145/2699343.2699354 Skłodowska Curie grant agreement ENLIGHTEM No. 814215; and [7] Purui Wang, Lilei Feng, Guojun Chen, Chenren Xu, Yue Wu, Kenuo Xu, Guobin by the LuxSenz project, a TOP-Grant, Module 1, Physical Sciences Shen, Kuntai Du, Gang Huang, and Xuanzhe Liu. 2020. Renovating Road Signs with project number 612.001.854, which is financed by the Dutch for Infrastructure-to-Vehicle Networking: A Visible Light Backscatter Communi- cation and Networking Approach. In Proceedings of the 26th Annual International Research Council (NWO). Conference on Mobile Computing and Networking (London, United Kingdom) (Mo- biCom ’20). Association for Computing Machinery, New York, NY, USA, Article REFERENCES 6, 13 pages. https://doi.org/10.1145/3372224.3380883 [8] Yue Wu, Purui Wang, Kenuo Xu, Lilei Feng, and Chenren Xu. 2020. Turboboosting [1] Karan Ahuja, Sujeath Pareddy, Robert Xiao, Mayank Goel, and Chris Harri- Visible Light Backscatter Communication. In Proceedings of the Annual Conference son. 2019. LightAnchors: Appropriating Point Lights for Spatially-Anchored of the ACM Special Interest Group on Data Communication on the Applications, Augmented Reality Interfaces. In Proceedings of the 32nd Annual ACM Sym- Technologies, Architectures, and Protocols for Computer Communication (Virtual posium on User Interface Software and Technology (New Orleans, LA, USA) Event, USA) (SIGCOMM ’20). Association for Computing Machinery, New York, (UIST ’19). Association for Computing Machinery, New York, NY, USA, 189–196. NY, USA, 186–197. https://doi.org/10.1145/3387514.3406229 https://doi.org/10.1145/3332165.3347884 [9] Xieyang Xu, Yang Shen, Junrui Yang, Chenren Xu, Guobin Shen, Guojun Chen, [2] Rens Bloom, Marco Zúñiga Zamalloa, and Chaitra Pai. 2019. LuxLink: Cre- and Yunzhe Ni. 2017. PassiveVLC: Enabling Practical Visible Light Backscatter ating a Wireless Link from Ambient Light. In Proceedings of the 17th Con- Communication for Battery-Free IoT Applications. In Proceedings of the 23rd ference on Embedded Networked Sensor Systems (New York, New York) (Sen- Annual International Conference on Mobile Computing and Networking (Snowbird, Sys ’19). Association for Computing Machinery, New York, NY, USA, 166–178. Utah, USA) (MobiCom ’17). Association for Computing Machinery, New York, https://doi.org/10.1145/3356250.3360021 NY, USA, 180–192. https://doi.org/10.1145/3117811.3117843 [3] R. Boubezari, H. Le Minh, Z. Ghassemlooy, and A. Bouridane. 2016. Smartphone [10] Zhice Yang, Zeyu Wang, Jiansong Zhang, Chenyu Huang, and Qian Zhang. 2015. Camera Based Visible Light Communication. Journal of Lightwave Technology Wearables Can Afford: Light-Weight Indoor Positioning with Visible Light. In 34, 17 (2016), 4121–4127. https://doi.org/10.1109/JLT.2016.2590880 Proceedings of the 13th Annual International Conference on Mobile Systems, Applica- [4] Rupam Kundu, Gopi Krishna Tummala, and Prasun Sinha. 2017. Visualloc: Vision tions, and Services (Florence, Italy) (MobiSys ’15). Association for Computing Ma- Based Localization Using a Single Smart-Bulb. In Proceedings of the 4th ACM chinery, New York, NY, USA, 317–330. https://doi.org/10.1145/2742647.2742648 International Conference on Systems for Energy-Efficient Built Environments (Delft,