Using Autonomous Robots to Diagnose Wireless Connectivity Richard Wang, Manuela Veloso, and Srinivasan Seshan Carnegie Mellon University Due to the proliferation of wireless devices, many wireless struggle to resolve the wireless issues because: 1. they users treat wireless connectivity as a black box. When have limited time due to the large number of users to wireless performance does not meet expectations, it can administrators (25,000 to 6 in our case), 2. the problem must be a frustrating experience to try and resolve wireless be easy to replicate, and 3. network administrators control the issues. Wireless problems are more significant for mobile infrastructure APs but have limited visibility of the wireless robots due to strenuous requirements for sustained wireless medium. connectivity while moving [1]. Unfortunately, it can be Autonomous robots as a wireless tool can augment diag- difficult to understand the cause of wireless problems in real nosis of wireless problems by: environments. First, wireless signals transmitted across the 1) capturing fine-grain wireless maps reflecting actual wireless medium are susceptible to attenuation, interference, propagation of wireless signals and reflections from the surrounding environment and other 2) serving as a vehicle to subject wireless devices to wireless devices. Second, wireless connectivity depends on repeatable motions decentralized cooperation across heterogeneous devices. As This is made possible due to their ability continuously autonomous robots are introduced in our environments, localize with high accuracy and autonomously and precisely we believe they can be a perfect tool to capture detailed navigate without human assistance. Detailed wireless maps snapshots about our wireless environments to help diagnose help to reveal how the wireless medium is being used in wireless connectivity issues. In this paper, we show how order to eliminate unlikely causes of poor connectivity. They these insights helped us to diagnose our robot’s own motion- would also allow wireless users to diagnose simple dead zone based wireless connectivity issues. coverage issues and perhaps also empower them to create I. I NTRODUCTION more meaningful trouble tickets. Since wireless problems Understanding wireless connectivity in real environments with motion are often short-lived, the ability to reliably repeat is hard. Much of the complexity stems from wireless trans- motions is essential for understanding more complex motion- missions occurring over an open, shared medium with a based wireless connectivity issues. mixture of decentralized, heterogeneous devices [2]. Once In this paper, we will first show that autonomous robots devices begin to move, wireless problems become even more can be used to collect detailed wireless measurements. Next, difficult to diagnose since wireless conditions around the we show fine-grain insights allow us to better understand device can change rapidly. The emergence of telepresence how our wireless infrastructure uses the wireless medium. robots has shown that wireless devices in motion struggle to Finally, we show how we were able to diagnose our device’s sustain uninterrupted wireless connectivity [1]. In this paper, own motion-based wireless connectivity issues. we will show that autonomous robots can be a valuable tool II. I NSIGHTS ABOUT S URROUNDING W IRELESS for identifying the cause of poor wireless performance with C ONDITIONS direct observations of the wireless environment. We now show the detailed insights that autonomous robots We focus on enterprise wireless networks composed of can capture without access to any sensitive wireless in- access points (APs) distributed throughout the environment frastructure APs. With these insights, we will be able to to provide Internet access to devices at all locations. Today, understand how the wireless medium is being utilize and see motion-based wireless connectivity issues are difficult for if possible infrastructure configuration issues may be causing users to resolve because: our wireless connectivity issues. 1) wireless infrastructures are complex and vary over space and time A. AP Coverage 2) users have visibility and control over only their own AP coverage ensures every location has at least one AP device in range. Avoiding wireless dead zones is the responsi- 3) wireless communication problems can require signif- bility of network administrators who manage the wireless icant domain knowledge to deal with the range of infrastructure. They often try to place APs to provide a hardware, drivers, and protocol layers high minimum received signal strength indicator (RSSI) at As a result, a natural reaction is to submit trouble tickets every location. Our network administrators target a minimum and wait some time for network administrators to come RSSI of -60 dBm, which is much higher than -90 dBm that and resolve the problem. Even network administrators may generally signifies no connectivity. The process of verifying FinE-R 2015 Page 11 IROS 2015, Hamburg - Germany The path to success: Failures in Real Robots October 2, 2015 coverage simply requires sampling RSSI at all locations in the environment. Unfortunately, there are no practical solutions that require little human effort and achieve fine- grain sampling of the environment. As a result, there are situations where trouble tickets result in the discovery of wireless dead zones in practice. We can automate this search for wireless dead zones by deploying autonomous robots to measure coverage across the environment. We were able to cover four floors of our en- terprise environment. Figure 1 shows a histogram of median RSSI of the best available AP after dividing the environment into 1m x 1m grid regions. We see that AP coverage across two floors is very strong with few regions falling below the -60 dBm target. If there had been wireless dead zones, they would have been apparent in these histograms. As a result, wireless issues for these floors are unlikely to be due to wireless dead zones. Fig. 2: Median throughput across two floors. connectivity issues still persist. From our own empirical observations, these wireless issues appear intermittent and seemingly random while moving around. When we bring the robot back to revisit locations where it lost connectivity, the connectivity issues would not occur again so these problems must arise with motion. Autonomous robots will (a) Floor 1 (b) Floor 2 help to better understand these motion issues since they can Fig. 1: Coverage summary showing histograms of the best continuously collect of wireless performance measurements median RSSI for each floor. Network administrators typically while also reliably executing controlled motions. With the aim for a minimum of -60 dBm coverage. autonomous robots, we will methodically diagnose the root cause by enabling humans to search for similar patterns that lead to these poor connectivity situations. B. Throughput Samples A. Repeated Motions Coverage is an important pre-requisite for wireless con- Many factors including location and speed of motion can nectivity but not necessarily reflective of the actual rate cause variations in wireless performance so we subject the of data transmission. Unlike RSSI that are instantaneous wireless device to nearly identical situations. An autonomous measurements, throughput samples depend on state and co- robot itself is perfectly suited for subjecting the device to ordination with other wireless devices. Throughput tends to repeated traverals over the same path with the exact same vary more than RSSI since congestion and dropped packets speeds and device orientations. Deploying an autonomous is affect the rate of data transmission. As a result, throughput much preferred over fixed contraptions that are cumbersome maps are unliklye to be a predictable as the coverage maps. and require modifications to the environment [3]. Figure 2 shows throughput maps collect by the robot as The robot is instructed to follow a simple three-quarter it moved across the environment. These measurements show loop path around three hallways in the environment where how wireless performance varies over space. We can see connectivity issues occur frequently, as shown in Figure 3. that our robot’s own wireless connectivity problems are not We even instruct the robot to move in both clockwise isolated to small regions but spread across large regions of (Figure 3b and 3d) and counterclockwise (Figure 3a and 3c) our building. This points to more systemic wireless issues directions. We intentionally select a path where the robot that our robot is struggling with. If the robot was facing traverses each location at most once. With no overlapping region-specific wireless issues due to excessive congestion, measurements at any location, it will be much easier to these types of throughput maps would have been helpful. analyze the wireless performance variations using wireless maps. III. D IAGNOSING M OTION -BASED W IRELESS C ONNECTIVITY B. Analyzing Variations in Wireless Performance We have shown that our wireless infrastructure is well- While being driven along the given path, the wireless de- configured and AP coverage is not an issue. Nevertheless, vice simultaneously captures RSSI, throughput, and current our throughput maps showed that motion-based wireless AP it is associated with. We present four noteworthy runs FinE-R 2015 Page 12 IROS 2015, Hamburg - Germany The path to success: Failures in Real Robots October 2, 2015 (a) Run #1 RSSI (b) Run #2 RSSI (c) Run #3 RSSI (d) Run #4 RSSI (e) Run #1 Throughput (f) Run #2 Throughput (g) Run #3 Throughput (h) Run #4 Throughput Fig. 3: Simultaneous associated RSSI and throughput for 4 runs over the same locations. Runs 1 (3a) and 3 (3a) began in the bottom left corner with the robot moving counterclockwise while Runs 2 (3b) and 4 (3d) started in the top left and moved clockwise. Numbered labels reflect the first point of association with each AP while the shape and color reflect a unique AP whose corresponding coverage is shown in Figure 4. (a) Coverage AP A (b) Coverage AP B (c) Coverage AP C (d) Coverage AP D Fig. 4: Coverage for each AP corresponding to APs in Figure 3 identified with a unique shape and color. in Figure 3 that shows RSSI (top), throughput (middle), and around timing disassociations before connectivity degrade coverage for each AP (bottom). In the RSSI maps (top), significantly and then intelligently selecting the next AP to the unique shapes reflect the location where the device first switch to. With the help of our autonomous robot, we are associated with the corresponding AP as identified by the able to distinguish the effects of AP coverage, changing color and shape. The numbered labels identify the order in wireless conditions, and device motion to conclude that poor which they were visited. We show AP coverage for each AP handoffs are the cause of our robot’s wireless issues. of these uniquely identified APs (bottom). Corresponding throughput while moving (middle) is also shown where large IV. R ELATED W ORK stretches of white space reflect absence of connectivity. Past efforts to collect wireless measurements are unable These four runs provide some noteworthy insights. First, to ensure fine-grain accuracy, densely cover spatially diverse RSSI changes gradually over several meters as a function of areas, and provide timely updates. Unfortunately, it is diffi- the device’s distance from the AP. Notice that run #1 and #2 cult to predict the propagation of wireless signals in realistic, remained associated with the same AP for the duration of indoor environments so fine-grain wireless maps require the traversal. Irrespective of the direction of motion, RSSI measuring signals captured at each location. Measurement for these runs nearly perfectly matches corresponding AP studies have been performed by having humans carefully coverage. For these runs, throughput resulted in lengthy traverse a building and mark their locations on a map [4], [5]. stretches of no connectivity since the AP was out of range. This is a tedious process that suffers from accuracy issues In run #3 and #4, the device switches to another AP in due to human errors that make it undesirable to repeat often the middle of the path. This AP switch particularly benefits so it will be difficult to ensure maps are up-to-date. run #3 but not as much for run #4. The difference is that Dense deployments of static WiFi monitors can ensure run #3 switches APs just as it is about to enter the strongest timeliness but are limited by placement options for fixed AP coverage region for the selected AP. In contrast, run #4 location monitors and incur significant human effort and switches to an AP that is almost out of range. costs to deploy so typically they cannot achieve high spatial We can see in Figure 4 that there is at least one AP granularity. While some use dedicated sensor hardware [6], with high RSSI along the entire path so AP coverage is [7], [8], others reduce costs by adding WiFi dongles to strong. The motion-based challenges must stem from poor available USB slots [9]. Distributed synchronization and AP handoffs. The key challenges appear to be centered hardware calibration enables creation of a single, unified FinE-R 2015 Page 13 IROS 2015, Hamburg - Germany The path to success: Failures in Real Robots October 2, 2015 view from measurements collected across all WiFi monitors. R EFERENCES A global view can be used to infer aggregate performance [1] “Life telepresent: working vicariously through the beam metrics like number of active wireless clients, interference, robot,” ”http://www.theverge.com/2012/11/7/3611510/suitable- loss rates, and utlization [6], [7], [10] and even infer missing beam-robot-aims-for-bulletproof-telepresence”. [Online]. Available: ”http://www.theverge.com/2012/11/7/3611510/ packets [8]. These approaches are limited to the perspective suitable-beam-robot-aims-for-bulletproof-telepresence” of the wireless infrastructure and have difficulty accounting [2] D. Eckhardt and P. Steenkiste, “Measurement and analysis of the error for unreceived wireless client transmissions. In this paper, characteristics of an in-building wireless network,” in ACM SIGCOMM Computer communication review, vol. 26, no. 4. ACM, 1996, pp. we view the wireless network from the perspective of the 243–254. wireless client by accounting for the client’s movement and [3] R. Fish, M. Flickinger, and J. Lepreau, “Mobile emulab: A robotic considering the client’s intent of transmitting wireless data. wireless and sensor network testbed,” in IEEE INFOCOM, 2006. [4] P. Bahl and V. N. Padmanabhan, “Radar: An in-building rf-based Other efforts attempt to crowd-source collection of wire- user location and tracking system,” in INFOCOM 2000. Nineteenth less maps. These approaches end up sacrificing accuracy in Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings. IEEE, vol. 2. Ieee, 2000, pp. 775–784. order to easily collect measurements. GPS can be used to [5] M. Youssef and A. Agrawala, “The horus wlan location determination provide location estimates [11], [12] but it operates primarily system,” in Proceedings of the 3rd international conference on Mobile in outdoor environment and suffers from poor location esti- systems, applications, and services. ACM, 2005, pp. 205–218. [6] Y.-C. Cheng, J. Bellardo, P. Benkö, A. C. Snoeren, G. M. Voelker, and mates of around 3 meters. FM signals [13] similarly suffer S. Savage, Jigsaw: solving the puzzle of enterprise 802.11 analysis. from the effects of indoor environments and cannot achieve ACM, 2006, vol. 36, no. 4. accurate location estimates. Recent efforts to take advantage [7] C. Reis, R. Mahajan, M. Rodrig, D. Wetherall, and J. Zahorjan, “Measurement-based models of delivery and interference in static of powerful sensors including odometry, magnetometer, and wireless networks,” in ACM SIGCOMM Computer Communication WiFi found in cell phones have been shown to have an Review, vol. 36, no. 4. ACM, 2006, pp. 51–62. accuracy of 1.69 m [14], [15]. Roomba robots have also been [8] R. Mahajan, M. Rodrig, D. Wetherall, and J. Zahorjan, “Analyzing the mac-level behavior of wireless networks in the wild,” ACM SIGCOMM used to collect wireless coverage maps by spinning in small Computer Communication Review, vol. 36, no. 4, pp. 75–86, 2006. grid areas [16], [17] but they cannot autonomously navigate [9] P. Bahl, J. Padhye, L. Ravindranath, M. Singh, A. Wolman, and B. Zill, to reduce human time and effort costs or execute complex “Dair: A framework for managing enterprise wireless networks using desktop infrastructure,” in HotNets IV, 2005. motions like our robot can. Our work takes advantage of [10] S. Rangwala, R. Gummadi, R. Govindan, and K. Psounis, much more powerful sensors that can localize within 10 “Interference-aware fair rate control in wireless sensor networks,” in cm using a wheeled platform that can reproduce complex ACM SIGCOMM Computer Communication Review, vol. 36, no. 4. ACM, 2006, pp. 63–74. movements. [11] Y.-C. Cheng, Y. Chawathe, A. LaMarca, and J. Krumm, “Accuracy Previous efforts have proposed techniques to use predic- characterization for metropolitan-scale wi-fi localization,” in Proceed- tions to reduce the duration of handoffs or inform applica- ings of the 3rd international conference on Mobile systems, applica- tions, and services. ACM, 2005, pp. 233–245. tions to allow for prefetching data and reduce the impact of [12] R. Mahajan, J. Zahorjan, and B. Zill, “Understanding wifi-based handoffs [18], [19]. Nearby access points have also been used connectivity from moving vehicles,” in Proceedings of the 7th ACM to opportunistically help mitigate WiFi handoffs for moving SIGCOMM conference on Internet measurement. ACM, 2007, pp. 321–326. vehicles when moving across multiple buildings [20]. Our [13] Y. Chen, D. Lymberopoulos, J. Liu, and B. Priyantha, “Fm-based in- work that helps to expose and reproduce fine-grain failures door localization,” in Proceedings of the 10th international conference in AP handoffs for moving devices is orthogonal to these on Mobile systems, applications, and services. ACM, 2012, pp. 169– 182. efforts as it provides a mechanism for robustly evaluating [14] H. Wang, S. Sen, A. Elgohary, M. Farid, M. Youssef, and R. R. handoff solutions. Choudhury, “No need to war-drive: Unsupervised indoor localization,” in Proceedings of the 10th international conference on Mobile systems, applications, and services. ACM, 2012, pp. 197–210. V. C ONCLUSION [15] A. Rai, K. K. Chintalapudi, V. N. Padmanabhan, and R. Sen, “Zee: Zero-effort crowdsourcing for indoor localization,” in Proceedings of Diagnosing wireless connectivity issues can be difficult the 18th annual international conference on Mobile computing and due to the many factors that potentially impact wireless networking. ACM, 2012, pp. 293–304. [16] S. Sen, B. Radunovic, R. R. Choudhury, and T. Minka, “Spot localiza- performance. We showed how autonomous robots can help tion using phy layer information,” in Proceedings of ACM MOBISYS, to methodically drill down to the root cause by capturing 2012. detailed wireless measurements that eliminate unlikely fac- [17] O. Rensfelt, F. Hermans, P. Gunningberg, L.-Å. Larzon, and E. Björnemo, “Repeatable experiments with mobile nodes in a re- tors. We were able to identify AP handoffs as the reason for locatable wsn testbed,” The Computer Journal, vol. 54, no. 12, pp. our own robot’s motion-based wireless connectivity issue by 1973–1986, 2011. analyzing variations in wireless performance while subjected [18] U. Javed, D. Han, R. Caceres, J. Pang, S. Seshan, and A. Varshavsky, “Predicting handoffs in 3g networks,” in Proceedings of the 3rd ACM to repeatable motions. This was a challenging wireless prob- SOSP Workshop on Networking, Systems, and Applications on Mobile lem that arose from poor decisions dependent on accurate Handhelds. ACM, 2011, p. 8. timing and it is unclear that we could have uncovered them [19] S. Seshan, H. Balakrishnan, and R. H. Katz, “Handoffs in cellular wireless networks: The daedalus implementation and experience,” without the accuracy and control of autonomous robots. Wireless Personal Communications, vol. 4, no. 2, pp. 141–162, 1997. Opportunities for future work include using these detailed [20] A. Balasubramanian, R. Mahajan, A. Venkataramani, B. N. Levine, wireless maps for better management of enterprise wireless and J. Zahorjan, “Interactive wifi connectivity for moving vehicles,” in ACM SIGCOMM Computer Communication Review, vol. 38, no. 4. networks, ensuring timely maps for wireless localization ACM, 2008, pp. 427–438. solutions, and automated diagnosis of wireless problems. FinE-R 2015 Page 14 IROS 2015, Hamburg - Germany The path to success: Failures in Real Robots October 2, 2015