Endowing Mobile Robot Teams with Ambient Intelligence for Improved Patient Care

Abstract

By networking mobile robots, personal smart devices, and smart space networks, we can provide for a more accurate data for patient care than when the former are used individually. We call this network of personal and smart space devices and robots “Robot-Assisted Ambient Intelligence (RAmI)”. Even more, with the application of distributed network optimization, not only can we improve the assistance of an individual patient, but we can also minimize conflict or congestion over multiple patients' usage of limited resources that are spatially and temporally constrained in such a system. The emphasis of RAmI is on the efficiency and effectiveness of the physical assistance of multiple users and on the influence of individual robot actions on the desired system's performance. In this paper, we propose a distributed RAmI system and put the basis for the architectural setup of such a system. This distributed system should be modular and should facilitate fast decision-making to multiple agents over limited available resources. The proposed architecture is showcased by means of a case study.

Keywords

• Service robotics
• ambient intelligence
• ambient assisted living
• multirobot systems
• multi-agent systems
• patient care

-

Ambient Intelligence (AmI) uses multiple sensors fixed in a smart space to assist user’s activities through recommendation, guidance, and appliance control. However, AmI is not capable of interacting with a user by physical contact since its user interfaces are usually tactile, auditive, and/or visual. On the other hand, mobile robots with installed robot arms are capable of physical user interaction, though with a world view that is limited to their local sensory and communication capabilities, see, e.g., [ 1 ].

The quality of service provided by mobile robot teams (MRT) to simultaneous multiple patients and elderly with decreased mobility depends on the efficiency of the robots’ coordination with one another and with humans. To keep a good MRT performance in simultaneous multiple tasks, an updated task information is required. Even though the MRT quality of service depends on the quality of the available information that can be facilitated by maintaining the MRT connectivity [ 14 ], MRT task assignment can be performed both in perfect (e.g., [ 2,4 ]) and imperfect robot networks, e.g., [ 9 ]. Due to the loss in the information quality, the efficiency of a MRT in the task execution can fall rapidly, e.g., [ 9,10 ]. The strategy to employ to mitigate this problem depends also on the environment that can be collaborative, neutral, or adversarial [ 9,10 ]. Providing redundant robots to keep the network’s connectivity is a possible approach to this problem. However, it is costly and can create congestion in narrow spaces. This is why, here, we propose to network mobile robots, patients’ smart devices, and AmI networks, such that we can use more accurate data for decision-making than when the former are used individually. Even more, with the application of distributed network optimization, not only can we improve the assistance of an individual patient, but we can also ensure that robots’ actions that are geographically and temporally constrained in the usage of limited resources do not result in conflict or congestion. We call this network of AmI, personal devices, and robots “Robot-Assisted Ambient Intelligence (RAmI)”. The emphasis of RAmI is on the quality of service in simultaneous multiple patients’ assistance and the influence of individual robot decisions on the desired system’s performance. One of the issues of RAmI in large installations is its computational efficiency. Mobile robot teams are intrinsically decentralized and should act quickly and efficiently in real-time in large smart spaces, e.g., [ 3,4 ].

A distributed ROS-based AmI architecture DAmIA integrating robotic and AmI sensors for human tracking has been proposed in [ 13 ]. A survey of cloud robotics that leverages the ad-hoc cloud formed by communicating robots, and an infrastructure cloud was presented in [ 6 ]. In [ 8 ], we proposed ORCAS architecture for manufacturing MRTs that configures and schedules robots based on robots’ and tasks’ semantic descriptions. In ORCAS, every robot is considered a collaborative agent whose architecture is made of three layers: semantic, scheduling and the execution layer. The aim of the semantic layer is to find feasible robots’ configurations which can satisfy customer demand based on given semantic descriptions about factory setting, available resources and product specifications. The semantic layer generates compatible subsets of resources for the given tasks. The scheduling layer determines robot-task assignments and sequencing of tasks assigned to each robot configuration considering task interrelations and the robot assembly capacities. The objective is to seamlessly optimize robots’ performance by dynamic reconfiguration and rescheduling in case of contingencies thus minimizing overall assembly costs and off-line times. The solution is found through distributed minimization of total production time and cost considering resource combinations obtained from the semantic layer. We apply a modification of dynamic auction-based negotiation [ 9 ]. The execution layer monitors the correct execution of the schedule in real-time. In case of unpredicted contingencies, the objective here is to carry out local actions to minimize their effects. The schedule’s quality and stability are controlled in real-time, e.g., [ 5 ].

In this paper, we formulate the RAmI problem in x2 and in x3 present its architecture for task assignment and routing of MRTs in congested AmI networks. The principles of the proposed architecture are demonstrated by means of a case study in x4. 2

Problem formulation

We represent a smart building layout by an undirected graph G = (N; A), where N is a set of smart space agent (SA) nodes representing rooms, offices, halls, and, in general, a relatively small portion of space within a building. Each arc (i; j) 2 A has an associated travel time tij , which depends on its length and the relative congestion. Each SA is responsible of monitoring its surrounding area (by, e.g., iBeacons and cameras) to locate users’ momentary positions and compute space congestion. Moreover, we assume that each mobile robot agent assumed with a limited communication range is positioned in one of nodes n 2 N and it can communicate with the rest of robots within its communication range and with the belonging SA. Alike, each user is represented by a user agent u installed on an app of a user’s personal smart device (e.g., tablet or a smartphone) containing user-relevant info and able to communicate with the closest SA and the robots if located within their communication range.

We consider multiple simultaneous item delivery by a MRT to patients in a building. The most frequent items for delivery to patients are a meal or a medicine. We assume that there is a set I N of item storage locations in the building. Furthermore, let O N and D N be the set of all robots and patients at their momentary positions, respectively. Moreover, we assume that the items are packaged such that only grasping is required to handle them. Then, the objective is to assign item delivery tasks to robots in O such that the overall delivery time is minimal considering travel time under congestion from the robots’ momentary locations through item storages in I, and delivering the items to patients in D. 3

RAmI architecture

The architecture used for the distributed coordination of robots in task assignment and routing is implemented in each one of the robots and is presented in Fig. 1. It contains semantic, scheduling and the execution layer. Contrary to ORCAS, in RAmI, we assume that robot configuration is fixed and that each robot’s delivery capacity is limited by maximum item’s weight and dimensions. Moreover, all resources and each item delivery are semantically described by a human operator: e.g., meal/medicine, time of delivery, weight, dimensions, and type of a meal/medicine. The semantic storage description contains the information of available items and their hospital locations.

In the semantic layer, a set of compatible robots for each patient demand is found by using a DL inference engine and SPARQL query language. Scheduling layer contains the task assignment and route planning module. Based on the semantically described delivery demand, each robot agent o 2 O coordinates with other robot agents for the task assignment through the bi-level task assignment algorithm in [ 7 ]. While MRT is responsible of the MRT task assignment, the AmI network is responsible of updating the travel times under congestion in the network and distributively optimizing robots’ routes by using the route finding algorithm in [ 11 ]. Robots receive updated routes and travel times info from the belonging SA. In the execution layer, the individual performance is monitored in real time and in case of unpredicted events, a robot tries to coordinate locally with its neighbors to lower their impact. If the local coordination is not sufficient, the scheduling layer recomputes the robots’ routes. In the case of larger contingencies that make the schedule infeasible or the addition of robots that can improve the MRT’s performance, semantic layer updates matchings between the tasks and the MRT. 4

Case study

We demonstrate the functionality of the proposed approach by means of a simple case study example in Figure 2. Given is a simple scenario of a building network with 5 nodes and 6 arcs. There are two mobile robots positioned at o1 and o2, two patients (at d1 and d2), and inventory node i. Moreover, given are arcs’ travel times tij , [min] for each arc (i; j). Patients’ delivery items are ontologically described through RDF. The objective is to find routes from the robots’ positions o1 and o2 through inventory i to patients d1 and d2 that minimize the overall patient delivery time.

Let us assume that both robots can deliver the demands of both patients d1 and d2. Then, in the scheduling layer, the robots get assigned to patients’ demands (tasks) following steps in the MRTA algorithm [ 2 ] based on the updated paths with shortest travel times given by SAs. The travel time computation is done by the AmI network where SA nodes compute distributively the routes through [ 11 ].

Let us analyze this simple example. Robots start the task assignment through [ 7 ]. From o1 to d1 and from o2 to d2, there is only one simple path available passing through i. The overall cost of this assignment is 16. From o1 to d2 and from o2 to d1, there are four simple paths available for each one of the patient nodes d1 and d2. The overall cost of optimal paths (o1; i); (i; d2) and (o2; i); (i; d1) is also 16. Since both assignments have the same cost, the solution is found lexicographically.

In the case of contingencies during the moving from one node to another, the robots try to coordinate among themselves by locally recomputing their routes by following the algorithm in [ 7 ]. If the solution is unsatisfactory, they recompute routes in the scheduling layer. If one of them breaks, then the other recomputes its route starting from the semantic layer.

In case of high travel time variations, robots should be able to reroute. This is where SA agents play a crucial role in observing congestion and updating travel times. The SA agents compute the routes and inform the robots of the available routes’ arrival times. MRT performance depends on the navigational maps (i.e. areas where the robots can safely go) by tracking human trajectories and integrating them within the probabilistic map which is built directly through the conventional sensory readings (see, e.g., [ 12 ]). Acknowledgements. This work has been partially supported by the COMRADES project within the framework “Fonds d’amorc¸age Sante´” by Institut Mines Telecom in France.

References

1. Chibani , A. , Amirat , Y. , Mohammed , S. , et al.: Ubiquitous robotics: Recent challenges and future trends . Robotics and Autonomous Systems 61 ( 11 ), 1162 - 1172 ( 2013 )

2. Giordani , S. , Lujak , M. , Martinelli , F. : A distributed algorithm for the multi-robot task allocation problem . In: Garc´ ıa- Pedrajas , N.e.a. (ed.) Trends in Applied Intelligent Systems, Proceedings of the 23rd Int. Conf. on Industrial Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2010 , LNCS , vol. 6096 , pp. 721 - 730 ( 2010 )

3. Julian , B.J. , Angermann , M. , Schwager , M. , Rus , D. : Distributed robotic sensor networks: An information-theoretic approach . The Int. J. of Robotics Res . 31 ( 10 ), 1134 - 1154 ( 2012 )

4. Kantaros , Y. , Zavlanos , M.M. : Global planning for multi-robot communication networks in complex environments . IEEE Transactions on Robotics 32 ( 5 ), 1045 - 1061 ( 2016 )

5. Katragjini , K. , Vallada , E. , Ruiz , R.: Flow shop rescheduling under different types of disruption . International Journal of Production Research 51 ( 3 ), 780 - 797 ( 2013 )

6. Kehoe , B. , Patil , S. , Abbeel , P. , Goldberg , K. : A survey of research on cloud robotics and automation . IEEE Transactions on automation science and engineering 12(2) , 398 - 409 ( 2015 )

7. Lujak , M. , Billhardt , H. , Ossowski , S. : Distributed coordination of emergency medical service for angioplasty patients . Annals of Mathematics and Artif. Intell . 78 ( 1 ), 73 - 100 ( 2016 )

8. Lujak , M. , Fernandez , A. : ORCAS: Optimized robots configuration and scheduling system . In: Proc. of the 30th ACM Symp. On Applied Computing . vol. 1 , pp. 327 - 330 ( 2015 )

9. Lujak , M. , Giordani , S. : On the communication range in the auction-based multi-agent target assignment . In: Bettstetter, C. , Gershenson , C. (eds.) Self-Organizing Systems . Proc. of the 5th Int. WS, IWSOS 2011 , LNCS , vol. 6557 , pp. 32 - 43 ( 2011 )

10. Lujak , M. , Giordani , S. , Ossowski , S. : Value of incomplete information in mobile target allocation . In: Klgl, F. , Ossowski , S. (eds.) Multiagent System Technologies . Proceedings of the 9th German Conference, MATES 2011 , LNCS , vol. 6973 , pp. 89 - 100 ( 2011 )

11. Lujak , M. , Giordani , S. , Ossowski , S. : Route guidance: Bridging system and user optimization in traffic assignment . Neurocomputing 151 ( 1 ), 449 - 460 ( 2015 )

12. Papadakis , P. , Rives , P. : Binding human spatial interactions with mapping for enhanced mobility in dynamic environments . Autonomous Robots 41 ( 5 ), 1047 - 1059 ( 2017 )

13. Petitti , A. , Di Paola , D. , Milella , A. , Lorusso , A. , Colella , R. , Attolico , G. , Caccia , M.: A network of stationary sensors and mobile robots for distributed ambient intelligence . IEEE Intelligent Systems 31 ( 6 ), 28 - 34 ( 2016 )

14. Sabattini , L. , Chopra , N. , Secchi , C. : Decentralized connectivity maintenance for cooperative control of mobile robotic systems . The International Journal of Robotics Research 32 ( 12 ), 1411 - 1423 ( 2013 )