As e-commerce continues its rapid expansion, the challenges surrounding delivery are becoming more pronounced. The surge in trafic, environmental concerns, and heightened customer expectations have compounded the complexities of the delivery process. Customers now demand quicker deliveries within increasingly narrow timeframes, placing greater pressure on last-mile logistics, a pivotal yet costly aspect of the delivery chain. To address these challenges, fully autonomous last-mile delivery robots ofer a promising and sustainable solution for eficient deliveries to their final destinations. Nevertheless, despite their advanced autonomous capabilities, it is widely acknowledged that, at least in the foreseeable future, autonomous robots operating in urban environments will frequently encounter situations beyond their capabilities. Factors such as road obstructions, adverse weather conditions, congested intersections, or human interactions may necessitate the intervention of a remote human operator. This work seeks to explore the specifications and design of a teleoperation interface tailored for remote human operators, enabling them to eficiently manage a multitude of delivery robots simultaneously.
With the growth of e-commerce, developing faster, more
affordable, and more sustainable last-mile delivery solutions
are needed. Autonomous Last-Mile Delivery Robots
(LMDRs) are emerging technologies which is seen as a
promising solution for delivery challenges in the near future [
Like other AVs, LMDRs are equipped with various
sensors enabling their autonomous mobility. These sensors
can include cameras, LIDAR, ultrasonic, and radar for
sensing objects in the environment. They are also equipped
with inertial measurement units (IMU) and global
positioning systems (GPS) used for navigation [
Despite their clear benefits, LMDRs are not flawless.
First, there are inherent limitations like their limited
delivery radius [
To address the situations in which LMDRs cannot handle independently, a remote human operator would be called upon for assistance. The current research seeks to explore how to design a teleoperation interface that would enable a remote operator to eficiently manage one or more LMDRs remotely. We take a human-centered design (UCD) approach by first understanding the needs and requirements of the users (i.e., the remote operators) and the task. Therefore, we conducted interviews with fiteen related stakeholders from the industry with the focus of understanding the main problems that might require remote assistance of LMDRs as well as the various issues that remote operators might have.
We conducted 15 interviews with industry stakeholders in the field of autonomous delivery robots. The semistructured interviews aimed at extracting the main issues experienced by teleoperators. To recruit participants, we contacted professionals through Linkedin, companies, and via networking. Table 1 shows the list of participants.
We conducted semi-structured interviews with the participants. While we aimed to ask the same questions, their diferent roles and associations to the field have led the interviews in slightly diferent directions, to allow participants to elaborate on the things they are more experienced and familiar with. The main question that led all the interviews was about the main challenges that a remote opera
Role description P1 built a prototype and conducted 3 experiments that include LMDR traveling from a starting point to a destination point The experiments were conducted in major cities across Europe.
P2 has been working in automotive industry for approximately 20 years. For the past 10 years P2 worked for a company that focused on indoor and outdoor robots The compaby provides and maintains LMDRs for various costumers.
The software of the company founded by P3 enables people to remotely operate, assist, and supervise vehicles from thousands of miles away including forklifts, tuggers, robots, trucks, and more.
The participant has worked for the past few years in a company that enables remote supervision and teleoperation of diferent vehicles and robots including LMDR. Works for the past 2 years in a company that developed market-leading on-demand transportation services, navigation products, and other mobile applications. Conducted experiments usind LMDR in various countries.
Managing projects at an asian company aiming at providing movement of people and goods via the use of autonomus robots. Tele-operation function has is been developed duo to a growing need for remote control in extreme cases.
The company enables teleoperations on public roads and public networks. It provides both a tele-driving platform that is costumable by the client and teledrivers service. The company develops and maintains robotic systems. The company deals mainly with teleoperation of heavy industrial indoor machines. In addition, it also deals with remote driving of outdoor industrial machines as well as some LMD that includes a teleoperation interface due to regulatory considerations.
The company has vast experience in the operation of drones and some additional experience in ground robotics operation especially for millitary purposes. Recently the company began collaborating with large vehicle and delivery compenies in the fiels of LMDR.
The company has created a robot that is a multi purpose helper to humanity. One of its services is LMD. They operate curently in a few ereas and cmpuses in USA. For the past 7 years works in product and business development roles in an AV simulation platform startup. The platform includes the option to perform teledriving simulations with AVs.
In the framework of the studio he was responsible for designing an autonomous LMD robot for a leading autonomous sidewalk delivery company that develops and operates robots that serve people in public spaces, mainly for the food industry. p13 founded a bootstrap company that assisted robots companies in autonomy development.
Responsible for the warehouses that maintain the delivery robots, responsible for providing technical support for the robots and the remote drivers.
The job entails finding new technologies that fit the strategic needs of the company. Now the company works on POC in the field of LMDR. It will soon begin experiments using a delivery ground robot in a controlled environment. tor of LMDRs encounters when performing their job. Interviews lasted around 50 minutes. Most of the interviews were held online and one was face-to-face. Interviews were recorded and transcribed for further analysis. A thematic analysis was conducted to extract the main issues. The analysis focused on two main questions: the first is when and why operators need to intervene and the second is the challenges they may experience while working
The analysis of the interviews is currently a work in progress. We report here on the initial findings from the analysis of 7 of the interviews.
Generally, teleoperation centers work like any other call center. Requests for assistance are often managed by an “administrating system” that channels them to the operators according to some criteria. These criteria and the way incoming calls are handled may difer among companies. A few models were described for assigning operators to handle requests. One of the interviewees described a situation he observed (as a bystander) in which five robots were standing in front of a crossroad but not crossing. His assumption was that this situation occurred because of the availability of operators. Since the common ratio today is one (operator)-to-many (robots), it is possible that in this situation there were enough operators to help all the robots to cross the road (a situation that requires one operator per robot).
The user interface for controlling the robots seems to vary greatly between companies both in terms of the number and size of the screens and the type and amount of information being displayed. For example, the information can be presented on a single or multiple monitors, for a single or multiple robots. The type of information can include any of the following (often in various combinations): front camera footage (of one or more robots), location and speed of robots, meta-data from the robot (e.g., type of vehicle), mode of operation (autonomous or manual) and much more. In essence, the information being displayed is based on the priorities of each company.
There is a great variety of issues requiring remote interventions. The interviewees described several types of requests that can be crudely divided into three main categories: 1. “Go/No-Go” question when arriving at identified locations; This refers to situations in which the robot requests permission to proceed when getting to a crossroad or another point that was predefined as a place requiring an operator’s assistance. 2. a robot gets stuck or cannot handle the situation independently. Robots can get stuck for many reasons, for example, their wheels might get stuck or obstacles block their pre-planned path and there is no alternative path to consider (as they are allowed to follow only certain paths). Obstacles can be fixed objects (e.g. a tree has fallen) or people or other moveable objects blocking the way. ROs can remotely drive the robot or choose a command to control it. 3. Communication problems. These often refer to loss or reduced quality of GPS or WIFI signal.
A few interviewees noted that sidewalks are a more dificult operational area compared to a road environment. Roads are more structured than sidewalks, the trafic generally moves in clear directions, and it is less crowded. It is relatively easier for autonomous technology and the operator to find a path for the robot to follow. Contrarily, the trafic on sidewalks is less predicted, denser, and highly dynamic – a bus may stop and block the way, construction work, people crowding, etc, which makes it harder for the operator to guide the robot through.
A few interviewees noted that teleoperators often follow scripts that guide them on how to handle various situations and instruct them on the steps that should be taken for a given problem. The scripts vary by the type and complexity of the problem. The more complicated the problem is, the more actions the operator is likely to be instructed to take.
Like with teleoperators of other vehicles, situation awareness (SA) is a major challenge for LMDRs teleoperators. Based on the interviews, we observed a few types of threats for situation awareness. First, problems in the communication (e.g., no video input) may be wrongly interpreted as no change in the scene/environment, which results in poor situation awareness. Another reason is the quick-shifting between the control of remote robots, which requires operators to recalibrate their situation awareness to the new situation and environment in hand. It takes a few seconds to look at the camera, understand what happens, and become aware of the situation. The lack of continuity and the need to rapidly develop an understanding of the situation requires a lot of focus and attention and may demand a high cognitive load.
Operators are expected to solve problems quickly while not always knowing what the problem is or what they need to do. This can induce a lot of stress and high cognitive load. Some operators may deal with these stressful situations by trying to fix the problem by addressing the symptoms and not the root cause. In other situations, they may prefer to call the onsite technician teams. While sometimes an onsite team can be the only solution, this is often a costly solution which preferably should be avoided.
Part of the role of teleoperators is to assist the customer if he or she has any problem with the robot. One interviewee noted that there might be situations when the request or assistance is not initiated by the robot, but rather by the customer who is facing a problem (e.g., the lid does not open). The remote operator should be able to communicate with the customer as well as be able to resolve simple problems remotely.
Last-mile delivery robots are an emerging technology that is seen as a promising solution for delivery challenges in urban areas. The current work investigates the teleoperation of such robots. This is a first step toward the design of a teleoperation interface that would enable to resolve the various issues that these autonomous robots face and cannot solve autonomously. Following the complete analysis of the interviews, we plan to make a complete list of use cases and tasks for which an LMDR needs assistance. This will be followed by the design and evaluation of a user interface for a remote LMDR operator.
This work was funded by the Israeli Smart Transportation Research Center (ISTRC).