Engaging stakeholders in capturing contextual requirements for automated transport systems can be a challenging task, especially when they are locally distributed and limitations such as traveling restriction hinder personal presence. This paper presents experiences from hybrid site visits applied in contextual inquiry studies related to automated ground transport systems. The results rely on in-depth interviews and a follow-up survey with organizers and participants of the hybrid site visits. Initial ifndings reveal that physical participation is the preferred mode of attendance for contextual inquiry. Online participation may complement physical observations. However, the interaction between remote and on-site participants needs to be well facilitated, e.g. by own remote facilitator taking care of questions in the online chat.
Capturing user requirements and understanding the context of use represents a main challenge
for ubiquitous computing systems [
A large number of methodologies for capturing requirements have been proposed, reaching
from traditional requirements engineering [
When dealing with automated transport logistics, such contextual investigations can be realized by so-called site visits. Site visits allow stakeholders to experience the logistics site, to inspect current workflows as well as working conditions and propose future opportunities for automation and assistance technologies. However, limitations such as (1) geographical distribution, (2) time and budget constraints, (3) restrictions due to climate emissions, or (4) special health-related restrictions (during the COVID-19 pandemics), may hinder the feasibility of face-to-face site visits. This demands for adapted site visits formats, which allow for locally distributed, collaborative requirements elicitation.
In this paper, we report experiences from a recent requirements gathering activity related to
future automated road transport logistics systems in the European research project AWARD [
Requirements capturing was realized through a mixed-methods approach, which aimed at
gathering data through several channels, to gain deep insights into the workplace requirements
of future automated road transport logistics systems. Apart from the site visits discussed in this
paper, the data gathering methods included an online survey, remote in-depth interviews, and
an online international workshop (see [
At each of the four use case sites, visits were organized that maximized attendance in times of COVID-caused travel restrictions, but that still provided information on the opportunities and challenges. This was realized by hybrid site visits - a mixture of a workshop format and a guided tour through the respective site, which could be attended physically and virtually.
For each use case, site visits in person as well as virtual tours provided an opportunity to get familiar with the site and to discuss specific user requirements. In a virtual visit, participants were able to live stream the use case site and operations via a Microsoft Teams teleconference. Depending on COVID-19 restrictions at the time of the visit, sites were also able to provide in-person attendance. Overall, four site visits, each dedicated to one use case (cf. Table 1), were conducted. Apart from the port use case, all other use cases allowed on-site participation.
The representatives for each use case usually first presented the specifics of the project in regard to their on-site operations. Afterwards, a guided tour was conducted. In the tour, aspects such as the transport infrastructure, transport vehicles, loading/unloading points and workplaces of relevant process participants were able to observe real live operations. Furthermore, site visit participants could ask questions. This was done using a smartphone for live streaming and when regulations allowed, a video material was recorded, anonymized if required and made available for internal use among the project partners.
After the site visits, dedicated interviews with the site visit organizers from each use case were conducted. In the interviews, their experience of organizing, guiding the tour, interacting with participants, and conclusions on the positive aspects as well as drawbacks were discussed. For the participants in the site visits, both physical and virtual, a post attendance survey with open-ended questions was implemented to assess their experience of both modes of the visits.
Reports from the site visit participants as well as from the organizers express expectations about understanding of the on-site processes and operations in each mode of attendance (physical and virtual) and suficient learning about the complexity of the automated logistics tasks ahead. In general, the opportunity ofered in hybrid site visits to inspect the usage context in a flexible setup, either on-site or remote, to get familiar with the local conditions and to ask questions, was perceived as a much-appreciated format.
As expected, there were many positive comments about this format due to the opportunity to make appointments for larger groups of people and to fit the site visit to individual working schedules and companies’ policies, without being forced to travel. The main benefit experienced within the physical mode was the possibility to grasp the actual size of surroundings, nature of movement on the site, timings of operations, as well as vehicles and participants’ interactions. However, outdoor operations are generally noisy and the hybrid form of the site visit allowed physically present participants to use the online communication channel to hear what the guide was talking about. With the hybrid possibility, no information was lost due to unexpected or noisy conditions. This way, it was possible to observe the complete workflow without interrupting it. Provided that technical broadcasting was good, virtual participants could feel well-oriented about the important topics, get the impression of being emerged in the test site, and explore some details of the procedures with expert explanations.
More particular comparisons and suggestions for how to improve future hybrid site visits are further presented from the point of view of organizers and participants.
Among the organizers, the general agreement was expressed in favor of the physical mode of the visit which ofers the opportunity for more fruitful interaction in their opinion. In the light of COVID restrictions, this attitude also relates to the more general working conditions where meeting people in person for collaboration was missing since the start of the project.
From the practical perspective of the guide of the visit, one way to improve online interaction in future situations would be for participants to instruct the guide on the site depending on what they would like to see, and possibly a separate moderator for chat and questions. Learning about the site of operations and the processes, as well as having good interactions was not dependent solely on technical and practical issues. In fact, social gatherings in pre-COVID times such as having a cofee together, meeting in the evening for a drink in the bar or in the hotel are considered to deepen the talks and general understanding of the topics.
Animating people to be more talkative in online communication is needed generally, and this is reflected in the small number of questions. Providing more structured possibilities for discussion between physical and virtual participants in hybrid mode would be the way to tackle these limitations. Also, involving online participants through the perspective of a specific scenario beforehand and discussing it during the visit in smaller groups could help interaction.
On the part of the virtual mode of attendance, the video stream was the main source of getting familiar with the surroundings and the site of operations. In the on-site events, the participants were able to extend their visit, explore more, look around, and interact which resulted in more questions asked and more engaging participation. This was compared to previous online participation where it was somewhat expected that people would have a limited range of questions because of restricted insight into surroundings and operations.
Technical aspects of video presentation where stream and recording would ofer more detailed visual perspectives and protocols with more opportunities for interaction and discussion would significantly improve future virtual events as agreed by participants and organizers. Organizers also suggest a pre-recording of the routes and trajectories on the site that would be available to the participants at any time and allow further discussion and reflection.
The biggest benefit of the virtual site visits was the possibility to introduce the project and operations to as many people as possible compared to physical participation which is limited by the number of allowed participants, organizational matters and safety on the site. Yet, organizers’ belief is that for the core partnership and people involved in the actual development, physical attendance can ofer a better understanding because these actors also have domain knowledge that directs discussion about future operations on the site.
Regardless of the mode of presentation, organizers recognized additional challenges and limitations when presenting the use case during the site visit. For example, in public areas apply diferent regulations for trafic and participants than on closed sites. Presentation challenges range from having noisy environments to complicated regulations preventing actual access to an area or not being allowed to live stream or record the video. For example, not being able to have a tour inside the factory prevented getting insight into interactions with the logistics system, how participants in the process see themselves working with the system, etc.
As respondents in the post attendance survey reported on both modes of attendance almost in equal numbers. Since the majority had a chance to experience both modes, a direct comparison could be drawn. The responses pointed to aspects such as possibilities for interaction, evaluation of participant’s own role during the visit, how others’ presence afected the experience, technical aspects, tools and protocols used, and guidance of the site visit tour.
Concerning the virtual mode of presentation, there were expectedly fewer possibilities for interactions. The big number of participants in online meetings may have caused some insecurity for the participants to pose a question and some comments in the chat were missed. Physical presence benefited from having the opportunity to ask questions and reflect during the demonstration. It was also easier to interact with other participants but more dificult to communicate with the guide of the tour. The reports from participants typically considered on-site tours better for accessing a larger portion of the site and easier for asking questions. Issues that come with the virtual mode of attendance were mostly technical such as the camera angle or connection problems. On the part of the tour guides and the information about the on-site operations, participants appreciated good and instructive explanations and where the situation allowed, detailed visual inspections and close-ups of the operations that helped them to learn about the complete workflow and the infrastructure. In the future, for hybrid modes, a written program or agenda was suggested to help participants understand better what they see and in a proper context.
In addition to the limitations and benefits of the modes of attendance that depend on external factors, the participants were also asked about their role as participants, whether they took a more active or passive role and the reasons that contributed to it. For virtual participation, participants reported being more passive and believed that on-site visits and physical attendance would have provided a more active role. In online communication, being fully aware of other participants and their interests can be lacking while physical interaction can more easily lead to background discussions that are not on the agenda. Virtual visits could benefit from separating participants into smaller dedicated groups instead of general big groups. Those who reported an active role in the physical site visit, participated with questions and reflections when they could not fully understand the presentation. However, the possibility for discussions in physical interactions was afected by on-site noise that made it dificult to follow.
Engaging stakeholders in capturing contextual requirements can be a challenging task, especially when the targeted application contexts and use case sites are dificult to reach. In this paper, we systematically compare diferent setups of site visits. We could generally confirm that site visits are perceived as a suitable way for gathering a detailed understanding and learning about the complexity of investigated operations.
Providing hybrid formats for site visits was recognized both by organizers and participants as an eficient means to get every project participant a common level of knowledge. Concluding from our comparative analysis of two modes of hybrid site visits – physical vs virtual – we conifrmed our expectation that there are many reasons for certain aspects of physical participation to be the preferred mode of attendance. This is reflected in the need for spontaneous face to face interactions and discussions as a way of learning about the project. The challenge of how to serve both modes of attendance simultaneously in a hybrid setting is related to moderation of discussions, technical obstacles and noise. Additionally, besides restrictions imposed by the COVID-19 pandemic, physical attendance was recognized as challenging due to distance and time spent on travel. Although more limited in terms of the number of participants, some participants would still choose physical mode while for others the virtual visit was the only possible attendance. In the future, recommendations for fostering both modes of attendance envision a stepwise program that would provide common grounds and share information about the context, and guidance for the attendees during the whole session.
This work is part of the project AWARD, which has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101006817. The content of this paper reflects only the author’s view. Neither the European Commission nor CINEA is responsible for any use that may be made of the information it contains.