Autonomous systems are becoming more prevalent across a diversity of industries and applications. The development and deployment of these systems is surpassing the promulgation of standards and regulations needed to govern their safety. As the potential applications of autonomous systems continue to broaden, segregating these systems from humans will become increasingly difficult and potentially not feasible in some contexts, such as human-machine teaming (HMT). A mechanism for categorising risk for HMT operations against levels of autonomy (LOA) and machine functions is proposed. The risk categorisation tool sits within a broader safety framework for HMT. The user centric framework will enable the safe operation of humans alongside machines in a teaming environment in which the machine will not be physically segregated from the human. A key factor to effective safety assurance is proportionality. Autonomous capabilities can vary widely for HMT operations, resulting in varying levels of risk. The proposed risk categorisation tool provides a mechanism for categorising risk for HMT operations.
The origin of the word autonomy stems from the Greek words “auto” meaning self and “nomos”
meaning governance [
While many LOA taxonomies have been proposed over the years, none of these taxonomies are specific to the application of human-machine teaming (HMT) [18]. [14] presents a framework for adaptive automation processes for human-robot teaming. While the framework presents varying LOA as a method for enhancing human-system performance, a taxonomy for categorising LOA for HMT is not presented.
There currently does not yet exist a globally agreed upon definition of HMT; however, the broader literature defines HMT, often coined the term human-autonomy teaming, around the notion of sharing authority to pursue common goals [12]. In the context of this research, HMT is defined as a combination of human and machine capabilities working together towards an aligned goal [20].
HMT operations have been actualised across a breadth of domains and applications, demonstrating a range of machine capabilities - what the machine is capable of doing - and machine functions - the role or purpose of the machine. LOA are an indication of machine capabilities as they describe the degree to which a system is automated and what level of human intervention is required [17]. How
© 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org) risk is measured and how safety is assured for HMT operations will differ depending on the varying capabilities that span across the spectrum that we call autonomy.
Currently, robust mechanisms for assuring the safety of autonomous systems are lacking across
most industries. There exists a patchwork of safety standards around robot systems, most prominently
in the industrial sector. ISO 15066 [
While standards such as ISO15066 “Robots and robotic devices — Collaborative robots” do exist
[
Risk assessment and management is a core pillar of safety assurance for systems, autonomous or
otherwise. Established as a scientific field in the 1970s, the practice of risk assessment and
management has matured significantly over the proceeding years and is now used across most
industries [
[
Subjective probability is a common approach to managing uncertainty in risk assessments [
A method for categorising HMT applications against machine capabilities, expressed through
LOA, and machine functions is proposed. Literature around LOA have propositioned taxonomies
specifying the degree to which a task is automated. While several taxonomies have been proposed [
At this level, the computer generates a list of decision op3ons that it selects from and carries out if the human consents. The human may approve of the computer’s selected op3on or select one from among those generated by the computer or the operator. The computer will then carry out the selected ac3on. This level represents a higher level decision support system that is capable of selec3ng among alterna3ves as well as implemen3ng the second op3on.
that ac3on, based upon a list of alterna3ves it generates (augmented by alterna3ves suggested by the human operator). This system, therefore, automates decision making in addi3on to the genera3on of op3ons (as with decision support systems).
The four LOA detailed in Table 1 were chosen as they reflect a more balanced relationship
between human and machine. Each of the levels demonstrates less of a hierarchical structure and
more of a collaborative relationship with opportunities for negotiation between the entities. HMT is
characterised by a more balanced relationship between human and machine with greater levels of
negotiation [12]. This type of relationship requires increased machine capability, which is why the
lower LOA identified in [
As machine capabilities cannot be isolated from machine functions, four machine functions were
have also been identified, Building on from [
Implemen1ng
The four machine functions identified represent the possible functions or purpose of a machine within HMT. The functions range from monitoring, which involves lower levels of decision making on part of the machine, through to implementing, which implies implementing decision making with or without human intervention.
To situate HMT operations in the context of machine capability, expressed through LOA, and machine functions, a categorisation matrix has been developed, and is depicted in Figure 1 below. The matrix is a tool for categorising HMT operations against three risk categories to support proportionate risk assessment and management of HMT operations.
The matrix presented in Figure 1 illustrates three risk categories for HMT operations. Risk category 1 encompasses capabilities that demonstrate lower levels of autonomy and greater levels of human supervision. Risk category 2 encompasses capabilities that demonstrate greater levels of autonomy that require less human supervision and risk category 3 encompasses capabilities that demonstrate high levels of autonomy that involve minimal human supervision. Situating HMT operations within these risk categories will ensure proportionate and effective safety assurance can be demonstrated through the broader HMT safety framework.
The presented risk categorisation matrix sits within a broader HMT safety framework as a mechanism for identifying appropriate levels of risk. The proposed broader framework will demonstrate the safety assurance of both entities -human and machine - within HMT. In a teaming context, the human role is less authoritative and more collaborative, as is demonstrated through increased opportunities for negotiation between the two entities [12].
Capturing all the broader risks that come with HMT can be challenging. As such, guiding principles have been developed to help guide users with identifying the risks of HMT. The guiding principles include: ● ● ● ● ●
Adaptability - understanding the capacity to which the human and the machine can adapt to their environment.
Goal setting and goal actualisation - as HMT is defined by the pursuit of a shared goal, it is necessary to understand how goals are determined and actualised for both humans and machines.
Communication - understanding how, what, why and when information is communicated between human and machine.
Ethics - understanding the ethical implications of humans operating in close proximity to a machine within specific environments.
Trust - understanding how trust between the two entities influences decision making.
The HMT safety framework will provide assurance of both entities, and in addition to addressing physical safety, the framework will also include psychosocial considerations such as trust. The framework will address how a system or capability operates in a specific environment and, more importantly, how humans operate alongside these capabilities. The HMT safety framework will be targeted at the implementation stage, with specific focus on user experience. It will act as a guiding set of processes for users to follow to ensure the safe operation of humans alongside machines in a teaming environment.
Machine capabilities exist across a spectrum of autonomy. LOA applicable to HMT were identified alongside machine capabilities. These factors are used to categorise HMT operations against three levels of risk. Different machine capabilities and functions will yield different risks. The risks that come with lower capabilities and functions, and thereby lower levels of uncertainty, will differ to the risks that emerge from higher machine capabilities and functions that entail greater levels of uncertainty. It follows that different risk analyses need to be applied to ensure proportionate measures of safety are being implemented.
The next stages of this research will include further development of the three risk analysis categories. Each category will be developed against case study analyses across multiple industries to ensure the outputs are applicable across a diversity of industries. The final output will be a crosssector safety management framework for HMT.
The research for this paper received funding from the Australian Government through Trusted Autonomous Systems, a Defence Cooperative Research Centre funded through the Next Generation Technologies Fund.
12. Lyons, J. B., Sycara, K., Lewis, M., & Capiola, A. (2021). Human–Autonomy Teaming: Definitions, Debates, and Directions. Frontiers in Psychology, 12, 19–32. https://doi.org/ 10.3389/fpsyg.2021.589585 13. Matthews, M., Chowdhary, G., & Kieson, E. (2017). Intent Communication between Autonomous
Vehicles and Pedestrians. https://arxiv.org/abs/1708.07123v1 14. Parasuraman, R., Barnes, M., & Cosenzo, K. (2007). Adaptive Automation for Human-Robot
Teaming in Future Command and Control Systems. The International C2 Journal, 1(2), 43–68. 15. Parasuraman, R., Sheridan, T., B., & Wickens, C., D. (2000). A Model for Types and Levels of Human Interaction with Automation. IEEE Transactions on Systems, Man, and Cybernetics Part A: Systems and Humans, 30(3), 286–297. 16. Parker, J. (2021). The Challenges Posed by the Advent of Maritime Autonomous Surface Ships for International Maritime Law. Australian and New Zealand Maritime Law Journal, 35(1), 31– 42. 17. Sheridan, T., B., & Verplank, W., L. (1978). Human and Computer Control of Undersea Teleoperators (Mechanical Engineering, Massachusetts Institute of Technology) [Technical Report]. https://apps.dtic.mil/sti/pdfs/ADA057655.pdf 18. Vagia, M., Transeth, A. A., & Fjerdingen, S. A. (2016). A literature review on the levels of automation during the years. What are the different taxonomies that have been proposed? Applied Ergonomics, 53, 190–202. http://dx.doi.org/10.1016/j.apergo.2015.09.013 19. Vine, R., & Kohn, E. (2020). Concept for Robotic and Autonomous Systems V1.0. Joint Warfare
Council. https://tasdcrc.com.au/wp-content/uploads/2020/12/ADF-Concept-Robotics.pdf 20. Walliser, James. C., de Visser, Ewart. J., & Shaw, Tyler. H. (2019). Team Structure and Team Building Improve HumanMachine Teaming With Autonomous Agents. Journal of Cognitive Engineering and Decision Making, 13(4), 258–278. https://doi.org/10.1177/1555343419867563 21. Weinstein, N., Przybylski, Andrew. K., & Ryan, Richard. M. (2012). The index of autonomous functioning: Development of a scale of human autonomy. Journal of Research in Personality, 46, 397–413. http://dx.doi.org/10.1016/j.jrp.2012.03.007