Since engineered systems, e.g. aviation control, have increasingly equipped with automated and computer-supported artifacts, human-system interaction has been an important issue. Understanding the infleunce of design on human performance requires phycological theories that explain human behaviour and cognition. One of the challenges of modelling such theories is to identify complex relations between systems and humans including different task perspectives. To address this problem, we have developed an ontology that specifies which relations are better comprehended when assisted with the theories and have used it to model human errors induced by designs. This paper presents the results of developing such ontology using a supporting tool, i.e. PCPACK.
An accident or incident event report contains useful information about evidence and it summarises the investigation results of probable causes [1]. Mostly the causes are due to either human errors or engineering failures. Whereas the number of accidents caused by the engineering failures has been reduced, the number of accidents caused by human errors is steadily increased. It is known that over 70% of accidents have human factors.
An event report is a good information source to identify such human errors. The descriptions of how these errors were made are often mentioned implicitly making it difficult to understand the real reasons of the errors. Understanding the infleunce of design on human performance requires phycological theories that explain human behaviour and cognition [2]. In particular, such theories help understand an operator’s mistakes within his or her task perspectives. The accident report is important for designers and safety analysts because knowing the reasons of accidents can help them to design more reliable systems.
There are methods like Fault-Tree Analysis (FTA) that helps visualise information contained in accident reports. However, these methods are not designed to capture relevant concepts directly from reports. Manual identifications by human experts are thus required. The Semantic Web technique can reduce the reliance on the experts through automatic semantic annotations. A challenge is to identify central concepts and their relations that are essential for designers in understanding the users’ behaviours affected by their designs [3].
An ontology is an explicit specification of conceptualization [4]. One of the main reasons of developing ontologies is to make domain knowledge explicit leading to more sharable. The ontology consists of concepts and relationships. In this research, the ontology is designed to support information sharing and extracting information from information sources. The main objective of this research is to develop an ontology that specifies information about human error and its relationships with designs. This paper presents the results of developing an ontology for the concept of Design-Induced Error (DIE). This research is based on the previous research that proposed a categorization of DIE [5].
Aviation accident reports are maintained by governments and published on the
Web sites. Most of the reports are downloadable to end-users. For example, the
Australian Transport Safety Bureau (ATSB) posts investigation reports into an
official website [6]. Each report contains background information about an accident,
e.g. aircraft model or accident date, and content in a free-text. The ATSB Web site
provides a limited search that enables to search based on the background information.
That is, it is difficult to retrieve the accidents only caused by Human Error that have
relationship with Design Error. A reason is that extracting specific information from
the unstructured texts requires analysing the texts from semantic perspectives. To
address, this research has developed an ontology for the following objectives:
(
Through discussions with knowledge engineering experts and examination of current methodologies, the following development process was adapted as shown in Figure 1.
applied methods Noy & McGuinness ontology
development method description analysis and metatheory of design-induced error
mark-up technique (PC PACK protocol tool) knowledge analysis technique (PC PACK ladder tool) knowledge representation
technique (PC PACK ladder, diagram tool) web browser publishing technique (PC PACK annotation and publish tool) development
stages determining the domain and scope of the
ontology knowledge elicitation: defining a domain
documents knowledge extraction from the domain documents knowledge analysis (generating concepts and
relations) knowledge model ing validation and publishing complete
refining concepts and relations source accident reports (web)
This is based on the development process proposed by Noy and McGuinness: [7]
(
To annotate information extracted from documents To structure the annotation using various knowledge models (such as trees, diagrams, grids and hypertext)
To acquire and validate knowledge from experts
This study is based on the reports downloaded from the ATSB. The ATSB was
chosen since: (
The development consists of six stages each of which is described below.
This stage is to determine the domain and scope of the ontology. The main role of
this ontology is to make information explicit in order for easier extractions. In order to
determine the scope of the ontology, a list of questions that the ontology should
answer was sketched as competency questions [10]:
(
These questions will help determine whether the ontology developed could provide answers to these questions and the answers require a particular level of details or representations. Reuse of existing ontologies was considered but unfortunately a relevant reusable ontology was not found. The ontology had to be developed from scratch.
Given the results of the Stage 1, the decision about what concepts and relations have
to be modelled in the ontology has to be made. There are a number of knowledge
elicitation methodologies e.g. an interview with domain experts or collecting
documents that contain domain knowledge [11]. This research tried to collect
documents (i.e. accident reports) that contain domain knowledge because it is
costeffective and can reduce human intervention. It was conducted by the first author and
based on the discussion between the author and experts in design and human error at
the Innovative Manufacturing Research Centre (IMRC) in the University of Bath. It
has the following steps:
(
The authors have examined a total number of 556 accident reports found until February 2005. After conducting the manual description analysis, 48 reports were selected as domain documents (i.e. testing documents) for an ontology construction. These reports were stored into a database system (i.e. Microsoft Access) for further analysis.
This stage is to identify ontology concepts from domain documents. The Protocol Tool of PCPACK was used to analyse transcripts of accident reports chosen in the previous stage. The tool helps identify concepts as well as instances. The tool simulates the way someone would mark-up a page of text using highlighter pens. Each concept is associated with a different colour, for example, blue for design concept, red for human error as shown in Figure 3. This process was conducted simultaneously with a knowledge analysis (described in the next stage). Basic concepts, attributes, and relationships were predefined by the knowledge analysis. Terms and phrase in the domain document were extracted as instances of concepts.
This stage is to define the concepts and relations and organise them into a hierarchy. It is to answer the questions of: What are relating concepts located in this process? How many of them we can capture in order to formalise?
(
This ontology is based on the meta-theory and on accident reports. The classification of concepts and the class hierarchy are therefore categorised and defined according to how the classification and terminology effectively capture the concepts.
With these preliminary defined concept categories, the classes and the class hierarchy were constructed using PCPACK ladder tool as shown in Figure 4. Knowledge captured in the previous step, a knowledge extraction step, can be automatically put into the related concepts.
(
In order to define the relationship between classes, an Entity and Relation (ER) diagram was drawn with classes. This ER diagram shows how the classes are related to each other. A total number of 16 relations were defined.
The PCPACK Ladder Tool and Diagram Template Editor were used to build concept and relation hierarchies. The ladder tool helps creating a tree-like hierarchical diagram by putting the classes into ladders. There are a concept ladder, a relation ladder, and an attribute ladder in the tool.
The Diagram Tool is used to create and edit diagrams. Concepts and relationships can be represented in the diagrams in the form of nodes and links. Nodes in a diagram represent knowledge objects in the knowledgebase, and links represent relationships between the knowledge objects. A diagram template determines the types of nodes and links used in a diagram. 48 accident documents cases were reconstructed with the tool as shown in Figure 5.
The Annotation Tool allows a page of information to be created and edited for each knowledge object (e.g. concept, attribute, task). The user can enter text or pictures to annotate what is known about that particular knowledge object. This tool uses a hypertext (html) format, hence words can be highlighted and linked to other pages. This allows web-like knowledge-structures to be constructed that can be based on the hierarchies produced in the Ladder Tool (if desired). Templates are used to define the structure, style and contents of annotation pages. These can include special commands to automatically insert information from the knowledgebase into the annotation page.
This step is to help ontology developers to look back on previous stages in order to check missed, wrong concepts or relations and then to revisit previous steps in order to modify inappropriate results. Annotated documents were published on the Web sites and the concepts and relations were checked several times. This process should continue for further examples and the authors expect the ontology to be continually refined because the ontology is not exhausted. Experts in human error and user (e.g. designers) can participate in further validation process.
The PCPACK Publisher Tool is used to publish the knowledgebase of designinduced error on a website as shown in Figure 6. As it was published in this way, PCPACK is no longer required to access the knowledge-base. Therefore, it is possible for the knowledgebase contents developed to be sent to other people and viewed by them without the need for PCPACK. This web browser can help to search and index relating concepts and their instances. This study provides 63 concepts, 16 relations, and more than 100 instances of the ontology in the knowledge-base.
This paper has presented the results of developing an ontology for DIE. The ontology was developed to model the concepts and relations related to DIE explicitly. According to the ‘meta-theory’, Design and Human error are related but it is difficult to notice the relationship in texts because their relations are often described indirectly and implicitly. It means that the interpretation of design functions and features that affect human cognition and performance need to be described more clearly. This research was an extension of the previous research that developed “meta-theory of design induced error” in order to address the relations between Design and Human Error.
The PCPACK used in this research is easy to use and effective in annotating texts with ontology concepts. However, it does not provide a learning capability that learns annotation patterns from examples. Manual annotation is time-consuming and errorprone, so automatic acquisition is required.
We plan to evaluate the usefulness of the DIE ontology with designers. In particular, we are interested in identifying whether or not the ontology helps better understanding the relationships between Design and Human Error.
Acknowledgments. The authors acknowledge the support of the Korean Government (MOL), EPSRC, and Rolls-Royce plc through the UTP for Design.