An accident or incident event report contains useful information about evidence and it summarises the investigation results of probable causes [1]. Mostly the causes are due to either human errors or engineering failures. Whereas the number of accidents caused by the engineering failures has been reduced, the number of accidents caused by human errors is steadily increased. It is known that over 70% of accidents have human factors.
An event report is a good information source to identify such human errors. The descriptions of how these errors were made are often mentioned implicitly making it difficult to understand the real reasons of the errors. Understanding the infleunce of design on human performance requires phycological theories that explain human behaviour and cognition [2]. In particular, such theories help understand an operator's mistakes within his or her task perspectives. The accident report is important for designers and safety analysts because knowing the reasons of accidents can help them to design more reliable systems.
There are methods like Fault-Tree Analysis (FTA) that helps visualise information contained in accident reports. However, these methods are not designed to capture relevant concepts directly from reports. Manual identifications by human experts are thus required. The Semantic Web technique can reduce the reliance on the experts through automatic semantic annotations. A challenge is to identify central concepts and their relations that are essential for designers in understanding the users' behaviours affected by their designs [3].
An ontology is an explicit specification of conceptualization [4]. One of the main reasons of developing ontologies is to make domain knowledge explicit leading to more sharable. The ontology consists of concepts and relationships. In this research, the ontology is designed to support information sharing and extracting information from information sources. The main objective of this research is to develop an ontology that specifies information about human error and its relationships with designs. This paper presents the results of developing an ontology for the concept of Design-Induced Error (DIE). This research is based on the previous research that proposed a categorization of DIE [5].
Aviation accident reports are maintained by governments and published on the Web sites. Most of the reports are downloadable to end-users. For example, the Australian Transport Safety Bureau (ATSB) posts investigation reports into an official website [6]. Each report contains background information about an accident, e.g. aircraft model or accident date, and content in a free-text. The ATSB Web site provides a limited search that enables to search based on the background information. That is, it is difficult to retrieve the accidents only caused by Human Error that have relationship with Design Error. A reason is that extracting specific information from the unstructured texts requires analysing the texts from semantic perspectives. To address, this research has developed an ontology for the following objectives:
(1) To formalise relations between design of a system and human error by analysing accident reports (2) To visualise a model of the relations (3) To examine the possibility to capture a psychological theory from the reports
Through discussions with knowledge engineering experts and examination of current methodologies, the following development process was adapted as shown in Figure 1. (1) determine ontology domain and scope, (2) consider reusing existing ontologies, (3) enumerate important terms in the ontology, (4) define the classes and class hierarchies, (5) define the properties of classes-slots, (6) define the facets of the slots, and (7) create instances. For a concept annotation, the PCPACK tools were used [8]. Figure 2 shows an overview of various tools within the PCPACK software that accepts documents in txt or html formats and generates the annotation outcomes into xml form. It is used:
To annotate information extracted from documents To structure the annotation using various knowledge models (such as trees, diagrams, grids and hypertext) To acquire and validate knowledge from experts This study is based on the reports downloaded from the ATSB. The ATSB was chosen since: (1) the reports are easily accessible; and (2) investigators tend to pay more attention to identify and describe human error. In particular, the investigators have expressed the importance of recognising human error and the underlying reasons of it. The analysis is based on Reasons' human error theory [9].
The development consists of six stages each of which is described below.
This stage is to determine the domain and scope of the ontology. The main role of this ontology is to make information explicit in order for easier extractions. In order to determine the scope of the ontology, a list of questions that the ontology should answer was sketched as competency questions [10]:
(1) Which design-induced error characteristics should I consider when designing a system?
(2) How does a design lead operators to make an error? -which design concepts easily make operators fall into design-induced error phenomena e.g. gulf of evaluation?
(3) Can an inadvertent action of an operator be influenced by design error?
(4) Why did an operator fail to follow designed rules?
(5) Which characteristics of a phenomenon affect its appropriateness for design? (6) What are different perspectives between designer and operator in human system interaction failures? These questions will help determine whether the ontology developed could provide answers to these questions and the answers require a particular level of details or representations. Reuse of existing ontologies was considered but unfortunately a relevant reusable ontology was not found. The ontology had to be developed from scratch.
Stage 2: Knowledge elicitation: Defining testing documents Given the results of the Stage 1, the decision about what concepts and relations have to be modelled in the ontology has to be made. There are a number of knowledge elicitation methodologies e.g. an interview with domain experts or collecting documents that contain domain knowledge [11]. This research tried to collect documents (i.e. accident reports) that contain domain knowledge because it is costeffective and can reduce human intervention. It was conducted by the first author and based on the discussion between the author and experts in design and human error at the Innovative Manufacturing Research Centre (IMRC) in the University of Bath. It has the following steps:
(1) Screening for documents that contain "human-system interaction failure" by picking up cases that were caused by "operator error"
(2) Analysing the screened cases in terms of design-induced error by applying theories
(3) Clustering necessary concepts (e.g. error inducing design, human error) for the ontology development (4) Enumerating important terms (or phrase) by categorising terminology (keywords) that frequently appear or are used to express a concept (5) Classifying documents according to evidence (6) Selecting domain documents (testing document) that will be used for the knowledge acquisition and representation process
The authors have examined a total number of 556 accident reports found until February 2005. After conducting the manual description analysis, 48 reports were selected as domain documents (i.e. testing documents) for an ontology construction. These reports were stored into a database system (i.e. Microsoft Access) for further analysis.
This stage is to identify ontology concepts from domain documents. The Protocol Tool of PCPACK was used to analyse transcripts of accident reports chosen in the previous stage. The tool helps identify concepts as well as instances. The tool simulates the way someone would mark-up a page of text using highlighter pens. Each concept is associated with a different colour, for example, blue for design concept, red for human error as shown in Figure 3. This process was conducted simultaneously with a knowledge analysis (described in the next stage). Basic concepts, attributes, and relationships were predefined by the knowledge analysis. Terms and phrase in the domain document were extracted as instances of concepts. The PCPACK Ladder Tool and Diagram Template Editor were used to build concept and relation hierarchies. The ladder tool helps creating a tree-like hierarchical diagram by putting the classes into ladders. There are a concept ladder, a relation ladder, and an attribute ladder in the tool.
The Diagram Tool is used to create and edit diagrams. Concepts and relationships can be represented in the diagrams in the form of nodes and links. Nodes in a diagram represent knowledge objects in the knowledgebase, and links represent relationships between the knowledge objects. A diagram template determines the types of nodes and links used in a diagram. 48 accident documents cases were reconstructed with the tool as shown in Figure 5.
The Annotation Tool allows a page of information to be created and edited for each knowledge object (e.g. concept, attribute, task). The user can enter text or pictures to annotate what is known about that particular knowledge object. This tool uses a hypertext (html) format, hence words can be highlighted and linked to other pages. This allows web-like knowledge-structures to be constructed that can be based on the hierarchies produced in the Ladder Tool (if desired). Templates are used to define the structure, style and contents of annotation pages. These can include special commands to automatically insert information from the knowledgebase into the annotation page. This step is to help ontology developers to look back on previous stages in order to check missed, wrong concepts or relations and then to revisit previous steps in order to modify inappropriate results. Annotated documents were published the Web sites and the concepts and relations were checked several times. This process should continue for further examples and the authors expect the ontology to be continually refined because the ontology is not exhausted. Experts in human error and user (e.g. designers) can participate in further validation process.
The PCPACK Publisher Tool is used to publish the knowledgebase of designinduced error on a website as shown in Figure 6. As it was published in this way, PCPACK is no longer required to access the knowledge-base. Therefore, it is possible for the knowledgebase contents developed to be sent to other people and viewed by them without the need for PCPACK. This web browser can help to search and index relating concepts and their instances. This study provides 63 concepts, 16 relations, and more than 100 instances of the ontology in the knowledge-base.
This paper has presented the results of developing an ontology for DIE. The ontology was to model the concepts and relations related to DIE explicitly. According to the 'meta-theory', Design and Human error are related but it is difficult to notice the relationship in texts because their relations are often described indirectly and implicitly. It means that the interpretation of design functions and features that affect human cognition and performance need to be described more clearly. This research was an extension of the previous research that developed "meta-theory of design induced error" in order to address the relations between Design and Human Error.
The PCPACK used in this research is easy to use and effective in annotating texts with ontology concepts. However, it does not provide a learning capability that learns annotation patterns from examples. Manual annotation is time-consuming and errorprone, so automatic acquisition is required.
We plan to evaluate the usefulness of the DIE ontology with designers. In particular, we are interested in identifying whether or not the ontology helps better understanding the relationships between Design and Human Error.
Acknowledgments. The authors acknowledge the support of the Korean Government (MOL), EPSRC, and Rolls-Royce plc through the UTP for Design.