Evaluation, Verification, Validation and Refinement of Intelligent systems have been an important issue from the very beginning of their applications. These issues were an important research area and engineering aspect in the 80's and 90's. A number of conceptual approaches as well as practical tools were developed then.
With time the focus of research in the design of intelligent systems moved away from these topics, towards knowledge representation, discovery and processing, the Semantic Web technologies, and a number of other AIinspired areas. However, recently a number of researchers have realized that the lack of systematic methods and formal techniques for the design, evaluation and refinement is often an important reason for limited applications of even mature intelligent systems. Therefore, there is a growing need to reconsider some of the basic issues in this field. Today, in fact, the classic approaches to the Design, Evaluation, Verification, Validation and Refinement have to be assessed from the new perspectives in order to transfer their principles to new approaches and application fields. The practical design issues are of prime importance. The integration of Intelligent Systems with mainstream technologies and related design approaches from other areas, e.g., from Software Engineering, from Machine Learning, or from the Social Sciences, is especially important. The quality issues need to be considered as early as possible during the Design phase of the system.
The goal of the workshop was to promote and further a community-wide discussion of ideas that will influence and foster continued research concerning the topics of Design, Evaluation, and Refinement, as well as attract new researchers to the field. The objective was to focus on the contributions in the above fields and to provide an environment for communicating different paradigms and approaches, thus hopefully stimulating future cooperation and synergistic activities.
The proceedings contain the papers presented at DERIS 2010 held on September 13th, 2010 in Ilmenau, Germany. In total, we received 10 submissions, from which we were able to accept seven submissions based on a rigorous reviewing process, as regular research papers. Each submission was reviewed by at least 2 program committee members.
The topics of interest of the DERIS workshop series were mainly located in the area of Design, Evaluation, Verification, Validation, and Refinement and include but are not limited to:
• Principles in knowledge systems and ontology design • Detecting and handling inconsistencies and other anomalies within knowledge bases • Fundamentals and formal methods for verification of AI systems • Fundamentals and formal methods and techniques of validity assessment of AI systems, AI principles, and intelligent behavior in general • Special approaches to verify and/or validate certain kinds of AI systems: Rule-based, case-based • Special approaches or tools to evaluate systems of a particular application field • Knowledge base refinement by using the results of evaluation • Development and evaluation of ontologies • Maintenance and evolution of knowledge systems and ontologies • Explanation in the context of evaluation and assessment • Problems in system certification • Ontology and knowledge capture • Design and evaluation issues in automatic knowledge capture and knowledge discovery • Design and evaluation of semantic web applications and systems • Formal methods in verification and evaluation of intelligent systems • Case studies in design and evaluation and the lessons learned
The organizers would like to thank all who contributed to the success of the workshop. We thank the authors for their submissions, and especially thank the Program Committee for their good work in carefully reviewing and collaboratively discussing the submissions. For the submission and reviewing process we used the Easy-Chair system, for which the organizers would like to thank Andrei Voronkov, the developer of the system.
Martin Atzmüller Rainer Knauf