The 18th European Conference on Artificial Intelligence Proceedings 1st International Workshop on Combinations of Intelligent Methods and Applications (CIMA 2008) Tuesday July 22, 2008 Patras, Greece Ioannis Hatzilygeroudis, Constantinos Koutsojannis and Vasile Palade Copyright © 2008 for the individual papers by the papers’ authors. Copying is permitted for private and academic purposes. Re-publication of material from this volume requires permission by the copyright owners. Table of Contents Workshop Organization ……………………………………………………………………….. ii Preface …………………………………………………………………………………………… iii Papers ANN for prognosis of abdominal pain in childhood: use of fuzzy modelling for convergence estimation George C. Anastassopoulos and Lazaros S. Iliadis ……………………………………… 1 Using Genetic Programming to Learn Models Containing Temporal Relations from Spatio-Temporal Data Andrew Bennett and Derek Magee ………………………………………………………… 7 Combining Intelligent Methods for Learner Modelling in Exploratory Learning Environments Mihaela Cocea and George D. Magoulas ……………………………………………….. 13 Belief Propagation in Fuzzy Bayesian Networks Christopher Fogelberg, Vasile Palade and Phil Assheton ……………………………... 19 Combining Goal Inference and Natural-Language Dialogue for Human-Robot Joint Action Mary Ellen Foster, Manuel Giuliani, Thomas Muller, Markus Rickert, Alois Knoll, Wolfram Erlhagen, Estela Bicho, Nzoji Hipolito and Luis Louro ………………………. 25 A Tool for Evolving Artificial Neural Networks Efstratios F. Georgopoulos, Adam V. Adamopoulos and Spiridon D. Likothanassis .. 31 Intelligently Raising Academic Performance Alerts Dimitris Kalles, Christos Pierrakeas and Michalis Xenos ………………………………. 37 Recognizing predictive patterns in chaotic maps Nicos G. Pavlidis, Adam Adamopoulos and Michael N. Vrahatis ……………………... 43 Improving the Accuracy of Neuro-Symbolic Rules with Case-Based Reasoning Jim Prentzas, Ioannis Hatzilygeroudis and Othon Michail …………………….……….. 49 Combinations of Case-Based Reasoning with Other Intelligent Methods (short paper) Jim Prentzas and Ioannis Hatzilygeroudis ……………………………………………..... 55 Combining Argumentation and Hybrid Evolutionary Systems in a Portfolio Construction Application Nikolaos Spanoudakis and Konstantina Pendaraki and Grigorios Beligiannis ………. 59 An Architecture for Multiple Heterogeneous Case-Based Reasoning Employing Agent Technologies (short paper) Elena I. Teodorescu and Miltos Petridis ...................................................................... 65 Workshop Organization Chairs-Organizers Ioannis Hatzilygeroudis University of Patras, Greece Constantinos Koutsojannis TEI of Patras, Greece Vasile Palade Oxford University, UK Program Committee Ajaith Abraham, IITA, South Korea Ao Sio Iong, Oxford University, UK Plamen Agelov, Lancaster University, UK Emilio Corchado, University of Burgos, Spain George Dounias, University of the Aegean, Greece Artur S. d’Avila Garcez, City University, UK Melanie Hilario, CUI - University of Geneva, Switzerland Elpida Keravnou-Papailiou, University of Cyprus, Cyprus Rudolf Kruse, University of Magdeburg, Germany George Magoulas, Birkbeck College, Univ. of London, UK Vasilis Megalooikonomou, University of Patras, Greece Toni Moreno, University Rovira i Virgili, Spain Amedeo Napoli, CNRS-INRIA-University of Nancy, France Ciprian-Daniel Neagu, University of Bradford, UK Jim Prentzas, TEI of Lamia, Greece Han Reichgelt, Southern Polytechnic State Univ., GA, USA David Sanchez, University Rovira i Virgili, Spain Douglas Vieira, University of Minas Gerais, Brazil Contact Chair Ioannis Hatzilygeroudis Dept. of Computer Engineering & Informatics University of Patras, Greece Email: ihatz@ceid.upatras.gr ii Preface The combination of different intelligent methods is a very active research area in Artificial Intelligence (AI). The aim is to create integrated or hybrid methods that benefit from each of their components. It is generally believed that complex problems can be easier solved with such integrated or hybrid methods. Some of the existing efforts combine what are called soft computing methods (fuzzy logic, neural networks and genetic algorithms) either among themselves or with more traditional AI methods such as logic and rules. Another stream of efforts integrates case- based reasoning or machine learning with soft-computing or traditional AI methods. Some of the combinations have been quite important and more extensively used, like neuro- symbolic methods, neuro-fuzzy methods and methods combining rule-based and case- based reasoning. However, there are other combinations that are still under investigation. In some cases, combinations are based on first principles, whereas in other cases they are created in the context of specific applications. The Workshop is intended to become a forum for exchanging experience and ideas among researchers and practitioners who are dealing with combining intelligent methods either based on first principles or in the context of specific applications. There were totally 20 papers submitted to the Workshop. Each paper was reviewed by at least two members of the PC. We finally accepted 12 papers (10 full and 2 short). Revised versions of the accepted papers (based on the comments of the reviewers) are included in these proceedings in alphabetic order (based on first author). Five of the accepted papers deal with combinations of Genetic Programming or Genetic Algorithms with either non-symbolic methods, like Neural Networks (NNs) and/or Kalman Filters (Georgopoulos etal, Spanoudakis etal), or symbolic ones, like Decision Trees (Kalles etal) and Temporal Logic (Bennett and Magee). Another four papers deal with combinations of Case-Based Reasoning (CBR). One of them presents a short survey of CBR combinations (Prentzas and Hatzilygeroudis) and another one a combination with Agents (Teodorescu and Petridis). The rest two of them present CBR combinations with a Neuro-Fuzzy (Cocea and Magoulas) and a Neuro-Symbolic (Prentzas etal) approach respectively, leading to multi-combinations. Also, another two papers concern combinations of Fuzzy Logic with either NNs (Anastassopoulos and Iliadis) or Bayesian iii Nets (Fogelberg etal). Finally, one of the papers combines a NN-based approach with a Natural Language Processing one (Foster etal). Four of the above papers present combinations developed in the context of an application. Applications involve Medicine (Anastassopoulos and Iliadis), Education (Cocea and Magoulas, Kalles etal) and Economy (Spanoudakis etal). We hope that this collection of papers will be useful to both researchers and developers. Given the success of this first Workshop on combinations of intelligent methods, we intend to continue our effort in the coming years. Ioannis Hatzilygeroudis Constantinos Koutsojannis Vasile Palade iv ANN for prognosis of abdominal pain in childhood: use of fuzzy modelling for convergence estimation George C. Anastassopoulos, Lazaros S. Iliadis Abstract. This paper focuses in two parallel objectives. First it decision rules can predict which children are at risk for aims in presenting a series of Artificial Neural Network models appendicitis (appendicitis is the most common surgical condition that are capable of performing prognosis of abdominal pain in of the abdomen). One such numerically based system is based on childhood. Clinical medical data records have been gathered and a 6-part scoring system: nausea (6 point), history of local RLQ used towards this direction. Its second target is the presentation and pain (2 point), migration of pain (1 point), difficulty walking (1 application of an innovative fuzzy algebraic model capable of point), rebound tenderness / pain with percussion (2 point), and evaluating Artificial Neural Networks’ performance [1]. This absolute neutrophil count of >6.75 x 10`3/μL (6 point). A score <5 model offers a flexible approach that uses fuzzy numbers, fuzzy had a sensitivity of 96.3% with a negative predictive value of sets and various fuzzy intensification and dilution techniques to 95.6% for AA. perform assessment of neural models under different perspectives. To date, all efforts to find clinical features or laboratory tests, It also produces partial and overall evaluation indices. The either alone or in combination, that are able to diagnose produced ANN models have proven to perform the classification appendicitis with 100% sensitivity or specificity have proven with significant success in the testing phase with first time seen futile. Also, there is only one research work [4] in bibliography data. based on ANN that deals with the abdominal pain prognosis in childhood. The incidence of Acute Appendicitis (AA) is 4 cases per 1000 1 INTRODUCTION children. However appendicitis despite pediatric surgeons’ best efforts remains the most commonly misdiagnosed surgical The wide range of problems in which Artificial Neural condition. Although diagnosis and treatment have improved, Networks can be used with promising results, is the reason of their appendicitis continues to cause significant morbidity and still growth [2, 3]. Some of the fields that ANNs are used are: medical remains, although rarely, a cause of death. Appendicitis has a systems [4-6], robotics [7], industry [8 – 11], image processing male-to-female ratio of 3:2 with a peak incidence between ages 12 [12], applied mathematics [13], financial analysis [14], and 18 years. The mean age in the pediatric population is 6-10 environmental risk modelling [15] and others. years. The lifetime risk is 8.6% for boys and 6.7% for girls. Prognosis is a medical term denoting an attempt of physician to The 15 factors that are used in the routine clinical practice for accurately estimate how a patient's disease will progress, and the assessment of AA in childhood are: Sex, Age, Religion, whether there is chance of recovery, based on an objective set of Demographic data, Duration of Pain, Vomitus, Diarrhea, Anorexia, factors that represent that situation. The inference about prognosis Tenderness, Rebound, Leucocytosis, Neutrophilia, Urinalysis, of a patient when presented with complex clinical and prognostic Temperature, Constipation. The sex (males), the age (peak of information is a common problem, in clinical medicine. The appearance of A.A in children aged 9 to 13 years), and the religion diagnosis of a disease is the outcome of combination of clinical (hygiene condition, feeding attitudes, genetic predisposition) were and laboratorial examinations through medical techniques. in relation with a higher frequency for AA. Anorexia, vomitus, In this paper various ANN architectures using different learning diarrhea or constipation and a slight elevation of the temperature rules, transfer functions and optimization algorithms have been (370 C - 380 C) were common manifestation of AA. Additionally, tried. This research effort was motivated form the fact that reliable abdominal tenderness principally in the RLQ of the abdomen and and seasonable detection of abdomen pain constitute attainments in the existence of the rebound sign, are strongly related with AA. effective treatment of disease and avoidance of relapses. That is Leucocytosis (>10.800 K/μl) with neutrophilia (neutrophil count > why the development of such an intelligent model that can 75%) is considered to be a significant clue for AA. Urinalysis is collaborate with the doctors will be very useful towards successful useful for detecting urinary tract disease, normal findings on treatment of potential patients. urinalysis are of limited diagnostic value for appendicitis. The role of race, ethnicity, health insurance, education, access to healthcare, and economic status on the development and treatment 2 DIAGNOSTIC FACTORS OF ABDOMINAL of appendicitis are widely debated. Cogent arguments have been PAIN made on both sides for and against the significance of each Several reports have described clinical scoring systems socioeconomic or racial condition. A genetic predisposition incorporating specific elements of the history, physical appears operative in some cases, particularly in children in whom examination, and laboratory studies designed to improve diagnostic appendicitis develops before age 6 years. Although the disorder is accuracy of abdominal pain [16]. Nothing is guaranteed, but uncommon in infants and elderly, these groups have a Democritus University of Thrace, Hellenic Open University disproportionate number of compilations because of delays in anasta@med.duth.gr, liliadis@fmenr.duth.gr diagnosis and the presence of comorbid conditions. 1 As diagnosis, there are four stages of appendicitis, including (TanH) transfer function the input data were normalized (divided acute focal appendicitis, acute supurative appendicitis, gangrenous properly) in order to be included in the acceptable range of [-3, 3] appendicitis and perforated appendicitis. These distinctions are to avoid problems such as saturation, where an element’s vague, and only the clinically relevant distinction of perforated summation value (the sum of the inputs times the weights) exceeds (gangrenous appendicitis includes into this entity as dead intestine the acceptable network range [17]. Standard back-propagation functionally acts as a perforation) versus non-perforated optimization algorithms using TanH, or Sigmoid or Digital Neural appendicitis (acute focal and supurative appendicitis) should be Network Architecture (DNNA) transfer functions, combined with made. the Extended Delta Bar Delta (ExtDBD) or with the Quick Prop The present study is based on data set that is obtained from the learning rules [18, 19] were employed. The ExtDBD is a heuristic Pediatric Surgery Clinical Information System of the University technique reinforcing good general trends and damping oscillations Hospital of Alexandroupolis, Greece. It consisted of 516 children’s [20]. medical records. Some of these children had different stages of Modular and radial basis function (RBF) ANN applying the appendicitis and, therefore, underwent operative treatment. This ExtDBD learning rule and the TanH transfer function were also data set was divided into a set of 422 records and another set of 94 used in an effort to determine the optimal networks. RBFs have an records. The former was used for training of the ANN, while the internal representation of hidden neurons which are radially latter for testing. A small number of data records were used as a symmetric, and the hidden layer consists of pattern units fully validation set during training to avoid overfitting. Table 1 connected to a linear output layer [21, 22]. represents the stages of appendicitis as well as the corresponding cases for each one. The 3rd column of Table 1 depicts the coding of possible diagnosis, as they used for ANN training and testing 3.2 ANN evaluation metrics applied stages. Traditional ANN evaluation measures like the Root Mean Square Table 1. Possible diagnosis and corresponding cases. Error (RMS error), R2 and the confusion matrix were used to Diagnosis Coding Cases validate the ensuing neural network models. It is well known that Discharge -2 236 the RMS error adds up the squares of the errors for each neuron in Normal the output layer, divides by the number of neurons in the output Observation -1 186 No findings 0 15 layer to obtain an average, and then takes the square root of that average. The confusion matrix is a graphical way of measuring the Operative Focal appendicitis 1 34 treatment Phlegmonous or network’s performance during the “training” and “testing” phases. 2 29 It also facilitates the correlation of the network output to the actual Supurative appendicitis Gangrenous appendicitis 3 8 observed values that belong to the testing set in a visual display Peritonitis 4 8 [17], and therefore provides a visual indication of the network’s performance. A network with the optimal configuration should have the “bins” (the cells in each matrix) on the diagonal from the 3 NEURAL NETWORK DESIGN lower left to the upper right of the output. An important aspect of the matrix is that the value of the vertical axis in the generated Data were divided into two groups, the training cases (TRAC) and histogram is the Common Mean Correlation (CMC) coefficient of the testing cases (TESC). The TRAC consisted of 417 concrete the desired (d), and the actual (predicted) output (y) across the medical data records and the TESC consisted of 101. Each input Epoch. record was organised in a format of fifteen fields, namely sex, age, Finally, the FUSETRESYS (Fuzzy Set Transformer Evaluation religion, area of residence, pain time period, vomit symptoms, System) that constitutes an innovative ANN evaluation system has diarrhoea, anorexia, located sensitivity, rebound, wbc, poly, been applied offering a more flexible approach [1]. general analysis of urine, body temperature, constipation. The output record contained a single field which corresponded to the potential outcome of each case. 3.3 Technical description of the FUSETRESYS The determination if the TRAC and TESC data sets was performed in a rather random manner. The training and testing ANN evaluation model sample size which would be sufficient for a good generalization Fuzzy logic enables the performance of calculations with was determined by using the Widrow’s rule of thumb for the LMS mathematically defined words called “Linguistics” [1, 23-25]. algorithm which is a distribution free, worst case formula [2] and it FUSETRESYS faces each training/testing example as a Fuzzy Set. is shown in the following equation 1. W is the total number of free It applies triangular or trapezoidal membership functions in order parameters in the network (synaptic weights and biases) and ε to determine the partial degree of convergence (PADECOV) of the denotes the fraction of the classification errors permitted during ANN for each training/testing example separately. The following testing. The O notation shows the order of quantity enclosed within equations 2 and 3 represent a triangular and a trapezoidal [2]. ⎛W ⎞ (1) membership functions respectively [1]. N = O⎜ ⎟ ⎝ε ⎠ μs(x;a,b,c)=max{min{ x − a c − x },0} a