The wide range of problems in which Artificial Neural Networks can be used with promising results, is the reason of their growth [2,3]. Some of the fields that ANNs are used are: medical systems [4][5][6], robotics [7], industry [8 -11], image processing [12], applied mathematics [13], financial analysis [14], environmental risk modelling [15] and others.
Prognosis is a medical term denoting an attempt of physician to accurately estimate how a patient's disease will progress, and whether there is chance of recovery, based on an objective set of factors that represent that situation. The inference about prognosis of a patient when presented with complex clinical and prognostic information is a common problem, in clinical medicine. The diagnosis of a disease is the outcome of combination of clinical and laboratorial examinations through medical techniques.
In this paper various ANN architectures using different learning rules, transfer functions and optimization algorithms have been tried. This research effort was motivated form the fact that reliable and seasonable detection of abdomen pain constitute attainments in effective treatment of disease and avoidance of relapses. That is why the development of such an intelligent model that can collaborate with the doctors will be very useful towards successful treatment of potential patients.
Several reports have described clinical scoring systems incorporating specific elements of the history, physical examination, and laboratory studies designed to improve diagnostic accuracy of abdominal pain [16]. Nothing is guaranteed, but Democritus University of Thrace, Hellenic Open University anasta@med.duth.gr, liliadis@fmenr.duth.gr decision rules can predict which children are at risk for appendicitis (appendicitis is the most common surgical condition of the abdomen). One such numerically based system is based on a 6-part scoring system: nausea (6 point), history of local RLQ pain (2 point), migration of pain (1 point), difficulty walking (1 point), rebound tenderness / pain with percussion (2 point), and absolute neutrophil count of >6.75 x 10`3/μL (6 point). A score <5 had a sensitivity of 96.3% with a negative predictive value of 95.6% for AA.
To date, all efforts to find clinical features or laboratory tests, either alone or in combination, that are able to diagnose appendicitis with 100% sensitivity or specificity have proven futile. Also, there is only one research work [4] in bibliography based on ANN that deals with the abdominal pain prognosis in childhood.
The incidence of Acute Appendicitis (AA) is 4 cases per 1000 children. However appendicitis despite pediatric surgeons' best efforts remains the most commonly misdiagnosed surgical condition. Although diagnosis and treatment have improved, appendicitis continues to cause significant morbidity and still remains, although rarely, a cause of death. Appendicitis has a male-to-female ratio of 3:2 with a peak incidence between ages 12 and 18 years. The mean age in the pediatric population is 6-10 years. The lifetime risk is 8.6% for boys and 6.7% for girls.
The 15 factors that are used in the routine clinical practice for the assessment of AA in childhood are: Sex, Age, Religion, Demographic data, Duration of Pain, Vomitus, Diarrhea, Anorexia, Tenderness, Rebound, Leucocytosis, Neutrophilia, Urinalysis, Temperature, Constipation. The sex (males), the age (peak of appearance of A.A in children aged 9 to 13 years), and the religion (hygiene condition, feeding attitudes, genetic predisposition) were in relation with a higher frequency for AA. Anorexia, vomitus, diarrhea or constipation and a slight elevation of the temperature (37 0 C -38 0 C) were common manifestation of AA. Additionally, abdominal tenderness principally in the RLQ of the abdomen and the existence of the rebound sign, are strongly related with AA. Leucocytosis (>10.800 K/μl) with neutrophilia (neutrophil count > 75%) is considered to be a significant clue for AA. Urinalysis is useful for detecting urinary tract disease, normal findings on urinalysis are of limited diagnostic value for appendicitis.
The role of race, ethnicity, health insurance, education, access to healthcare, and economic status on the development and treatment of appendicitis are widely debated. Cogent arguments have been made on both sides for and against the significance of each socioeconomic or racial condition. A genetic predisposition appears operative in some cases, particularly in children in whom appendicitis develops before age 6 years. Although the disorder is uncommon in infants and elderly, these groups have a disproportionate number of compilations because of delays in diagnosis and the presence of comorbid conditions.
As diagnosis, there are four stages of appendicitis, including acute focal appendicitis, acute supurative appendicitis, gangrenous appendicitis and perforated appendicitis. These distinctions are vague, and only the clinically relevant distinction of perforated (gangrenous appendicitis includes into this entity as dead intestine functionally acts as a perforation) versus non-perforated appendicitis (acute focal and supurative appendicitis) should be made.
The present study is based on data set that is obtained from the Pediatric Surgery Clinical Information System of the University Hospital of Alexandroupolis, Greece. It consisted of 516 children's medical records. Some of these children had different stages of appendicitis and, therefore, underwent operative treatment. This data set was divided into a set of 422 records and another set of 94 records. The former was used for training of the ANN, while the latter for testing. A small number of data records were used as a validation set during training to avoid overfitting. Table 1 represents the stages of appendicitis as well as the corresponding cases for each one. The 3rd column of Table 1 depicts the coding of possible diagnosis, as they used for ANN training and testing stages.
Data were divided into two groups, the training cases (TRAC) and the testing cases (TESC). The TRAC consisted of 417 concrete medical data records and the TESC consisted of 101. Each input record was organised in a format of fifteen fields, namely sex, age, religion, area of residence, pain time period, vomit symptoms, diarrhoea, anorexia, located sensitivity, rebound, wbc, poly, general analysis of urine, body temperature, constipation. The output record contained a single field which corresponded to the potential outcome of each case.
The determination if the TRAC and TESC data sets was performed in a rather random manner. The training and testing sample size which would be sufficient for a good generalization was determined by using the Widrow's rule of thumb for the LMS algorithm which is a distribution free, worst case formula [2] and it is shown in the following equation 1. W is the total number of free parameters in the network (synaptic weights and biases) and ε denotes the fraction of the classification errors permitted during testing. The O notation shows the order of quantity enclosed within [2].
In the case examined here with 417 training examples used, the classification error that could be tolerated would be about 4%.
During experimentations, numerous ANN architectures, learning algorithms and transfer functions were combined in an effort to obtain the optimal network. For the Tangent Hyperbolic (TanH) transfer function the input data were normalized (divided properly) in order to be included in the acceptable range of [-3, 3] to avoid problems such as saturation, where an element's summation value (the sum of the inputs times the weights) exceeds the acceptable network range [17]. Standard back-propagation optimization algorithms using TanH, or Sigmoid or Digital Neural Network Architecture (DNNA) transfer functions, combined with the Extended Delta Bar Delta (ExtDBD) or with the Quick Prop learning rules [18,19] were employed. The ExtDBD is a heuristic technique reinforcing good general trends and damping oscillations [20].
Modular and radial basis function (RBF) ANN applying the ExtDBD learning rule and the TanH transfer function were also used in an effort to determine the optimal networks. RBFs have an internal representation of hidden neurons which are radially symmetric, and the hidden layer consists of pattern units fully connected to a linear output layer [21,22].
Traditional ANN evaluation measures like the Root Mean Square Error (RMS error), R 2 and the confusion matrix were used to validate the ensuing neural network models. It is well known that the RMS error adds up the squares of the errors for each neuron in the output layer, divides by the number of neurons in the output layer to obtain an average, and then takes the square root of that average. The confusion matrix is a graphical way of measuring the network's performance during the "training" and "testing" phases. It also facilitates the correlation of the network output to the actual observed values that belong to the testing set in a visual display [17], and therefore provides a visual indication of the network's performance. A network with the optimal configuration should have the "bins" (the cells in each matrix) on the diagonal from the lower left to the upper right of the output. An important aspect of the matrix is that the value of the vertical axis in the generated histogram is the Common Mean Correlation (CMC) coefficient of the desired (d), and the actual (predicted) output (y) across the Epoch.
Finally, the FUSETRESYS (Fuzzy Set Transformer Evaluation System) that constitutes an innovative ANN evaluation system has been applied offering a more flexible approach [1].
Fuzzy logic enables the performance of calculations with mathematically defined words called "Linguistics" [1,[23][24][25]. FUSETRESYS faces each training/testing example as a Fuzzy Set. It applies triangular or trapezoidal membership functions in order to determine the partial degree of convergence (PADECOV) of the ANN for each training/testing example separately. The following equations 2 and 3 represent a triangular and a trapezoidal membership functions respectively [1].
The model can produce various overall degrees of convergence (OVDECOV) for all of the training examples by applying either fuzzy T-Norm or fuzzy S-Norm conjunction operations, depending on the optimistic or pessimistic point of view of the developer. T-Norms tend to produce lower aggregation indices so in the case of ANN evaluation they can be considered as a pessimistic approach, whereas the opposite happens with S-Norms [26]. In fact, each distinct Norm evaluates the performance of an ANN under a different perspective. For example the drastic product assigns the ANN a high OVDECOV only if it does not have extreme deviations between the desired and the produced classifications during the training/testing process [1] whereas the Einstein T-Norm acts in a more average mode. The following equations 4 and 5 present the drastic product and the Einstein product T-Norms. More details on fuzzy conjunction operators can be found in [26][27][28].
The fact that the FUSETRESYS evaluates each training/testing example separately, offers a more clear view of the ANN's performance. In this way the developers know if the network operates extremely bad or well in specific cases.
Also when there are several neurons in the output layer, the traditional approaches produce separate evaluation results for each one whereas the FUSETRESYS can produce an additive performance index (ADPERI) of the ANN. This could be done under different perspectives and under different degrees of optimism [1].
Finally the application of fuzzy set hedges offers the "dilution" and the "intensification" options. In this way by using the dilution approach the developer softens the membership function over the fuzzy set and weakens the membership constraints so that a point of the Universe of discourse is "truer" than it would be before [1,27]. On the contrary the intensification hardens the MF over the FS and strengthens the membership constraints so that a point on the domain is "less true" than it used to be [1,27]. The following equations 6 and 7 correspond to the intensification and dilution functions respectively.
In this way the ANN can be evaluated strictly by using a "very well fit" evaluation option, or in a more relaxed way by using the "somewhat fit" option. Of course it is in the developer's hand to decide the potential type of the ANN's evaluation and the degree of dilution or intensification. For a more detailed description of FUSETRESYS please see [1].
Several experiments were performed. The following table 2 presents the structure of the four most effective Back Propagation (BP) multilayer (ML) neural networks. In all cases of ANN models, the classical approach for overcoming the overfitting problem has been followed. More specifically, a set of validation data have been provided to the algorithm in addition to the training data. The algorithm has monitored the error with respect to this validation set, while using the training set to drive the gradient descent search. The number of weight tuning iterations performed by the system, were determined in each case based on the criterion of lowest error over the validation set. Two copies of the best performing weights are kept: one copy for training and another one of the best performing weights thus far.
The following Table 4 presents the training and testing results for the most successful ML and RBF ANN using R 2 . Also for the three most successful networks, namely 2,3,4 the FUSETRESYS model was applied to determine the average degree of convergence. According to the results, the most suitable network was ML#3. The structure and the architecture of this successful network have been described in the above Table 2. The following figure 1 is a graphical representation of the all the PADECOV of the ML#2, for each testing example. The absolute degree of convergence has the value of one. A serious effort was made towards the development of modular ANN (MODANN) for the classification problem solution. The term MODANN refers to the "adaptive" mixtures of local experts (LOCEXP) as proposed by [29].
They consist of a group of BP ANN referred to as local experts competing to learn different aspects of a problem. A "gating ANN" controls the competition and learns to assign different parts of the data space to different networks. The LOCEXP have the same architecture but they can apply distinct learning rules or transfer functions. Also the number of the output processing elements of the gating network is equal to the number of LOCEXP used. The number of the neurons in the hidden layer of the gating network should be larger than the number of the output processing elements [17]. The above table 5 presents the structure and the architecture of the optimal MODANN that was developed for the medical classification problem examined here. The performance of the developed modular network is very satisfying, having an R 2 value of 0.9434 and a FUSETRESYS produced average PADECOV equal to 0.9733 (using the Triangular membership function) in the testing process using the first time seen testing data set.
The following figure 2 depicts the gating probabilities for the optimal MODANN.. The above Table 6 presents a small sample of the 101 distinct PADECOV values produced by the FUSTRESYS. Also the Einstein T-Norm was applied for the determination of the overall degree of convergence of the ANN. The ML#2 ANN had a very high OVEDECOV index with a value of 0.98299 whereas the other ML#3 ANN and the MODANN #REF1 had OVEDECOV indices as high as 0.97. The Drastic Product T-Norm was not applied in this research effort because it was proven unnecessary from the data in table 5 where there were no serious indications of extreme bad ANN performance in any of the testing examples.
The above research has obtained six ANNs with good level of convergence and it has proven that there exist at least four ANNs that have high performance indices, in the case of abdominal pain classification. Namely the best ANNs are two ML BP ANN, a RBF ANN and a MODANN using a referee gating network and two local experts. All of them have been described in the previous sections.
A very interesting part of the whole research effort is the application of an innovative ANN evaluation model called FUSETRESYS that uses fuzzy logic and fuzzy algebra proposed in [11].
The new evaluation scheme has performed individual convergence indices namely PADECOV, for the output of each single data record used in the testing phase. The worst PADECOV value equals to 0.6666 which actually is the degree of membership of each data record to the FS "Actual output value equal to the desired value". This worst case appears three times exactly in the same cases of data records, for the ML#2, ML#3, #1REF ANN and it shows that the classification capacity of the developed networks is not bad even in the worst cases. This conclusion becomes stronger by considering the fact that the second worst PADECOV index has a value of 0.833.
If an overall ANN validation is performed the traditional evaluation instruments agree with the FUSETRESYS that the most suitable ANN is the ML BP with code# 4 whereas all of the other developed ANN have almost an equally good performance. The Einstein T-Norm produces a higher "good performance index" for the MODANN than the traditional methods.
As it can be seen in table 7, the OVDECOV indices have very high values for ML#2 and for REF#1 and ML#3 networks when a "Partly fit" validation is performed. There is significant differentiation when a very strict evaluation is done under the linguistic "Very well fit". The OVDECOV indices fall from 0.99 to 0.75 for ML#2, from 0.99 to 0.65 for #REF and from 0.99 to 0.71 for ML#3 respectively. This is a very useful approach and it shows the actual power of FUSETRESYS due to the fact that it shows the differentiation of the average convergence degree of the three ANN when more strict validation methods are applied. So ANN fed with the same data records in testing and appearing to have more or less the same performance, they are very seriously differentiated when more strict convergence validation methods are performed.
The proposed ANN architecture faces the appendicitis prediction quite satisfactory, based on both the above presented results, and the pediatric surgeon's opinion that used these ANNs in their everyday routine clinical practice.
The innovative ANN evaluation model that was applied successfully in this research effort will be used extensively in the future, in an integrated effort to check its validity under various perspectives.
We would like to thank the pediatric surgeons of the Pediatric Surgeon Department of Medical School of Democritus University of Thrace, for their contribution in the concession of the medical records.