This paper presents a case-study of a knowledge-based recommender system capable to diagnose post-harvest diseases of apples. It describes the process of knowledge elicitation and construction of a Bayesian Network reasoning system as well as its evaluation with three diferent types of studies involving diseased apples. The ground truth of diseased instances has been established by genome sequencing in a lab. The paper demonstrates the performance diferences of knowledge-based reasoning mechanisms due to diferent users interacting with the system under diferent conditions and proposes methods for boosting the performance by likelihood evidence learned from the estimated consensus of users' and expert's interactions.
mender for
Apple trees are the most common temperate fruit tree species, since their fruits can be stored for prolonged periods of time under controlled atmosphere conditions. However, physiological disorders and pathogenic microorganisms can deteriorate the quality and quantity of the production during storage, and lead to considerable economic losses [1]. For instance, in Northern Europe, storage losses due to pathogenic microorganisms were estimated to reach up to 10% in integrated production and up to 30% in organic production [2]. Therefore, an efective knowledge-based recommender system, able to timely suggest a correct diagnosis of diseases manifested on stored apples, is of crucial importance. For instance, it depends on the exact pathogen species to decide on the right strategy for immediate damage containment and/or to recommend a plant protection scheme for the nEvelop-O (M. Zanker) ture of the disease, several macroscopic symptoms, such as appearance, color, texture and consistency of the rot need to be considered by the system. Hence, we should provide a practical interface to elicit user feedback on manifested symptoms on a diseased apple in order to guide the reasoning to recommend a diagnosis. Thus, we Systems (KaRS) & 5th Edition of Recommendation in Complex
The presented knowledge-based decision support system, named BN-DSSApple, is conceptualized as an interactive easy-to-use web application that allows users with different levels of domain expertise in the area of apple production (e.g., farmers, quality controllers, and storage workers), to perform in-field diagnosis of post-harvest diseases of apple fruit, relying solely on the observed macroscopic symptoms on the stored fruit. The system is designed as a recommender engine which collects the feedback of the user (i.e., the evidence) on a specific apple fruit (i.e., the target apple), in order to suggest a suitable diagnosis (i.e., a set of recommended diseases). The reasoning mechanism is performed by a Bayesian Network (BN) based on an ad-hoc knowledge base, constructed with the help of a domain expert (as described in 3.2).
Specifically, the system collects user’s feedback about the target apple by asking a set of dynamic multiplechoice questions related to the macroscopic features of the observed symptoms (e.g., the shape of the rot, the origin of the infection, etc.). Each question is illustrated with exemplary pictures, facilitating also non-expert users in their understanding. Each question is mapped to a specific variable in the BN model. This part of the system is dynamic, since the system incrementally adapts the questions path based on the previous answers given by the user. For instance, when the system gets the information that spores are visible on the infected apple, it will inquiry mass distribution, colour, and origin). Furthermore, the system provides full flexibility to the user, i.e., it allows to navigate the questions path back and forth in order to revise previous answers, to provide multiple answers, or to skip questions in case of lacking confidence.
In order to build a diagnostic reasoning system based on Bayesian network (i.e., both the network structure and the CPTs) two options are available: learn from the data or elicit the knowledge from the domain literature or the experts, or any combination of the above. At the best of our knowledge, no datasets are publicly available to learn significant relationships among apple diseases and macroscopic symptoms. Thus, we started by analysing a large OWL ontology which captures the entire life cycle of apple cultivation, production, handling, and storage, presented in [5]. Hence, we extracted a smaller quantitative part of the presented ontology suitable for our goal, which allows a simple reasoning mechanism connecting ∈ ( ∈ ( A Bayesian Network (BN) [3, 4] is defined by its two main components: the qualitative part represented by its graphical structure and the quantitative part consisting of the conditional probabilities. More formally, a BN is graphically represented as a directed acyclic graph (DAG) = ( , )
, where = { of nodes and ⊆ ×
1, 2, … , } denotes the set the set of directed edges between pairs of nodes. Each node ∈ in the DAG is mapped one-to-one with a random variable ∈ , where
denotes the set of random variables involved in the model. A random variable ∈ is represented by a set of exclusive values (or states) in which the variable might be observed (
) = { 1, 2, … , }, where the notation = variable ∈ is observed (or instantiated) in the state ) denotes the -th value of variable . We use
for an observed event, to express that ). A conditional probability table (CPT) is assothe conditional probability distribution ( ciated to each random variable ∈ . The CPT specifies over the states of . Where, conditional probabilities in the model, and (
represents the set of denotes the set of the so-called parents of the variable associated to the node in the DAG . Specifically, the parent set of is composed by every variable ∈ associated to the node in the DAG , connected with a directed edge to (the so-called child node). More for )) ∈ ) ⊂ |( mally, ( define an ) = { ∈ ∶ ( ancestor variable ( a descendant variable ( , ) ∈ } . We can further
) of the variable , and rected path (i.e., a set of directed edges) connecting node (associated with variable (
)) to (associated with variable ), and to (associated with variable ( namely {( , ), ( , ), ( , ℎ), … , ( , )} ⊂ . It is important to mention that the DAG of the BN specifies
)); a set of probabilistic relationships among variables in the model. Namely, if an edge ( , ) ∈ exists in the graph, this generally implies that a causal relation holds between the variables and , associated to nodes and . Specifically, we typically assume that the parent represents the cause and child represents the effect in the domain. Thus, a fundamental assumption of conditional (in)dependence between variables could be derived. This assumption is the Local Markov Assumption (or Local Independence Assumption), and it states that: given its parents ( ) ⊂ , defined in the DAG , a variable is conditionally independent of all its nondescendant variables. More formally, for each variable : ( ⟂ |( )), where ∉ (
), set of descendants of . This property allows to specify the joint distribution over the space of the variables in the BN model through the probability factorization ( ) =
∏=1 ( |(
)), ) of variable , if exists a di- the user about further features of those spores (i.e., their usually referred to as the chain rule for Bayesian networks. symptoms to diseases, thanks to a set of SWRL rules [6]. The graphical structure of this ontology is represented in of advancement of the post-harvest infection, namely Val(Stage) = {early, medium, late}. This workaround alto (semi)-automatically construct a BN from a domain lows the expert to visualize a specific condition of the ontology is still under-explored in the literature. Few disease and thus specify a more reliable likelihood of the practical, heuristic solutions can be found [7, 8], which symptoms. can hardly be applied to our case. The main limitation of The final BN-DSSApple graph is reported in Figure 2. such an efort lays in the fact that the two frameworks difThe central nodes in the network, bolded and empty, repfer in the purpose they are used for. An ontology is more suitable to describe concepts and qualitative relationships (of diferent nature), while the BN requires quantitative definitions (i.e., probabilistic) of correlation relationships related to the reasoning mechanism of phenomena [9]. resent the two hidden diagnosis variables, namely Disease and Stage. On the top part of the network, coloured in grey, are the nodes related to the lesion properties. On the right-most part, colored in yellow, are the rot properties, while on the left-most part, colored in green, are the lesion origin nodes. Finally, in the central-bottom part, colored in orange, are represented the nodes related to the lesion type and other symptoms, under those, colored in cyan, the nodes representing the properties of the other symptoms. asked the domain expert to review the available ontol- from a situation where all the symptom variables are concies among variables. For simplicity, we decided to start ogy, enrich and adapt it in order to obtain an efective tool for the diagnosis of post-harvest diseases of apple based on visible macroscopic symptoms on it. After few rounds of interaction, we agreed with a set of 27 discrete random variables (12 boolean and 15 categorical) related to macroscopical symptoms and signs that could be observed on the infected apple skin and pulp, together with two hidden (target) variables, namely Disease and Stage.
We assumed that a target apple could be infected by one and only one disease and thus, the random variable Disease encodes the whole set of bacterial diseases of our study, namely the 7 diseases Val(Disease) = {alternaria_rot, alternaria_spot, bitter_rot, botrytis, mucor_rot, neofabraea, penicillium}. The Stage random variable was introduced to facilitate the experts’ probability elicitation task. The variable represents three discrete and symbolic stages ditionally independent among each other, given the states of Disease and Stage. Furthermore, they all depends from the two hidden variables responsible for the assessment of the diagnosis (i.e., Disease and Stage). We indicate the Disease variable as ∈
, where hidden variables for the model. () = { defines the set of 1, 2, … }
represents the set of states of the variable , where is the -th state of the Disease variable (i.e., the -th disease in our pool). The Stage variable is referred as ∈ and ( ) = {
1, 2, … } represents the set of states of variable , where is the -th state of the Stage variable.
All other (observed) variables in the model are referred as symptom variables and they belong to the set . A generic symptom variable ∈ is represented by a set of states ( ) = {
1, 2, … }, where is the -th state of the symptom variable . Moreover, we adapted the procedures described in [10] for eliciting expert proba- value in order to avoid null probabilities, then values bilities of our network. Specifically, we adopted a mixed are normalized such that ∑∈ () ( ) = 1.0 . This prosymbolic questionnaire to facilitate the expert express- cess completely defines a probability distribution for the the expert was invited to answer the question: “How fre- In this section, we detail how a ranked list of recoming the conditional probability of each event. In more details, two techniques were applied depending on the support of the variable. For boolean variables (for each symptom variable ∈ such that ( ) = { , }
), quently do you observe symptom =
, given that you have an apple infected by disease = at stage = ?”.
We allowed her to select one option on a pre-defined 6point scale, including Always (A), Very often (V) , Often (O) , Sometimes (S), Rarely (R), and Never (N). The expert had to fill a form, providing the answer for each combination of ∈ × ∈ . The symbolic scale is converted into an actual probability ( = | =
, = ) according to the scheme reported in Table 1. The complementary probability is consequentially defined as ( = | = , = ) = 1 − ( = | =
, = ). answer
( = | 0.999 0.8 0.6 0.3 0.01 0.001 , ) 3-point symbolic annotation to denote the likelihood of each reported value, namely, common (no parenthesis), less common (one parenthesis), and rare (two parenthesis). The assumption underneath this choice is that many symptom values are never observed under some conditions (i.e., resulting CPTs are sparse) and could be ignored to speed up the elicitation process. In order to convert likelihood annotations into actual probability distribution values we adopted the following heuristic. Please consider a random variable with () = {, , , } which is annotated as follows by the the expert: a: common, b: less common, c: rare, and d is ignored; then , () = 2 () = 4 () = 1.0 and () = 0.0 . Furthermore, a small value = 0.001 is added to each probability categorical random variable .
mended diseases (i.e., a diagnosis) is computed after the user provides the feedback on a target apple, answering the questions asked by the system.
The reasoning mechanism of the BN allows to perform the inference, namely, to estimate the posterior probability distribution on a target unobserved variable (i.e., the Disease variable ), given any set S ∈ of observed variables as provided by the user (i.e., the evidence E).
The evidence set E is constructed incrementally by the application. At each step, the application requests the user to answer a multiple-choice question, related to a symptom variable ∈ . When the user submits the observed state ∈ , the diagnosis score related to target apple for to as the transferability problem [12, 13]. For instance, is associated with it (i.e., the diference of knowledge and as the mismatch between the BN probability distribu- asked by the application, while the random variables in disease ∈ is computed as: ( ) = ( =
|E) The ranked list of the suggested diseases { 1, 2, … , } shown to the user is then based on the score for each disease, such that (
) ≥ ( +1 ). The parameter controls for the flexibility of the system to show more or less recommended diseases to the user. In our evaluation, the parameter is fixed to = 3 .
(1) =
In knowledge-based modeling, but also with standard supervised learning, we often face the problem of transferring such a model on a diferent environment (i.e., providing external validity). This type of situation is referred it might be dificult to allow a vast set of users, with diferent expertise level, to efectively exploit a diagnostic expert model, based on domain-specific knowledge. In our application, the knowledge base of BN-DSSApple has been built with the information derived from domain literature and empirical knowledge of a domain expert. Nevertheless, diferent sets of users, with less experience in the field, might perceive the same attributes (i.e., the symptoms) in a diferent way. In fact, the user perception is mediated by her personal experience and specific knowledge biases. This mismatch invalidates the efectiveness and hence the diagnostic performance of BN-DSSApple. In this section, we formalize the problem of transferability and we propose a practical solution to bridge the gap between the expert model and the user perception.
In our scenario, the transferability problem is defined tions (CPTs) defined by the expert, and the probability distributions derived by the usage of the system. Formally, the expert during the knowledge elicitation phase (as described in Section 3.2) implicitly defined a complete set of probability = { ( S| = 1), ( S| = 2), … ( S| = )} ⊆ , for each set of symptom random variables S, given the target disease = . At testing time, the users of our application produced a set of where E = { 1 = 1, 2 = 2 , … = observations ℰ = {(E1, 1), (E2, 2), … (E , )} ⊆ × }, represent the evidence provided by a user during the -th diagnosis ses, sion, as a set of instantiations of symptom variables, and is the corresponding ground-truth disease. These set of user observations define a diferent set of probabilities which is generally diferent from the one defined by the = { ( S| = 1), ( S| = 2), … , ( S| = )} ⊆ , expert, ≠
. The problem becomes the one to expert model such that = ( ). ifnd a transferability function (.) to be applied to the The problem of transferability is long-lasting in machine learning and statistics and it has been addressed in causal terms, referred to as transportability [12, 14], as well as in statistical terms, in the context of supervised learning, where it is also known as covariate shift or sample selection bias [15, 16]. One of the most common approaches applies a direct correction to the learned probability distribution based on the estimates on the testing set [13]. Specifically inspired by the work presented in [17], we proposed a methodology, referred to as likelihood evidence and tailored to our BN-based application, to correct the expert-defined distribution one derived by users . We define the likelihood evidence (or likelihood finding) for each random symptom variable ∈ of our BN-DSSApple. Specifically, when a symptom variables is observed and thus instantiated by a user, we assume that a certain degree of uncertainty towards the expertise between the user and the expert). We define the actual user observation with another random variable , such that ( ) = ( ), to distinguish it from the variable as it should be observed by an expert . We represent the uncertainty degree with a likelihood ratio ( ), formally defined as:
( = ) = ( = | = ) (2) which represents the probability of a user observing value ) given that, in the same situation, the expert ∈ ( would have observed BN by adding, for each symptom variable , a virtual likelihood evidence node that encodes the likelihood ratio ( variable = { ), with (
) = { }. The added set of random 1, 2, … } is now the one observed by the user while providing the evidence E on the questions ). Thus, we enrich our ( , ) ∈ × become hidden. We finally need to define a new set of conditional probability tables ( | ) for each pair . We adopt a direct estimation of these probabilities from the observed interactions of users with a set of apples for which we know the actual observed value by the expert. Namely, for each state of each variable ∈ we define a subset of the expert is = . Thus, the conditional probability of the observed value = by the users is defined as: ( = | = ) =
1 | | ∑ ∈ 1 ( )
(3) where 1 (
) is an indicator function which is equal to 1 if the user observed = in apple , and 0 otherwise. The defined conditional probability for the likelihood ratio is also referred as consensus among expert and users.
three distinct phases to test the system behaviour under (SUS), Single User Study (SUS), and Multiple User Study
BN-DSSApple. is a ranked list of suggested diagnosis
We conducted a large user study to evaluate the efectiveness of BN-DSSApple in recommending the correct diagnosis. Specifically, we divided the user study into diferent circumstances. The task submitted to the users involved in our study was the same in all cases. The user received a “bucket” of infected apples, for which she had to find the correct diagnosis leveraging Each target apple was simulated as a set of two highdefinition photos depicting an internal and an external view of the target apple, and for which the ground-truth disease was collected in lab by genome sequencing. In each diagnostic round, the user had to carefully inspect the target apple and interact with the system by providing information (i.e., the evidence) about the symptoms and signs she was able to identify on the apple. At the end, BN-DSSApple returned a ranked list of three suggested diagnosis, i.e., the three diseases with the highest posterior given the available evidence, as computed by the BN. The three phases of the presented study difered in the number of users, their expertise level, and the number of distinct target apples involved. In details, we performed: • Single Expert Study (SES): a domain expert (the one which collaborate in the construction of the BN) interacted with the system to diagnose 21 target apples in a time-span of around 2 weeks. • Single User Study (SUS): a single user (a MSc student in Biology), interacted with the system during the course of an internship, lasting around 3 months, to diagnose 131 target apples. • Multiple Users Study (MUS): a group of 11 students of a Phytopatology class interacted with the system to diagnose a bucket of 7 target apples each. The apples were randomly sampled from the same set of 21 apples used for SES. The activity lasted for a total of 4 hours.
In Table 2 we summarize the diferent characteristics of the three user studies performed.
SES SUS
MUS for apple . Thus, we formally 1
∑ ∈ 1 ( )
(4) Where the function 1 ( which is equal 1 if ∈ and 0 otherwise. ) is an indicator function
SES .905 1. 1.
SUS .489 .656 .763
MUS .286 .403 .571
ZeroR .143 .286 .429
From the results presented in Table 3 we highlight how the theoretical efectiveness of the
BN-DSSApple model is very high. Specifically, an expert user (SES), with strong knowledge in the domain of post-harvest diseases of apples and a good capability of correctly identify symptoms on a diseased apple, is able to reach a recall@1 above the 90%. The performance of the system increases up to 100% of recall when evaluated at a larger cut-of of suggested diseases. Of course, we have to consider that in the SES evaluation, we are in the ideal situation where the expert user knows exactly how to look and evaluate the symptoms requested by BN-DSSApple. A more realistic situation is depicted by the SUS evaluation. In this situation, a single user with a medium-high level of expertise had months of time to interact with the system by evaluating a very large set of apples (131). The performance of the system for the recall@1 are still conall diagnoses. The other metrics testify how the system is not able to scale-up well for further cut-of of recall, rect disease is within the first 3 recommendations in 3/4 of the cases). Finally, BN-DSSApple showed some limits In Table 3 we report the results of the three user stud- vincing (49%), i.e. correct disease identification by half of = { please consider a situation in which a set of diagnosis is performed by BN-DSSApple. The set is composed by ranked lists of recommended diagnosis, namely 1, 2, … }, where represents the -th apple
processed by the system. A generic = { 1 , 2 , … , } in the situation where the users have a limited expertise
and training, and a limited amount of time (few hours) rank attribute consensus to use the system as in the MUS evaluation. In addition 1 Sclerotia 0.988 to the time and skill aspect, also less intrinsic motiva- 2 Calyx 0.985 tion to interact as accurate as possible with the system 3 Rot 0.964 could be a partial explanation for the deviation. In this 4 Spot 0.950 case, the measured recall of the system is significantly 5 Stalk 0.926 lower than the one of the two previous evaluations. Par- 6 Core 0.917 ticularly, the recall@1 doesn’t reach the 30%, while the 7 Spore_distribution 0.872 best result is achieved by the recall@3 with a value of 8 Lesion_size 0.837 57% (slightly more than half of the diagnosis include the 190 NLuesmiobne_r_sulersfiaocnes 00..883177 correct disease in the top-3 recommendations). Neverthe- 11 Mycelium_spore 0.809 less, despite the poor performances of BN-DSSApple in 12 Lesion_form 0.792 MUS, the collected results are still superior to the ZeroR 13 Lesion_crack 0.790 benchmark, namely, a classifier which always suggest 14 Halo 0.782 the class with a priori higher probability. Important to 15 Rot_shape 0.760 notice that the reported results for ZeroR are related to 16 Rot_texture_dry 0.755 the situation in which the class (ground-truth disease) 17 Halo_colour 0.750 distribution is perfectly balanced, like for SES and MUS. 18 Rot_margin 0.740 In the comparison with ZeroR, MUS evaluation for BN- 19 Spore_colour 0.731 DSSApple shows the double of recall@1 (28.6% against 20 Spore_origin 0.694 14.3%), while recall@2 and recall@3 are closer but still 2212 LeLseisoino_nm_aarregain 00..663263 significantly better (+12% and +14%, respectively). The 23 Rot_texture_opaque 0.607 main cause of this mismatch of performances among ex- 24 Wound 0.594 pert and averaged users can be identified in the problem 25 Lenticel 0.588 of transferability of a knowledge-aware model. In the 26 Lesion_appearance 0.417 remaining of this section, we are going to empirically an- 27 Rot_texture_pressure 0.321 alyze and explain such a phenomenon, and test possible solutions to correct and alleviate it. Table 4
Foremost, we want to understand the impact of each Asetntrsiubsu)toefs trhaenkuisnegrsboafseMdUoSnwthitehrtahtee doofmagarieneemxepnetrt(io.ef.S,cEoSn.expert-defined attribute in the model. In Table 4 we report the ranked list of attributes, based on the likelihood ratio (i.e., consensus) computed between users of MUS and the expert of SES (which we consider as a user, with a consensus above the 90% with the expert. ground-truth) in identifying the symptoms on the same Nevertheless, two of them, namely Wound and Lenticel, set of 21 target apples. It is interesting to notice how are equally dificult to be recognized with a consensus of the users are efective in identifying the principal symp- around 59%. This is probably due to the fact that the two toms and signs, presented by the application as boolean origins might be perceived as quite similar and could be variables. Namely, Sclerotia (99%), Rot (96%), and Spot confused, without a careful inspection of the apple skin. (95%) present a very high level of agreement with the In Figure 3 we plot the recall@k achieved by BNdomain expert, while Mycelium_spore (81%) and Halo DSSApple for MUS and SES, by incrementally selecting (78%) receive an high consensus. Vice versa, some quali- the attributes based on the consensus ranking reported tative attributes related to the appearance or the consis- in Table 4. On the x-axis, we report the number of attency of the lesion and the rot are among the hardest to tributes in each model configuration. Namely, the -th be correctly recognized by the users (i.e., they show a value represents the BN model built with the attribute poor consensus with the expert). For example, Lesion_ap- set = { 1, 2, … −1 , }, where the rank of attribute pearance and Rot_texture_pressure achieve a consensus is defined by expert consensus, as reported in Table below the 50%, while Lesion_margin, Lesion_area, and 4. From the graph in Figure 3a for MUS evaluation, we Rot_texture_opaque are below 65%. Nevertheless, other immediately notice how the model achieves the best percategorical variables more related to quantitative aspects formances for recall@1 and recall@2 with around 8-9 of the lesion are easier for the users to be spotted. This attributes. A larger set of attributes is detrimental, causis the case of the variables Lesion_size, Lesion_surface, ing a drop of recall of at least 10% in both situations. Lesion_form, and Lesion_crack which show a consensus Interesting to notice how these performances seem to between 84% and 79%. Finally, it is interesting to notice recover with the models based on 21-22 attributes, withthe behavior of the variables of the Lesion origin cate- out reaching the optimal level. In fact, for the recall@3 gory. Most of them are quite easy to be identified by the metric the global optimum is achieved by the model with (a) (b) 20 attributes, with a significant improvement of around data with the Maximum Likelihood Estimation (MLE) al10% on the smaller attribute set configurations. Opposite gorithm. The recall@1 improvement is marginal (around considerations emerge from the graph in Figure 3b for +2.5%), while recall@2 shows a +6.5% with respect to the SES evaluation. In this case, the recall@k metrics are lin- plain BN model. We already commented the large imearly correlated to the number of attributes, and the best provements achieved by selecting the optimal attribute performances are always achieved with the full set of set (BEST-ATTR model), whereas the gain in recall is beattributes. This means that the expert is able to correctly tween +14% and +21%. Of course, this analysis is derived instantiate even the harder variables, by understanding a posteriori, where the optimal number of attributes is the status of an infected apple. Furthermore, this “hard- fixed after the evaluation. For this reason, the achieveto-recognize” attributes are necessary to significantly ment of the model equipped with likelihood evidence improve the diagnostic efectiveness of the model and (LH-EV, methodology detailed in Section 3.4, where exreach the highest performances in term of recall@k. For pert ground-truth data are derived from SES) is even instance, in both recall@2 and recall@3 the BN model greater. For recall@1 the LH-EV outperforms TRAIN-BN registers around +20% improvement by considering the of around +4%, while being inferior to BEST-ATTR by full set of 27 attributes instead of just considering 21 around -8%. For recall@2, instead, the likelihood eviattributes (i.e., by discarding the 6 “hardest” attributes, dence achieves the best result outperforming also BESTwith lowest consensus). ATTR by a +2.5%. Finally, for recall@3 the LH-EV model significantly outscores TRAIN-BN (+13%), while being
BN TRAIN-BN BEST-ATTR LH-EV comparable with the results of BEST-ATTR. .286 .403 .571 .312 .468 .636
Table 5 This case study focused on knowledge elicitation and Recall@k for MUS when applying the plain BN-DSSApple construction as well as discussed the application of likeli(BN), the trained BN-DSSApple on MUS data (TRAIN-BN), hood evidence to enhance performance and transferabilthe incremental best attribute selection (BEST-ATTR), and ity of the knowledge-based recommendation system BNthe BN-DSSApple with likelihood evidence (LH-EV). In BEST- DSSApple. Major limitations of the presented approach ATTR column, we report the results for the optimal attribute concern the fact that the knowledge base is fully based set, with the number of selected attributes in parenthesis. on qualitatively probability elicitation from a single human expert. Furthermore, transferability problem of the
Finally, in Table 5 we compare the recall@k results for crafted BN must be additionally investigated. Further the MUS evaluation of the improved versions of the BN development of the method to other domains as well as model, in order to cope with the transferability problem additional testing is required. Currently, deployment for discussed in Section 3.4. Firstly, the smallest improve- real-life evaluation is ongoing. In future work, the intement is provided by the trained BN model (dubbed as gration of additional evidence like microscopic images TRAIN-BN), where the parameters are fine-tuned on MUS of fungal spores will be considered. in: Proceedings of the 2011 IEEE 11th International Conference on Data Mining Workshops, [1] T. B. Sutton, H. S. Aldwinckle, A. Agnello, J. F. Wal- ICDMW ’11, IEEE Computer Society, USA, 2011, genbach (Eds.), Compendium of apple and pear dis- p. 540–547. URL: https://doi.org/10.1109/ICDMW. eases and pests, 2 ed., APS press, 2014. 2011.169. doi:1 0 . 1 1 0 9 / I C D M W . 2 0 1 1 . 1 6 9 . [2] P. Maxin, M. Williams, R. W. Weber, Control of fun- [13] J. Lu, V. Behbood, P. Hao, H. Zuo, S. Xue, G. Zhang, gal storage rots of apples by hot-water treatments: Transfer learning using computational intelligence: A northern european perspective, Erwerbs-Obstbau A survey, Knowl. Based Syst. 80 (2015) 14–23. 56 (2014) 25–34. [14] A. Subbaswamy, S. Saria, Counterfactual normal[3] D. Koller, N. Friedman, Probabilistic Graphical ization: Proactively addressing dataset shift using Models: Principles and Techniques, Adaptive causal mechanisms, in: R. Silva, A. Globerson, computation and machine learning, MIT Press, A. Globerson (Eds.), 34th Conference on Uncer2009. URL: https://books.google.co.in/books?id= tainty in Artificial Intelligence 2018, UAI 2018, vol7dzpHCHzNQ4C. ume 2, Association For Uncertainty in Artificial [4] U. B. Kjaerulf, A. L. Madsen, Bayesian Networks Intelligence (AUAI), 2018, pp. 947–957. 34th Conferand Influence Diagrams: A Guide to Construction ence on Uncertainty in Artificial Intelligence 2018, and Analysis, 1st ed., Springer Publishing Company, UAI 2018 ; Conference date: 06-08-2018 Through Incorporated, 2010. 10-08-2018. [5] A. Niederkofler, S. Baric, G. Guizzardi, G. Sotto- [15] J. Huang, A. J. Smola, A. Gretton, K. M. Borgwardt, cornola, M. Zanker, Knowledge models for diag- B. Scholkopf, Correcting sample selection bias by nosing postharvest diseases of apples, in: Proceed- unlabeled data, in: Proceedings of the 19th Internaings of the Joint Ontology Workshops 2019 Episode tional Conference on Neural Information ProcessV: The Styrian Autumn of Ontology, Graz, Aus- ing Systems, NIPS’06, MIT Press, Cambridge, MA, tria, September 23-25, 2019, volume 2518 of CEUR USA, 2006, p. 601–608.
Workshop Proceedings, CEUR-WS.org, 2019. URL: [16] M. Sugiyama, S. Nakajima, H. Kashima, P. v. Bühttp://ceur-ws.org/Vol-2518/paper-ODLS6.pdf. nau, M. Kawanabe, Direct importance estimation [6] M. Zanker, M. Jessenitschnig, W. Schmid, Prefer- with model selection and its application to covarience reasoning with soft constraints in constraint- ate shift adaptation, in: Proceedings of the 20th based recommender systems, Constraints 15 (2010) International Conference on Neural Information 574–595. Processing Systems, NIPS’07, Curran Associates [7] M. B. Messaoud, P. Leray, N. B. Amor, Sem- Inc., Red Hook, NY, USA, 2007, p. 1433–1440. cado: A serendipitous strategy for causal discov- [17] A. B. Mrad, V. Delcroix, S. Piechowiak, P. Leicester, ery and ontology evolution., Knowl.-Based Syst. M. Abid, An explication of uncertain evidence in 76 (2015) 79–95. URL: http://dblp.uni-trier.de/db/ bayesian networks: likelihood evidence and probajournals/kbs/kbs76.html#MessaoudLA15. bilistic evidence - uncertain evidence in bayesian [8] A. M. Kalet, J. N. Doctor, J. H. Gennari, M. H. networks, Appl. Intell. 43 (2015) 802–824. URL: Phillips, Developing bayesian networks from a https://doi.org/10.1007/s10489-015-0678-6. doi:1 0 . dependency‐layered ontology: A proof‐of‐concept 1 0 0 7 / s 1 0 4 8 9 - 0 1 5 - 0 6 7 8 - 6 . in radiation oncology, Medical Physics 44 (2017) 4350–4359. doi:1 0 . 1 0 0 2 / m p . 1 2 3 4 0 . [9] S. Fenz, An ontology-based approach for constructing bayesian networks, Data Knowl. Eng. 73 (2012) 73–88. URL: http://dx.doi.org/10.1016/j.datak.2011.
12.001. doi:1 0 . 1 0 1 6 / j . d a t a k . 2 0 1 1 . 1 2 . 0 0 1 . [10] L. C. van der Gaag, S. Renooij, C. L. M. Witteman,
B. M. P. Aleman, B. G. Taal, How to elicit many probabilities, in: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, UAI’99, Morgan Kaufmann Publishers Inc., San
Francisco, CA, USA, 1999, p. 647–654. [11] A. T. Ihler, J. W. Fischer III, A. S. Willsky, Loopy belief propagation: Convergence and efects of message errors, J. Mach. Learn. Res. 6 (2005) 905–936. [12] J. Pearl, E. Bareinboim, Transportability of causal and statistical relations: A formal approach,