=Paper=
{{Paper
|id=Vol-2567/paper8
|storemode=property
|title=Hypothyroid Disease Diagnosis with Causal Explanation using Case-based Reasoning and Domain-specific Ontology
|pdfUrl=https://ceur-ws.org/Vol-2567/paper8.pdf
|volume=Vol-2567
|authors=Mir Riyanul Islam,Shaibal Barua,Shahina Begum,Mobyen Uddin Ahmed
|dblpUrl=https://dblp.org/rec/conf/iccbr/IslamBBA19
}}
==Hypothyroid Disease Diagnosis with Causal Explanation using Case-based Reasoning and Domain-specific Ontology==
https://ceur-ws.org/Vol-2567/paper8.pdf
Hypothyroid Disease Diagnosis with Causal
Explanation using Case-based Reasoning and
Domain-specific Ontology
Mir Riyanul Islam, Shaibal Barua, Shahina Begum, and Mobyen Uddin Ahmed
School of Innovation, Design and Engineering
Mälardalen University, Västerås, Sweden
{mir.riyanul.islam, shaibal.barua, shahina.begum, mobyen.ahmed}@mdh.se
Abstract. Explainability of intelligent systems in health-care domain
is still in its initial state. Recently, more efforts are made to leverage
machine learning in solving causal inference problems of disease diag-
nosis, prediction and treatments. This research work presents an on-
tology based causal inference model for hypothyroid disease diagnosis
using case-based reasoning. The effectiveness of the proposed method is
demonstrated with an example from hypothyroid disease domain. Here,
the domain knowledge is mapped into an expert defined ontology and
causal inference is performed based on this domain-specific ontology. The
goal is to incorporate this causal inference model in traditional case-based
reasoning cycle enabling explanation for each solved problem. Finally, a
mechanism is defined to deduce explanation for a solution to a problem
case from the combined causal statements of similar cases. The initial
result shows that case-based reasoning can retrieve relevant cases with
95% accuracy.
Keywords: Case-based Reasoning · Causal Model · Explainability · Ex-
plainable Artificial Intelligence · Hypothyroid · Diagnosis · Ontology.
1 Introduction
With the outburst of AI applications, expectations have been increased for in-
telligent systems in all domains including the health-care domain. The growing
capabilities of AI, especially the applications of Machine Learning (ML) leverage
new requirements to be fulfilled such as human-level intelligence. According to
Pearl, at present three prime obstacles in AI and/or ML applications are there
to achieve human-level intelligence, which are; i) adaptability or robustness, ii)
explainability and iii) understanding of cause-effect connections. Explainability
is one of the major characteristics of human-level intelligence. In recent years, a
good number of works have been done to equip systems with causal models in
disease diagnosis. Incorporating causal model would facilitate explainability of
the prevailing systems [18]. State-of-the-art techniques to develop AI applications
are gaining more accuracy gradually but lack of proper explanation with the so-
lution inhibits the reliability of those techniques. Another significant issue is to
Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0
International (CC BY 4.0).
include causality in the explanation to a solution. If these issues can be solved,
most of the challenges to produce AI/ML applications with human-level intelli-
gence will be conquered. Causality in intelligent health-care applications is still
in an immature state and there are rooms for further research and improvement.
According to Sene, medical information is doubling every 5 years but only 20%
of the evidence based knowledge is used by medical practitioners [22]. Several
research works have been carried out to make evident based knowledge useful to
clinicians as well as intelligent health-care systems. Case-based reasoning (CBR)
has been being used for health systems due to its close characteristics to human
behaviour, intimidating previous experience as the medical practitioners do. In
CBR, only the solution is produced for a new problem case with responsible
features only. In this paper, we have adapted the traditional structure of the
CBR cycle and introduce causal inference using domain ontology. This research
work is aimed at making CBR systems more explainable for health-care services
with the support of domain knowledge from an ontology. The objective of this
work is further elaborated by considering the following example scenario.
Problem scenario: A patient is reported to be diagnosed for having thyroid
diseases provided he/she has taken several tests and recorded his/her history of
medication, treatment and symptoms. For this scenario, a physician with access
to machines that are capable of classifying based on historical data, can diag-
nose the patient primarily. From state-of-the-art techniques, for example rough
sets learning [19], the physician could get the final verdict of the diagnosis. The
verdict may provide that due to having Thyroid Stimulating Hormone (TSH)
test value less than or equal to 6 mU/L, the patient may have negative hypothy-
roid. But for T SH > 6 mU/L, the patient might be diagnosed for negative,
primary hypothyroid and compensated hypothyroid with 40%, 30% and 30%
possibilities respectively. In this uncertain scenario, some causal statements in
support of each verdict along with the contribution of other features from the di-
agnosis would facilitate the decision making for a physician. To overcome these
uncertainties and obstacle for explainable AI applications, this research work
is destined to produce human understandable explanation in natural language
with the label or verdict generated by a classification mechanism from predefined
causal statements.
The remaining parts of this paper is divided into several sections. Section
2 describes the background of the concerned topics with several related works.
Detailed description of the proposed causal model with formal definitions and
brief descriptions of developed components are discussed in Section 3. Section
4 presents a brief evaluation on case retrieval model and the description of ex-
tracting explanation for extracted solution. Finally, Section 5 states some of the
future possibilities of this research work and conclusive statements respectively.
2 Background and State-of-the-Art
This section contains a short description of the methods used to achieve the goal
of this work followed by a brief discussion on recent research works.
CBR is an AI approach that uses previous experiences to solve a current
problem which perfectly aligns with the characteristics of physicians in case
of patient for diseases. According to Kolodner [13], CBR is a reasoner that
solves a new problem by remembering and using past situations similar to the
current. CBR is an instance-based lazy learning method, i.e., it does not try to
reason until it has to [15]. The term “case” represents an experience achieved
from a previously solved problem. The term “based” means in CBR cases are
the source of reasoning. Finally, the term “reasoning” means the approach of
problem-solving, i.e., solving a problem by concluding using previously solved
cases [21]. Based on implementation techniques, CBR can be distinguished into
four main types: i) CBR using nearest neighbour, ii) CBR using induction, iii)
CBR using fuzzy logic and iv) CBR using database technologies [22]. In addition
to the basic implementation methodologies, Barua et al. have shown a distributed
architecture of CBR using XML files containing individual cases [1].
Ontology structures the concepts with definition and relations. The term on-
tology has been evolved from philosophy which means a systematic account of
existence [9]. A number of definitions exist about ontology in terms of computer
science. Gruber has defined, “Ontology is an explicit specification of a conceptu-
alization” [9]. The previous definition rises another fundamental question, what
is conceptualisation? According to Gruber, “A conceptualisation is an abstract,
simplified view of the world that we wish to represent for some purpose.” [9].
There are other definitions of ontology. Ontology is defined as a formal, explicit
specification of a shared conceptualisation [7]. A number of domain independent
ontologies are developed for knowledge representation. For instance, BioPortal1
is a collection of ontologies in biomedical domain.
Various forms of causal models are being investigated for long to be used in
disease diagnosis. Mostly causal models are comprised of mathematical models
that represent the causal relationships in a system [10]. Pearl introduced causal
models based on the Bayesian Network (BN) [17], a commonly used methodol-
ogy for prediction and classification tasks in different domains. Basically, a BN
consists of a directed acyclic graph (DAG) and a set of conditional probability
tables such that each node of the graph represents a variable and it is associated
with a conditional probability table that contains probability of each form of the
variable with every possible state of its parent states. Causal Bayesian Network
(CBN) [16] is upgraded from BN with autonomous causal relations with the do
operator by Pearl [25]. Several works have been done to learn and adopt the
CBN for developing causal models. A theoretical study has been carried out by
Eberhardth et al. on lower bound of worst case for the number of experiments
to be done to recover causal structures [6]. Tong and Daphne have developed a
score-based technique to learn CBN from experimental data [24].
Recent works have been done to incorporate ontologies in systems to achieve
explainability. Besnard et al. have proposed ontology-based inference rules for
causal explanation [3]. In order to facilitate causal models, researchers have
also worked on fusing ontology to BN [5][2]. Ishak et al. have proposed Object-
1
http://bioportal.bioontology.org/
Oriented Bayesian Networks based on Ontologies [12]. CBR and ontologies have
often be used collaboratively by researchers to facilitate the works of physicians
[23][22]. These methodologies achieve the primary goal but lack in overcom-
ing the challenges of modern AI applications. In recent years, there have been
some researches on disease diagnosis using causal models. Raghu et al. have pro-
posed a probabilistic causal model for lung cancer prediction [20]. In another
research work, Huang et al. have developed a causal discovery of autism based
on constrained functional causal models [11]. Wang and Tansel have proposed
an ontology based decision support system for medical diagnosis where case re-
trieval has been done on the basis of semantic similarity [26]. Lamy et al. have
proposed an approach to detect breast cancer with explanation using CBR and
visual reasoning [14].
3 Proposed Casual Inference Model
The proposed model is an accumulation of several components from different
concepts i.e., CBR, ontology and causal inference model. The formal definitions
followed by the description of each of the components and their overall collabo-
ration are stated in the following subsections.
3.1 Formal Definition
An ontology is a representation of a domain with respect to its entities, relation-
ship among entities and their attributes which is known as DERA knowledge
representation framework [8]. Formally, D = < E, R, A >, where, D represents
the domain of interest, E is the set of all the entities i.e. concepts and indi-
viduals, R is the set of relations among the entities or object properties and A
is the set of data properties or attributes. In ontology, classes are arranged in
hierarchy which is reflected in statements as is a i.e., Sub-class is a Class. ABox
is a fact used in description logic (DL) to define an individual with respect to
its respective class in the form of Class(Individual). It is used to represent the
causal statements associated with each of the cases in case base. For example, to
express this statement; “I131 is a specific Therapy that is a type of Treatment”,
we use the propositional statements: Therapy is a Treatment and Therapy(I131).
To facilitate causation in the system, we propose to define causal statement
C comprised of two atomic statements, α and β as ABox connected with the
keyword “causes” in the form of “C : α causes β”. Causal statements will be
associated with cases as individual or in a set. For example, a possible causal
statement is “C : TT4(Abnormal) causes Hypothyroid(Primary)” which sum-
marises “Primary hypothyroid is caused by abnormal free thyroxine level (TT4)”
Explanation for the solution case will be generated by translating causal
statements associated with the similar cases – γ because of Φ, where γ is the
solution and Φ is the set of probable explanations to the solution. “because of ”
is used to make statements shorter which can be translated in natural language
as “the probable solution is given because of the facts”.
Primary Hypothyroidism Secondary Hypothyroidism
Negative Hypothyroidism Hypothyroidism Compensated Hypothyroidism
Therapy
Symptoms Thyroid Test Value Level FTI
T3
Surgery
Treatment owl: Thing Test Values Hormone T4U
Medication
TSH
Patient Person Diagnosis Blood Test TTU
Thyroxine Utilization Rates Hormone Test Free Thyroxine
Thyroxine Stimulating Hormone Free Thyroxine Index Free T3 Index
Class Hierarchy Object Property
Fig. 1. Class hierarchy and object properties of hypothyroid ontology. Class hierarchy
is built on is a relation and object properties among different classes differ on the basis
of their characteristics.
3.2 Building Hypothyroid Ontology
An ontology was built to facilitate explanations of solutions produced using
CBR. Thyroid disease dataset [19] was used to build the concepts of the ontol-
ogy. This dataset was created by the Garvan Institute, Sydney, Australia which
is now available at UCI Machine Learning Repository2 . The thyroid disease
dataset contains 3772 instances with 26 attributes each. The attributes repre-
sent information of diagnosed patients for hypothyroid i.e. age, sex, ongoing
medications, disease history, values for different diagnosis tests for determining
levels of hormones – TSH, triiodothyronine (T3), thyroxine (TT4), thyroxine
utilization rates (T4U) and free thyroxine index (FTI). Instances of the dataset
are labelled with four classes: primary hypothyroid, compensated hypothyroid,
secondary hypothyroid and negative. Concepts and relations in the ontology
were developed and validated in respect of two expert curated medical ontolo-
gies built by Shen et al. [23] and Can et al. [4]. Ontology building tool Protégé3
was used to define the entities, relations and attributes in the ontology. Figure
1 illustrates the classes of hypothyroid ontology which is a subset of E from the
equation of DERA given in section 3.1. In the figure, the solid arrows represent
the is a relation and dotted arrows shows the object properties of the concepts.
At first the concept hierarchy was built. Afterwards, object properties were de-
fined to create relation among the concepts. The object properties from the set
R of the equation defined in section 3.1 for the developed hypothyroid ontology
are given below:
– defines: Represents the relation between Diagnosis class and Thyroid class.
2
http://archive.ics.uci.edu/ml/datasets/thyroid+disease
3
https://protege.stanford.edu/
– hasDisease: Represents the relationship between Thyroid class and Patient
class.
– hasReceived: Represents the relationship between Patient class and Treat-
ment class.
– hasSymptoms: Represents the relationship between Patient class and Symp-
toms class.
– hasTestDone: Represents the relationship between Patient class and Hor-
moneTest class.
– hasValueType: Represents the relationship between HormoneTest class and
Hormone class.
– hasValueLevel: Represents the relationship between HormoneTest class
and TestValueLevel class.
Data properties of the developed ontology were defined afterwards which
represent values for various entities in the ontology. Set A holds these data
properties in equation defined for DERA in section 3.1. The data properties of
the hypothyroid ontology are described briefly below.
– hasAge: Represents the age of a patient.
– hasGender: Represents the gender of a patient.
– hasTestDefinition: Represents the information of a diagnostic test.
– hasTestName: Represents the name of a diagnostic test.
– hasTestValue: Represents the value of a diagnostic test.
– hasValue: Represents the reference values of a diagnostic test.
– lowerLimit: Represents the regular lower limit of a diagnostic test.
– upperLimit: Represents the regular upper limit of a diagnostic test.
Finally, the individuals were added to the ontology based on the experimental
data from Thyroid Disease dataset. Most prominent way to store an ontology is
to use web ontology language (OWL) or resource description format (RDF). In
this work, the built ontology is stored using RDF since it facilitates to hold the
inferred axioms of the ontology whereas OWL holds the defined axioms only.
3.3 Case Representation
Cases in the case base are stored in the form of XML files. This method is
adopted from the work of Barua et al. [1] to facilitate inclusion of the causal
statements to the case representation. In each of the files, there will be a case id,
list of features, tested solution, causal statements and index to similar cases to
reduce the computation for retrieving similar cases in distributed architecture.
Graphical representation of a sample case from the case base of our proposed
model is shown in Figure 2. During the preparation of cases, causal statements
were added according to the relations among the attributes defined in the domain
ontology and their contributions to the final solution. In our concerned scenario,
attributes are the levels of various hormone tests and other clinical issues of the
patient.
id
true
21.41
:
:
Instance_1
Instance_2
:
case_id_1
case_id_2
:
Fig. 2. Sample case representation.
3.4 CBR with Causal Model
The proposed causal model is incorporated within the naive CBR architecture.
Figure 3 illustrates the working mechanism of the model. This model is described
based on the basic four steps of a CBR system: i) retrieve, ii) reuse, iii) revise and
iv) retain [21]. The first step of the mechanism is to represent the new problem
into defined format of the old cases. After the new case is built, following steps
are followed to generate solution with probable explanations.
Retrieve: Similar cases were retrieved from the case base using similarity func-
tion developed based on k-nearest neighbour (k-NN) algorithm associated with
voting from the most similar cases. Detailed evaluation of the accuracy of re-
trieval model is discussed in section 4.
Reuse: In this step, the new solution was formed from the similar cases. To
represent the causal statement associated with the solution, the inference en-
gine combined all the relevant statements to the new case and prepared for the
translation into explanations. For example, if three similar cases were found with
two different solutions α and β. According to the definition in section 3.1, the
representation of the causal statements – Cα,1 : γ causes α, Cα,2 : δ causes α
and Cβ,1 : η causes β.
Causal inference engine combines these causal statements into a single state-
ment with union operator – C : α because of {γ, δ} t β because of {η}.
Revise: The solved case from the previous step containing a solution with
explanation translated from the associated causal statements by the explanation
Problem Similarity
Function
fs New
New Case
Retrieve
Case Retrieved
Cases
Reuse
Learned
Previous
Case
Cases
Case Base Causal
Inference
Engine
Retain
Ontology
Tested Solved
Revise
Case Case
Confirmed Explanation Suggested
Solution Translator Solution
Fig. 3. Overview of proposed CBR architecture with Causal Model.
translator. Brief discussion on the explanation translator is discussed in the
section 4. Finally, the suggested solution was manually curated by an expert on
the basis of domain expertise to produce tested case.
Retain: By further generating explanation from modified suggested solution,
confirmed solution was developed. This solution was inserted in the case base
for future reference. In some cases, there were requirements to modify facts in
the ontology, this was also done in this step.
4 Evaluation
To retrieve similar cases from the case base, k-NN was used. The value of k was
determined after tuning the model parameters using grid search over different
values of k. Moreover, 5-fold cross validation for all the values were done and
k = 3 produced the highest mean cross-validation accuracy. Table 1 shows the
accuracy for 5-fold cross validation with different values of k.
After retrieving similar cases from case base, explanation for the solution case
was deduced by inferring associated causal statements of the solved cases. For
better understanding of the mechanism, consider the following example demon-
strating a translation mechanism to generate explanation from causal state-
ments.
Table 1. Accuracy (%) for 5-fold cross validation of k-NN with different values of k.
Iterations
k-NN Mean
1 2 3 4 5
k=1 94.70 94.04 94.56 94.83 94.95 94.62
k=3 94.57 94.17 95.36 96.02 95.61 95.15
k=5 94.44 94.17 94.43 95.76 95.74 94.91
k=7 94.04 94.17 94.56 95.62 95.74 94.83
Example 1. Let us consider the causal statements associated with a solved case:
C1 : T SH(Low) because of Symptom(Pregnancy)
C2 : Hypothyroidism(Secondary) because of
{T SH(Low), T T 4(Low)} (1)
For generalisation, these statements can be represented with symbols –
C1 : A(α) because of B(β)
C2 : Γ (γ) because of {A(α), ∆(δ)} (2)
Splitting the previous statements into atomic ones, we get –
C1,1 : B(β) causes A(α)
C2,1 : A(α) causes Γ (γ)
C2,2 : ∆(δ) causes Γ (γ) (3)
Α Β Δ
𝛼 𝛽 𝛿
𝛾 𝑖𝑠_𝑎
𝑐𝑎𝑢𝑠𝑒𝑠
𝑒𝑥𝑝𝑙𝑎𝑖𝑛𝑠
Γ
Fig. 4. Generic diagram for deducing explanation.
Figure 4 illustrates the hierarchy and causation of the concerned entities in
this example. The path from the nodes representing individual causes to the
verdict node leads to an explanation for the predicted verdict produced from
CBR. From the generic statements C1,1 , C2,1 and C2,2 it is found that, Preg-
nancy causes low TSH, Low TSH causes secondary hypothyroidism and Low
TT4 causes hypothyroidism respectively. Finally, the probable explanation is
extracted with predefined natural language for each of the object properties
as, “Pregnancy causes low TSH. Low TSH with low TT4 causes secondary hy-
pothyroidism. Therefore, Pregnancy can be an explanation of secondary hypothy-
roidism. ”
5 Conclusion
The rapidly growing nature of AI applications in the aspects of capabilities
and performance leverages the need of upgrading the intelligence. People seek
more humanly intelligence from the machines. To be specific, explanations are
expected more often in addition to the result of any task. This work represents
a framework for adding causation to CBR architecture using domain specific
ontology on hypothyroid disease diagnosis. Thyroid disease dataset was used to
develop the ontology and causal model. The outcome of this work would have
been more credible if there were instances of all types of hypothyroidism in the
selected dataset though some evaluation method was applied to justify the use
of k-NN in case retrieval step. Progressive works are still on to make this system
more robust and test with more feasible datasets. However, this causal model was
developed in a generalised fashion that can be adopted to any domain by using a
domain specific ontology and tuning several parameters in explanation translator
which enables the model with adaptability. Moreover, representing cases in XML
files can make provision to use this type of case-bases in distributed architecture
which would contribute to overcoming challenges of being scalable.
References
1. Barua, S., Begum, S., Uddin, M.: Towards Distributed K-NN Similarity for Scalable
Case Retrieval. In: ICCBR 2018: The 26th International Conference on Case-Based
Reasoning. pp. 151–160 (2018)
2. Ben Messaoud, M., Leray, P., Ben Amor, N.: Integrating Ontological Knowledge
for Iterative Causal Discovery and Visualization. In: Lecture Notes in Computer
Science (including subseries Lecture Notes in Artificial Intelligence and Lecture
Notes in Bioinformatics). vol. 5590 LNAI, pp. 168–179 (2009)
3. Besnard, P., Cordier, M.O., Moinard, Y.: Ontology-based inference for causal
explanation. Integrated Computer-Aided Engineering 15(4), 351—-367 (2010),
http://arxiv.org/abs/1004.4801
4. Can, Ö., Sezer, E., Ünalır, M.O., Bursa, O., Öz, M.: Implementing Thyroid Ontol-
ogy for Diagnosis and Care. In: GLOBAL HEALTH 2017 : The Sixth International
Conference on Global Health Challenges Implementing. pp. 49–53 (2017)
5. Cutic, D., Gini, G.: Creating Causal Representations from Ontologies and Bayesian
Networks. In: Workhop on New Research Frontiers for Intelligent Autonomous Sys-
tems (NRF-IAS). pp. 1–12 (2014), http://home.deib.polimi.it/gini/papers/
2014NFR.pdf
6. Eberhardt, F., Glymour, C., Scheines, R.: N-1 experiments suffice to determine
the causal relations among n variables. In: Innovations in Machine Learning, pp.
97–112. Springer (2006)
7. Fensel, D., McGuiness, D., Schulten, E., Ng, W.K., Lim, G.P., Yan, G.: Ontologies
and electronic commerce. IEEE Intelligent Systems 16(1), 8–14 (2001)
8. Giunchiglia, F., Dutta, B.: Dera: A faceted knowledge organization framework.
Tech. rep., University of Trento (2011)
9. Gruber, T.R.: Toward principles for the design of ontologies used for knowl-
edge sharing? International Journal of Human-computer Studies 43(5-6), 907–928
(1995)
10. Hitchcock, C.: Causal models. The Stanford encyclopedia of philosophy (2008)
11. Huang, B., Zhang, K., Sanchez-Romero, R., Ramsey, J., Glymour, M., Glymour,
C.: Diagnosis of autism spectrum disorder by causal influence strength learned
from resting-state fmri data. arXiv preprint arXiv:1902.10073 (2019)
12. Ishak, M.B., Leray, P., Amor, N.B.: Ontology-based generation of object oriented
bayesian networks. In: CEUR Workshop Proceedings. vol. 818, pp. 9–17 (2011)
13. Kolodner, J.L.: An introduction to case-based reasoning. Artificial Intelligence Re-
view 6(1), 3–34 (Mar 1992). https://doi.org/10.1007/BF00155578, https://doi.
org/10.1007/BF00155578
14. Lamy, J.B., Sekar, B., Guezennec, G., Bouaud, J., Séroussi, B.: Explainable artifi-
cial intelligence for breast cancer: A visual case-based reasoning approach. Artificial
intelligence in medicine 94, 42–53 (2019)
15. Mitchell, T.M.: Machine Learning. McGraw-Hill, Inc., New York, NY, USA, 1 edn.
(1997)
16. Pearl, J.: Causality. Cambridge University Press, New York, 2nd edn. (2009)
17. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible
Inference. Elsevier (2014)
18. Pearl, J.: The Seven Tools of Causal Inference with Reflections on Machine Learn-
ing. Forthcoming, Communications of Association for Computing Machinery 1(1)
(2018)
19. Radwan, E., Assiri, A.M.A.: Thyroid Diagnosis based Technique on Rough Sets
with Modified Similarity Relation. International Journal of Advanced Computer
Science and Applications 4(10), 120–126 (2013)
20. Raghu, V.K., Zhao, W., Pu, J., Leader, J.K., Wang, R., Herman, J., Yuan, J.M.,
Benos, P.V., Wilson, D.O.: Feasibility of lung cancer prediction from low-dose ct
scan and smoking factors using causal models. Thorax pp. thoraxjnl–2018 (2019)
21. Richter, M.M., Weber, R.O.: Basic CBR Elements, pp. 17–40. Springer Berlin
Heidelberg, Berlin, Heidelberg (2013)
22. Sene, A., Kamsu-Foguem, B., Rumeau, P.: Telemedicine framework using case-
based reasoning with evidences. Computer Methods and Programs in Biomedicine
121(1), 21–35 (2015). https://doi.org/10.1016/j.cmpb.2015.04.012, http://dx.
doi.org/10.1016/j.cmpb.2015.04.012
23. Shen, Y., Colloc, J., Jacquet-Andrieu, A., Lei, K.: Emerging medical in-
formatics with case-based reasoning for aiding clinical decision in multi-
agent system. Journal of Biomedical Informatics 56, 307–317 (2015).
https://doi.org/10.1016/j.jbi.2015.06.012, http://dx.doi.org/10.1016/j.
jbi.2015.06.012
24. Tong, S., Koller, D.: Active learning for structure in bayesian networks. In: Inter-
national Joint Conference on Artificial Intelligence. vol. 17, pp. 863–869. Citeseer
(2001)
25. Tucci, R.R.: Introduction to Judea Pearl’s Do-Calculus. arXiv preprint
arXiv:1305.5506 (2013), http://arxiv.org/abs/1305.5506
26. Wang, H.T., Tansel, A.U.: Composite ontology-based medical diagnosis decision
support system framework. Communications of the IIMA 13(2), 4 (2013)