We describe a new approach for resolving conflicts for automatic identifying human organs from a medical CT images. The main premise of this approach is the use of classifier created with using two-level classifier and domain knowledge advisers decisions. We test our approach on multiple CT images of chest organs (trachea, lungs, bronchus) and demonstrate usefulness and effectiveness of the resulting classifications. The presented approach can be used to assist in solving more complex medical problems.
A design of human–machine interface is the most important aspect of computer aided
interpretation of medical image exams. Assists include decision support, reminder and
navigation techniques to help avoid diagnosis errors, content-based data mining
capabilities, and access to reference libraries. Human–machine systems should take advantage
of computer capabilities to increase physicians interpretation capabilities [
An automatic identification of medical objects visualized by Computed
Tomography (CT) imagery (e.g., organs, blood vessels, bones, etc.), without any doubt, could
be useful, to support solving many complex medical problems using computer tools.
Our approach is based on a two-level classifier. On the lower level, our approach uses
a classical classifier based on a decision tree that is calculated on the basis of the local
discretization (see, e.g., [
In the Section 2, we describe the problem of medical image understanding. Second
section present conception of design a system for automatic medical objects
classification. Finally, we present the complete structure of two-level classifier with method for
resolving conflicts for the automatic classification of chest organs (see Section 4).
A process of radiological interpretation generally includes the understanding of
medical image content resulting in recognition of possible pathology symptoms, most often
called detection, and assessment of comprehensive image information in a context of
current clinical case-knowledge. It involves image-based detection of disease, defining
disease extent, determining etiology of the disease process, assisting in designing of the
clinical management plans for the patient, based on imaging findings, and following
response to the therapy [
CENTER Center of the object region (e.g.R1, R2 . . . )
OIL The number of objects on the right side OIR The number of objects on the left side OIA The number of objects above
OIB The number of objects below DTNLO Distance to the nearest object on the left side DTNRO Distance to the nearest object on the right side DTNAO Distance to the nearest object above DTNBO Distance to the nearest object below SNLO The size of the nearest object on the left side SNRO The size of the nearest object on the right side SNAO The size of the nearest object above
SNBO The size of the nearest object below
The other area of application of the automatic image understanding technique is
deep and requires a detailed analysis of particularly difficult images, especially in case
of doubts and difficulties in deciding on final diagnosis. A very important difference
between all traditional methods of automatic image processing (or recognition) and the
new paradigm for image understanding is that there is one directional scheme of the data
flow in the traditional methods; there are two-directional interactions between signals
(features) extracted from the image analysis and expectations resulting from the
knowledge of image contents, as given by experts (physicians). The results of all analyses
of medical image characteristics and objects visible in them, generated by computers,
allow the physician to base his/her reasoning on much more reliable and quantifiable
premises than just a visual assessment of that image, improving both the effectiveness
of his/her activities, and the feeling of reliability and security. Finally, the increasing
acceptance of techniques for the automatic recognition and classification of biological
objects distinguished in medical images can help the doctor make the right diagnostic
decisions, although these techniques sometimes require the doctor to be able to
critically assess the automatically suggested categories, as every recognition technique
carries some level of error, while nothing excuses the doctor’s personal responsibility for
his/her decisions [
Medical image analysis is one of the areas of computer vision where domain knowledge plays a very important role, because localized pixel information obtained from CT related to various medical problems. To understand medical image correctly, a computer should detect and recognize quite correctly all medical objects located on the image by using domain knowledge (extremely challenging task even for a man). 3
3.1
In order to understand the medical images, it is important to create a tool for
understanding the interior of the human body on different levels of abstraction and tracking
of interaction between the observed medical objects. The main issues to be addressed
include problems with the quality of the medical image data, problems with domain
knowledge descriptions and problems with modeling and exploration of the human
body, which is very complex. The system should include the assumptions, such that
the system should support work of doctors (not replace), expert always decide,
system should allow for future sharing of knowledge and should naturally communicate in
order to exchange knowledge (speech).
There is no ”ideal set of features” which characterize the object. Features are selected
individually depending on the recognized objects. In the computer analysis of the
images, extracted features from the image, can be assigned to one of the categories, such as
non-transformed structural characteristics (e.g.moments, power, amplitude information,
energy, etc.),transformed structural characteristics (e.g.frequency and amplitude
spectra, subspace transformation methods, etc.), structural descriptions (formal languages
and their grammars, parsing techniques, and string matching techniques) and graph
descriptors (e.g.attributed graphs, relational graphs, and semantic networks) described in
detail in [
”Domain Knowledge” Features (DKF) 3.3 To understand the image, it is also necessary to define the additional features that will define the acquired domain knowledge from experts. We call these features Domain Knowledge Features (DKF). DKF can be assigned to one of the categories, such as: – features used to describe domain knowledge about the number of objects that surround an analyzed object, – features used to describe domain knowledge about the distance from analyzed object to surrounding objects, – features used to describe domain knowledge about the size of objects that surround an analyzed object, – features used to describe domain knowledge about position of an object.
In total, for the purposes of the experiments we define 13 DKF features (see Table 2). Our experiments were carried out on the data obtained from the clinical hospital Jagiellonian University Medical College in Kraków (the patients were diagnosed with asthma). The entire data set counted 26 patients (19 woman, 7 man). The average age of patients was 58.12 years (st.dev. 6.78 years, age range from 47 to 70 years). In all patients, volumetric CT torso scans were performed at both full inspiration and expiration with using 16-channel multi-detector CT scanner Toshiba (manufacturer’s model name: Aquilion). The acquired data were reconstructed using a kernel (FC86) with 1 mm increments. Images were stored in the Digital Imaging and Communications in Medicine (DICOM) format. For each patient was taken 300 to 400 images (full inspiration) with a resolution of 512x512 pixels. The total size of the data set for the experiment count 9655 CT images.
From all images we select every fifth image (20% of all images, 5mm increments) to pre-processing. As a result of the segmentation process, we acquired 7491 objects for experiments. For all the objects we set LLF and DKF features, further all objects are classified by an expert to one of the 7 classes (chest organs) presented in the Table 3.
The entire data set was divided 20 times randomly into two sets - a set with training data and a set with test data (around 70% of the data getting into a training set - 18 patients, other (around 30%) into test set - 8 patients). Experiments 1,2,3 and 4 we conducted on these datasets. In 5-th experiment the entire data set was divided 20 times randomly into three sets - a set with training data and a set with test data (9 patients of the data getting into a training set, 9 patients into valid set and other 8 patients into test set).
To verify the effectiveness of classification we prepare five methods. In methods one to four with using training data we built a classifier, which has been tested on test data. In method five with using training data we built a two-level classifier. Decision from this classifier has been used with advisers decisions to create second classifier on valid data. This classifier is used to resolve conflicts between domain knowledge advisers and two-level classifier.
We designed a classifier to the automatic classification of chest organs. In all
methods we have implemented classifiers in the IMPLA (Image Processing Laboratory),
which is a continuation of the RSES-lib library (forming the kernel of the RSES system
[
The second method was similar to the method 1 and based on the decision tree with local discretization. This method has used both LLF and DKF features (see Table 4). 4.1
Method 3 was similar to the method 1 and based on the decision tree with local discretization. This method has used only DKF features (see Table 4). 4.4
Method 4 is a method based on two-level classifier with ”advisers” (Figure 2). In this
approach classification decision is dependent on suggestions of domain knowledge
advisers. DKA suggest decisions based on domain knowledge e.g.”Left lung is located
on the right side of medical image”, ”Object located on the left side of medical image
is probably not a left lunge”. We prepare 15 DKA for all chest organs. Advisers are
divided into two groups:
– YES advisers - Advisers to advise on YES e.g.”yes, this is probably the left lung”
(6 DKA),
– NO advisers - Advisers to advise on NO e.g.”no, this is probably not the left lung”
(9 DKA).
Cssts
C-LL
Verification was followed on the basis of the DKF features e.g.if object center is located
in region R3, R6 or R9 then YES adviser for right lunge take false decision. Advisers
suggest what should be a decision (YES advisers) or suggested what should not be a
decision (NO advisers). If any of the advisors suggested otherwise than the classifier
(in some sense, the low-level classifier), decision was suspended (see [
This method was similar to the method 4. In this experiment the entire data set was
divided 20 times randomly into three sets - a set with training data, a set with validation
data and a set with test data (9 patients of the data getting into a training set, 9 patients
into valid set and other 8 patients into test set). All the experiments (experiment with
method 1 and 4 was prepared on merged training and validation data sets) we conducted
on these datasets. Classification decision is dependent on suggestions of domain
knowledge advisers (see [
The results of experiments performed on medical data sets indicate that the presented approach seems to be promising. The use of domain knowledge and the addition of a classifier resolving conflicts between advisers significantly improved the quality of the medical object identification. The next steps will focus on the use of time dependencies between medical images (object tracking in time).
The presented approach can be used in the future to support solving more
complex medical problems. We plan to use the results of research, among other things, to
treatment of an asthmatic airway remodeling (see, e.g., [
This work was partially supported by the Polish National Science Centre grant DEC2013/09/B/ST6/01568 and by the Centre for Innovation and Transfer of Natural Sciences and Engineering Knowledge of University of Rzeszów, Poland.