We describe the FM-Ultranet approach to provide decision support and experience management for a medical doctor in the course of conducting an ultrasonographical examination of a fetus in the mother's uterus. Based on the medical background and the requirements of the examination we introduce a knowledge representation of fetal malformation and its utilization in a Case-Based Reasoning system to assist the examiner. We conclude with a description of our results from working with domain experts and give directions for future work.
As compared to other medical imaging techniques, ultrasound imaging is a real noninvasive and cost-effective solution in many diagnostic approaches. In the field of obstetrics (fetal ultrasonography), this technique is used in a different way than in other areas. Scanning a fetus is done mainly to detect abnormalities at a very early stage (e.g. of the head, the heart, abdominal wall, umbilical cord, urinary tract, skeleton, etc.). This is done not only by observing the anatomical features of each organ but also by dynamically examining the respective positions of these organs as well as movements of the fetus.
This is a difficult task done by an ultrasonographist (i.e. a radiologist, an obstetrician, a general practitioner) which requires an average of 1/2 hour of examination. This task is operator dependent and subject to a strong personal interpretation as reference cases are not easily accessible: sequences of high definition images are more difficult to store and to browse than traditional radiology books. This adds difficulties to the limits of today ultrasound diagnostic imaging inherent to the use of the technology (such as use of a probe) and coming from technical weaknesses (such as image resolution). The specificity of this exam is far lower than for other areas (in Europe, 60% to 75% of the malformations remain unnoticed before birth) [GLL99] and it is worse in North America [WPCSB95]. Moreover in most European countries, when the ultrasonographist believes that there is a risk of malformation, he is compelled to get an external advice from an expert often attached to a regional reference center. On the average, the expert invalidates the presence of the malformation on 40% of the cases (second level exam). As 1 person out of 2 is a woman and, in Europe, women have more than one children on the average, the corresponding “market” is very large. Better-trained ultrasonographists should reduce the risk of diagnosis errors, limit demands for control to reference centers and consequently benefit to public health and reduce health costs.
The goal of our work is to progress significantly towards the above objective. Ultrasonography still delivers often low quality images. Furthermore, it requires the interpretation of moving pictures. For these reasons our approach is to support the human image interpretation task rather than to attempt to automate the recognition process. Ultrasound scans interpretation and diagnosis can largely be improved through comparison of past existing similar cases and with guidance of an expert. Hence, FMUltranet combines Case-Based Reasoning (CBR) and Expert networking to improve ultrasonography through decision support. CBR seems to be best suited to achieve this goal. In the past there have been a number of applications of CBR in medicine. [GBS98] presents a comprehensive overview. It also shows that the integration of such systems into the medical work process is crucial for the success. Following we present our approach in detail. 2
When an ultrasonographist detects a malformation during a morphological examination of a fetus then he may remember a previous case from his experience and/or he may ask an expert for a second opinion. These are two ways to obtain support for his decision, i.e. the kind of malformation and, in the worst case, the termination of pregnancy. In day to day practice this kind of support is not available during the examination process. Hence, FM-Ultranet’s approach is to extend the workplace by a software tool that provides access to experience through a database of reference cases and active support for the examination process.
In the course of an examination the ultrasonographist usually, i.e. in 95% of the cases, documents a normal morphology with no special symptoms. This includes some administrative data of the patient as well as a standard description of the morphology. Only in the case of a suspicion of a malformation a more detailed description is
Kidney +Aspect of Kidney : Simple Biometry +Biometry Of Kidney : Biometry of Kidney +Kidney Pelvis : Normal/Abnormal +Change of Aspect of Kidney : Change +Change of Kidney Pelvis : Change
Status of Kidney +Kidney shape : Normal/Abnormal +Cortico-medullar differentiation : Normal/Abnormal +Change of shape of kidney : Change +Change of Cortico-medullar differentiation : Change +Change of Duplication : Change +Change of Status : Change +Duplication : boolean(idl) +Status : Normal/Abnormal «enumeration» Simple Biometry -small -normal -big «enumeration»
Change -decrease -unchanged -increase
«enumeration» Normal/Abnormal -normal -abnormal necessary. This encompasses biometry data and symbolic information on the suspected malformed organ, e.g. the kidney. This data is collected with the FM-Ultranet tool. It is built on top of a CBR engine and hence able to retrieve the most similar past cases form a reference case base as well as from the ultrasonographist’s local case base. This past case data is utilized to support the decision process, i.e. the ultrasonographist can compare the found cases with his present examination, e.g. in the case of the suspect kidney he will see an attributed description and a full motion ultrasonography. This enables him to evaluate his own findings including the visual impression. Additional support is provided by medical background knowledge that, e.g. ensures the completeness of collection of medical data. In the case of a malformed kidney it is recommended to examine the fetus’ head. If the head is too small then this is an additional indicator for a malformed kidney.
Finally, the conclusions of the past cases can be adopted to the present case to ensure the consideration of all relevant aspects such as further requirements for examinations, termination of pregnancy, follow-up of pregnancy, pediatric surgeon counseling, etc. If the level of confidence into the examination’s findings is still too weak, then the collected data together with attached images from the present examination can be posted to an expert. The structure of the data within FM-Ultranet forms a standard that allows for easy exchange of data among ultrasonographists. It thereby leverages the collaboration within this community of expertise.
FM-Ultranet acts as an electronic assistant to its user. It trains the apprentice to do complete examinations and to recognize the relevant symptoms of the diverse malformations. It supports the practitioner by easing his daily work while supporting him in the exceptional case of a malformed fetus. Last but not least it assist the expert in exploring large collections of cases and helps him to share his experience. 3
Technically, FM-Ultranet is a knowledge-based support system. Its kernel is a CBR system that utilizes an object-oriented case model. The similarity assessment applies specific medical measures that were developed with gynecologists. These two knowledge containers, i.e. cases and similarity measures, are augmented with medical background knowledge in the form of rules to express constraints and consistency conditions, among others.
The case base currently contains 120 reference cases of malformation of the urinary tract because it is the most frequent malformation and in the meantime it is difficult to detect. Each case is given by an object-oriented representation, i.e. a hierarchical structure containing 140 attributes. These are grouped into classes that represent a set of characteristics in a medical sense. These characteristics comprise all relevant information of an exam: maternal age, clinical data of the mother (medical history, obstetrical medical history), date of exam, clinical data of the fetus (amniotic fluid, urinary biology, RMI), data of the medical conclusion (further exam required, termination of pregnancy, follow up of pregnancy), pregnancy issues (delivery, abortion, termination of pregnancy, status of child), general report etc.
In figure 1 we show an UML like diagram of a part of the kidney representation. The Biometry of the kidney class is a specialization of the Kidney class. The latter contains all information required in the absence of a malformation. Its specialization adds 8 attributes to describe the basic characteristics of a malformation of this organ. Two further levels of detail can be utilized. An excerpt from the overall structure of the part-of hierarchy of a case is shown in figure 2. Each attribute is either typed by a class, thereby representing the part-of structure of the fetal anatomy or by a simple data type. The majority of the simple types are given by symbolic value enumerations, e.g. increase, unchanged and decrease for Change. Numerical data is used for biometrical information. Each attribute can also carry a special value, i.e. seen, not seen, It cannot be seen and I don’t know to express uncertainty and the state of examination. Often it is obligatory for the ultrasonographist to specify whether an organ is normal or abnormal, e.g. if the kidney is normal it is sufficient to enter a normal value for the attribute status of kidney; if the kidney is abnormal it is necessary to enter the value abnormal in the corresponding attribute. Thereby the navigation in the structure towards the extended description is controlled. For each attribute a similarity measure is defined by a mathematical function or a table, depending on the attribute’s type. For example the similarity measure for the gestational age is a function that reflects the usual time frame of eight weeks in which the regular examinations take place, i.e. around week 14, 22 and 34. Figure 3 depicts this function, where e.g. a case that is from a week that is 6 weeks later than the present one is considered as being 50% similar.
FM-Ultranet
Case
Liquid images
Kidney
Adrenal
Bladder Biometry Urogenital System Morphology
Sex Ureter Abdomen Stomach Thorax
Biometry of kidney
Head Cerebral structures Heart
Lung Additional medical background knowledge is expressed in rules. There are rules to derive the value of attributes from others; to show deviations between the present exam and the last one of the same patient; to guide the practitioner, i.e. to draw his attention to relevant pathological characteristics of the exam; and to alert him if he enters inconsistent or incomplete data. An example for a rule is (plain English is used for readability): “If attribute status of class status of kidney is abnormal then the weight of attribute head of the class Morphology is increased by 10”. This rule expresses the dependency between the kidney and the head in case of a malformation, i.e. when the kidney is malformed, then check the head for malformation. By changing the weight of the relevant attribute the CBR system will retrieve cases that reflect this situation. 4
The whole system has been implemented with the CBR tool CBR-Works from empolis, Germany. We have collected 120 reference cases of the malformations of the urinary tract including the related image data. The knowledge representation fully covers the sane and normal fetus. According to the representation of malformations the urinary tract is fully described, 25% of the heart and 10% of the abdomen are covered. The model consists of 40 classes with 140 attributes. Each class has a rule set to ensure completeness and consistency of the description and to provide guidance for the image interpretation. All discriminant attributes have appropriate medical similarity measures. The system is used at the ultrasonographer’s workplace. Data collection is done easily from a single page that offers all relevant entries for a sane fetus. In the case of an abnormal value the related detailed descriptions are provided. Thereby only in the rare cases of a potential malformation an additional effort is required. In 95% of the cases with a normal fetus, the required overhead for documentation is reduced. Instead of creating a report manually, the examiner just annotates picture information where necessary to the filled attributes and then generates his case documentation report with a single click. The resulting document can be further edited in any RTF word processor, e.g. Microsoft Word©.
The results provided by the CBR system, i.e. reference cases were evaluated and have met the expert’s expectations. The system is appropriate to train examiners and thereby to improve there ability to detect malformations accurately.
The system was deployed and tested in a trial with medical practitioners in Nimes and Liège. Most appreciated by these users was the similarity based retrieval of reference case data to improve their diagnostic quality and the use of the system as a personal experience management system. All participants actually want such a system as integrated part of their work environment. The latter requires a better mapping of the actual medical work flow and the technical interfacing with the ultrasonography equipment. 5
The FM-Ultranet approach has proven its feasibility in the work with subject matter experts. Focusing on decision support and assistance rather than diagnosis proved to be appropriate for the process of examination. The medical doctor has to follow and react on the movements of his patient, i.e. the fetus in the uterus of the mother rather than following a diagnostic procedure that is defined, e.g. by a decision tree.
With the help of the system, data collection turned out to be much completer than before. In particular, the creation of the reference case base produced a new asset in itself. The presentation of a relevant reference case during a present exam leads to quicker and safer decisions, hence improving the confidence level of the ultrasonographist. The representation of malformation of the fetus needs to be extended, e.g. for the abdomen. However, the present structure already forms an important step towards standardizing the description of malformations. It is compliant with the documentation duties of the ultrasonographist and it contains the ICD10 classification.
In a recent work [JMGSC98] argues for the integration of CBR with Image Based Reasoning in the domain of In Vitro Fertilization (IVF) analysis. While the human interpretation of images is more flexible, the machine interpretation can provide precise and objective results. The FM-Ultranet feature set should be analyzed according to which of its features can be extracted from the image data automatically, when the quality of images increases and when the field of automatic image analysis advances. Finally, FM-Ultranet is an excellent example of the expressive power of a CBR system. It utilizes all four knowledge containers as introduced by [Ri95] in an exemplary way: (1) Case base: Storage of experience (2) Vocabulary: Model or structure of the domain (fetal anatomy) (3) Solution Transformation: Explicit diagnostic knowledge (4) Similarity Measure: Expert evaluation of symptoms As such, the implementation can serve as a source for teaching CBR. The work presented in this paper has been partly funded by the European Union within the IST research programme under contract no. IST-1999-20865: Fetal Malformation – Ultrasonography training networking platform. 7 [GLL99]