Making a correct diagnosis is a complex effort. Depending on the specialty, physicians can have to work with a long list of diseases with intersecting, often not obvious symptoms. Despite that patient treatment should be personal, the nature of diseases is rather repetitive. Patients often share common symptoms based on the location, communication network, season of year, etc. Such repetitiveness introduces a value of having an information system that can make suggestion to the physicians based on a set of factors - symptoms, location, season, diagnosis of other patients, etc. The work described below is based on the industry-standard medical datasets and technologies, but it's not limited to a particular dataset, clinic, language, country, or technology. The system itself provides benefits both for physicians, where they become aware of the situation in their area or departments, and hospitals, where they can collect statistics regarding health situation in the given region or other data projection and analyze the info.
Ukraine Luchkevych)
Healthcare is one of the most important parts of our live. Good healthcare services result in better quality of live, longevity, attractiveness of the country or specific region for migration. Making a correct diagnosis of a patient’s disease is crucial to the patient, physician, and healthcare system in general since mistakes can lead to severe consequences or even patient’s death. Usually, physicians rely only on their knowledge and patient feedback, but they does not consider situation in their neighborhood, region, and other factors like trending diseases (viruses), etc.
Having information system that allows to consolidate cross-patient information allows physicians to see insights about trending diseases and understand actual trends in real-time. The same system helps hospitals to analyze trends and make proper forecasts and preparation.
This study involved the development of an expert prototype [
A system that helps patients diagnose their illness and provide them with appropriate advice.
In addition, the knowledge management used in the system is learned by the expert system. The aim
of this study was to find an appropriate language to represent the patient's medical history and current
situation in the knowledge base for expert systems to be able to perform an effective consultation. An
example of an expert system was developed using CLIPS (C Language Integrated Production System)
with a Java interface [
An expert system solves problems by modeling the human reasoning process and applying specific
knowledge and interfaces. Expert systems also use human knowledge to solve problems that normally
require human intelligence. These expert systems represent expert knowledge as data or rules in a
computer. These rules and data can be used when needed to solve a problem. A person must read and
interpret the knowledge that is used. A computer program designed to simulate the ability of a human
expert to solve problems [
2023 Copyright for this paper by its authors.
Expert systems and artificial intelligence covers such diverse activities as game automated reasoning, natural language, automatic programming, machine learning, robotics and vision, software tools, modeling human performance and expert systems for complex decision making. Comprehensive medical solutions take center stage and every step and our system helps solve this problem with an expert system.
There are many medical diagnostic expert systems MYCIN [
MYCIN was the first known medical expert system developed by Shortliffe at Stanford to assist physicians rather than antimicrobial experts. A limitation of MYCIN was that its knowledge base is incomplete because it does not cover the full spectrum of infectious diseases. Running it would require more computing power than most hospitals at the time could afford. Doctors don't like to type in a terminal and demand a much better user interface than the one provided.
EasyDiagnosis is expert system software that lists and clinically describes the most likely conditions
based on an analysis of your particular symptoms. EasyDiagnosis focuses on the most common medical
complaints for most doctor visits and hospitalizations [
The system designed at the article is a distributed search system targeted both for physicians and
hospital’s analytics department. At the same time, the system is designed to work with personal data,
so it should comply to international healthcare and privacy standards [
The system should comply to the following functional and non-functional requirements [
It must be clearly stated that the system is an assistant to the physician [
During the article the following terms and keywords will be repeated multiple times [
The system is designed to replace in daily use both public solutions and in-house competitors. Due
to an NDA, it’s not allowed to name specific vendors or systems. The further comparison will be done
in the following way [
The component diagram below describes how the application looks like on high-level view [
Theoretically, it should be possible to use a NoSQL database like MongoDB and other search engines like Solr or Sphinx, however those components were not evaluated for the system feasibility and most likely will require certain changes to the codebase.
The Diagram Below illustrates data flow (via action/sequence diagram) of a typical search request
Whenever physician selects an item on UI, a feedback action is required to update usage statistics which is essential for the system functionality in terms of user experience and statistics
Diagnosis search and suggestion is one of the key components of the system. To be able to provide
suggestions, the search engine must be pre-populated with medical vocabulary (dataset). For the
illustrative purposes, the system will be populated with the ICD-10 dataset having the following field
mappings:
code – a code used to identify the diagnosis
text – description of the diagnosis
symptoms – a list of symptoms relevant for the diagnosis
facets – a list of categories, where a particular diagnosis is applicable
Below you can find a typical ICD-10 record mapped to the fields used by the system [
At the same time, it’s not enough just to map records to the fields [21]. A search engine should be configured to understand field mappings and types [22]. At the same time, since indexing is happening on insert or update, a search engine must be additionally configured with the indexing (field analysis) rules. Having correct indexing rules are crucial both for the search accuracy and performance.
At the Figure 5 there are two main blocks: “settings” and “mappings”. “Mappings” block is responsible for field types and analyzer selection, while “settings” block is responsible for analysis configuration. Whenever record is inserted into the search engine, it’s being indexed according to the settings (show Figure 5). In the current use-case, Elasticsearch is configured to do the following analysis operations on the records inserted (apart from facets fields): 1. Convert text to lowercase 2. Remove stop words 3. Use stemmer 4. Remove duplicated tokens
Once the data is analyzed, it’s being stored in the search index using edge-ngram tokenizer which is configured to store tokens of 2 and up to 10 characters. Such approach allows to implement search-asyou-type style suggestions. For example, let’s analyze the text of R50 diagnosis: Fever of other and unknown origin.
As it’s demonstrated on Figure 6, the diagnosis text is split to a set of tokens like “fe”, “fev”, “feve”, etc. These tokens are being used to match the search input and to provide the most relevant results. At the same time, stop words, punctuation symbols and duplicated tokens are removed.
Having the above index, it’s easy to make a search having either full or partial input of the term. The search for user input “feve” would result in item:
Additionally, by applying “fuzziness” factor to the query a search behavior can be altered to support a certain number of mistakes. For instance, if “fuzziness: 1” is specified in the search query the system would provide results both for “feve” and “fevr” search input.
Being able to re-order search results is crucial for the system to provide a high-quality search result. The system should score items differently if a search input is found in code, text, or keyword field. Typically, the code match must be ranked higher and symptom match must be ranked lower then description field. To achieve such results, the boosting functions are applied.
The similar technique is used when the system wants to suggest an item that is frequently used by other physicians. For instance, if a particular item must be ranked higher a script-based filter function is applied which can manipulate the final search score of the item.
Since different countries naturally use their own languages and localized versions of medical diseases classification, it’s crucial for the system to support multiple languages. The same applies for a revision of medical classification systems which can add new records once new diseases appear. A good example is COVID-19, which was not initially present in the medical classification systems.
To add support for the custom language and vocabulary, a simple three steps must be taken:
A local dictionary must be loaded, for instance ICD-10-FR for France Analysis settings (Figure 5) must be adjusted to use French version of the snowball filter A separate index must be used (for instance icd10fr)
In case there is a need to update the existing dictionary, it’s simply enough to re-insert data into the existing index. The system will be fully operational during the system update. Physicians might experience a slight slowdown of the system, but typically it’s not the case. The data consistency is guaranteed by re-using the same disease codes. The records in the search engine and database may be added or updated, but not removed. The removal of the items is strictly forbidden because it will lead the system to an inconsistent state. Instead, certain items may be marked as excluded for the search.
It must be noted that the system is designed to have 24\7 operations. Eventually, during system evolution there is a need to introduce braking changes to the system. In order to achieve 0-downtime, the changes must be applied in the following order: A database or search engine or and API is modified with a backward-compatible change, e.g. a new field, property or endpoint is added Consuming components are updated to use a new field, property or API endpoint instead of a legacy one The legacy field. Property or API endpoint is removed
System performance heavily depends on multiple factors like hardware, number users, data size, etc. At the same time, it’s crucial to follow the following key principles which will heavily impact scalability: API layer should be stateless to allow horizontal scalability Search Engine should support data sharding and use separate indexes for each dataset Database engine should use separate schema per tenant (hospital or network of hospitals) If the above characteristics are met the modern 4-core/16 GB RAM AWS instances (or their physical counterparts) can serve thousands of search requests per second.
An ability to provide multi-dimensional reports is a last but not least goal of the system. The system itself is not generating reports but it acts as a data source for another reporting system. By combining vocabulary data with the export from the database (preferably due to performance) it’s possible to achieve a free-form report generation by a 3rd party reporting system. Another option would be to embed a reporting suite directly into the application however it’s not recommended due to the extra cost and overall increased complexity of the system. Depending on the requirements, the variety of reporting tools can be used, such as MicroStrategy (commercial), Kibana or even Microsoft Excel.
Any medical system, as well as the system that works with personal data should obey a set of rules and regulations. There are various rules and regulations across the globe, the most known are GDPR for EU and CCPA for US. Besides mentioned rulesets, there is a set of medical regulations such as HIPAA and HITECH which must be considered.
The system achieves security by combining a set of best practices (OWASP Top 10) with the set of special measures applied at the individual layers of the system. Let’s look on the layers: Search engine is a layer does not require any special treatment because it contains dictionary data. Database is the place where all the selected diagnosis are stored. Ideally the database should not contain any PHII (personal health identifiable information) to minimize security risks API Layer, should follow industry best practices, but does not require any special treatment All the communication should be secured and use proper credentials as well as the encryption-intransit approach.
To understand how system performs, let’s do a set of comparisons with the competitors:
1. The smart search systems are valuable both for physicians and hospitals. They allow to enhance physician workflow as well as provide analytics for hospitals. 2. Usage of open-source components allowed to rapidly create a production-ready prototype of the system with the support of a multiple languages, dictionaries, facets. Moreover, the interchangeable nature of components allows to host the system on premises or in the cloud, powered by cloud technologies like Aurora or OpenSearch. 3. The result of the experiments demonstrates clear advantage of the system in the multiple areas in comparison to the existing public and proprietary classification search systems. 4. While being focused on healthcare domain the system can be adapted to the non-healthcare domain and be used, for instance, in retail by re-using similar concepts and approaches. [21] Medykovsky, M., Droniuk, I., Nazarkevich, M., & Fedevych, O. Modelling the pertubation of traffic based on ateb-functions. In Computer Networks: 20th International Conference, CN 2013, Lwówek Śląski, Poland, June 17-21, 2013. 2013. Proceedings 20 (pp. 38-44). Springer Berlin Heidelberg. [22] Ahn, J. K., Choi, S., Hasegawa, S., Hayakawa, S. H., Hong, J., Ichikawa, Y., Yoshida, J.
Superconducting dipole magnet for Hyperon spectrometer. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 2023, 1047, 167775.