Digitized world demands data integration systems that combine data repositories from multiple data sources. Vast amounts of existing clinical and biomedical research data are considered a primary force enabling data-driven research toward advancing health research and for introducing efficiencies in healthcare delivery. Datadriven research may have many goals, including but not limited to improved diagnostics processes, novel biomedical discoveries, epidemiology, and education. However, finding and gaining access to relevant data remains an elusive goal. We identified different data integration challenges and developed an Integrated Radiology Image Search (IRIS) framework that could be a step toward aiding data-driven research. We propose building a biomedical data categorization and integration framework using human-in-the-loop and developing data bridges to support search and retrieval of relevant documents from the integrated repository. My research focuses on biomedical data integration, indexing systems, and providing relevance-ranked document retrieval from an integrated repository. Although we currently focus on integrating biomedical data sources (for medical professionals), we believe that our proposed framework and methodologies can be used in other domains as well.
A growing amount of available biomedical data poses new
challenges in data management. Data re-usability is a highly
desirable goal, both for advancing science as well as for replicating
or validating results of previous studies. Recognizing this need,
publishers and funding bodies may require researchers to submit
data generated in their work and make it available to the research
community. For example, National Institutes of Health (NIH) is
encouraging funded investigators to use cloud computing to
conduct research and make their work accessible to larger audiences1.
1https://commonfund.nih.gov/strides/
However, in the healthcare domain, datasets are often not shared
because of security concerns, lack of integration, or limitations of
retrieval engines. A data integration framework should make data
available, accessible, and support fine-grained access control for
different users [
An integrated repository of teaching files can retrieve thousands of results for a text search. A search can thus become effectively useless without being able to show the most relevant results first. Publicly available radiology teaching file search engines do not provide text relevance ranking or combined text-and-image search. Lack of such systems motivated us to build Integrated Radiology Image Search (IRIS) and develop the ranking algorithm presented here. We presented IRIS at the annual Society for Imaging Informatics in Medicine (SIIM 2018) meeting (two posters: one focusing on search and another on data integration) and received feedback from doctors indicating that this work would be useful for the medical domain practitioners. 2.
In this section we discuss papers that addressed the need for data
integration and retrieval systems along with an overview of
existing medical data retrieval systems. Several studies have highlighted
2https://www.mypacs.net/
3https://www.myesr.org/eurorad
4http://mirc.rsna.org/query
the need for integration of healthcare data [
Mypacs.net: Publicly available teaching file resource with more than 35,000 cases and 200,000 images.
RadLex: RadLex is an ontological system that provides a comprehensive lexicon vocabulary for radiologists.
SNOMED CT: ontology provides a standardized, multilingual vocabulary of clinical terminology that is used by physicians and other healthcare providers for the electronic exchange of clinical health information. 3.
In this section, we discuss major biomedical data sources and significant goals that we identified as a part of my PhD proposal. 3.1
We currently focus on three types of data a) Electronic health records; b) Radiology teaching files or teaching files used by doctors and radiologists; c) Research datasets.
Electronic Health Records (EHRs): An electronic health record is a digital version of a patient’s record. EHRs are maintained at hospitals and provide patient information such as history of patient, medical test results, allergies, immunization details, radiology images, and clinical reports.
Medical Teaching Files: A radiology teaching files system is a collection of important cases for teaching and clinical follow-up. Teaching files share a similar overall structure but significant variations exist even within the same data sources and can include information such as patient history, findings, diagnosis, discussion, comments, references, and images related to clinical reports. Research datasets: From our survey with different research institute datasets, we observed that most of the data in healthcare domain are images (e.g., CT, X-ray, MRI). Those images are most typically stored in formats such as JPEG, DICOM, or PNG and include associated text data describing patient and case information. 3.2
We have organized this project into three phases (I finished the first two phases and working on the last phase of my PhD work). 5https://openi.nlm.nih.gov/ 6http://www.radlex.org/ 7https://www.nlm.nih.gov/healthit/snomedct/
SummarTyable 1: Research work summary IRIS 1.0 Teaching file text pre-processing and indexing.
Smart search through substitution of synonyms
and interpreting negation. Query expansion using
RadLex through an exact term match. [
Data integration as an iterative process, showing how
each integration step improved IRIS results [
Cluster analysis and coverage analysis for both ontologies and radiology data sources.
Unsupervised machine learning to identify data source properties – to identify best data sources and ontologies for integration (Journal paper – under review).
IRIS 1.2 Multimodal ranked retrieval for integrated radiology data sources using context of search term by considering weighted ontology and category terms (Conference paper – under review).
Toward using FAIR Principles for Fine-Grained
Access to aid Biomedical Data Driven Research [
For each phase we have identified a research question. Publications related to this work are briefly summarized in Table 1 3.2.1
Design an integrated smart database with heterogeneous data sources
Research question #1: How to determine which data sources and ontologies need to be integrated?
Most hospitals maintain a collection of teaching files, but many
public teaching file collections are also available through curated
online sources (e.g., RSNA MIRC, MyPacs, and EURORAD). We
developed IRIS engine as a pilot for a data integration system for
the healthcare domain [
Ranked retrieval search engine with multimodal text and image-based search capabilities
Research question #2: How to find relevant documents given a keyword query or hybrid (text+image) query? Figure 1 shows the architecture of IRIS engine. When a user enters a text query, IRIS performs query expansion using relevant ontologies, and retrieves relevant results to the query term. Our database also stores accuracy feedback from users which is then used to evaluate and iteratively improve IRIS results.
An integrated search may result in thousands of matches; thus, we are designing a search algorithm that ranks results by incorporating context computed through a weighted ontology terms. For text-based search ranking evaluation we used Normalized Discounted Cumulative Gain (NDCG)8 algorithm to measure the quality of search result ranking. Our analysis showed an improvement in ranked retrieval as compared to other search engines (addressing research question #2). 3.2.3
Data bridges and indexing mechanism to integrate biomedical data sources
Research question #3: How data integration performance (time) and scalability (adding variety of data sources) can be improved using data bridges? In order to make our integration solution applicable to other biomedical data sources (e.g., EHR’s, clinical reports), we plan to create data adapters that will serve as a bridge between data providers and data integration systems (this work was a part of my internship at NIH). Data providers can share their data in any file format and bridges will interpret that data in a uniform manner. As shown in Figure 2, our data clustering indexing approach starts with collecting different biomedical data sources. From our literature survey we observed that data preparation accounts 80% of data scientist work. Data preparation includes finding relevant data sources, extracting data from those data sources, data cleaning, and data integration. Our proposed data integration system would help data scientists and researchers optimize and streamline data preparation. We collected different biomedical data sources and working 8https://en.wikipedia.org/wiki/Discounted_ cumulative_gain on defining standard data cleaning technique that would be applicable to the most of the similar data sources that we proposed in this work. Our data categorization module categorizes data items into different sets based on the usage of those data elements in search operation. We need support from a human to check the accuracy of data categorization, to set similarity thresholds between different data items, and apply additional domain knowledge to categorize these data items based on relevance between data objects. Our data categorization algorithm will differentiate data items based on diagnostic relevance. For example, teaching cases with title, findings, and diagnosis would be treated as one sub-category in teaching cases (that would also integrate clinical reports) while another subcategory could integrate fields those are medically less relevant e.g., discussion, history, or comments. Based on data categorization we will be designing database schema and would also evaluate schema based on standard database schema benchmark techniques. Data write bridges would be responsible for the extracting data from different data categories and loading data to the respective database schema. This data categorization work is ongoing and we do not have any experimental results yet. We will address research question #3 by implementing this module. 4.
In this section we briefly discuss the current results from proposed system. 4.0.1
We evaluated IRIS search ranking using a combination of queries received from radiologists at a well-known hospital and other queries chosen from an extensive literature survey. We have initially tested a total of 28 text queries, out of which we picked a subset of 10 queries (Q1:Cardiomegaly, Q2: ACL Tear, Q3: Annular Pancreas, Q4: Pseudocoxalgia, Q5: Varicocele, Q6: Angiosarcoma, Q7: Tracheal dilation, Q8: Appendicitis, Q9: Bronchus intermedius, Q10: Cystitis glandularis) to perform an in depth evaluation. Due to space constraints we briefly discuss text based results. We evaluated text-based results on a scale from 0 (“not relevant”) to 2 (“very relevant”). We defined five categories to score text search results: “not relevant” = 0 (when term and synonyms do not appear anywhere in the results), “relevant” = 0.5 (if term or synonyms appear in any category of teaching file), “more relevant” = 1 (if term or synonyms appear in discussion category), “most relevant” = 1.5 (if term or synonyms appears in history or ddx category), and “very relevant” = 2 (if term or synonyms appears in title, findings, or diagnosis categories).
We compared IRIS relevance rank algorithm with MIRC using the same dataset. We considered top four teaching file results from IRIS, MIRC, and Google site search. We calculated relevance score by scoring top four teaching files from each engine, using weighted ontology ranking algorithm . Figure 3 shows an overall analysis of results from these 3 search engines. score for each search engine shows that IRIS relevance rank algorithm performs better than other two engines.
We also considered how other public medical radiology teaching file search engines rank their search results. We used the same query set and performed a search using MIRC, MyPacs, EURORAD, and Open-i search engines. We discuss only two queries (Q1:“cardiomegaly” and Q8:“appedicitus”) in detail and reporting scores for the top 10 search results. Figure 4 shows a comparative analysis of ranked results from these four engines using the relevance scores based on our metric described above. Open-i can rank search results based on different categories (e.g., based on diagnosis or based on teaching file date) – we used a diagnosis based search in Open-i. MIRC ranks results based on the date of modification with no other option available. Our analysis shows that none of the search engines return the most relevant results first. Interestingly, top results are often less relevant than the subsequent search results. For example for “cardiomegaly” MyPacs fourth result is more relevant than the top three results. EURORAD does not retrieve any results for “cardiomegaly” but we checked “appendicits” results – and those were also not ranked based on the relevance of the search term. 4.0.2
IRIS hybrid algorithm augments the text search with image search and re-ranks results based on the relevance to the query. Due to space constraints we briefly discuss hybrid search result. IRIS textbased and hybrid search results scored an score of 0.83 out of 1. Image search scored only about 0.53 out of 1, validating our use of the image search as an enhancement to the text search (rather than a standalone search). Hybrid search scored 0.84 out of 1 because of text results were augmented with image-based results. For hybrid search Some of the results were noticeable better than text-based search.By combining text search with image results, we are striving to get a text-based match that also includes a similar image.
The ranking approach presented in this paper is significant because it enables IRIS to present the user with top relevant reference cases first. Through integrating term frequency, adding more weight to ontology terms we show that teaching files can be better ranked in order of their relevance to a search query. Currently I am working on data write bridges and categorization algorithm to improve biomedical data integration process. 6.
This research was supported in part by the Intramural Research Program of the National Institutes of Health (NIH), National Library of Medicine (NLM), and Lister Hill National Center for Biomedical Communications (LHNCBC). 7.