Building a neurological patient multimedia database for information retrieval Suela Maxhelaku Alda Kika Silvana Greca Arben Rroji Department of Department of Department of Neuroradiology Informatics Informatics Informatics Service University of University of University of University Tirana Tirana, Tirana Hospital Center suela.maxhelaku@ alda.kika@fshn.ed silvana.greca@fsh Mother Teresa fshn.edu.al u.al n.edu.al arbenrroji@yahoo. com extraction and classification of images for easy and efficient retrieval. Content Based Image Abstract Retrieval(CBIR) is an automatic retrieval of images generally based on some particular properties such as This paper will focus on presenting the color composition, shape and texture. Every day large challenges to retrieve information from volumes of different types of medical images such as medical data such as Digital Imaging stored dental, endoscopy, skull, MRI, ultrasound, radiology in a multimedia database. This database will are produced in various hospitals as well as in various store information about patients and high medical centers. Medical image retrieval has many significant applications especially in medical resolution images scans. diagnosis, education and research fields The data was gathered from patients in [Ash,Man12]. Neuroradiology Service at UHC “Mother Teresa” in Tirana. This is a national Technological capabilities in the field of medical reference center and performs more than imaging contribute to the increasing use of image 5000 scanned images in year. It would be analysis in the diagnostic medical systems. Medical very useful for the physicians to use the imaging is derived from a number of tomography gathered data of different format to retrieve studies, primary including radiography, information about each patient or for a ultrasonography, computed tomography and magnetic specific disease. The characteristics of resonance imaging. The methods of collecting and multimedia database, technologies that can storing medical images can be performed using be used and the structure of the multimedia almost any database system. However, the analysis of database from the gathered data to retrieve this type of information is a complex issue and information are presented in this paper. requires advanced information technologies. The appropriate management of medical image and patient information is related to the issues concerning 1. Introduction database design and specificity of multimedia data [Byc, Wos11]. Over the years, certain standards have been Developing a high quality medical multimedia formulated for medical departments, endowed with modalities using digital technology (Ultrasonography, database will give the opportunity to compare medical CT, MR etc.) and peripheral devices like laser images, to retrieve similar cases and to see their printers. Digital Imaging and Communications in treatments, diagnosis. The radiology information Medicine (DICOM), is one such standard, which system (RIS) is considered the core system for the deals with imaging equipment, printers, picture electronic management of imaging departments. The archival and communication systems (PACS), etc. It electronic medical record (EMR) is the core also offers assorted functions such as film printing or informational system for patient management across CD burning, which are distinctly determined by the health-care system. Within a radiology service classes. [Ind,Ver16]. department, major functions of the RIS can include Medical images are usually fused, subject to high patient scheduling, resource management, inconsistency and composed of different minor examination performance tracking, examination structures. So there is a necessity for feature interpretation, results distribution, and procedure billing. The widespread adoption of picture archiving e) Automatic feature extraction and indexing using and communication systems (PACS) requires advanced tools. additional practice management workflow coordination, including the creation and distribution Content based retrieval was first introduced in the of images within the imaging department and early 1980 as a new tool and is the most popular throughout the imaging enterprise [McE13]. group of optimization technique. CBR uses visual content of an image as features to represent and index image to be searched from large scale image databases. It is the main motivation behind recent 2. Characteristics of Multimedia research in multimedia databases. Database Multimedia database management system (MMDBMS) is mainly used for the retrieval and storage of the multimedia data content. The development of the multimedia system depends on the process of inserting, indexing, querying and retrieving, etc. In recent years, many researchers have designed the multimedia data model, but these models have some drawbacks [GUO13]. A multimedia database is a collection of related multimedia data. Common multimedia data types that can be found in a multimedia database include the following: Text, Graphics: drawing, sketches, and illustrations, Images: color and black & white pictures, photographs, maps and paintings, Animation sequences: animated images or graphic objects, Video: a sequence of images (frames), Audio: Figure 1: Architecture of CBIR systems in medical generated from an aural recording device, Composite [Fat,Bal12] multimedia: a combination of two or more of the Semantic based search is defined as a type of above data types [Yu, Bra11]. searching technique that compares the original Multimedia databases thus should provide (1) multimedia data to a prototypical category. Compared content-based access, (2) knowledge discovery to content based retrieval, semantic based retrieval is methods, (3) scalability to large data volumes, (4) categorized as a high level features that implements scalability to high dimensionality of features, (5) user’s perception. Semantic query uses knowledge good runtime performance. Multimedia database about the domain of relations, nature of data, and management system support facilities for the constraints related to database elements. Element indexing, storage, retrieval and provides a suitable extracted from different modalities of a video, such as environment for using and managing multimedia data visual information, auditory information, and text in [Far, Nor, Yuz, Sai12]. the video frames are generated to model the semantic There are several differences in the processing of of the video. Keyword based retrieval is considered as multimedia data compared to traditional data which a traditional method to retrieve data using textual can be divided into five as follows: [Mar,Sub96] description (metadata). Metadata is defined as structured information describing characteristics that a) Format of multimedia data. assist users to identify digital content itself. Metadata b) Presentation of the output results. is the data or semantic information to classify the c) Size of multimedia data. d) Temporal characteristics of multimedia data. content, quality, condition and other characteristics of concept query [Fat, Bal10], [Ram,Chan11], [Sim, the data. [Ras, Haw8] Jom11]. With the increasing variety and decreasing cost of various types of sensors, there will be an increase in 3.Digital Imaging and Communications in the use of radically different media such as infrared, motion sensor information, text in assorted formats, Medicine optical sensor data, telemetric data of various sorts The Digital Imaging and Communications in (biological and satellite), transducers data, location Medicine (DICOM) standard was created by the data captured by GPS devices, spatial data, graphics National Electrical Manufacturers Association and animation data. [Kan,Rui7]. (NEMA) to aid the distribution and viewing of Kehua Guo and Shigeng Zhang developed A medical images, such as CT scans and ultrasound. Semantic Medical Multimedia Retrieval Approach New technologies such as Java should always be used Using Ontology Information Hiding. Their as complements of the de facto standard in medical architecture consists of semantic annotation, ontology imagine, DICOM. DICOM allows the interchange of representation, semantic multimedia storage, and images from different modalities, archives, and medical multimedia retrieval steps. [Keh, Shig13] workstations from different vendors. java technology can be used to build a storage system and to make this 2.1Query formulation by image content service accessible for different clients. However, this storage service should also incorporate DICOM Representation of images needs to discuss which services to store and access examination data from features are most useful for representing the contents DICOM workstations and DICOM modalities. of images and which approaches can effectively code DICOM is the universal standard for sharing medical the attributes of the images. Some of the Processes of imaging resources between heterogeneous and multi- Image Retrieval will include: vendor equipments (acquisition device, workstation, storage server, patient management system, etc.). a) The query image and database images are [Noo,Sam9] compared to retrieval of very similar images to query DICOM Service Class is defined as a group of image from the database. operations that a user wants to perform on data from a modality. Typical examples of Service Classes b) In radiology feature Extraction, generally used include Print Management Service Class that deals image features for content-based image retrieval were with printing images on film or paper printer, with color, shape and texture. If a user wants to perform a flexible film formats, Storage Service Class that query, three parameters have to be specified: 1) the implies “sending” images and Query/Retrieve Service location of the idle is containing the future query Class that deals with issues of “find”, “move” and image, 2) the system will give the query a number “get”. SOP (Service Object Pair) Classes. While that uniquely identities the group of fragments with “find” is used to query for images, “move” and “get” the same dimension (the “query index”), 3) the type are used to commence a transfer. Other classes of of the algorithm used in the query, Content Based service include Verification Service Class, Media Image Retrieval (CBIR). storage, Study content Notification, Print management, Patient management, Study c) Providing a sample of the kind of output is desired management, Result management, Modality and asking the system to retrieve further examples of Performed Procedure Step Management States and the same kind. Several alternative query formulation Structured reporting [Ind, Ver13]. approaches have been proposed: category browsing, DICOM enables the integration of scanners, servers, simple visual feature query, feature combination workstations, printers, and network hardware from query, localized feature query, query by sketch, user- multiple vendors into a picture archiving and communication system (PACS). dawned attribute query, object relationship query, The DICOM standard has a series of advantages: Generally, all medical equipments acquiring medical images support this standard and communicate among offer the history of a patients and all the medications them using it; DICOM can store besides the actual 2D that he took. image additional information, such as: the patient’s 3D position, physical size of the objects presents in the image, slice thickness, exposure parameters, and Table 2: Family History and Complaints others. These are used for a better later processing and interpretation. The DICOM files and messages support more than 2000 standardized attributes that Family History Complaints maintain patient’s medical data and images. Images are acquired and stored using parameters that are asthma Eye pain device independent. Likewise, DICOM images can be bleeding disorder sinus pain processed without taking into account the actual cancer fever device used in the acquisition process. drug addiction acne heart disease diarrhea 4.The proposed multimedia database hypertension laceration The database used for storing information about mental illness pain patients will include patient information, physical examination, neurologic facts, scanned images, strokes hoarseness intervention in patients and histology. This structure alcohol addiction infection of storing information will give the opportunity to group patients according to the disease, diagnosis and Another important entity in the database is the treatment. We will use DICOM for managing the physical examination that will include vital signs and medical images information. the physical examination records by the doctor Table1: Information about Patients Table 3: Physical Examinations Vital Signs Physical Examination Basic information Risk factors height weight change First name Diabetes weight anorexia Last name, High cholesterol body temperature heat or cold Date of birth Smoke respiratory rate fever Gender LDL under control systolic blood pressure insomnia Health insurance Allergies from penicillin blood glucose change in vision number heart rate polyuria Birthplace neurologic situation throat Address vertigo Family history also plays an important role in Another important information for the patient is the determining the diagnoses and the treatment for a neurologic situation. First of all, in the database certain disease. Also it is important to store in the should be saved information if the patient has database the complaints of the patients and the current headache, seizures, incoordination, significant past medication that he is attending so this database will history, head injury, tremors, numbness dizziness, weakness or difficulty swallowing. The neurologic situation will include also the evaluation of the mental Theoretical and Applied Information Technology, status, the evaluation of reflexes etc. The database 2013 with offer the opportunity to save the scale in which [Yu, Bra11] C. Yu, T. Brandenburg. Multimedia the patient opens the eyes, the patients give the Database Applications: Issues and Concerns for responses, etc. The most important thing is the classroom teaching. The International Journal of diagnosis in which the patient should be identified. Multimedia & Its Applications, 2011 All the diagnoses should have a unique code and the [Far, Nor, Yuz, Sai12] M. Farham, R. Nordin, L. description of the diagnosis, so when the doctor will Yuzarimi, Mohamed Saiful. Managing Multimedia register the patient, should also register the exact Data: A Temporal-Based Approach. International diagnosis of the patient. And in the end the database Journal of Multimedia and Ubiquitous Engineering, should register the interventions, histology of the 7, 2012 patient and the medications of the patient. [Mar, Sub96] S. Marcus, V. Subrahmania. Foundations of Multimedia Database Systems. ACM, 5. Conclusion 474-523, 1996 In this paper are described the steps for construction [Kan, Rui7] S. Kankanhalli, Y. Rui. Application the medical multimedia database in neurology Potential of Multimedia Information. IEEE, 2007 department in Mother Teresa Hospital. Developing [Fat,Bal12] Fathabad, Balafar. Content based image the multimedia database will give the possibility to retrieval for medical images. Technical and Physical analyze diagnosis, treatments and medical images to Problems of Engineering, 117-822, 2012 improve the process of identifying the diagnose of patients and the disease. Medical image retrieval for [Ram,Chan11] B. Ramamurthy, K. Chandran. diagnostic purposes is important because the Content Based Image Retrieval for Medical Images historical images of different patients in medical Using Canny Edge Detection Algorithm. centers have valuable information for the upcoming International Journal of Computer Applications, 2011 diagnosis with a system which retrieves similar cases, [Sim, Jom11] J.Simily, J. Jomy. Content Based Image make more accurate diagnosis and decide on Retrieval System for Malayalam Handwritten appropriate treatment. Characters. IEEE, 2011 [Ind, Ver16] S. Indrajit, C. Verma. DICOM, HL7 6. References and IHE: A basic primer on Healthcare Standards for [Ash,Man12] O. Ashish, S. Manpreet. Content Based Radiologists. Computers in radiology, 2016 Image Retrieval System for Medical Databases - [Keh, Shig13] G. Kehua, Zh. Shigeng. A Semantic Lucratively tested on Endoscopy, Dental and Skull Medical Multimedia Retrieval Approach Using Images. International Journal of Computer Science, Ontology Information Hiding. Computational and 9, 2012 Mathematical Methods in Medicine, 2-3, 2013 [Byc, Wos11] L. Byczkowska-Lipińska, A. Wosiak. [ Haw, Ras12] R. Rasli, S.Haw. Survey on Multimedia Database Techniques for Medical Diagnosis Processes Support, 2011 Optimizing Image, Video, and Audio Query Retrieval in Multimedia Databases. International Journal of [McE13] K. McEreny. Radiology Information Advanced Computer Science , 2, 229-236, 2012 Systems and Electronic Medical Records, 2013 [Noo,Sam9] A.Noor, M.Saman. Distributed Object [GUO13] C. Guo. Design and implementation of a Medical Imaging Model. IJCSI International Journal multimedia database application system. Journal of of Computer Science Issues, 2009