PURPOSE: The aim of this study was to provide an overview of Artificial intelligence in Forensic science with the aid of radio-diagnostic modalities. DATA SOURCES and SYNTHESIS: The data is gathered by searching the articles in varioussearch engines which have been published between January 2010 to December 2020. A total of 20 studies were found eligible after following inclusion and exclusion criteria described in the below article. Prisma Guidelines and Prisma Flowchart was followed. CONCLUSION: Artificial intelligence (AI) is a technology that involves computerised algorithms to dichotomize complex data. AI is widely used in diagnostic imaging for detection and quantification of a clinical condition. This systematic review aimed to explain the role of AI with diagnostic imaging modality of radiology in forensic. AI technology is now widely used for age and sex estimation. Most of the AI models are based on machine learning (ML) programs, artificial neural network(ANN) and convolutional neural network (CNN). The results of the studies are promising, providing great accuracy and decision making. These different AI based models will be act as identification tools in mass disasters cases, medicolegal cases. Further improvement in AI programs and diagnostic tool is needed for better accuracy and specificity in Forensic investigations.
With the advancement in technology the application of artificial intelligence (AI) in diagnostic imaging is in the phase of evolution. AI has provided the accuracy and specificity in diagnosing several diseases and tumors. Investigations for the determination of small radiographic abnormality with the help of computer-aided program have shown excellent precision and sensitivity. The modalities of diagnostic imaging are gaining importance in terms of Forensic Science, anthropology, archaeology and Forensic Medicine. The combine application of Forensic and Diagnostic imaging for the purpose of Identification, Mass disaster, Medico-legal investigations is well known. One of the key benefits is its nature that is non-destructive in nature as compare to other tools of Forensic. The radiological study is meant both for live as well as dead cases. Radiographic estimation is considered more feasible as they are simple and consume less time. Role of radiology in forensic is wide from identification of human, injuries, sex and stature estimation from bones, cause of death of the victim and location of any foreign material in the body and many more (Fig. 1). CAD technologies (Computer-aided detection/diagnosis) in the 1960s were first used in chest x-ray and mammography applications [1]are now common to radiologists. However, developments in algorithm creation, together with the ease of access to computing tools, enable Artificial Intelligence chosen to implement at a higher functional level in radiological decision-making [2]. The given table 1 includes use of AI in diagnosis of medical imaging in terms of collecting the data and its features computational techniques and imaging applications.
c i s n e r o F n i y g o l o i d a R
intervention, a homicide or suicide, genocide, Mass causalities, sudden unexpected death in infants or sudden infant death syndrome. 1.1.
The suggested Artificial Intelligence automation levels in diagnostic radiology varying from 0 to 4 (Fig. 2) in a study generated by a discussion of the Ministry of Health, Welfare, and Labour [3]. Level 0 refers to image pre-processing and does not provide computer-assisted diagnosis. Level 0 is split into two categories: image pre-processing without AI (level 0) and image pre-processing with AI (level 0+). In Recent Times, recent synthetic imaging, which is picture pre-processing with AI (level 0+), has advanced quickly. Tier 1 is the computer-assisted diagnosis of a particular form of visual identification, like lung nodule identification on a chest CT scan. Grade 2 requires complex pattern identification in a number of areas, for example pneumonia lesions, liver mass lesions, and a lung nodule. Stage 3 medical imaging skills are similar to human being. Level 4 applies to medical imaging capacities that are comparable to those of humans.
The radiologist's functions are complex, including caretaker for a valued facility, specialist diagnostician, and patient care protector. AI is attempting to question the diagnostic position. Developments in AI technologies and imaging have increased analysis on the radiologist's diagnostician function, which involves dual methods: image analysis tracked through analysis of results. This necessitates the capacity to visually interpret a picture as well as the cognitive abilities to use pattern detection to distinguish between regular and abnormal [4]. This is difficult since human understanding of photos often overlooks observations and attracts interpretation errors. Clearly, radiologist negligence leads to missed diagnoses and complications in treatment, which may contribute to worse medical care[5]. 1.3.
ML, being as a branch of AI, also known as standard AI, was first used in medical imaging in the 1980s. Users initially specify precise imaging factors and functions based on professional experience[6-8]. Shapes, zones, and histograms of image pixels from areas of concern (i.e., tumour regions) may be removed. Typically, with a specified number of accessible records entries, a portion of them is applied for preparation and the remainder for research. To understand the functionality, a particular ML algorithm is chosen for preparation. CNN (Convolutional neural networks), PCA (Principal component analysis), SVM (support vector machines), and other algorithms are instances[912]. The qualified algorithm is then expected to recognize the features and label the picture for a given testing image[13-16]. Some of the issues with ML is that clients must pick the features that determine the class of the picture. However, certain contributing variables could be ignored[17-18]. For example, in order to diagnose a lung tumour, the consumer would segment the tumour area as structure features. The accuracy of manual function selection has always been a concern owing to patient and consumer variety. Deep learning, on the other hand, does not necessitate specific user feedback of the functions. Deep learning, as the name means, learns from a much greater volume of info. It employs deep artificial neural network models[19]. Deep learning utilizes several levels to derive higher-level functionality from raw image input. It aids in disentangling abstractions and identifying features that can increase efficiency. Deep learning was first recommended decades earlier. Only in the last decade has the implementation of deep learning been possible due to the tremendous number of medical images generated and advances in hardware growth, such as GPU (graphics processing units). Though, as ML gains significance and value on a daily basis, even GPU has been quite deficient. To fix this problem, Google built an accelerator integrated (AI) circuit that will be applied for its TensorFlow AI framework — TPU (tensor processing unit). TPU is primarily developed for neural network ML, although it may also be used in medical imaging studies.
The incredible advancement in AI and ML (machine learning) can prove a revolution in providing consistent information in conclusion making. Hence this systematic review aimed to account on application of AI of diagnostic radiology in Forensic science.
The systematic review is conducted by following the guidelines of PRISMA (preferred reporting items for systematic reviews). The data for the present article is gathered from the databases which are available free and do not require any institutional access. The article published in PubMed, Google Scholar are mainly collected and also from the other research engines like Open Science Directory, Free Medical Journals, Directory of Open Access Journals and OpenMD.com that have been published between (January, 2010 to December, 2020). The strategy of search mainly focusses on the articles that use the keywords like artificial intelligence, machine learning, diagnostic imaging, radiology, forensic radiology, forensic, forensic medicine. 2.2.
The articles were selected on the basis of title and after that a preliminary search was done on referring the abstract of the articles. On first stage of selection 189 number of articles were selected. 26 number of articles were removed on the basis of duplication by using Mendeley software and 77 articles related to conference article, review article irrelevant to our study title were also removed. Further 38 records were removed on the basis of inclusion and exclusion criteria. During screening of articles 13 records were removed after studying the abstract and full article.
A protocol was not registered and ethical reviews are not required for this review article. For selection and extraction, the discussion between first two authors were done and whenever a disagreement was there the last author acted as an arbitrator.
• • • •
The articles must focus on the Forensic benefits from modalities of diagnostic Imaging. The article must use the technology like AI and ML.
The articles without full- text available.
There should be an outcome of the study without any disagreement between authors.
• • •
Articles available in language other than English Articles without full texts.
Articles that are not focussed on AI and radiology imaging.
After applying above criteria, the number of articles for our study further reduced to 20. The articles were studied thoroughly and a PRISMA flow chart is used (Fig. 3) for qualitative extraction. These articles were thoroughly studied for quantified assessment with the reference of year of publication to know the current trend of AI in forensic applications.
Identification
Screening Included
Records removed before screening: Duplicate records removed (n =26) Records removed of conference, review paper
(n =77)
A total of 20 studies was analysed for the review paper on analysis the articles shows that majority of the studies were conducted in last ten years. The articles that were taken in this review article mainly focus on the role of AI, ML technology using diagnostic modalities of Diagnostic Radiology for benefit of Forensic. Most of these studies uses CNN (convolutional neural networks), NN(neural networks). Most of the studies uses AI approach for assessment of bone age, facial reconstruction, injury identification and sex estimation (Fig. 4). CT scan and radiographs were the most common choice of modalities used. MRI was also used in three studies. The data included in the studies for the AI based models were highly standardized, so there is no effect on the final output.
Role of AI with Diagnostic Modality in Forensic 12 10 8 6 4 2 0
Age Sex estimati determin Injury on ation uction
Facial reconstr Identific ation
Brain mapping
Fracture s Series 1 10 5 1 1 2 1 1
Forensic investigations mainly deal with the victim identifications (age, sex, stature, race) [20]. In case of mass casualties like earthquake, floods the gender identification and age estimation are of great importance [21-22]. Despite of very limited studies confirming the use of artificial intelligence in forensic medicine, these studies have shown and proved the efficiency in predicting age, sex and localization of fractures.
In this systematic review paper, we have analysed the articles that have use AI based technology in forensic perspective. One of the major advantages of AI based algorithms is that they can help in identification from previous radiographs taken. The AI based identifications are superior in terms of human eyes and free from biasness [23]. 4.1.
The age estimation in radiology department is generally carried out with the help of hand radiographs with other long bones radiograph [24]. To overcome the subjectivity of the radiologist and to have more precise result several ML and AI programs are developed. Chronological age estimation is necessary in terms of medicolegal cases and for court room trials. In past few years several AI based models are been made to estimate the age [25-26]. Darko Stern et al [27] uses MRI (magnetic resonance imaging) for age estimation in 2016 and 2019, the study conducted in 2019 uses multifactorial method based on MRI and able to estimate age up to 25 years where as earlier it was 19 years only in many similar studies. A study by Christian Booz [28] in 2020 conducted to investigate the accuracy of AI based model to Greulich- Pyle method 514 radiographs were analysed the correlation between AI and reference was significantly higher (r=0.99) as compare to other method (r=0.90) and mean reading time was also reduced to 87% (table 2).
Another author Daniela Giordano [29] examined 360 hand radiographs, 180 of male and female each. For age range of 0-6 years by a modified version of the Tanner and Whitehouse with collaborating Markov models. The success rate in age estimation was high with mean error rate of 0.41 ± 0.33 years, a tool was also released to speed up the practice. Jeong Rye Kim et al [30] used a method based on combination of Greulich-Pyle method and deep learning program to make an AI software for bone age evaluation. 200 radiographs of left hand were taken with the age range of 3-17 years. Three combinations for estimation were use first- software only, second- computer assisted with two radiologist and third- Greulich-Pyle atlas with two radiologists. The results showed increase in concordance with automatic software and also reduce reading time. Hsiu-Hsia Lin et al [31] method was based on phalangeal image of hand radiograph. Segmentation method was used. Bone age assessment was done using Fuzzy Neural Network, the system included two parts the first part help in adjustment of feature weights in accordance of four stages defined in the article to specify development of epiphyses and metaphysis. Second step depends upon the result of first for assessing bone age. The result of the study revealed the use of FNN for better quantitative accuracy. Hyunkwang Lee et al in 2017 [32]used an CNN (convolutional neural network) based model for bone age assessment. Using radiographs and gave accuracy of 61.4% in females and 57.32% in males. Initial results were seemed to be very fruitful but at the end were not very promising. David B. Larson et al [33] also used CNN model for age assessment in children using 14306 radiographs. The performance was measured in terms of root mean square and mean absolute difference which was 0.63 and 0.50 years respectively. This model gave a promise of high accuracy over existing automated models. Another study done by Jang Hung Lee et al. [34] for age estimation used deep learning-based technique. Formulated as a regression formula where radiographs of hand were used as an input material and estimated age as the output. The name of the tool used was Caffe was demonstrated. The studies done for age estimation shows that it can be time and cost effective and also eliminates the use of atlas and more user friendly.
Skeletal bones play a key role in gender estimation [35]. Gender identification is very crucial whenever skeletal remains are found for medicolegal and court room purpose. The skeleton bones play the major role in sex determination and with help of radiology modalities the sex identification become easier. Skull and pelvis play a vital role in gender identification, the shape and size of the bones are different in male and females.
James Bewes et al [36] in 2019 used artificial neural network for gender determination. This study was conducted on 900 skulls virtually constructed using CT scans. The ANN showed the accuracy of 95% and rapid to use. Hongjuan Gao et al. [37] with the help of CT scans 78 landmarks and MKDSIF-FCM model, skull of Chinese ethnic group was used for gender identification and the results were 98% in females and whereas in males it was 93%. The result was of high accuracy and good stability. Wen Yang et al. [38] proposed a BPNN (backpropagation neural network) which was an improved version by using skull. A total of 267 skulls from whole skull database of CT scan were studied out of which 153 were of females. Six parameters were used as input to get the desired result, for improving generalization ability Adaboost algorithm were used. The accuracy rate was 96.76% with 0.01 mean square error. In year 2019 another study based on CT scan for sex determination was done by Angelique Franchi et al. [39]. It was extraction of key points based on algorithms; 83 scans of living individual were taken from database VISCERAL which is public. A Probabilistic Sex Diagnosis tool was used on the landmarks which gave accuracy of 62%. The main limiting factor in their study was population size. Some models are very efficient in terms of accuracy and sensitivity like a study conducted by Mumtaz Kaloi et al. [40] gave 98% accuracy in children using CNN. Left hand radiograph of age range one month to 18 years were used. According to author this kind of technique was first of its nature.
Identification and recognition of individual from the skull obtained is earlier done by the help of wax, clay, and plasticine modelling on to the skull replica to sculpt the face which is a lengthy and time-consuming process. With recent advancement in science and technology 3-D facial reconstruction can be done which is rapid and more flexible computer-based technology. In the superimposition method, a skeletonized skull is compared to an antemortem picture of the victim, and the thickness of soft tissues or the anatomic structure is analysed. The precision of three-dimensional (3D) reconstructed images is important for the superimposition method. A virtual copy of skull is produced. But for unknown skull the major problem is of measurement of tissues depth. In 20th century different anatomists collected and studied tissue depths for different races and ethnic groups [41]. Cone-beam CT scanning has the advantage of acquiring images of subjects in upright positions compare to conventional CT. 2-D facial reconstruction generally based on ante mortem photographs of the victim. But with recent time skull radiographs are also use for construction. 3-D facial reconstruction based on use of computer images. A study conducted by Ayaka Sakuma et al. [42] in 2010 used a mobile CT scanner unit. The CT scan of 2mm slices were taken the data on the DICOM software were imported and constructed into 3D polygon data by using Micro AVS and VIRTUAL place Lexus. The findings of the study assures that CT images using simple computer learning program can be used for superimpositions. Their results show high reproducibility of thickness and also suggest for certain landmarks refinement.
Identification of individual is of great importancein forensic context and medicolegal process. Antemortem and post-mortem records play a vital role in terms of personal identification. The study done by Ayaka Sakuma et al for facial reconstruction can also be used for personal identification with superimposition technique with PMCT for unidentified cases. And suggested the use of cone beam CT scan as a more precise tool. O. Gomez et al. [43] in 2020 used frontal sinuses for identification method based on an algorithm model. Frontal sinuses are used for identification because of their high identification and individuality. The result of this study showed accuracy rate of more than 80%. In this study 50 samples were of x-ray images and another 50 were of CT. the work was formulated as 2D – 3D image registration problem two algorithm model DE (Differential Evolution) and MVMO (Mean- variance mapping optimization) were compared and the best MVMO-SH was applied for identification.
Jack Garland et al. [44] and Jakob Heimer et al. [45] used deep learning methods for identification of injuries and fractures in skull respectively. J. Garland suggested the technique involving PMCT and AI for head imaging, that involved construction of CNN program with Keras and was trained against the training data before the testing dataset. PMCT images of head at the level of frontal sinus were taken, 25 cases were of fatal head injury and 25 with non- head injury control sample. The accuracy was between 70% to 92% but there was difficulty in recognition of haemorrhage of subarachnoid. The result obtained gave a potential application in screening of injuries. Method used by Jakob et al. for fracture identification with help of DNN (deep neural network) depends upon the skull fractures on curved maximum intensity projections of 75 cases. The author suggested the use of pre-scanning PMCT data with 0.75 classification threshold can be applied. The author expects the role of the deep learning program in post mortem radiology will increase and gives more resource efficient information.
MRI diagnostic tool with application of AI in it will help in studying in neurodegenerative changes in brain suggested by, Shane O’Sullivan et al. [46] and can help in brain mapping. Neurodegenerative diseases are associated with loss of volume in the brain. The author described the protocol and limitation for study and management of large data. The tool developed a bridge between virtopsy and histology to overcome the gap between neuropathology and virtual reality. The author concluded that presently virtual autopsy cannot substitute conventional method of autopsy, both methods should be used parallel to one another keeping other factors like tissue, organ involved in consideration.
It allows non-invasive or minimum invasion for several findings that may not be visible in routine autopsy. Digitization of bodies is possible through imaging. Each technique has its advantage and disadvantage. From the above analysis it can be determined that conventional image processing methods is slowly replaced by methods that acquire data from multiple image modalities and with the help of different image processing techniques like enhancement, segmentation and image restoration with advance systems of machine learning and artificial intelligence are increasing the accuracy and sensitivity of the test. Also reducing the examination time and reporting time which will help the forensic experts and improve diagnosis. The human biasness which can alter any opinion will also be overcome by the incorporation of these algorithms. AI is playing an important part in medical imaging science. It altered how people processed the massive number of photographs. There are also problems to iron out before AI may influence clinical procedures. AI would undoubtedly have an effect on radiology, and it will do so faster than in other medical fields. Radiologists should take the lead in this upcoming transition. Cardiac imaging was an early adopter of AI strategies in image analysis, standardized documentation, and clinical decision support systems, and it has the ability to continue to set the benchmark for the remainder of diagnostic imaging and the practice of medicine. Although AI may be used to forecast possible results, the automated production of management judgments tends to be less optimal than a human neural network dealing with each particular patient's health, personal, environmental, and social aspects. In the near future, therefore, for imaging professionals and clinicians, AI's value-adding ability is more likely to be as intelligent precision medicine instruments.
Thecapability of machine and human are different the aim offorensic and medical health care can be strength by using both. Machine never get tired so some work can be more suitable for computers, they can produce repeatable and consistent in the results. Have better analysing power at high speed. Whereas human can synthesize disparate points of data. Human can extract valuable information more precisely using these techniques. As describe in this review article variety of modalities are available. Different researches have demonstrated that imaging techniques can be superior to routine procedures so it should be applied with combination of earlier and new methods. For better implication of these new modalities more forensic examiners should be trained and motivate to use it in daily routine caseworks. For better improvement of AI in radiology the features like data augmentation, algorithm and selection combination should be enhanced. AI for identification and age estimation will be a key application in the field of forensic science. The application of radiology with AI programs for identification of fractures, injury, calculation of organs weight, cause of death, brain mapping will help in solving many cases of forensic interest. Although all these AI based models are promising in nature but lack of real-life experience and limited studies is a major limitation of the present review. 7. References
Lee, June-Goo, et al. "Deep learning in medical imaging: general overview." Korean journal of radiology 18.4 (2017): 570-584.
Pesapane, Filippo, Marina Codari, and Francesco Sardanelli. "Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine." European radiology experimental 2.1 (2018): 1-10.
Mohamed, Afifah, et al. "Multimodality imaging demonstrates reduced right-ventricular function independent of pulmonary physiology in moderately preterm-born adults." Cardiovascular Imaging 13.9 (2020): 2046-2048.
Krupinski, Elizabeth A. "The future of image perception in radiology: synergy between humans and computers." Academic radiology 10.1 (2003): 1-3.
George, Gina, and Anisha M. Lal. "A Personalized Approach to Course Recommendation in Higher Education." International Journal on Semantic Web and Information Systems (IJSWIS) 17.2 (2021): 100-114.
Dagi, T. Forcht, Fred G. Barker, and Jacob Glass. "Machine Learning and Artificial Intelligence in Neurosurgery: Status, Prospects, and Challenges." (2021): 133-142.
Trayanova, Natalia A., Dan M. Popescu, and Julie K. Shade. "Machine learning in arrhythmia and electrophysiology." Circulation Research 128.4 (2021): 544-566.
Wichmann, Julian L., Martin J. Willemink, and Carlo N. De Cecco. "Artificial intelligence and machine learning in radiology: current state and considerations for routine clinical implementation." Investigative Radiology 55.9 (2020): 619-627. [9] Chowdhury, Aritra, et al. "Image driven machine learning methods for microstructure recognition." Computational Materials Science 123 (2016): 176-187. [10] Jozdani, Shahab Eddin, Brian Alan Johnson, and Dongmei Chen. "Comparing deep neural networks, ensemble classifiers, and support vector machine algorithms for object-based urban land use/land cover classification." Remote Sensing 11.14 (2019): 1713. [11] Rahman, Shelia, Tanusree Sharma, and Mufti Mahmud. "Improving alcoholism diagnosis: comparing instance-based classifiers against neural networks for classifying EEG signal." International Conference on Brain Informatics. Springer, Cham, 2020. [12] Mani, Nag, Melody Moh, and Teng-Sheng Moh. "Defending deep learning models against adversarial attacks." International Journal of Software Science and Computational Intelligence (IJSSCI) 13.1 (2021): 72-89. [13] Brunelli, Roberto, and Tomaso Poggio. "Face recognition: Features versus templates." IEEE transactions on pattern analysis and machine intelligence 15.10 (1993): 1042-1052. [14] Matrone, Giulia, et al. "The delay multiply and sum beamforming algorithm in ultrasound Bmode medical imaging." IEEE transactions on medical imaging 34.4 (2014): 940-949. [15] Kachelrieß, Marc, and Willi A. Kalender. "Presampling, algorithm factors, and noise: Considerations for CT in particular and for medical imaging in general." Medical physics 32.5 (2005): 1321-1334. [16] Kaddour, Sidi Mohammed, and Mohamed Lehsaini. "Electricity Consumption Data Analysis Using Various Outlier Detection Methods." International Journal of Software Science and Computational Intelligence (IJSSCI) 13.3 (2021): 12-27. [17] Hosny, Ahmed, et al. "Artificial intelligence in radiology." Nature Reviews Cancer 18.8 (2018): 500-510. [18] Pesapane, Filippo, Marina Codari, and Francesco Sardanelli. "Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine." European radiology experimental 2.1 (2018): 1-10. [19] Bouarara, Hadj Ahmed. "Recurrent Neural Network (RNN) to Analyse Mental Behaviour in Social Media." International Journal of Software Science and Computational Intelligence (IJSSCI) 13.3 (2021): 1-11. [20] Simmons, Tal, and William D. Haglund. "Anthropology in a forensic context." Forensic
Archaeology. Routledge, 2005. 173-190. [21] Ayoub, Fouad, et al. "Correlation of oral, genetic, and radiological parameters involved in human identification in forensic dentistry." Journal of International Oral Health 8.6 (2016): 725. [22] Sledzik, Paul S., and William C. Rodriguez. "Damnum fatale: the taphonomic fate of human remains in mass disasters." Advances in forensic taphonomy: method, theory, and archaeological perspectives (2002): 321-330. [23] Hu, Chuili. "Implementation of Online Guiding System Based on VR and Face Recognition Algorithms." 2021 5th International Conference on Computing Methodologies and Communication (ICCMC). IEEE, 2021. [24] Petrovečki, Vedrana, et al. "Prediction of stature based on radiographic measurements of cadaver long bones: a study of the Croatian population." Journal of forensic sciences 52.3 (2007): 547-552. [25] Chen, Ke, et al. "Cumulative attribute space for age and crowd density estimation." Proceedings of the IEEE conference on computer vision and pattern recognition 2013. [26] Westerberg, Erik. "AI-based Age Estimation using X-ray Hand Images: A comparison of Object
Detection and Deep Learning models." (2020). [27] Štern, Darko, Christian Payer, and Martin Urschler. "Automated age estimation from MRI volumes of the hand." Medical image analysis 58 (2019): 101538. https://doi.org/10.1016/j.media.2019.101538. [28] Booz, Christian, et al. "Artificial intelligence in bone age assessment: accuracy and efficiency of a novel fully automated algorithm compared to the Greulich-Pyle method." European radiology experimental 4.1 (2020): 1-8.https://doi.org/10.1186/s41747-019-0139-9.