Each day a vast amount of unstructured content is generated in the biomedical domain from various sources such as clinical notes, research articles and medical reports. Such content contain a suficient amount of eficient and meaningful information that needs to be converted into actionable knowledge for secondary use. However, accessing precise biomedical content is quite challenging because of content heterogeneity, missing and imprecise metadata and unavailability of associated semantic tags required for search engine optimization. We have introduced a socio-technical semantic annotation optimization approach that enhance the semantic search of biomedical contents. The proposed approach consist of layered architecture. At First layer (Preliminary Semantic Enrichment), it annotates the biomedical contents with the ontological concepts from NCBO BioPortal. With the growing biomedical information, the suggested semantic annotations from NCBO Bioportal are not always correct. Therefore, in the second layer (Optimizing the Enriched Semantic Information), we introduce a knowledge sharing scheme through which authors/users could request for recommendations from other users to optimize the semantic enrichment process. To guage the credibility of the the human recommended, our systems records the recommender confidence score, collects community voting against previous recommendations, stores percentage of correctly suggested annotation and translates that into an index to later connect right users to get suggestions to optimize the semantic enrichment of biomedical contents. At the preliminary layer of annotation from NCBO, we analyzed the n-gram strategy for biomedical word boundary identification. We have found that NCBO recognizes biomedical terms for n-gram-1 more than for n-gram-2 to n-gram-5. Similarly, a statistical measure conducted on significant features using the Wilson score and data normalization. In contrast, the proposed methodology achieves an suitable accuracy of ≈90% for the semantic optimization approach.
tal unstructured textual content has been generated in Over the last few decades, a huge volume of the digi- require the metadata to properly index contents in a context-aware fashion for the precise search of biomedibiomedical research and practice, including various con- cal literature and to foster secondary activities such as tent types such as scientific papers, medical reports, and physician notes. This explosive growth in the biomedical domain has introduced several access-level challenges for researchers and practitioners. These valuable information are available in the web contents but still opaque to information retrieval and knowledge extraction search (A. Bisram); https://bukharilab.org (N. Iqbal); https://bukharilab.org (F. Keshtkar); https://bukharilab.org (S. A. C. Bukhari)
0000-0001-6374-0397 (A. Abbas); 0000-0002-6517-5261
metadata (semantic annotations) [
The main components of the semantic annotation
process are ontologies which are sets of machine-readable
controlled vocabularies that provide the “explicit
specifimantic annotators are designed to facilitate
tagging/annotating the related ontology concepts with pre-defined
terminologies in a manual, automatic, or hybrid way [
Recommendation annotation improvement. As soon authors receive a reSubsequently at initial level semantic annotation, the sponse from the crowd, they are notified, and all suggesauthor is enabled to approach and get recommendation tions start the display with the option to accept or reject. from peer review for a correct and high quality anno- Here the authors decide to choose a particular suggestation through seeking help module Figure 1(b). The tion based on social indexing. Our system calculates the authors are required to select the biomedical term from social index and displays each suggestion based on its the preliminary annotation interface for correct annota- index score in descending order. To gauge the credibility tion by peer review. Additionally the author is facilitated of the human recommender, our systems record the recwith an interface to smoothly query with available op- ommender confidence score, collect community voting tions such as a drop down menu of recommended queries against previous recommendations, store the percentage for an author. Similarly author can explain their query of correctly suggested annotation, and translate that into and provide evidence and links to better convey their an index to later connect the right users to get suggesquery to the expert or peer review. Finally when the tions to optimize the semantic enrichment of biomedical author submit their query, it is posted on “Semantically contents. All the process information is stored in the Knowledge Cafe” forum for peer review response and a backend knowledge base. notification is send to the community users as shown in Consider an author is required to find correct ontology Figure 1(c). annotations from peer review for the biomedical term “worsening shortness of breath” as shown in Figure 2. The author posts the query on a “Semantically Knowledge Cafe” forum such as “Which Ontology should I use for medical content ‘worsening shortness of breath’”? and receives replies from fellow users or experts . We categorized users who replied as expert users as with “No of Reply-post” and suggested the correct annotation for a required biomedical content as “Expert Annotation” see (0.458, 0.381, 0.518 ). Finally, ( self confidence and author credibility score for each expert that suggested annotation as “aggregate score” of ) function is applied on the aggregate score to obtain the maximum and each expert suggested the annotation as (“RCD”,”UP- score earned by each expert annotation which is 0.518.
their confidence score which they recorded as of (4,6 Eventually, the high proficient and ranking annotation is recommended to the author as “NCIT” and “Replyand 7) out of a scale from 1 to 10. The community user- post=3” for the biomedical content “worsening shortness s/crowd at ’Semantically Knowledge Cafe’ can observe the suggested recommendation and record their up and of breath”, see Figure 2. The same process is applied for another biomedical content, “Acute Flaccid Myelitis”, yet down voting about a particular suggestions. From users the scenario or query can be changed. , we recorded upvotes (9,10,11) and downvotes (9,8,7) to the expert recommended annotation. Whenever the author accepts recommended annotation from experts , a credibility score is recorded. We used Wilson score conifdence interval for a Bernoulli parameter to normalize and aggregate the recorded scores, see Equ. (1).
= Where, ( +̂ 2 22 ± [ ̂ (1 − ̂)+ 42 ])
2 2 √
2 (1 + 2 ) =1 1.
= ( − ())/(() − ()) ∗ Where is the ℎ normalized value in the dataset. Where is the ℎ value in the dataset e.g the user confidence score. Similarly, ()
is the minimum value in the dataset, e.g the minimum value between 1 and 10 is 1, so the () = 1 and ()
is the maximum value in the dataset, e.g the maximum value between 1 and 10 is 10, so the () = 10
. Consequently, a mean ̂= =0 applied on Wilson score, normalize the (1) (2) (4)
30 people participated in our proposed model. We recruited participants by via social media request and asking them to participate in the study. Further, We categorize participants as the most graduate-level students with computer and biological science backgrounds.
Accordingly, We considered a set of 30 articles from pubmed.org [13] and randomly distributed it to the participants. Similarly, We provided a user manual of systems along with a pre-recorded video about system usage. Afterward, We asked each participant to generate queries on the “Semantically Knowledge Cafe” about the biomedical content annotation about which they like to seek social help. Collectively, our participants post 140 questions to the system. All the participants have also (3) recorded their confidence scores between 1 and 10 from the suggestions they received as a satisfaction score. Consequently, Our system recorded 421 responses against 140 questions from expert users. Similarly, 2929 and 3149 up and down votes were also recorded against the suggestions annotations. Table 1 illustrates participants and their responses. cal word or concept boundary detection process. A set of 30 pubmed.org [13] articles is processed at the initial level, 3.2. Performance Measurement: consequently obtaining annotated biomedical terms in Knowledge-sharing based Semantic the range of n-gram-5. Subsequently scrutinized, we Enrichment Optimization found the proposed annotation system identifies biomedical terms of n-gram-1 quantitatively higher than n-gram- A domain expert from the academia at the professor level 2 to n-gram-5. A very few biomedical terms identify of is engaged to evaluate these results manually based on n-gram-5 see Figure 3. However, the biomedical terms their knowledge and experience. After that, calculate of − > 1 deliver extra meaningful and coher- the system level accuracy for semantic annotation beent information to the user contextually. For example, fore socio-techinical semantic annotation optimization “blood pressure is high”, “he has coronary artery disease”, and after socio-technical semantic annotation optimizaand “liver function test is normal” are more meaningful tion approach Figure 4. A document’s level accuracy terms as compared to a single term such as “pressure”, is recorded with-out a socio-technical and with a socio“blood”, “coronary”, and others. As the − word technical approach. The Figure 4 on X-axis represents size increases, the accuracy of composite terms decreases, the number of 30 documents processed. In contrast, the as shown in Figure 3. Because the proposed system em- Y-axis at the left represents the level of accuracy with-out ploys exact word matching to the terminology (Bioportal) a socio-technical approach, and the Y-axis at the right approach, the primary characteristic of the exact word represents the level of accuracy with a socio-technical matching approach is that a single word matches more approach. Consequently, scrutinizing the results of a accurately than a combined or compound word. system with a socio-technical approach performed better than with-out a socio-technical at document level. Until high accuracy of 90% has been gained by nine documents and lower accuracy of 87% has gained by three documents and maximum number of documents has gained accuracy between 87% and 90% with socio-technical approach see
interface with possible option is available to the expert for reply post. Subsequently reply by the expert to the author post with precise annotation, same while other community users are enable to give up-vote or downvote to the expert reply post shown in Figure 5(e). Finally a high quality annotation recommendation notification is generated to the author by aggregating wilson score, and expert self confidence score as shown in Figure 5(f).
Whenever the author click on ”New Recommendation” link, a high quality expert recommended annotation is pop-up see Figure 5(g). Now here author is allow either accept the recommended annotation or reject, while accepting annotation a credibility score is recorded to the author profile between 1-5, vise versa no score is recorded to the author profile. Similarly by accepting recommended annotation, initial annotation for specific terminology is replaced by recommended one and thus annotation optimization process is completed Figure 5(g).
Initially the author is enable to import or write the biomedical content in the editor and click on annotation button to get preliminary annotation see Figure 5. The underline word with green color presented annotated term, subsequently when author select any term the underline color change to pink and ”Need Help” option is appeared on left side panel to the author see Figure 5(a). After click 4. Conclusion on ”Need Help” an interface is open, where author can write there query from expert for recommended anno- This research advances state-of-the-art biomedical setation to explicit terminology Figure 5(b). Additionally mantic research and systems, enabling various biomedauthor is facilitated with primary options for quick query. ical users to author context-aware content with no When the author click on ” ” button, the query is prior technical skills needed. An out-of-the-box socioposted on the ”Semantically Knowledge Cafe” forum and technical semantic annotation optimization approach is a new post notification received to the community users presented to automate the semantic enrichment mecha as shown in Figure 5(c). Whenever users click on nism and discover the precise semantic annotation while ”Semantically Knowledge Cafe” , the new posted query is keeping the original content creator in the loop. The appeared as shown in Figure 5(d). Now that if user know end user is facilitated with an authoring interface similar the answer of the posted query, he/she is enable to click to the MS Word editor type/write biomedical contents. on ” ” button to reply author post/query as shown To cater the preliminary semantic annotation or enrichin Figure 5(d). Subsequently reply to the post by user ment at the content level, we utilized Bioportal endpoint with record self confidence score, now the role of this APIs and automated the configuration process for auuser is consider as a domain expert. Similarly, a smooth
thors. Similarly, the semantic annotation optimization approach is designed where the author can post their query for optimized annotation recommendation. In our future work, we plan to expand the backend knowledge graph and apply the neural graph networks. The semantic annotation optimization system is available at https://gosemantically.com. 2022 IEEE 16th International Conference on Semantic Computing (ICSC), IEEE, 2022, pp. 175–176. [13] PubMed, National Center for Biotechnology In- formation, 2022. https://pubmed.ncbi.nlm.nih.gov/.