As a novel contribution to the problem of semantifying biological assays, in this paper, we propose a neural-network-based approach to automatically semantify, thereby structure, unstructured bioassay text descriptions. Experimental evaluations, to this end, show promise as the neural-based semanti cation signi cantly outperforms a naive frequencybased baseline approach. Speci cally, the neural method attains 72% F 1 versus 47% F 1 from the frequency-based method. The work in this paper aligns with the present cutting-edge trend of the scholarly knowledge digitalization impetus which aim to convert the long-standing document-based format of scholarly content into knowledge graphs (KG). To this end, our selected data domain of bioassays are a prime candidate for structuring into KGs.
Biological assays are de ned as standard biochemical test procedures used to
determine the concentration or potency of a stimulus (physical, chemical, or
biological) by its e ect on living cells or tissues [
In the context of the current Covid-19 pandemic, bioassays are critical, for example, for vaccine development. They reveal the functional and biologically relevant immunological responses that correlate with vaccine e cacy. However, massive volumes of bioassays are being produced and researchers are inundated with this information. Apart from their sheer quantity, bioassay diversity presents enormous challenges to organizing, standardizing, and integrating the data with the goal to maximize their scienti c and ultimately their public health impact as the screening results are carried forward into drug development programs. ? Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Against this broad societal application setting, we present a solution as a
step in the easier knowledge acquisition of bioassays for researchers: the
neuralbased automated structuring of unstructured, non-standardized bioassays based
on the standardized BioAssay Ontology (BAO) [
For automated bioassay semanti cation, we carry out the supervised machine
learning of semantic statements (i.e., subject-predicate-object triples) based on
the BioAssay Ontology (BAO) [
Our dataset for learning comprises an expert manually annotated collection of 983
semanti ed bioasssays [
Problem Formulation The dataset can be formalized as follows. Let b be a bioassay from the assays dataset B. Each bi is annotated with an annotation sequence asi such that asi 2 S, where S is a set of all possible semantic statements seen in the training dataset. Speci cally, asi = fs1; s2; s3; :::; skg, such that sx is a semantic statement 2 S; asi has k di erent statements. In general, annotation sequences are of varying lengths. The dataset we use has jSj = 1756 unique statements (after ltering for non-informative ones).
In the supervised task, the input data instance corresponds to a pair (b; s; c) where c 2 ftrue; f alseg is the classi cation label. Thus, speci cally, our semanti cation problem is formulated as a binary classi cation task. (b; s) is true if s 2 b's annotation sequence (as), else f alse. Where f alse instances are formed by pairing b with any other label not in the annotation sequence as of b. As an aggregate, the semanti cation of each bioassay is a multi-label, multi-class classi cation problem which we have broken up into binary classi cation decisions.
Intuitively, our task formulation is meaningful because it emulates the way the human expert annotates the data. Basically, the expert, from their memory of all semantic statements S, simply assigns s to a given b if they deem it as true; irrelevant statements are not considered, thus implicitly deemed f alse. 2.3
SciBERT-based Machine Learning
Our machine learning system is the state-of-the-art, bidirectional transformer-based
SciBERT [
For robust evaluations, we perform 3-fold cross validation (2:1 train-test split). In each
fold experiment, training data contains roughly 655 bioassays and the remaining 328
bioassays are used for testing, where the test assays are unique across the folds.
Standard precision (P ), recall (R), and f-score (F 1) metrics are used. We refer the reader
to the SciBERT paper [
Results and Discussion Our results are depicted in Tables 2 and 3. And we examine the RQ: can advanced neural technologies be leveraged to automatically semantify bioassays? We nd that the cumulative obtainable F 1 by the SciBERT classi er out-of-the-box is 0.72 (bold in Table 3)|signi cantly higher than 0.47 from a naive frequency-based semanti cation approach. Furthermore, the di erence of the neural approach from the frequency method is clearly evident in the hit-and-miss illustration in Fig 1. The top thin neck of the curve in Fig 1(a) indicates that the neural approach, for most bioassays, had faster true semantic statement hits among its top-scoring predictions. Thus, answering RQ, neural technologies can indeed perform reliable semanti cation of bioassays. They are also practically e cient, since, given the 1756 unique statements considered as labels, each test assay is semanti ed at a rate of 4 seconds. f alse labels The discovery of cures during pandemics such as Covid-19 can be greatly expedited if scientists are given intelligent information access tools, and our work toward automatically semantifying bioassays are a step in this direction. We refer the reader to the Appendix for an illustrated use case of semanti ed bioassays data in next-generation digital libraries.
s s p W C u o r c f o e o h tw ca n sa ra h rpo c teh em fro po lsu O o te se
d x io p u t en le -b a n
a n R in se e se n d ru e e b a rcah t-xge issao l-sab K n a e n e y d o r s lew itao tbo sem d n e a g d i n e i n t G ig te ifi rahp llitab trage itcao . r d n a ie sa ap r s a b i o a s s a y s i g i t a l L i b r a r i e s D
S
B
A o
o f o r S c i e n c e a n d T e c h c h l a r l y K n o l e d e G r a p i o a s s a y s f o r e x t G e n e r a 2 in an ,. 0 e n S 41 lae tao cuh .) irgnn itno ,rreS a o . n f , d b V u io is s a se e s r r - , rg it A B S ahp edfi aun iao rvu
b t ta i o s e w ao m s y
a a w ss it y C w ay c
a s o ..rrkgoo itsehn llygen apm g O re r
i R ta s K e o G d
n c d o s ig m
o it p a a f llib irso tS ra n r r s u y o c (D f t L se u .) m r F a e u n d ll c C i H fa uo eu no fr C W O
n la la g it lse r s n om lra e u irssefi irssefi isaoB --&M lltsaebo irxeeepm itrssagn iftranom euhbPC ,.ltcake saeudbo traaD (bo (to say iss tabo ,tsn litrae liave .Fm tano raan tse ttom )vp syb loP iann vew lrsem liseb raeo iignn lssye ) .iscSERB rceeunFq ftseSom ll.itaedpoomm itraeeeenbdw iiitaghnn17w litcgeuhnnndo ,isscaayehbow isssaaybo389 tttsaaeehndo
T y - - a b b n a a t s s i e e fy d d
Unique statements (labels) distribution Each bioassays present on average 53 labels. The distribution is visible in Figure 2 B
Snapshot of Semanti ed Bioassay in the Open Research Knowledge Graph Next generation DLs target semanti ed scholarly knowledge. The ORKG with the semanti ed bioassays integrated, automatically computes their survey comparisons depending on how many of the machine-actionable assays were selected to be compared by the user. Such tools must be available to scientists to assist them in such massive knowledge ingestion scenarios to quickly grasp the scholarly knowledge highlights fostering faster progress with discoveries. semanti ed Bioassay assay. Accessible at: