IBID is a virtual librarian that processes biology articles and builds semantic annotations based on the contents of an article. It then assists human designers by locating and presenting biology articles related to a design query. IBID's use of ontologies allows for knowledge extraction and assists users with the identification of key information in an article and comparison of the contents of two articles. In this paper, we describe how the addition of an environment ontology enhances IBID's capability to understand the habitats of various organisms. In a pilot study, we evaluated IBID's performance against human subjects who read the same passage and highlighted phrases pertaining to locations and habitats. The preliminary results indicate that the ability to add ontologies to IBID allows it to extract meaning from new documents.
Scientific documents are information-rich and are more common and more available than ever before. However, with this proliferation comes the challenge of tracking and understanding scientific documents at scale. Traditionally, a scientist could work with a librarian to find the literature relevant to the problem of interest. Now, most scientific literature has moved online, real librarians are hard to find, and it is increasingly difficult, even for experts, to track, read and understand all the new scientific documents that are being generated on a given topic.
Understanding scientific documents is an involved process: there is a big difference between just reading text and actually understanding it. We view scientific document understanding as the ability to process information and then be able to draw useful inferences from it and not draw spurious inferences. This view supports higher level tasks like comparing the contents of two different documents and identifying similarities and differences between them.
We posit that AI can be a powerful ally in tracking and
understanding scientific documents and that
knowledgebased methods that use ontologies can augment the
understanding capability of AI agents. This kind of AI agent can
serve as a sort of virtual librarian for scientific literature. The
IBID
2
Biologically inspired design, also known as biomimicry
If engineers could successfully and efficiently mimic this ability in technological systems at scale, it might be possible to solve many water crises that exist in the world (Chen & Zhang 2020).
However, there are several major hurdles in putting
biologically inspired design paradigms into practice. From an
information-processing perspective, one big hurdle is
locating biological cases relevant to a design problem. Given a
Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
problem, most designers search for articles describing
relevant biological systems online. Observations of online
information-seeking behavior of (student) designers indicate
three problems
As a result, there have been several attempts in using
natural language processing techniques to help designers locate
biology articles relevant to their problem. Shu (2010)
describes an early approach in engineering for using natural
language processing for this task. Shu uses keywords for
anchoring the natural language processing, but points out that
the benefits of information extraction through natural
language processing is not restricted to known patterns. Nagel,
Stone & McAdams (2010) use an engineering to biology
thesaurus that translates design queries in engineering to
equivalent keywords in biology. Krupier et. al (2017)
provide a more recent effort coming from biology. Their work
is based on a domain-specific ontology of biological
systems
Of course, the goal of using publicly available scientific
literature to support human creativity extends far beyond the
domain of biologically inspired design. In the context of
computational creativity more generally,
3
The goal of the IBID project is to address the above mentioned problems of findability, recognizability and understandability in the context of biologically inspired design. Figure 1 shows the full functionality of IBID for its three use cases: (1) End users such as engineers and designers looking for biology articles relevant to their design problems, (2) Knowledge engineers extending IBID’s knowledge representation ontologies, and (3) System administrators adding to its repository of analyzed papers. Figure 1 also specifies the actions available to each user type; the arrows in the figure indicate progression of steps and/or access to/from the database.
The core of IBID’s approach to these problems is the use
of the Structure-Behavior-Function (SBF) models of
technological and natural systems
In earlier work on the KA project in the 1990s
In IBID, the SBF ontologies come from several sources: • Structure Ontology is borrowed from Vincent’s (2014) ontology of biological systems. • Behavior Ontology builds on Khoo et al.’s (1998) patterns of cause and effect. • Function Ontology was developed in our laboratory (Rugaber et al. 2016). Functional concepts are organized hierarchically: similar concepts are grouped together as families and more nuanced concepts are found deeper in the hierarchy.
The current version of IBID does not directly relate structure, behaviors and functions of a biological system into a complete SBF model.
These ontologies help IBID construct a partial SBF model of the biological system described in a biology article. Rugaber et. al. (2016) provide an example of how IBID processes the following passage from Norgaard and Dacke (2010): The mechanism by which fog water forms into large droplets on a beaded surface has been described from the study of the elytra of beetles from the genus Stenocara. The structures behind this process are believed to be hydrophilic peaks surrounded by hydrophobic areas; water carried by the fog settles on the hydrophilic peaks of the smooth bumps on the elytra of the beetle and form fast-growing droplets that - once large enough to move against the wind - roll down towards the head.
IBID processes the above paragraph and identifies the structure, behavior and function specified in it: • Structure: IBID identifies the entity in question as elytra. • Behavior: IBID identifies the cause as droplets grow in size and the effect as they roll down towards the head. • Function: IBID identifies the result of the action as move the water droplets.
This list is only illustrative of IBID’s capabilities, not comprehensive. IBID performs this kind of automatic extraction of structure, behavior and function for whole articles and annotates the articles with the extracted structure, behaviors and functions.
Given IBID’s annotation of biology articles in a corpus with the structure, behaviors and functions of biological systems described in them, users can perform faceted search on the corpus (Prieto-Diaz 1991). Thus, a user may search for the function move, or the structural element elytra, or both. A user may also use IBID to perform a search using a design query expressed in plain English: given such a query, IBID extracts the structure, behaviors and functions of the desired technological system from the query and then matches the extractions with the SBF annotations on the articles in the corpus in a manner similar to the earlier KA project (Peterson, Mahesh & Goel 1994). This helps IBID address the problems of findability and recognizability we described earlier.
IBID also highlights the SBF annotations on a biology article. This helps IBID address the problem of understandability even for dense and long articles, such as the Norgaard and Dacke (2010) article quoted above. This can potentially help the user process biology articles more efficiently and easily, where the users may include not only biologists, but also engineers, designers, or even citizen scientists. 4
While the paragraph from the Norgaard and Dacke (2010)
article briefly mentions the location of elytra (elytra of
beetles), the above description of IBID has no way of
identifying the location of the structural elements of a biological
system. However, for many biological systems, the
location, habitat, and, more generally, the environment of the
system is very important. The external environment is also
important for technological systems: the specification of
many design problems includes a specification of the
environment of the desired technological system
Actually, the environment always was a part of SBF
modeling
Instead of building a new environment ontology from
scratch, we decided to explore already existing ontologies.
After examining several candidates, we selected the
Environment Ontology (ENVO) described by Buttigieg et al.
(2013) in the Journal of Biomedical Semantics. This
ontology is hosted on the OBO Foundry (Smith et. al. 2007) and
is quite comprehensive. A big advantage of this ontology
over many others is that it can be exported as a Web
Ontology Language (OWL) file. OWL files written in the
Semantic Web Language are “designed to represent rich and
complex knowledge about things, groups of things, and relations
between things.”
To provide a simpler testing ground of adding an ontology into IBID and testing its effectiveness, we reduced ENVO to just contain extremely basic concepts relating to ecosystems and their key environmental concepts. Figure 2 illustrates a small excerpt from the stripped-down ENVO.
Three factors were especially important in adding the ENVO ontology to IBID: 1. The ability to have a standard format by which to import ontologies. 2. The ability to add information quickly without breaking previous implementations.
3. The ability to use & export this information easily.
By using ENVO, it was clear that Objective 1 could be reached just by establishing that all future ontologies would use the OWL format. Not only can OWL files be imported, parsed, modified, and exported easily, there are many tools to help visualize and act on these OWL files such as Protégé (Musen 2015). Protégé became the software used to scale down ENVO, as well as rebuild the Structure, Behavior, and Function ontologies so they also conformed to the new OWL standard. Adding new concepts or modifying existing concepts was simple using the Protégé software, thereby addressing Objective 2.
With the new converted ontologies, the issue of how to store these ontologies in a relational database arose. To resolve this, we developed a script that would take in an OWL file and convert it into its relational database equivalent. By the end of the implementation, the structure, behavior, and function ontologies were updated and reimported into IBID’s relational database using the new OWL format. The environment ontology was also imported into IBID allowing for articles to be analyzed to extract environment concepts. In addition to this, IBID now has a pipeline for integrating new ontologies that are in the OWL format in an easy manner. Given that all of the data was imported into a relational database, exporting this information from the database was simple, and even using Protégé to export the OWL files into other formats was simple, addressing Objective 3. 5
With the ability to import new knowledge executed, the next step was experimenting and evaluating how well IBID could leverage this knowledge. An experiment was conducted to test the effectiveness of the environment ontology with ten participants outside of the IBID project in the Design and Intelligence Lab. In conjunction with this experiment, a validation page was developed to test the functionality of the environment ontology. The use case of comparing scientific documents was also explored qualitatively.
IBID’s validation took a passage of text and ran it through
IBID’s analysis pipeline and returned a list of results
specific to the environment ontology. The experiment
compared IBID’s results with human subjects analyzing the
same passages. The results of this experiment would reveal
gaps in the environment ontology’s functionality that could
be used to make it more robust. The text for the experiments
came from Szalay (2014), en.wikipedia.org/wiki/Elephant,
and
In total, 10 human participants completed the experiment. Each participant read the same three passages on three different organisms. The instructions were to underline terms in the passages they considered to be related to the “environment” or the “habitat” in which organisms live. The organisms in question were the King Cobra, with a passage containing 4 sentences, the African Bush Elephant with a passage containing 14 sentences, and the Highland Streaked Tenrac with a passage containing 6 sentences.
Passage King Cobra (4 sentences) – Passage 1 African Bush Elephant (14 sentences) – Passage 2 Highland Streaked Tenrac (6 sentences) – Passage 3
Avg. # of Terms by Humans # of Terms by IBID 8 10 11 2 4 4 The highlighted phrases were pulled out exactly as they were marked by the participant. The assumption here was that there was a difference between a term having been highlighted in one straight stroke, and a term being highlighted with spaces in between. This meant that in this sentence from en.wikipedia.org/wiki/Elephant:
The African bush elephant can be found in habitats as diverse as dry savannahs, deserts, marshes, and lake shores, and in elevations from sea level to mountain areas above the snow line.
There was a difference if a participant highlighted, “dry savannahs, deserts, marshes, and lake shores” in one go to count as one phrase, or they highlighted “dry savannahs,” then “deserts,” then “marshes,” then “lake shores” separately to count as 4 different phrases. Of the 4 sentences based on the King Cobra’s habitat, the humans were on average able to locate ~8 different environment terms. Running the same passage in IBID led to it finding only 2. Of Passage 2 dry thorn-scrub forests (x5) the 14 sentences based on the African Bush Elephant, the humans were able to on average find 10 different environment terms; IBID was able to identify 4. Finally, the passage on the Highland Streaked Tenrac had humans denoting around 11 environment terms while IBID was able to extract 4. The results are shown in Table 1. Table 2 contains the concepts that a majority of participants agreed on. The criteria for “agreeing” means that of the aggregated list of results, at least 50% of the participants agreed that the selected sentence was one that contained an environment concept and at least 5 participants also agreed on the concept that indicated it related to the environment.
As mentioned earlier, scientific document understanding allows an agent to perform higher level tasks and one such task that is paramount in any kind of research is the ability to quickly compare the key points of two different documents. IBID is able to take in two documents and run its analysis and display the results side-by-side. This process involves the same pipeline as discussed earlier and leverages the same knowledge base. We tested this process with several different excerpts taken from descriptions of the habitats of different species, an example of which is shown in Figure 3. It can be seen that IBID’s ability to understand the key concepts in a document helps the researcher quickly compare two documents. If we have two documents about the same or similar species, IBID can help the researcher compare and contrast information and see where two different documents are in agreement and where they disagree. We believe that this can be a powerful tool and a major feature in the realm of scientific document understanding. 6
Based on the experiment above, we can see that the addition of a new ontology, in this case the environment ontology, improves IBID’s understanding in this domain. IBID initially had no understanding ability when it came to habitats and locations, but the addition of this ontology led to increased understanding as seen in Table 1. However, we acknowledge that the number of participants in our experiment was small and IBID did not reach human level performance. We still feel that these preliminary results show that IBID’s ability to integrate new knowledge moves it towards becoming a true virtual librarian.
The experiment also showed some of the weaknesses
IBID has. For example, there are many proper noun location
words (country names, cardinal directions, etc.) that many
participants deemed relevant to the environment of an
animal. IBID’s knowledge base is strictly that of habitats as
described in ENVO. Take for instance the simple sentence
from Passage 3
They are most commonly found at forest fringes on the central plateau edge and near cultivated fields and rice paddies The key term was “forest” and it was pulled out by IBID; the term “forest” maps to an environment concept in ENVO. In contrast to this, humans are able to look at a sentence saying, “southern Indian desert” and see that the whole phrase indicates location while IBID would only be able to recognize the term “desert”.
Looking at the “Phrase Selected” column in Table 2, it is clear that there are many examples where humans agreed that adjectives describing habitats are just as important as the habitat itself. Descriptive words like “dense mangrove swamps” and “dry savannahs” might be difficult for IBID to parse because they are compound terms containing a descriptive word followed by a habitat word. This issue could be addressed by extending IBID’s parser to include adjectives that might describe an environment term.
One thing IBID does really well is identify the verb
predicates from a sentence. Verbs like “prefer”, “occur”, and
“find”, occur frequently with environment related phrases
that were marked by the human participants. For example,
in Passage 3
There are sentences where IBID identified information that was right, but the term used to do so was not. For example, in the sentence from en.wikipedia.org/wiki/Elephant, The African bush elephant can be found in habitats as diverse as dry savannahs, deserts, marshes, and lake shores, and in elevations from sea level to mountain areas above the snow line.
IBID pulled out the word “bush” instead of one of the environment terms, even though “bush” is just part of the species name. This means that in passages where the name of an animal is an environment concept, IBID may pull out a false positive.
Finally, there were cases where humans identified vague habitat phrases like “under a tree,” or “near water sources” that IBID missed. For example, in the sentence from en.wikipedia.org/wiki/Elephant:
Asian elephants prefer areas with a mix of grasses, low woody plants, and trees, primarily inhabiting dry thornscrub forests in southern India and Sri Lanka and evergreen forests in Malaya.
It makes sense that humans marked “mix of grasses” and “low woody plants, and trees.” However, there aren’t any real concepts in ENVO that are mapped to by these phrases. However, the verb “prefer” was identified by IBID and allowed the sentence to be extracted independent of the environment terms found by the participants.
These results show that IBID’s knowledge-based methods show promise in efficiently extracting information from a scientific document and that the use of ontologies allows for it to quickly integrate and leverage new knowledge, without the need for extensive data collection or training. Another major benefit of IBID’s approach is better explainability. It is easy to determine gaps in IBID’s knowledge, like those identified in regard to proper nouns and cardinal directions. It is also easy to see which knowledge IBID used to extract information. The use of an ontology also allows IBID and its users to leverage the relationships that are found for downstream inferencing tasks. The use of the standard OWL file format also allows users to edit the knowledge using tools like Protégé.
IBID demonstrates the effectiveness of knowledge-based methods in augmenting scientific document understanding and moves us towards a true virtual librarian. IBID’s use of standardized ontologies allows it to quickly gain a deeper understanding of a new domain, without the need to acquire lots of new data or to spend time learning a complex model. This ability also allows IBID to be extensible. The Environment Ontology was a working example, but the same process can be applied to new ontologies, thus growing IBID’s understanding capability. These abilities allow IBID to facilitate higher level tasks like document comparison, which can help users of IBID compare and contrast different approaches to their engineering problem. We acknowledge that there is a need for augmenting the analysis and filling the gaps in IBID’s knowledge, but the use of knowledgebased methods helps the user to efficiently identify these gaps and easily make modifications or add extra processing.
We are grateful to Julian Vincent for sharing his ontology of biological systems; IBID’s structure ontology is a subset of his structure ontology. We thank Pablo Boserman and Daniel Dias for their work on making the IBID system functional, as well as members of the Georgia Tech Design & Intelligence Lab for assisting with the IBID experiment including the evaluation of interactive system.