Cellulose, the most abundant and versatile biopolymer on earth found in plant cells and some bacteria, is the building block of cellulosic materials, which have many applications in various domains because of their sustainable, renewable, and biodegradable nature. One of the most significant applications of cellulosic materials is the production of paper and paperboard. The unique dimensions and characteristics of cellulose nanofibrils (CNFs) make them crucial in papermaking for enhancing the strength properties of paper [1]. In addition to their use in paper production, several nanocelluloses (NCs) are alternatives for the textile industry because of their higher mechanical resistance [2], and as a substitute for petroleum-based packaging [3], and as a natural polymer with low toxicity, high crystallinity, biocompatibility, and biosafety for biomedical applications [4].

However, the knowledge on cellulose, much of which is stored only in scientific publication and, even when available in digital formats like PDF or HTML, is not easily processed at a large scale due to the scattered and ambiguous nature of text. Information extraction (IE) approaches are needed to extract this information from scientific publications and make it accessible in structured formats. By putting the information into an ontology, the knowledge can be subsequently queried and reasoned with more efficiently. One of the key steps in IE is NER, or Named Entity Recognition, which is the task of extracting nouns and noun chunks, called entities or named entities, from text. Identification and classification of named entities is the key part of the information extraction process. Though, the current NER approaches are limited in their ability to recognize cellulosic named entities and to handle variations in the naming conventions of them. ChemSpot [5] is a hybrid chemical named entity recognizer that utilized a CRF (Conditional Random Field) model to identify chemical named entities in natural language texts. In the biomedical domain, tmChem [6] employed a model combination approach using two different CRF models to recognize chemical mentions, properties, and their relationships. Akkasi et al. (2016) introduced ChemTok [7], a rule-based tokenizer specifically designed for chemical named entity recognition. Swain et al. (2016) presented ChemDataExtractor [8], a toolkit capable of extracting chemical entities along with their properties, measurements, and relationships. Corbett and Boyle (2018) developed Chemlistem [9], a chemical named entity recognizer based on recurrent neural networks. Although there are other methods available for extracting material entities from text, they also struggle in recognizing the diverse range of entities encountered in the cellulosic domain. Zhao et al. ( 2021) introduced a fine-tuned BERT model [10] specifically designed for materials named entity recognition. Similarly, Miah and Sulaiman (2023) proposed a deep neural network-based model [11] tailored for materials named entity recognition. Shetty et al. (2023) presented an alternative approach [12] for extracting material property data. Furthermore, Weston et al. (2019) presented a comprehensive approach [13] that not only extracts material properties but also captures their applications and mentions of inorganic materials.

In order to comprehensively capture cellulosic knowledge, it is crucial to extract a wide range of relevant entities beyond just chemicals and materials. This includes extracting properties associated with materials and chemicals, manufacturing processes, as well as names of products and equipment. Therefore, existing methods often fail to accurately identify a significant portion of cellulosic entities due to their limited familiarity with cellulosic data. The ultimate goal of the project is to contribute in growing an ontology-guided knowledge body about cellulose by extracting relevant terms from the scientific literatures which are the preferred source of knowledge. Initially, a manually made cellulose ontology will play a significant role in NER by providing a structured representation of the knowledge and relationships among entities. It can also assist to improve NER performance by providing additional context and disambiguous information, as well as enabling more sophisticated reasoning and inference. Later on the cellulose ontology itself will grow by adding newly identified cellulosic entities to speed up ontology development. The manual amendment of ontologies can be a time-consuming and costly process which limits their usefulness in practice. In contrast, an automated process can help to overcome these limitations and enable more efficient and effective use of ontologies in NER and other NLP tasks. Additionally, the automatic amendment of ontologies can help to ensure that they are up-to-date and reflect the latest developments in the domain, but there is little work in leveraging the synergies between NER and ontologies: in (1) utilizing ontologies for NER and (2) using NER to amend and populate ontologies.

In scientific domains where accurate organization of terms is important to avoid misrepresentation of knowledge, relying solely on an automatic ontology construction method that builds the ontology from scratch may not be effective. Instead, a semi-automated method, where domain experts contribute initial concepts and relationships to establish a core domain ontology, can be utilized to amend the ontology with additional terms. The purpose of incorporating an ontology in the named entity recognition process is to improve its performance in the cellulosic domain. This integration will enable the system to identify the named entities that align with the concepts and relationships of the ontology and classify them accordingly.

The proposed dissertation aims to leverage the synergies between ontologies and NER by specifically addressing the following three research questions:

1. Under what conditions can pre-trained language models be incrementally re-trained for improved named entity recognition of terms in the cellulosic domain? 2. How can cellulose-related terms that are identified by such NER approaches be categorized more effectively and precisely by leveraging a small hand-curated ontology of cellulose materials? 3. What methods are suitable for determining whether a particular identified term refers to a concept that already exists in the ontology, or a new concept that requires amending the ontology?

The objective of the project is to develop an automated system that will identify cellulosic terms from given text with a higher accuracy. Additionally, the system aims to classify these identified terms, as much as possible, with the most relevant concepts available in a cellulose ontology which is currently being developed. To accomplish this, the proposed system will establish internal communication with the ontology, enabling the identification and classification of new cellulosic terms that are not yet incorporated in the ontology. The resulting set of new terms will be shared with domain experts for review, allowing them to assess the relevance of those terms and determine their appropriate placement within the taxonomy. If a new term is found to carry a more refined semantic meaning than an existing term in the ontology, the ontology will be amended accordingly. Furthermore, the identification and exclusion of irrelevant terms in every NER process will accelerate the continual assessment process for recognized cellulosic terms and their association with the ontology over time.

A ScispaCy [14] language model will be selected considering its current performance on the cellulosic data. This performance will be measured based on the number of correctly recognized cellulosic terms from a curated corpus of evaluation data. The best performing model will go through an incremental training process using spaCy [15] model training pipeline. In the first phase of training, the selected model will be introduced with CHEMDNER corpus [16] which is currently the largest corpus for chemical terms. Then the model will undergo training using various corpus of training data, including materials, properties, processes, and more, across different training phases. This iterative process will enable the model to progressively enhance its understanding and knowledge on diverse terms of the cellulosic domain. After each phase of training the performance of the model will be evaluated using standard metrics such as precision, recall, and F1-score. To assess the improvement, a comparison will also be conducted between the performance of the model in the current training phase and the performance of the models in the previous training phases using an identical evaluation dataset. Finally, the improved language model will be employed to extract cellulosic named entities from a vast collection of text documents. The extracted cellulosic terms will be comprehensively compared to the existing terms in the ontology, generating a set of candidate terms to be forwarded for further verification by domain experts. The domain experts will assess the suitability of these terms and make informed decisions regarding their rejection or integration into the ontology. This collaborative process will ensure that the ontology is enriched with relevant and accurate terms, enhancing its overall effectiveness and comprehensiveness. The effectiveness of the approach will be measured by calculating the percentage of terms accepted by domain experts. This metric will provide valuable insights into the success and acceptance of the proposed method in enriching the ontology with relevant and authoritative terms.