The use of SPARQL is evergreen for querying, extracting and expanding named entities from RDF datasets, significantly enhancing text processing applications such as Automatic Question Generation (AQG). Recent research employs pretrained Large Language Models (LLMs) for AQG to reduce manual costs, but these models are limited by their dependence on training data. This in turn makes LLMs to conterfeit the answer-revealingness challenge in automatic Multi-hop Question Generation. This challenge occurs in the process of Multi-hop Question Generation as it requires integration of named entities from multiple sources and deep comprehension of these interconnected concepts, which is quite challenging. In this context, Retrieval Augmented Models (RAM) have gained attention in NLP for improving text processing through enhanced information extraction, yet their application in AQG is limited. This research addresses this gap by RAM's workflow with attention to its first phase - Input Text Enrichment via Named Entity Expansion-is crucial for generating diverse, comprehensive questions. But, Effective named entity expansion is facilitated by ontology mapping to align entities to various ontologies, which is more demanding. To address this requirement, SPARQL querying techniques such as multi-querying, step-back querying, and sub-querying are examined to enhance named entity expansion accuracy, thereby improving RAM's efficacy, leading to well-formed auto-generated questions.
The SPARQL Protocol and RDF Query Language remains crucial need for querying, extracting, and
expanding relevant named entities from RDF (Resource Description Framework) datasets [
In this regard, Retrieval Augmented Models (RAM) have gained attention in Natural Language
Processing (NLP) for enhancing text processing and information retrieval. However, their
application in Automatic Question Generation (AQG) remains limited [
The role of ontologies and Ontology Mappings are evergreen in Automated Question Generation
(AQG), particularly in text expansion [
Meanwhile, SPARQL querying techniques have also been evolving to address challenges in RDF
data processing [
However, the application of RAM in AQG remains underexplored, which is highly essential to
handle the limitations of pretrained Large Language Models (LLMs) in dealing with the
domainspecific knowledge for Multi-hop Question Generation. This research gap emphasises the need for
analysis of OM’s contributive factors [
• • • •
RQ-1: How can RAM be effectively utilized to incorporate domain-specific knowledge for Multi-hop Question Generation using LLMs ? RQ-2: What are the impacts of diverse SPARQL querying techniques on the accuracy of named entity expansion and efficiency in RAM-based AQG systems? RO-1: To study the effectiveness of RAM-AQG integration in overcoming the limits of incorporating domain-specific knowledge LLM based Multi-hop Question Generation. RO-2: To analyze the significance of OM’s contributive factors such as multi-querying, step-back querying, and sub-querying on the accuracy of named entity expansion for AQG.
The proposed methodology employs an Anchor-Align approach, a variant of CLASH approach to link the named entities in the sources text to that in the Ontology to bring out named entity expansion. If all entities are connected by one or more edges, the named entity network becomes a connected graph. Conversely, if an entity remains disconnected due to a lack of association with other entities, it becomes impossible to infer any meaningful information about the entity from the network. Once processed, the Ontology is updated using shared attributes: an entity vector (ndimensional vector) and the similarity between two entities (cosine similarity between these vectors).
In this phase, the source text is pre-processed using various Natural Language Processing (NLP) techniques such as tokenisation, case folding, entity extraction etc. Entity extraction based categorisation of source text using IsimScore (Equation 1) calculated with bigram relevance and semantic relationships intersection for further Ontology Mapping (OM) based entity expansion process. The random subset selection process is based on the assumption that using 63% of records in source data and log(F) of Feature set can generate feasible number of Random Subsets. In addition, heuristics are applied (Table 1) to carry out additive feature extraction for interpretable ensemble model generation in the consecutive stages of the work.
• (1)
= ∏!"#$∑%*)$%("∑# $'&$((&"##'$((|&&##,,&&++')( |&+)
Where, Sim = Similarity between unigram/bigram entities, Ei+1, Ei = Entities extracted from input text and Ej+1|Ej = Entities extracted from knowledge graphs.
The methodology for anchoring links between entities of different ontologies in the ontology mapping process focuses on establishing Basic Mappings as an interim outcome. This process is predicated on the execution of SPARQL queries, primarily employing three query types: sub query, multi query, and step-back query. Sub queries are utilized to extract specific subsets of data from a larger dataset, identifying entities that meet particular criteria within a single ontology. Multi queries perform simultaneous searches across multiple ontologies, facilitating the identification of potential links by comparing entities in parallel. Step-back queries trace relationships by moving backward through the data hierarchy, ensuring the consistency and contextual integrity of mapped entities. In the Basic Mapping process for two conceptual ontologies, sub queries extract relevant properties from the source ontology. Multi queries then identify equivalent properties in the target ontology by retrieving and comparing similar entities. Step-back queries verify and refine these mappings by ensuring contextual consistency. Using the above specified two-step empirical technique the mappings are generated as entity-relationship sets, which is then encoding into vectors for further text processing.
Three dataset under varied domain and size are considered for empirical analysis and comprehensive evaluation of the OM process in the proposed work. The first dataset, The BPMN ontology [12] serves as a structured framework for representing Business Process Model and Notation (BPMN) concepts in a machine-readable format. It encompasses 270 classes, 176 object properties, and 70 data properties, enabling a comprehensive representation of BPMN elements, attributes, and relationships. The 270 classes represent core elements like processes, activities, events, gateways, data objects, and participants, which are being mapped into classeses using retinal shapes such as triangles, polygons etc. Object properties detail relationships like control flow (e.g., sequenceFlow), associations (e.g., dataInputAssociation), containment (e.g., contains), and interactions (e.g., participant). The second dataset is the Shape Ontology (SO) [14]. The shape ontology comprises 40 classes, with endurant or continuant entities forming the top-level category and specific shapes as subtypes. It defines 15 data properties, categorizing shapes by characteristics such as dimensionality, symmetry, boundary conditions, and curvature. The ontology includes 30 object properties, detailing relationships like hasShape (between objects and geometric shapes) and approximates (between individual objects and shape property types). The third one is the Geometry Ontology (GO) [13]. The Geometry Ontology encompasses 9 classes, including foundational geometric representations such as Point, LineString, Polygon, and their composite forms like MultiLineString, Triangles, MultiPolygon etc. he ontology includes 10 object properties, including "geometry, symmetry" linking resources to their geometric shapes, and "boundary," grouping properties defining polygonal boundaries.
The study (Table 2) shows that the proposed approach surpasses the baselines through improved triple type driven relation extraction for entity context based AQG.
It is noted that a wide margin of 2-3% accuracy improvement can be achieved using the proposed A-A approach in entity selection for AQG. In particular, the generation of ERR category triples (relationships) are more benefited when used with sub-query and multi-query types, showing an accuracy improvement of around 4%. On the other side, the URR category triples are commonly achieving around 94% upon all three query types, where Basic Mapping (BM) is relatively improved. In this line, it is noted that the proposed A-A approach can also generate improved mappings, only when Basic Mapping (BM) process is required. Figure 2 shows the sample mapping generated using the three diverse query types. Based on these observations (Table 3) the proposed FFRF model is inferred to have improved potential to automatically generate questions, balancing diversity and relevance factors.
Discuss the geometric characteristics of an equilateral triangle and elaborate on the relationship between its sides and angles.
Generated Question: Define equilateral triangle and compare its geometric properties with other types.
In conclusion, this research paper explores the utilization of three SPARQL query types to enhance OM andRAM for AQG. The study focused on addressing challenges in generating non-answer revealing multi-hop questions using Retrieval Augmented Models (RAM). By employing SPARQL techniques like multi-querying, step-back querying, and sub-querying, the research aimed to improve the accuracy of named entity set expansion, a critical requirement for AQG tasks. Empirical analysis across diverse datasets shows significant advancements in triple type-driven relation extraction, achieving notable improvements in precision, recall, and F1-score metrics compared to baseline methods. The paper highlights the significance of OM across three common triple types, facilitated by SPARQL to complement Large Language Models (LLMs) with reliance on traditional approaches. Future research could further broaden OM across different domains, and integrate advanced techniques into RAM-based AQG systems.
This research was carried out at E-Learning and HCI Lab, Department of Computer Applications, National Institute of Technology, Tiruchirappalli. [12] Singer, R. (2019). An ontological analysis of business process modeling and execution. arXiv preprint arXiv:1905.00499. [13] Katsumi, M. Geometry Ontology. Enterprise Integration Lab. Retrieved June 16, 2024, from https://enterpriseintegrationlab.github.io/icity/Geom/doc/index-en.html [14] Rovetto, R. J. (2011). The Shape of Shapes: An Ontological Exploration. Shapes, 1.