In this position paper, we propose a detailed technical outline of what needs to be done, for example in a Master project, to bridge the research gap for the problem of the existence of terminological saturation. The problem is studied regarding a sequence of incrementally growing sub-collections of documents describing an arbitrary subject domain, using the OntoElect approach. After reviewing the related work, we present the formal basics of the approach and experimental evidence of the existence of terminological saturation. Consequently, we formulate the research hypotheses, and outline the methodology and plan for further research elaborating on this position.
Extracting a set of terms from a document collection, describing a subject domain, is
an important initial step in figuring out a complete set of requirements for building an
ontology for the domain [
In frame of OntoElect methodology [
The following research question has been left, however, without a proper attention
in our prior research: Is there a way to prove formally the existence of terminological
saturation for an arbitrary collection of textual documents? This question is important,
as terminology extraction from texts is computationally hard, even if optimized [
In this position paper, we aim at blueprinting the formal framework to solve the outlined problem. For that, the hypothesis about the conditions for the existence of terminological saturation need to be formulated. However, we leave this proof for a separate research project leading to a degree in Computer Science. Therefore, this paper is the proposal of a potential master project.
The remainder of the paper is structured as follows. In Section 2, we review the existing related work on terminological saturation. In Sect. 3 we present our background experimental evidence of the phenomenon of terminological saturation and deliberate on relevant research questions. In Sect. 4, we present the structure of the formal theoretical framework and sketch out some of its basic components, including the hypothesis about the formal conditions for the existence of terminological saturation. In Sect. 5 we present a vision of the plan of the future research work towards developing this theoretical framework. Finally, we make conclusive remarks in Sect. 6. 2
An ontology as an artifact [
To design a domain ontology to be well supporting the requirements of the relevant
knowledge stakeholder community, it is imperative to be informed sufficiently fully
about their views. This poses a challenge, as it is hard to elicit directly the
interpretations of the domain from the knowledge stakeholders in an explicit form [
To overcome the abovementioned difficulty in ensuring representativeness, it has
been proposed [
One of the relevant types of these artifacts is professional textual documents.
Ontology learning from texts is now a noticeable subfield in ontology learning with
developed methodologies and processing pipelines [
While there is a bunch of approaches to select relevant documents in the literature,
the problem of checking if a document collection is sufficiently complete has not been
adequately resolved. One reason for under-estimating the importance of ensuring the
representativeness of a text collection is that text resources are abundant. Hence, one
may always expect to be able to have enough if the domain of her interest is well
circumscribed. For example, it is feasible to collect high-quality research papers on a
particular topic or within a field, as we did for Knowledge Management (KM) [
Furthermore, it might happen that only a small part of a big document collection is sufficiently complete in terms of domain knowledge coverage. Therefore, having a method to find this terminological core within the entire collection would help substantially decrease the effort needed for ontology learning from these texts.
The only relevant Computer Science publication we found in the context of ensuring
the completeness of the set of processed documents is [
The use of saturation phenomenon has received little attention in the Computer
Science literature, in particular in Text Mining and Ontology Learning. Saturation of
clauses was used in Theorem Proving [
The only broadly exploited analogy to terminological saturation, which is directly
relevant for our purposes, we found in qualitative research methodologies for Sociology
and Medical Sciences, which is theoretical (or data) saturation [
In Qualitative Research, the phenomenon of data saturation finds its origin in the
Grounded Theory method by Glaser and Strauss [
Notably, a mainstream ontology engineering methodology could be very similarly
termed as the discovery of a descriptive domain theory based on the data obtained from
qualitative research – c.f. [
Numerous attempts to operationalize the detection of data saturation has been
mentioned in the Qualitative Analysis literature. However, it is still a “mysterious step” [
We now briefly present our background knowledge in the context of detecting terminological saturation in document collections. This background has been developed in the OntoElect project1. We first outline the formal basics of the technique proposed for detecting terminological saturation in Sect. 3.1. In Sect 3.2, we then summarize our experimental evidence of the validity of this technique and emphasize the problem. This problem represents the research gap, which needs to be further studied and narrowed. 3.1
In OntoElect [
), for which a domain ontology needs to be developed or refined.
Our supposition is that, if we have a sufficiently bounded to describe it is finite and not very large. These terms could be extracted from the doc, the set of terms used plete document collection. uments (Doc), belonging to a documents collection ( = { Hypothetically, one may collect all the existing documents that describe }) that describes – a com.
Definition 1: A Complete Document Collection for . A containing all the documents describing
For any realistic , its is a Complete ( ). representative set of terms from it would be a tedious and resource consuming task. This is why we look at the document sub-collections of a .
Definition 2: A document sub-collection. A document sub-collection ( ) for the may be very big in volume. Therefore, extracting a is the subset of the
:
We are interested in finding a
cally the same set of terms as the
these
features that characterise
, form the terminological basis
= { }, = 1, … ,
Definition 3: A Terminological Basis. The finite set of terms , identifying all the
might not be equally important for describing the domain,
as reflected in the documents of the
with every term used in the documents of the
. Let a real positive value (score) be associated
. The more significant the term is for
describing the domain, the higher is the score. Hence, a vector space model (VSM) [
= { } having dimension . It is not easy, however, to build a for any realistic domain. We have to have a technique to: extract from the documents describing the ; make sure that an extracted ∈ ; and that all significant have been exExtracted Terms. Let there be a mapping that transforms a into a bag of terms ( ) extracted from the documents of this . Every element in is a pair < , >, where t is a candidate string and is the estimate of the likelihood The term candidates having high scores are denoted as significant terms. that is a relevant term for the
: the higher the , the more likely ∈ .
Retained Significant Terms. Let a term significance threshold ( ) be rationally above which the terms are regarded as significant, hence belong to the chosen (or estimated) for the s of individual terms in a . It indicates a boundary . After having built a , let us retain the s with the score > eps in the corresponding Bag of Significant
is either chosen empirically, or selected based on common sense considerations. In OntoElect, we offer a rationale for the estimation of an eps.
A Simple Majority Vote on Terms. A subset of term candidates is considered the core, containing all significant terms, if the terms in this core reflect the sentiment of a simple majority of the knowledge stakeholders in the domain. Consequently, the score of a term might be interpreted as the sum of the votes for this term by the domain knowledge stakeholders.
Definition 4: An Individual Term Significance Threshold. Let be sorted in the descending order of the scores. Then eps threshold for is computed as follows (1):
= : ∑ =1 > 1/2 ∑ =1 , ‖ ‖ ing condition (2) holds: at the top of the ordered list is found, whose voters are the simple majority. where is the minimal number such that the condition after the semicolon holds.
Definition 5: A Bag of Retained Significant Terms. Let ⊂ such that the follow(1) (2) ∀ , > . in the bag of significant terms hardly tractable for any realistic will constitute the because of at least two reasons: (i) one has to Then denotes the bag of retained significant terms.
Successive Approximation for building a a might be to extract and retain all the significant terms from the . One (superficial) way for building ensure that the collection of documents s/he possesses is a (ii) processing a , that qualifies to be a for a realistic tionally expensive because of its volume. One feasible way might be to use a statistifrom this cally representative will form a basis that is very similar to , with statistically negligible instead of
. Then, the relevant and valid terms extracted difference. Hence, we need a way to figure out if a In this regard, a successive approximation technique is plausible. is statistically representative.
. The terms . This is however , which is hard; and , is very computa
Terminological Saturation. Let: ment sub-collections such
that 1, 2, … , , … 1, 2 , … , , +1 retained from the successive positive value. If, at some : (i) ℎ goes below the threshold of the statistical error ; and (ii) there is a convincing evidence that it will never go above this threshold; then and returning the difference as a real be the bags of retained
significant terms , … . Let ℎ be a function comparing the bags of significant terms 1,
, , … be the sequence of docu2 ⊂ ⋯ ⊂
⊂ ⋯ ⊂ extracted
; from is a saturated
for
, is a saturated term set; the difference (distance) between and such a could be used as an -approximation of is not higher than . The set of terms in
. Such a , labelled further as is the bag of terms from which . The difference (ℎ ) between is retained; and and any successive , including , is within the statistical error: ℎ ( , ) < .
Terminological Saturation Threshold. The premise for a rational choice of the threshold ( ) for detecting terminological saturation is that a set of terms becomes saturated if it already contains all the terms from . Hence, whatever the terms are added in the subsequent s, these are not significant. Let us set ε = – the (1) computed for . Then
(2) will contain statistically the same set of terms as .
Terminological Difference Measure. Let us now define the function (ℎ ) for measuring the distance between term sets. It takes in a pair of term sets and maps these arguments into a real positive value of the distance between them: 3.2
Experiments show that terminological saturation, measured using ℎ , exists in the collections, having appropriate volume, composed of carefully selected documents describing the same topics (different
(Fig. 1(a)). From the other hand, if the documents on arbitrary s) are randomly taken into , then terminological saturation is not observed (Fig. 1(b)) and appears to be not reachable. Furthermore, in some collections ℎ measurements result in an unclear picture – as pictured in Fig. 1(c). In the latter case, ℎ values are volatile and oscillate quite sharply around the curve of , hence, cannot reliably indicate if terminological saturation is reachable. Fig. 1(c) pictures that there is often a chance that “might be” saturation observed in several measurement steps is further disproved by an additional measurement. In such cases, the ℎ : {< , >} ⟶ ℜ+. (3) Due to the incremental nature of sub-collections , not only the number of extracted terms will grow in + compared to , but also the absolute values of the s of the terms. Therefore, for making the s comparable in the pairs, normalized values have to be used. Let of the term, in a bag of terms , is the
of the first element. Then a normalized score ( having maximal value. In a , sorted in the descending order of the s, ) could be computed as . To measure ℎ we need to: (i) extract from and from
; (ii) compute for and for using (1); (iii) retain significant
terms in and using (2); (iv) compute
for and ; (v) compute ℎ ,
based on (i)-(iv).
collection might be not representative and, therefore, more documents have to be added
to it. One more reason for high ℎ volatility might be that the collection is too noisy,
as it appeared to be in the case of DAC2 [
(a) Saturation in DMKD-3003 (b) Absence of saturation in RAW4 (c) Saturation is not clear in DAC Legend: individual term significance threshold eps; terminological difference ℎ ( , +1) Hence, the open problem that needs to be further researched is finding the sufficient conditions for terminological saturation to exist after a necessary condition, its indication, have been observed in ℎ measurements.
Our position is that the problem needs to be solved formally and the theoretical framework for the solution has to be elaborated as a structured set of proven formal statements. 4
Based on the OntoElect method and the experimental evidence of its validity, presented
in Sect. 3, we now outline the open research questions and put forward the hypotheses
that need to be proven. The hypotheses are formulated in a way to resolve the issues
mentioned in the context of the volatility of ℎ measurements and, therefore,
instability in terminological saturation. These statements form the logic of the sought
theoretical framework.
The ℎ function is the mapping of the vectors , , representing respective partial
collections, into ℜ+. Several distance functions are known from the literature in this
context – c.f. [
We put forward the following research hypotheses about this function.
Hypothesis 1: ℎ is Manhattan distance function. 2 3 4
DAC is the collection containing 506 full text papers published in the proceedings of the Design Automation Conference between 2004 and 2010.
DMKD-300 is the collection containing 300 full text articles published in the Journal of Data Mining and Knowledge Discovery between 1997 and 2010.
RAW collection was synthetically formed of 80 randomly articles from English Wikipedia
such that no two of them were about a similar topic and the size of an article was not too small.
(4)
an
Manhattan (or often also called taxicab) distance [
= ∑ =1 − ,
Hypothesis 2: Let sible document sub-collections of a
is a metric space. where = ( 1, 2, … , are the vectors in to be proven that (4) is the formula for computing ℎ . n-dimensional ℜ+ vector space with fixed Cartesian coordinate system. Hence, it has be a vector space formed of VSM representations of all posin . Then ℎ , is a metric function For proving this statement, the metric conditions for ℎ , need to be checked: (i) non-negativity; (ii) triangle inequality; (iii) symmetry; and (iv) identity of indiscernibles. Further, it has to be proven that a is a metric space with ℎ , as its distance metric. 4.2
approximation, decreasing function. when ℎ For terminological saturation measurement, we are interested in computing ℎ not for arbitrary , , but for successive pairs , +1, = 1, 2, … . Hence, the function ( ) = ℎ ( , +1) is of our practical interest. In particular, we are interested ( ) goes below
. To find this out, we have to analyse: • The values of ℎ
( ), = 1, 2, … : { , +1} → ℜ+ • The values of individual term significance thresholds , used for retaining terms in , which could be regarded as a function ( ) = , = 1, 2, …: → ℜ+
ically non-decreasing function. However, in general it might be possible to build its ( ), as a lower envelope function, that is a monotonically nonenvelope function, that is a monotonically non-increasing function.
( ), ℎ
( ) is not necessarily a monotonically non-increasing function. It might also be possible to build its approximation, ℎ ( ) as an upper
Hypothesis 3: Terminological saturation exists if there exist: (i) a monotonically non-decreasing function ℎ ( ); the intersection of these functions at some . monotonically non-increasing function 4.3
Hypothesis 3 may be formulated as an existence theorem for terminological saturation in a sequence of incrementally growing document sub-collections as follows.
Theorem 1 (sufficient conditions of terminological saturation). Let: (i)
2 … ⊂ arbitrary domain tracted from 1, … be the sequence of document sub-collections, each describing an ; (ii) 1, 2, … , , … be the sequence of the bags of terms ex
… and ( ) is the function of individual term significance thresholds for , = 1, 2, … ; (iii) 1, 2, … , , +1, … be the sequence of the 1 ⊂ bags of retained significant terms for which pairwise successive terminological differlogically saturated, i is the saturation point, and ence is computed using the ℎ ( ) function. Then, the sequence of = = is termino(iii)
( ) ≥ ℎ ( ) (i)
There exist a non-decreasing lower envelope function
(ii) There exist a non-increasing upper envelope function ℎ ( ) for ℎ ( )
( ) for ( )
have to be verified in the experiments using the instruments and datasets available in
the OntoElect Project [
30 percent of measurements and proven existence Theorem 1 measurements (ii) Terminological saturation will be predicted based on these envelope functions (iii) Terminological saturation will be checked using the remaining 70 percent of Further, the same experiments will be done using the datasets of the RAW and DAC collections to verify if the prediction of terminological saturation works reliably in complex conditions and across domains.
( ) and ℎ ( ) will be predicted based on the first 5
The objective of this position paper is the proposal of a Master project aimed at developing a rigorous theoretical framework proving the existence of terminological saturation in the sequence of incrementally enlarged sub-collections of documents describing an arbitrary subject domain. The proposal is based on the background knowledge of the OntoElect project.
In the literature study focused on this topic, we found out that little attention has
been paid, to date, to the problem of terminology saturation in textual corpora. In
particular, the only measure of such a saturation is our own prior work [
The proposal of the framework is structured along the facets of: terminological distance measure and its metric properties; the envelope functions to cope with non-monotonicity of saturation measurement; and the existence theorem that states the required conditions.
Our plans for the future work are: (i) elaborate and validate experimentally the
formal proofs of Hypotheses 1-3; (ii) modify our processing pipeline using the knowledge
from the theoretical framework; and (iii) evaluate the instrumental software in the
experiments on industrial-scale textual collections, like Springer KM [