LDA [BNJ03] is currently the most common probabilistic topic model compared to similar models, such as probabilistic Latent Semantic Analysis (pLSA) [Hof01], with a wide range of applications [Ble12]. LDA statistically discovers hidden topics from documents as features to be used for di↵erent tasks in information retrieval (IR) [WC06, WMW07], information filtering (IF) [GXL15] and for many other text mining and machine learning applications. LDA represents documents by a set of topics, and each topic is a set of semantically related terms1. Thus, it is capable of clustering related words in a document collection, which can reduce the impact of common problems like polysemy, synonymy and information overload [AZ12].

The core and critical part of any text FS method is the weighting function. It assigns a numerical value (usually a real number) to each feature, which specifies how informative the feature is to the user’s information needs [ALA13]. In the context of probabilistic topic modelling in general and LDA specifically, calculating a term weight is done locally at its document-level based on two components; the term local documenttopics distributions and the global term-topics assignment. Therefore, in a set of similar documents, a specific term might receive a di↵erent weight in each single document even though this term is semantically identical across all these documents. Such approach does not accurately reflect on the semantic meaning and usefulness of this term to the entire user’s information needs. It badly influences the performance of LDA 1In this paper, terms, words, keywords or unigrams are used interchangeably. for FS as it is uncertain and dicult to know which weight is more representative and should be assigned to the intended term. Would it be the average weight? The highest? The lowest? The aggregated? Several experiments in various studies confirm that the localglobal weighting approach of the LDA is ine↵ective for relevant FS [GXL15].

Given a document set that describes user information needs, global statistics, such as document frequency (df), reveal the discriminatory power of terms [LTSL09]. However, in IR, selecting terms based on global weighting schemes did not show better retrieval performance [MO10], because global statistics cannot describe the local importance of terms [MC13]. From the LDA’s perspective, it is challenging and still uncertain on how to use LDA’s local-global term weighting function in a global context due to the complex relationships between terms and many entities that represent the entire collection. A term, for example, might appear in multiple documents and LDA topics, and each topic may also cover many documents or paragraphs that contain the same term. Therefore, the hard question this research tries to answer is: how to generalise the local topic weight (at document level) and combine it with global topical statistics such as the term frequency in both topics and relevant documents for more discriminative and semantically representative global term weighting scheme?

The aim of this research is to develop an e↵ective topic-based FS model for relevance discovery. The model uses a hierarchical framework based on ERS theory to assign a more representative weight to terms based on their appearance in LDA topics and all relevant documents. Therefore, two major contributions have been made in this paper to the fields of text FS and IF: (a) A new theoretical model based on multiple ERS [Mol06] to represent and interpret the complex relationships between long documents, their paragraphs, LDA topics and all terms in the collection, where a function describes each relationship; (b) A new and e↵ective term weighting formula that assigns a more discriminately accurate weight to topical terms that represent their relevance to the user information needs. The formula generalises LDA’s local topic weight to a global one using the proposed ERS theory and then combines it with the frequency ratio of words in both documents and topics to answer the question asked by the authors. To test the e↵ectiveness of our model, we conducted extensive experiments on RCV1 dataset and the assessors’ relevance judgements of the TREC filtering track. The results show that our model significantly outperforms all used baseline FS models for IF despite the type of text features they use (terms, phrases, patterns, topics or even a di↵erent combination of them).

In the literature, there is a significant amount of work that extends and improves LDA to suit di↵erent needs including text FS [ZPH08, TG09]. However, our model is intended for IF, and, to the best of our knowledge, it is the first attempt to extend random sets [Mol06] to functionally describe and interpret complex relationships that involve topical terms and other entities in a document collection to enhance the semantic of topical words for relevance FS. Relevance is a fundamental concept in both IR and IF. IR mainly concerns about document’s relevance to a query for a specific subject. However, IF discusses the document’s relevance to user information needs [LAZ10]. In relevance discovery, FS is a method that selects a subset of features that are relevant to user’s needs and thus removing those that are irrelevant, redundant and noisy. Existing methods adopt di↵erent type of text features such as terms [LTSL09], phrases (n-grams) [ALA13], patterns (a pattern is a set of associated terms) [LAA+15], topics [DDF+90, Hof01, BNJ03] or a combination of them for better performance [WMW07, LAZ10, GXL15].

The most ecient FS methods for relevance, are the ones that are developed based on weighting function, which is the core and critical part of the selection algorithm [LAA+15]. Using LDA words weighting function for relevance is still limited and does not show encouraging results [GXL15] including similar topicbased models such as the pLSA [Hof01]. For better performance, Gao et al (2015) [GXL15] integrate pattern mining techniques into topic models to discover discriminative features. Such work is expensive and susceptible to the features-loss problem and also might be impacted by the uncertainty of the probabilistic topic model. ERS is proven to be e↵ective in describing complex relations between di↵erent entities and interprets them as a function (weighting function) [Li03]. Thus, the ERS-based models can be used to weight closed sequential patterns more accurately and thus facilitate the discovery of specific ones as appears in [ALX14]. However, selecting the most useful patterns is challenging due to a large number of patterns generated from relevant documents using various minimum supports (min sup), and also may lead to feature-loss. 3

For a given corpus C, the relevant long documents set D✓ C represents user’s information needs that might have multiple subjects. The proposed model uses D for training where each document dx2 D has a set of paragraphs P S and each paragraph has a set of terms T . ⇥ is the set of all paragraphs in D and P S✓ ⇥. A set of terms ⌦ is the set of all unique words in D. The proposed model uses LDA to reduce the dimensionality of D to a set of manageable topics Z, where V is the number of topics. LDA assumes that each document has multiple latent topics [GXL15], and defines each topic zj 2 Z as a multinomial probability distribution over all words in ⌦ as p(wi|zj ) in which wi2 ⌦ and 1  j V such that P|⌦ | p(wi|zj )=1. LDA i also represents a document d as a probabilistic mixture of topics as p(zj |d). As a result, and based on the number of latent topics, the probability (local weight) of word wi in document d can be calculated as p(wi|d)= PV

j=1 p(wi|zj )⇥ p(zj |d) . Finally, all hidden variables, p(wi|zj ) and p(zj |d), are statistically estimated by the Gibbs sampling algorithm [SG07]. 3.2

The proposed model (Figure 1) deals with the local weight problem of terms that is assigned by the LDA probability function (described in section 3.1) by exploring all possible relationships between di↵erent entities that influence the weighting process. The targeting entities in our model are documents, paragraphs, topics, and terms. The possible relationships between these entities are complex (a set of one-to-many relationships). For example, a document can have many paragraphs and terms; a paragraph can have multiple topics; a topic can have many terms. Inversely, a topic can cover many paragraphs, and a term can appear in many documents and topics.

In this model, we proposed three ERSs to describe such complex relationships, where each ERS can be interpreted as a function by which we can determine the importance of the main entity in the relationship. Then, the proposed ERS theory is used to develop a new weighting scheme to accurately weight topical words by generalising the topic’s local weight, and, then, combine it with the frequency ratio of words in both documents and topics. Let assume we have a set of topics Z={z1, z2, z3, . . . , zV } in ⇥ and let D= {d1, d2, d3, . . . , dN } is a set of N relevant long documents. Each document dx consists of M paragraphs such as dx= {p1, p2, p3, . . . , pM }. A paragraph py consists of a set of L words, for example, py= {w1, w2, w3, . . . , wL}. A word w is a keyword or unigram, where the function words(p) returns a set of words appear in paragraph p. A topic z can be defined as a probability distribution over the set of words ⌦ where words(p)✓ ⌦ for every paragraph p2 ⇥.

For each zi2 Z, let fi(w, zi) be a frequency function on ⌦, such that ( zi)={w|w2 ⌦ , fi(w, zi) 0} while the inverse mapping of is defined as 1 : ⌦ ! 2Z ; 1(w)={z2 Z|w2 ( z)}. Also, for each dj 2 D, let fj (w, dj ) be a frequency function on ⌦, such that ( dj ) = {w|w2 ⌦ , fj (w, dj )>0} while the inverse mapping of is defined as 1 : ⌦ ! 2D; 1(w)={d2 D|w2 ( d)}. These extended setvalued mappings can decide a weighting function on ⌦, which satisfies sr :: ⌦ ! [0, +1 ) such that ing the conditional probability function Pxy(z|dxpy) as 1 : ⇥ ! 2Z⇥ [0,1]; 1(dxpy)={(z1, Pxy(z1|dxpy)), . . .}.

Similarly 2 is also proposed to describe the relationship between topics and terms using the defined frequency function fi(w, zi) as 2 : Z ! 2⌦ ⇥ [0,+1 ); 2(zi)={(w1, Pi(w1|zi)), . . .}.

Lastly, 3 is also proposed to describe the relationship between documents and terms using the defined frequency function fj (w, dj ) as 3 : D ! 2⌦ ⇥ [0,+1 ); 3(dj )={(w1, fj (w1, dj )), . . .}

Based on the inverse mapping described above, we have 1 1, 2 1 and 3 1. 1 1 describes the inverse relationships between topics and paragraphs using the probability function Pz(zi) such that 1 1(z)={dxpy|z2 1(dxpy)} while 2 1, on the other hand, describes the inverse relationships between terms and topics using fi(w, zi) function such that 2 1(w)={z|w2 2(z)}. 3 1 describes the inverse relationships between terms and documents using fj (w, dj ) function such that 3(w)={d|w 2 3(d)} To estimate the generalised topic weight in D, we need to calculate the probability of each topic Pz(zi) in each paragraph of document d and similarly for all documents in D based on 1 1 in which we assume P⇥ (dxpy) = N1 , where N is the total number of paragraphs as follows:

Pz(zi) =

P To verify the proposed model, we designed two hypotheses. First, our ERS model can e↵ectively generalise the topic’s local weight that is estimated from all documents paragraphs. The generalisation has led to a more accurate term weighting scheme especially when it is combined with the term frequency ratio in both documents and topics. Second, our model, overall, is more e↵ective in selecting relevant features than most, state-of-the-art, term-based, patternbased, topic-based or even mix-based FS models. To support these two hypotheses, we conducted experiments and evaluated their performance. 5.1

This paper presents an innovative and e↵ective topicbased feature ranking model to enhance the semantic of topical words to acquire user needs. The model extends random sets to generalise the LDA topic weight at the document level. Then, a term weighting scheme is developed to accurately rank topical terms based on their frequent appearance in the LDA topics distributions and all relevant documents. The new calculated weight e↵ectively reflects the relevance of a term to user’s information needs and maintains the same semantic meaning of terms across all relevant documents. The proposed model is tested for IF on the standard RCV1 dataset, TREC topics, five di↵erent performance measurement metrics and eight stateof-the-art baseline models. The experimental results show that our model achieved significant performance compared to all other baseline models. [ALA13] line models on all performance measures. A model’s result is considered significantly di↵erent from other model’s if the p-value is less than 0.05 [Wil45].Clearly, the p-value for all metrics is largely less than 0.05 confirming that our model’s performance is significantly di↵erent from all baselines. This shows that our model gains substantial improvement compared to the used baseline models. Based on the results presented earlier, we are confident in claiming that our extended random sets model can e↵ectively generalise the local topic weight at the document level in the LDA term scoring function and, thus, provide a more globally representative term weight when it combined the term frequency in document and topics. Also, our model is more e↵ective in selecting relevant features to acquire user’s information needs that represented by a set of long documents. [DDF+90] Scott Deerwester, Susan T Dumais, George W Furnas, Thomas K Landauer, and Richard Harshman. Indexing by latent semantic analysis. Journal of the American society for information science, 41(6):391, 1990. [ALX14] [AZ12] [Ble12] [BNJ03] [BV00]