This paper aims at the generationdoefnials of expectations in the domain of vehicle reviews. A denial of expectation expresses an apparent contradiction between some probabilistically motivated rule and a current circumstance expressed by a contrastive sentence. For generating such an argumentative sentence, we present a new approach for content selection in a neural data-to-text generation framework. In addition to selecting relevant information from tabular data that should appear in the text, further methods are required for determining evaluations that are rooted in this data, but express individual appraisals of the respective vehicle. We show how a content selection module is able to decide when expressing a denial of expectation. We use multi-label and binary classification for content selection on automatically extracted training data and Random Forest Regression with varying knowledge limitation for predicting expectations about feature values. These predictions are compared to manually annotated corpus instances of contrast relations in order to show that the concept of denial of expectation is a reasonable approach to determining contrasts and evaluative content at the early stage of content selection.
When arguing in favor of (or against) a new device in order to convince a potential buyer to purchase (or disregard) that device, it is often useful to compare features of that device with feature-related expected values, and to indicate possible consequences of discrepancies between real and expected value. One of the linguistic means for realizing this is the so-cdaellneidal of expectation, which is an apparent contradiction between some rule, be it grounded in domain knowledge, personal experience or norms of social behaviour, and a current circumstance expressed in the corresponding sentence. In sentences like: 1. The sports car is not that big in terms of external dimensions, but it does weigh quite a bit. 2. Although the sports car is not that big in terms of external dimensions, it does weigh quite a bit. 3. (Un)fortunately, the sports car does weigh quite a bit. we have diferent linguistic realizations of the denial of expectation. Exam1p)lee x(presses a contradiction that is based on technical specifications and their consequences for a car’s weight and the weight of the corresponding sports car. According to the external dimensions, weight is expected to be lower than the real value. However, the conjuncbtuitonis ambiguous, it can also be used to express a contrast that is grounded in other means than rule violation (as in John is short but Lea is tall). Contrary tbout, the concessive conjunctionalthough in example (2) can only be used to express the denial of expectation. Adverbs l(iukne)fortunately are typically not considered as items expressing a denial of expectation, but in fact it is possible that they are grounded in the same semantic mechanism the conjunctions are based on. For example, unfortunately expresses that some part of the proposition that is in the scope of this adverb is typically evaluated neutrally, and in this special case it has been shifted to the negative end of this evaluation scale. Example3() can be interpreted in the following way: The writer’s use of this sentence is based on her estimation of the weight of sports cars, which is grounded in their typical dimensions and other technical factors, but the sports car that is relevant here violates this expectation. It should be noted that other uses of adverbs of this type are not grounded in a denial of expectation, but in other incompatibilities. For exa munpfloer,tunately this car is red might just express the diference between the speaker’s colour preference and the actual colour of the car. Therefore, in this paper we consider only those adverb uses with a reference to a denial of expectation.
A denial of expectation as a special case of linguistically expressed contrast is inherently argumentatively motivate1d],[since the addressee’s inferred opposition between both conjuncts of such a sentence is quite decisive for the evaluation of the object or state of afairs at hand. Using Toulmin’s argumentation schemes2][, examples (1) and (2) are claims based on expected and real values for the car as grounds, and the underlying expectation functions as the warrant.
In this paper, we are analyzing denials of expectations from a Natural Language Generation (NLG) perspective. Hence, we do not determine plausible oppositions for a given denial of expectation, but we are motivating decisions for realizing such a sentence. As application domain, we are using driving reports and an associated database with technical specifications for this. During content selection – the first stage in a pipelined NLG system – possible data correlations must be determined, together with a violation of this correlation in the current message to be verbalized.
In traditional NLG pipelines for data-to-text generation, an interpretation module that encodes
domain-expert knowledge decides which information should be contained in a message within
the document plan. The avoidance of hand-coding a heuristic selection module is desirable for
time and efort reasons and can be realized with the help of data-driven learning techniques.
A premise to this is the availability of a reasonable amount of data. In general, data-to-text
generation is the field of transforming tabular data into surface t3e,x4t,5[]. Due to the rise
of neural networks, traditional pipeline approaches were abandoned in favor of end-to-end
trainable encoder-decoder network6s, 7[
As an improvement in regard to content and informational correctness, copy mechanisms
[
Another important point to mention is that most models are trained to generate short
phrases only, e.g. short biographies from Wikipedia tabl6e,s1[
Ferreira et al[.4] propose a re-modularization of neural generation networks, chaining separately trainable and evaluable networks that are specialized for the diferent tasks of content selection, document planning and surface realisation. They show that these pipelined neural generation models outperform end-to-end networks, especially on unseen data, where the latter tend to produce topic-unrelated, incoherent texts and hallucinations. Turning back to a pipelined neural generation system necessitates to find a suitable content selection model that determines what the document plan shall contain.
Many papers have been published on research how to punish toxicity and inappropriate language use in neural NLG, resulting in more neutral word choice, but to our knowledge, the controlled generation of a denial of expectation as a model of contrast relations and evaluative content has not yet been dealt with in the context of neural NLG.
We present a classification approach to content selection on automatically augmented data which predicts what information shall be present in the document plan. Furthermore, we use regression models in order to predict, given diferent knowledge limitations, whether the information to be produced in the surface text agrees with expectations about the information in order to determine in a data-driven manner whether expressive content is legitimate to use for putting the information into perspective or not.
A domain for data-to-text generation needs tabular data from which surface text shall be produced. The German Automotive Club ADAC supplied us with a proprietary data set of 1300 road test reports with 3000 to 6000 words including the respective database with 127 technical and economic properties for each vehicle. The road test reports are written by domain experts and contain a subset of the properties given in the database as well as surplus information, e.g. a guess of resale values of used cars, or their opinion on vehicle characteristics. Therefore, the texts also include evaluative information which is naturally grounded in subjective estimation. These evaluative decisions, in turn, will be reflected in the use of corresponding evaluative • The A3 completes the intermediate sprint from 60 to 100 km/h in a brisk 6.1 s. • The petrol engine completes the intermediate sprint from 60 to 100 km/h in 5.6 s. • The 108 kW/147 hp mean that the simulated overtaking maneuver (sprint from 60 to 100 km/h) ends after just 6.0 seconds. lexical items like adverbs or conjuncts. There is no information in the database which properties were named in the respective road test report. In order to produce a data set that is usable for a learning-based content selection module, we need a function that maps a road test review to a binary decision on properties in the database that either marks the absence of the respective piece of information from the text w0itohr its presence in the road test report 1w. iTthe result is a tripl(e , , ) for each road test repo rtconsisting of the te x t,a database row , which is a 127-tupel of mixed type information on the vehicle, and a mapwhich is a 127-tupel of binary values for the presence or absence of each property .inWe selected a subset of 15 properties from the database which are related to the technical details on engine, chassis and driving performance. In order to determ in,ewe extracted a sample of 200 database rows and respective test reports and build a heuristic information extraction module, which uses the tracking of keywords and pattern matching for determining the presence of a piece of information in the text. At the same time, we manually annotated a set of 50 texts from the corpus in regard to the presence of the 15 target features we selected beforehand. This set of annotated texts serves as the gold corpus for evaluating the heuristic IE module. The set of 200 reports for building the IE heuristic and the set of 50 texts for manual annotations are disjoint, and only the remaining 1050 road test reports and database rows are used for the machine learning models of content selection and denial of expectation.
For information extraction, we searched the subset of 200 reports for keywords and patterns in which the 15 target properties are verbalized in order to extract rules in form of regular expressions. Fortunately, across test reports, the verbalisation of each piece of information is relatively uniform, despite the self-evident diferences between properties (see T1a).ble
As the examples in Table1 show, the data poinatcceleration is stereotypically verbalized as a quantity of seconds the car needs to accelerate from 60 to 100 km/h. Additionally, key words like sprint and overtaking maneuver often indicate the target informatioancceleration. Not all of the features are as straightforward in extractiaocncealesration. The featuredisplacement, for example, occurs in highly aggregated compounds liktehe 2.5 litre turbo combustor motor and is less accurately extractable.
This heuristic information extraction approach is applied to the remaining 1050 pairs of road reports and database rows in order to produce the necessary training instances for neural content selection. The output is a table containing 15 columns such that for each road test report there is a 15-tuple of binary values that indicate whether the respective property was found in the text or not.
Content selection has only little bearing in neural end-to-end networks and encoder-decoder
models, especially in regard to the fact that the generated texts often only comprise a few
sentences or a single paragraph6,[
Our content selector takes the very first step in an NLG pipeline and decides which pieces of information shall be integrated into the document plan, which in turn, after adding the database values for the respective properties, can also be represented as RDF triples, based on the ADAC database and the human written car reviews. We assume that the authors of the car reviews have domain-specific reasons for choosing certain properties from the database and leaving others aside. For exampleh,orse power is used in nearly all texts, whereavsalves are never even mentioned. Hence we assume that given the texts, the database entries, and our generated training data, we are able to train a neural classifier that is capable of predicting which features should be produced. This would mean that the classifier could encode domain expert knowledge on what to say by finding the same patterns in the technical data as the experts would do.
Before using classification, we tried to determine feature importance patterns in the data. Assuming that some of the technical details are interdependent, not all pieces of information are relevant for predicting the presence of each property. This is also reasonable from an engineering perspective, since properties of the motor might condition each other, while these have no influence on the design of the interior.
The heatmap in Figure1 reveals some interesting relations. The slightly recognizable line, which resembles a linear function with gradient -1, is self-evident, since each piece of information is mapped onto itself. More important are the really strong outliers beside this line, showing that each data point only has a few, but important points that condition the target point’s presence.
According to the feature importance and usability of feature type, we selected 40 categorical
and numerical values for predicting the 15 properties. The 15 data points were first
labelencoded and then one-hot encoded. By applying the heuristic information extraction module to
the texts, we gained about 931 usable text array pairs. The remaining car reviews only provided
partial or fragmental test reports, which we therefore excluded. These arrays contain 15 binary
values [
An important point to mention here is that the input to the classifier is not a tensor with 15 elements of the binary decision on presence, but the real values of the 40 relevant properties, for e.g. , , ℎ and so on taken from the database. The network is therefore trained on determining the presence or absence of a piece of information on the basis of the vehicle’s data, and not on the presence or absence of the other data points.
A multi-label classifier [16, 17] with a dense network of 5 layers (Keras, Tensorflow), RELU activation function and sigmoid activation as final layer did not lead to satisfying results. We received only a few percent of accuracy per feature when predicting all 15 target features at once. Reasonable accuracy is only reachable when performing binary classification for each target feature separately.
For surface realization we used the multi-lingual T5 (MT158,][); the model is fine-tuned on
the German webNLG corpus (1[
Input: type: combustor; motor power: 130 HP; cubic: 1.3L; torque: 240 Nm; ...
Content: torque, supercharging, motor power Planning: Renault Clio TCe 130 GPF| torque| 240 Nm
Renault Clio TCe 130 GPF| supercharging| Turbo Renault Clio TCe 130 GPF| motor power|130 PS (torque, supercharging, motor power (HP)) Output: Der Turbobenziner leistet 130 PS und entwickelt ein maximales
Drehmoment von 240. ‘The turbocharged petrol engine makes 130 PS and produces a torque of 240.’ Reference: Der 1,3 Liter Vierzylinderbenziner leistet dank Turboaufladung 130 PS und entwickelt ein maximales Drehmoment von kräftigen 240 Nm, das bereits bei 1.600 Umdrehungen pro Minute bereitsteht. ‘The 1.3 liter four-cylinder petrol engine has, thanks to supercharging, 130 PS and produces a maximal torque of powerful 240 nm, which is available already at 1.600 turns per minute.’
Vehicle testers have certain expectations about features of a vehicle when writing the reviews. These are based on domain-specific experience, such that experts have an intuition what weight a sports car should have, given a set of exterior dimensions, motor block size and so on. Many evaluations and contrasting details are formulated in regard to technical details which contradict each other or do not agree to the expected value, both in positive and negative polarity.
A denial of expectation when the expected value does not agree to the real value may be a trigger for generating a concessive or evaluative marker that signals this mismatch to the reader. Independent of the question how such a contrast or evaluation shall be lexicalised, the underlying semantic mechanism for determining such a mismatch is data-driven and is, therefore, installed at the interface of content selection and document planning.
A straightforward approach to predicting values, given a set of features, is regression. Using a Random Forest regression implementation (Scikit, 100 estimators), which often faces limitations for linguistic data2[0], but is a reasonable choice for our task at hand, we predict numerical values for each of the 15 target features on the basis of the set of residual features in the database (excluding the target feature). By comparing these predicted values to the real values, we can determine whether there is a significant deviation that may trigger a contrast relation or the usage of an evaluative adverb.
From the corpus of 1300 car reports we automatically extracted instances of concessive markers, namelyobwohl (‘although’), and the evaluative adverbesrstaunlicherweise (‘surprisingly’), bedauerlicherweise (‘regrettably’) anldeider (‘unfortunately’), and filtered them by the assessibility of the contrasting information in our database. Contrast relations, to which the denial of expectation applies, but which cannot be modelled in a data-driven way, dealed with subjective driving experience, e.g. the adjustability of the arm rest or the noise level of the motor, or would necessitate additional reasoning or information we did not have access to. 19 instances of concessives and evaluative expressions remain for analysis, listed in Ta4b.le Note that we confined the evaluative adverbs to those cases with a denial of expectation as underlying contrasting motivation. This is a small number of instances, but it relates to the fact that we only searched for the specific markers above. The polarity defines either positive or negative direction of the denial of expectation, whereas the arrows beside the source and target properties names indicate whether their values need to be high↑(-)eorr)(low(-er) ↓() in order to match the polarity.
Furthermore, Tabl4e lists four numerical values: the real values for the target data point which were retrieved from the database and three possible thresholds for modeling denial of expectation. These are the predicted value on the basis of all 40 relevant data points, the ‘naive’ prediction where the input to the regressor is limited to the source information the authors name in their contrasting relation, and finally the average value of the target data point across all vehicles in the database. The numerical values that difer with correct sign from the real value such that the denial of expectation can be captured by the threshold are printed in boldface.
A diference to the real value can be interpreted as a denial of expectation. The expected value is higher or lower than the real value - the sign of the diference and semantic relation between the source and target features, e.g. hig hermay entail higher , determine the polarity of the whole expression. The polarity (+/-) determines whether the denial is a surprise (+) or a disappointment (-). For example, instance (8) in Ta4bclean be paraphrased as although the car has many horse powers, the mileage is comparably low. The expected values (both regression values, 5.94 and 6.1) are higher than the real value (5.9), meaning that the real falls below what is expected for a car of the respec t ive. Since less is positive, the polarity of the whole expression is positive.
Before going into the analytic details, a few technical relations need clarification in order to understand the expression for the target proprearntgye(↑)@mileage(↓). Five of the contrasting relations are established between the size of the fuel tank and the possible range of the car, indicating that despite a given comparably small tank, a high range is possible. The range as a numeric value is listed in the database only for electric cars, but for combustors, the range can be inferred by the tank size and the minimal consumption. Range and mileage are anti-proportional, meaning that the smaller the mileage, the higher the possible range at constant tank size. Our target feature is therefore the fuel consumption, where less is generally considered better. Furthermoarcec,eleration should be explained. Acceleration is quantified in seconds needed to reach a certain velocity, e.g. 100 km/h. Higher acceleration leads to fewer seconds needed.
In Table 2 the results of the heuristic information extraction approach tested on the gold corpus of 50 reports are listed. We received perfect or near perfect valuheosrfsoerpower, torque, fuel consumption, acceleration and brakes. While velocity and transmission still show good recall and precision values around 0.85d,isplacement and supercharging reveal that the highly aggregated expressions for the motor description are too diverse to be captured as accurately as the other feature horse powers torque fuel consumption acceleration motor type price brakes max. velocity transmission cylinders displacement supercharging weight valves attributes. Fovralves we cannot ofer data since the feature did not occur in the reviews at all, which shows its irrelevance to the reader from the domain experts’ perspective. The low recall and precision values owfeight is due to the rather indefinite nature of the car’s weight. Diferent weight-related features exist, e.g. trailing load, maximal allowed weight of cargo on the roof or in the trunk, and sums of varying subsets of those are used in the text. The weight of the car is therefore complicated to distinguish from other weight-related expressions.
For evaluating the content selection module, we used 8-fold cross validation and calculated the average scores. In Table3(), the evaluation metrics for each feature are listed separately. The featuresacceleration and . , which were very well recognized by IE, are the properties with worst prediction accuracy and only marginally better than random. The data pointssupercharging and displacement agree in accuracy with the low precision and recall values the IE already suggestedH.orse power and valves are predicted with best accuracy, which is explainable by the simple categorical use of the former and the utter absence of the latter, making their selection deterministic.
The output of the system is evaluated against the original human-produced car reviews using BLEU. Surprisingly, the model reaches 0.57 BLEU despite the small data set. Depending on the order of triples, the BLEU score may drop to 0.19 (random order), showing that the linear order, based on topic extraction, is of essential importance for the successful transformation into surface text. This is a clear indication that the discourse planner plays a central role in the acceptance of the generated texts.
The graphs show the error of the Random Forest Regression model for the two fefauteulres consumption and acceleration, which are from the upper and the lower end, respectively, of the accuracy scale for content selection. Figu3rsehows that the predicted values agree perfectly with the real values of fuel consumption, and the confidence intervals are uniform, at least in the area lower than 9 liters of fuel per 100 km. Above this threshold, only few data points are available in the data, for which the prediction is much less accurate and the confidence much id token pol. source property 1 obwohl + fuel tank(↓) 2 obwohl + fuel tank(↓) 3 obwohl + fuel tank(↓) 4 obwohl + weight(↑) 5 obwohl + weight(↑) 6 obwohl - dimensions(↓) 7 obwohl + fuel tank(↓) 8 obwohl + hp(↑) 9 obwohl + weight(↑) 10 obwohl + fuel tank(↓) 11 obwohl + weight(↑) 12 obwohl + weight(↑) 13 leider - – 14 leider - – 15 leider - – 16 obwohl + Supercharging(↓) 17 obwohl + displacement(↓) 18 obwohl + supercharging(↓) 19 erstaunl. + hp(↓) target property real range(↑)@mileage(↓) 4.1 range(↑)@mileage(↓) 6 range(↑)@mileage(↓) 3.9 vehicle payload(↑) 520 acceleration(↑) 7.8 weight(↑) 2173 range(↑)@mileage(↓) 3.7 mileage(↓) 5.9 acceleration(↑) 7.6 range(↑)mileage(↓) 5.1 mileage(↓) 4.8 acceleration(↑) 5.2 price(↑) 31900 price(↑) 26295 mileage(↑) 5.2 torque(↑) 213 max. velocity(↑) 182 acceleration(↑) 6.1 trailing payload(↑) 4900 weaker. A less accurate picture is drawn by the predictionsacfocerleration, which are still close to the real values, but fewer predictions are perfectly on point. There is also more variance in the confidence intervals, but in contrast ftuoel consumption, there are fewer outliers and the values are spread across the interval of 2.5 seconds to 15 seconds in a more balanced way.
On average, regression seems to yield good results when predicting the 15 target features on the basis of the 40 most relevant features in the database. But does regression model the domain experts’ decisions in a suficiently accurate way? A closer look at instances of denial of expectation in the corpus sheds light on the relation of author expertise and the motivation for generating this kind of contrast.
As the column of predicted values in Tabl4edisplays, many of the expected values related to fuel consumption are modeled nearly perfect on point, leaving no or only marginal diferences. Reducing the input features of the regression model to the source information causes more deviation, which allows us to model contrast with ‘naive’ prediction with limited knowledge. The average mileage across all database entries is also a good threshold estimator. The outlier in regard to mileage is instance 2. The original text gives a reasonable explanation for this - the scale by which the positiveness of reduced fuel consumption is explained is a direct comparison to the previous version of the same car model. The newer version has a smaller tank, but longer range. Therefore, the contrast is triggered by the direct comparison of tank and mileage of two versions of the same car model.
All contrasts concerninagcceleration can either be modeled with the predicted value or the naive prediction that is limited to the knowledge given by the respective source information. The average ofacceleration also captures all instances correctly. Examp3l,estated in sectio1n, is the only instance ofobwohl with negative polarity, denying the expectation of a lower weight given the comparably small dimensions. This contrast is also correctly modeled with all of the three possible thresholds. The evaluative adverlebider, which semantically expresses a negative point, is not correctly predictable with the regressed values. Only for the mileage-related instance (15) the average seems to be a reasonable threshold. The other instancleesidoefr deal with price, which is highly dependent on build quality, brand and prestige and therefore the possibly most problematic feature for evaluative content. The average price is not a good estimator in this case. Instances (16) and (18), contrasting the lacksuinpercharging with surprisingly good torque and acceleration values, are captured by naively predicted values, the former even by the value predicted on the whole relevant dataset. Example (17) contrasts the minor motor displacement with a surprisingly highmaximum velocity, which is captured by both predictions, while the average value is far away from proximity. The contrast relation in (19), marked by erstaunlicherweise (‘surprisingly’), deals with an extraordinarily high payload given a rather low . All thresholds capture this contrast correctly, while the regressed value with full input features is still the closest to the original value. The usageersotafunlicherweise instead of using obwohl may indicate that the authors have a proper classification of contrast markers and evaluative adverbs that express a certain degree of deviation from the expectation – the distance to the expected value in either positive or negative direction may trigger the usage of an expression that semantically quantifies this distance.
First, the heuristic approach to information extraction and data augmentation introduces noise into the training data, possibly to the same extend to which errors occur in the manually annotated data. The consequence to be drawn from this is that any network trained on this data may also incorrectly predict on the basis of error-prone input. The only limitation this imposes on the neural models is that it cannot outperform the correctness of neither heuristic nor the human annotations, which is an inherent issue of the challenge to apply machine learning models to noisy data.
Regarding the content selection module in general we can state that the performance of the content classifier is acceptable, given the small data set and the complexity of the task of both IE and annotation, which produced the model input. The content selector can, for unseen data, generalize from encoded input properties, and predict the presence of the pieces of information in the text to be generated. Although the accuracy values for some properties are still imperfect, this is a huge leap towards modeling domain expert knowledge for content selection with minimal resources of annotations. As already mentioned, the error rate of the heuristic IE approach is passed on to the content selector, which also entails that its accuracy will presumably improve with increasing recall and precision of its input.
In regard to surface realisation, it is important to mention that using Transformer models for producing surface text has a downside; neural models often hallucinate fact2s2[] and diferent methods have already been applied to prevent 2it3][. A special case is expressive content such as evaluative adverbs and contrast relations, which go beyond the purely propositional content. Intentional production of such expressive, non-propositional, constructions means that any sort of non-at-issue content should be suppressed for decoding where the input does not demand for such verbalisation. Measures against hallucination intend to do that, but for expressives it would work on word level only. Measures against hallucination that base on fact-checking and ranking cannot be applied here, because non-at-issue content only puts facts into perspective; it reflects the author’s opinion. In training instances, where the input data motivate expressive content, the expressive content in the text output is either preserved, or it has been enriched with it in case no expressives are present. This training includes the systematic annotation of non-at-issue content24[]. Further research will show how such an approach dovetails with methods to minimize fact hallucination.
Summing up the empirical findings on modeling the denial of expectation, we can state that 42% of the evaluative expressions and contrasts we dealt with are explainable through regression of the target feature with full domain knowledge. Limiting the knowledge to the source features of the contrasting relations improves the coverage to 86%, excluding the insta nleciedsero, f where no source features are mentioned explicitly (72% including 13, 14 and 15). The average value as a threshold scores diferently for diferent features. Although no reliable conclusion can be drawn from the statistics due to data sparseness, features like and seem more comparable to a global average than details like and .
A very interesting point is the diference between predictions on the full set of input features and the knowledge limitation of ‘naive’ prediction the authors of the report anticipate. The empirical data shows that limiting the input knowledge of the regression to the features on the basis of which the car reviewers make their assumptions doubles the percentage of correct predictions. Due to data sparseness, we only have a very small number of instances we can use for evaluation. Nevertheless, the score of 42% and 86% respectively exceed our expectations and are a good indicator that the acquisition of more instances for evaluation will validate our ifndings and the adequateness of our modeling approach.
A topic we did not yet address in regard to the generation of denials of expectation are false positives. Regression may determine a trigger for evaluative expressions where no such expression shows up in the corpus of texts. There might be non-linguistic motivations behind (not) producing evaluative content, e.g. an economic bias or personal preferences. These motivations cannot be modeled, but may explain a mismatch between observed and data-driven usage of evaluatives. This amounts to the necessity, once we have built a data set of verbalisations of properties containing both, neutral ones and those enriched with evaluative content, to determine the minimal diference between expected value and real value that triggers a contrast or evaluative adverb, such that the distribution of evaluatives in the empirical data best matches their distribution in the document plans.
With more data, we are sure to be able to determine these property-wise minimal deviations and deploy them for determining evaluative content at content selection level.