is paper revisits the learning-to-rank approach we proposed for automatically identifying the target entity types of queries [6]. A er presenting our contributions and results, we draw on the learned lessons and encountered challenges to identify directions for future enhancements. A signi cant portion of information needs in web search target entities [11]. Entities, such as people, organizations, or locations are natural units for organizing information and for providing direct answers. A characteristic property of entities is that they are typed, where types are typically organized in a hierarchical structure, i.e., a type taxonomy. Previous work has shown that entity retrieval performance can be signi cantly improved when a query is complemented with explicit target type information, see, e.g., [1, 8, 10]. Recently, Gariglio i and Balog[5] have conducted a systematic evaluation of dimensions of type information. Identifying and exploiting target type information falls within the broad area of query understanding, which, according to Cro et al. [4], refers to process of “identifying the underlying intent of the queries, based on a particular representation.” In a realistic Web search scenario, automatically detected target types avoid the possible cognitive e ort of the user to provide type information for her query. Furthermore, they can be used as facets, for ltering the results. Motivated by the above reasons, our main objective is to generate target type annotations of queries automatically. Following the hierarchical target type identi cationtask proposed in [2], we wish to identify the most speci c target types for a query, from a given type taxonomy, such that they are su cient to cover all relevant results. We introduced a relaxation to the task de nition, by allowing for a query to have multiple target types (or none). One main contribution of this work is a test collection we built for the revised hierarchical target type identi cation task. We used the DBpedia ontology (version 2015-10) as our type taxonomy and collect relevance labels via crowdsourcing for the 485 queries in the entity ranking DBpedia-Entity collection [3]. We noted that none of the elements of our approach are speci c to this taxonomy, and our methods could be applied on top of any type taxonomy. As our second main contribution, we developed a learning-torank (LTR) approach with a rich set of features, including termbased, linguistic, and distributional similarity, as well as taxonomic features. We compared our LTR method versus two competitive baselines from the literature. One approach is an entity-centric model [2, 9, 12], which rstly ranks the entities based on their relevance to the query, then look at what types the top ranked entities have. Alternatively, a type-centric model presented in [2] ranks direct term-based representations (pseudo type description documents), built for each type, by aggregating descriptions of entities
LEARNER’17, October 1, 2017, Amsterdam, e Netherlands
Copyright ©2017 for this paper by its authors. Copying permi ed for private and
academic purposes.
of that type. ese baseline models also t, respectively, the late
fusion and early fusion design pa erns for object retrieval1[
In the forward-looking spirit of the workshop, we identify three main themes for future development. Some of these are closely tied to the target type identi cation task, while others concern learningbased approaches in general. A rst challenge is the generalization of the proposed approach to other type systems. So far, the suitability of our learning-to-rank target type detector was demonstrated only using training data manually labeled with target types from the DBpedia Ontology. It remains an open question whether these particular features are applicable to other type systems. Prior to that, an issue to be addressed is how to ease the acquisition of type labels for training. While manual annotation can be performed via crowdsourcing, with very large type systems it becomes practically unfeasible (selecting a single/handful of target type(s) from several hundred options would create cognitive overload). One possible strategy that we can imagine would be an instance of knowledge transfer from the available test collection of DBpedia target types.
As part of a more general line of discussion, another item to be mentioned has to do with how training data is obtained. It is still to be answered whether more training data helps. And, if that is the case, the acquisition of high-quality labeled data becomes a bottleneck. Automatically obtaining target labels by weak supervision, thereby avoiding human annotation e ort, may be a plausible way to alleviate this challenge.
Finally, we draw our a ention to more recent developments with an increasing impact in many problem domains within information retrieval. Speci cally, using features obtained by representation learning via deep arti cial neural networks. Our approach exploits semantic similarities purely based on pre-trained word embeddings. What we do here, using distinguished neural features in a LTR approach, has become a noticeable trend in recent years. e question that arises is whether it is possible to embrace an alternative, fully neural learning approach. We emphasize that this question applies beyond our speci c task of target types identi cation. Indeed, it likely extends to a whole range of tasks where the current state of the art is constituted by a learning-to-rank approach, using a manually engineered set of features.