Exploiting ERP Systems in Enterprise Search

Abstract

Enterprise resource planning (ERP) systems are the core of many companies: they contain entities which are the focus of enterprise searches [5]. In this paper, a model which exploits those entities to improve the search experience of enterprise users is proposed. Speci cally, a graph knowledge base called entity graph is de ned. It is used both to o er a novel data exploration experience that re ects the business processes and to improve the search accuracy contributing to the score of a search result into a weighted linear model. The applicability of the model is demonstrated by implementing an enterprise search prototype called SeNSE (Skyline eNterprise Search Engine).

Keywords

• Enterprise search
• Entity centric retrieval
• Entity graph exploration

Introduction

According to [ 5 ], enterprise search on small data is much more important than web search on big data for many companies, but this issue still receives little attention from the information retrieval community. However, last advances in enterprise search focus on the extraction of concepts or entities from enterprise data, which might be a promising way to enhance the search performances. Among the di erent sources of information of an enterprise (such as relational databases, le system documents, web pages, etc.), a key role is played by ERP systems [ 10 ], which are typically composed of several modules, such as sales, nance and production, or business intelligence. Since ERP systems capture information among modules and provide an integrated view of information through enterprise business processes, we decide to model its main entities and the related entity links. The latter are arranged in a graph knowledge base that we called entity graph (EG), which can be used to boost the search results and to explore data in a way that re ects the business processes and the work- ow of enterprise users. Despite some state-of-the-art enterprise search systems, based on entities [ 1,5 ], ours are a small number of complex concepts (such as orders, invoices, estimates, etc.). This choice has two main advantages: from the enterprise user point of view, entities and their links can be displayed as a meaningful graph that can be exploited for the everyday work; from the machine learning point of view, since we have a small number of entities and entity links, it is easier to assign them weighs that can be used to improve search results.

Our contributions are summarized as follows: (1) to our best knowledge, we are the rst one to use ERP entities and their relations to build a knowledge base to improve the search experience; (2) we propose a novel data navigation model based on the EG; (3) we build an enterprise search prototype that demonstrate the applicability of our model. 2

System Design

ERP is the core of a company [ 10 ] because it contains most of the fundamental entities searched by enterprise users. Despite that, most of the enterprise search solutions are not able to achieve satisfying search performances because they still aim at working at word level. However, there are remarkable recent works that show how to extract concepts or entities from data automatically [ 1,2,4,5,8 ], but they still cannot deal with complex ERP entities made up of many relational tables. To improve the search experience, we decided to model the fundamental entities and their relations explicitly by exploiting our knowledge of ERP systems and enterprise user needs, which is necessary to build an e ective search system [ 13 ]. Therefore, we asked our users which were the most relevant type of ERP entities and what kind of relationships connected them. We obtained a list of 33 entity types that are connected by 70 relationship types that represent the core of the work for most of our users. These entities are made of structured and unstructured data that are represented as documents. In order to preserve the structure of the entities, documents are organized as a set of elds (see [ 7 ] for more details). Furthermore, according to our ERP domain experts, we de ned a set of components, detailed in Sec. 2.1, that must in uence the rank of an entity. These are related to the following fundamental aspects of an entity: content, context (in terms of its relationships), and last modi ed date. To combine the contributions of the components, we follow the idea proposed by [ 12 ], which led us to design a modular enterprise search engine. The modules are organized into a pipeline and the contribution of each of them is computed sequentially. 2.1

Entity Ranking

When a search is performed, the nal rank of the results is a weighted linear combination of contributions computed by a pipeline of components. More formally, let f igi=1;:::;N a set of scores and fwigi=1;:::;N a set of weights, the nal rank r of an entity " is given by r(") =

N X wi i s.t. 0 i=1 i 1; (1) where i represents the normalized version of i through the min-max normalization method [ 9 ]. We instantiated the model (Eq. (1)) considering the following contributions:

cnt Given the document representation d of entities ", this is the TF-IDF score that re ects how relevant an entity is by its content (see [ 6 ]). More speci cally, we computed the cosine similarity between the user query q and an indexed document d represented as vectors. Therefore, the cnt can be expressed as cnt(d) = cos(q; d) =

V (q)V (d) jV (q)jjV (d)j ; d 2 D where V () is the vector form of a document and D is the set of indexed documents. where t.

dte It is a linear score that boosts recent entities [ 7 ]. Given the date of a document expressed in days t and a normalization constant n = max(t 2 T ), where T represents the set of dates of the indexed documents, the score is de ned as dte(d) = n t ; n is a boost factor that we set to 2 and d is the document associated with egs Considering the subgraph S of EG provided by the top results of cnt ranking, this is a logarithmic score that boosts connected documents. egs is de ned as egs(") = log 1 + ;

"; "0 2 S 1 '("; "0) where '() is a weighted distance computed by summing the weight of the edges on the shortest path between a pair of entities ("; "0) such as " 6= "0.

prk The score provided by Page Rank which is proven to lead to better search performances [ 12 ].

Therefore the rank model used by our system is r(") = w1 cnt + w2 dte + w3 egs + w4 prk; (2) where w1; : : : ; w4 are assigned by analyzing search results as explained in Sec. 3.3. Exploiting the click-through data [ 6 ], it could be interesting to try to compute weights automatically by using a machine learning technique such as SVM, boosting, or neural networks [ 9 ]. 2.2

Entity Graph (EG)

To meet the user need of exploring ERP entities, we build the EG, enhancing enterprise search with an exploration experience complementary to faceted navigation and full text search. EG is a graph which consists of nodes that represent entities extracted from a set of queries on the data sources, one for each entity type. Edges represent the underlying business relations among the entity types. They are extracted by queries that link pairs of entity references. Formally, an EG is a directed graph G = (V; E; W), where V is a set of nodes, E V V a set of edges, and W a set of edge weights. We place an entity identi er into each node, while edges contain labels that explain the meaning of the relations. A con guration le determines the queries to extract the relations, their direction, and the weights of each type of relation. By analyzing the links of EG, we found that there are huge node hubs because there are some types of entities (i.e., master data type) that are linked to almost all the others. This is a problem for the computation of prk, because ranking methods such as PageRank or HITS [ 3 ] are built to rank web pages. So, they give higher rank to hub nodes which are not necessarily relevant for each enterprise information need. Even if the problem is still open, our system gives to prk a lower weight in order to mitigate the huge hub nodes e ect. 3

Prototype

SeNSE (Skyline eNterprise Search Engine) is the name of the prototype that demonstrates the applicability of the model described in Sec. 2. The prototype is based on the ERP system Freeway Skyline1. 3.1

Architecture

1 www.freewayskyline.com 2 senseidb.github.io/bobo

ENTERPRISE DATA

Unstructured Data (docs, emails,

web etc.) Structured Data (rdbms) ERP Data

INDEXING SERVER

SEARCHING SERVER

Preview Computation

Text Analysis

Entity Extraction

Entity Graph

Analysis Entity Graph

Building Entity Links

Extraction Entity Definition

Entity

Preview Entity Graph

Exploration Full Text Search

Faceted

Search Search Pipeline (TF-IDF score, date score, EG score, etc.)

Content Search computes the score cnt by exploiting the full text search capabilities of Lucene [ 7 ]; Date Boost computes the score dte; Link Score computes the score egs; Page Rank computes the score prk; Final Score computes the equation (2) given the result of the previous components; Abstract Highlighting highlights terms of the result documents that match the user query; Entity Security de nes a cached security lter that is provided by the Content Search component. The searching server provides two other services, namely the entity graph exploration and the entity preview services independent of the search pipeline. We store into document elds the security information such as user name, company name, and database table grants. For each , we de ne the allow a and deny d policies. To establish if a result can be listed into the SERP the following boolean expression is evaluated ( 1a ^ : d) _ 1 _ ( ia ^ : d) _ i _ ( aL ^ : dL); where i 2 1; : : : ; L is the index of a security information. The presence of id is not strictly necessary, but it allows to implement security roles such as \allow all but . . . ".

One of the major problems we found in designing the architecture of SeNSE is that it needs di erent representations of an entity (namely sparse vector, node of a graph, and database entry) to provide its services. This is not only a scalability issue but also a modeling one. In fact, the extension of the search pipeline with further components could introduce novel representations for the entities. In particular, for many machine learning techniques a dense vector representation is necessary [ 12 ]. To the best of our knowledge there is not a uni ed representation to search, analyze and explore entities.

Another tricky problem concerns the update of the indexed entities, because enterprise search engines updates should be processed in near real-time. The system has to deal with all the type of updates, in particular it has to manage the cancellation of entities which is the most di cult case. To tackle the update problem, SeNSE implements three update policies: batch full that updates all the entities of a certain type, batch delta that updates entities modi ed up to a speci c date, and real time. The rst two policies can be scheduled depending on the number of entities involved into the update and their indexing speed. The current implementations of the policies is speci c for each data source, but there is still room to improve because the performances of the update noti cation infrastructures provided by data sources are not always satisfactory, since the infrastructures provide too many false positive update noti cations or too generic noti cations. 3.2

User Experience 1 4 5 6 2 3

The most relevant pieces of the user interface of our system is shown in Fig. 2, Fig. 3, and at www.freewayskyline.com/demosense. In particular, Fig. 2 shows a small part of the SERP which is divided into three main areas. The rst area contains the search box as depicted in Fig. 2.1. According to our users, we provide the possibility to choose the type of entity before entering the search query. Once the search is performed, the faceted navigation can be started from the left part of the UI as shown in Fig. 2.2. Simultaneously, the results are listed in the right part of the interface (Fig. 2.3). For each result three functions are available: starting from the left, the rst function is the EG exploration (Fig. 2.4) which is detailed below. In the middle (Fig. 2.5), it is placed the preview function that displays the image of the entity associated with the result into a exbox according to its type and format. Finally, on the right (Fig. 2.6) there is the user actions function that list a set of user de ned business actions available for the result such as compile an order or print a bill.

Back to search

Restart the navigation Categories:

Order confirmation Order request

Estimate

Order Request: 07ORDXV 11/01/2013 - IBM ITALY SPA

Order Confirmation: 1234 08/02/2013 - IBM ITALY SPA

Estimate: 000785 11/12/2012 - IBM ITALY SPA

To implement the exploration of the EG, we use the Vis.js3 library that is able to display automatically and interact with the graph at the same time. When a result of a search is displayed into the SERP, the exploration can be started from the entity associated with the selected result and its neighborhood as shown in Fig. 3.1, then it is possible to continue the exploration experience by selecting a neighbor. Since the ER is interactive, from each node it is possible to execute its business actions. On the top part of the EG exploration page (Fig. 3.2) the map legend and the main navigation functions are displayed. Users found the EG exploration e ective and intuitive on both tablet and pc and ask to personalize the appearance of each entity type. 3.3

Experiments

We experimented SeNSE with success on an X64 Intel Xeon E5450 3:00 Ghz processor with 10 Gbytes of RAM server. Since we are not aware of any public database that t our ERP entity model, we built three di erent enterprise datasets with real data. They contain approximately 1 million entities and 10 million entity links which are typical magnitude of data for small and mediumsized enterprises. To evaluate the performance of our system we chose the largest 3 visjs.org dataset and we computed the Precision at k (Pk) [ 6 ] on a testing set of 100 user information needs. We collected the needs both by interviewing users and by logging their search queries, then relevance judgments are obtained by merging the user ranking on the top 5 entities. The performance baseline of SeNSE is given by the cnt rank. It yields that the top 5 entities in user queries are recognized with an average precision of 54%. To improve the performances up to 15%, we added all the others score components ( dte, egs, and prk). We assigned a weight fwigi=1;:::;4 performing a grid search [ 9 ] that maximize Pk. For this purpose all the scores are normalized (see Eq. (1)) and weights are selected by searching into a range 0 wi 1 using a step of 0:1. Final weights are not uniformly distributed, in fact cnt is the most important contribution with respect to the others. 4

Conclusions and Future Works

We presented an enterprise search model that exploits ERP entities to enhance the enterprise search experience and its implementation: SeNSE. We discussed the main design aspects of the model and the related open issues. Then we present the architecture of the prototype and its user experience. In future work, we aim to clarify the bene t given by each contribution to entity ranking and we will implement an automatic method to compute the weights for those contributions.

References

1. Brauer , F. , Huber , M. , Hackenbroich , G. , Leser , U. , Naumann , F. , Barczynski , W.M. : Graph-based concept identi cation and disambiguation for enterprise search . In: WWW ( 2010 )

2. Graus , D. , Tsagkias , M. , Weerkamp , W. , Meij , E., de Rijke, M.: Dynamic collective entity representations for entity ranking . In: WSDM ( 2016 )

3. Leskovec , J. , Rajaraman , A. , Ullman , J.D. : Mining of massive datasets . CUP ( 2014 )

4. Li , J. , Yang , J.J. , Liu, C. , Zhao , Y. , Liu , B. , Shi , Y. : Exploiting semantic linkages among multiple sources for semantic information retrieval . EIS ( 2014 )

5. Liu , X. , Chen , F. , Fang , H. , Wang , M. : Exploiting entity relationship for query expansion in enterprise search . IR 17 ( 3 ), 265 { 294 ( 2014 )

6. Manning , C.D. , Raghavan , P. , Schutze, H.: Introduction to Information Retrieval. Cambridge University Press ( 2008 )

7. McCandless , M. , Hatcher , E. , Gospodnetic , O. : Lucene in Action: Covers Apache Lucene 3.0 . Manning Publications Co. ( 2010 )

8. Meij , E. , Balog , K. , Odijk , D. : Entity linking and retrieval for semantic search . In: WSDM ( 2014 )

9. Murphy , K.P.: Machine learning: a probabilistic perspective . MIT press ( 2012 )

10. Nazemi , E. , Tarokh , M.J. , Djavanshir , G.R. : Erp: a literature survey . IJAMT ( 2012 )

11. Owens , M. , Allen , G.: SQLite . Springer ( 2010 )

12. Turney , P.D. , Pantel , P. , et al.: From frequency to meaning: Vector space models of semantics . JAIR ( 2010 )

13. White , M. : Critical success factors for enterprise search . BIR ( 2015 )