Modern enterprise applications have high degree of complexity. Usage recommendations help to address this complexity from the user perspective. The paper uses recommendations to guide business process execution in enterprise applications. To provide timely support, the recommendations should be generated in near real-time without compromising performance. That is achieved by developing a platform integrating data streams and generating recommendations autonomously of the enterprise application. The platform is shown capable of processing high volume data streams and generate recommendations in near real-time. Keyword : Software development, performance testing, load testing
Enterprise applications (EA) such as ERP systems are used by companies to execute
their business processes. They have a large number of functions and high complexity.
Despite of recent developments towards better integrated and more open systems [
Adaptation helps to address these challenges [
This paper investigates an ability of the proposed solution to handle large volumes
of incoming data and generate recommendations in near real-time. RaAP is developed
as a specialization of the data streams processing solution described in [
The rest of the paper is organized as follows. Section 2 describes the conceptual model of the adaptation and recommendation platform. Section 3 describes architecture of the platform. The experimental studies on performance of the platform are reported in Section 4. Section 5 concludes. 2
Reliability, process execution consistence, performance and user acceptance are
among key requirements for EAs [
Context Provider
Entity Measurable Property
0..* uses 1 0..* 0..* Context Calculation
0..* associated with 0..* measures 1 0..* 0..* Calculates
Context Element 1
1..* uses 1..* Trigger 1 1..* performson 1..*
1..* uses 0..* trigers 0..*
Action
1 Recommendation 1 Adaptation
It is assumed that an EA consists of software entities (e.g., forms, reports, lists) (Fig. 1). The entities are executed in specific context defined by context elements (CE), which are measured using measurable properties (MP). MP are raw data from various sensors and CE are logical interpretation of these data (e.g., user experience, customer importance). The distinction between MP and CE acts as a filter to reduce processing of unnecessary data and to simplify definition of recommendations. If CE assume specific values (e.g., inexperienced user confirms orders one-by-one) a recommendation is triggered. The recommendation is combination of software entity and action (e.g., suggest mass-processing function). It is important to note that software entities and actions are standardized and can be reused to specify multitude of recommendations.
RaAP is developed as an independent module, which is integrated with an EA in a loosely coupled manner. This way it can handle various data sources, generate recommendations for different EAs in a uniform manner, separate computational effort needed for generating recommendations from the core application and decouple recommendation and adaptation logics from business logics of the EA. The platform is specifically geared towards generating recommendations from live data streams and ensuring scalability of data processing. The architecture of RaAP is shown in Fig. 2. Horizontally scalable stream processing units are responsible for receiving data from context providers (1) and handling internal data streams. The archiving jobs store MP in persistence storage and CE jobs use MP to evaluate CE and their values are sent to internal stream processing (4), where they are forwarded for evaluation of recommendations triggers (6). If triggering conditions are met (7), a recommendation is generated and posted to EA (8,9). All potentially computationally intensive tasks are executed in dedicated containers in cluster to ensure high performance. There are alternative possibilities for enacting recommendations in EA depending on preferences of EA developers. One alternative is recommendation window though direct execution of recommendations is also possible, for instance, using sockets.
providers 1
3,5
10 P1 PL 2 K1 4
... 6 CE jobs KM 7
9 Recommendation and adaptation engine (RAE) R1 ... RN
SP is implemented Apache Kafka1 streaming platform. CoE is built using Apache Spark2 big data analytics engine and RAE is based on Docker3 containers. The infrastructure is provided using CloudStack4 cloud infrastructure tools. 4
The platform currently is being piloted on an ERP system specialized of municipalities. It has around 30 000 users and highly variable workload. The context data sources are log files, live user observation as well as external data sources. The platform provides recommendations to improve the system’s usage efficiency.
To evaluate an ability of RaAP to generate near real-time recommendations, high volume data streams are simulated and processed by the platform. The platform’s throughput, recommendations generation time and workload generated on EA are evaluated. Three configurations of the platform are considered: C1) 1 core for every SP, CoE and RAE, C2) 2 cores for every SP, CoE and RAE and C3) 3 cores for every SP and RAE. The data stream flow rate S is varied from 5000 requests per second to 20 000 rps.
Fig. 3a shows ability of the configurations to handle high volume data streams. The throughput without queuing tails-off after the capacity limit is achieved (e.g., C1 has its limit at S-5000 rps). This should be sufficient for medium to large deployments of EA (in the example considered peek flow is less than 1.5 MB/s). Recommendations are generated within 536 ms and 793 ms for C1 and C3 and S=1000 rps, respectively, and 1078 ms for C3 and S=15 000 rps. C3 has higher throughput thought the latency increases implying that oversized configuration is not desirable for low data streams. Fig. 3b shows workload created by RAE on EA. T denotes recommendation processing time by EA (received to be displayed) and z characterizes how often recommendations are generated (i.e., what is a share of input requests transformed into useful recommendations and sent to EA). If many recommendations are generated, nonscalable EA is not able to handle the incoming flow. In the case of z=40% many requests are dropped. EA can be designed to handle large flow of recommendations though that diverts development and computational resources from the core needs of enterprise applications. That demonstrates that RaAP is important to filter out useful information. 1 https://kafka.apache.org/ 2 https://spark.apache.org/ 3 https://www.docker.com/ 4 https://cloudstack.apache.org/ RaAP allows generating recommendations for users of EAs. It allows specification and implementation of recommendations in reusable manner without altering the business logics of the core EA and generates recommendations in near real-time. The platform is highly scalable and is able to handle high volume of incoming data and reduces overhead associated with generating recommendations.