Quality of Service in Event-based Systems Stefan Appel Kai Sachs Alejandro Buchmann TU Darmstadt, Germany TU Darmstadt, Germany TU Darmstadt, Germany appel@dvs.tu- sachs@dvs.tu- buchmann@dvs.tu- darmstadt.de darmstadt.de darmstadt.de ABSTRACT becomes a key aspect. Defining QoS and developing the nec- Future software systems must be responsive to events and essary monitoring mechanisms is a major challenge. First, must be able to adapt the software to enhance business pro- the system boundaries must clearly be defined: is only event cesses. Examples are production and logistics processes that transport of interest, or have event production, composition must be rescheduled based on relevant traffic information. and consumption taken into consideration as well. Once the It is therefore essential that relevant events are detected and scope of the event system is defined, overall quality criteria transported to the software components responsible for dy- and criteria for each component must be defined, monitored namic changes. The trust in such reactive systems depends and enforced. Quality of Service in EBSs is strongly influ- to a large extent on the Quality of Service (QoS) provided enced both by functional properties, such as ordered delivery by the underlying event system. This paper introduces QoS of events, and non-functional properties, such as throughput aspects in event-based systems and discusses ways of identi- an availability. fying and evaluating QoS needs. This is done by identifying the different system layers, and the quality requirements at In this paper a general overview of QoS in event-based sys- each layer based on the layer’s functionality. tems is given and an architecture that supports the different types of QoS in an EBS is presented. The main focus is on QoS of the notification service (Message-Oriented Middle- 1. INTRODUCTION ware, MOM, sometimes referred to as Event Bus) as well Quality of Service (QoS) in event-based systems (EBSs) is as the QoS in terms of Complex Event Processing (CEP). important when developing solutions to dynamically sup- The paper is organized as follows: first, related work is pre- port and monitor business processes. A user requires reli- sented pointing out current directions of research. Based on ability guarantees in order to trust the output of a system those, a generic approach is chosen to identify QoS factors supporting business critical operations. The QoS aspects in event-based systems. The paper continues with a sys- discussed in this paper are driven by the research project tem architecture describing the different qualities occurring ADiWa 1 . The goal of ADiWa is letting events from the In- within the scope of an EBS. Finally performance evalua- ternet of Things (IoT) dynamically influence and enhance tion and monitoring of event-based systems is addressed as business processes. Industrial partners provide different use a controlling instrument for ensuring QoS. cases from the areas of logistics, business services, retail and production; for these scenarios, event-based system proto- types are developed that enhance the underlying business 2. RELATED WORK processes automatically. Research in event-based systems can be categorized into three broad areas: event production, event transportation In event-based systems event producers and event consumers and event consumption. Typical event producers are wire- are decoupled and communicate via asynchronous communi- less sensor nodes and RFID readers but also complex event cation patterns. Event consumers may subscribe to events of processors (CEPs) producing, e.g., composite events. Event interest. Whenever those events are detected, the subscriber consumers are, for example, complex event processing en- is notified. Since the invocation of business critical processes gines or business applications in general. To connect pro- is now triggered by events, the QoS of the event mechanism ducers and consumers a middleware is necessary and also 1 responsible for transporting the event notifications. Events Allianz Digitaler Warenfluss (ADiWa). Funded by the German Fed- eral Ministry of Education and Research under grant 01IA08006 are represented by event objects which are packaged into notifications [7, 12]. A schematic view of an EBS is shown in Figure 1. For the communication between producers and consumers the paradigm of choice is publish/subscribe, allowing a de- coupled many-to-many communication. The pub/sub mid- dleware must provide a certain QoS [8]. At present, one of the most widely used technologies providing pub/sub capa- GvD Workshop 2010, 25.-28.05.2010, Bad Helmstedt, Germany. bilities is the Java Message Service (JMS) [23]. A variety of Copyright is held by the author/owner(s). different implementations exist; popular ones are ActiveMQ Event routing, filtering and transactions is given in [5] and [18]. Observation Generally, QoS has not been addressed extensively in re- Event-Based Interaction search; a basis is provided in [4] where basic QoS metrics Producer Event observed Event Notification (e.g., latency, bandwidth, delivery guarantees) are discussed. ….:……. Time Consumer …….Location… Consumer In the ADiWa project we tried to apply these metrics di- Event Notification Envelope rectly and found out that they are very technical on one Time…. Consumer ……. hand, and incomplete on the other, since they do not ad- dress many of the critical functional aspects that have an Notification Service Notification Service impact on QoS. Therefore, we used a list of EBS features Communication Layer as a starting point to determine the functionality needs of EBSs. From them we derived the QoS requirements. Figure 1: Structure of Event-based Systems 3. QOS IN EVENT-BASED SYSTEMS 3.1 Features in EBSs [24], HornetQ [20], IBM Websphere, TIBCO Enterprise Mes- ADiWa started with the identification of business processes sage Service or Oracle WebLogic Server. All these products which can be dynamically supported by an event-based sys- follow the JMS specification and provide the corresponding tem. From the technical perspective this is a high-level view QoS functionalities. For JMS these are support for transac- and many common QoS criteria, like performance of the tions, persistence and durability. EBS, are not applicable at this early stage. To address this issue we categorized features of EBSs to enable a QoS-aware Besides JMS, the Data Distribution Service (DDS) [17] and specification and development process. The used features the Advanced Message Queuing Protocol (AMQP) [1] are are based upon an analysis of real-world event-based sys- other standards for MOM. AMQP is a not yet finally speci- tems [12]. Examples are: support for mobility, support for fied network wire-level protocol which enables interoperabil- transactions, early filtering, real time capabilities, or sup- ity among different MOM implementations and program- port for interval events. As first step those more than 40 ming languages; it is comparable to JMS in terms of reli- single features were grouped into five main categories: event ability and messaging capabilities. The focus of DDS are types (e.g., support for interval events), general require- real-time distributed systems whereas JMS was originally ments (e.g., handling of out of order events), (legal) lim- designed for business applications. In terms of QoS, DDS itations and specifications (e.g., privacy protection), tech- does support more options than JMS does; [17] gives an nical properties (e.g., support of prioritization), and run- overview over the supported QoS policies, e.g., DDS allows time requirements & performance (e.g., latency). These fea- the specification of transport priorities for notifications. De- tures determine which type of EBS is needed with respect pending on the area of application JMS, AMQP, or DDS to the relevant business cases. Thus, rather than building might be the messaging standard of choice. a Swiss army knife EBS supporting all imaginable features and QoS needs, the requirements are matched during the Although JMS, AMQP, and DDS support a variety of QoS requirements engineering and development process. features, important QoS needs, e.g., ordering events from multiple producers or the occurrence of false positives and Comprehensive Understanding false negatives, are not addressed and still topics of research of Quality Needs and of interest for future systems. JMS and DDS are used Event Types in QoS research: [13] presents concepts for the management of QoS in DDS, [22] uses the SPECjms2007 benchmark to General Requirements evaluate compliance and performance of JMS middleware. (Legal) Limitations and Specifications Technical Properties In addition to JMS, AMQP, and DDS, which are used in Runtime Requirements & Performance real-world applications, many research prototypes of mid- - Development Process - dleware systems exist to evaluate new concepts and features. While JMS follows a centralized, topic-based approach, cur- rent research systems are highly distributed and support, for Figure 2: Features of Event-based Systems example, content or concept based pub/sub [2]. The use of these distributed event-based systems (DEBSs) is necessary Figure 2 shows the above mentioned categories with respect to cope with high volume of events expected to occur in fu- to the time line of a development process. As it can be seen, ture applications. With the spread of mobile smart devices, general requirements as well as event types should be iden- events are produced and received anywhere making some tifiable rather early, during the first steps of requirements sort of intelligent middleware indispensable. Research pro- engineering. Later on, legal limitations and further system totypes of DEBSs are for example PADRES [11], SIENA [6], specifications can be determined before technical properties HERMES [19] or REBECA [2]. Prominent research topics can be addressed. As last step, runtime requirements and include routing [10] and filtering [9] in DEBSs. Further, the performance needs can be identified. A comprehensive un- support of transactions [16] is an important property in or- derstanding of QoS is only possible after all required fea- der to build systems supporting business-critical processes. tures have been identified. A detailed specification of QoS A system overview of DEBS addressing, amongst others, requirements can now be produced. 3.2 Characterizing the EBS QUALITY CATEGORY SYSTEM COMPONENTS Based on the relevance of the various features in a given Controlling Quality Dynamic Business Processes application scenario a specific EBS can be configured that Subscribe Publish Processing conforms to the required QoS. Figure 3 shows the concep- Quality Complex Event tual approach of defining QoS for certain features. For each (e.g. Throughput, Processing Accuracy) feature one or more metrics are necessary to measure and Subscribe Publish monitor this feature. For some features the metric to mea- Notification Quality sure QoS might be as simple as Is supported [yes/no], for (e.g. Latency, Reliability) Message-oriented Middleware (MOM) / Event Bus other features multiple metrics might be necessary. An ex- ample is the support of the feature early filtering with filter Publish Publish Publish Publish Event Producer merging and the tradeoff between reduced number of filters Production Quality … and increased number of false positives. (e.g. accuracy of sensors) RFID Reader Wireless Sensor Networks Mobile Data Entry Other Producers By merging filters we reduce the number of filters that must be maintained and evaluated but increase the number of Figure 4: QoS Architecture false positives and the number of forwarded messages be- cause the filter is now less precise. As shown in Figure 2, early filtering results in multiple QoS attributes such as cern the inner parts of the system: the event transporting accuracy and efficiency. Efficiency is ultimately measured layer as well as the event processing layer. through resource utilization of the broker node. Event-based System Since QoS affects all layers of a system a system-wide ap- proach for managing quality information is needed. Every Features Metrics Quality of Service single event has its related production quality and is then Accuracy transported and processed with a certain quality. During Early Filtering Efficiency all these steps quality information for each event has to be … maintained either directly at the event level or at higher granularity like event streams. The complexity of the QoS Low Response Time Response Time information suggests that in addition to attaching quality data to events a central registry for QoS data is helpful (Fig- Interval Event Support Yes / No ure 5). … … … Event Transporting Middleware Figure 3: Defining QoS Transport Quality Data A major problem that must be considered when addressing Complex Event Event QoS in EBSs is the fact that features and derived QoS re- Quality Quality quirements are not orthogonal, i.e. they may influence each Producer Data CEP Data Consumer other. However, the more concrete the requirements for an EBS are specified and as more design decisions are made during the design process, the easier it becomes defining Quality Data Repository and monitoring QoS. 4. A QOS AWARE INFRASTRUCTURE Figure 5: Types of Quality Data So far we limited the scope to event transport (MOM) and complex event processing (CEP). As shown in Figure 4, the QoS data can be archived. This QoS history is then accessi- complete system includes also the event production layer ble and the middleware might draw conclusions during the and the dynamic business process layer. event transport: events originating from a sensor reporting critical situations might require a highly reliable real-time At the lowest level the events enter into system; at this handling within the middleware. To support features along point there is an associated production quality. This quality these lines an appropriate event format and a system-wide is determined by the event producers, for example wireless understanding of the existence of QoS is necessary. sensor nodes. These nodes can monitor the environment and have, for example, a limited accuracy (Temperature A special challenge in terms of QoS is the support of transac- +/- 1 degree) which might influence later event processing. tions since transactional processing does involve producers, The highest levels of the system are the business processes middleware, and consumers and thus affects all QoS lay- driven by events where the goal is to dynamically adapt ers. Transactions are, for instance, needed in case event these processes according to occurring events. The adaption producers require acknowledgments from a consumer before of processes is associated with QoS again: controlling qual- publishing another event (e.g., shipment event before billing ity. Associated QoS metrics describe how well processes are event). To support this functionality in a flexible and reli- adapted and whether changes lead to improvements. This able way, transaction management has to be supported by can be measured with Key Performance Indicators (KPI) the middleware [16]. Without middleware mediated trans- assigned to processes. The two remaining levels of QoS con- actions it would not be possible to inform the event producer whether a message was just lost or whether there is no ap- 5.3 Monitoring propriate subscriber. Depending on this information further Besides modeling and benchmarking the monitoring of a pro- steps can differ: resending the message vs. reacting to not ductive event-based system is important to ensure QoS. Col- available subscriber. lected data can be incorporated with models, validated by simulations, and utilized to develop a characteristic bench- 5. EVALUATING & MONITORING MOM mark workload. Further, monitoring allows the early detec- tion of potential bottlenecks and gives developers the pos- To ensure the compliance of a system with the QoS re- sibility to adapt and extend the system in advance. Moni- quirements techniques based on modeling, benchmarking toring is done by collecting runtime information. This can and monitoring are used. be accomplished in a pub/sub manner: each event producer or consumer does not only publish events but also statistical 5.1 Modeling information. In addition, the event transporting middleware Various techniques exist for the modeling of event-based sys- itself can publish data accounting for QoS. Parties interested tems. A promising approach is the use of Queuing Petri Nets in statistical data can then simply subscribe to messages (QPNs) [3, 15], which combine the advantages of Queuing and perform further analysis. The middleware can act as a Networks and Petri Nets. The basic building blocks of a consumer, too. By this a self-adapting middleware can be QPN are places, transitions and tokens. Tokens flow through realized which dynamically reconfigures to constantly assure the net via transitions; at each place the ”processing” of a to- the required QoS. ken takes a certain amount of time, the so called service time. In the context of event-based systems, an event is repre- To allow the reporting of statistical data middleware imple- sented by a token. After building the model and calibrating mentations need to be adapted. Unfortunately monitoring a the service times, simulation tools (e.g., SimQPN [14]) allow system introduces an overhead. The overhead increases with predicting the throughput and latency of the whole system the granularity of monitoring, thus a tradeoff between mon- in a reasonable time. In addition, resource utilization (e.g., itoring granularity and potential monitoring benefits has to CPU utilization) of single components, like brokers, can be be found in productive systems. It is also possible to build estimated. a system with adjustable monitoring capabilities. If such a monitoring framework exists, it becomes possible to in- 5.2 Benchmarking crease the monitoring granularity either on a regular basis Another approach is the use of benchmarks to evaluate the or whenever an in-depth problem analysis is needed. performance and QoS of systems. At first, a benchmark can test whether the promised QoS is provided by the underly- 6. CONCLUSION ing EBS, e.g., in terms of latency. Second, a well designed Dynamically enhancing business processes using events orig- benchmark has a scalable workload allowing determining the inating from a variety of producers requires a middleware for limits of systems and comparing implementations of differ- transporting the data. Within the ADiWa project require- ent vendors in an independent way. One of the challenges ments for those middleware systems are a topic of research. when developing a benchmark is choosing an appropriate In this paper an approach for identifying, defining and en- workload. Since building separate workloads for various ap- suring QoS needs is presented. At first, relevant features plication scenarios is usually not possible, a benchmark uses of an EBS are identified; this can be accomplished stepwise a workload with properties which are characteristic for many as the requirements engineering process proceeds. Based applications. Further, it can be designed to support soft- upon the determined functionalities, QoS metrics can be de- ware developers and architects during the development pro- veloped and evaluated using, e.g., benchmarking, modeling, cess. When choosing a current middleware which supports and monitoring mechanisms. This QoS-aware software de- pub/sub, e.g., following the JMS standard, the developers velopment process assures that quality needs are taken into still have the choice whether to use a single topic for all consideration early in the design process. In addition to the messages or various topics. In terms of ADiWa the decision identification of QoS needs the management of QoS relevant would be whether to use an event bus where all events flow data is a challenge. QoS data does occur at various places in through the same stream (destination), or whether to struc- systems and thus collecting relevant parts of it in a central ture events and use various streams (destinations) forming repository is necessary. Based upon archived QoS data the the overall event bus. To evaluate, amongst others, these system can be evaluated and tuned. different alternatives jms2009-PS was developed [21]; it uses the SPECjms2007 workload as a basis and emphasizes on pub/sub messaging. SPECjms2007 is the current industry- 7. REFERENCES standard benchmark for MOM servers based on the JMS and [1] AMQP Working Group. Advanced Message Queuing models a supermarket supply chain where RFID technology Protocol, 2010. http://www.amqp.org. is used to track the flow of goods. [2] J. Antollini, M. Antollini, P. Guerrero, and M. Cilia. Extending rebeca to support concept-based jms2009-PS provides more than 80 configuration parame- addressing. In Proceedings of ASIS, 2004. ters allowing the user to customize the workload in terms of [3] F. Bause. Queueing Petri Nets - A formalism for the the number of destinations (topics), the number of subscrip- combined qualitative andquantitative analysis of tions, the number and type of selectors, and the message de- systems. In Proceedings of Petri Nets and livery modes (QoS). With these parameters alternative ways Performance Models, 1993. to implement pub/sub communication can be evaluated in [4] S. Behnel, L. Fiege, and G. Muehl. On terms of their overhead, performance and scalability. quality-of-service and publish-subscribe. In Proceedings of ICDCSW, 2006. middleware using the SPECjms2007 benchmark. [5] A. Buchmann, C. Bornhoevd, M. Cilia, L. Fiege, Performance Evaluation, 66(8), 2009. F. Gaertner, C. Liebig, M. Meixner, and G. Muehl. [23] Sun Microsystems. Inc. Java Message Service Dream: Distributed reliable event-based application Specification Final Release 1.1, 2002. management. In M. Levene and A. Poulovassilis, [24] The Apache Software Foundation. Apache ActiveMQ, editors, Web Dynamics: Adapting to Change in http://activemq.apache.org/, 2009. Content, Size, Topology and Use. Springer, 2004. [6] A. Carzaniga, D. S. Rosenblum, and A. L. Wolf. Achieving scalability and expressiveness in an internet-scale event notification service. In Proceedings of PODC, 2000. [7] K. Chandy and W. Schulte. Event Processing: Designing IT Systems for Agile Companies. McGraw-Hill, Inc., 2010. [8] P. T. Eugster, P. A. Felber, R. Guerraoui, and A.-M. Kermarrec. The many faces of publish/subscribe. ACM Comput. Surv., 35(2), 2003. [9] F. Fabret, H. A. Jacobsen, F. Llirbat, J. Pereira, K. A. Ross, and D. Shasha. Filtering algorithms and implementation for very fast publish/subscribe systems. In Proceedings of SIGMOD, 2001. [10] C. Fengyun and J. P. Singh. Efficient event routing in content-based publish-subscribe service networks. In Proceedings of INFOCOM, 2004. [11] E. Fidler, H.-A. Jacobsen, G. Li, and S. Mankovski. The PADRES distributed publish/subscribe system. Feature Interactions in Telecommunications and Software Systems, VIII, 2005. [12] A. Hinze, K. Sachs, and A. Buchmann. Event-based applications and enabling technologies. In Proceedings of DEBS, 2009. [13] J. Hoffert, D. Schmidt, and A. Gokhale. A QoS policy configuration modeling language for publish/subscribe middleware platforms. In Proceedings of DEBS, 2007. [14] S. Kounev and A. Buchmann. SimQPN - a tool and methodology for analyzing queueing Petri net models by means of simulation. Performance Evaluation, 63(4–5), 2006. [15] S. Kounev, K. Sachs, J. Bacon, and A. P. Buchmann. A methodology for performance modeling of distributed Event-Based systems. In Proceedings of ISORC, 2008. [16] C. Liebig and S. Tai. Middleware mediated transactions. In G. Blair, D. Schmidt, and M. Takizawa, editors, Proceedings of DOA, 2001. [17] Object Management Group (OMG). OMG Data Distribution Service (DDS) Specifications. 2007. http://www.omg.org/spec/DDS/. [18] P. Pietzuch, D. Eyers, S. Kounev, and B. Shand. Towards a Common API for Publish/Subscribe. In Proceedings of DEBS, 2007. [19] P. R. Pietzuch and J. Bacon. Hermes: A distributed event-based middleware architecture. In Proceedings of ICDCSW, 2002. [20] Red Hat, Inc. JBoss HornetQ, http://www.jboss.org/hornetq, 2009. [21] K. Sachs, S. Appel, S. Kounev, and A. Buchmann. Benchmarking publish/subscribe-based messaging systems. In Proceedings of BenchmarX, 2010. [22] K. Sachs, S. Kounev, J. Bacon, and A. Buchmann. Performance evaluation of message-oriented