Organizations exchange data electronically to perform business transactions using Electronic Data Interchange (EDI). In order to gain insights on such transactions, approaches for inter-organizational business process mining based on the observation of exchanged EDI messages have been recently proposed. In recent approaches, however, only meta-information about the exchanged messages, such as message type, interchange time and sender/receiver information, has been used as data base for generating event logs. This neglects the opportunity of using business information from observed EDI messages to arrive at more detailed event logs, which in turn enable mining of detailed process models and ne-grained process performance analyses. In addressing this shortcoming, we present EDIminer, a toolset that allows for (i) enhanced visualization of contents of EDI messages, (ii) automatic and/or user-driven de nition of mappings of EDI artifacts to events, (iii) generation of events from such mappings, (iv) semi-automatic correlation of events to process instances and (v) generation of industry-standard XES event logs for subsequent application of conventional process mining techniques. We demonstrate the utility of EDIminer by means of an exemplary EDI-based purchase order process based on real-world data.
Companies and organizations exchange data electronically to perform business
transactions (e.g., requests for quotes, purchase orders, etc.). If the interchange
of data is carried out in an automated and standardized manner, such processes
may be referred to as Electronic Data Interchange (EDI) [
Message Flow Mining (MFM). In the approach presented in [
Physical Activity Mining (PAM). Business information conveyed in EDI messages may be used to infer additional events. For example, an invoice message may, in addition to general invoicing information, contain information about a shipping date of invoiced line items. Consequently, from such a particular shipping date one may infer that an activity \Ship goods" has occurred on that date. Events resulting from such information generally re ect activities that represent physical product ows, cash ows or other activities as opposed to message ows. Hence, we refer to such approaches as physical activity mining (PAM). Thereby, a major challenge is to identify and de ne appropriate mappings of business information in EDI messages to events and their attributes in event sequences (\EDI/event mappings"). Such mappings need to specify rules that de ne (i) what EDI artifacts constitute events and (ii) which EDI artifacts shall be used to populate event attributes. In this paper we illustrate PAM by means of manual mapping de nitions. As in MFM, correlation of so-created events to process instances can pose a challenge in PAM as well.
We propose that MFM and PAM may be combined for process mining from EDI messages in order to arrive at detailed event logs. Such event logs may allow for the mining of detailed process models, performance analyses, dataaware conformance checking, and identi cation of bottlenecks. For example, a process model mined from ne-grained shipping dates documented in an invoice message may lead to insights about speci c products that cause regular delays in the invoicing process. In this paper, we present a toolset named EDIminer that enables both MFM and PAM from EDI messages. 3 More precisely, each sent or received EDI message is considered to represent at most one event per process instance, but can potentially map to multiple process instances.
EDI messages 1
Parsing/ preprocessing
EDIminer 2 Mapping definition
Events (uncorrelated)
XES event log 3
Event generation Conventional process mining techniques
Process models Diagnostic information Performance analyses
EDIminer 4 is a toolset implemented as a stand-alone Java Swing application
that allows a user to perform the following tasks (cf. Fig. 1; cf. Sections 2.1-2.5):
1. Parse, preprocess and visualize a set of EDI messages using an enhanced
visualization method based on semantic technologies (automatic, Mark 1 )
2. Automatically (for MFM) or manually (for PAM) de ne EDI/event
mappings (Mark 2 )
3. Execute the de ned mappings on the parsed EDI messages to generate events
(automatic, Mark 3 )
4. Correlate generated events to process instances (semi-automatic, Mark 4 )
5. Export correlated events to an XES [
Running Example. In the remainder of this paper, we provide a proof of concept as well as illustrate the use of EDIminer by means of a running example that is based on real-world EDI data of an Austrian company from the consumer goods sector. For the sake of con dentiality, we will further on refer to this company as SupplierCo. The mentioned EDI data set is a snapshot of 466 inbound EDIFACT ORDERS (order) messages and 427 outbound EDIFACT INVOIC (invoice) messages collected between May 2012 and March 2013. It is related to a purchase order process which, according to the explanation of a company representative, consists of the following sequential activities (cf. Fig. 2): 1. The process is initiated when a customer sends an order message to SupplierCo, who receives and processes the order. 2. SupplierCo ships the ordered goods. If this can't be done all at once, the shipments are partitioned. 3. For each shipment, SupplierCo sends a corresponding invoice message to the customer. Usually, no invoices are sent before all ordered goods have been shipped. In addition to general invoicing data, the message contains information about the shipping date of the corresponding goods. 4 As of May 2013, EDIminer is publicly available and can be downloaded from http://edimine.ec.tuwien.ac.at.
caisedemo 4 In order to enable user-driven mapping of EDI artifacts to events and process instances, the contents of EDI messages rst have to be visualized to the user. A central question in this regard is what kind of visualization can be considered useful for this purpose. We argue that naive approaches of visualizing and mapping (only) plain data elements from traditional EDI standards will likely not lead to viable results for the purpose of creating event logs. The reason for this is that semantically accurate interpretation and visualization of EDI messages based on standards such as EDIFACT or X12 are non-trivial due to the existence of so-called quali ed and non-quali ed data elements in these standards. For example, in various EDIFACT standards, such as release D.01B, data element 3039 (Party identi er ), found in NAD (Name and address ) segment5 instances, may have a plethora of di erent concrete semantics (e.g., buyer's party identi er, seller's party identi er, etc.) depending on the value of a corresponding qualifying code in data element 3035 (Party function code quali er )6. We refer to the di erent concrete semantics of a data element as data element variants (DEVs). For example, an instance of data element 3039 quali ed by the code \BY" (Buyer ) in an instance of data element 3035 represents one DEV of data element 3039 - this DEV could be labeled, for instance, \Buyer PartyIdenti er'.
In ERoDbeIrt mEnignel er, we tackled the problem of handling quali catio1nofr1elationships
by implementing the approach described in [
The employed ontological approach allows for the visualization of the
contents of EDI messages in a tree structure of \plain" data elements and DEVs, as
illustrated on the left hand side of Fig. 3. The tree entries in red italics represent
DEVs. The hierarchichal structure of the tree entries resembles the
corresponding message type speci cations according to the underlying EDI standards. We
consider this kind of visualization an extension to the state-of-the-art in EDI
mapping tools, which are generally not capable of visualizing variants of data
elements, except at most for limited subsets of speci c EDI standards [
Autogenerate MFM mappings feature
Manually defined PAM mapping
Auto-generated
MFM mappings
Visualization of data element variants (DEVs) in
parsed EDI messages Drag&Drop Drag area
Drag&Drop Drop area
Generation of
events
SupplierCo Example. As shown in the tree on the left hand side of Fig. 3, the contained data elements/DEVs of the parsed ORDERS and INVOIC messages are displayed to the user. Thereby, DEVs such as order number reference identi ers (Mark 1 ) and actual delivery dates (Mark 2 ) are visualized in addition to corresponding \plain" data elements, such as reference identi er and date or time or period value, respectively. 2.2
De nition of EDI/Event Mappings Having this visualization of the parsed EDI messages at hand, the next step is to de ne EDI/event mappings specifying (i) what EDI artifacts constitute events and (ii) which EDI artifacts shall be used to populate event attributes. In the table shown on the right hand side of Fig. 3 (Mark 3 ), each row represents a mapping rule. The rst column (\Event trigger") speci es which values of data elements/DEVs in messages shall trigger the creation of events. The remaining columns specify the values of data elements/DEVs that shall be used to populate the attributes of these events, such as timestamp, activity and organizational resource. Additional event attributes may be added by a user through adding columns to the table using the \Add event attribute" button. In addition to data elements/DEVs, also user-de ned xed strings and some special variables (e.g., interchange meta-data) may be used in mappings.
To automatically de ne mapping rules for the purpose of MFM, users can
click the \Autogenerate MFM mappings" button. This leads to the creation of
one mapping rule per parsed message type where the event trigger is set to
the special variable message instance (i.e., every parsed EDI message triggers
an event), the timestamp attribute is set to the timestamp of the message
interchange, the activity attribute is set to the name of the message type (e.g.,
\INVOIC") and the resource attribute is set to the name of the party who
initiated the message interchange. Such auto-generated mappings resemble the MFM
approach described in [
SupplierCo Example. For SupplierCo's purchase order process we de ne combined mappings for MFM and PAM. By using the \Autogenerate MFM mappings" feature, two mappings (for ORDERS and for INVOIC messages) are created automatically ( rst two mappings in Fig. 3). A third PAM mapping is de ned manually such that the creation of events is triggered by occurrences of actual delivery dates in INVOIC messages (third mapping in Fig. 3). These events' timestamps are set to these same delivery dates and their activity labels are set to the user-de ned xed string \Ship goods".
With a view to facilitating subsequent event correlation, we manually add an additional event attribute invoiceNumber that is applicable for the two INVOICbased mappings and populate it with values of document identi ers. Additionally, we add an attribute orderNumber to all three mappings. In the ORDERSbased mapping, this attribute is populated by values of document/message number. For the other two mappings, it is populated by values of order number reference identi er. 2.3
Generation of Events Once the user has completed the de nition of EDI/event mappings, EDIminer can be instructed to automatically generate a set of events from the parsed EDI messages. However, since a single EDI message may contain multiple instantiations of identical data elements/DEVs at di erent positions, the question arises of how to decide which speci c event-triggering values of data elements/DEVs are to be related to which speci c attribute-populating values. In EDIminer, a heuristic method is used to arrive at reasonable matchings; due to space limitations, however, we refrain from describing this method in detail. Generally speaking, the closer an attribute-populating value is located to an event-triggering value in the hierarchical structure of segment group instances in a message, the more likely it is used for populating the corresponding event's attributes.
Visualization options (e.g., filtering, grid size) Configuration of synonymous correlators and concatenation of traces parameters
List of correlators with average trace
length and coverage statistics Configuration of key/value-correlation parameters
List of events List of process instances resulting from current
parameterization Visualization of traces resulting from current
parameterization
Event log
generation
Correlation of events originating from EDI messages to process instances may
pose a signi cant challenge [
SupplierCo Example. For the set of events generated from the SupplierCo example, events can be correlated by key/value-correlation using the previously de ned orderNumber event attribute. A visualization of the resulting traces ( ltered by a minimum trace length of three events) is shown in Fig. 4. 2.5
Generation of XES Event Logs
Once the events are correlated to process instances, EDIminer can be instructed
to export them to an XES event log [
In this paper we presented EDIminer, a software toolset that enables process mining from EDI messages supporting two di erent methodologies: (i) message ow mining (MFM) and (ii) physical activity mining (PAM). EDIminer employs semantic technologies to provide an enhanced visualization of contents of EDI messages and allows users to automatically or manually de ne EDI/event mappings. These mappings are used to automatically generate events, which are subsequently correlated to process instances (cases) using a semi-automatic approach. The correlated events can be exported to XES event logs and used with existing process mining tools.
Our future research plans focus on conducting an extended evaluation of EDIminer by means of case studies using real-world EDI data from di erent industries. Furthermore, we intend to evaluate and enhance the MFM and PAM methodologies using larger data sets than the one used in this paper.
Acknowledgment. This research has been conducted in the context of the EDImine project and has been funded by the Vienna Science and Technology Fund (WWTF) through project ICT10-010.