Process mining is a young research discipline combining process and data science to analyze processes. Part of the success of process mining can arguably be attributed to the availability of open-source tools, datasets and standards that facilitate the exchange of ideas. The current leading process mining frameworks, ProM and PM4Py, present diferent tradeofs in terms of scalability, ease-of-use, and feature completeness. We present ProM4Py, a Python wrapper for the ProM framework combining the scalability and feature-richness of ProM with PM4Py's ease-of-use. ProM4Py seamlessly integrates with PM4Py, such that existing PM4Py scripts can be migrated to it with minimal code changes.
Value
Open-source tools play a pivotal role in the development of process mining. By centralizing existing algorithms and making them easily accessible, these tools facilitate the rapid exchange of ideas. Currently, ProM and PM4Py are the leading academic frameworks for process mining. Both tools strike diferent trade-ofs in terms of scalability, provided features, and ease-of-use.
The ProM framework [
In comparison, PM4Py [
In this demo paper, we present ProM4Py, a Python wrapper for the ProM framework, aiming
to fully and seamlessly integrate ProM’s CLI API into PM4Py. While there have been attempts
in the community to expose ProM’s functionality to other environments [
We demonstrate how ProM4Py can be used to, with minimal code changes, speed up common process mining workflows that were previously implemented with PM4Py. The tool is currently in early development and while the evaluation demonstrates the feasibility and practicality of the approach, further work is needed to improve its robustness and usability.
It follows from the first two requirements that ProM4Py must ofer an API that is similar to PM4Py and that PM4Py and ProM4Py’s data structures must be interoperable. The third requirement implies that ProM4Py must provide a generic solution to support ProM plugins.
The resulting tool consists of a JAVA backend and a Python client. The JAVA backend is implemented in ProM as a new PluginContext (ProM4PyContext) and runs on a separate process. At startup, the ProM4PyContext initializes the ProM framework, loads available plugins, and starts a HTTP server implementing a REST API that exposes ProM’s functionality. The Python client is responsible for establishing a connection with the ProM backend, coordinating API calls, and coverting Python objects from/to ProM, such that the user never interacts with the REST API. To illustrate ProM4Py’s usability, Listing 1 provides an example code demonstrating how to use ProM4Py to mine a PM4Py event log using ProM’s inductive miner plugin.
To use ProM4Py, one must first create a ProM4Py client object (line 2), through which methods can be called. This only requires passing the path where ProM4Py’s JAVA client is installed. If an instance of the backend service is already running, the client will refer to that one. Otherwise, a new instance of the service is started as a separate process. ProM4Py exposes Python functions corresponding to each supported ProM plugin (line 6). Notice that a PM4Py log is passed as parameter to the function. When calling a plugin, ProM4Py takes care of converting PM4Py objects into their ProM counter-parts. 1 # Create ProM4Py instance 2 prom4py = ProM4Py(prom4py_script_path) 3 # Load the EventLog using PM4Py 4 log = pm4py.read_xes(log_file_path) 5 # Mine the event log using ProM’s inductive miner with noise threshold = 0.2 6 net = prom4py.mine_petri_net_with_inductive_miner_with_parameters(log, 0.2) 7 # Extract the resulting net and visualize it with PM4Py 8 net = prom4py.create_accepting_petri_net(net[0]).extract() 9 pm4py.view_petri_net(net.net, net.initial_marking, net.final_marking) 10 # Explicitly terminate the engine 11 prom4py.terminate() 12 # CachedProMObjects are always garbage-collected when the script terminates
Listing 1: Sample code to discover a process model from a PM4Py event log.
ProM4Py avoids transferring any data from JAVA to Python. Instead, plugin calls always return a reference (called CachedProMObject) to the JAVA object representing the computation. CachedProMObjects can be converted to their respective Python objects using the extract() method (line 8), as long as an appropriate serializer is implemented on both sides (see Section 2.2). The user can choose whether the backend process should also be terminated (line 11), or kept alive to speed-up the startup of subsequent script executions. Last, upon termination (or process abort) ProM4Py’s client cleans up the objects in the backend’s cache store (see Section 2.1).
Notice that ProM4Py and PM4Py code can be freely mixed, and changing from a PM4Py function to its ProM4Py equivalent consists of a mere change in the name of the callback.
Figure 1 depicts ProM4Py’s architecture, highlighting the roundtrip process involved in discovering a process model. When a ProM4Py method is called (step 1), the Python client first checks whether its input are Python objects or CachedProMObjects. In the first case, it first transfers the objects to ProM. This is done by serializing the object and calling ProM4Py’s /cache endpoint (step 2), which de-serializes the object and stores it into ProM4Py’s CacheStore (step 3). The CacheStore returns the object’s key, which is forwarded to the Python client (steps 4 and 5). In case the inputs are already CachedProMObjects, these steps are skipped.
Then, the Python client issues a call to the /plugin endpoint with the name of the plugin and a list of parameters cache keys (step 6). The /plugin endpoint ofers a standard interface to call all existing ProM CLI plugins. Upon request of a plugin execution, ProM4Py retrieves the JAVA objects corresponding to the plugin’s arguments from the CacheStore (steps 7 and 8). It then uses the plugin’s name and the parameters’ typing information to determine the appropriate method to be called. The method is then called (steps 9 and 10). Its results are also stored in the CacheStore (steps 11 and 12). The cache keys are returned to the Python client (step 13), which creates a CachedProMObject and returns to the user (step 14).
This choice of architecture brings a few advantages and challenges. On the positive side, using CachedProMObjects minimizes the amount of data copying, which can be a potential bottleneck. Furthermore, the /plugin endpoint’s generic interface means that ProM4Py can,
ProM4Py 1. mine_petri_net(log) 14. net_obj, im_obj, fm_obj 2. /cache log 5. return log_key
6. /plugin mine_petri_net log_key 13. return net_key, im_key, fm_key 3. store(log)
CacheRequest
Handler 4. return log_key Cache Store 7. get(log_key) 8. return log
Plugin Request
Handler 11. store(net, im, fm)
10. net, im, fm 12. return net_key, im_key, fm_key 9. execute ('mine_petri_net', log) Script Executor
InductiveMiner
Package 1
. . .
Package N in principle, support any ProM CLI plugin (see Section 2.2). On the negative side, since there are two separate processes, the system may reach inconsistent states such as orphan ProM processes, or ProM4Py objects pointing at non-existing instances.
We distinguish between two levels of plugin support. A fully supported plugin is a plugin for which its input and output types can be freely interchanged between both runtimes. A partially supported plugin is a plugin that cannot serialize at least one of its inputs or outputs types. Among them, we distinguish between uni-directional support (for which its inputs can be serialized, but not the output, or vice-versa), and fully-internal plugins, when neither its inputs nor its outputs can be serialized. Since ProM4Py’s API accepts CachedProMObjects as arguments, it is still possible to interact with these plugins by using other ProM plugins to read/write inputs from/to the file system, and convert between internal data-structures in ProM.
The notion of a fully/partially supported plugin assumes that the plugin is compatible with the schema from Figure 1, i.e. it assumes that parameters can be independently parsed and that adding support for them is merely a matter of implementing the required serialization methods. However, some plugins are incompatible with this schema. Supporting such plugins requires an extra implementation efort and result in a user-unfriendly API. Unfortunately, ProM’s alignment package, arguably one its most used plugins, falls into this category.
ProM’s alignment plugins require a mapping from transitions in the Petri net to event classes in the event log. However, the mapping relies on the ID of the JAVA object of the Petri net’s transitions, which is only assigned once the Petri net is loaded into ProM. Therefore, the approach of serializing all input arguments separately does not work. Instead, ProM4Py must implement additional functionality to create this mapping. As a result, a custom alignment end-point was developed to explicitly support this plugin. Luckily, most ProM plugins are compatible with the schema from Figure 1, such that the majority of ProM’s functionality can be supported without resorting to such workarounds.
We demonstrate how to use ProM4Py as a replacement for PM4Py in a common process mining
workflow consisting of mining a process model and aligning the event log with the discovered
model. For the evaluation, we use the BPI Challenge 2020 event log [
The first thing to notice is that loading the event log is consistently faster in ProM compared to Python. Loading the log is a bottleneck in simpler input scenarios such as DD and RFP, where the subsequent computation steps are relatively inexpensive. Generally, ProM4Py implements all computation steps more eficiently than PM4Py, the only exception being the discovery steps for DD and RFP. This can be partially explained by the usage of native DataFrame APIs in PM4Py’s inductive miner implementation. However, during our experiments, we observed that PM4Py’s inductive miner produces diferent results from those of ProM, i.e. the implementations of the two frameworks diverge. Thus, even without providing a performance improvement, ProM4Py can still be useful to provide access to the reference implementation of an algorithm.
When considering the alignment computation, we observe that performing this step in ProM
is significantly (up to five times) faster than in Python and that the overhead associated with
transferring the alignment results is, in most cases, negligible. Overall, ProM4Py is between 2 to
4 times faster than PM4Py. Since we used the same algorithms in both scenarios, this diference
is attributed to the inherent performance diferences between Java and Python, as well as other
implementation-specific details. But notice that ProM also implements more eficient alignment
plugins which are not available in PM4Py[
We believe that it is possible to improve the integration between the Python and JAVA run
times by using a standardized (and faster) data exchange format [
We thank the Alexander von Humboldt (AvH) Stiftung for supporting our research.