Process mining requires event logs. XES is a widely accepted format for storing and exchanging event log data, however, it only has XML serialization, leading to sub-optimal storage and time requirements in certain scenarios. JXES is a recently proposed serialization format for XES based on JSON, a more lightweight data interchange format. This paper presents Python-JXES: a Python implementation of JXES. Its read and write performance is evaluated against a certified state-of-the-art tool, and file sizes of JXES and XES serializations of open-source event logs are compared. JXES achieves up to 33% storage savings and up to 73% faster read speeds. Python-JXES can be used to facilitate eficient event log storage, streaming process mining and process mining on IoT devices.
CEUR
ceur-ws.org Screencast video Value Python-JXES 0.1 Apache 2.0 Python Microsoft Windows, GNU/Linux https://pypi.org/project/jxes/ https://github.com/MaxVidgof/python-jxes https://github.com/MaxVidgof/python-jxes https://youtu.be/8adiYqeczAs
Exchanging event logs is crucial for research and practice of Process Mining (PM) [
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
JSON serialization for XES has already been proposed by Narayana, Khalifa & van der Aalst [
XES standard 1849-2023[
Python-JXES defines a set of tools to convert PM4Py [
JSON ofers a smaller set of possible data formats than XML, thus time and ID attribute types are represented as strings in JXES. Time is serialized and de-serialized according to ISO 86011 using a regular expression.
While this implementation strives to be strictly conforming to the XES standard as defined in
Clause 8.1. and supports all features described in Section 2.1, some discrepancies exist. First, as
the standard only defines XML serialization, conforming log instances are also defined only
in terms of XML, making all other serialization formats technically non-conforming. Second,
log attribute xes.version is assigned string value "1849-2023", although it had to be of type
xs:decimal (i.e., float in JSON) as per standard Clause 5.1.2. Finally, as the original JXES
proposal included containers described in XES 2.0 standard [
The implementation and basic examples are publicly available on PyPi2 and GitHub3.
In this section, the implementation will be evaluated. The file size in JXES will be compared
to standard XML serialization. Read and write speeds of python-jxes will be compared to a
state-of-the-art tool PM4Py [
The evaluation was performed the following way: first, an XES event log was read by PM4Py, 5 times for each importer variant5 except rustxes, then, this log was serialized back to XES using PM4Py, also 5 times for each of the two exporter variants. Following that, the in-memory log was serialized as JXES for 5 times. Finally, the JXES log was imported for 5 times. In order to prevent caching, cache was flushed before each of the 5 iterations of every test. For every combination of file, tool and variant, the median of the 5 runs was used. All evaluation results can be found in a separate GitHub repository6.
(a) XES logs smaller than 150 MB (b) XES logs larger than 150 MB 4.1. Setup All tests ran on a laptop with Intel®Core ™ i7-1260P CPU, 32 GB DDR4 RAM and Corsair MP600 CORE XT NVMe SSD, running Ubuntu 22.04, Python 3.11 and PM4Py version 2.7.11.13. 2https://pypi.org/project/jxes/ 3https://github.com/MaxVidgof/python-jxes 4https://www.tf-pm.org/resources/xes-standard/for-vendors/tool-support 5https://github.com/pm4py/pm4py-core/tree/release/pm4py/objects/log/importer/xes/variants 6https://github.com/MaxVidgof/python-jxes-evaluation Open-source event logs such as BPI Challenge 2011-2013, 2015, 2017-2019, as well as Italian helpdesk log and Road trafic fines log were user for evaluation. 4.2. Size
Write benchmarks show similar results. Serializing event logs in JXES is faster than the best PM4Py XES serialization by at least 28%. Median write performance improvement is 40%, and maximum improvement almost reaches 47%.
Space savings by JXES are achieved because a lot less characters are needed. First,
JSON removes the necessity to specify data types as opposed to XML. Second,
there is no need for opening and closing tags in JSON. Because of this, an event
<event><string key="concept:name" value="example"/></event> can be written as
{"concept:name":"example"}, leading to significant space reduction. This is important for at
least the following scenarios:
1. Storing large amounts of historical data. This might not seem critical for smaller
event logs ranging between tens of kilobytes and tens of megabytes in size. However, for
real-life logs reaching tens of gigabytes in size, this becomes relevant.
2. Data transfer. When logs have to be transferred, e.g., between the system collecting
them and the system analyzing them, data size is also crucial. Also important is the case
of Streaming Process Mining, where smaller sizes may also reduce latency because it will
take less time to transmit an event completely.
3. Streaming Process Mining. In previous work [
It must be noted at this stage that binary serialization formats can ofer even more compression. For instance, pickle7 requires even less space, e.g., 1.1 GB for pickle versus 1.3 GB for JXES versus 1.9 GB XES for BPIC 2018. However, in contrast to both XES and JXES, pickle is a binary format that is not readable for humans and only allows to share data between Python programs, leaving out a significant fraction of the PM ecosystem. Similarly, file compression can reduce the file sizes significantly, however, it might still be impractical for streaming or IoT scenarios as (de-)compressing requires additional time and computational resources. In addition, the compressed files also cannot be read without decompression. It is worth pointing out though, that compressed JXES files are still about 10% smaller than compressed XES files containing the same event log.
While JXES shows significant improvements in read and write speeds, it must be noted that, in comparison to XML, Python’s default implementation of JSON does not support line-by-line reading, which could improve reading of large event logs or streams even further. However, non-standard implementations for incremental JSON parsing do exist, and exploring them is left for future work.
XES is a standard XML-based serialization format for event logs. As JSON-based formats are becoming increasingly adopted, JXES – a JSON serialization format for XES was proposed. This paper presents Python implementation of JXES and evaluates it against state-of-the-art XES 7https://docs.python.org/3/library/pickle.html tools for open-source event logs, showcasing significant improvements in storage requirements and read and write performance.
Future work is aimed at evaluating JXES in scenarios where it can be most beneficial, such as Streaming Process Mining and PM of IoT devices. Also evaluation in regular PM scenarios would be beneficial. Finally, the implementation might receive new features such as incremental JSON parsing.