Currently, nancial statements auditors perform the tests of controls based on sampling. However, when using a sampling approach, information is lost. To counter this drawback, data analytics has been applied as a method for auditors to provide assurance while using all data. Speci cally for testing controls, the potential of process mining has been explained in literature. Indeed, conformance checking can be used to compare real process executions with a normative model. However, the outcome of current conformance checking techniques is too vast for an auditor to inspect further. The identi ed deviations are at an atomic level (skipped and inserted tasks) and there is no feasible approach to gain a quick overview of the deviations. In this paper, we propose an approach to categorize deviations, which enables auditors to quickly gain an overview of di erent types of existing deviations along with their frequencies. Categorizing deviating process instances can also give an insight for assessing the risk at case level.
The objective of nancial audit practice, for auditors is to express their opinion
on correctness and fairness of organizations' nancial statements. To achieve this
goal, auditors test business process's controls e ectiveness and perform
substantive tests of balances and accounts which impact the nancial reporting. Test
of process controls should be done before investigation of balance sheets and
accounts. The rationale is that if the control settings are rigid, there is
reasonable assurance, for the auditor, to rely on the organization's internal controls.
If the results of tests of controls are not satisfactory, more substantive evidence
should be gathered. That means more time and e ort should be dedicated while
testing the balance sheets and statements. To test the controls, as required by
Audit Standard No.5 [
The advent of process mining techniques [
The question is how the idea of conformance checking (comparing a log with a model) can be made actionable for auditors. The approach we consider in this paper is to create a di erent layer of deviations that would allow us to categorize those deviations in sets that are meaningful and manageable for an auditor. Concretely, we would like to develop (or alter) a conformance checking algorithm that identi es deviations in such a way that i) they will be meaningful for auditors, ii) gives them an insight of di erent types of existing deviations and their frequencies in the executed behavior, and iii) helps auditors to prioritize deviations based on their risk level.
In the remainder of this paper, we propose an e cient interpretation of deviations in section 2, which can answer the above question. In section 3, we explain how we will implement the idea and we conclude in section 4.
To address the problem of too ne-grained deviation types (skipped and inserted) mentioned in the previous section, we de ne a set of six deviation types that we presume would be meaningful to an auditor, even without the context of the complete trace. These deviation types interpret deviations at a higher level, while keeping the location and root cause of the mismatches in each case. The six types that we propose are: missing a sequence, existence of an extra sequence, loop on a sequence, repetition of a sequence, swapping two sequences, replacing one sequence by another sequence. Note that a sequence can contain one or more activities. These proposed types are explained in table 1 due to lack of space. For each deviation type an example is provided, based on the procurement model presented in section 1.
Type Description Example 1 Missing a se- A sequence of events <Create PO, Release, IR, GR, quence that should have taken Pay>
place, is not executed Sign is missing 2 Existence of ex- A sequence of events <Create PO, Sign, Change Line, tra sequence is executed that it was Release, IR, GR, Pay>) not designed Extra event Change Line exists 3 Loop on a se- A sequence of events <Create PO, Sign, Release, IR, GR, quence is repeated while only Pay, Pay)> a single occurrence was Loop on Pay designed 4 Repetition of a An executed sequence <Create PO, Sign, Release, IR, GR, sequence is repeated later in an- Sign, Pay)>
other part of trace Repetition of Sign after GR 5 Swapping two Two sequences have <Create PO, IR, Sign, Release, sequences changed their order GR, Pay)> <Sign, Release> and IR are swapped 6 Replacing one A sequence takes place <Create PO, Sign, Pay, IR, GR, sequence by an- instead of another Pay> other sequence missing sequence. Release is replaced by Pay
The idea is to develop an algorithm that provides auditors with an overview of deviations according to their categories. Next, it should be feasible to drill down to the sequences that were subject to the deviations. For instance, the deviating trace <Create PO, IR, Release, Pay> will be described as follow: Missing an event (with two sub-categories: Sign is missed and GR is missed ) and Swapping events (Release and IR are swapped ). On a log level, for example, this could give: "This logs shows 500 times a "Repetition of a sequence", this is stemming from 150 repetition of <Sign, Release>, 150 repetition of IR, and 200 repetition of GR." which describes existing deviations in the log in categories along with their frequencies.
Therefore, the contribution of this paper is to interpret deviations using these types, which enables auditors to perceive di erent types of devisions and possibly related risk, as one sees them intuitively (rather than on an atomic level). 3
Before the development of the desired algorithm, we will perform a eld research. The methodology is to interview auditors to test our proposed deviation types in their approach. The eld research will be executed to gain insights in how complex deviation types might be interpreted by human experts (as apposed to our assumptions). Consider again, the deviating example in section 1, <Create PO, IR, Release, Pay>. The deviation type 'swapped Release and IR' can be interpreted in a di erent way like 'Release is postponed after IR' or even 'IR is advanced before Release'.
When various deviation types exist in one trace, the combination of them also
can be interpreted di erently. Avoiding di erent interpretation of deviations is
important because it may lead to assigning wrong risk level to them. Hence, we
believe performing the eld work research is necessary before implementation
of the idea. After the eld research, we plan to build the algorithm with the
desired requirements. Current known algorithms have already been investigated
to what extend they could help in achieving our goal. The shortcoming of
costbased conformance checking tool, proposed by Adriansyah et al. [
This paper introduces the idea of developing a technique for organizing the identi ed deviations in event logs into certain categories. A set of six di erent deviation types are proposed to enable auditors to gain an overview of existing deviations. During our eld research, we will also study what type of information auditors use to assess the risk of deviation types. This insight will be used in a followup phase to go from deviation to risk classi cation. Moreover, the correlation between risk level and the complexity of each deviating trace (i.e., the number and the variety of the mismatches in each deviating trace), in a real life setting will be investigated.