Road accidents cause millions of fatalities worldwide and impose a significant economic burden on societies. To address this problem, road safety researchers have mainly applied statistical or machine learning methods to predict accident occurrences and identify the causes of such crashes, but to the best of our knowledge, no work has explored how to optimally address the causes of such crashes to minimise accidents' severity. Recently, eXplainable AI (XAI) techniques have been applied in transportation to evaluate the efect of accidents' contributing factors. Limited work however delved into optimum ways to reduce accident severity by minimising changes needed to road network's infrastructure using XAI. In this work, we apply counterfactual explanations, a popular XAI technique, to road accident data, to identify optimal changes to infrastructure to improve road safety, by converting severe accidents into minor accidents. Traditionally, counterfactual explanations are used for single instances(accidents), which is not appropriate to this problem since the goal is to find actionable changes to the road infrastructure to convert as many severe accidents to non-severe accidents. Thus, our proposed methodology is based on multi-instance explanations. The proposed methodology is evaluated in a case study with real accident data from Cyprus.
According to the World Health Organisation (WHO), approximately 1.4 million accidents occur each year worldwide leaving millions of people injured. Road accidents constitute the eighth leading cause of death worldwide and this number is likely to increase if this problem is not addressed efectively.
Predominantly research on road safety utilizes classical statistical techniques such as Logistic, Poisson,
and Negative binomial regression [
During road safety analysis, trafic accident data is used to develop models to predict and understand
the causes of accidents through the identification of relationships among contributing factors and the
outcome of the models. Contributing factors can be combinations of properties relating to the road
infrastructure, driving behavior, and environmental conditions. These are considered independent
variables and the outcome (accident occurrence or severity) is the dependent variable. Predicting an
accident before it happens gives the chance to policymakers to take precautionary measures to minimise
or prevent accidents from happening. In the machine learning domain, researchers use techniques such
as eXplainable AI (XAI) [
In transportation, one of the main approaches to improve safety is by addressing one or more causes or
risk factors that are associated with accident occurrence. The trafic safety literature highlights diferent
modifications to either the policy or road infrastructure to improve safety. For instance, infrastructural
modifications could be, changes in horizontal curvature of roads, shoulder widths, the width of the
median that separate lanes, etc. Such modifications, however, are usually introduced after analysing
the problem, designing a solution, simulating it, and then applying it, without having any guarantees
of the efect that these changes can make. Therefore, most of such projects fail to meet their goals.
Additionally, such modifications are usually introduced without knowing the optimum degree of change
(for example speed limit) to archive the desired efect. Optimisation has been applied in transportation
to minimise the budget required to improve safety as reported in work by [
Explainable AI (XAI) [
The goal of this work is the maximisation of road safety (reducing severe accidents) by minimising the infrastructural changes to a road network to improve safety. This is achieved by explaining using counterfactual explanations a machine learning model trained to predict accident severity using historical accident data of the specific road network. Counterfactuals, however, usually provide explanations for single instances (accidents) rather than a group of cases. Moreover, counterfactual explanations are applied on numerical variables (e.g. road width, speed limit, etc.), even though categorical variables (e.g. type or intersection) are a key type of features in diferent domains, including transportation. The approach proposed herein addresses these two problems by finding counterfactuals that satisfy multiple instances(accidents) simultaneously, characterised by both numerical and categorical features. Therefore, the method provides policymakers with recommendations that not only indicate the factors that contribute to the problem but also provide them with information regarding the degree of change to these factors to archive the desired efect. Therefore, the method can potentially minimise the costs of satisfying stakeholders’ goals.
The remainder of this work is organised as follows: Section 2 elaborates on the background of counterfactual explanations, the related literature, and the motivation behind the proposed counterfactual methodology. The next section introduces the methodology (Section 3) and elaborates on its application in a road safety case study using accident data (Section 4). Finally, we summarize and discuss future directions (Section 5).
A counterfactual explanation [
To be useful in practice, a counterfactual must not only be feasible (i.e. valid) but also as simple as
possible - e.g. not too many recommendations or big changes [
In order to not make any assumptions on the data domain, we use the symbol ⊕ to denote the application/execution of the counterfactual ⃗cf to the original instance ⃗orig. While in the case of real and integer numbers (e.g. = R) this reduces to the translation (i.e. (⃗cf) = (⃗orig) + (⃗cf), in the case of categorical features it denotes a substitution – i.e. (⃗cf) = (⃗cf).
Note that Definition 1 constitutes a non-causal approach - i.e. no causal model of the world is included.
There exists an entirely diferent line of research on counterfactuals utilizing structural causal models
to incorporate causal knowledge [
There exists a wide variety of methods for computing counterfactual explanations – i.e. solving
the optimization problem. The model-agnostic methods utilize gradient-based optimization methods
that can be applied on any black box model [11, 12, 13], while the model-specific methods use details
of the specific model’s architecture to calculate recommendations [
In many real-world applications of counterfactuals, one is interested in gaining knowledge about a set or group of instances instead of a single instance, which is the usual case. For instance, in any organization, the human resource department is interested in minimizing employees’ attrition, since this causes several problems. To understand the cause of attrition and deploy appropriate (global) countermeasures, an organization needs to consider all cases (employees intending to leave) and find a change (maybe increase employee salary) that will guarantee the retention of as many employees as possible [14, 15].
To find a common feasible recommendation for a group of instances (i.e. employees), counterfactual explanations have been recently extended towards multi-instance counterfactual explanations (also called group-counterfactuals) [14, 15, 16, 17]. A multi-instance counterfactual states what to change on a group level (e.g. increasing some specific attribute for all instances in the group by the same amount) such that the outcome for this group of instances changes simultaneously in some desired way. Like counterfactual explanations, the computation of a multi-instance counterfactual explanation ⃗cf for a set of cases can be formalized as an optimization problem: Definition 2 (Multi-instance Counterfactual Explanation). Let ℎ : → denote a prediction function, and let be a set of labeled instances with the same prediction ∈ under ℎ(· ) – i.e. ℎ(⃗) = ∀⃗ ∈ . We are looking for a single change ⃗cf ∈ R that, if applied to the instances in , changes as many of their predictions to some requested output cf ∈ .
We call all pareto-optimal solutions ⃗cf to the following multi-objective optimization problem multiinstance counterfactuals: min (︀ (⃗cf) , ℓ(ℎ(⃗1 ⊕ ⃗cf), cf), . . . , ℓ(ℎ(⃗|| ⊕ ⃗cf), cf)︀) ⃗cf ∈ (2) where (· ) denotes the cost of the counterfactual, and ℓ(· ) denotes a suitable loss function penalizing deviations from the requested outcome cf – suitable loss functions might be the mean-squared error or cross-entropy loss, while the cost (· ) might be implemented by a -norm.
Note that in contrast to normal counterfactuals (Definition 1), multi-instance counterfactuals (Definition 2) require multiple constraints – one constraint for each case in the set . Similar to normal counterfactuals, one could merge all constraints into a single objective which would then enable the use of general gradient-based or black-box methods for solving the optimization problem.
A major challenge in the computation of multi-instance counterfactuals is that there might be no feasible solution – i.e. it might be impossible to find a single change ⃗cf that is feasible for all instances in . Therefore, one might either want to relax the constraints and compute a change ⃗cf that is feasible for as many as possible instances in ⃗cf, or find a grouping/clustering of such that there exists a change ⃗cf for each of the groups that is feasible for all instances within this sub-group – here the additional challenges of finding such sub-groups arise. We denote the percentage of instances for which the explanation is feasible as accuracy.
Unlike counterfactual explanations (Definition 1), multi-instance counterfactuals (Definition 2) is a novel concept and consequently, existing work on this is rather limited. For instance, the work by [15] applies multi-instance counterfactuals to the employee attrition problem but only considers a linear classifier. One of the latest works such as [ 14] proposes a counterfactual explanation tree, which assigns counterfactual explanations to a learned decision tree that assigns samples to groups – this method is only applicable if an automatic clustering into sub-groups is needed. Besides that, most existing work for multi-instance counterfactuals can be interpreted as summarizing or aggregating local counterfactuals. In the work of [16], multi-instance counterfactuals are generated by first computing individual counterfactuals and then selecting those that maximizes the cover of a given set of instances. Similarly, [18] tries to obtain a global explanation by simply aggregating local explanations. However, these methods cannot guarantee high accuracy because they consider all instances separately.
Herein we propose a methodology that utilizes multi-instance counterfactuals to analyze road accident records and compute suggestions on how to improve road safety – the only assumption we make is that the collected road accident records are labeled with accident severity. The proposed methodology consists of three main steps as illustrated in Figure 1: 1. Train a binary classifier using road accident data to predict the severity of accidents. Train classifier to predict serverity of accidents
Cluster accidents into groups
Multi-instance Counterfactuals
Generate recommendations how to improve road safety
2. (Optional:) Cluster all severe accidents into groups of similar characteristics. 3. Compute a multi-instance counterfactual explanation for each group - i.e. what to change to turn a severe accident into a non-severe (slight).
We interpret the computed multi-instance counterfactuals as potential suggestions on how road safety can be improved. Note that the grouping in the second step of the methodology is optional because one could simply consider all severe accidents as one large group. However, depending on the use case a more refined grouping might allow the computation of suggestions for more specific questions: For instance, depending on the nature of the collected data, it might be possible to group road accidents based on their location (e.g. rural area vs. urban) or timing of the accident (day, night). In this way, one could generate more specific recommendations (e.g. area/time specific) for certain types of accidents.
We empirically evaluate our proposed methodology (see Section 3) in a case study using real-world accident data. 4.1. Data The original data consists of accidents that occurred in Nicosia, Cyprus during 2007 - 2012. After merging and cleaning the data, 9829 cases were left each consisting of 58 attributes. To ensure that the recommendations to be made are actionable, we only consider 12 attributes that could be changed in practice, in contrast to attributes that cannot be changed, such as the age of drivers. Examples of mutable attributes are the road width, speed limit, trafic control, pedestrians crossing, the existence of a median in the road, etc. The accident type attribute is converted into a binary variable by combining fatal with severe accidents and labelling them as severe, and slight(minor) with property damage accidents and labelling these as non-severe accidents.
Because the data mainly consists of numerical (e.g. integer) and categorical attributes, most standard machine learning models are not suitable. Thus, in this work, we decided to use a tree-based classifier (i.e. Xgboost [19]) that can handle such categorical attributes without transforming them. The classifier is trained to predict the severity of a given accident using diferent hyperparameters that have been tuned to improve the model’s performance such as depth of trees, number of estimators, learning rate, and scale-pos-weight to address data imbalance. The classifier achieves an average F1-score of approx. 80% on the test data – i.e. we split the data into train (70%) and test data (30%).
We consider three groups of accidents (among the severe accidents only) in our experiments: All Severe accidents; Severe accidents in rural areas; and Severe accidents in urban areas. These groupings can be changed depending on the case study. For instance, a transportation engineer might use only accidents at a specific location in the road network (black spot). Table 1 shows diferent multi-instance recommendations for each of these groups. The column multi-instance counterfactuals show the changes that need to be made so that the severe accidents are converted to non-severe. The variables in parenthesis refer to the property of the infrastructure and the number next to each one defines the type and degree of change. Therefore for the case (Trafic-control, 3) which is a categorical variable, it denotes that the type of trafic control needs to change to trafic-light (this refers to the number 3) for all of these accidents so that these change to non-severe. Similarly for continuous variables such as ’speed’ the recommended change is defined as a number with a sign indicating positive or negative change. In the case of speed limit, most recommendations indicate a reduction in speed limit which abides to road safety literature [20, 21].
Because in this case study the data also include non-continuous attributes, existing methods for computing multi-instance counterfactuals cannot be applied out of the box. To address this issue, we built an evolutionary algorithm for computing multi-instance counterfactuals that are guaranteed to adhere to the specific attribute ranges and yield feasible solutions. The evolutionary (i.e. genetic) algorithm treats all variables as discrete and iteratively mutates and merges (cross-over) candidate solutions until convergence. To guarantee the feasibility of the final multi-instance counterfactual ⃗cf, we construct the set of feasible changes for each feature of numerical variables as follows – assuming non-negativity which can be achieved by adding a constant:
= − min{(⃗ )} and = − max{(⃗ )} where and denote the maximum and minimum feasible value of the -th feature, and the final set of feasible changes is then given as [, ]. These sets are used when computing mutations in our evolutionary algorithm of existing individuals during the optimization. As an objective, we use the zero-norm (i.e. setting p=0 in the p-norm) - by this, we aim to minimize the number of suggested changes. Together with the constraints, this yields the following optimization problem: arg min ︁( ‖⃗cf‖0 , ℓ(ℎ(⃗1 + ⃗cf), cf), . . . , ℓ(ℎ(⃗|| + ⃗cf), cf)︁)
⃗cf
To generate reliable results and avoid recommendations that are based on a bad train-test data split, the experiments are conducted using a 3-fold cross-validation. Thus, we generate three multi-instance explanations for each accident groups (i.e. all severe accidents, severe accidents in urban areas, and rural areas) as shown in Table 1.
Besides listing the explanations in Table 1 (tuple of attribute and recommended change), we also added a column for the accuracy of the explanation which refers to the percentage of instances for which the explanation correctly changes their prediction (Table 1). From the results, we observe that the generated multi-instance counterfactuals are almost always feasible (the accuracy is close to one). This demonstrates that our implemented evolutionary algorithm is able to compute feasible solutions with high reliability.
The results from the multi-instance counterfactuals show the required changes to infrastructure to convert severe accidents into non-severe. The variables that are considered important and thus are part of the recommended changes are the Trafic Control, which takes the values trafic signals, roundabout, police, stop sign and none; Road Width, stating the width in meters; Speed Limit in Km/h; Pedestrian Crossing type, that takes values: zebra crossing, pedestrian trafic signal crossing, and pelican crossing; Constriction variable that takes the values: one-way, two-way bridge, none; and Brake (3) (4) Algorithm 1 Multi-instance Counterfactuals for Road-safety Analysis the number of evolutionary steps Output: Recommendations (i.e. multi-instance counterfactual explanation) ⃗cf Input: Set of labeled (severe vs. non-severe) accidents = {(⃗, )}, hyper-parameter denoting 6: 9: 10: 11: 12: 14: 13: end for ◁ Repeat in k-fold cross validation ◁ Severe accidents only ◁ Optional ◁ Evolutionary algorithm for computing a multi-instance counterfactual
◁ Random initial population of solutions – respect feature 2: Fit XGBoost classifier 1: Split data into train train and test test set
ℎ(· ) to train 4: (Group accidents and pick group of interest) 3: Consider all accidents ⃗ ∈ test with ℎ(⃗) = "severe" – create severe 5: Compute feature bounds {[, ]} Eq. (3) on severe ⃗ bounds! 7: { cf } = random_init({[, ]})
Evaluate fitness of each ⃗cf using Eq. (4) Select best ⃗cf for next generation Apply random mutations considering feature bounds {[, ]}
Apply cross-over to create next generation of solutions {⃗cf } 15: ⃗cf = arg min ︁( ‖⃗cf‖0 , ℓ(ℎ(⃗1 + ⃗cf), cf), . . . , ℓ(ℎ(⃗ |severe| + ⃗cf), cf) ︁) Grouping
Accuracy ↑ {⃗cf}
All All
All Urban Urban Urban Rural Rural Rural 1.0 ◁ Select best solution as the final recommendation ⃗cf Multi-instance counterfactual explanations from the conducted case study.
Lane, that describes the width of the footway/shoulder in meters. The recommended changes mainly highlight the importance of trafic control type and speed limit, factors that are known in the literature to afect road safety [ 20, 21]. Additional changes include an increase in road width and break lane width, recommendations which are also consistent with the literature [22].
To validate our approach we used SHAP (SHapley Additive exPlanations) [23], an XAI model-agnostic method, to identify which features are influencing mostly accident type. The SHAP summary plot in Figure 2 shows that the road width, break lane width, and trafic control are key features along with conjunction type and speed limit. These results verify that our method uses these key features to make counterfactual recommendations. The limitation of SHAP however is that it does not indicate which combinations of features must change and by how much so that the accidents are converted from severe to non-severe.
In this work, we proposed a methodology for analyzing road accident data and using XAI to generate suggestions on how to improve road safety. Our methodology requires training a classifier to predict the severity of accidents (severe, non-severe) and then recommending changes to the input of a set of severe accidents using multi-instance counterfactual explanations, so that the predictions of the model change to non-severe accidents. The approach enable us to compute a global recommendation on how to reduce the severity of accidents by converting all severe accidents into non-severe using minimum alterations to chargeable features (referring to infrastructure), and by doing this improve road safety. We also conducted an empirical case study on real-world data, which showed that our proposed methodology computes reasonable recommendations that abide with the literature.
Despite the promising results, some aspects need further investigation: • We observed that the suggested changes are often large (for example the speed limit), and it is not clear how plausible those changes would be in practice. We think this is mainly due to the fact that the designed evolutionary algorithm does not have any distributional knowledge about the data - i.e. which variable values or combinations are more often observed in the real world. Such distributional information might be used as part of the objective function, thus, automatically punishing large changes that would be costly to implement. Currently, we are working on an extension where the evolutionary algorithm is given distributional information which it utilizes when generating random mutations, new individuals, and cross-overs. By this, we hope to improve the quality of the recommended infrastructural changes significantly. • In the presented case study, we either considered all severe accidents as one large group or manually split them into two sub-groups based on a location attribute from the dataset. While the computation of multi-instance counterfactuals worked well for both groups (large group, and two smaller sub-groups), it remains unclear if other interesting or beneficial (for the generated suggestions) groupings exist. An automatic clustering might not only reveal interesting clusters within the accident data set but also give rise to better and more specific suggestions on how to improve road safety. Currently, we are investigating how to cluster cases into groups such that the resulting multi-instance counterfactuals are as simple as possible.
This research was supported by the Ministry of Culture and Science NRW (Germany) as part of the Lamarr Fellow Network. This publication reflects the views of the authors only.