Algorithmic Fairness in Geo-intelligence Workflows through Causality

Abstract

In this paper, we investigate how causality (causal inference) can be used to detect bias and ensure fairness in geo-intelligence workflows . We investigate the usefulness of such a causality-based approach in the context of an early warning system that predicts building damage at municipality levels in The Philippines. We use directed acyclic graphs to reason about the causal relationships in the model case study and quantify the relationships using structural equation modelling. Mediation analysis is also used to validate the causal relationships between variables. We find cases of confounder bias and Simpsons paradox that could potentially bias the damage predictions. However we note that the objective and outcome variable in the early warning system needs to be defined in a manner that allows for more nuanced investigation on fairness (i.e., from damage assessment to impact assessment).

Keywords

• eol>Biases
• algorithmic fairness
• geo-intelligence
• disaster early warning systems

1. Introduction

fairness and transparency as data biases and technical limitations can afect the reliability and trustworthiness of geo-intelligence workflows.

In this paper, we investigate how causality (causal inference) can be used to promote fairness by detecting bias in the data and design of geo-intelligence workflows. As a case/study we consider an early warning system that predicts building damage by tropical cyclones (TC) at municipality levels in The Philippines. Fairness is a pertinent issue in disaster response/management because of the limited resources and the lives and livelihoods at stake. We explore how causality can be used for algorithmic fairness since non-causal (associational) models have been shown to replicate, reinforce and propagate biases in observational data. Furthermore, some biases and statistical anomalies cannot be addressed using association/correlation language used in non-causal models (e.g., confounder bias [Figure 3] and the Simpson’s paradox) [ 4 ].

We follow Friedman and Nissenbaum’s [ 5 ] definition of a bias in computer systems. By their definition, a bias is a systematic, and unfair discrimination of individuals or groups of individuals. Friedman and Nissenbaum develop a framework of understanding bias in computer systems as either preexisting bias (e.g., data generated from biased society or institutions), technical bias (technical limitations of the algorithms) and or emergent bias (i.e., biases changing real world use such as changing societal values). In the context of our case study, the data does not contain the quintessential sensitive variables (e.g., gender, race/ethnicity). However, the case study uses building typologies as predictor variables which are often considered an indicator of socio-economic well-being. Therefore a biased damage assessment in this context would consequently cause unfair distribution of aid resources to municipalities with a high number of building typologies vulnerable to tropical cyclones.

Although there is literature on using causality for quantifying disaster risk (e.g., [ 6, 7 ]), to the best of our knowledge there are none so far on assessing biases and fairness using causality in this context. Though our case study is on aggregated data and does not contain the typical problematic sensitive variables (e.g., gender, ethnicity), it is still important to ensure that there are no biases in the early warning system and that it is fair. In this paper we present preliminary results on detecting biases through causality.

2. The Case Study and Data

The Philippines is prone to tropical cyclones (characterized by high speed winds and heavy rainfall) which cause loss of lives and infrastructure damage. While there are accurate models that predict the characteristics of tropical cyclones (e.g., wind speed and rainfall) it is still a challenge to predict and quantify the impact it would have on people. Because of socioeconomic inequalities that influence choice of building materials, often this impact is measured by assessing building damage. Because of these challenges (high frequency of tropical cyclone events and cascading efects), geo-intelligence workflows are increasingly leveraged to guide Impact-based Forecasting (IbF) that in turn inform disaster anticipatory actions such as Forecastbased Financing (FbF) [ 8 ]. 510, an initiative of The Netherlands Red Cross developed an early warning system for The Philippines. Their model and data serves as our case study.

The observation data is from previous 39 tropical cyclone incidents in The Philippines. The data is aggregated at the municipality level (1486 municipalities), therefore the variables are an aggregate representation of the hazard efects in each municipality. Hazard variables include wind speed and amount of rainfall and physical vulnerability variables are captured by house typology numbers (grouped by roof type and wall type). Geographical variables are also included (e.g., coastal length). The resulting early warning system under study is a regression model that predicts the damage percentage in municipalities identifying the most impacted (see Table 1 for metrics).

3. Methods: Causality

Causal inference is a methodology that aims at quantifying cause and efect from observational data. Causal inference has found applications in algorithmic fairness [e.g., 9, 10, 11] and in explainable AI [ 12 ]. There are diferent methods on causal inference such as Bayesian Networks and Structural Equation Models (SEMs). BNs and SEMs are popular because of the ease of representing causal relationships using graphs, specifically directed acylic graphs (DAGs) which do not have feedback loops [ 4 ]. In this paper we use SEMs (Equation 1) because they can also be used in mediation analysis to estimate and validate causal chain graphs (Figure 2).

= (, ), = 1, . . . ,

Where is a function model (e.g., linear regression models) with as parent nodes of and a random error.

Mediation analysis aims to quantify both the direct efects and indirect efects of an exposure variable to an outcome. This is done in two steps, one regressing on the outcome variable on the exposure variable (Equation 2) and two regressing the outcome on both exposure and mediator variable (Equation 3). The diference of the exposure coeficients in the two model gives the efects of the exposure (E) interfered by mediator variable (M) (Equation 4) [ 13 ].

= 0 + × + = 0 + 1 × + 2 × +

= − 1

To detect potential biases through causality in the early warning system we first define a DAG informed by literature (e.g., [ 14 ]) of how tropical cyclones cause damage. This is however limited by the available variables in the data. We use correlation analysis to identify relationships, mediation analysis to confirm chain relationships (Figure 2), and linear regression models to identify cases of Simpsons paradox. Importantly, the DAG help identify potential confounder biases. Secondly, we conduct a path analysis, quantifying the efects of variables on the outcome. We use SEMs with the functions specified as linear regression models to quantify the direct efects among the variables. These steps are a precursor to conducting further causal fairness checks, for example path-specific counterfactual fairness as illustrated by Chiappa [ 15 ]. (1) (2) (3) (4)

4. Preliminary results

Figure 1 shows our causal representation of the tropical cyclones early warning system. It is informed by literature of how tropical cyclones cause infrastructure damage [e.g., 14] and this damage varies because of the building typologies1. With many variables to consider, we focus on the ones that have a strong correlation with the outcome variable and among each other. Figure 5 shows the correlation between the variables. Furthermore, departure from the initial associational ML design to a causal model leads to removing variables that do not have a causal link (or interpretation) to the other variables. For example, the number of households registered for social protection can be associated with higher damage levels but there’s no causal link/path between this variable and the outcome.

The DAG is based on the causal reasoning that natural hazards have a direct causal efect on damage of buildings. The hazard components - wind speed and total rainfall2 - have a strong correlation (see Figure 5H) and statistical significance to the outcome (damage). Since every tropical cyclone event is diferent and there might be other characteristics not captured in the data, we consider the event itself as a parent node to wind speed and total rainfall. The event and municipalities are parent nodes of distance which causes the municipalities to experience diferent levels of wind speed and rainfall. For example, Table 6 shows that wind speed is indeed a significant mediator between distance and damage. Tables 4 and 3 show that the significance of rainfall as a predictor of damage changes depending on whether or not we account for the specific tropical cyclone events in the model. This is therefore a case of the Simpsons paradox. Furthermore, here we consider distance to express other hazard efects on damage and therefore it becomes a confounder variable. The hazard components may not be entirely representative of tropical cyclones since it does not also consider storm surges as a side efect of the wind speed.

For simplicity, we do not add paths between hazard nodes and build typologies nodes because this introduces a time component. That is over time communities adjust to impact tropical cyclones rebuilding damaged buildings with stronger materials. Mediation analysis however confirms that build typologies can be mediators of efect of hazard on damage levels in our data (see Table 5). Furthermore, the relationship between build typologies could be bidirectional but doing so in our model introduces cycles and structural causal models do not account for cyclic relationships [ 4 ].

5. Discussion

In this causal re-interpretation of the efect of tropical cyclones on damage levels in municipalities, based on our data, we find the problems of confounder bias (distance confounds both hazard components and the outcome damage levels). In addition, we find a case of Simpsons paradox with the direct efect of rainfall on damage. Following the correct causal path ways to establish efect of hazard on damage plays an important role in ensuring fairness. Path analysis is a precursor to carrying out further fairness tests (e.g., counterfactual analysis). In this case the weaker build typologies (e.g., light roof, light walls) are sometimes proxies for vulnerable

1Stronger building materials are more resilient to natural hazards 2We exclude rainfall measurements at 6 hours and 24 hours because of high correlation to total rainfall, see Figure 5

socio-economic groups of people and hence can be considered as sensitive variables. Because of the identified complexities (i.e., confounder bias and Simpsons paradox) further analysis is required to quantify the efect of these biases on the accuracy of damage prediction and how they would afect fair distribution of aid resources. It is however apparent that the outcome variable and the objectives also need to be designed or specified in a fair manner that ensures fairness. In this context we note that predicting impact instead of damage would allow for a deeper engagement on fairness.

6. Acknowledgments

Authors received funding from the Netherlands Organization for Scientific Research (NWOMVI) and UNICEF (grant number MVI.19.007). Authors also benefit from collaborations with 510, an Initiative of the Netherlands Red Cross.

Authors also acknowledge Anna Manchens for comments on mediation reasoning.

A. Basic DAGs

Chain graph conditional independence.

⊥ | (5)

C. Tables

Mean Absolute Error (MAE) Root Mean Square Error (RMSE)

Description Percentage of damage buildings observed in the previous typhoons Hazard - wind speed (/) Hazard - total rainfall () during the tropical cyclone Geography - Distance from the epicentre of the tropical cyclone Percentage of strong roof and strong wall build types Percentage of strong roof and salvage wall build types Percentage of strong roof and light wall build types Percentage of salvage roof and strong wall build types Percentage of salvage roof and salvage wall build types Percentage of salvage roof and light wall build type Percentage of light roof and strong wall build type Percentage of light roof and salvage wall build type

Percentage of light roof and light wall build types Estimate 0.005 0.235 0.240 0.021 Average Causal Mediation Efects(ACME) Average Direct Efects (ADE) Total Efect Prop. Mediated

References

[1] S. Boeke , M. van den Homberg, A. Teklesadik, J. Fabila , D. Riquet , M. Alimardani , Towards predicting rice loss due to typhoons in the philippines , The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 42 ( 2019 ) 63 - 70 .

[2] M. J. Van den Homberg , C. M. Gevaert , Y. Georgiadou, The changing face of accountability in humanitarianism: Using artificial intelligence for anticipatory action , Politics and Governance 8 ( 2020 ) 456 - 467 .

[3] C. M. Gevaert , M. Carman , B. Rosman , Y. Georgiadou , R. Soden , Fairness and accountability of ai in disaster risk management: Opportunities and challenges , Patterns 2 ( 2021 ).

[4] J. Pearl , Causality: Models, Reasoning , and Inference, 2nd ed., Cambridge Univeristy Press, 2009 .

[5] B. Friedman , H. Nissenbaum , Bias in computer systems , ACM Transactions on information systems (TOIS) 14 ( 1996 ) 330 - 347 .

[6] S. Xu , J. Dimasaka , D. J. Wald , H. Y. Noh , Seismic multi-hazard and impact estimation via causal inference from satellite imagery , Nature Communications 13 ( 2022 ) 7793 .

[7] H. Burton , Causal inference on observational data: Opportunities and challenges in earthquake engineering , Earthquake Spectra 39 ( 2023 ) 54 - 76 .

[8] A. Teklesadik , M. van den Homberg, Forecasting impacts of tropical cyclones with machine learning: A case study in the philippines , in: EGU General Assembly Conference Abstracts, 2022 , pp. EGU22 - 12917 .

[9] X. Wang , Y. Zhang , R. Zhu , A brief review on algorithmic fairness , Management System Engineering 1 ( 2022 ) 7 .

[10] J. R. Loftus , C. Russell , M. J. Kusner , R. Silva , Causal reasoning for algorithmic fairness , arXiv preprint arXiv: 1805 . 05859 ( 2018 ).

[11] C. Su , G. Yu , J. Wang , Z. Yan , L. Cui , A review of causality-based fairness machine learning ( 2022 ).

[12] Y.-L. Chou , C. Moreira , P. Bruza , C. Ouyang , J. Jorge , Counterfactuals and causability in explainable artificial intelligence: Theory, algorithms , and applications, Information Fusion 81 ( 2022 ) 59 - 83 .

[13] S. J. Jung , Introduction to mediation analysis and examples of its application to real-world data , Journal of Preventive Medicine and Public Health 54 ( 2021 ) 166 .

[14] U. M. K. Eidsvig , K. Kristensen , B. V. Vangelsten , Assessing the risk posed by natural hazards to infrastructures , Natural Hazards and Earth System Sciences 17 ( 2017 ) 481 - 504 .

[15] S. Chiappa , Path-specific counterfactual fairness , in: Proceedings of the AAAI conference on artificial intelligence , volume 33 , 2019 , pp. 7801 - 7808 .

P-value < 2e- 16 *** < 2e - 16 *** < 2e - 16 *** < 2e - 16 ***