Decision making in many areas is based on data about individual behavior often measured using different surveys. The study investigates the proposed approach for behavior modelling on the base of Bayesian belief networks that allows predicting behavior characteristics using small and incomplete data from surveys about behavior episodes. We explored the characteristics of the models using the automatically generated dataset that included 44350 cases. During our experiment, we considered three different model structures and compared three different discretization strategies. We found that simpler structures showed better prediction quality for all measures (average accuracy, precision, recall, F1 score). The observed difference was statistically significant but did not exceed 1% that can be considered unimportant if error price is low. Our findings suggested that ways of data transformation, particularly discretization strategies for input data, had a significant impact on prediction quality: background knowledge about distributions, theoretical assumptions about behavior led to higher prediction quality.
Decision making in many areas is based on data about individual behavior: personnel behavior [
The idea of analysis of such kind of data is based on Bayesian Belief Network (BBN) theory that
allows complementing empirical data with inputs from other models and expert knowledge [
Bayesian Belief Network is a type of probabilistic graphical models that represents a set of random
variables and their conditional dependencies [
Previous studies showed high prediction quality of BBN models for estimating risky behavior
characteristics [
The input data for the model [
Adding data about minimum and maximum intervals decreases the influence of recent behavior
represented by the last episodes. However, combining all the data about episodes leads to very
complicated joint distribution [
On the contrary, Bayesian belief networks allow determining complex relationships in terms of
simpler dependencies between small parts. Modelling risky behaviour as BBN gives a way to add all
available data into the model as well as include expert assumptions about relationships between them
and their distributions. Moreover, the existed software tools for BBNs representation, visualization,
structure and parameter learning, inference and analysis, for example [
The structure of BBN model is a directed acyclic graph G(V , L) with vertices V le1, le2 , le3, tmin , tmax , rate, n and edges (or links) L u, v : u, v V , where rate is random variable for behavior rate; lei is random variable for the length of the interval between (i-1)-th and i-th episodes from the end (0 corresponds to interview moment); tmin and tmax (min and max in figures) are random variables for the length of minimum and maximum intervals; n is random variable for the number of episodes during period of interest.
Previous studies [
Further studies [
The aim of this study is to explore the next step: if the number of episodes can be directly estimated on the base of the rate and vice versa when T is given, let’s exclude one of these variables and explore a new, even more simplified model (figure 3). Thus, we plan to estimate the influence of latent variable n .
Since collecting the real data about behavior episodes is extremely time-consuming and requires
financial and organizational resources, we explored the characteristics of the models using
automatically generated dataset. The generation process assumes that the behaviour follows Poisson
random process: the occurrence of the next episode is independent from the previous ones, length of
interval between concurrent episodes follows exponential distribution. This assumption corresponds to
the features of real-life risky behavior and it is widely used in previous studies [
On the first step we generated 1500 values for behavior rate from Gamma distribution (shape k 1.2, scale 0.3 ). The choice of parameters’ value was aimed to get dataset that corresponded to “reallife” risky behavior: behavior rate in most cases (in 94% of cases in our dataset) was less than 1 and generated about one episode in 3–4 days.
Next step included generation of 30 “respondents” (sequences of behavior episodes) for each rate value for period of 365 days in total, that summed up to 45000 sequences of episodes. Then we calculated initial data for the model: lengths of minimum, maximum intervals between episodes and lengths of intervals between the last three episodes. After deletion of incomplete cases (e.g. cases with only one episode during 365 days) the final dataset included 44350 cases (or “respondents”).
To estimate prediction quality of the models we randomly select 5000 cases to the test dataset and do not use them in model learning.
As mentioned in Section 2 we conducted the series of experiments to explore the influence of the latent variable n (the number of behavior episodes during period of interest T ) on the prediction quality of the model. Hence, we considered three different model structures (Figure 1–3). Since we explored discrete Bayesian networks, the discretization strategy can influence the prediction quality measures. So, in this study, we compared three different discretization strategies. The detailed description of each strategy is provided below. We used the same number of intervals for all strategies for further comparison.
The first discretization is based on breaks provided by experts. The general idea is to make smaller intervals for more frequent values. In addition, this discretisation uses more interpretable breaks. For example, rate 0.5,1 means that there was one behavior episode in 1–2 days. Moreover, interpretability becomes more important for variables corresponded to interval lengths: le1 7,14 means that the last behaviour episode was 1–2 weeks ago. The usage of weeks / months / year notation can simplify the questionnaire design for real-world applications.
The second discretization strategy is quite simple: we divided possible range into intervals with equal lengths and then added the last interval with infinity as the upper limit. We included background knowledge about behavior characteristics in this strategy too: the infinite interval started at 1 for rate variable (since we assumed that most of the cases had rate less than 1) and it started at 180 for timelength variables (since we assumed at least two behavior episodes).
The last strategy uses the idea of equal interval probabilities not lengths and is based on quantile calculation. At the experiment we used sample quantiles but for real-world problems when the rate distribution is unknown it is possible to use theoretical quantiles based on distribution assumptions.
The example of the variable discretization according to the strategies is shown on figure 4.
First, we applied the described discretization strategies to the train dataset that consisted of 39350
cases in total. On each iteration we randomly selected 10000 cases from our train dataset. Then we
learned the parameters of Bayesian belief network for all three structure models described in
Section 2: 1) the initial model, 2) the simplified model with fewer links and 3) the last model without
n variable. Thus, we had nine different settings at each iteration (all possible combinations of three
model structures and three discretization strategies). Finally, we used test dataset with corresponding
discretization to estimate prediction quality (accuracy, precision, recall, F1 score). All measures were
calculated according to multi-class classification metrics (average accuracy, macro precision, macro
recall, macro F1 score) [
The calculations and statistical analysis were performed using R [
The average accuracy through 50 iterations is shown on figure 5. The discretization with equal probabilities presented the highest results for all model structures, while the discrete intervals with equal length were the worst approach.
The F1 score presented the same pattern: the highest measures for discretization with equal probabilities, the lowest for the one with equal interval lengths (figure 6). F1 score was even more skewed: it was about 0.6 for equal probability strategy, about 0.54 for expert-defined breaks and 0.16– 0.22 only for equal length strategy. The latter was explained with further analysis of precision and recall measures. The discretization with equal lengths had a poor prediction quality for all classes except the first two of them. Due to extremely skewed distribution (see figure 4), neither model, regardless of its structure, predicted rates higher than 0.3, so predictions were all at the first two classes.
Mean quality measures and the standard deviations (SD) for all model structures and discretization strategies are summarized in table 1. Since the discretization with equal lengths showed the similar results for the initial structure and the structure without n variable and slightly better but still poor results for the simplified model (structure on figure 2) we focused on other discretization strategies.
To compare all quality measures among the proposed models we run pairwise t-test for multiple comparisons with Bonferroni correction. The simplified model had statistically significant higher prediction quality measures comparing to both initial model and model without n variable in case of expert-defined discrete intervals (table 1 (mean values) and table 2 (p-values)). There was no statistically significant difference in quality measures between initial model and model without latent n variable.
In case of discretization with equal probabilities of intervals, both the model with simplified structure and the model with structure without n variable outperformed the initial model. The simplified model also had significantly higher precision comparing to the model without n variable with the same levels of other measures.
Risky behavior modelling, behavior parameter estimating and prediction became the focus of
studies in many research areas. An increased attention is given to investigation of online
behavior and prediction of social media user’s characteristics [
The current study explores the proposed behavior model on the base of Bayesian belief networks that allows processing incomplete data about behavior episodes and predicting behavior characteristics.
Our findings suggested that ways of data transformation, particularly discretization strategies for input data, had a significant impact on prediction quality: background knowledge about distributions, theoretical assumptions about behavior led to higher prediction quality.
We found that for the same period of interest the simpler structures showed better prediction quality but it is important to mention that the quality difference did not exceed 1% (for example, 89.7% vs 89.9% for average 8-class accuracy for equal probability discretization). The effect of this difference can be estimated in practical settings only: sometimes the error price is high; in other circumstances it is diminishing.
In general, proposed model showed high prediction quality and has a great potential for analyzing real-life behavior problems.
This work was partially supported by the by RFBR according to the research project No. 16-31-60063 and No. 18-01-00626