Arteriovenous fistula (AVF) is the conjunction of an artery and a venous, surgically created as point of access to blood stream in dialysis patients. The measure of blood flow in AVF (Qa) is an important indicator to assess the functioning of AVF and the consequent adequacy for effective dialysis treatment. The measurement of Qa requires time-consuming and costly procedures which are generally performed only for a subset of patients considered at risk of problems. Our hypothesis is that there is a set of information collected during dialysis treatment that can be exploited to estimate Qa. The objective of this study is twofold: analyze the accuracy of Qa estimation from routinely collected clinical and dialysis ma-chine data variables and compare different approaches for this problem. We tack-led this estimation as an ordinal classification problem. As in machine learning domain there are different approaches to include order information in multiclass classification, we explored pros and cons of such different methods.
Chronic kidney disease is an increasing long-term condition characterized by a deterioration of kidneys functioning. After a certain level of renal damage, the patient needs renal replacement therapy (RRT) to survive. There are two kinds of RRT: hemodialysis and peritoneal dialysis. The former (and most common) consists in re-moving fluid and waste products from the blood using a special filter called dialyzer. To take the blood from the patient and filter through the dialyzer and return it to the patient, the doctor needs to make access to the patient’s blood vessel, called vascular access. There are several types of vascular access, arteriovenous fistula (AVF) is one associated with lower incidence of complication and longer patient’s survival. It is a conjunction of an artery and a venous, surgically created, to exploit the blood pressure of the artery and guarantee an adequate blood flow through the dialyzer.
Continuous monitoring of AVF functioning is an important aspect of dialysis patient management
[
The assessment of AVF blood flow (Qa) is an important prognostic test for evaluating the status of
vascular access [
On the other hand, most of the dialysis machines automatically collect, at each dialysis treatment, a set of data related to treatment blood flow and needles pressure which can be exploited to estimate the AVF blood flow. Furthermore, the information present in the patient health records, such as age, laboratory tests, comorbidities, and others, might influence the AVF blood flow. In this study we focused on the prediction of the AVF flow exploiting all this routinely collected data.
The value of AVF blood flow is generally categorized in clinical meaningful levels: < 525 ml/h (very low level), 525 ml/h - 925 ml/h (low level), > 925 ml/h (normal level).
Considering this categorization, AVF flow estimate can be seen as an ordinal classification problem, as there is a clear inherent order among the classes but not a well-defined numerical distance.
This kind of multiclassification can be tackled with different machine learning approaches. We have
selected two kinds of ordinal classification methods: the first transforms ordinal targets into binary
classification subtasks [
In this study we also compare the ordinal classification methods with a model based on a standard regression methos.
We already explored the application of machine learning approaches for the monitoring of AVF
status in [
Ordinal multi-classification problem consists in a task of predicting labels characterized by an ordinal scale. An ordinal classifier C maps each object [ ] ∈ into an ordered set : ⟶ , where = { 1 ≺ ⋯ ≺ }The labels are characterized by the fact that it is possible to define an order among them but not a measure of distance.
Although, in Machine Learning there are many algorithms for general multi-classification problems, there are much less able to consider the order information
There are mainly two classes of algorithms for this kind of problem. The first [
The estimation of the first and the last class depends on a single classifier, ( < 525) is given by the first classifier and ( ≥ 925) is given by the last classifier. The probability of the middle class is computed as ( ≥ 525) ⋅ (1 − ( ≥ 925))
This approach suffers from the possible inconsistency of the predicted probabilities [
( ≥ 925) > (525 < < 925) and ( ≥ 925) < ( ( ≥ 525)).
The second approach, called rank-consistent ordinal regression framework (CORN), is based on a
neural network with an output layer that implements a set of − 1 binary learning tasks associated
with the ranks [
In this study we applied the method described in [
Given a dataset = { [ ], [ ]} =1, a rank [ ] is transformed into − 1 binary labels 1[ ], … , [−] 1 such that [ ] ∈ {0,1} and indicates if [ ] is greater than . To ensure classifiers consistency, the model estimates a series of conditional probabilities such that the output of the -th binary task ( [ ]) represents the conditional probability
( [ ]) = ( [ ] > | [ ] > −1), where { [ ] > } ⊆ { [ ] > −1}
The unconditional probabilities can be derived through the chain rule for probabilities to the model outputs: ( [ ] > ) = ∏ ( [ ]) Since ∀ 0 ≤ ( [ ]) ≤ 1, we have
( [ ] > 1) ≥ ( [ ] > 2) ≥ ⋯ ≥ ( [ ] > −1) This guarantees the rank consistency among the − 1 binary tasks.
To evaluate the benefits of using ordinal classifier approaches than a classical regression approach, we used as benchmark an XGBoost regressor to estimate the exact value of Qa and, after that, we map these values to the three classes (very low flow, low flow, normal flow) using the thresholds 525 ml/h and 925 ml/h. We call this model REGM.
The accuracy comparison among the three methods was assessed computing mean absolute error (MAE), precision and recall, F1-score and confusion matrix 2.2.
Our dataset was composed of 46,292 Qa measurements referred to 5,940 hemodialysis patients from four different European countries (Czech Republic, Portugal, Slovakia, Spain) and collected between 2015 and 2022. The Qa distribution was shown in Figure 1.
Written informed consent for statistical analysis was obtained from all the patients. Qa measurements are mainly obtained through dilution technique (thermal 93.34% and other 6.27%) and only a few measures (0.39%) were obtained through doppler ultrasound.
The input dataset was composed of 49 parameters that characterize patient, AVF and treatments routinely recorded through Fresenius Medical Care clinical database (EuClID®). The input variables were computed collecting the information in the 90 days before the Qa measure. 70% of the dataset was randomly selected to train each model, 10% was used as validation set (for CORN) and the remaining 20% was used as test set.
In Figures 2, 3, and 4 we reported the confusion matrix with the percentage of labels correctly and incorrectly classified for the three classes for: REGM, BINCM and CORN respectively. The overall MAE was 0.37, 0.27 and 0.36.
The results show that both ordinal classification models performed better than the benchmark in terms of overall MAE, and BINCM outperformed CORN (0.27 Vs. 0.36). It is interesting to note that when we looked at the best recall for single class, we found that the best one for “very low” and “low” flow is still BINCM but for normal flow the best one is REGM. On the other hand, the best one in terms of precision for “very low” flow is REGM and for “low” and “normal” flow it is BINCM.
In this exploratory paper we studied the estimation of AVF blood flow exploiting routinely collected clinical data extracted from a clinical database. AVF blood flow assessment is critically important to assess the status of the fistula and evaluate preemptive intervention to avoid AVF failure, that may lead to the risk of complications.
As AVF blood flow is generally categorized in three medical meaningful classes (very low when less than 525 ml/h, low when between 525 and 925 ml/h and normal when greater than 925 ml/h), we tackled the problem as an ordinal classification task.
There are several approaches in machine learning domain for ordinal classification, we focused on two techniques: ordinal classification with K-1 binary classifiers and CORN. We used a popular regression model (XGBoost) with post classification as benchmark.
The objective of this study can be summarized in two points: first, evaluate the accuracy of AVF blood estimation to assess its possible medical use and second, compare different Machine Learning approaches for this problem.
Regarding the first point, the preliminary results are very promising. The ability to detect fistula with very low flow with a precision of 0.9 (with BINCM) is a remarkable result per se. It should be also noted that 7.3 % of the missed “very low flow” AVFs are predicted as “low flow” and only 2.7% are predicted as normal flow. This suggests that the model could be used to build an alert system to warn physicians about AVFs in critical situations. It might substantially improve the AVF surveillance process, referring the patient for further investigation when it is really needed and potentially decrease the risk of AVF failure.
Regarding the comparison among different approaches, the results are interesting as well. If we focus on the overall MAE and F1-score, it seems that the possible inconsistency of probabilities classes might be an acceptable tradeoff for our problem as those metrics are much better for the BINCM than CORN (0.27 and 0.76 Vs. 0.36 and 0.69). Although CORN results less accurate than BINCM, it can be a useful tool for the application where it is important to show to the physician the single probability of the three classes. In this case, the inconsistency of the probabilities would be difficult to understand.
It is also important to consider that if we focus on the performance of the models in the single classes, the best model is not always the same. This suggests that those approaches have different peculiarities that can be exploited to create a more robust and effective medical support decision system.
Considering the promising preliminary results, we see different next steps for this study, for example: a deep feature analysis to explore the contribution of each variable to the prediction; evaluating further ordinal classification methods; discuss with clinicians for a possible use case of the model in real clinic setting and how integrate it in the clinical system and in clinical practice.