Receiver operating characteristic analysis is widely used in biomedical studies for evaluating the diagnostic accuracy of continuous markers. Continuous status severity evaluation and the accurate prediction of mortality risk for the polytrauma patients is crucial for triage, quality management, assessment of mortality prediction and the scientific study of trauma. The aim of this study is to investigate the possibility of the receiver operating characteristic analysis for determination of lethal outcome predictive markers for multiple trauma patients with severe thoracic trauma during early posttraumatic period. A single-center prospective observational cohort study involved 73 male patients. Patients' examinations were performed on the 1st-2nd, 3rd-4th and 5th-6th days after trauma. A biochemical assay was conducted for estimation of biomarkers dynamics during observed posttraumatic period. Receiver operating characteristic analyses with the areas under receiver operating characteristic curves estimation was performed for the investigated biomarkers with the most significant differences between survivors and non-survivors for each of the time periods. According to Youden's index the -coufft values of investigated biomarkers with contingency table statistics were calculated as possible predictive tests for negative outcomes during the first 56 days after trauma. This study demonstrates that receiver operating characteristic analysis is a useful tool for decision-making in clinical medicine. The clinical example suggests that the same biomarkers and cut-off values cannot be equally useful for lethal outcome prediction for several days in patients with multiple trauma with severe thoracic trauma. These additional biomarkers for each of the investigated time periods can serve as criteria for the clinical course monitoring of polytraumatized patients via recognizing of those with a high risk of lethal outcome for improving the quality of patient care.
The receiver operating characteristic (ROC) analysis with an estimation of the area under the curve is the most common metric for evaluating the prediction of binary outcomes [1]. It was first used for detection of radio signals in the presence of noise following the Pearl Harbor battle [2]. After that, contributions were made by researchers in engineering, psychology, radiology and mathematics [3]. At the present time, ROC analysis is widely used in biomedical studies for evaluating the diagnostic accuracy of continuous markers [4]. Application of ROC analysis is independent on data following a normal distribution. It is not substantially affected by sample asymmetry of positive or negative cases. Still, it is fundamentally dependent on unequivocal classification of cases and controls, generally using a gold standard diagnostic test, examination or the final outcome [5]. Besides, it allows determining the best cut-off value with the highest true positive rate together with the lowest false positive rate according to Youden’s i[n1d].exAlso calculation of the area under receiver operating characteristic curve (AUROC) gives a measure of the general test usefulness, with the possibility of its comparison [2].
Healthcare has long pursued an understanding of the personal risk factors that contribute to disease onset [6, 7]. The decision from a clinical diagnostic test is mostly based on whether the marker value exceeds a cut-off value, in which case the diagnosis is “dnisdeas“endo”-ndiaseased” otherwise. It is not rare in modern medicine that classification criteria are not completely predictive, leading to incorrect classification, so it is important to compare the effectiveness of the different classification systems [5]. There is always a probability that the diagnostic test is positive for a non-diseased individual or a negative for a diseased patient [4]. Continuous status severity evaluation and the accurate prediction of mortality risk for polytrauma patients is crucial for triage, quality management, assessment of mortality prediction and the scientific study of trauma [8]. Some researchers incorporated the dependency of time in the sensitivity and specificity in disease for individuals instead of the standard ROC curve method. The application of time-dependent setting with the observation of disease status at each time point yields different values of sensitivity and specificity throughout the study. These methods have better effectiveness, but still are not fully used in medical research [4]. Accurate estimation of the mortality and morbidity risks can not only improve our understanding of the pathophysiological mechanisms involved in disease progression but also provide early warning for patients at high risk of developing complications. This knowledge could help guide clinical decisions and improve the quality of patient care through early intensive focused care.
Most predictive tools for outcome evaluation of polytrauma patients were designed only for the
first 24-hours or on the time in admission to the hospital. But on the other hand, it is well-known that
patients with multiple trauma, especially with severe thoracic trauma, change their clinical status
dramatically, so intense monitoring, especially during early posttraumatic period, is mandatory [9].
Besides, the pathophysiology of polytrauma is complex and consists of certain stages of systemic
reactions with different predominant mechanisms that are responsible for secondary insults and early
and late systemic post-injury complications [
The combination of severe thoracic trauma with other injuries of the body regions significantly complicates patient treatment [13, 14]. Management of such multiple trauma patients requires a multidisciplinary approach and involves different medical specialists: emergency physicians both out of hospital and in hospital settings, anesthesiologists, intensivists, radiologists, advanced care practitioners, surgeons, respiratory therapy personnel and others [15, 16, 17]. Variety invasive and noninvasive interventions can be effective in treating severe chest injuries in multiple trauma patients requiring multiple medical and allied health disciplines. To ensure the best quality of coordination, implementation, monitoring and evaluation of recommended care, an organized trauma care system is required with proper continuous status severity understanding [18, 19]. Its evaluation for the polytraumatized patient during the early posttraumatic period is crucial for the triage, quality management, the assessment of mortality prediction and the scientific study of trauma. 2. Aim
The aim of this study is to investigate the possibility of the ROC analysis for the determination of lethal outcome predictive markers for multiple trauma patients with severe thoracic trauma during early posttraumatic period.
This single-center prospective observational cohort study was conducted in anesthesiology and intensive care for patients with multiple trauma of Kharkiv Municipal Clinical Emergency Hospital named after prof. O.I. Meshchaninov. 3.1.
Seventy three patients with a blunt mechanism of multiple trauma with severe thoracic trauma were included in this study. The presence of two or more injured body regions, Injury Severity Score more than 16 with the severe thoracic component of multiple trauma were determined as inclusion criteria. As exclusion criteria was set the presence of concomitant chronic disease in the decompensation and compensation phase. Patients' examinations with blood samplings were performed three times: on the 1st-2nd day after trauma (10.75-33.5 hours), 3rd-4th day (48-75.2 hours) and 5th-6th day (97-122 hours). The survival/non-survival ratio was 42/31. The main demographic characteristics are shown in table 1. There were no significant differences in age, number of patients with concomitant alcohol exposure, admission time and the etiology of polytrauma between patient groups.
Biochemical assay was conducted in the biochemistry department of Kharkiv National Medical University according to spectrophotometric methods. Total protein concentration was determined according to biuret reaction in patients’ plasm[a20]. The level of the proteins’ carbonyl groups was determined with the help of dinitrophenylhydrazine reaction [21]. Enzyme-linked immunosorbent assay was used for the determination of interleukin-4 and interleukin-10. White blood cells count with leukocyte formula estimation was performed in the clinical laboratory of Kharkiv Municipal Clinical Emergency Hospital according to a conventional clinical method using Giemsa stains.
The Microsoft Excel spreadsheet was used for primary data collection. ROC analysis was performed with the help of GraphPad Prism 5.03. Youden’s idnex was used to choose an appropriate cut-off value for biochemical markers [1]. The Mann-Whitney test was used for determining differences between groups for quantitative data. Two-sided Fisher`s exact test and chi-square test for trends were performed to consider differences in nominal data of demographic characteristics. All quantitative variables are presented as median with 95 % confidence interval in round brackets. Qualitative data are presented as numbers with percentage of the patients’ populatironundin brackets. The level of statistical significance was specified as p <0.05.
The dynamics of investigated biomarkers for multiple trauma patients with severe thoracic trauma during early posttraumatic period are represented in Table 2. It can be seen that there is no normal distribution of all data presented in the table, therefore, Mann-Whitney test was used for comparing results of the patient groups.
Besides that, the dynamics of investigated biomarkers are not similar nor for each biomarker during the estimated time period, nor between patient groups. The most significant differences between survivors and non-survivors for the patients with the multiple trauma with severe thoracic trauma on 1st-2nd day after trauma were observed according to the total -gplroobtueliinn,s aγnd albumin concentrations. For the 3rd-4th day of the early posttraumatic period the most significant differences were found for th-egloαb1ulins, interleukin-10 concentrations and the stab neutrophils count in white blood cells analysis. Concentration-sgloobfuliαn1s, protein carbonyls and interleukin10 on the 5th-6th day after trauma were the most different between patients groups. 4.2.
ROC-analyses with AUROC calculation were performed for the biomarkers with the most significant differences between survivors and non-survivors for every estimated time period. ROC curve shows the relationship between sensitivity and specificity for every possible cut-off value of the estimated biomarker [5]. The AUROC is the test that is used as a criterion to measure the te discriminative ability and is interpreted as the probability that a patient who dies has a biomarker value worthier than that for a patient who survives [1]. Figure 1 represents investigated ROC curves. For the 1st-2nd day after trauma AUROC 0.9616 (0.9135 – 1.01); р<0.0001 was obtained for albumin concentration, 0.808 (0.7039 – 0.9121); р<.00001 for -γglobulins concentration and 0.828 (0.7333 – 0.9226); р<.00001 for total protein concentration.
For the 3rd-4th day after trauma AUROC 0.9607 (0.9149 – 1.006); р<.00001 was calculated for - α1 globulins concentration, 0.7785 (0.6539 – 0.9031); =р0.0002262 for stab neutrophils count and 0.8851 (0.7882 – 0.982); р<.00001 for interleukin-10 concentration. For the 5th-6th day after trauma AUROC 0.9989 (0.9951 – 1.003); р<.00001 was calculated for -gαl1obulins concentration, 0.9671 (0.9295 – 1.005); р<.00001 for the proteins carbonyls concentration and 0.8662 (0.778 – 0.9544); р<0.0001 for interleukin-4 concentration. 4.3.
Youden’s indexestimation was performed for the determination of clinically significant cut-off values with optimal predictive properties from the series of investigated biomarkers. In fact, this index maximizes the difference between sensitivity and 1-specificity across various cut-off points, so that the optimal cut-off point can be calculated [22]. The results of Youden’s indexand contingency table statistics are presented in tables 3-5.
The sensitivity of a diagnostic test is the proportion of patients for whom the test correctly classifies the positive outcome. The specificity is the proportion of patients for whom the test correctly identifies the negative outcome. Sensitivity and specificity are characteristics of a test and are not affected by the prevalence of the disease [1]. Sensitivity depends only on those who have died due to multiple trauma with severe thoracic trauma and specificity only on those who survived. From Table 3 the highest sensitivity was obtained for albumin concentration <20.6015 g/L and the highest specificity were observed for both albumin concentration <20.6015 g/L and total protein concentration <49.36 g/L. Besides that, for the last biomarker cut-off value, the specificity value is higher than sensitivity indicating better use of the total protein concentration <49.36 g/L as a screening test for multiple trauma patients with severe thoracic trauma on the 1st-2nd day of early posttraumatic period.
The positive predictive value is the fraction of patients with positive test results who actually had the lethal outcome and the negative predictive value is the fraction of patients with negative test results who actually survived. The highest positive and negative predictive values were observed for albumin concentration <20.6015 g/L. In contrast to sensitivity and specificity, positive and negative predictive values directly assess the test usefulness, but they are affected by the disease prevalence [1].
The odds ratio is the ratio of the odds of lethal outcome in the presence of an estimated positive diagnostic test and the odds of lethal outcome in the presence of negative diagnostic test. This test quantifies the strength of the association between lethal outcome and presence of positive test according to estimated cut-off values of investigated biomarkers. And close to odds ratio, the likelihood ratio is the ratio of the probability of a positive test result if the outcome is lethal to the probability of a positive test result if the patient survives. The highest odds and likelihood ratios were obtained for albumin concentration <20.6015 g/L. Also, the highest accuracy was observed for albumin concentration <20.6015 g/L, indicating that this test is the most sensitive and accurate for predicting of lethal outcome in case of multiple trauma with severe thoracic trauma on the 1st-2nd day of early posttraumatic period.
Interestingly, that ROC curves of - gthloebuγlins and total protein concentrations are crossed (Figure 1) and AUROC curve value for the -gγlobulins concentration is lower than that for total protein concentration, but Youden’s index f othre -γglobulins concentration >12.24 g/L is higher than the total protein concentration <49.36 g/L.
From Table 4 the highest sensitivity was calculated-glfoobruliαn1s concentration <2.596 g/L and the highest specificity were obtained for b-ogtlhobuαli1ns concentration <2.596 g/L and stab neutrophils count >227.6×71/0L. For all biomarkers’-ocffutvalues, the specificity values are higher than sensitivity indicating their usefulness as screening tests rather than a test for estimating the severity of multiorgan disturbances on the 3rd-4th posttraumatic day for patients with multiple trauma with severe thoracic trauma. The highest positive and negative predictive values were observed for α1-globulins concentration <2.596 g/L. The highest odds ratio, likelihood ratio and test accuracy were obtained for α1-globulins concentration <2.596 g/L too.
For the 5th-6th day of the trauma, the highest sensitivity was calculate-dglobfourlinsα1 concentration <2.719 g/L and proteins carbonyls’ concentration >15.86 µmol/g protein (Table 5). The highest specificity was obtained only for α-1globulins concentration <2.719 g/L. For all biomarkers’ cut-off values the sensitivity values are higher than specificity indicating good usefulness as tests for estimating the severity of posttraumatic complications. The highest positive and negative predictive values, the odds, likelihood ratios and test accuracy were observed-glfoobruliαn1s concentration <2.719 g/L.
Receiver operating characteristic analysis is a useful tool for decision making in clinical medicine. This is an effective way of determining the effectiveness of a diagnostic test. Better outcome prediction in case of multiple trauma with severe thoracic trauma can be estimated according to albumin concentration less than 20.5015 g/L on the 1st-2nd day, α1-globulins less than 2.596 g/L on the 3rd-4th day and α1-globulins less than 2.719 g/L on the 5th-6th day of early posttraumatic period. This clinical example suggests that the same biomarkers and their cut-off values cannot be fixed for the lethal outcome prediction for the whole early posttraumatic period in patients with multiple trauma with severe thoracic trauma, because each day after trauma has its specific predictive biomarkers with different parameters of contingency table statistics and test accuracy. These additional biomarkers can serve as criteria for the clinical course monitoring of polytraumatized patients via recognizing those with a high risk of lethal outcomes for improving the quality of patient care. Also, these statistics cannot become the substitution for clinical thinking, but provide a systematic approach to dealing with medical decision-making tools in clinical practice and operationalization in research through providing support for choice of cut-off values to optimize the classification process. 6. References