Emergency Department Chief Complaints have been used to detect the size and the spread of disease outbreaks in the past. Chief complaints are readily available in digital formats and provide a good data source for syndromic surveillance. This paper reports our ndings on the identi cation of the distribution of a few syndromes over time using the Victorian Syndromic Surveillance (SynSurv) data set. We utilized a machine learning-based Nave Bayes classi er to predict the syndromic group of unseen chief complaints. Then, we analyzed the patterns of the distributions of three syndromes in the SynSurv data, speci cally the Flu-like Illness, Acute Respiratory, and Diarrhoea syndromes, over sliding windows of time using the EARS C1, C2, and C3 aberrancy detection algorithms. The results of our analyses demonstrate that applying aberrancy algorithms over the variance data between two consecutive weeks reduces the large number of possible disease outbreaks detected using raw frequencies of the syndromic groups in the same time period, resulting in a more feasible approach for practical syndromic surveillance.
The risk of bio-terrorist attacks and pathogenic diseases such as SARS have resulted in an increased need for ongoing syndromic surveillance of life-threatening diseases. During the past few decades, Natural Language Processing techniques have been widely used in biosurveillance for detecting disease outbreaks from healthrelated data sets [2]. Chief complaints and triage notes at an Emergency Department provide a good data source for the detection of such disease outbreaks. A chief complaint is a short summary of symptoms experienced by the patient at the time of arrival at an Emergency Department. The con rmation of an infectious disease or pathogen-borne illness usually requires extensive laboratory testing and may take days. This is a time consuming process which may result in the late-identi cation of a signi cant disease outbreak; however, with constant monitoring of chief complaints the usual delays in the process can be signi cantly reduced.
Syndromic surveillance has a focus on following the size, spread, and tempo of outbreaks with the aim of
monitoring disease trends and providing reassurance that an outbreak has not occurred [11]. If chief complaints
can be classi ed into syndromic groups, then abnormally high visit rates with a speci c syndrome can easily
send an early signal of a disease outbreak in real-time. After the anthrax letter threat in 2001, a range of
projects were initiated to classify chief complaints into syndromic categories and detect disease outbreaks using
supervised machine learning methods [
In this work, we aim to nd disease outbreaks through the analysis of chief complaint texts in the emergency department. We approach this task using a combination of supervised machine learning methods and statistical aberrancy detection algorithms over shifting windows of time. We analyse the performance of the approach using the Victorian Syndromic Surveillance (SynSurv) data set, and consider the e ectiveness of the algorithm to retrospectively identify outbreaks. We compare the raw frequencies over consecutive time periods with the use of the di erences between total frequencies of the positive classi ed syndromes in detecting aberrancies. 3 3.1
The SynSurv syndromic surveillance data was collected from the Emergency Departments of two of the main hospitals in the city of Melbourne, Australia, the Royal Melbourne Hospital and the Alfred Hospital. The data was collected on behalf of the Victorian Department of Health, initially to enable monitoring during the 2006 Commonwealth Games held in Melbourne. The data covers the period July 2005 to August 2009. The syndromes that we focused on in the data set were Flu-like Illness, Diarrhoea and Acute Respiratory. The SynSurv data set contained a total number of 314,630 chief complaints already labeled with one of the syndromic groups. In the SynSurv data set, a chief complaint can be a part of more than one syndromic group at a time. For example, a chief complaint can belong to the Flu-like Illness category as well as Acute Respiratory at the same time since these two diseases are very similar. Table 1 summarizes the distribution of the di erent syndromic groups in the SynSurv data set. The data was split into training and testing portions by using 2/3 of the total data set for each syndrome for training and the remainder for testing.
Syndromic group
Flu-like Illness
Acute Respiratory
Diarrhoea
Other
Total:
#Training records
11,398
7,431
5,066
185,965
209,860
#Testing records
5,829
3,877
2,601
92,462
104,769
To analyze the distribution of syndromes over a period of time, the rst step is to classify the chief complaints
reported in that period of time into speci c syndromic groups. This requires either an o -the-shelf classi er
to be applied on the data or a machine learning-based classi er to be trained with some pre-labelled chief
complaints. In a previous work [
In this study, a new classi er was trained on the SynSurv data set for chief complaint classi cation. Once the
chief complaints were pre-processed, the Nave Bayes classi er in MALLET [
As shown in Table 2, the results of the new classi er in this study show an improvement over those reported
on the same data set in [
Syndromic group
Flu-like Illness
Diarrhoea
Acute Respiratory
Statistical aberrancy detection algorithms have been a vital method for syndromic surveillance [4]. These
algorithms can detect large sudden deviations of occurrences of speci c events that signi cantly depart from
the norm over time. We applied Early Aberration Reporting System's C1, C2, and C3 aberrancy detection
algorithms [
The formulas to calculate the EARS C1, C2, and C3 algorithms can be found in [4, 10]. Brie y, they are based on a calculation of the occurrence of speci c diseases, relative to an expected value for that occurrence. The C1 algorithm requires a 7-day baseline data starting from t 7 to t 1, where t = present day, to calculate the mean and standard deviation over a sample. It then calculates how much the value at day t varies from the expected value. If the variance exceeds a pre-set threshold, then an aberrancy in the data is detected. The C2 algorithm adds a 2-day lag to the baseline, starting from t 9 to t 7, while C3 uses the current and previous two values of C2 to detect possible aberrancies. The thresholds for C1 and C2 were set to [(sample mean + (3 sample standard deviation)]; C1(t) > 3 and C2(t) > 3; while for the C3 Algorithm, any C3(t) > 2 would signal an aberrancy at t.
The C2 algorithm is known to perform better on serially correlated data [
The trained classi cation system (MALLET's Nave Bayes) was used to predict the syndromic group of the chief complaints in the test SynSurv data set. Then, the positive classi ed cases for each syndrome and the dates they occurred on were tracked. Once we knew how many cases of Flu-like Illness, Diarrhoea, and Acute Respiratory syndromic groups occurred per day in the SynSurv test data set, we calculated the di erences between the frequencies of cases for each syndromic group over shifting windows of time. The time windows were seven days long, and each subsequent window was shifted by one day. Therefore, the rst window of time was from day 1 to day 7, the second from day 2 to day 8, and so on. This procedure formed the data that we refer to as the predicted data set. A similar procedure was utilized on the same SynSurv test data set, but using the actual gold standard labels of the chief complaints rather than text-based predictions to derive syndrome frequencies; we refer to this as the actual data set. Finally, the aberrancy detection algorithms C1, C2, and C3 were applied to both Predicted and Actual data sets to nd any outbreaks of the three syndromic groups in the SynSurv data set.
Since C1, C2, and C3 algorithms do not require prior training, their application introduces a hybrid approach combining unsupervised statistical methods with supervised classi cation techniques. This hybrid method will enable the analysis of large volumes of data collected at emergency departments and will draw health practitioners' attention to any statistical aberrancies in the data that could indicate signi cant outbreaks.
The aberrancy detection algorithms were also applied to raw frequencies of each syndromic group per window of time, in addition to the di erences between the consecutive time windows. Therefore, we discuss two types of methods here: i) the raw frequency method, that focuses on the raw frequencies of positive cases of each syndromic groups over a period of time, and ii) the variance method, that considers the di erences between the frequencies in consecutive time windows for positive cases of each syndrome. We treated the raw frequency methodology as the baseline method for comparison with new variance method. 4.2
Before applying the aberrancy detection algorithms using the raw and variance methods, we wanted to understand how the distribution of the positive cases of each syndromic group compare using both the classi cation system and the gold standard labels. The positive instances were counted and the summary result is shown in Figure 1. As shown in this gure, during the syndromic class prediction process, the chief complaint classi er produced a number of false positives as indicated by the larger numbers of the positive instances compared with those of the actual cases for all of the three syndromic groups. However, the trends in the predicted cases followed those of the actual cases, i.e., the distribution of false positives is uniform.
Since we are using aberrancy detection algorithms, a uniform distribution of false positives will not negatively a ect the detection process. The aberrancy detection algorithms consider the mean and the standard deviation of time windows and set a threshold accordingly before indicating an abnormal deviation in the number of positive cases of the syndromes. The false positives are therefore adjusted by the sample means and sample standard deviations. In other words, the aberrancy detection algorithms can play a pivotal role in situations where the underlying data are noisy and the classi cation system produces a reasonable amount of false positives.
Another advantage of using the EARS aberrancy detection algorithms is that only a 7-day baseline is required.
This helps to quickly identify any health-related outbreaks within a short period of time without the need for
a longer-term data background. Moreover, due to varying seasonal trends and varying numbers of patients
visiting medical centres over weekends and weekdays, a more systematic way of detecting disease outbreaks is
required. Aberrancy detection algorithms provide inherent measures to control such false alarm rates and the
di erent algorithms are categorized according to their sensitivity in nding abnormal deviations; C1 having
mild, C2 medium, and C3 ultra sensitivity[
We applied the aberrancy detection algorithms in the way discussed in the last section to the SynSurv data set. C3 detected approximately 400 aberrancies for each syndrome using the raw data. Analysing each of these 400 alerts is an intensive process for a health practitioner. Although this large number of aberrancies were cut to nearly half for the variance data, the number of aberrancies is still reasonably high. Therefore, we used the C2 algorithm as it has medium sensitivity, resulting in more manageable numbers of aberrancies. Figure 2 depicts the results for the C2 algorithm on the predicted data only for space limitations.
From Figure 2, it can be seen that there are a large number of peak lines above the threshold value 3 on the left, while the curves on the right have a smaller number of peaks crossing the threshold value detecting possible outbreaks. Therefore, nding the di erences in the number of diseases between two consecutive weeks, i.e., the variance method, results in a less noisy output and is a more feasible method for nding possible outbreaks compared with using raw frequencies between consecutive time periods.
It is di cult to assess whether the detected aberrancies represent real disease outbreaks. To address this, we compared the onset dates detected by the C2 algorithm with the predicted data (the variance method) with the Australian National In uenza Surveillance Scheme reports, which we consider to be the best available source of information pertaining to in uenza outbreaks.
In 2005, our system predicted aberrancies in the SynSurv data in early July, which falls in the u season in
Australia (i.e., from June to August each year). \In uenza infections are seasonal in temperate climates (June
to September in the Southern Hemisphere and December to April in the Northern Hemisphere)" [9]. Another
detected date was in the week ending October 2, 2005. \The 2006 Australian in uenza season was mild in
comparison to previous years and was predominantly due to in uenza A infections" as reported in the In uenza
Annual Report 2007 [
It should be noted that the Australian Annual In uenza reports make use of various surveillance methods including reports from emergency departments, general practitioners, and laboratory con rmed cases all over the country. The SynSurv data, however, includes Victorian emergency department data only which may not be a comprehensive representation of the national data. We interpret our results to be reliable as long as the detected aberrancies fall within the u season of Australia. No o cial reports exist for the Acute Respiratory and Diarrhoea groups and therefore we cannot directly assess performance of the method for these diseases. We employed a new technique to perform retrospective syndromic surveillance of three speci c syndromic groups, i.e., Flu-like Illness, Acute Respiratory, and Diarrhoea over the Victorian Syndromic Surveillance (SynSurv) data set consisting of a large number of emergency department chief complaints. The process started with the analysis of the unstructured text of the chief complaints and the classi cation of these into the three syndromic groups using supervised machine learning methods. Then, aberrancy detection algorithms were utilized on both the raw frequencies of positive disease cases and the variances between the number of positive cases of each syndromic group over shifting windows of time (each window comprising 7 days). While the machine learning classi er produced a number of false positives for each syndromic group, the aberrancy detection procedure was insensitive to those (uniformly distributed) false positives, due to its consideration of the mean and standard deviation of frequency di erences over time. More importantly, we found that the detection of possible disease outbreaks using our new variance method, which considers the di erences between disease frequencies as the inputs to the aberrancy detection process, will result in a more e ective outbreak detection compared with the standard method that uses the raw frequencies of the positive cases.
Our study has limitations based on the e ectiveness of the SynSurv data set for outbreak detection. The SynSurv data set only contains data from Emergency Departments of hospitals. Real-time syndromic surveillance may require incorporation of other sources of data such as those from medical practitioners for more e ective surveillance.
In future work, we are planning to further investigate the textual features of chief complaints using more
in-depth natural language processing techniques to nd other (complementary) methods for identifying disease
outbreaks from chief complaints. Similar to our related work on Ebola [