Anomaly detection is essential in various domains, including healthcare, where early identification of irregular patterns in data can significantly impact patient outcomes. This paper presents a novel approach to unsupervised anomaly detection using the ECG5000 dataset, focusing on electrocardiogram (ECG) data. We introduce multiple autoencoder architectures-linear, convolutional, and LSTM-based-reframing the traditionally supervised classification task as an unsupervised anomaly detection problem. By disregarding original labels, we emphasize the models' ability to generalize across diferent ECG abnormalities. Our extensive experiments reveal that a denoising linear autoencoder outperforms more complex architectures, achieving an accuracy of 97.73%, within 0.7% of the current state-of-the-art. Furthermore, we conduct a comprehensive analysis of the latent space representations, providing insights into the models' feature extraction capabilities. These findings suggest that our approach not only reduces model complexity but also maintains high accuracy, ofering a viable solution for real-time anomaly detection in medical settings.
k-NN) have shown improved eficacy but tend to be
computationally intensive and less scalable for large
Anomaly detection is a crucial task in data science and datasets[
Recent advancements in anomaly detection within explore a variety of autoencoder architectures—linear, healthcare have progressed from traditional statisti- convolutional, and LSTM-based—to evaluate their efeccal methods, such as Z-Score and Interquartile Range tiveness in identifying anomalies without the guidance of (IQR)[18], to more sophisticated machine learning and labels[40]. The research focuses on two main variations deep learning approaches[19, 20], capable of handling of autoencoders: denoising and contractive. Our objeccomplex, high-dimensional data like ECG signals[21, tive is to identify an architecture that balances model 22, 23, 24]. Traditional methods often struggle with complexity with performance, making it suitable for realsuch intricate temporal patterns, while density-based[25] time medical applications where computational resources (e.g., LOF, DBSCAN) and distance-based techniques (e.g., may be limited.
Through rigorous experimentation, we demonstrate that a denoising linear autoencoder achieves near stateof-the-art performance with significantly reduced complexity. Additionally, we perform an in-depth analysis of els, ofering insights into how these architectures capture essential features for anomaly detection in ECG data.
In this section, we outline the methodologies employed to explore the efectiveness of various autoencoder architectures for unsupervised anomaly detection in ECG data. The methodology encompasses the design and testing of multiple autoencoder variants, the evaluation of their performance, and the analysis of the latent space representations. • Denoising Autoencoders: Here, Gaussian noise is added to the input data during training. The model is trained to reconstruct the clean input from the noisy version, enhancing its ability to iflter out noise and focus on the underlying signal structure. • Mixed Models: Combining both contractive and denoising strategies, these models aim to leverage the strengths of both approaches, though they require careful tuning of hyperparameters to avoid over-regularization.
We investigated three diferent autoencoder architectures: Linear, Convolutional, and LSTM-based autoencoders[41]. These architectures were selected based on their ability to capture diferent characteristics of the ECG data—spatial hierarchies in the case of convolutional layers and temporal dependencies for LSTM layers.
Each of the aforementioned architectures was further developed into diferent variants to assess their robustness and efectiveness: • Contractive Autoencoders: These models introduce a regularization term based on the Frobenius norm of the Jacobian matrix, which encourages the model to learn stable latent representations that are less sensitive to small input perturbations.
A key aspect of our methodology was the analysis of the latent space produced by the autoencoders. Principal Component Analysis (PCA) was applied to reduce the dimensionality of the latent space and visualize the separation between normal and abnormal samples. Additionally, a simple logistic regression discriminator was trained on the latent representations to assess their ability to distinguish between normal and anomalous data.
The efectiveness of the latent space was quantified by measuring the accuracy of the discriminator, with higher accuracy indicating a more distinct and informative latent representation. This analysis was crucial in understanding the models’ capacity to encode meaningful features in the latent space, which directly impacts the anomaly detection performance.
Figures 6 and 7 present the latent space and decision boundary for the contractive linear model, which also demonstrated lower efectiveness in distinguishing between normal and abnormal samples compared to the denoising linear model.
Further analysis was conducted on the mixed linear model (Figures 8 and 9), and the denoising convolutional model (Figures 10 and 11). These results highlighted the strengths and limitations of combining contractive and denoising approaches in the same architecture.
optimize the hyperparameters for each model variant. The final training was conducted over 15 epochs, with a learning rate of 0.01 using the Adam optimizer, which proved more stable and faster than stochastic gradient descent (SGD). The denoising linear autoencoder achieved the best performance, with training early-stopped at 11 epochs based on validation performance.
The optimal hyperparameters included a contractive lambda of 0.0001 and a noise level of 0.05 (representing the maximum absolute value for the Gaussian noise added to the input). Notably, the contractive variants took approximately 10 to 15 times longer to train than their denoising counterparts, without yielding superior
to evaluate the performance of the various autoencoder architectures on the ECG5000 dataset. This includes a description of the dataset, data preprocessing steps, model training configurations, and evaluation metrics.
The ECG5000 dataset is a well-known benchmark for anomaly detection tasks involving electrocardiogram (ECG) signals. The dataset consists of 5000 onedimensional ECG recordings, each containing 140 time steps. The dataset is divided into five classes, with one representing normal heartbeats and the remaining four representing various types of anomalies, including: • Class 1: Normal beats. • Class 2: Premature ventricular contractions. • Class 3: Fusion beats. • Class 4: Unclassifiable beats.
• Class 5: Anomalous beats.
the original labels are disregarded during training. Only samples from the normal class (Class 1) are used to train the autoencoders, with the goal of identifying anomalies based on reconstruction error during testing.
the shape of the signal rather than its absolute amplitude, 3.3. Model Training Configuration which is crucial for the generalization of the anomaly detection task. The models were trained using the Adam optimizer with
Given the nature of the dataset, no further data aug- a learning rate of 0.01. The training process was conmentation techniques were applied, as the goal was to ducted over 15 epochs, with early stopping applied if evaluate the autoencoders’ performance on raw, unal- the validation loss did not improve for three consecutive tered ECG signals. The dataset was split into training, epochs. The batch size was set to 64, which provided a validation, and test sets with the following proportions: balance between computational eficiency and gradient estimation accuracy. • Training Set: 60% of the normal samples. • Validation Set: 20% of the normal samples. • Test Set: 20% of the normal samples, along with
all abnormal samples.
The validation set was used for hyperparameter tuning and early stopping, while the test set was used for final performance evaluation. • Contractive Autoencoders: A contractive loss term with a regularization coeficient ( ) of 0.0001 was added to the standard reconstruction loss.
This encouraged the model to learn more stable representations. • Denoising Autoencoders: Gaussian noise with a standard deviation of 0.05 was added to the input during training. The model was trained to reconstruct the original, clean ECG signal from the noisy input. Table 1 • Mixed Models: Both contractive loss and denois- Accuracy of the Various Autoencoder Models ing noise were applied. These models required careful tuning of the regularization coeficient and noise level to avoid over-regularization.
Linear Convolutional
LSTM
Accuracy (%) Denoising Contractive 97.73 95.69 94.94 94.51 93.60
tection was a critical aspect of the experimental setup. The threshold was determined by analyzing the MSE distribution for normal and anomalous samples. The final threshold was set as the midpoint between one standard deviation above the mean MSE of the normal samples and one standard deviation below the mean MSE of the anomalous samples.
This approach ensured that the threshold was not overly conservative, allowing the models to generalize better to unseen data, especially in scenarios where the separation between normal and anomalous samples was subtle.
computing cluster equipped with NVIDIA GPUs. The use of GPU acceleration significantly reduced training times, particularly for the more complex models like the LSTM-based autoencoder and the contractive autoencoders, which require extensive matrix computations.
diferent autoencoder models evaluated on the ECG5000 dataset. The results are organized to highlight the efectiveness of each model in detecting anomalies based on the various metrics discussed in the experimental setup.
each model. The denoising linear autoencoder demonstrated consistent and stable training, with the lowest overall loss and a clear separation between the reconstruction errors of normal and anomalous samples. (b) MSE score
The latent space analysis revealed significant diferences in the representation quality of each model. The denoising linear autoencoder produced a well-separated latent space, allowing for clear discrimination between normal and anomalous samples. This is reflected in the high accuracy of the discriminator applied to the latent space, as shown in Table 2.
The performance of the LSTM-based model was notably lower, as evidenced by the near-random discriminator accuracy and the poor separation in the latent space (Figures 20 and 21). This suggests that the LSTM model struggled to capture relevant features in the latent space for efective anomaly detection.
on the ECG5000 dataset. It achieved the highest accuracy, the most distinct latent space separation, and outperformed the contractive and LSTM-based models. The success of the denoising approach highlights the importance of handling noise in ECG signals and suggests that simpler models with well-tuned noise management can outperform more complex architectures in this context.
The performance results indicate that the denoising linear autoencoder outperformed other models in terms of accuracy and robustness in anomaly detection. This suggests that for the task of unsupervised anomaly detection on ECG data, simpler models with well-managed noise handling capabilities can provide superior performance compared to more complex architectures such as LSTM or convolutional models.
The denoising approach efectively enhances the autoencoder’s ability to learn meaningful representations by forcing the network to reconstruct clean data from noisy inputs. This technique seems to be particularly well-suited for ECG data, where noise is prevalent due to various sources of interference during signal acquisition. The success of the denoising linear autoencoder demonstrates that the simplicity of the architecture, combined with an efective noise reduction strategy, can lead to robust performance in anomaly detection tasks.
The comparison between contractive and denoising autoencoders highlights distinct diferences in how these architectures manage latent space representations. While contractive autoencoders aim to enforce robustness by minimizing the sensitivity of the latent space to small perturbations in the input, they tend to be more computationally expensive and, in this study, did not outperform the denoising models. This suggests that, at least in the context of ECG anomaly detection, denoising strategies are more efective in maintaining the balance between reconstruction accuracy and computational eficiency.
One notable observation is the performance of the contractive convolutional autoencoder, which sufered from latent space collapse—where all inputs were mapped to nearly identical latent representations. This outcome highlights the potential risks of over-regularization in contractive models, particularly when the regularization strength is not carefully tuned.
the internal workings of the diferent autoencoder models. The use of Principal Component Analysis (PCA) and a simple discriminator allowed us to visualize and quantify the quality of the latent space representations.
The denoising linear autoencoder produced a wellseparated latent space, as evidenced by the clear clusters corresponding to normal and anomalous data. This welldefined separation is crucial for efective anomaly detection, as it allows for the reliable identification of outliers based on the reconstruction error. The high discriminator detection; future work could explore semi-supervised or accuracy (93.10%) further confirms the efectiveness of fully supervised approaches to leverage labeled data for the latent space representation in distinguishing between enhanced performance. normal and anomalous samples. Furthermore, the contractive autoencoders showed po
In contrast, the latent space produced by the denoising tential issues with over-regularization, which warrants LSTM and contractive convolutional models exhibited further investigation. Future work could explore adapsignificant overlap between normal and anomalous data, tive regularization techniques or alternative forms of resulting in poor discriminator performance. The near- regularization to address these challenges. Similarly, the random accuracy of the discriminator (around 50%) in exploration of more sophisticated noise models for the dethese cases indicates that the latent space failed to cap- noising autoencoder could provide insights into further ture the essential features needed for efective anomaly improving robustness and accuracy. detection. This finding suggests that while LSTM-based Finally, expanding the latent space analysis to include models are well-suited for capturing temporal dependen- other dimensionality reduction techniques or incorpocies, they may struggle with unsupervised tasks where rating more advanced discriminative models could prothe latent space must be highly informative for anomaly vide deeper insights into the structure and utility of the detection. learned representations.
The issues observed with the contractive convolutional model, including the latent space collapse, underscore the importance of balancing regularization strength to avoid 6. Conclusion over-constraining the model. When the contractive loss term is too strong, the model may prioritize minimizing the latent space sensitivity to the extent that it disregards the actual data structure, leading to poor performance.
autoencoders, particularly those with simple linear architectures, are highly efective for unsupervised anomaly detection in ECG signals. The ability of these models to generate robust latent representations, coupled with 5.4. Practical Implications their computational eficiency, makes them strong canThe findings from this study have several practical impli- didates for deployment in clinical settings where rapid cations. First, the success of the denoising linear autoen- and accurate detection of cardiac anomalies is crucial. coder suggests that for ECG anomaly detection, model The findings also underscore the importance of careful simplicity combined with efective noise management selection and tuning of regularization techniques, as inapcan yield strong performance. This insight is particularly propriate combinations can hinder rather than enhance relevant for deployment in resource-constrained envi- model performance. Future research should continue to ronments, where computational eficiency is paramount. explore these themes, with a focus on generalizability,
Second, the latent space analysis highlights the impor- interpretability, and computational eficiency. tance of selecting appropriate architectures and regularization strategies to ensure that the latent space remains 7. Declaration on Generative AI informative and well-structured. The trade-ofs between model complexity, regularization strength, and perfor- During the preparation of this work, the authors used mance must be carefully considered when designing au- ChatGPT, Grammarly in order to: Grammar and spelling toencoders for anomaly detection. check, Paraphrase and reword. After using this tool/ser
Finally, the limitations observed with LSTM-based vice, the authors reviewed and edited the content as models in this study suggest that alternative strategies, needed and take full responsibility for the publication’s such as attention mechanisms or hybrid models, may be content. needed to efectively capture temporal dependencies in unsupervised anomaly detection tasks. Future research could explore these alternatives to improve the perfor- References mance of sequential models in this context.
itations should be noted. The models were tested on a single dataset (ECG5000), and the findings may not generalize to other types of ECG data or diferent domains. Additionally, the study focused on unsupervised anomaly