Novel continual learning techniques on noisy label datasets

Abstract

Many Machine Learning and Deep Learning algorithms are widely used with remarkable success in scenarios whose benchmark datasets consist of reliable data. However, they often struggle to handle realistic scenarios, particularly those in the financial sector, where available data constantly vary, increase daily, and may contain noise. As a result, we present an overview of the ongoing research at the AImageLab research laboratory of the University of Modena and Reggio Emilia, in collaboration with AxyonAI, focused on exploring Continual Learning methods in the presence of noisy data, with a special focus on noisy labels. To the best of our knowledge, this is a problem that has received limited attention from the scientific community thus far.

Keywords

• deep learning
• finance
• noisy label
• noise
• classification
• continual learning

1. Introduction

The use of artificial neural networks, enabled by the availability of large datasets, has yielded remarkable results sequently, finance. Recently, an increasing trend has emerged to enhance investment performance via the use of Deep Learning methods to forecast market price fluctuations. This provides asset managers with valuable els, as well as analyzing portfolio performance under uncertain circumstances. However, these applications are often afected by the high complexity and volatility of the financial market, which makes them susceptible to noisy data. One major reason is the unpredictable and often irrational behaviour of market participants that can cause sudden and unexpected changes(see the latest event at SVB [ 1 ]). Furthermore, financial data is often subject to measurement errors, data gaps, class overlap, and contexts where labels are ambiguous or subjective.

Additionally, other external factors such as geopolitical events and economic policies can significantly contribute to adding noise to the data. Therefore, the prediction problems in this domain involve datasets with several

nEvelop-O

0000-0001-7090-7159 (M. Millunzi); 0000-0002-9717-5602 (L. Bonicelli); 0000-0003-2656-2696 (A. Salman); 0000-0003-2656-2696 (J. Credi); 0000-0001-9056-1538 (S. Calderara) challenges: (i) uncertain labels, (ii) imbalanced classes, and (iii) data that change significantly over time, leading to the well-known problem of concept drift [ 2 ]. While DNNs represent a strong tool for simple classification appropriately in standard deep learning scenarios for two main reasons: data comes continually from nonstationary environments and may change with high frequency, causing the data drifting phenomenon. This to learn from the most recent data. Indeed, as new data arrive as a stream, the network naturally adapts to the last-seen ones, making it prone to forget the previously acquired knowledge. This aspect is usually not considered, assuming that old financial trends do not afect future ones. Conversely, market trends are often cyclical; hence, making financial models retain the knowledge of past data while learning from the most recent ones may make them more reliable in already-experienced market conditions. By using Continual Learning techniques, models can adapt to changing financial data and improve their predictive power over time. Therefore, the application of CL to finance has the potential to enhance investment performance by providing more accurate and reliable predictions in the face of noisy data.

The research under discussion endeavours to explore complex CL scenarios that involve training data with incorrect and noisy labels.

2. Related

Works

There is a growing body of work on the application of machine learning techniques to financial problems. The

ifnancial machine learning techniques in [ 3, 4 ]. In this

© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License interested reader can find a comprehensive overview of work, however, we are specifically interested in the prob- ing without Forgetting [20] (LwF) employs Knowledge lem of training supervised models with noisy financial Distillation (KD) to distill the model learned during past labels in a continual learning scenario. Both problems tasks into the current one. Parameter-isolation methhave been studied independently, see e.g. [ 5 ] for the for- ods explicitly define a separate sub-network per task to mer and [ 6, 7 ] for the latter. avoid interference between the parameters during learn

Let us now delve more deeply into the two problems ing. Notably, in Progressive Neural Networks [21] (PNN) separately to show an overview of the state-of-the-art the learner is expanded at each subsequent task, while in contribution in both fields, which will then allow us to Context-dependent Gating [22] (XdG) only sparse, mostly deal with the current existing works that address both non-overlapping patterns of units are active for any task. continual learning ad noisy labels simultaneously. While these methods usually achieve high performance, they usually rely on the knowledge of the task identity 2.1. Noisy Labels during inference, thus limiting their applicability to real scenarios. Finally, Reharsal methods [23, 24] are based Learning with noisy labels (LNL) is a common and long- on the idea of interleaving data for the current task with standing challenge in many machine learning applica- a bufer of data from the previous ones, a strategy comtions, where the labels assigned to the data points may monly referred to as Experience Replay (ER). iCARL [25] not be entirely accurate. Consequently, the model perfor- combines replay with KD and uses the herding strategy mances tend to degrade rapidly with respect to classical to select the most representative samples for each class. scenarios since DNNs can easily overfit noisy labels and DER [26] and X-DER [27] use the past responses of the show poor generalization performances. In this study, model as a means for knowledge distillation. we focus mostly on the supervised classification learning task. In [ 8 ] the authors present a comprehensive 2.3. Continual with Noisy Labels overview of the existing approaches proposed to enhance the robustness of models against noisy labels. We can Bridging the gap between CL and LNL, a preliminary distinguish two main types of methods: those that at- work [28] proposes a self-supervised loss term to learn a tempt to learn the noise transition matrix and those that representation that is independent of label noise, while prioritize cleaning the data stream before using it to train distilling a small, purified bufer of samples for later use the model. The former approach focuses on building a ro- in fine-tuning and classification. Although this method bust architecture that is capable to estimate a noise model achieves good initial performance when compared with that captures the label distribution pattern [ 9, 10 ]. In- common CL baselines, its learning objective features a stead, the latter approach addresses the issue of keeping low sample eficiency which limits its real-world applitraining data as clean as possible via various “sample- cability. To address this limitation, [29] exploits the loss selection” techniques that try to identify clean examples diference between correct and noisy samples to split based on the small-loss hypothesis [ 11, 12, 13 ] and then the incoming data, using the latter as unlabeled data in apply semi-supervised learning to re-label the leftover a FixMatch [30] objective. Finally, in [31] the authors and hence train on the whole training set. Notably, some seek a balance between the purity and the efectiveness methods [ 14, 15, 16 ] feature a two-component Gaussian of samples stored in the bufer. To achieve this, they Mixture Model to model and split the loss distribution of promote the removal of samples from the bufer based correctly-labeled and incorrectly-labeled examples. on a combination of the sample loss with a measure of diversity in the bufer.

2.2. Continual Learning

The field of Continual Learning ( CL) seeks to learn from

a non-stationary non-i.i.d. stream of data without incurring the forgetting problem. Methods that tackle CL are commonly categorized into three main groups [17]. Regularization-based methods introduce regularization terms to prevent significant changes in the performance of the model on past tasks. Elastic Weight Consolidation (EWC) [18] and Synaptic Intelligence (SI) [19] seek to prevent parameters deemed as important for the current task from changing in the future. In EWC, the importance of each parameter is estimated as the Fisher information matrix while SI preserves connections that strongly afected the past task loss. Diferently, Learn

3. Datasets with noisy labels

Researchers have developed several datasets specifically

designed to address the challenge of LNL. Although these consist of images and therefore difer from the typical ifnancial datasets, we are interested in understanding model response to noise, regardless of the data itself.

Here, we briefly describe the dataset commonly used in LNL literature: • Clothing1M: this large-scale dataset contains 1 million images of clothing items from diferent online stores. The labels for the images were be the input and ground-truth label space respectively. the latter, since in this controlled scenario, we know obtained through a web search and thus may con- identically distributed according to a certain data gentain incorrect labels. •

WebVision: this dataset contains over 2.4 mil- comes from a noisy distribution =̃ {( , ̃ )} =1 , with ̃ lion images from 1000 diferent categories collected from the web. The dataset has been used for various tasks such as image classification, object detection, and fine-grained recognition under noisy label scenarios. • CIFAR-10-N and CIFAR-100-N [32]: these are

Amazon Mechanical Turk1. variations of the popular CIFAR-10 and CIFAR100 datasets [33] respectively, with humanannotated real-world noisy labels collected from

Overall, these datasets provide valuable resources for developing and evaluating machine learning models under noisy label scenarios, which are common in many real-world applications.

In addition to these, another popular choice consists in artificially applying noise on the annotations of a clean dataset by flipping the label with a fixed probability. Such a synthetic setting is particularly helpful since the availability of ground-truth labels, coupled with the noisy ones, facilitates monitoring models’ behaviour with respect to diferent noise patterns.

4. Problem formulation

For this proof preliminary analysis, we focus on a -class

image classification

problem, which we split in -fold classification tasks. Let ∈ ℛ × and ∈ {1, … , } {( , In a standard CL setting, each task receives data = )}=1 , where each pair ( , ) is independently and 1https://www.mturk.com erating distribution. Instead, in the LNL scenario, data being the noisy label space. Here, we assume that the corruption process that produces ̃ from is independent of the input data; hence, one true label may be flipped into another label with equal probability (noise rate). We will refer to this process as “symmetric noise”.

5. Experiments

For the sake of simplicity and a fair comparison with other existing works [28, 29], we will conduct experiments on CIFAR-10. This consists of 60000 32×32 images, usually split into 50000 images for the training set and 10000 for the test set. The classification task involves 10 non-overlapping classes and the number of examples per class is uniform. We modify the dataset by adding symmetric noise in each of the classes. To be compliant with Class-IL scenario [17], we split the 10-fold classification into 5 binary tasks and let the model learn from each one sequentially. We denote the version of the dataset obtained by such two modifications as Split-N-CIFAR-10.

As emerges from [28, 29, 31], keeping the memory bufer as clean as possible during training is a key aspect of dealing with the label noise issue in a continual learning scenario. Therefore, we decided to test the GMM technique used in [ 15, 14 ] to separate exemplars into noisy and clean against a strategy we call “oracle”. For which samples are associated with a noisy label, we can use this information to prevent all the noisy ones from being stored inside the replay bufer. Both techniques are tested using ER-ACE as a base strategy for CL. This method extends the common Experience Replay baseline by adding an asymmetrical cross-entropy loss between stream and bufer examples.

We used the ResNet18 [35] model initialized from scratch and trained with Stochastic Gradient Descent (SGD) for 50 epochs per task, in a standard ofline fashion. Additionally, to allow comparison against [28, 29] we evaluate the online (single epoch) scenario.

5.1. Results

In Table 1 we report performances in terms of Final Av

erage Accuracy (FAA) at the end of all tasks. Results are averaged across five runs and the bufer size is set to 500. We provide a lower and an upper bound, respectively finetuning without any countermeasure to forgetting and “Multitask”, given by training all tasks jointly. Expectedly, the model is strongly afected by the presence of noise, and indeed its performances decrease as the noise rate increases. When in presence of low noise levels (20%), the standard ER-ACE model and its variants with bufer cleaning techniques perform on par, whereas for noise levels above 40%, the model seems to benefit from replaying a clean bufer. Surprisingly, in some scenarios the adoption of the GMM makes the model outperform the oracle for the same noise conditions, suggesting that having some noisy examples stored in the bufer may reduce generalization error by providing some extra regularization. Finally, while the GMM-based bufer cleaning provides an initial benefit, by looking at results from the online scenario it is clear that using semi-supervised learning on mislabeled examples (as CNLL does) provides a significant boost in terms of performance.

6. Conclusion

In this manuscript, we have investigated the challenging

problem of learning from non-stationary and noisy data, which is highly relevant to the domain of financial data analysis. Through a joint efort between the AImageLab research laboratory at the University of Modena and Reggio Emilia and AxyonAI, we have conducted a preliminary analysis to assess the efectiveness of rehearsalbased continual learning baselines equipped with common label noise learning strategies. Our findings shed light on the complex interaction between these challenging fields and provide a proof of concept for future research in this area. Identifying mislabeled data using the area under the lini, E. D. Cubuk, A. Kurakin, H. Zhang, C. Rafel, margin ranking, Advances in Neural Information Fixmatch: Simplifying semi-supervised learning Processing Systems 33 (2020) 17044–17056. with consistency and confidence, arXiv preprint [17] G. M. van de Ven, A. S. Tolias, Three continual learn- arXiv:2001.07685 (2020).

ing scenarios, in: Neural Information Processing [31] J. Bang, H. Koh, S. Park, H. Song, J.-W. Ha, J. Choi, Systems Workshops, 2018. Online continual learning on a contaminated data [18] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, stream with blurry task boundaries, in: Proceedings G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ra- of the IEEE/CVF Conference on Computer Vision malho, A. Grabska-Barwinska, et al., Overcoming and Pattern Recognition, 2022, pp. 9275–9284. catastrophic forgetting in neural networks, Pro- [32] J. Wei, Z. Zhu, H. Cheng, T. Liu, G. Niu, Y. Liu, ceedings of the national academy of sciences 114 Learning with noisy labels revisited: A study us(2017) 3521–3526. ing real-world human annotations, arXiv preprint [19] F. Zenke, B. Poole, S. Ganguli, Continual learning arXiv:2110.12088 (2021).

through synaptic intelligence, in: International [33] A. Krizhevsky, et al., Learning multiple layers of feaConference on Machine Learning, 2017. tures from tiny images, Technical Report, Citeseer, [20] Z. Li, D. Hoiem, Learning without forgetting, IEEE 2009.

Transactions on Pattern Analysis and Machine In- [34] L. Caccia, R. Aljundi, N. Asadi, T. Tuytelaars, telligence (2017). J. Pineau, E. Belilovsky, New insights on reducing [21] A. A. Rusu, N. C. Rabinowitz, G. Desjardins, abrupt representation change in online continual H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, learning, arXiv preprint arXiv:2203.03798 (2022). R. Hadsell, Progressive neural networks, arXiv [35] K. He, X. Zhang, S. Ren, J. Sun, Deep residual preprint arXiv:1606.04671 (2016). learning for image recognition. arxiv 2015, arXiv [22] N. Y. Masse, G. D. Grant, D. J. Freedman, Alleviating preprint arXiv:1512.03385 14 (2015). catastrophic forgetting using context-dependent gating and synaptic stabilization, Proceedings of the National Academy of Sciences 115 (2018)

E10467–E10475. [23] R. Ratclif, Connectionist models of recognition memory: constraints imposed by learning and forgetting functions., Psychological Review (1990). [24] A. Robins, Catastrophic forgetting, rehearsal and

pseudorehearsal, Connection Science (1995). [25] S.-A. Rebufi, A. Kolesnikov, G. Sperl, C. H. Lampert, iCaRL: Incremental classifier and representation learning, in: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2017. [26] P. Buzzega, M. Boschini, A. Porrello, D. Abati,

S. Calderara, Dark Experience for General Continual Learning: a Strong, Simple Baseline, in: Advances in Neural Information Processing Systems, 2020. [27] M. Boschini, L. Bonicelli, P. Buzzega, A. Porrello,

S. Calderara, Class-incremental continual learning into the extended der-verse, IEEE Transactions on

Pattern Analysis and Machine Intelligence (2022). [28] C. D. Kim, J. Jeong, S. Moon, G. Kim, Continual learning on noisy data streams via self-purified replay, in: Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 537–547. [29] N. Karim, U. Khalid, A. Esmaeili, N. Rahnavard, Cnll:

A semi-supervised approach for continual noisy label learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 3878–3888. [30] K. Sohn, D. Berthelot, C.-L. Li, Z. Zhang, N. Car

References

[1] M. T. Fennell , Svb financial group: Santa clara, california, Fed. Res. Bull . 107 ( 2021 ) 49 .

[2] A. Tsymbal , The problem of concept drift: definitions and related work , Computer Science Department, Trinity College Dublin 106 ( 2004 ) 58 .

[3] M. L. De Prado , Advances in financial machine learning , John Wiley & Sons, 2018 .

[4] M. F. Dixon , I. Halperin , P. Bilokon , Machine learning in Finance, volume 1170 , Springer, 2020 .

[5] Y.-H. Lien , Y.-S. Lin , Y.-S. Wang , Uncertainty awareness for predicting noisy stock price movements , in: Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2022 , Grenoble, France, September 19-23 , 2022 , Proceedings, Part VI , Springer, 2023 , pp. 154 - 169 .

[6] D. Philps , T. Weyde , A. d. Garcez, R. Batchelor , Continual learning augmented investment decisions , arXiv preprint arXiv: 1812 . 02340 ( 2018 ).

[7] A. Zurli , A. Bertugli , J. Credi , Does catastrophic forgetting negatively afect financial predictions? , in: Machine Learning, Optimization, and Data Science: 8th International Workshop , LOD 2022, Certosa di Pontignano , Italy, September 19-22 , 2022 , Revised Selected Papers , Part I , Springer, 2023 , pp. 501 - 515 .

[8] H. Song , M. Kim , D. Park , Y. Shin , J.-G. Lee , Learning from noisy labels with deep neural networks: A survey , IEEE Transactions on Neural Networks and Learning Systems ( 2022 ).

[9] X. Chen , A. Gupta , Webly supervised learning of convolutional networks , 2015 . a r X i v : 1 5 0 5 . 0 1 5 5 4 .

[10] T. Xiao , T. Xia , Y. Yang , C. Huang , X. Wang , Learning from massive noisy labeled data for image classification , in: Proceedings of the IEEE conference on computer vision and pattern recognition , 2015 , pp. 2691 - 2699 .

[11] L. Jiang , Z. Zhou , T. Leung , L.-J. Li , L. Fei-Fei , Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels , in: International Conference on Machine Learning , 2018 .

[12] B. Han , Q. Yao , X. Yu , G. Niu , M. Xu , W. Hu , I. Tsang , M. Sugiyama , Co-teaching: Robust training of deep neural networks with extremely noisy labels , Advances in neural information processing systems 31 ( 2018 ).

[13] H. Song , M. Kim , J.-G. Lee , Selfie: Refurbishing unclean samples for robust deep learning , in: International Conference on Machine Learning, PMLR , 2019 , pp. 5907 - 5915 .

[14] E. Arazo , D. Ortego , P. Albert , N. O 'Connor , K. McGuinness , Unsupervised label noise modeling and loss correction , in: International conference on machine learning, PMLR , 2019 , pp. 312 - 321 .

[15] J. Li , R. Socher , S. C. Hoi , Dividemix Learning with noisy labels as semi-supervised learning , in: International Conference on Learning Representations , 2020 .

[16] G. Pleiss , T. Zhang , E. Elenberg, K. Q. Weinberger ,