Introduction

Interactive adaptive learning refers to methods that help improve the entire lifecycle of machine learning models. This includes how the models interact with human experts or other systems and how they adapt to different types of emerging data rather than just training them on a fixed dataset. This allows the models to improve and adapt over time, which is critical for many real-world applications. Active learning is the most prominent field of interactive adaptive learning [1,2,3]. Therefore, we explore different aspects of active learning in this tutorial and then discuss recent advances in the field in a workshop session.

This tutorial is divided into three main parts, which are described in detail in the following sections. Their titles and presenters are as follows:

1. Introduction to Uncertainty-Based Active Learning (A. Tharwat), 2. Hands-on Pool-based Active Learning via scikit-activeml (M. Herde), 3. Towards Pool-based Active Learning with Error-prone Annotators (M. Herde).

Part I -Introduction to Uncertainty-Based Active Learning

The motivation behind active learning stems from the common scenario where large amounts of unlabeled data are readily available and free, but the process of obtaining labeled data is time-consuming and expensive. In many real-world applications, such as medical image analysis, document classification, or sensor network monitoring, the cost of manual labeling or expert annotation can be prohibitively high. Traditional passive learning approaches, where the model is trained on a fixed dataset, may not be the most efficient use of limited labeling resources [4]. Active learning offers a solution to this challenge by enabling the model to actively select the most informative and/or representative samples from the unlabeled pool for labeling. By focusing the annotation effort on the most valuable data points, active learning can achieve better model performance with fewer labeled samples compared to passive learning [3]. There are two main strategies for querying points in active learning as follows:

• Exploration-focused query strategies: This category aims to identify representative samples that can help the model explore the underlying data distribution more effectively. By querying for diverse and representative data points, the model can gain a better understanding of the IAL@ECML-PKDD'24: 8 th Intl. Worksh. & Tutorial on Interactive Adaptive Learning, Sep. 9 th , 2024, Vilnius, Lithuania † These authors contributed equally.

marek.herde@uni-kassel.de (M. Herde); tuan.pham@uni-kassel.de (M. T. Pham); alaa.othman@hsbi.de (A. Tharwat); bsick@uni-kassel.de (B. Sick) true structure of the problem domain, even with a limited labeled dataset [5]. Some common exploration-focused query strategies include (i) Diversity-based sampling: selecting dissimilar samples to explore different regions, (ii) Density-based sampling: prioritizing samples in dense areas, as they are more representative, (iii) Clustering-based sampling: identifying samples near cluster centroids to ensure coverage of data subgroups [3].

• Informativeness (or exploitation)-focused strategies: The type in which active learning employs informativeness-focused query strategies with the goal of identifying the most informative samples, i.e., the data points that, if labeled, would provide the maximum information gain to the model [6]. Some common exploitation-focused approaches include (i) Margin-based sampling: prioritizing samples near the decision boundary [1], (ii) Entropy-based sampling: selecting the most uncertain samples [7], and (iii) Variance-based sampling: prioritizing samples with high prediction variance [7].

However, by exploiting the model's own uncertainty about the unlabeled data, active learning can more effectively identify the samples that, if labeled, would provide the maximum information gain to improve the model's performance. The key idea behind uncertainty-based active learning is that the model's uncertainty serves as a proxy for the informativeness of a data point. Samples with higher uncertainty are more likely to be informative because they represent areas of the input space where the model's predictions are less confident or reliable [8,9]. Common uncertainty-based approaches include

• Uncertainty sampling: Selecting the most uncertain samples, as they are likely to be the most informative [7]. • Expected information gain: Choosing samples with the highest expected information gain [7].

• Bayesian optimization: Using a Bayesian model to quantify uncertainty and guide sample selection [10].

Recent advances in uncertainty quantification have further extended the capabilities of active learning. The distinction between (i) epistemic uncertainty, which occurs due to lack of knowledge, and samples with high epistemic uncertainty are often the most informative for improving model understanding, and (ii) aleatory uncertainty, which captures the inherent noise or randomness in the data, and samples with high aleatory uncertainty may be less useful for model training [11,12,13].

By distinguishing between epistemic and aleatory uncertainty, active learning can more effectively identify the most informative samples to improve model performance with less labeled data. The integration of advanced uncertainty quantification with active learning strategies creates a powerful framework for efficient and effective model training, even with large amounts of unlabeled data [9].

Part II -Hands-on Pool-based Active Learning via scikit-activeml

Active learning is a versatile approach to reducing the labeling cost. Assumptions regarding data availability and training of the classifiers can vary greatly depending on the use case. Active learning libraries often abstract parts of active learning experiments, such as the whole experiment (AliPy [14] and Baal [15]), the classifier training (modAL [16] and small-text [17]), and data management (libact [18]). These abstractions help simplify active learning experiments where the use case matches the library's scope. However, it requires considerable work if the assumptions differ. The scikit-activeml [19] library has been conceptualized with this in mind, with its modular design inspired by and built on top of scikit-learn [20], a flexible general-purpose machine learning library.

The goal of scikit-activeml is to bridge active learning research and its application in real-world use cases. The library is flexible enough for researchers to allow for many assumptions and promotes reproducibility. For practitioners, it provides extensive documentation, many tutorials for different use cases, and examples for each query strategy with animations visualizing the strategies' behavior that can be used as a starting ground.

scikit-activeml provides query strategies for pool-based active learning in classification and regression with single or multiple annotators with varying batch sizes. Stream-based active learning [21] for classification is also supported for single annotator scenarios. In this tutorial, we focus on pool-based active learning. Algorithm 1 shows a small pool-based active learning script using scikit-activeml. Labeled and unlabeled data are stored together in X (samples) and y (labels), where unlabeled data is marked with a user-specified MISSING_LABEL constant (cf. lines 9-13). The classifier and query strategy are initialized independently and support using random seeds to ensure reproducibility (cf. lines [16][17][18][19][20][21]. The for-loop shows the active learning cycle, where sample indices are queried (cf. line 26), and their corresponding missing label is replaced with the ground truth (cf. line 27). Figure 1 shows the fitted classifier, labeled, and unlabeled data after 10 and 30 cycles. Additionally, areas where it is beneficial to query more labels, according to uncertainty sampling, are highlighted in dark green.

Starting from such a basic learning cycle with uncertainty sampling as the employed query strategy, this tutorial's part outlines other popular and state-of-the-art query strategies for pool-based active learning, e.g., core set [22], batch active learning by diverse gradient embeddings (BADGE) [23], typical clustering (TypiClust) [24], probability coverage (ProbCover) [25], clustering uncertainty-weighted embeddings (CLUE) [26], and contrastive active learning (CAL) [27]. Specifically, we analyze these query strategies regarding informativeness, representativeness, and batch diversity as central concepts in pool-based active learning (cf. Section 2). Further, we introduce the differentiation between low-and high-budget active learning scenarios. Depending on the scenario, the importance of the aforementioned concepts changes. Throughout this tutorial's part, illustrations of synthetic two-dimensional datasets (cf. Fig. 1) provide a more intuitive understanding of the query strategies' main ideas and sample selection behaviors. Beyond such toy examples, we also present an empirical evaluation study as a potential application for scikt-activeml, where we compare query strategies' performances across tabular, image, and text data. In doing so, we leverage feature representations (embeddings) learned by the pre-trained model self-distillation with no labels (DINOv2) [28] for the image and bidirectional encoder representations from transformers (BERT) [29] for the text data. This tutorial's part concludes with a hands-on session, where participants can access a Jupyter notebook to apply their newly acquired knowledge by implementing their own active learning experiment via scikit-activeml.

Part III -Towards Pool-based Active Learning with Error-prone Annotators

In pool-based active learning, a common assumption is that the queried class labels originate from a single omniscient annotator [30]. However, many annotation campaigns involve querying class labels from multiple humans, e.g., crowdworkers, who are prone to error for various reasons, e.g., lack of expertise, tiredness, or missing motivation [31]. As a result, the queried class labels are subject to noise.

Training with such noisy class labels can strongly deteriorate the classifier's performance. With a focus on neural networks, numerous techniques have been proposed to improve the robustness against noisy class labels [32]. A common approach is the joint training of the classifier and an annotator performance model, which corrects the noisy class labels by modeling each annotator's individual performance [33]. Depending on the assumptions about the annotators' noise patterns, confusion matrices are estimated per annotator [34] or even for each sample-annotator pair [35], for example. Such techniques are typically employed to train a neural network after completing an annotation campaign. Yet, the annotators' performance estimates could be used to guide the annotator selection during an ongoing annotation campaign. In conjunction with intelligent sample selection, we refer to this scenario as pool-based active learning with multiple error-prone annotators. Corresponding query strategies [36,37] must also balance the added exploration-exploitation trade-off when assigning annotators to provide class labels for given instances. The goal of this tutorial's part is to give a basic understanding of such challenges and outline potential baselines, which leverage common pool-based query strategies for the sample selection and performance estimates for the annotator selection.

project "Künstliche Intelligenz zur Fremdkörperdetektion in befüllten Getränkeflaschen (KI4FKD)" (493 22_0022_2B). This project is receiving funding from the program "Distr@l", an initiative of the Ministry for Digitalization and Innovation of the State of Hesse. The sole responsibility for the content of this publication lies with the authors.