Introduction

Artificial Intelligence (AI) is a field in considerable and continuous expansion that is part of our lives and has spread to many sectors, like economy [1], industry [2], sports [3], medicine [4,5] and healthcare [6]. Focusing on these two latter fields, AI provides a valid support for helping doctors and other professionals to make diagnosis [7] and predictions [8], explain and analyze medical data [9,10]. Moreover, the use of assistive robots in rehabilitation and elderly monitoring is widespread nowadays [11,12]. For each adulterant, the list of colors identifying normality and abnormality of the test is provided. Image taken from [13].

In particular, AI is a unique tool for analyzing huge amounts of data effectively and on time [14]. Human evaluation, constrained by various factors such as subjectivity, limited computational capacity, past and personal experiences, fatigue, stress, and data quality (such as image resolution and/or lighting conditions), may be prone to generate inaccurate predictions and/or faults. Especially in medicine and healthcare, the error minimization is paramount, since it might affect the diagnosis of potential diseases, prompt interventions, therapies for rehabilitation and other aspects. A largely applied approach in the medical data analysis and classification relies on the use of Convolutional Neural Networks (CNNs) [15][16][17]. CNNs enable the analysis of broad datasets containing thousands of data faster than human operators. Notwithstanding the abundance of the examples, a major concern is the lack of explainability of some models proposed in the literature, posing a challenge that even involves the developers themselves. This turns out to be critical in the fields of medicine and healthcare, where the use of explainable AI approaches is pivotal [18][19][20].

In this work, we analyzed the issue of detecting the presence of substances/drugs in rapid lateral-flow tests (Fig. 1) [21]. Similar works investigating this topic are those reported in [22][23][24]. In particular, in [23] the authors propose an image processing algorithm combined with a Least Squares Support Vector Machine (LS-SVM) to investigate pH indicator paper assays. Their approach achieve excellent performances in terms of accuracy.

The analysis of the test results is generally made by human operators. As we stated above, the interpretation of the test is affected by factors like the subjectivity of the person and/or her/his physical and mental conditions, with consequent possible errors. Instead, we propose a novel Computer Vision (CV) approach based on the use of a deep CNN. Specifically, we employed the model introduced in [25,26] with the addition of pooling layers [27] in the convolutional part of the network. The use of pooling allows us to reduce the complexity of the problem without losing accuracy. Indeed, pooling helps the model to become invariant to small translations of the input [27]. The model must distinguish between normal and abnormal results in lateral-flow tests analyzed for the detection of drug abuse. The primary goal of the model is to verify the suitability of the sample to ensure it has not been compromised in any way. Once the sample's suitability is confirmed, the analysis can proceed to investigate the presence of narcotic substances. The cartridge undergoes a color change upon contact with the human biological sample (urine). Based on the detected color gradation, it can be concluded whether the sample has been adulterated or not. A test is considered as "abnormal" if any of the adulterants is not compliant with the corresponding guide (Fig. 2).

While the detection of strips in rapid check tests with Deep Learning techniques has already been addressed in the literature [22,[28][29][30], to our knowledge the use of a CNN model to verify that the biological sample is indeed urine and has not been tampered with in the adulterant section of lateral-flow tests has not been investigated. Collected results indicate the validity of our approach: the proposed model manages to discriminate between normal and abnormal tests. Moreover, we discuss the reasons of why such model is effective, thus providing evidence of its explainability, which represents a paramount property to successfully apply the methodology in real-world scenarios. The main contributions of our work can be summarized as follows:

• we propose a novel Deep Learning (DL) approach to address the issues related to the visual inspection of lateral-flow tests, which are generally examined by human operators, with a focus on the adulterant section of the assay; • we apply the recently proposed ConvNet3_4 model [25,26] to discriminate between normal and abnormal tests; • we achieve an excellent classification capability proving the validity of the model; • we compare the model with two state-of-the-art deep networks and we demonstrate the superiority of our approach; • we provide a thorough analysis of the relevant features extracted by the model in order to associate the proper output class to each image.

The remainder of the article is structured as follows: section 2 contains a description of the methodology we applied with respect to the considered problem. Results of our experiments are provided in section 3. Finally, our conclusions and final remarks are reported in section 4. The adulterants taken into account are: pH, OX and GL. We filled the image with a black box in order to minimize the impact of meaningless pixels on the model's prediction.

Materials and methods

As stated in section 1, our study focuses on assessing the suitability of samples through the analysis of lateral-flow tests for the detection of substance abuse (Fig. 1). All images used were obtained from a specialized medical devices company and were labeled by professional laboratory technicians. An example of images obtained with this setup is shown in Fig. 3.

Our analysis focuses on the suitability of the sample by examining the portion of the test image containing the six different adulterants (see Fig. 4 left, highlighted area in the blue rectangle): Specific Gravity (SG), pH, Oxidant (OX), Creatinine (CRE), Nitrite (NI), and Glutaraldehyde (GL). Due to the presence of elements belonging to a specific class only (i.e., samples being either always normal or always abnormal), we concentrated our analysis on only three adulterants -pH, OX, and GL -for which we managed to collect data belonging to both classes. Consequently, we created a dataset consisting of 181 images. Fig. 4 right provides an example of the input image where we cropped the specific portion of interest, filling the remaining part with black pixels. The size of pictures is 215 × 225 pixels. Our model was then trained exclusively on images considering these components.

Given the small size of our dataset, we employed the data augmentation technique [31], which is crucial to avoid overfitting when the amount of available data is limited [32]. Furthermore, due to the difficulty to collect normal assays, the original dataset is unbalanced between the two classes, with 133 images of abnormal tests and only 48 pictures of normal assays (the ratio is around 2.77). The class imbalance problem is a major concern in Machine Learning (ML) and Deep Learning (DL) [33]. In fact, training models on unbalanced data may result in learning most from the larger class, with consequent sub-optimal performances and poor generalization capabilities (for instance, a model could associate one class to all the input data regardless of the image features). Aiming to mitigate such issue, we first apply transformations so as to balance the two types of data (see Table 1), thus obtaining 300 images equally split between the two classes, 80% of which constitute the training set and the remaining 20% are the test set. The balancing operation has been performed in order to ensure that each image and its variation(s) cannot be in both training and test sets, hence excluding the possibility of overfitting. Then, we use the RandomAdjustSharpness transformation [34] (parameters: 𝑓 𝑎𝑐𝑡𝑜𝑟 ∈ [0, 0.25, 0.5, 0.75, 1.25, 1.5, 2, 2.5, 3]; 𝑝𝑟𝑜𝑏 = 1.0) to widen the set of input images in both training and test sets. The type of transformations employed to increase the number of data has been chosen carefully by taking into account the specific nature of the problem and the criticality of modifying image colors (see Fig. 2). Overall, the final training set consists of 2400 images, while the final test set contains 600 images.

Our model has been trained 10 times, each one starting with a different network weight initialization. The use of multiple replications minimize the risk of overestimating the model's performance due to lucky conditions. Training lasts 50 epochs, the learning rate is set to 10 −4 and the batch size is set to 16. The model's optimizer is Adam [35] with weight decay, whose value is 10 −2 . The size of pooling filters is 2 × 2. The experimental parameters have been derived from [26] and are empirically determined. Before training our model, we applied the k-fold cross-validation technique [36] to verify whether our deep network is suitable for the considered problem and mitigate the data overfitting issue [37]. We set 𝑘 = 5 and measured the average accuracy of the model by computing the Cross-Entropy (CE) loss metric. The average CE obtained during cross-validation phase is 92.917%, that is the proposed model achieves sufficiently good categorization performances and correctly discriminates between normal and abnormal tests. Therefore, we can state that our model is suitable for the considered problem.

Aiming to demonstrate the novelty and efficacy of the proposed model, we perform a comparison with the DenseNet121 [38] and ResNet18 [39] pre-trained networks, which represent two state-of-the-art models. We choose these networks since they are characterized by a number of trainable parameters of similar orders of magnitude compared to our approach, as we will illustrate in the next section. This allows us to perform a fair evaluation of the presented model.

Results

In this section we provide the outcomes of our experiments. As for the k-fold cross-validation phase, we employed the CE loss as a performance measure. List of transformations applied to the original images in order to balance pictures between the two output classes (i.e., "normal" and "abnormal"). Specifically, we generate 17 images of abnormal tests and 102 pictures of normal assays. The resulting set contains 300 images equally distributed among the two classes. For further details about the transformations, the reader is referred to [34]. As we mentioned above, we compared the outcomes of our model with those achieved with the DenseNet121 and ResNet18 networks. The analysis is illustrated in Table 2 and reveals that the ConvNet3_4 model is notably superior to both DenseNet121 and ResNet18 with respect to the accuracy: pre-trained networks manage to correctly classify only around half of the images, i.e. they are not able to discriminate between the two possible output classes (see also Fig. B.1). The result is in line with those reported in [26]. Moreover, our ConvNet3_4 model is also remarkably better than DenseNet121 and ResNet18 in terms of efficiency (see the significant discrepancy concerning the training time in Table 2), which represents a pivotal property for a model applicability in real scenarios.

Output class

The results we presented so far demonstrate that our model is suitable to address the considered problem and achieve excellent performances in terms of classification capability. Nonetheless, as we stated in section 1, a worthwhile aspect in the fields of medicine and healthcare is the explainability of the proposed models, which is necessary to practically employ them in real-application cases. To this end, we performed a feature analysis by using two different techniques: Saliency [40] and Integrated Gradients (IG) [41]. Both methods are widely employed to interpret the outcomes of a model's classification [42][43][44]. The former method allows the identification of the parts of the image contributing more to the output prediction. An example of the feature map extracted through the Saliency method is shown in Fig. 7 middle. Saliency values range from 0 (absence of saliency) to 1 (positive saliency), as indicated in Fig. 7 middle. For a more detailed description of the Saliency algorithm, the reader is referred to [40]. Conversely, the IG method identifies the regions of the image that most influenced the model's classification decision by considering the entire input-output trajectory and the reference input distribution (baselines) used in the attribution calculation. As pointed out in [41], the IG method first considers the input image 𝑥 and a baseline 𝑥 ′ , which is an input characterized by absence of features. Then, a straightline path from 𝑥 ′ to 𝑥 is taken into account. IG computes the gradients at all points along such path. The integrated gradients are obtained as the cumulative sum of these gradients. Put more formally, if we denote our CNN model with 𝐹 : R 𝑛 → [0, 1], the integrated gradients along the 𝑖 𝑡ℎ dimension is calculated as:

𝐼𝐺 𝑖 (𝑥) = (𝑥 𝑖 − 𝑥 ′ 𝑖 ) × ∫︁ 1 𝛼=0 𝜕𝐹 (𝑥 ′ + 𝛼 × (𝑥 − 𝑥 ′ )) 𝜕𝑥 𝑖 𝑑𝛼(1)

where 𝜕𝐹 (𝑥) 𝜕𝑥 𝑖 indicates the gradient of 𝐹 (𝑥) along the 𝑖 𝑡ℎ dimension. Overall, the IG method enables to detect the portions of the picture providing positive (parts in green, see Fig. 7 right) and negative (parts in red, see Fig. 7 right) contributions to the output prediction. Fig. 7 shows the image of a lateral-flow test categorized as "normal" (Fig. 7 left), the Saliency feature map (Fig. 7 middle) and the Integrated Gradients heat map (Fig. 7 right). Fig. 8 contains the same data for an "abnormal" assay. The colorbars below the heat maps specify respectively the intensity of the saliency and the magnitude of the importance attribution of each region of the image to the model's prediction.

By examining the feature maps associated to a normal lateral-flow test (Fig. 7), we can observe that the Saliency method returns a positive saliency for the pH and GL adulterants and a slightly positive saliency for the OX adulterant (Fig. 7 middle). Concerning the Integrated Gradients technique, the heat map displays a positive attribution for the pH and GL adulterants and a slightly positive attribution for the OX adulterant (see Fig. 7 right). Therefore, in the case of a normal assay, the model assigns the same importance to the portions of the image containing the considered adulterants. This outcome is in line with the actual test result.

If we look at the relevant features highlighted by the Integrated Gradients technique with regard to an abnormal assay, we can see that a slightly positive attribution is conferred to the pH, OX and GL adulterants (Fig. 8 right). As far as the Saliency algorithm is concerned, it returns a positive saliency for the GL adulterant and a slightly positive attribution for the pH and OX adulterants (Fig. 8 middle). Also in this case, the results are coherent with the actual test outcome, which indicates non-compliance for the OX and GL adulterants. Overall, our findings suggest that the proposed model primarily relies on the portions of the image containing the pH, OX and GL adulterants. However, the amount of contribution strongly depends on the test result (e.g., normal or abnormal) and the specific color gradation of the adulterants. Indeed, except for one case, the OX adulterant is characterized by soft nuances tending to be as similar as the background color of the image. Similarly, the normality and abnormality of the pH adulterant are defined based on subtle color gradations. Therefore, distinguishing between the two cases might be challenging even for laboratory operators. Finally, it is worth noting that cropping the picture does not affect the classification capability of the model. Indeed, the use of black pixels providing no information allows the model to focus only on the remaining parts of the image, which contains the relevant data.

To summarize, our outcomes demonstrate the capability of the ConvNet3_4 model to extract the most relevant features of the input image in order to generate a precise prediction of the output class. In particular, the identification of the portions containing the adulterants as the key elements of the input image implies that the model is capable of making assumptions from a medical point of view. Indeed, discriminating between the color nuances of the considered adulterants (see Fig. 2) is far from trivial even for experienced and well-trained operators.

Discussion and conclusions

The spread of AI methods poses questions about the explainability of such models, particularly when they are applied in real-world contexts. Especially in medicine and healthcare, using explainable and trustworthy approaches is paramount in order to help doctors and other professionals to make diagnoses of possible diseases, design adequate therapies for prevention or rehabilitation, explain and collect historical data. Generally, analyzing huge amount of medical data is addressed through DL methods and CNNs represent a widespread tool, although they are often tailored to specific applications. This represents a major concern in the possibility to develop cross-cutting tools. In this work, we proposed a novel approach for the problem of automatically classify lateral-flow tests for drug abuse detection. Specifically, we considered the adulterant section of an assay and we trained a CNN model for the ability to categorize tests (i.e., normal or abnormal) by analyzing the pH, OX and GL adulterants only. We used the network introduced in [25,26] with the addition of pooling layers in the convolutional part of the model. The use of pooling enables the development of slim networks that can be used in real-world scenarios, particularly when dealing with limited hardware resources. We verified the suitability of the model through a 5-fold cross-validation and we ran the training 10 times. We collected promising results on the chosen task, with an excellent average accuracy. The proposed approach is also notably superior to two state-of-the-art deep networks. Moreover, we provided an explainability of our model by performing a feature analysis. Our outcomes reveal the importance of some portions of the input image (those containing the adulterants), while other parts affect the final prediction only partially.

In spite of the good results we achieved, further research is needed to generalize our approach. First, we are collecting samples so as to broaden our analyses to the SG, CRE and NI adulterants, which are out of the scope of this work, and increase the size of our dataset. Owning a significant high number of images is pivotal to apply the model in a real case, since medical data usually include hundreds or thousands of pictures. However, extending the analysis to all adulterants implies considering input images of different sizes, with an unavoidable effect on the model's performance. Furthermore, the ConvNet3_4 model proves effective at dealing with the considered problem, with a very good classification capability, and outperforms the DenseNet121 and ResNet18 pre-trained networks. Nonetheless, we observe the presence of oscillations during training due to the sensitivity to specific batches of input images. Aiming to address such undesired behavior, in the future we will consider the possibility to adapt the learning rate and/or the weight decay during training. In addition, with respect to the explainability issue, we provide evidence of the reasons behind the success of our model. Nonetheless, because the