In this study, we introduce a pioneering web-based application designed to enhance melanoma detection accuracy through the innovative use of a Vision Transformer (ViT) model. Leveraging the power of advanced deep learning architectures, our application processes dermatological images to identify potential melanomas with a precision previously unattainable in conventional screening methods. The development process involved finetuning the ViT model on a diverse dataset of dermatoscopic images, ensuring robustness and reliability across a wide range of images. The web application is intuitively designed, allowing for easy access and use by dermatologists and potentially by the general public for preliminary screening purposes. This research not only underscores the viability of ViT models in medical imaging but also ofers a practical tool for early melanoma detection, thereby contributing to better clinical outcomes and facilitating early treatment interventions.
and the general public, facilitating widespread screening and awareness.
The early diagnosis of melanoma, a highly aggressive Such web applications can transform smartphones and
form of skin cancer, is crucial for improving patient personal computers into powerful tools for preliminary
outcomes [
The integration of AI into medical imaging for To date, the ABCDE method is the standard
apmelanoma detection allows for the analysis of derma- proach used by medical professionals for the
diagnotological images with a level of detail and precision that sis of melanoma, emphasizing the evaluation of lesions
surpasses human capability [
In this study, we have developed a comprehensive and ISIC 2020, the fine-tuned model was subsequently
dataset derived from the ISIC (International Skin Imaging tested on an entirely diferent dataset, MEDNODE [
In this work, we have constructed a dataset starting The concept of transfer learning, applied to the context
from the data of the ISIC Challenges of 2019 [
For the dataset preparation, with the objective of melanoma detection. In this phase, the model is specifiachieving balanced classes between Melanoma and No cally adapted to the characteristics of dermatological imMelanoma, images were extracted from ISIC 2019 and ages. Although the network already has a broad recogniISIC 2020. The resulting dataset comprises 8,223 training tion capability thanks to its previous training, fine-tuning images and 1,450 test images, witnessing a significant allows for the refinement of its precision and sensitivincrease in the sample size, especially in the number of ity to the peculiarities of skin lesions, thus enhancing images within the Melanoma class of interest (3,890 No diagnostic accuracy.
Melanoma, 4,333 Melanoma). This dataset was synthe- The development environment utilized for training sized from images sourced from ISIC 2019 and ISIC 2020. the model features the following specifications in AWS Specifically, the “Melanoma” class benefited from con- Environment: p3.2xlarge, Intel(R) Xeon(R) CPU E5-2686 tributions from both datasets, while the “No-Melanoma” v4 @ 2.30GHz, 64GB RAM, Graphics Card: Tesla V100class was formed using images exclusively from ISIC 2019. SXM2 with 16GB of VRAM.
This balanced approach ensures a more equitable distri- For the inference phase, no specific dedicated hardbution of classes, enhancing the model’s ability to learn ware with GPU is required. This flexibility in hardware and accurately diferentiate between Melanoma and No requirements for inference ensures that the fine-tuned Melanoma, which is crucial for the model’s performance model can be deployed in a wide range of environments, making it accessible for clinical use without the need for high-performance computing resources.
The results obtained from the model on the test set, as detailed in Table 1, underscore its adeptness in distinguishing between melanoma and non-melanoma lesions, showcasing substantial precision, recall, and F1-score Figure 1: Diagnostic Performance of the Vision Transformer metrics across both categories. For the "No Melanoma" Model for Melanoma Detection on ISIC test set. On the left, class, precision was marked at 0.86, with a recall of 0.83 the confusion matrix illustrates the model’s accuracy in clasand an F1-score of 0.85, across a support of 677 cases. sifying 1: ’Melanoma’ and 0: ’No Melanoma’ cases, with the This high level of accuracy in identifying non-melanoma number of true positives, true negatives, false positives, and instances indicates a strong balance between the preci- fCahlsaerancetgearitsitviecs).cOurnvet.he right, the ROC (Receiver Operating sion and recall, reflecting the model’s eficiency in minimizing false positives while efectively recognizing true negatives.
Conversely, the "Melanoma" class demonstrated pre- ize melanoma screening and diagnosis through the incision and recall scores of 0.86 and 0.88, respectively, tegration of cutting-edge AI technologies into clinical achieving an F1-score of 0.87 over 773 instances. These practices, thereby enhancing diagnostic accuracy and metrics highlight the model’s capability in reliably detect- facilitating improved patient care. ing melanoma lesions, with the elevated recall indicating a particular strength in reducing false negatives—crucial 3.3. Validation on MEDNODE Dataset for melanoma screening where the cost of missing a positive diagnosis is exceedingly high. The model’s performance on the MEDNODE dataset, as
The consistency in precision across both classes, paired detailed in Table 2, reflects its diagnostic precision in with the balanced F1-scores, attests to the model’s robust- distinguishing between melanoma and non-melanoma ness, suggesting its potential as a dependable tool in the cases. With a precision of 0.83 and a recall of 0.98 for "No diagnostic toolkit. Such performance, detailed in Table 1, Melanoma," the model demonstrates a high capability in afirms the advanced AI models, like the Vision Trans- correctly identifying non-melanoma cases, as evidenced former’s, significant role in dermatological diagnostics. by an F1-score of 0.90 across 100 instances. This high
Further insight into the model’s performance is pro- recall rate is crucial, indicating the model’s strength in vided by Figure 1, which depicts the confusion matrix minimizing the risk of false negatives in non-melanoma and the ROC curve for the model’s predictions. The con- diagnoses, which is vital for avoiding unnecessary further fusion matrix visually illustrates the model’s accuracy testing and anxiety for patients. in classifying the test cases, ofering a clear depiction Conversely, for the "Melanoma" category, the preciof the true positive and negative rates, alongside the in- sion stands at an impressive 0.96, showing the model’s stances of false positives and negatives. The ROC curve, reliability in its melanoma predictions. However, the reaccompanying this matrix, further elucidates the model’s call of 0.71 highlights a potential area for improvement diagnostic ability across diferent thresholds, showcas- in capturing all true melanoma cases, with an F1-score of ing its exceptional capability to balance sensitivity and 0.82 across 70 instances reflecting the balance between specificity efectively. Together, Table 1 and Figure 1 precision and the need to improve recall. ofer a comprehensive overview of the model’s diagnos- Figure 2 provides further insight into these results tic performance, highlighting its potential to revolution- through a visual representation. The left side of the figure features the confusion matrix, illustrating the model’s taining high diagnostic accuracy, the model paves the way for broader clinical adoption, ofering a promising tool for early melanoma detection and thereby improving patient outcomes through timely and accurate diagnoses.
visualizes the diagnostic results, including the probabil- AI-driven diagnostics accessible to a broad audience. ity of melanoma presence. This section exemplifies the In essence, this work illustrates the synergy between critical role of data visualization in making complex AI cutting-edge AI technology and user-centric application analyses accessible and understandable to users, facilitat- design in addressing the critical healthcare challenge of ing an informed interpretation of the results. early melanoma detection. By marrying the technical
The emphasis on data visualization and ease of use prowess of Vision Transformers with the accessibility within the Streamlit application not only democratizes and clarity provided by Streamlit, this initiative paves access to advanced melanoma detection tools but also sig- the way for future advancements in the field of medinificantly enhances the user experience. By translating cal imaging and diagnosis. It stands as a testament to sophisticated AI diagnostics into intuitive visual outputs, the potential of AI to not only revolutionize diagnosthe application bridges the gap between complex medical tic processes but also to empower individuals with the data and actionable insights, making it an invaluable tool tools and knowledge necessary for early detection and in the early detection of melanoma. intervention, ultimately contributing to better healthcare outcomes and the broader goal of reducing melanomarelated mortality.
In conclusion, this article has presented a comprehensive exploration of an innovative web application developed for the early detection of melanoma, leveraging the advanced capabilities of Vision Transformers (ViTs) and the intuitive platform of Streamlit. The initial stages of this work involved the meticulous construction of a dataset from the ISIC Challenges of 2019 and 2020, followed by the fine-tuning of a pre-trained ViT Large model from Google, with the dual objectives of distinguishing between "Melanoma" and "No Melanoma" cases and adapting the model to the unique characteristics of dermatological images. This process was underscored by the strategic preparation of the dataset to ensure balanced classes, thereby enhancing the model’s learning and predictive accuracy.
Subsequent testing on the distinct MEDNODE dataset confirmed the model’s robustness and adaptability, demonstrating significant diagnostic precision across varying conditions. The application of transfer learning techniques further exemplified the utility of leveraging existing AI models for specialized tasks, reducing both the time and resources required for model development from scratch. The deployment environment, characterized by high-performance computing resources, facilitated the model’s training and validation phases, while the streamlined requirements for the inference phase underscored the model’s practical applicability in diverse clinical settings.
The Streamlit-based web application represents a significant stride towards democratizing access to advanced diagnostic tools. By ofering an intuitive interface for uploading and analyzing dermatological images, coupled with real-time presentation of diagnostic results, the application emphasizes the critical role of data visualization in enhancing user engagement and understanding. Each segment of the application, from providing background on melanoma detection to visualizing AI-generated predictions, is designed to make the complex process of