<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>A Self-Supervised Learning Approach for Detecting BRCA Mutations in Breast Cancer Histopathological Images</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Faycal Touazi</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Djamel Gaceb</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Chaima Belkadi</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Besma Loubar</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>LIMOSE Laboratory, Computer Science Department, University M'hamed Bougara</institution>
          ,
          <addr-line>Independence Avenue, 35000 Boumerdes</addr-line>
          ,
          <country country="DZ">Algeria</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Breast and ovarian cancers are among the most pressing health issues afecting women globally, with genetic mutations, particularly in the BRCA1 and BRCA2 genes, significantly influencing their development. This thesis ofers a comprehensive overview of these cancers, emphasizing the genetic, anatomical, and histopathological factors that contribute to their onset and progression. A detailed examination of the anatomy of the female breast and ovaries provides insight into the origins of these malignancies. The critical role of histopathology in identifying specific cancer subtypes and gene mutations is explored, underscoring its vital importance in diagnosis and treatment. Our results demonstrate that the developed deep learning framework, integrating Vector Quantized-Variational Autoencoders (VQ-VAE) and DBSCAN for clustering, achieved an accuracy of 95% in classifying BRCA mutation-positive and negative cases, outperforming traditional diagnostic methods. By investigating the interplay between genetic predisposition and histopathological analysis, this thesis aims to enhance the understanding of breast and ovarian cancers and their implications for public health.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Breast Cancer</kwd>
        <kwd>BRCA mutation</kwd>
        <kwd>deep learning</kwd>
        <kwd>Self supervised learning</kwd>
        <kwd>BRCA 1</kwd>
        <kwd>BRCA 2</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>Breast cancer remains one of the most prevalent cancers globally, afecting millions of women each
year. Early detection is a critical factor in improving survival rates, as it allows for timely intervention
and treatment. Traditional methods for breast cancer detection, such as mammography, have long been
the gold standard in screening programs. In recent years, deep learning has emerged as a powerful tool
for enhancing breast cancer detection, particularly in medical imaging tasks such as mammography
interpretation.</p>
      <p>
        Deep learning algorithms, particularly Convolutional Neural Networks (CNNs), have demonstrated
remarkable success in breast cancer detection from mammography images. Studies have shown that
deep learning models can achieve accuracy levels comparable to radiologists in identifying tumors
[
        <xref ref-type="bibr" rid="ref1 ref2 ref3">1, 2, 3, 4</xref>
        ].
      </p>
      <p>Mutations in the BRCA1 and BRCA2 genes are among the most well-known genetic risk factors for
breast cancer. These mutations, which can be inherited, significantly increase a woman’s lifetime risk
of developing breast cancer. Women who carry a BRCA1 or BRCA2 mutation have an elevated risk of
50 to 85% of developing breast cancer by the age of 70, compared to a 12% risk in the general population
[5].</p>
      <p>The discovery of a BRCA mutation in a patient is of crucial importance, not only for early diagnosis
and cancer management, but also for informed decision making regarding preventive measures and
treatment options. Identifying such mutations can lead to personalized surveillance strategies,
riskreducing surgeries, and targeted therapies, thus improving the overall prognosis and quality of life of
high-risk patients [6] [7] [8].</p>
      <p>Deep learning has revolutionized various domains and its impact on the medical field is particularly
profound. The ability of deep learning algorithms to analyze complex patterns in large datasets has led
to significant advances in medical diagnostics, treatment planning, and personalized medicine. In the
context of medical imaging, deep learning models have demonstrated exceptional accuracy in tasks
such as detecting abnormalities, classifying diseases, and predicting patient outcomes. These models,
which often outperform traditional methods, have the potential to assist clinicians in making more
informed decisions and improving patient care.</p>
      <p>Our study leverages deep learning techniques to address the challenges associated with detecting
BRCA1 and BRCA2 mutations in histopathological images of breast and ovarian cancer. By training a
robust model on a curated dataset, we aim to provide a reliable tool for identifying these genetic
mutations. The results presented in this work highlight the efectiveness of our approach and demonstrate
the potential of deep learning in enhancing the accuracy of cancer detection and prognosis. Through
this research, we contribute to the growing body of evidence supporting the integration of deep learning
into clinical practice, ultimately aiming to improve outcomes for patients with hereditary cancer risks.</p>
      <p>This paper is organized as follows: Section 2 reviews related works. Section 3 outlines our proposed
approach, focusing on the Vector Quantized Variational AutoEncoder (VQ-VAE). Section 4 describes the
experimental setup, including the TCGA-BRCA dataset, preprocessing, and evaluation metrics. Section
5 presents the results, covering clustering, BRCA patch classification, and SVS image classification,
along with comparisons to related work. Finally, Section 6 concludes with a summary of findings and
future research directions.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Related Works</title>
      <p>In this section, we ofer a comprehensive review of recent studies that focus on detecting BRCA
mutations in breast cancer using deep learning methods.</p>
      <p>Shen Zhao et al. [9] developed a deep learning framework for comprehensive molecular and prognostic
stratifications of triple-negative breast cancer (TNBC). The framework features two CNNs in series: the
ifrst, a tissue type classifier based on ResNet-18, achieved a weighted F1 score of 0.96, classifying tissue
types with near 90% accuracy. The second CNN predicted molecular features and relapse risks with
AUCs ranging from 0.71 to 0.76.</p>
      <p>Xiaoxiao Wang et al. [10] proposed a deep learning model based on CNNs to predict BRCA gene
mutations from histopathological images. Trained on the JSPHCM and JSCH datasets, their model
demonstrated high performance with AUC values ranging from 79%.</p>
      <p>Tristan Lazard et al. [11] employed multiple instance learning (MIL) techniques to identify
morphological patterns indicative of homologous recombination deficiency in luminal breast cancers. Their
model, tested on a dataset of 673 WSIs from TCGA and an in-house dataset, achieved an AUC of 71%.</p>
      <p>Nam Nhut Phan et al. [12] developed a deep learning pipeline for classifying breast cancer molecular
subtypes from unannotated pathological images. Their approach utilized a two-step transfer learning
process with CNNs such as ResNet50, ResNet101, VGG16, and Xception. Initially, the models were
pre-trained on ImageNet and then fine-tuned on an internal dataset. They were subsequently trained on
the TCGA-BRCA dataset to classify breast cancer into basal, HER2, luminal A, and luminal B subtypes.
The images were normalized to 512x512 pixels, and patches were extracted from WSIs. The models
achieved average AUCs ranging from 88 to 92%.</p>
      <p>Kurian et al. [13] proposed a semi-supervised learning approach to classify breast cancer subtypes
using histopathological images from the TCGA-BRCA dataset. They focused on diferentiating between
Basal and Luminal A PAM50 subtypes by analyzing a curated subset of 180 whole slide images (WSIs)
selected to minimize heterogeneity. Their model leveraged a Deep Neural Network (DNN) architecture
based on SimCLR with a ResNet18 backbone for out-of-distribution (OOD) detection, pre-trained on a
large histology image dataset. Patch extraction from annotated tumor regions enabled the model to
focus on relevant regions, although it introduced potential label noise. They achieved a patient-level
accuracy of 81.43%.</p>
      <p>The methodology employed by Valieris et al. [14] involved developing a deep learning framework to
detect homologous recombination (HR) deficiency in breast tumors using the TCGA-BRCA dataset. The
model leveraged whole slide images (WSI), utilizing advanced image processing techniques to extract
histopathological features indicative of HR deficiency. To address the complexity and variability in
these images, the authors implemented a multiple instance learning (MIL) approach, allowing the model
to learn from entire tumor samples without the need for manual segmentation. Their model achieved
an area under the curve (AUC) of 80%.</p>
      <p>Table 1 provides a comparative summary of the performance achieved in state-of-the-art studies for
breast cancer classification, highlighting diferent datasets, methods, and evaluation metrics used.</p>
      <p>Dataset
Tristan Lazard et al. [11] TCGA
Xiaoxiao Wang et al. [10] JSPHCM, JSCH
Kurian et al. [13] TCGA-BRCA
Valieris et al. [14] TCGA
Nam Nhut Phan et al. [12] TCGA-BRCA</p>
      <p>Year</p>
    </sec>
    <sec id="sec-3">
      <title>3. Proposed approach</title>
      <p>In this section, we describe our proposed approach for the detection and diagnosis of breast cancer using
advanced deep learning techniques. Our approach is designed to address the challenges of analyzing
histopathological images and aims to provide a comprehensive solution to detect and classify breast
masses. But first we will introduce a key architecture in our proposal.</p>
      <sec id="sec-3-1">
        <title>3.1. Vector Quantized Variational AutoEncoder</title>
        <p>The Vector Quantized Variational AutoEncoder (VQ-VAE)[15] is a type of variational autoencoder that
introduces vector quantization to obtain a discrete latent representation, distinguishing itself from
traditional VAEs, which produce continuous latent codes. The VQ-VAE uses a codebook, a matrix e
of dimensions  × , where  represents the number of embeddings and  is the dimensionality
of each embedding (see Figure 1). This architecture enables the encoding of data into discrete codes,
which helps in learning compact and structured representations. The model consists of three main
components:
• An encoder: that maps input data  (such as images) into continuous latent representations .
• A vector quantizer: that transforms these continuous representations into discrete vectors
 using the codebook, by selecting the closest embedding through minimizing the Euclidean
distance:
quantization() = argmin|| − ||2
∈
(1)
• A decoder: that reconstructs the original data from the discrete latent codes.</p>
        <p>The codebook, which contains the learned embeddings, plays a critical role in the quantization process.
It allows the continuous output of the encoder to be mapped to discrete codes, facilitating the generation
of data from these discrete codes. By using a discrete latent space, VQ-VAE simplifies model optimization
and enables the use of generative models based on discrete distributions, such as PixelCNN or other
autoregressive models, to model the latent codes.</p>
        <p>One of the key advantages of VQ-VAE is its ability to capture meaningful structural representations in
the data, making it particularly useful for high-quality generation tasks, especially in areas like medical
imaging and discrete signal modeling.</p>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. Methodology</title>
        <p>In this section, we detail the steps of our methodology for analyzing histopathological images to identify
BRCA mutations and classify cancerous versus normal cells. Our approach employs image preprocessing,
feature extraction using Vector Quantized-Variational Autoencoder (VQ-VAE), and clustering techniques
to organize images based on mutation status, enabling precise classification and improving diagnostic
accuracy (see Figure 2).</p>
        <p>• Step 1: Input Images The process begins with the acquisition of large histopathology images in
SVS (Scalable Vector Graphics) format. These images are particularly challenging to handle due
to their high resolution and substantial size, which necessitates advanced processing techniques.
To manage this, the SVS images are divided into smaller patches of size 1400 × 920 pixels. This
approach simplifies the analysis and processing of the images while retaining important details.</p>
        <p>The dataset comprises a diverse set of patches,
• Step 2: Dataset Categorization: After patchifing the dataset into small images were CNN
models can process theme, further refinement involves categorizing patches based on BRCA
mutation status. The patches are divided into three distinct categories: those related to SVS
images where BRCA1 is identified, those where BRCA2 is identified, and those with no identified
BRCA mutations. This detailed categorization enables a more focused analysis of IDC patches
in relation to specific BRCA mutations, enhancing the understanding of their histopathological
features.
• Step 3: Image Characterization Using Vector Quantized-Variational Autoencoder
(VQVAE)
In our approach, we utilize a Vector Quantized-Variational Autoencoder (VQ-VAE) [16] with
a codebook of 128 discrete vectors to handle and analyze high-resolution histopathological
images. This model is crucial for efectively managing the complexity of these images through its
encoder-decoder architecture.</p>
        <p>– Encoder Network: The encoder network transforms high-resolution input images into a
continuous latent representation. It consists of multiple neural network layers that extract
significant features and reduce the dimensionality of the images while retaining important
details.
– Vector Quantization: Following the generation of the continuous latent representation,
VQ-VAE applies vector quantization. This process maps the continuous latent vectors to the
nearest discrete vectors in a predefined codebook of 1024 entries. This quantization step
converts the latent space into a more manageable and structured form, which simplifies
further analysis.
– Codebook: The codebook, comprising 1024 discrete vectors, is updated during training
to minimize reconstruction loss. This ensures that the codebook efectively captures the
essential characteristics of the input images.
– Decoder Network: The decoder network reconstructs the high-resolution images from
the quantized latent representation. Using the discrete codes produced by the encoder, the
decoder aims to accurately recreate the original images, preserving critical features and
details.
– Dimensionality Reduction and Eficient Analysis: The combination of the encoder,
vector quantization, and decoder facilitates dimensionality reduction of high-resolution images.
This reduction compresses the data into a latent space that retains essential information,
making the data more suitable for eficient analysis and processing.
• Step 4: Feature Extraction with VQ-VAE: To extract meaningful features from the images,
we utilize the VQ-VAE model. The VQ-VAE’s encoder network processes the histopathological
images to generate continuous latent representations, which are then quantized into discrete
vectors using the codebook. This approach captures intricate patterns and features within the
images. The aim is to characterize the images with a reduced dimensionality representation,
which simplifies and enhances the clustering operation. This method provides a comprehensive
and structured feature representation by reducing the dimensionality of the high-resolution
images, making it easier to perform efective clustering.
• Step 5: Latent Space Representation: After feature extraction, each image is encoded into a
latent vector. These latent vectors collectively form a dataset that is used for subsequent analysis.
This latent space representation simplifies the data and prepares it for clustering and other
operations.
• Step 6: Clustering and Annotation: After encoding the images into latent vectors, we apply
the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm to group
similar vectors into clusters. This clustering approach organizes the images into meaningful
groups based on their feature vectors, distinguishing between BRCA mutation-positive and BRCA
mutation-negative cases.
• Step 7: Classification: The final stage involves classifying the images into two main categories:
normal cells and cancerous cells. This classification is based on the previously obtained clusters
and latent space representations. The model aims to enhance the accuracy of diferentiating
between various types of cancerous and non-cancerous tissues, thereby improving diagnostic
capabilities.</p>
        <p>By integrating VQ-VAE with advanced CNNs and clustering techniques, our approach provides a
robust framework for analyzing histopathological images. This methodology is designed to improve
the performance of breast cancer detection and diagnosis, ofering a more accurate and comprehensive
analysis of histological samples.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Experimentations and results</title>
      <sec id="sec-4-1">
        <title>4.1. TCGA-BRCA Dataset</title>
        <p>The TCGA-BRCA dataset, referenced in [17], is part of the Cancer Genome Atlas (TCGA) project, which
aims to enhance the understanding of cancer through comprehensive genomic studies. This dataset
includes RNA sequencing, somatic mutation profiles, and gene-level copy number variation data from
1,098 breast invasive carcinoma cases. It contains 1978 images from these 1,098 patients, with 763 tumor
samples that include single nucleotide polymorphism (SNP) and copy number variation (CNV) data
generated using the Afymetrix 6.0 SNP array, alongside somatic mutation information obtained from
the Illumina sequencing platform. Data sources include the Genomic Data Commons (GDC) Data Portal,
Pan-Cancer Atlas, and The Broad Institute’s TCGA GDAC Firehose. The dataset is publicly available
through both the GDC Data Portal and the Cancer Imaging Archive (TCIA).
4.2. Data Pre-processing
In this study, we undertake a comprehensive pre-processing procedure to prepare histopathology images
for deep learning analysis.</p>
        <p>First, 200 SVS image files were collected from the TCGA-BRCA dataset, including images with
BRCA1 or BRCA2 mutations, as well as some without these mutations. These images can be as large as
130,000x99,000 pixels. To facilitate eficient processing and analysis, the images were divided into smaller
patches of 1400 x 920 pixels (see Figure 4 for exemples). Then each patch was classified according to the
status of the BRCA mutation, distinguishing between the BRCA mutation positive and BRCA mutation
negative cases. This classification is critical for investigating the role of BRCA mutations in breast
cancer. The dataset was organized according to the BRCA mutation status, ensuring a comprehensive
range of examples for model training. Subsequently, the dataset was split into training and validation
sets to prepare for model evaluation (see Table 2 for the distribution of images).</p>
      </sec>
      <sec id="sec-4-2">
        <title>4.3. Used Metrics and Loss Functions</title>
        <p>In this study, we use a variety of metrics and loss functions to evaluate and optimize our deep learning
models for breast cancer detection and diagnosis. This includes the VQ-VAE model, which employs a
specialized loss function. Below, we outline the metrics and loss functions used:</p>
        <p>1 ∑︁ [ log(ˆ) + (1 − ) log(1 − ˆ)]
LossBCE = −</p>
        <p>=1
where  denotes the ground truth label and ˆ is the predicted probability.
• VQ-VAE Loss Function: The VQ-VAE model utilizes a specialized loss function that includes
three key components:
– Reconstruction Loss: Measures how well the decoder reconstructs the input from the
quantized representation. It ensures that the reconstructed image is similar to the original
input image. The formula is:
4.4. Loss Functions:
• Binary Cross-Entropy Loss: Applied for binary classification tasks, such as distinguishing
between cancerous and normal patches. Measures the performance of a classification model with
output probabilities between 0 and 1. The formula is:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
LossRecon =</p>
        <p>1 ∑︁ ‖ − ˆ‖2
 =1
where  is the original image and ˆ is the reconstructed image.
– Codebook Loss: Encourages the codebook vectors to move closer to the encoder output,
ensuring that the quantization process efectively captures the data’s structure. This
component helps to learn a better representation by minimizing the distance between the encoder
output and codebook vectors. The formula is:</p>
        <p />
        <p>LossCodebook = 1 ∑=︁1 ‖ − ()‖2
where  is the continuous latent vector and () is the quantized vector.
– Commitment Loss: Penalizes the encoder for not committing to a specific codebook vector,
promoting stability in the learned representations. This component helps to stabilize the
learning process and ensure that the encoder uses the codebook vectors efectively. The
formula is:</p>
        <p />
        <p>LossCommitment =  1 ∑=︁1 ‖ − ()‖2
where  is a hyperparameter that controls the weight of the commitment loss term.</p>
      </sec>
      <sec id="sec-4-3">
        <title>4.5. Evaluation Metrics:</title>
        <p>• Accuracy: Measures the proportion of correctly classified patches out of the total number of
patches. The formula is as follows:
• Precision and Recall: Precision measures the proportion of true positive predictions among all
positive predictions, while Recall measures the proportion of true positive predictions among all
actual positives. These metrics are defined as:</p>
        <p>Accuracy =</p>
        <sec id="sec-4-3-1">
          <title>Number of Correct Predictions Total Number of Predictions</title>
          <p>Precision =</p>
        </sec>
        <sec id="sec-4-3-2">
          <title>True Positives</title>
          <p>True Positives + False Positives
Recall =</p>
        </sec>
        <sec id="sec-4-3-3">
          <title>True Positives</title>
          <p>True Positives + False Negatives
• F1 Score: The harmonic mean of Precision and Recall, providing a balanced measure of model
performance. The formula is:
• Area Under the Receiver Operating Characteristic Curve (AUC-ROC): Measures the model’s
ability to distinguish between classes across all classification thresholds. A higher AUC value
indicates better model performance. The AUC is calculated as the integral of the Receiver
Operating Characteristic (ROC) curve:</p>
          <p>Precision · Recall
F1 Score = 2 · Precision + Recall
  =
∫︁ 1
0
  ()
(9)
(10)</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Results and discussion</title>
      <p>The discussion section is dedicated to analyzing and interpreting the results obtained from our
experiments:</p>
      <sec id="sec-5-1">
        <title>5.1. Clustering Results</title>
        <p>The clustering process using the DBSCAN algorithm [18] (Density-Based Spatial Clustering of
Applications with Noise) aimed to categorize patches into specific groups based on the presence or absence of
BRCA mutations. The parameters for DBSCAN were set with eps = 8.7 and min_samples = 180000,
guiding the clustering of the latent space representations.</p>
        <p>The goal of this clustering was to classify patches according to their status of BRCA mutation. The
clustering process identified two distinct categories of clusters. The first cluster contains patches from
both images with BRCA mutations and images without these mutations. The second cluster, however,
exclusively contains patches from images identified with BRCA mutations. This clustering approach
enables a more focused separation, supporting targeted analysis and model training based on the
presence or absence of the BRCA mutation.</p>
      </sec>
      <sec id="sec-5-2">
        <title>5.2. BRCA Patch Classification</title>
        <p>In this section, we present the results of classifying BRCA patches using three diferent deep learning
models: VGG16[19], ResNet [20], EficientNet [ 21], and Inception V3 [22]. The classification task
involves distinguishing between patches with BRCA mutations and those without. The data set used
for the classification of BRCA mutation consists of histopathological image patches, divided into three
subsets: training, validation, and testing. Table 4 summarizes the distribution of labels across the
training, validation, and test sets.</p>
        <p>The training set comprises a total of 57,152 samples, with 48,176 labeled as NO_BRCA and 8,976 as
BRCA. The validation set includes 11,431 samples, of which 9,639 are labeled as NO_BRCA and 1,792
as BRCA. Finally, the test set contains 14,288 samples, with 12,044 labeled as NO_BRCA and 2,244 as
BRCA.</p>
        <p>The classification results for the detection of BRCA mutation in patches using four diferent models
are presented in Table 5, which outlines key metrics such as precision, AUC, precision, recall and F1
score for both the BRCA and No Mutation classes. These metrics provide a comprehensive evaluation
of each model’s performance, highlighting their ability to distinguish between patients with BRCA
mutations and those without.</p>
        <p>All models exhibited exceptional performance with accuracy that exceeded 98%. Inception V3
achieved the highest accuracy at 98.94%, closely followed by EficientNet and VGG16, both at 98.81%,
while ResNet achieved 98.71%. The Area Under the Curve (AUC) further supports these findings, with
all models surpassing the 97% threshold, led by Inception V3 at 97.36%.</p>
        <p>Detailed precision, recall, and F1-score reveal that Inception V3 consistently outperformed the other
models in all metrics. For the BRCA class, Inception V3 achieved a precision of 98%, a recall of 95%,
and an F1 score of 97%. For the No Mutation class, Inception V3 reached near-perfect performance,
with a recall of 100%, precision of 99%, and an F1-score of 99%. These results highlight Inception V3’s
balanced sensitivity (recall) and precision across both classes, making it a reliable model for BRCA
mutation detection.</p>
        <p>Although all models show strong performance, Inception V3 stands out with the best overall metrics
in accuracy, AUC, and F1 score. EficientNet and VGG16 shared similar results, achieving an accuracy
of 98.81% and maintaining high precision and recall for both classes. ResNet, although slightly lower
in performance compared to the other models, still achieved competitive results with a precision of
97% for the BRCA class and high recall values.</p>
        <p>The consistently high performance of all models underscores the efectiveness of deep learning
architectures for histopathological image classification. However, the slight edge of Inception V3 in
both AUC and F1-score suggests that its architecture may be better suited for extracting subtle features
in histopathological images related to BRCA mutations, possibly due to its ability to capture multi-scale
features.</p>
        <p>To further analyze the classification performance, confusion matrices for the four
models—EficientNet, VGG16, ResNet, and Inception V3—are illustrated in Figure 5, showing the distribution
of true positives, false positives, true negatives, and false negatives for both the BRCA and No Mutation
classes.</p>
      </sec>
      <sec id="sec-5-3">
        <title>5.3. SVS Image Classification Results</title>
        <p>The classification performance of our model for detecting BRCA mutations in histopathological SVS
images is summarized in Table 5. The model achieved strong metrics for both the BRCA Mutation
and No Mutation classes. Specifically, precision, recall, and F1-score for both classes were balanced,
indicating robust classification results.</p>
        <p>(a) Confusion Matrix - VGG Model
(b) Confusion Matrix - ResNet Model
(c) Confusion Matrix - EficientNet Model
(d) Confusion Matrix - InceptionV3 Model</p>
        <p>As shown in Table 6, the model achieved an overall accuracy of 95%, with an AUC score of 93.27%.
The high precision and recall for both classes demonstrate the efectiveness of our approach in detecting
BRCA mutations, reducing the risk of false positives and false negatives.</p>
      </sec>
      <sec id="sec-5-4">
        <title>5.4. Comparison with Related Work</title>
        <p>Table 7 presents a comparison of our method with related work in the field of BRCA mutation detection
from histopathological images. Our approach, combining VQVAE and DBSCAN with InceptionV3,
outperformed previous studies, achieving the highest AUC of 93.27%.</p>
        <p>Our approach ofers several distinct advantages over other methods for the detection of BRCA
mutations, primarily due to the integration of advanced unsupervised learning and clustering techniques.
Using the VQVAE model, we eficiently encode high-dimensional histopathological images into a
compact latent space, allowing the extraction of critical features while preserving key image details.
Unlike conventional methods, which may struggle to capture subtle variations in tissue morphology, our
model’s ability to reconstruct intricate patterns enhances the detection of relevant features. Moreover,
the incorporation of DBSCAN for clustering within this latent space adds a significant layer of robustness,
efectively grouping similar patterns, and reducing noise. This method ensures that irrelevant or noisy
data are filtered out, improving classification accuracy.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>6. Conclusion</title>
      <p>In this study, we present a pioneering deep learning framework designed to predict mutations of
the BRCA gene in breast cancer utilizing histopathological images. By integrating Vector
QuantizedVariational Autoencoders (VQ-VAE) for efective feature extraction and employing DBSCAN for
clustering, we have established a robust model that demonstrates superior accuracy in classifying cases
as BRCA mutation-positive or negative. This innovative approach surpasses conventional methods
and highlights the potential of artificial intelligence to automate complex diagnostic processes within
medical imaging.</p>
      <p>In perspective, our future work will focus on the improvement of data enhancement techniques to
further enhance the accuracy of the model in the detection of BRCA mutations. By generating synthetic
samples that capture the variability in the expression of the BRCA mutation, our aim is to improve the
robustness and generalization of our model. This will be particularly valuable for addressing imbalances
in the dataset and improving the classification of rare mutation cases. In addition, our exploration will
extend to investigating the roles of other genetic mutations in breast cancer.</p>
    </sec>
    <sec id="sec-7">
      <title>Declaration on Generative AI</title>
      <p>During the preparation of this work, the authors used ChatGPT for grammar and spelling checks, as
well as paraphrasing. After utilizing this tool, the authors reviewed and edited the content as necessary,
taking full responsibility for the final publication.
[4] R. A. Dar, M. Rasool, A. Assad, et al., Breast cancer detection using deep learning: Datasets,
methods, and challenges ahead, Computers in biology and medicine 149 (2022) 106073.
[5] N. Petrucelli, M. B. Daly, T. Pal, Brca1-and brca2-associated hereditary breast and ovarian cancer
(2022).
[6] A. Hodgson, G. Turashvili, Pathology of hereditary breast and ovarian cancer, Frontiers in</p>
      <p>Oncology 10 (2020) 531790.
[7] V. Talwar, A. Rauthan, Brca mutations: implications of genetic testing in ovarian cancer, Indian</p>
      <p>Journal of Cancer 59 (2022) S56–S67.
[8] L. F. Madrigal, M. Y. R. Garcés, F. J. J. Ruiz, Impact of non-brca genes in the indication of
riskreducing surgery in hereditary breast and ovarian cancer syndrome (hboc), Current Problems in
Cancer 47 (2023) 101008.
[9] S. Zhao, C.-Y. Yan, H. Lv, J.-C. Yang, C. You, Z.-A. Li, D. Ma, Y. Xiao, J. Hu, W.-T. Yang, et al., Deep
learning framework for comprehensive molecular and prognostic stratifications of triple-negative
breast cancer, Fundamental Research (2022).
[10] X. Wang, C. Zou, Y. Zhang, X. Li, C. Wang, F. Ke, J. Chen, W. Wang, D. Wang, X. Xu, et al.,
Prediction of brca gene mutation in breast cancer based on deep learning and histopathology
images, Frontiers in Genetics 12 (2021) 661109.
[11] T. Lazard, G. Bataillon, P. Naylor, T. Popova, F.-C. Bidard, D. Stoppa-Lyonnet, M.-H. Stern, E.
Decencière, T. Walter, A. Vincent-Salomon, Deep learning identifies morphological patterns of
homologous recombination deficiency in luminal breast cancers from whole slide images, Cell
Reports Medicine 3 (2022).
[12] N. N. Phan, C.-C. Huang, L.-M. Tseng, E. Y. Chuang, Predicting breast cancer gene expression
signature by applying deep convolutional neural networks from unannotated pathological images,
Frontiers in oncology 11 (2021) 769447.
[13] N. C. Kurian, S. Varsha, A. Patil, S. Khade, A. Sethi, Robust semi-supervised learning for
histopathology images through self-supervision guided out-of-distribution scoring, in: 2023 IEEE 23rd
International Conference on Bioinformatics and Bioengineering (BIBE), IEEE, 2023, pp. 121–128.
[14] R. Valieris, L. Amaro, C. A. B. d. T. Osório, A. P. Bueno, R. A. Rosales Mitrowsky, D. M. Carraro,
D. N. Nunes, E. Dias-Neto, I. T. d. Silva, Deep learning predicts underlying features on pathology
images with therapeutic relevance for breast and gastric cancer, Cancers 12 (2020) 3687.
[15] A. van den Oord, O. Vinyals, K. Kavukcuoglu, Neural discrete representation learning, CoRR
abs/1711.00937 (2017). URL: http://arxiv.org/abs/1711.00937. arXiv:1711.00937.
[16] A. Van Den Oord, O. Vinyals, et al., Neural discrete representation learning, Advances in neural
information processing systems 30 (2017).
[17] A. Thennavan, F. Beca, Y. Xia, S. Garcia-Recio, K. Allison, L. C. Collins, M. T. Gary, Y.-Y. Chen,
S. J. Schnitt, K. A. Hoadley, et al., Molecular analysis of tcga breast cancer histologic types, Cell
genomics 1 (2021).
[18] M. Ester, H.-P. Kriegel, J. Sander, X. Xu, et al., A density-based algorithm for discovering clusters
in large spatial databases with noise, in: kdd, volume 96, 1996, pp. 226–231.
[19] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition,
arXiv preprint arXiv:1409.1556 (2014).
[20] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Conference on
computer vision and pattern recognition, 2016, pp. 770–778.
[21] M. Tan, Q. Le, Eficientnet: Rethinking model scaling for convolutional neural networks, in:</p>
      <p>International conference on machine learning, PMLR, 2019, pp. 6105–6114.
[22] C. Szegedy, V. Vanhoucke, S. Iofe, J. Shlens, Z. Wojna, Rethinking the inception architecture
for computer vision, in: Proceedings of the IEEE conference on computer vision and pattern
recognition, 2016, pp. 2818–2826.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>F.</given-names>
            <surname>Touazi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Gaceb</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Chirane</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Herzallah</surname>
          </string-name>
          ,
          <article-title>Two-stage approach for semantic image segmentation of breast cancer: Deep learning and mass detection in mammographic images</article-title>
          ., in: IDDM,
          <year>2023</year>
          , pp.
          <fpage>62</fpage>
          -
          <lpage>76</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>M.</given-names>
            <surname>Khaled</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Touazi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Gaceb</surname>
          </string-name>
          ,
          <article-title>Improving breast cancer diagnosis in mammograms with progressive transfer learning and ensemble deep learning</article-title>
          ,
          <source>Arabian Journal for Science and Engineering</source>
          (
          <year>2024</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>F.</given-names>
            <surname>Touazi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Gaceb</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Boudissa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Assas</surname>
          </string-name>
          ,
          <article-title>Enhancing breast mass cancer detection through hybrid vit-based image segmentation model</article-title>
          ,
          <source>in: The 6th Conference on Computing Systems and Applications</source>
          , Algiers, Algeria,
          <year>2024</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>10</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>