Unsupervised domain adaptation (UDA) has emerged as a promising approach to address domain shifts in remote-sensing scene classification. The acquisition of labelled data from diverse geographic, temporal, and sensor domains often presents significant challenges, rendering the UDA an essential tool for real-world applications. Traditional UDA methodologies typically focus on single-source domains. However, real-world scenarios frequently involve multi-source domains with diverse distributions, which introduce additional challenges such as inter-source discrepancy, label noise, class imbalance and explainability. To address these challenges, an explainable multi-source UDA framework is proposed which integrates feature extraction through contrastive-learning with an adaptive clustering-based pseudo-labeling named as XMUDA-CLAC. The pseudo-label generation process is further refined through a multi-objective optimization approach. To enhance transparency and interpretability, Explainable Artificial Intelligence (XAI) methodologies are employed to visualize the attention maps generated by contrastive learning-based Vision Transformer (ViT). The proposed XMUDA-CLAC framework is assessed using four benchmark remote sensing datasets-AID (A), NWPU-RESISC45 (N), PatternNet (P), and UC Merced (U)-under various domain-shift scenarios: (A → U), (P → N), (U → P), (A, P → U), (A, N → U), (P, U → N), (A, P, N → U), and (A, U, P → N). In this context, the proposed method demonstrates absolute accuracy improvements of 0.62%, 0.94%, 0.73%, 0.41%, 0.51%, 1.66%, 0.20%, and 0.10% over the best-performing baselines, respectively. This illustrates the eficacy of the framework in scenarios involving multi-source domain adaptation with better interpretability. The source code is available at https://github.com/BinuJoseA/XMUDA-CLAC.
The UDA-based methods become an essential approach for mitigating the issue of performance
degradation caused by domain shifts [
In the field of UDA, most existing approaches are designed for single-source domains. Nevertheless,
real-world scene classification frequently necessitates multi-source UDA (MUDA), where data are
derived from various domains with unique distributions [
Recent advancements in UDA, particularly those incorporating contrastive learning and Vision
Transformers (ViTs) [
To address these challenges, the proposed XMUDA-CLAC framework integrates contrastive learningbased feature extraction, adaptive incremental clustering, pseudo-label generation and class-aware pseudo-label refinement through a multi-objective optimization technique. This cohesive design not only enhances domain alignment and pseudo-label quality but also improves model interpretability through attention visualization. By uniting these components, our approach ofers a robust, scalable, and interpretable solution for remote sensing scene classification under domain shift conditions.
The major contributions of the paper are as follows. 1. A contrastive learning-based feature extraction mechanism for acquiring domain-invariant representations in UDA. 2. An adaptive incremental clustering module designed to produce interpretable high-quality pseudolabels. 3. A multi-objective optimization strategy aimed at enhancing cluster reliability, pseudo-label consistency and domain alignment. 4. A class-aware pseudo-label refinement mechanism alongside dynamic centroid alignment to address issues of class imbalance, mode collapse, and temporal feature drift. 5. An XAI component for visualizing ViT attention maps and interpreting focused decisions, thereby augmenting transparency and trust in model predictions.
The XMUDA-CLAC framework, as depicted in Figure 1, introduces a MUDA approach for remote sensing scene classification based on substantial domain shifts. Initially, a feature encoder is pretrained using a contrastive learning model on both the source and target datasets to acquire domain-invariant representations. These features are subsequently extracted for all source domains and unlabelled target domain. A source classifier is trained on the labelled source features, and the source class centroids are calculated. An adaptive incremental density-based clustering algorithm is then employed on the target features to assign pseudo-labels by aligning the cluster centroids with the mean feature vector. Simultaneously, a domain discriminator with a gradient reversal layer (GRL) facilitates domain alignment to produce indistinguishable features across the domains. The training process is directed by four ∑︀+ [ log ( ()) + (1 − ) log(1 − ( ()))]
=1 Source Domain 1 Source Domain 2 . .
.
Source Domain N
Target Domain
Contrastive Learning (ViT
+ SimCLR)
Shared
Weights Contrastive Learning
(ViT+ SimCLR)
Source Features 1
Source Features 2
Source Features N
Target Features
No
Optimized features / Maximum no of generations ?
Yes
Source Classifier
Domain discriminator
Adaptive incremental density based clustering and pseudo labelling souce casssifier loss domain discriminator loss cluster compactness pseudo-label consistency loss
Multi-objective
Optimization Class aware pseudo-label refinement update set of source and target features
Adaptive incremental density based clustering and pseudo labelling
XAI
Interpretable scene predictions on the target domain parameters , and s (·): classifier network with parameters . objective functions: supervised classification loss on the source domain ( ℒcls), adversarial alignment loss (ℒadv), clustering compactness loss (ℒclust), and pseudo-label consistency loss on the target (ℒpl). These are collectively optimized using a multi-objective strategy, such as deep learning-based paretofront generation. Finally, interpretability is achieved through Grad-CAM, providing insights into both scene predictions and pseudo-label assignments in the target domain. The description of the objective functions are presented in Table 1. The notations used in Table 1 are : number of labelled source samples, (): -th source sample,
(): ground truth label of the -th source sample, (·) : feature extractor with parameters , (·) : classifier network with parameters , ( ()) is the domain discriminator output, = 1 if ∈ source and = 0 if ∈ target, : number of clusters, : a sample in cluster , (): Feature vector of sample from feature extractor, : is the mean feature vector of cluster ,
(): -th target sample, (): pseudo-label of the -th sample, (·) : feature extractor with The components in the proposed framework are detailed in subsequent subsections. Target Clusters Sort-wise
Top-k Filtering
Pseudo Labels
The source domains (1, 2, ..., ) comprise several labelled datasets derived from diverse remote sensing sources, each distinguished by variations in geographic location, acquisition time, and sensor type. Conversely, the target domain ( ) is an unlabelled dataset that requires domain adaptation to facilitate accurate scene classification despite distributional changes.
Images from both the source and target domains are encoded using a SimCLR-based contrastive learning framework [16], to acquire domain-invariant representations. A ViT, pretrained through contrastive learning, is subsequently employed to extract high-level semantic features that demonstrate robustness to domain shifts. These features are then input into three parallel modules: source classifier, domain discriminator, and clustering module, facilitating classification, domain alignment, and pseudo-label generation.
A multilayer perceptron (MLP) functions as the source classifier to examine and categorize various features or characteristics of the source domain. The Adversarial Domain Discrepancy Gradient Reversal Layer (ADD-GRL) [17] is employed to mitigate the domain discrepancy between the features of the source and target domains.
The adaptive incremental cluster formation with dynamic density estimation and neural network-based merging algorithm, presented as Algorithm 1. The algorithm is initiated by calculating the global average distance to inform the local parameter selection. Each incoming feature vector adaptively determines the neighbours based on the local distance distribution and its relationship to the global threshold. This process facilitates the dynamic estimation of the neighbourhood radius () and local density, from which sample-specific are derived. Subsequently, the algorithm identifies -neighbours and evaluates whether the sample qualifies as a core point. If so, the sample is either incorporated into an existing cluster, initiates a new cluster, or prompts merging when multiple clusters overlap. Merging decisions are executed using a two-stage neural network. The initial model computes a merge score by evaluating cluster proximity, density, and feature similarity. Subsequently, the second model dynamically adjusts the merging threshold based on these inputs and the computed score. Only the pairs that surpassed the threshold are merged. Samples that do not meet core criteria are initially classified as noise and are later re-evaluated during post-processing for potential cluster reassignment based on updated local densities.
The proposed framework integrates unsupervised deep learning with multi-objective optimization to generate Pareto front rankings, as detailed in our previous work [18]. To enhance the diversity and generalization within the feature space, crowding distance-based selection [19] is employed to ensure a well-distributed set of solutions, which is crucial for UDA in geo-spatial contexts. Simulated Binary Crossover (SBX) [20] is used to efectively balances exploration and exploitation by preserving linear relationships among parent solutions, thereby aiding spatial and spectral coherence. Furthermore, polynomial mutation [21] is also used to reinforce spatial and spectral consistency, aligns adapted features with inherent geo-spatial structures, and enhances cross-domain generalization.
Algorithm 1: Adaptive incremental cluster formation with dynamic density estimation and neural network-based merging
Input : Target feature set = {1, 2, . . . , }, new feature vector new, initial clusters , scaling factor 1, adaptive range constants 1, 2, 3, 4 Output : Updated cluster list updated // Precompute global statistics 1 Compute pairwise distance matrix across ; 2 Compute global distance mean = (1−1) ∑︀̸= ; 3 foreach new ∈ do // Estimate Local Distance Characteristics Let = {(new, ) | ∈ } ; if mean() ≤ then
Select ∼ Uniform( 1, 2) ;
// Noise Re-Assessment Phase 33 foreach point previously labelled as noise do 34 Recompute neighbors within local ; 35 if ||≥ MinPts then 36 Assign to the nearest valid cluster ; else
Select ∼ Uniform( 3, 4) ; Sort in ascending order ; Set = [] ; // Infer Local Density Let = { ∈ | (new, ) ≤ } ; Compute local density = || ; Compute adaptive threshold MinPts = 1 · ; // Decision: Assign or Evaluate if ||≥ MinPts then
Identify intersecting clusters near = { ∈ | ∩ ̸= ∅} ; if |near|= 0 then
Create new cluster new = {new}, add to ; else if |near|= 1 then
Append new to the matched cluster ; else foreach pair (, ) ⊆ near do
Extract features: proximity, compactness, cross-similarity ; Use trained neural model to compute: ; merge_score ← Net 1(·), ← Net 2(·) ; if merge_score ≥ then
Merge ∪ and add new ; else // Handle potential noise if none of belongs to any cluster then
Mark new as temporary noise ; else
Find the nearest neighbor ∈ ∩ ;
Assign new to cluster of ;
The class-aware adaptive pseudo-labeling algorithm, presented as Algorithm 2, ofers a robust approach to MUDA without dependence on target domain centroids. The algorithm is initiated by calculating class-specific prototypes from labelled source features. During each training cycle, soft pseudo-labels are allocated to target samples based on their similarity to these prototypes, utilizing a scaled softmax function. The reliability of pseudo-labels is assessed using cluster-wise confidence scores, which are derived from adaptive incremental density-based clustering. Only samples within high-confidence clusters are fully accepted, while others are incorporated with reduced weight. To ensure class balance, the top- most confident samples per class are selected and employed to update the source prototypes in a weighted manner, facilitating gradual adaptation to the target domain. Subsequently, prototype-based contrastive loss is computed to align pseudo-labelled target features with their corresponding prototypes. The classifier is trained using a combined loss: a standard classification loss for confident samples and a contrastive loss for alignment.
The contrastive learning model, adapted through UDA, efectively employs both Grad-CAM-based activations and global Rollout to make informed decisions regarding scene classification. Upon determining the optimal set of source and target features, the adaptive incremental density-based clustering algorithm is applied to the optimal target features to produce pseudo-labels. These pseudo-labels are then employed to further train the classifier, incorporating both the optimal source and target features.
In order to evaluate performance, four prominent datasets for remote sensing scene classification have
been chosen: AID () [22], NWPU-RESISC45 ( ) [23], PatternNet ( )[24] and UC Merced ( ) [25]. To
ensure a uniform basis for comparison, five shared classes such as Farmland, Forest, Parking, Residential
and River present in all four datasets are utilized. Experiments are conducted on an NVIDIA DGX Station
A100 equipped with an AMD EPYC 7742 64-core CPU, four NVIDIA A100 (40 GB) GPUs, and 512 GB of
DDR4 RAM. Training is conducted for 200 epochs, each comprising 100 genetic-algorithm generations.
The crossover and mutation rates are fixed at 0.82 and 0.018, respectively. The neighbor-rank parameters
1, 2, 3, and 4 are assigned the values 5, 10, 20, and 50. The density scale 1 is computed as the
standard deviation of local neighbor distances. The confidence gate 2 and the prototype-contrastive
weight proto values are in the range of [
The performance analysis of XMUDA-CLAC is conducted using the classification accuracy, average Receiver Operating Characteristic (ROC) curve, computational cost and worst-case time complexity. Table 3 presents a comparative classification accuracy analysis of several state-of-the-art UDA methods applied to remote sensing scene classification. In the (A → U) task, the proposed framework achieves an accuracy of 0.965, representing a 0.62% improvement over the next best method (0.959). In the (P → N) task, the proposed method surpasses Hy-MSDA (0.953) with an accuracy of 0.962, indicating a 0.94% increase. In the (U → P) task, the proposed method also demonstrates significant improvements, with an increase of 0.73%. Specifically, for multi-source domain tasks such as (A, P → U), (A, N → U), (P, U → N), (A, P, N → U), and (A, U, P → N), the proposed method exhibits superiority over existing methods. The findings highlight the efectiveness of the proposed method in improving accuracy across various domain adaptation tasks when compared to other leading UDA methods.
Figure 2 illustrates the ROC curves for five classifiers: support vector machine (SVM), multi-layer perceptron (MLP), XGBoost, random forest, and logistic regression. These classifiers were assessed on
Compute initial prototype (0) = |1| ∑︀∈ (); // Iteratively Update Pseudo-Labels and Source Prototypes 3 for each training epoch do
// Assign Soft Pseudo-Labels with Class-Wise Top- Filtering 4 Initialize [] = ∅ for each class ;
// Pseudo-label generation 5 for each target sample ∈ do 6 Identify cluster of from ; 7 Compute soft pseudo-label probabilities ( = ) using: ( = ) =
(−1) )/ ) exp(sim( (),
(−1) )/ ) ∑︀=1 exp(sim( (), else if ≥ then
Add (, ( = * ), (), weight = 1.0) to [* ]; Let * be pseudo-label; * = arg max ( = ); // Class-aware pseudo-label refinement Compute cluster-level confidence using intra-cluster similarity or density; Compute threshold = 2 · mean( );
Add (, ( = * ), (), weight = 0.5) to [* ]; // Top-k Selection for each class do
Sort [] by confidence and retain top- samples;
Compute updated prototype () = ∑︀(,∑︀)·( ) from top- []; // Update Source Prototypes from Top-k Pseudo-Labelled Target Samples for each class do // Compute Contrastive Loss for each pseudo-labelled sample (, ()) do
Compute: ℒproto() = − log
())/ ) exp(sim( (), *
())/ ) ∑︀=1 exp(sim( (), Total loss: ℒtotal = ℒcls + proto · ℒ proto; Update network parameters using ℒtotal; // Classifier Training
Compute cross-entropy loss ℒcls over confident samples; 25 return Refined pseudo-labels * ; domain adaptation tasks (A, P, N → U) and (A, U, P → N). Each curve depicts the mean performance across all five scene classes. The Area Under the Curve (AUC) scores indicate that the MLP classifier achieves the highest performance, with AUC values of 0.99 and 0.98 for the respective tasks.
Table 4 presents a comparative computational cost analysis using parameter count, GFLOPs, model size, and training time for (A, U, P → N) across various state-of-the-art models. Although the XMUDACLAC achieves significant classification accuracy utilizing ViT+SimCLR, the slight increase in training time is negligible, which ofers scalable solution compared to some of the existing benchmark models.
The comparative time complexity analysis of XMUDA-CLAC with some of the state-of-art approaches are detailed in Table 5. The notations used in Table 5 are : number of source images, : number of target images, : feature width, : number of classes, : number of target clusters, , , are the epochs for SimCLR pretrain, source classifier, adversarial alignment, : number of generations, : population size , : number of objectives, : ViT per sample cost, : classifier cost and : discriminator cost, : number of source domains, : training epochs, fwd: per-sample cost of a forward pass, bwd: per-sample cost of one forward+backward pass, : number of epochs for the initial source-only pretraining, : number of outer training epochs, and : mini-batch size. From the table, it is observed that the time complexity of the proposed approach is at par with existing models, while performing well with respect to classification accuracy.
The explainability of the XMUDA-CLAC are performed using Grad-CAM and Attention Rollout methods and are presented in Figures 3 and 4. Figure 3 depicts the alteration in attention distribution within a contrastive learning-based ViT model, observed before and after domain adaptation on a target image from NWPU-RESISC45 dataset. Figure 3a shows the original river scenario. Figure 3b presents the Grad-CAM output from the source-only model, which is trained on AID, UC Merced, and PatternNet, highlighting the initial regions of attention. Figure 3c displays the attention map post-adaptation. The regions in red/yellow signify high attention, whereas blue/green regions indicate low attention. Figure 3d exhibits the Attention Change Map, where red/yellow areas highlight shifts in attention and black areas denote stable focus, underscoring the interpretability improvements resulting from adaptation. dataset utilizing a contrastive learning-based UDA model. source domains, target domain, updated centroids, and top-k filtered pseudo-labelled samples in a UDA scenario (A, U, P → N). Source samples are represented as circles, target samples as crosses, top-k samples are outlined with circular borders, and the updated centroids are depicted as black pentagons.
feature extractors and objective functions for UDA across multiple remote sensing datasets. The study examines four objective functions, utilizing feature extractors such as ResNet50, Vision Transformer (ViT), and ViT pretrained with SimCLR. Among the configurations tested, the combination of ViT with SimCLR and the full set of objective functions achieves the highest classification accuracy across all eight domain adaptation tasks, including both single-source and multi-source scenarios. Notably, it attains superior performance on tasks such as (A → U) (0.965), (A, N → U) (0.977), and (A, U, P → N) (0.978). These results highlight the eficacy of transformer-based representations enhanced by contrastive learning and the synergistic efect of multiple complementary objective functions.
This study introduces a comprehensive XMUDA-CLAC framework for remote sensing scene classification in the context of significant domain shifts. By integrating contrastive-pretrained ViTs with adaptive incremental density-based clustering, the framework efectively extracts domain-invariant features and generates high-confidence pseudo-labels for the unlabelled target domain. The robustness to class imbalance and feature drift is further enhanced through class-aware pseudo-label refinement and dynamic centroid alignment. By framing pseudo-labeling, clustering, and domain alignment as a unified multi-objective optimization problem, the framework facilitates reliable learning in the absence of target labels. Explainability is incorporated through XAI techniques, such as Grad-CAM and attention rollout, thereby improving the interpretability and trustworthiness of model predictions. Experimental results demonstrate the framework’s superior accuracy and generalization compared with state-of-the-art UDA methods. Although this approach incurs additional computational cost due to its optimization complexity, it ofers solution with better classification accuracy, scalability and interpretability. Future research endeavors will focus on developing a multi-source universal domain adaptation (DA) variant of XMUDA-CLAC, incorporating minimum-cost-flow matching and conformally calibrated energy-based mechanisms for unknown rejection.
During the preparation of this work, the author(s) used ChatGPT, Grammarly to: Grammar and spelling check. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content. [12] M. Litrico, A. Del Bue, P. Morerio, Guiding pseudo-labels with uncertainty estimation for sourcefree unsupervised domain adaptation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7640–7650. [13] T. Burgert, M. Ravanbakhsh, B. Demir, On the efects of diferent types of label noise in multi-label remote sensing image classification, IEEE Transactions on Geoscience and Remote Sensing 60 (2022) 1–13. [14] A. Abbas, M. Linardi, E. Vareille, V. Christophides, C. Paris, Towards explainable ai4eo: An explainable deep learning approach for crop type mapping using satellite images time series, in: IGARSS 2023-2023 IEEE International Geoscience and Remote Sensing Symposium, IEEE, 2023, pp. 1088–1091. [15] K. Xu, Z. Zhu, W. Wang, C. Fan, B. Wu, Z. Jia, Enhancing remote sensing scene classification with hy-msda: A hybrid cnn-transformer for multi-source domain adaptation, IEEE Transactions on Geoscience and Remote Sensing (2024). [16] T. Chen, S. Kornblith, M. Norouzi, G. Hinton, A simple framework for contrastive learning of visual representations, in: International conference on machine learning, PmLR, 2020, pp. 1597–1607. [17] E. Tzeng, J. Hofman, K. Saenko, T. Darrell, Adversarial discriminative domain adaptation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 7167– 7176. [18] B. Jose A, P. Das, An automated incremental density-based clustering approach using unsupervised deep learning and multi-objective optimization, Computers and Electrical Engineering 123 (2025) 110109. [19] D. Feng, Y. Li, J. Liu, Y. Liu, A particle swarm optimization algorithm based on modified crowding distance for multimodal multi-objective problems, Applied Soft Computing 152 (2024) 111280. [20] L. Pan, W. Xu, L. Li, C. He, R. Cheng, Adaptive simulated binary crossover for rotated multiobjective optimization, Swarm and Evolutionary Computation 60 (2021) 100759. [21] J. L. Carles-Bou, S. F. Galán, Self-adaptive polynomial mutation in nsga-ii, Soft Computing 27 (2023) 17711–17727. [22] G.-S. Xia, J. Hu, F. Hu, B. Shi, X. Bai, Y. Zhong, L. Zhang, X. Lu, Aid: A benchmark data set for performance evaluation of aerial scene classification, IEEE Transactions on Geoscience and Remote Sensing 55 (2017) 3965–3981. [23] G. Cheng, J. Han, X. Lu, Remote sensing image scene classification: Benchmark and state of the art, Proceedings of the IEEE 105 (2017) 1865–1883. [24] W. Zhou, S. Newsam, C. Li, Z. Shao, Patternnet: A benchmark dataset for performance evaluation of remote sensing image retrieval, ISPRS journal of photogrammetry and remote sensing 145 (2018) 197–209. [25] Y. Yang, S. Newsam, Bag-of-visual-words and spatial extensions for land-use classification, in: Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems, 2010, pp. 270–279. [26] Y. Wei, L. Yang, Y. Han, Q. Hu, Multi-source collaborative contrastive learning for decentralized domain adaptation, IEEE Transactions on Circuits and Systems for Video Technology 33 (2022) 2202–2216. [27] C.-X. Ren, Y.-H. Liu, X.-W. Zhang, K.-K. Huang, Multi-source unsupervised domain adaptation via pseudo target domain, IEEE Transactions on Image Processing 31 (2022) 2122–2135. [28] M. Li, C. Zhang, W. Zhao, W. Zhou, Cross-domain urban land use classification via scenewise unsupervised multisource domain adaptation with transformer, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 17 (2024) 10051–10066. [29] S. Chen, L. Zheng, H. Wu, Riemannian representation learning for multi-source domain adaptation, Pattern Recognition 137 (2023) 109271.