Thе study examines how different activation functions influence the performance of a U-Net model applied to binary water-body segmentation in Sentinel-2 imagery. Using an identical training setup for each experiment, six nonlinearities-ReLU, Leaky ReLU, ELU, PReLU, Swish and RReLU-are individually substituted into the network while all other parameters remain fixed. Comparative evaluation on a held-out validation set reveals that Leaky ReLU provides the most balanced trade-off between precision and recall, making it the preferred choice for accurate water-mask generation. PReLU offers a similar but slightly lower performance, whereas ELU excels at capturing additional water pixels at the cost of more false positives. The findings highlight the importance of activation-function selection in remote-sensing segmentation tasks and suggest further exploration of advanced nonlinearities and larger, more diverse datasets to enhance generalization.
Monitoring water resources, including rivers, lakes, and coastal areas, plays a crucial role in modern research related to sustainable environmental management, agrotechnology, and ecology. Thanks to an extensive satellite network, particularly the Sentinel-2 program, it is now possible to obtain highresolution multispectral imagery for the regular assessment of water bodies. However, the task of automatically and accurately separating water from land (segmentation) remains challenging due to factors such as water turbidity, seasonal variability, cloud coverage, and the spectral similarity of various landscape elements.
Traditional algorithms based on indices such as the Normalized Difference Water Index (NDWI)
offer fast solutions, but they are often vulnerable to complex environmental conditions. With advances
in deep learning within the field of computer vision, there is a growing trend towards the use of deep
convolutional neural networks (CNNs), which enable more accurate identification of image features
[
Nonetheless, the effectiveness of CNN training — including that of U-Net — depends not only on architectural design but also on the choice of activation functions, which define how neurons respond to incoming signals. The ReLU (Rectified Linear Unit) family remains the most used due to its simplicity and immunity to the vanishing gradient problem in the positive domain. However, numerous modifications of ReLU (e.g., Leaky ReLU, PReLU, RReLU) as well as alternative functions (ELU, Swish, Mish) have been proposed to improve convergence and address the limitations of standard ReLU.
This study presents a comparative analysis of six activation functions (ReLU, Leaky ReLU, ELU, PReLU, Swish, RReLU) in the context of binary segmentation of water bodies using Sentinel-2 satellite imagery. To ensure objective evaluation, all training parameters (number of epochs, dataset) were fixed so that the only changing variable was the activation function. The evaluation criteria included F1 score, Intersection over Union (IoU), precision, recall, and convergence rate. The results provide insights into which activation function contributes most effectively to accurate and reliable water segmentation under diverse landscapes and imaging conditions.
This study employs a modified U-Net architecture specifically designed for the binary segmentation of water bodies based on Sentinel-2 satellite data. The choice of U-Net is justified by its ability to integrate high-level (global) features with fine-grained local details—an essential quality for detecting small water bodies and complex shoreline structures. To enhance performance, the input data undergoes preprocessing, including the generation of training patches from satellite scenes and the creation of corresponding target masks. This enables the model to operate effectively on multispectral imagery by adjusting the number of input channels and scale according to the available spectral bands.
During model development, the unique characteristics of Sentinel-2 imagery are considered such as the varying spatial resolution of individual bands and uneven surface illumination [23]. Each image is normalized and divided into fixed-size patches, which simplifies the training process and reduces the need to store large full-resolution intermediate results. Each patch is input into the U-Net as a tensor, typically with 3 or 4 channels (RGB or RGB plus near-infrared). In this project, 512×512 sized patches are used, striking a balance between computational cost and spatial detail preservation.
The architecture retains the classic encoder-decoder structure with convolutional and transposed
convolutional blocks. The encoder progressively reduces spatial resolution while extracting
increasingly abstract features, whereas the decoder reconstructs the original image dimensions and
focuses on the accurate localization of segmented objects. Skip connections between corresponding
levels of the encoder and decoder help retain crucial fine-grained information that would otherwise be
lost—essential for delineating the boundaries of water bodies [
The advantages of U-Net in this project are further supported by multiple studies demonstrating its
effectiveness for remote sensing data [
The research begins with the selection of Sentinel-2 satellite image that contains the relevant spectral bands for distinguishing between water and land. The imageis normalized to minimize differences in illumination and acquisition conditions.
The fragment (5376x5376 pixels) of the original satellite image was selected for training dataset. The image is then divided into fixed-size patches of 512×512 pixels to simplify the training process and optimize computational efficiency. The total number of patches used for training is 110.
Next, binary masks are generated to indicate the presence of water at the pixel level. Some of these masks are refined manually, while others are derived from spectral indices and later validated for labeling errors. This approach enables the creation of a robust training and validation dataset with a balanced representation of water and non-water regions. A baseline U-Net model is used for segmentation, with the number of input channels tailored to the selected spectral bands. Batch size, learning rate, and other hyperparameters remain constant across all experiments to ensure fair comparison among activation functions.
The encoder-decoder structure, along with max-pooling and transposed convolution operations, allows the network to preserve local details while reconstructing spatial features of the input image.
For each activation function under consideration (ReLU, Leaky ReLU, ELU, PReLU, Swish, RReLU), a separate variant of the model is implemented in which only the activation layers are modified. Aside from the activation function, all other components—including the dataset and training duration— remain unchanged.
Upon completion of training, the models are evaluated on a validation set using F1, IoU, precision, recall, and convergence metrics.
The predicted outputs are stitched into complete segmentation maps, enabling both visual and quantitative assessment of water body detection quality. Finally, a comparative analysis of all six activation functions is performed to identify the most effective one for binary water segmentation from Sentinel-2 imagery.
The Rectified Linear Unit (ReLU) is one of the most used activation functions in modern deep
convolutional networks [
ReLU was introduced to mitigate the vanishing gradient problem often encountered with sigmoid
or tanh activations [
Its main advantages include computational simplicity and the ability to maintain non-zero gradients when x > 0, which facilitates effective optimization in deep architectures [12]. Additionally, the lack of saturation for positive inputs allows neurons to output arbitrarily large values, assuming supportive data and weights. However, ReLU has a significant drawback in the form of "dead neurons"—units that output zero across all inputs if they remain in the negative region during training [8]. Despite this, ReLU continues to demonstrate reliable performance in high-resolution image segmentation, including satellite imagery [13].
In this study, ReLU serves as the baseline activation function. The U-Net model with ReLU is used
as a reference to evaluate the performance gains achieved by its alternatives (Leaky ReLU, ELU, etc.). It
remains a widely adopted standard due to its proven efficacy in segmentation, classification, and
various deep learning tasks [
The performance of Leaky ReLU is sensitive to the choice of α. A very small value makes it behave like ReLU, whereas a large value may weaken its ability to discriminate between signal polarities [15]. In practice, α is often chosen between 0.01 and 0.1 to balance learning speed and neuron activity, especially in vision tasks and satellite image segmentation [16].
In water segmentation tasks, Leaky ReLU may enhance model adaptability in regions with high spectral variability, such as vegetated shorelines or partially flooded zones. This study examines whether Leaky ReLU enables the network to retain more informative neurons and achieve superior segmentation performance compared to baseline ReLU [10]. ELU The Exponential Linear Unit (ELU) was introduced to accelerate convergence and reduce bias shift during training. It is defined as:
, ≥ 0 (3)
( ) = α( − ), < 0
where α is a positive parameter, typically set to 1 [17]. Unlike ReLU, ELU produces smooth negative
outputs rather than hard zeros, preserving gradients in the negative region [14]. For x ≥ 0, it behaves
similarly to ReLU, ensuring simple optimization and avoiding saturation [
ELU also helps to center activation values around zero, which can facilitate learning and reduce reliance on normalization techniques [18]. However, it incurs higher computational costs due to the exponential term and may generate large negative outputs that destabilize training in some cases [19].
In this study, ELU is evaluated in contexts where nuanced control over negative inputs is beneficial—for instance, near noisy land-water transitions. The goal is to assess whether ELU can accelerate learning and improve segmentation metrics compared to ReLU and Leaky ReLU.
Parametric ReLU (PReLU) generalizes Leaky ReLU by learning the coefficient α during training rather than fixing it manually [26]. It is defined as:
, > 0 (4) ( ) = , ≤ 0
Here, α is initialized to a small positive value and optimized alongside other network parameters [8]. This adaptability allows the model to fine-tune the "leakiness" for each channel or neuron [14].
The main advantage of PReLU is its ability to dynamically adjust the negative slope to the data distribution, potentially improving accuracy [12]. However, it increases the number of parameters, necessitating stronger regularization. In water segmentation, PReLU may prove useful in cases where land-water boundaries are highly variable and require distinct sensitivity across channels [10]. depending on the input magnitude [21]. This prevents neuron inactivation and may allow for better feature representation.
Swish avoids abrupt changes around x = 0, resulting in smoother gradients [
In this study, Swish is considered a promising alternative for scenarios where water-land boundaries are fuzzy or ill-defined. Its effectiveness, however, remains dependent on dataset size and training conditions [11]. [20]. It is defined as: Swish is considered a smoother alternative to ReLU that can enhance gradient flow in deep networks ( ) = , , ≥ 0 < 0 , ∈ [ , (5) (6) specified range during training [14]. Formally: where αᵣis a random value. This randomness can act as a regularizer, helping the model avoid overfitting or over-reliance on specific activation patterns [8]. However, it may also slow convergence if the variation range is too broad [16].
RReLU is potentially beneficial for heterogeneous datasets with varied conditions (e.g., seasonal differences, diverse lighting). This study explores whether its built-in variability leads to more generalized segmentation performance when compared to deterministic counterparts like Leaky ReLU and PReLU.
The table below summarizes the comparative performance of all six activation functions based on four key evaluation metrics: F1 score, Precision, Recall, and Intersection over Union (IoU). Among the tested functions, Leaky ReLU achieved the highest F1 score (0.7386), along with the best precision (0.8395) and overall IoU (0.5856). PReLU ranked second in terms of F1 score (0.7253), showing a balanced performance with Precision of 0.7712 and Recall of 0.6845.
While ELU reached the highest Recall (0.7286), it suffered from low precision (0.5293), which led to the lowest overall F1 (0.6132) and IoU (0.4421) scores. ReLU and Swish produced similar mid-range results, with ReLU slightly outperforming Swish in Recall (0.7067 vs. 0.7028).
RReLU demonstrated relatively high precision (0.8261), exceeding that of PReLU, but its lower recall (0.6273) placed its F1 score (0.7131) and IoU (0.5541) between those of baseline ReLU and the topperforming Leaky ReLU. Thus, if the primary goal is to maximize F1 or IoU, Leaky ReLU is the most optimal. If recall is prioritized—for instance, to reduce false negatives in water detection—ELU may be considered, albeit at the cost of lower precision.
The results of this comparative experiment confirm that the choice of activation function has a substantial impact on the performance of the U-Net model for binary segmentation of water bodies in Sentinel-2 satellite imagery. Leaky ReLU demonstrated the best overall results, achieving the highest values for F1 score and IoU while maintaining the strongest precision. PReLU followed closely, offering a balanced trade-off between precision and recall, though it still Leaky ReLU in all major metrics. ELU stood out by achieving the highest recall, but this came at the expense of significant precision loss, resulting in the lowest F1 and IoU scores. The standard ReLU and Swish functions delivered average performance with no significant advantages over the more adaptive alternatives. RReLU offered high precision but somewhat reduced recall, placing its overall results between those of ReLU and Leaky ReLU.
In conclusion, for the segmentation of water surfaces from Sentinel-2 imagery, Leaky ReLU is the most effective activation function, offering the best balance between accuracy, completeness, and spatial consistency. In scenarios where maximizing recall is critical—such as minimizing omission of water pixels—ELU may be considered, albeit with a higher risk of false positives. To further enhance segmentation quality, future work should include expanding the training dataset with diverse geographic regions, optimizing activation-related hyperparameters, and exploring newer functions such as Mish or SELU, particularly in the context of multispectral Sentinel-2 data.
During the preparation of this work, the author(s) usedGPT-4o in order to: Grammar and spelling check. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content.