=Paper=
{{Paper
|id=Vol-2930/paper17
|storemode=property
|title=Comparison of Al-Based Approaches for Statistical Downscaling of Surface Wind Fields in the North Atlantic
|pdfUrl=https://ceur-ws.org/Vol-2930/paper17.pdf
|volume=Vol-2930
|authors=Vadim Rezvov,Mikhail Krinitskiy,Alexander Gavrikov,Sergey Gulev
}}
==Comparison of Al-Based Approaches for Statistical Downscaling of Surface Wind Fields in the North Atlantic==
https://ceur-ws.org/Vol-2930/paper17.pdf
Comparison of AI-Based Approaches for Statistical Downscaling
of Surface Wind Fields in the North Atlantic
Vadim Rezvova,b, Mikhail Krinitskiya , Alexander Gavrikova and Sergey Guleva
a
Shirshov Institute of Oceanology, Russian Academy of Sciences, Moscow, Russia
b
Moscow Institute of Physics and Technology (MIPT), Dolgoprudny, Russia
Abstract
Surface wind is one of the most important physical fields in climate research. Accurate
prediction of high-resolution near-surface winds has a wide variety of applications. Statistical
downscaling methods obtain high-resolution information about the physical quantity
distribution using available low-resolution data. They avoid high-resolution hydrodynamic
simulations that are computationally expensive. Deep learning methods are one of the typical
examples of the machine learning approaches to complex nonlinear functions approximating.
In this work, we consider statistical downscaling of near-surface wind in the North Atlantic.
For this, cubic interpolation, various architectures of convolutional networks, and generative
adversarial network are applied. Based on the results obtained, the quality of these statistical
downscaling methods is compared, and their advantages and disadvantages are identified.
Keywords 1
Statistical downscaling, neural networks, North Atlantic, near-surface wind
1. Introduction
Climate change is one of the most serious problems of the modern world. It leads to an increase in
temperature and changes in local patterns in different seasons caused by local changes in wind speed.
General circulation models are used to study the climate system and its changes on a global scale. The
results of general circulation models have low resolution and large spatial scale of the computational
cells. General circulation models are extremely computationally expensive even for low resolution
outputs. It restricts the ability to predict high-resolution climate variables [1]. The low resolution of
climate models results and the systematic errors lead to implausible predictions of future climate
scenarios, especially for extreme events [2].
The results of general circulation models can be corrected to increase their resolution, using
downscaling. Downscaling allows obtaining high-resolution information about physical quantities
based on low-resolution modeling data. Such methods can be divided into two large groups: dynamic
and statistical ones [3]. In dynamic downscaling, high-resolution numerical models are applied in sub-
domains of the area of interest, and the results of the coarser model are used as boundary conditions
for high-resolution modeling in the sub-domains [4-5]. This approach significantly reduces
computational costs, since high-resolution modeling is not carried out simultaneously in the entire
workspace. Statistical downscaling avoids high-resolution numerical simulations altogether. In this
group of methods, the values of physical quantities obtained as a result of a low-resolution numerical
simulation are inputs of a certain function. The outputs are local values of the same variables. The
functional relationship between low and high resolution data is approximated by training statistical
models on a set of data pairs.
VI International Conference Information Technologies and High-Performance Computing (ITHPC-2021),
September 14–16, 2021, Khabarovsk, Russia
EMAIL: rezvov.vyu@phystech.edu
ORCID: 0000-0003-1470-647X (A. 1); 0000-0001-5943-0695 (A. 2); 0000-0002-4198-4400 (A. 3)
- ©️ 2021 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
129
Subsynoptic and mesoscale atmospheric dynamics over the North Atlantic is considerably
interesting for understanding the mechanisms of highly localized precipitation, heat and moisture
transport. Analysis of intensity and characteristic trajectories of hurricanes in the North Atlantic is an
integral part of the quantitative assessment of the impact of highly variable atmospheric processes on
cyclonic activity and fluxes during ocean-atmosphere interaction [6]. Extremely strong turbulent
surface heat and momentum fluxes associated with the above atmospheric processes are highly
localized in space and time and require high temporal and spatial resolution for their adequate study
[7].
One of the most important questions is whether modern deep learning methods are better for
solving the problem of statistical downscaling compared to traditional statistical methods. Recent
advances in deep neural networks have led to a significant increase in statistical downscaling
techniques. At the moment, the prospect of moving away from numerical downscaling methods based
on physical equations towards statistical methods is unclear. In addition, emerging artificial neural
network architectures, including convolutional neural networks and GANs capable of solving the
downscaling problem, are trained first on standard publicly available sets of photographs and images.
Therefore, a certain gap arises in understanding how applicable the models trained on such data to
real fields of climate variables.
This paper discusses the statistical downscaling of the surface wind speed and direction in the
North Atlantic. Model exploration analyzes whether the quality of downscaling improves with
increasing depth of convolutional models or when an adversarial learning process is used. In this
work, two-dimensional cubic interpolation of the wind from a low-resolution grid to a high-resolution
grid is chosen as a reference solution for comparing various methods. Thus, the purpose of this work
is to compare the capabilities of existing approaches of statistical downscaling, which are different
degrees of complexity and depth of artificial neural networks, in relation to the fields of the surface
wind and, in general, to identify the advantages and disadvantages of neural network methods in
solving this problem.
2. Materials and methods
2.1. Initial data
Long-term atmospheric reanalysis performed using high-resolution model configurations for the
North Atlantic is provided by a retrospective dynamic model NAAD [8]. The result is atmospheric
fields with a high resolution (14 km) for the North Atlantic region. Area of modelling covers the
North Atlantic region from 10⁰N up to 80⁰N and from 90⁰W up to 5⁰E. The center of the area is
located at the point with coordinates 45⁰N, 45⁰W. The main NAAD experiment in which high-
resolution calculations were performed is HiRes. In this experiment, the work area is a regular grid of
110 X 110 nodes. The spacing between the nodes of the regular grid in HiRes is approximately 14
km. The lower near-surface level is 10–12 m above the ocean surface. In addition to the HiRes
experiment, the NAAD model also ran a moderately low resolution experiment LoRes. The
calculations in this experiment are carried out on a regular grid of 550 X 550 nodes, the distance
between which is 77 km. All experiments of the NAAD model were conducted over a 38-year period
from January 1979 to December 2016 with three-hour time resolution. Both the low-resolution input
variables and the high-resolution target variables are composed of two orthogonal horizontal
components of wind speed at 10-12 m above the surface and sea-level atmospheric pressure
2.2. Methods
The first downscaling method used in this work is cubic interpolation from a low-resolution to a
high-resolution grid. The results of all other approaches associated with the use of artificial neural
networks will be compared with the result of this method, chosen as the reference solution.
The recent publications on the use of artificial neural networks for statistical downscaling of
climate variables described the use of convolutional neural networks. Convolutional neural networks
130
are quite effective for tasks with spatially distributed data, given in the form of arrays on regular
grids, which is often used in climatology and meteorology.
The simplest artificial neural network studied in this work is a linear convolutional neural network.
The idea for this model was taken from the LinearCNN architecture proposed in [9]. Linear
convolutional network allows to associate low-resolution input with high-resolution data using only
convolutional layers without the use of non-linear activation functions.
Increasing the depth of the convolutional neural network improves the quality of prediction.
However, such an increase in the number of convolutional layers can lead to instability in the training
of the model as a result of problems that arise during optimization by the backpropagation algorithm.
Batch normalization has been proposed to stabilize the training of networks. Another effective way to
solve the problem of learning instability is to include connections allowing the output of earlier
network layers to be passed to a later stage directly, bypassing the intermediate layers of the model.
One example of such connections is residual connections. Thus, we propose to combine the
advantages of using deeper neural networks, batch normalization and residual connections. As a
result, in this paper we investigate a residual convolutional neural network based on the EnhanceNet
model [9]. Another example of a solution to the problem of learning instability in deep neural
networks is skip connections. That’s why we also investigate U-Net-based convolutional neural
network with skip connections.
Unlike all the neural network models described above, which are separate convolutional neural
networks, training adversarial networks is an adversarial process in which two models are
simultaneously trained. In adversarial network architecture, the generative model, or generator, is
opposed to its adversary, the discriminative model, or discriminator, which learns to determine
whether a sample is taken from the distribution generated by the generator or from the true
distribution of the data. In particular, if the generative and discriminative models are artificial neural
networks, then the network as a whole is called a generative adversarial network. In this case, both
models are trained using an error backpropagation algorithm. In this paper, we investigate a network
model based on SRGAN [10], which is a generative adversarial one.
2.3. Quality metrics
To measure the error between scaled and true values of climate variables at high resolution,
various loss functions are considered to highlight some aspect of the accuracy of the scaling. For
optimization purposes, in this work, spatially averaged loss functions are used, and to assess the
quality, metrics are used that take into account both average and local deviations of the model output
from the true values. To compare the various models studied in this paper in terms of the quality of
scaling, various quality metrics are considered that assess the degree of deviation between the target
values of the variables and the result of the models.
One of the most important indicators of the scaling quality is the accuracy of determining the
absolute value of the wind speed by the model. Therefore, as the simplest metric of the scaling quality
in this work, we use the root mean squared error (RMSE) of determining the wind speed. The
introduction of the RMSE-95 metric makes it possible to evaluate the scaling quality of extremely
high wind speed values.
The peak signal-to-noise ratio (PSNR) tends to infinity as the mean square error MSE (“noise”)
approaches zero. Since the goal of training the neural network is to minimize MSE, a higher PSNR
value may indicate a higher image quality. As the maximum value of the variables (“signal”)
increases, which indicates that higher values appear in the scaled data, the peak signal-to-noise ratio
also increases. An increase in the “signal”, and, consequently, an increase in PSNR, may mean a
weakening of data smoothing, which is necessary for a better visual perception of the scaled fields of
climatic variables.
3. Results
For all neural network models studied in this work, the same training and validation sets are used.
The data describing the fields of atmospheric variables are quite strongly correlated on time scales of
131
the order of several days. In addition, seasonal recurrence of atmospheric processes is observed,
leading to correlations on scales of an integer number of years [9]. To correctly take into account such
a feature of a given time series, both the training and validation samples consist of a certain number of
full consecutive years. The presence of an integer number of seasonal cycles in the training sample
also makes it possible to average seasonal variability, which can affect the learning outcome.
All models, including cubic interpolation, convolutional neural networks, and the generative
adversarial network, were trained and evaluated using the capabilities of the PyTorch machine
learning framework for the Python language. A general overview of the performance of the studied
models is presented in the Table 1. The comparison is carried out according to the number of trained
parameters of the model and memory consumption. All calculations were performed on GPU with 32
GB memory.
Table 1
Model performance comparison
Model Number of parameters Memory consumption, Mb
Linear CNN 11,328 0,01
Residual CNN 1,334,475 5,1
CNN with skip connections 72,127,620 275,2
GAN 1,361,994 (generator) 5,2 (generator)
5,215,425 (discriminator) 19,9 (discriminator)
Deeper nonlinear models consist of significant number of convolutional layers and therefore
require more memory to store the trained parameters. Thus, the overall memory consumption
increases as the model becomes more complex. The smallest number of parameters is trained in a
linear convolutional network, which obviously follows from its simplest two-layer architecture. Since
the architectures of the residual convolutional network and the generator of the generating adversarial
network are similar in depth, the number of trained parameters is practically the same. The generative
network discriminator contains 5 times more parameters than the generator. Despite the fact that the
total number of parameters of the most complex model - a convolutional network with skip
connections - is almost 11 times more than the total number of parameters in the generator and
discriminator of the generating network, the generating adversarial network shows visually much
better results.
Comparison of the values of quality metrics shown by all investigated methods in this work on the
validation set is given in the Table 2.
Table 2
Downscaling quality comparison
Method RMSE, m/s RMSE-95, m/s PSNR
Cubic interpolation 1.44 1.90 35.16
Linear CNN 2.85 5.32 27.68
Residual CNN 1.42 2.21 32.87
CNN with skip connections 1.32 1.97 34.46
GAN 1.88 3.3 33.99
Convolutional network with skip connections outperforms residual convolutional network in all
quality metrics. This is due to better downscaling over continents, which is not in line with the
purpose of this study. Despite the fact that the generative adversarial network is inferior to methods
based on convolutional neural networks in terms of quality metrics, this architecture can be
considered as the most promising of all the investigated. The generating network model is the only
one that detects the small-scale structure of wind fields. The best values of the RMSE-95 and PSNR
quality metrics are shown by the cubic interpolation method. Convolutional network with added skip
connections is the best method according to the RMSE metric.
132
Despite the fact that the generative adversarial network does not outperform the other methods,
including the reference solution, in any of the selected quality metrics, it can be considered as
showing the most encouraging results. Simultaneous training of the generator and the discriminator
changes the training direction of the generator so that it learns to repeat the small-scale wind pattern
over the ocean without overfitting over the continent (Figure 1).
(a) (b) (c)
(d) (e)
Figure 1: Difference between downscaled and true wind speed (m/s), 00:00, 1 January 2010:
(a) Cubic interpolation; (b) Linear CNN; (c) Residual CNN; (d) CNN with skip connections; (e)
Generative adversarial network.
4. Discussion
This study does not imply that neural network downscaling methods will be used directly for
operational prediction based on data on a coarse grid. The results have shown that in terms of the
spatial resolution of the downscaling models, the resulting data are not competitive in comparison
with the existing dynamic methods. Nevertheless, some use of the generative adversarial network in
the downscaling of climatic variables can be considered a promising basis for the further development
of statistical scaling methods. One of the problems that needs to be solved is the increase in the
number of predictors for training. Perhaps datasets with more climate variables and static predictors
will allow the model to train more efficiently.
There are many applications for statistical downscaling techniques that require more accurate local
wind speed predictions. These include renewable energy, local distribution of air pollutants, water
transport, sailing, etc. In cases where average speed predictions are important, as for renewable
energy sources, computationally cheap neural network scaling methods can be most widely used. For
other needs requiring accurate extreme wind speed values, additional research will be necessary. For
example, the training time of the models may be insufficient, and the scaled predictions, as the results
of this study show, may be too smooth.
In conclusion, it should be noted that the emergence of new methods for solving the problem of
scaling the wind speed increases the scope of the forecasts obtained in this way, which is especially
important for regions with complex topography. In turn, further research of neural network methods
will improve their quality, expanding their application in addition to numerical weather forecasting
models.
133
5. Acknowledgements
This work was undertaken with financial support by the Russian Science Foundation grant № 17-
77-20112-P.
The studies were carried out using the resources of the Center for Shared Use of Scientific
Equipment "Center for Processing and Storage of Scientific Data of the Far Eastern Branch of the
Russian Academy of Sciences", funded by the Russian Federation represented by the Ministry of
Science and Higher Education of the Russian Federation under project No. 075-15-2021-663.
6. References
[1] Hersbach, Hans, et al. "The ERA5 global reanalysis." Quarterly Journal of the Royal
Meteorological Society 146.730 (2020): 1999-2049.
[2] Maraun, Douglas, et al. "Towards process-informed bias correction of climate change
simulations." Nature Climate Change 7.11 (2017): 764-773.
[3] Hewitson, Bruce C., and Robert George Crane. "Climate downscaling: techniques and
application." Climate Research 7.2 (1996): 85-95.
[4] Rummukainen, Markku. "State‐of‐the‐art with regional climate models." Wiley Interdisciplinary
Reviews: Climate Change 1.1 (2010): 82-96.
[5] Xue, Yongkang, et al. "A review on regional dynamical downscaling in intraseasonal to seasonal
simulation/prediction and major factors that affect downscaling ability." Atmospheric research
147 (2014): 68-85.
[6] Tilinina, Natalia, Alexander Gavrikov, and Sergey K. Gulev. "Association of the North Atlantic
surface turbulent heat fluxes with midlatitude cyclones." Monthly Weather Review 146.11
(2018): 3691-3715.
[7] Gulev, Sergey K., and Konstantin Belyaev. "Probability distribution characteristics for surface
air–sea turbulent heat fluxes over the global ocean." Journal of Climate 25.1 (2012): 184-206.
[8] Gavrikov, Alexander, et al. "RAS-NAAD: 40-yr High-Resolution North Atlantic atmospheric
hindcast for multipurpose applications (new dataset for the regional mesoscale studies in the
atmosphere and the ocean)." Journal of Applied Meteorology and Climatology 59.5 (2020): 793-
817.
[9] Höhlein, Kevin, et al. "A comparative study of convolutional neural network models for wind
field downscaling." Meteorological Applications 27.6 (2020): e1961.
[10] Ledig, Christian, et al. "Photo-realistic single image super-resolution using a generative
adversarial network." Proceedings of the IEEE conference on computer vision and pattern
recognition. 2017.
134