=Paper=
{{Paper
|id=Vol-1536/paper11
|storemode=property
|title=
Information-computational System for Online Analysis of Georeferenced Climatological Data
|pdfUrl=https://ceur-ws.org/Vol-1536/paper11.pdf
|volume=Vol-1536
|dblpUrl=https://dblp.org/rec/conf/rcdl/OkladnikovGTS15
}}
==
Information-computational System for Online Analysis of Georeferenced Climatological Data
==
https://ceur-ws.org/Vol-1536/paper11.pdf
Information-computational system for online analysis of
georeferenced climatological data
© I.G. Okladnikov © E.P. Gordov © A.G. Titov © T.M.Shulgina
Institute of Monitoring of Climatic and Ecological Systems SB RAS, Tomsk
Tomsk branch of Institute of Computational Technologies SB RAS, Tomsk
Tomsk State University, Tomsk
oig@scert.ru gordov@scert.ru titov@scert.ru stm@scert.ru
Abstract actively used in numerous applications including
modeling, interpretation and forecast of climatic and
An Internet accessible information-computational ecosystem changes for various spatial and temporal
system (ICS) is dedicated to visualization and analysis scales [9]. Due to inherent heterogeneity of
of geospatial gridded datasets in the area of Earth environmental datasets as well as their huge size which
system science (local observations, remote sensing and might constitute up to tens terabytes for a single dataset,
modeling) and climatology in particular. This online at present, studies in the area of climate and
ICS is based on a dedicated software framework environmental change require a special software
consisting of three key parts: server-side computational support [5]. A dedicated online information-
back-end, server-side middleware represented by computational system for analysis of georeferenced
Geoserver software and PHP controllers, and JavaScript climatological and meteorological data is presented in
library for building typical components of web mapping this work. It is based on Open Geospatial Consortium
client GUI developed using AJAX techniques. Huge (OGC) standards and involves a number of modern
multidimensional geospatial datasets are stored in a solutions such as object-oriented programming model,
Network Common Data Form (NetCDF) and processed modular composition, and JavaScript components based
by a set of validated software modules included in the on GeoExt library [12], ExtJS Framework [23] and
back-end. Results of a processing are represented by OpenLayers software [15]. This system is being
overlapped raster and vector layers placed over a developed in the framework of a big research project
georeferenced background. Data processing aimed to creation of a distributed hardware and
functionality allows performing basic and complex software platform for monitoring and forecasting of
statistical analysis of climatological data whilst online regional climatic and ecological changes and supporting
geo-information system (GIS) instruments give a user of continuous education. And the work is still not
an ability to combine and place georeferenced results finished.
over a chosen cartographical basis. It can provide
specialists involved into multidisciplinary research 2 Related work
projects, and even users without programming skills,
with reliable and practical online instruments for At present, only few web-based information systems
integrated research of climate and ecosystems changes devoted to handling of geophysical data are known:
through a unified web-interface. GIOVANNI – GES-DISC (Goddard Earth Sciences
Data and Information Services Center) Interactive
1 Introduction Online Visualization ANd aNalysis Infrastructure ([1],
[8]); dissemination system based on Coupled Climate
Specifics and regional environmental applications of Model of Institute for Numerical Mathematics of the
basic Earth system sciences make them RAS (INM RAS, [3]); Climate explorer by Royal
multidisciplinary, involving into studies a number of Netherlands Meteorological Institute (KNMI, [2]);
nationally and internationally distributed research distributed informational-analytical system for
groups. Success of this cooperative work depends on searching, processing and analysis of spatial data based
ability of these groups to rapidly exchange data and on the combination of GIS and Web technologies
knowledge, coordinating activities and optimizing currently being developed at Institute of Computational
usage of information-computational resources, services Technologies of the SB RAS (ICT SB RAS, [24]); An
and applications. Georeferenced datasets are currently Integrated Mapping and Analysis System with
Application to Siberia (RIMS, [19], [21]). Nevertheless,
Proceedings of the XVII International Conference in spite of presence of some efforts made in Earth and
«Data Analytics and Management in Data Intensive Space Sciences Informatics research area there is still a
Domains» (DAMDID/RCDL’2015), Obninsk, Russia, lack of powerful online tools combining various
October 13 - 16, 2015
76
capabilities to perform online processing, analysis and To satisfy these requirements the information-
visualization of multidimensional heterogeneous data computational system was based on a dedicated
collections using unified web interface for integrated software framework for developing thematic online ICS
study of global and regional environmental changes. for analysis of dynamics of Earth system components
[26]. This software framework includes:
3 Results 1) A server-side computational back-end (core),
providing data access and processing, and output of
Basic requirements to functional capabilities of the results;
system were defined through meetings with specialists 2) A server-side middleware including PHP [16]
in the area (climatologists, ecologists and biologists controllers run by web portal and providing
from Russian and international institutions) and analysis management of cartographical web services,
of literature ([10], [13], [22]). These requirements also computational core and components of graphical user
include key features of a typical thematic web GIS: interface (GUI);
Internet-access by means of a common graphical 3) A JavaScript library for developing components
Internet-browser to all data processing and visualization of client web GIS application GUI based on AJAX
tools; abilities to overlay processing results as layers on techniques.
top of geo-referenced background with basic functions Computational backend of the information-
of desktop GIS, such as map navigation, synchronous computation system is developed using high-level open-
overview of layers at different scales with capability to source GNU Data Language (GDL, [6]) and Python
show/hide/rearrange them, associating specific colors in [20]. These are powerful programming languages
a legend with range of values, and inspecting data providing reliable computational and visualization
values at specific coordinates of several spatial data routines, as well as application programming interface
layers at once. To provide interoperability with other (API) for NetCDF [14] library and PostgreSQL [18]
information systems requirements were expanded to database controller. Middleware implementing basic
include support for exporting of results in NetCDF files, functionality such as user authentication, database
geo-referenced Tagged Image File Format (geoTIFF, connectivity, HyperText Markup Language (HTML)
[7]), Open Geospatial Consortium (OGC) Keyhole templates usage, language localization, content
Markup Language (KML) and Geography Markup management system and so on is represented by a web
Language (GML). portal and an integrated set of
Fig.1. Main window of the system
dedicated PHP modules (controllers). They are (WMS/WFS) and geospatial data storage. Client-side
organized into the package structure and provide server- part of the ICS is developed using framework GeoExt,
side API to interact with graphical user interface, ExtJS and OpenLayers JavaScript libraries. It provides
computational backend, cartographical web services AJAX-based client-side API for implementation of
77
corresponding PHP controllers as well as basic GUI Spatially distributed results of a processing are
elements of typical desktop application. represented by shaded color plots stored in geoTIFF
Several geophysical datasets are available for files. Spatially distributed results in the form of contour
processing by the system: NCEP/NCAR Reanalysis II, plots, vector wind streams and local point observations
JMA/CRIEPI JRA-25 Reanalysis, ECMWF ERA-40 are stored in ESRI Shapefiles [4]. 2-D plots and
Reanalysis, ECMWF ERA Interim Reanalysis, diagrams are stored in Encapsulated PostScript (EPS)
MRI/JMA APHRODITE’s Water Resources Project format files. Additionally, files with raw values (CSV,
Reanalysis, DWD Global Precipitation Climatology XML, NetCDF) could be written to accompany
Centre’s data, GMAO Modern Era-Retrospective GeoTIFF and EPS files.
analysis for Research and Applications, reanalysis of The main advantage of the system is that it gives
Monitoring atmospheric composition and climate users an ability to perform mathematical and statistical
(MACC) Collaborated Project, NOAA-CIRES data analysis of huge georeferenced datasets without
Twentieth Century Global Reanalysis Version II, NCEP downloading them, remotely through a window of a
Climate Forecast System Reanalysis (CFSR), web browser installed on a common desktop computer
meteorological observational data for the territory of the connected to Internet [27]. In contrast to many scientific
former USSR for the 20th century, results of modeling gateways and analytical web-systems existing
by global and regional climatological models, and this nowadays (http://sciencegateways.org/), this system
list is extending. Flexibility of the system allows easy allows not only to do remote data analysis and
and fast expansion of number of datasets available for visualization, but to represent results in a web interface
processing and visualization. as a stack of georeferenced layers on an interactive map
Reanalyses and modeling data are stored as NetCDF implementing GIS features (Fig. 1 and 2), and to export
files organized in a hierarchy of a file system’s them as OGC-compliant (in terms of metadata) layers
directories: using web mapping (WMS) and web feature (WFS)
// services, thanks to a tight fusion of web-, GIS- and
/ cloud-computing technologies. At present, for the Earth
/ system science area there is no similar information-
/ computational system with analogous capabilities and
data archive. The system’s ability to export results as
Here, is an absolute path to a downloadable binary files and WMS/WFS-layers
common root directory of all data archives, is a name of a subdirectory containing NetCDF offline analytical software including user’s desktop
files of a single data archive, is a GIS. Since results obtained are much smaller than raw
name of a subdirectory containing files of a single datasets they can be downloaded, visualized and
spatial resolution, is a name of a processed on a user’s desktop computer in a reasonable
subdirectory containing files of a single time resolution. time.
Names of underlying subfolders as well as NetCDF Currently, experimental version of the system is
files are not regulated and depend on a dataset. Each available for registered users on the web site:
NetCDF file contains georeferenced multidimensional http://climate.climate.scert.ru/. It is hosted at IMCES
sets of climatological data and corresponding metadata SB RAS (Institute of Monitoring of Climatic and
(georeferencing and description) and spans some spatial Ecological Systems, Siberian Branch of Russian
and time ranges. Such data model allows to organize Academy of Sciences, Tomsk, Russia) on a hardware
datasets on a storage systems. And a dedicated metadata that includes: high performance server HP Proliant
base, developed as a part of the system, allows fast DL585 G7 (four 12-core processors AMD Opteron
location of required data files according to a processing 6172 and 32 Gb RAM) for the computational core,
task. dedicated server (Intel Pentium 4 and 2 Gb RAM) for
Meteorological observational data for the territory of the web-portal, dedicated server (two Intel Xeon 5130
the former USSR for the 20th century are represented and 4 Gb RAM) for Geoserver software. All servers are
by time series of a several characteristics observed at interconnected using 1 Gb/s Ethernet. Such
meteorological stations. To provide fast location and configuration is able to process up to 20 simultaneous
extraction these data were converted and placed in a user tasks.
PostgreSQL database with PostGIS extension.
Also a functionality to run the Weather research and 4 Discussion
forecasting model (WRF, [28]) and “Planet simulator”
model by Meteorological Institute of University of The information-computational system has proved
Hamburg [17] was implemented in the system. Due to its reliability and effectiveness during investigation of
many preset parameters, as well as limited temporal and modern climatic changes in Siberia [25]. Surface air
spatial ranges set in the system these models have low temperatures in Siberia for the last few decades were
computational power requirements and are used in taken from ECMWF ERA Interim dataset, and
educational process thus providing better understanding precipitations for the same period were taken from JMA
of basic climatological and meteorological processes by APHRODITE’s project dataset tied to the system.
students and young scientists [11]. Researchers have benefited the ability to easily perform
78
analysis of huge multidimensional spatially distributed controllers and reduced development overhead for
datasets, to visualize results obtained and investigate construction basic GUI elements by using standard
data in overlapping layers at particular geographical library component, thus producing reliable web
points. Analysis results obtained were found to be in application with fewer lines of code.
good correlation with results obtained by another Although the system was developed for an analysis
authors, and qualitatively precise spatial scales of the of global and regional climate changes and an
climate changes were revealed. Modular software assessment of an impact of these changes on
architecture and continuous contact between researchers ecosystems and human activities, it has found its
and developers allowed to timely implement required purpose in an education process. The ICS was
features in the ICS. Internet-accessible instruments and accompanied by electronic courses based on an
“cloud” data processing allowed distributed educational environment MOODLE (Modular Object-
multidisciplinary group of researches to perform Oriented Dynamic Learning Environment,
complex statistical analysis using familiar Internet https://moodle.org/), thus giving opportunity to use it to
browsers on their desktop computers without study high-grade and post-graduate students with
downloading huge amounts of geospatial data. profile specialties [11]. In 2012 high-grade students of
The decision to use framework GeoExt along with chair of meteorology and climatology in Tomsk state
ExtJS and OpenLayers JavaScript libraries has been university studied basics of climate change analysis
very beneficial for development of web GUI imitating using the web GIS and performed labs on courses
desktop application. These libraries have provided “Regional climate change analysis” and “Future climate
AJAX-based client-side API for corresponding PHP analysis”.
Fig. 2. Visualization of meteorological stations data analysis
The ICS is the first thematic web GIS for climatological 4 Conclusion
research based on the dedicated software framework
designed by the authors [27], and its still under hard The web information-computational system for
development. Progress of extending its functionality geophysical data analysis provides specialists involved
depends on needs of partners within the climatological into multidisciplinary research projects with reliable
and ecological communities. Modular composition of and practical instruments for complex analysis of
the system allows an easy addition of new online climate and ecosystems changes on global and regional
processing and visualization tools as well as new GUI scales. Using it even unskilled user without specific
elements implementing new requirements of users. knowledge of software development data standards can
perform computational processing and visualization of
large meteorological, climatological and satellite
monitoring datasets through unified web-interface in a
79
common graphical web-browser. Applications. http://www.geoext.org
This work is partially supported by SB RAS Basic [13] Kadochnikov, A.A., Popov, V.G., Tokarev A.V.,
research project VIII.80.2.1 and Russian Foundation for Yakubailik, O.E., 2008. Implementation of
Basic Research (RFBR grants 13-05-12034 and 14-05- Internet GIS Portal for Environment and Natural
00502). Resources Monitoring Tasks. J. of Siberian
Federal University. Engineering & Technologies,
References 1(4), 375-384.
[14] Network Common Data Form.
[1] Berrick, S.W., Leptoukh, G., Farley, J.D., and Rui,
H., 2009. Giovanni: A Web service workflow- http://www.unidata.ucar.edu/software/netcdf/
based data visualization and analysis system. [15] OpenLayers 3. http://openlayers.org
IEEE Trans. Geosci. Remote Sens., 47(1), 106- [16] PHP: Hypertext preprocessor. http://php.net/
113. [17] Planet Simulator model. http://www.mi.uni-
[2] Climate Explorer, http://climexp.knmi.nl/ hamburg.de/index.php?id=216
[3] Coupled Climate Model of the INM RAS, [18] PostgreSQL. http://www.postgresql.org/
http://ksv.inm.ras.ru/GCM_DATA_PLOTTING/G [19] Proussevitch, A.A., Vörösmarty, C.J., Glidden, S.,
CM_INM_DATA.html Fekete, B.M., Green,P., Lammers, R.B. (in review
[4] ESRI Shapefile Technical Description. 2009). Global-RIMS: A Global Rapid Integrated
http://www.esri.com/library/whitepapers/pdfs/shap Mapping System for Hydrology Visualization and
efile.pdf Analysis. Computers and the Geosciences.
[5] Felice Frankel, Rosalind Reid. Big data: Distilling [20] Python. http://python.org
meaning from data // Nature. Vol. 455. N. 7209. [21] RIMS, http://RIMS.unh.edu/
P. 30. [22] Romañach, S.S., M. McKelvy, K. Suir, C.
[6] GDL – GNU Data Language. Conzelmann, 2015. EverVIEW: A visualization
http://gnudatalanguage.sourceforge.net/ platform for hydrologic and Earth science gridded
[7] GeoTIFF. http://trac.osgeo.org/geotiff/ data // Computers & Geosciences 76 (2015), P.
[8] GIOVANNI, 88–95.
http://daac.gsfc.nasa.gov/techlab/giovanni/ [23] Sencha Ext JS. The most comprehensive
[9] Gordov E.P., Lykosov V.N., Krupchatnikov V.N., MVC/MVVM JavaScript framework for building
Okladnikov I.G., Titov A.G., Shulgina T.M. feature-rich cross-platform web applications.
Computational and information technologies for http://www.sencha.com/products/extjs
monitoring and modeling of climate changes and [24] Shokin, Y.I., Zhizhimov, O.L., Pestunov I.A.,
their consequences. – Novosibirsk: Nauka, Sinyavsky Y.N., Smirnov V.V., 2007. Distributed
Siberian branch, 2013. – 195 p. information-analytical system for searching,
[10] Gordov, E., K. Bryant, O. Bulygina, I. Csiszar, J. processing and analysis of spatial data.
Eberle, S. Fritz, I. Gerasimov, R. Gerlach, S. Hese, Computational Technologies, 12, Special issue,
F. Kraxner, R. B. Lammers, G. Leptoukh, T. 108-115.
Loboda, I. McCallum, M. Obersteiner, I. [25] Shulgina, T.M., Genina, E.Yu., Gordov, E.P.
Okladnikov, J. Pan, A. A. Prusevich, V. Razuvaev, Dynamics of climatic characteristics influencing
P. Romanov, H. Rui, D. Schepaschenko, C. vegetation in Siberia. // Environ. Res. Lett., 2011,
Schmullius, S. Shen, A. I. Shiklomanov, T. 6(4). doi: 10.1088/1748-9326/6/4/045210.
Shulgina, A. Shvidenko, A. Titov, 2013. [26] T.M. Shulgina, E.P. Gordov, I.G. Okladnikov,
Development of Information-Computational A.G., Titov, E.Yu. Genina, N.P. Gorbatenko, I.V.
Infrastructure for Environmental research in Kuzhevskaya, A.S. Akhmetshina. Software
Siberia as a baseline component of the Northern complex for a regional climate change analysis. //
Eurasia Earth Science Partnership Initiative Vestnik NGU. Series: Information technologies.
(NEESPI) Studies / Regional Environmental 2013. Vol. 11. Issue 1. P. 124 – 131.
Changes in Siberia and Their Global
[27] Titov, A.G., Gordov, E.P., Okladnikov, I.G.͒//
Consequences // Series: Springer Environmental
Science and Engineering. Ed.: Groisman, Pavel Hardware-software platform “Climate” as a basis
Ya., Gutman, Garik. Vol. XII, 2013. P. 19-55. for local spatial data infrastructure geoportal.
Vestnik NGU. Series: Information technologies.
[11] Gordova Yu.E., Genina E.Yu, Gorbatenko V.P.,
2012. Vol. 10. Issue 4. P. 104 – 111.
Gordov E.P., Kuzhevskaya I.V., Martyanova
[28] Weather Research and Forecasting Model.
Yu.V., Okladnikov I.G., Titov A.G., Shulgina
T.M., Barashkova N.K. Supporting educational http://www.wrf-model.org/
process in modern climatology by web-GIS
platform “Climate” // Open and remote education.
Tomsk. 2013. N.1 (49). P. 14 – 19.
[12] JavaScript Toolkit for Rich Web Mapping
80