Integrated Modeling of Hydrological Processes through OpenMI and Web Services Konstantinos Kokkinos1, Athanasios Loukas2, Nicholas Samaras1 and Omiros Iatrellis1 1 Department of Information Technology and Telecommunications, Technological Educational Institute of Larissa, Greece, e-mail: k_kokkinos, nsamaras, iatrellis@.teilar.gr 2 Civil Engineering Department, University of Thessaly, Volos, Greece, e-mail: aloukas@civ.uth.gr Abstract. Difficulty in linking data and models across organizations is one of the barriers to be overcome in developing integrated decision-making systems since not all models exist in the same location. OpenMI is a popular standard for coupling spatially and temporarily hydrological models but it requires that all involved models exist on the same machine. In this paper we present a Web Services based collaborative framework to couple hydrological models. This is achieved by converting the interface of the OpenMI configuration to be web- based and to remotely invoke the computational engines of models. Our case study shows the remote linking of a water balance model and a reservoir model applied for the reservoir of the restored Lake Karla in Thessaly, Greece. The results show that the collaboration process is not affected by the communication overhead introduced and it is bounded by the time, space and optimization characteristics of the coupled models. Keywords: Environmental, web, services, coupling, OpenMI. 1 Introduction Integrated environmental management systems have arisen, because managing environmental processes independently does not always produce sensible decisions when the wider view is taken. However, a major deficiency in integrated management is the complexity of the involved processes we attempt to manage. Managers are therefore turning to decision support systems which in this context comprise one or more models, their associated data and a user interface. These models predict outcomes of various scenarios of interest and therefore, they help the managers in decision making for given scenarios and thus help the manager chose the most appropriate option (Moore and Tindall, 2005; Browning-Aiken et al., 2004, 2006; Matthies et al., 2007). The most important step in building such systems is the accurate transformation of the conceptual models that map the physical processes BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB &RS\ULJKW‹E\WKHSDSHU¶VDXWKRUV&RS\LQJSHUPLWWHGRQO\IRUSULYDWHDQGDFDGHPLFSXUSRVHV  ,Q06DODPSDVLV$0DWRSRXORV HGV 3URFHHGLQJVRIWKH,QWHUQDWLRQDO&RQIHUHQFHRQ,QIRUPDWLRQ  DQG&RPPXQLFDWLRQ7HFKQRORJLHV  IRU6XVWDLQDEOH$JULSURGXFWLRQDQG(QYLURQPHQW +$,&7$ 6NLDWKRV6HSWHPEHU    409 into computational models to produce a numerical simulation (Gregersen et al., 2007). Recently, the research in collaborative decision making in Hydrology and other sciences has been elaborated through the introduction of the Open Modeling Interface, (OpenMI) standard ((Gregersen, 2005; Gregersen et al., 2007). OpenMI offers a means under which compatible models can be spatially and temporarily synchronized, interlinked and interchange data. Prior to linking, all models must be migrated into the OpenMI-standard to become compliant for coupling. The migration process enriches the model computational engines with a migration wrapper which must be present to interact with other external models by just promoting the standard. In that way, models are marked by a two-way link to define mutual dependence and they may run asynchronously. The above methodology however can be applied only by implementing the necessary infrastructure in order to achieve compliance with OpenMI and only if the models involved in the interlinking process are all installed in the same machine on which OpenMI is installed. The motivation for this research work was to design a web service for the remote model interlinking of models. To the best of our knowledge, such a standard does not currently exist, yet is needed to achieve an end-to-end integration for modeling water resource systems. While this approach has obvious benefits this methodology for integrated modeling has also a collection of challenges to overcome that are either due to model specifics or due to the technology limitations that exist. The need for model calibration prior to model coupling is apparent and thus restricts the collaborative environment to approach the linking process holistically. On the other hand, the need for modeler authenticated access in a web based framework, the apparent existence of state variables to synchronize both ends of the system and the simultaneous model simulations force for boundary communication conditions that need to be optimized in order to avoid the introduction of overheads. In this paper, we propose and experimentally verify with a case study a semi-web services approach to overcome the above challenges. Assuming that, the OpenMI standard is installed on the server side then, modeler-clients of the system need to upload their models in the server repository and need a web-interface of the OpenMI configuration editor for setting up the coupling process. The simulation runs are triggered by the modeler remotely however, the whole computation process is done on the server. A simulation monitoring tool (in a form of logs) must also be provided for modelers to control the coupling and initiate interrupts, if needed. Such an approach is limited compared to a fully web-service methodology which needs implementation of service registries, proxy servers, service orchestration and extensions to the SOAP protocol to accommodate the spatial and temporal synchronization of the models. However, it is faster and open-ended on the number of models that exist on the repository. In the following sections of the paper we illustrate a case study of the migration through OpenMI using the semi-web service approach discussed above of two water resources models: The first model, called UTHBAL is a conceptual monthly water balance hydrological model. The model has been developed by (Loukas et al., 2003) and updated in its present form for simulation of hydrologic cycle components in (Loukas et al., 2007). The second one is a reservoir model, called UTHRL which simulates the natural water runoff inflows from a specific watershed area, any water 410 transfers from nearby water resources, the water withdrawals, and the net water losses (Loukas et al., 2007). Prior to this work we have implemented a stand-alone framework called UTH-MODELER under which the above models have been plugged and interchange data. Section 2 provides an in depth yet compact description of the UTHBAL and UTHRL models. In section 3, we explain the UTH-MODELLER framework from a software engineering point-of view summarizing its functional requirements and providing all the additional extensions to become a web service. That is, we excluded from the framework all added models and provided model portability via an XML- schema incorporating all time-series data along with the geometries of the nodes of where the data were observed. Section 4 shows the coupling mechanism between the models in the web service, the uploading of models and provides details of the simulation runs, studies and results produced. Finally, in the last section we elaborate on the future challenges of our methodology. 2 Model Description 2.1 The Hydrological Water Balance Model The hydrologic model used for this research is a monthly conceptual water balance model, called UTHBAL which was developed and optimized by (Loukas et al., 2003; Loukas et al., 2007). The model observes three input variables: precipitation, temperature and potential evapotranspiration. The precipitation is divided into rainfall and snowfall. The estimations of the rainfall and snowfall percentages are done using a logistic relationship based on the mean monthly air temperature with a simple degree-day method which is described in (Semadeni-Davies, 1997; Knight et al., 2001). More specifically, let %S be the snow percentage then, according to the model: (1) 0 %S = 0 if T ³ 12.22 C 1 %S = if - 10 0 C £ T £ 12.22 0 C (1.35T *1.61) + 1 %S = 1 if T £ -10 0 C The snowmelt of the jth month, SM(j), is calculated to be SM ( j ) = Cm .T ( j ) . (2) where, T(j) is the jth mean monthly temperature and Cm is the monthly melt rate factor. The snow water equivalent of the accumulated snowpack is another variable under interest here. It is denoted as SWESP and it is estimated to be equal to 411 SWESP ( j ) = SWESP ( j - 1) + S ( j ) - SM ( j ) . (3) where, S(j) is the snowfall during the jth month, which is equal to S(j)=P(j)x%S(j) and P(j) is the total precipitation of that month. Fig. 1. A logical diagram of the hydrologic balancing model UTHBAL. According to the model, the total watershed runoff of the jth month is divided into three separate components: the surface runoff, SR(j), the medium runoff, MR(j) and the base flow runoff using a soil moisture mechanism. The first priority of the model is to fulfill the actual evapotranspiration. The monthly actual evapotranspiration Ea of the jth month depends on the available soil moisture and the average surface potential evapotranspiration Ep of that month. The monthly actual evapotranspiration is estimated using a relationship proposed by (Loukas et al., 2007): { E a (J ) = min E p (J ) ( 1 - a Smoist( J ) E p ( J ) } ), Smoist(J ) . (4) As depicted above in Fig.1, the model depends on five empirical factors namely α, β, Κ, CN, Cm that must be optimized in order for the model to produce optimal output. For that reason a calibration process is applied. The time-series subset used for the calibration can be arbitrary large. The calibration of the model is based on non-linear optimization techniques. More specifically the Reduced Gradient Decent is applied to the above time-series data to optimize the factors. 2.2 The Water Reservoir Model To manage the renewable surface water resources in a study area we use a reservoir model, called UTHRL which is an optimized version of the reservoir model given in 412 (Loukas et al., 2007). This is a simple monthly conceptual model using a general equation to describe the operation of the reservoir in a monthly time step: V ( j ) = V ( j - 1) + Q( j ) - E( j ) - A( j ) - Y ( j ) . (5) where V(j) and V(j-1) correspond to the stored water volumes in the reservoir on the months j and j-1 respectively, Q(j) is the inflow to the reservoir on the month j, E(j) is the net water loss from the reservoir for the month j, A(j) is the real withdrawal for the month j, and Y(j) is the real overflow during that month. The reservoir storage and overflow are calculated at each monthly time step using (5). The monthly net water loses of the reservoir are estimated from the equation: E ( j ) = E0 ( j ) - P0 ( j ) + L( j ) + Q( j ) . (6) where, E(j) are the net water loses of month j, Eo(j) is the evaporation from the reservoir water surface of month j, Po(j) is the direct precipitation on the reservoir during month j, L(j) are the estimated deep percolation loses to groundwater and Q(j) is the natural surface runoff that would have been generated from the area of the reservoir if the reservoir does not exist. The deep percolation loses are usually estimated by field geological measurements before the development of the reservoir. In case that such measurements do not exist, estimates of the deep percolation loses are used or the deep percolation loses are taken equal to zero. All the above quantities are expressed in millimeters. The above quantities are expressed to volume units (hm3) by multiplying them with the reservoir surface area. The water level and the reservoir surface area are estimated using the reservoir storage-water level and surface area-water level curves. Using these curves, an expression is developed relating the reservoir water surface area, F, to reservoir storage, V: F = a + bV c . (7) 3 The UTH-MODELER Framework as a Web Service The UTH-MODELER is an integrated simulation framework that was introduced in (Kokkinos et al., 2010) for the coupling of hydrological models as a stand-alone application. The framework has the open-ended inclusion of models property. It accepts models carrying their time-series data along with their geometry idiosyncrasies. The UTH-MODELLER incorporates the extensibility feature that allows the introduction of other hydrological and/or water resources models. Furthermore, this framework provides: a) a visualization component for creating time-series data histograms of the simulated variables, b) a calibration and optimization component for the conceptual models and c) an automated migration into OpenMI component for the incorporated models in this stand-alone version. This stand-alone migration methodology via OpenMI has been presented in (Loukas et al., 2008; Kokkinos et al., 2010). Incorporating the OpenMI-standard as a primary 413 coupling mechanism, it solves the problem of model spatial variance. Each model carries its own data acquisition nodes as mapping points of a specific location as a composite data structure with a unique ID for each node or as grid cells (i.e. XYZ- polygons). Furthermore, it accommodates simultaneous simulations of models in lumped mode (treating the overall geographical area of study as a whole) or in the opposite extreme in fully distributed mode achieving spatial desegregation of the data acquisition nodes. The framework is implemented as a set of various components including: a) a storage component of the time-series data accepting Excel spreadsheets, DEM input files or GIS-shape input files, b) a calibration and optimization component for the included models, c) a visualization component for the graphical representation of the input and all output variables of the simulations, and d) an automatic migration and export component that adds to the models the property of OpenMI-compliance. Fig. 2. The Web OpenMI-Configurator Interface. We added to the framework the functionality of automatic storage of model data in XML format where each node of the model carries out its own spatial and temporal characteristics along with its data. In this way models that currently the UTH-MODELER supports may be loaded to the framework. For this to happen, we also changed the model interoperability functionality of the framework to become a web service. Furthermore, we extended the framework by implementing a web interface for the OpenMI-Configuration Editor so that modelers can now couple models into UTH-MODELER after loading them to the service. At the same time, the new framework allows the permanent deposition of models into the service 414 repository, thus prompting future reusability. For the experimental verification of the service, we added models that were incorporated in the stand-alone version of the framework. The web application of the OpenMI configurator includes all the operations that the stand-alone version included. Fig. 2 illustrates the model inclusion process into the web interface. It implements a web service designed to serialize all the attributes of the model coupling composition. We added to the functional requirements of the framework the ability of the user to select which model variables should be available through the model data exchange. For our already programmed models (UTHBAL, UTHRL) this automation is practical because it helps to show all intermediate output data variables. Furthermore event listeners for the monitoring of the simulation process are added as Fig. 3 depicts. Fig. 3. Variability of input/output exchange items in the linked simulation and monitoring of process interrupts. The web application is based on the composition class which implements all the web methods relating to the coupling composition consisting namely: adding models, initiating triggers, setting up connections, indicating input and output exchanged items and monitoring output properties. A session variable is used to store the serialized composition class every time when an update or an interrupt occurs. The framework repository saves all coupled models that have been implemented within the UTH-MODELER in a binary file (named UTHMODELER.ubm) which must exist in the same directory with the manifest files (.OMI extension) that accompany the models for coupling. This file needs to be present so that, the engines of the models, (namely UthBalModelOMIEngine and UthRLModelOMIEngine) 415 know which data series can be exchanged. The OpenMI SDK provides a template class called IEngine that is used as an interface of each model engine making it ready to interchange data. The web Configurator instantiates a separate IEngine object for each model engine for the simulation. These components discover the directory path of the corresponding .ubm file so that they can instantiate the OpenMI-compliant model accordingly. It is worth noting that, we cannot create just the web engine components for the models without the corresponding .ubm file since, the simulation cannot run without data. 4 Collaborative simulation case study through the Web Configurator. In this section we describe the coupling of the hydrological model UTHBAL with the reservoir operation model UTHRL so that, we will have data exchange between the models under simultaneous simulation. This operation is handled via the web service described previously. We set up a case study of a specific region in Lake Karla, Thessaly, Greece in order to experimentally verify our models in terms of surface water inflow assessment into to the reservoir of the lake at a monthly time scale. The area under consideration is depicted in Fig. 4 which contains a digital elevation model of the Lake Karla watershed in the location of the reservoir. Furthermore in the same figure we illustrate how the spatial characteristics of the models are visualized in the Web Configurator. Fig. 4. A) Digital Elevation Model of Lake Karla watershed in the location of the reservoir and B) Visual representation of the model grid cells in the Web Configurator. 416 The study area is an intensely cultivated agricultural region. The over-exploitation of groundwater and the unsustainable water resources management has led to a remarkable drawdown of the water table. However, the necessary functions have been initiated for the lake restoration recently. The partial restoration of the former Lake Karla is expected to reverse this situation and the relevant environmental problems caused by the lake drainage and provide new irrigation water resources for the local farmers. The available data for the estimation of the water resources were precipitation data, temperature data, and discharge data. These data span for a period of time of over 50 years and they are on a monthly basis. The original meteorological and discharge measurements were collected by regional and prefecture water resources agencies and the competent authorities. The measurements have been checked for errors, homogenized and processed according to the World Meteorological Organization techniques and standards. The UTHBAL model runs on three spatial modes: a) fully distributed mode, b) semi-distributed mode and lumped simulation mode. Fig. 5. Illustrative representation of the simulation time step between the UTHBAL and the UTHRL model. (The joining variable is the surface runoff) The spatially lumped mode was chosen for the experiment. In lumped mode, and for each monthly time step, the UTHBAL model feeds the UTHRL model with a single value for the total surface runoff. This event triggers the UTHRL model to include as inflow the value Q(j), in order to execute its time step compotation and to wait for the next feeding. The initiation of the simulation is invoked by the UTHRL 417 through a call to the GetValues() OpenMI-function. For that reason, we explicitly connect to it an OpenMI-trigger in the composition design phase. This process becomes then repetitive for the overlapping time period of the two models. Fig. 5 shows such a simulation step where in fig. 6 we illustrate the web log of the whole simulation produced. The inclusion of data of the two models in our experiment makes then partially synchronized. However, for an overlapping period of 10 years (i.e. 120 time steps) the system succeeds to interchange values. Fig. 6. A screen shot of the simulation web log produced. Apart from the fact that, the user can set parameters in visualizing exchange quantities in the OpenMI configuration editor, our modeler also produces output in text files, EXCEL files and ascii-DEM files which can be seamlessly read by a GIS system. More specifically and for the case of the UTHBAL model, it is also our intention to provide data acquisition methodologies for real time input coming from wireless sensor networks or networks of meteorological stations for the regions under study. Finally, it is worth noting that comparing the simulation of simultaneous runs for the models in stand-alone mode with the remote simulation over the web service implemented we did not find any significant delays. Any communication overhead produced is meaningless since, the simulation starts after both models are loaded onto the server. Thus the set of all HTTP-requests is limited. 5 Conclusions and Future Challenges. We implemented a web-based hydrological decision support system to meet the challenges of current desktop-based implementations. Such a system is based on the OpenMI-coupling standard and has achieved to convert the OpenMI-Configurator in to a web service that can be remotely accessed and configured to couple our prior implemented models UTHBAL and UTHRL. Our prototype avoided the step-by-step data interchange of models over the SOAP protocol by introducing a mechanism to upload all models of interest in the server repository. The benefits of such an 418 infrastructure are essential and they span into: a) provision of remote access to large time series databases, b) deployment of simulations on one machine while the simulation runs are migrated to dedicated servers, c) model linking of multiple providers without the need for moving the data and d) effective usage of simulation licenses. In this way modelers need only a simple web interface that: a) uploads model engines and data series assuming that are migrated to the OpenMI-paradigm, b) designs a coupling composition c) triggers the simulation start and d) provides an observation/monitoring tool for the simulation. Our future research challenges span in two directions. Primary concern is given to achieving the conversion of the UTH-MODELER framework into a fully functional web service. For this to happen, we need to provide a functionality named “Any-to- UBM” that can accept arbitrary models and automatically convert their mode and data specifications into our XML schema structure our service implements. Secondly and arguably more difficult task is the creation of a web mapping service that detects and automatically resolves any model spatial and temporal heterogeneities. Unfortunately the intricacies of the model mathematics involved along with the special visualization tools needed for the modelers to control such simulations indicates the computational complexity of such implementation. Acknowledgments. Part of this research has been funded by EC under the LIFE Framework Program and contributing to the implementation of the thematic component LIFE-Environment under the policy area "Sustainable management of ground water and surface water management". Contract no: LIFE06 ENV/UK/000409. References 1. Moore, R., and Tindall I. (2005) An overview of the open modeling interface and environment (the OpenMI), Environmental Science & Policy, 8, 279-286. 2. Browning-Aiken, A., Richter, H., Goodrich, D., Strain, B., Varady, R. (2004) Upper San Pedro Basin: Collaborative binational watershed management. International Journal of Water Resources and Development 20 (3), 353-367. 3. Browning-Aiken, A., Varady, R.G., Goodrich, D., Richter, H., Sprouse, T., Shuttleworth, W.J. (2006) Integrating science and policy for water management: A case study of the Upper San Pedro River Basin. In: Wallace, J.S., Wouters, P. (Eds.), Hydrology and Water LawdBridging the Gap: A Case Study of HELP Basins. 4. Gregersen, J.B., (2005) OpenMI: The essential concepts and their implications for legacy software, Advances in Geosciences, vol. 4, 37-44 5. Gregersen, J.B., Gijsbers, P.J. and Westen, S.J. (2007) OpenMI: Open Modelling Interface, Journal of Hydroinformatics, 9(3),175-191. 6. Knight, C.G., Chang, H., Staneva, P.M. and Kostov, D.A., (2001) A simplified basin model for simulating runoff: the Struma River GIS, Professional Geographer, vol. 53(4), 533-545. 419 7. Kokkinos, K. and Loukas, A. (2010) Collaborative Migration, Coupling and Simulation of Water Resources Models through OpenMI, Enabling Technologies: Infrastructures for Collaborative Enterprises (WETICE), 19th IEEE International Workshop ISSN: 1524-4547, 166 – 171. 8. Loukas, A., Vasiliades, L. and Mpastrogiannis, N., (2003) Hydrologic simulation of surface water balance in Yermasoyia watershed, Cyprus, In Sidiropoulos E., Iacovides I., (eds) Research of water resources in Cyprus, Nicosia, Cyprus, 83- 114, (in Greek). 9. Loukas, A., Mylopoulos, N. and Vasiliades, L., (2007) A modeling system for the evaluation of water resources management strategies in Thessaly, Greece, Water Resources Management, 21(10), 1673-1702. 10. Loukas, A., Kokkinos, K., Vasiliades, L. and Liakopoulos, A., (2008) The Migration of the UTHBAL Hydrological Model into OpenMI, Proceedings of the iEMSs 2008: International Congress on Environmental Modelling and Software Integrating Sciences and Information Technology for Environmental Assessment and Decision Making, Vol 2., 1102-1109. 11. Matthies, M., Giupponi, C., Ostendorf, B., (2007) Environmental decision support systems: Current issues, methods and tools. Environmental Modeling & Software 22 (2), 123-127. 12. Semadeni-Davies, A., (1997) Monthly snowmelt modeling for large-scale climate change studies using the degree day approach”, Ecological Modeling, vol. 101, 303-323. 420