Environmental computational spreadsheets are important tools in supporting decision making. However, as the underlying concepts and relations are not made explicit, the transparency and re-use of these spreadsheets is severely limited. The goal of this project is to provide a semi-automatic methodology for constructing the underlying knowledge level model of environmental computational spreadsheets. We develop and test this methodology in a limited number of case studies. Our methodology combines heuristics on spreadsheet layout and formulas, with existing methods from computer science. We evaluate our constructed model with both the original developers and their peers.
Current environmental issues, like climate change and biodiversity loss, are
universal in their scale and long-term in their impact, their mechanisms are complex,
and empirical data are scarce [1{3]. In addition there is an urgent need to nd
strategies to cope with these issues, and political pressure on the research
community is high [
Environmental computer models are mainly developed and used by domain
scientists and typically implemented as spreadsheets, Fortran programs or in
MatLab. These domain scientists have a knowledge level model [
It is hardly possible to obtain the knowledge level model from the domain
scientists themselves. They may give a limited textual explanation about their
ideas and choices in their publications, but they rather focus on the
computational side of modeling [
Spreadsheet Implicit domain knowledge
Domain scientists
Design spreadsheets
Describe research
Research
results
Publication
level model in their mind [
The focus of this research is on environmental computer models that are
implemented as spreadsheets, from now on called `environmental computational
spreadsheets'. Spreadsheets are widely used by domain scientists to store and
manipulate quantitative data from their research projects [
Results of this research could enable peers to discuss and assess the scienti c quality of environmental computational spreadsheets and to reuse corresponding results and insights. This could contribute to both scienti c cooperation and progress, and reliable environmental decision making.
Our research is focused on spreadsheets from the domain of environmental science. However, scientists from other domains may have a similar way of designing and using their spreadsheet models as environmental scientists. We therefore think that the methods and insights from this study might also be applied to spreadsheets from other domains, provided that these spreadsheets contain both domain knowledge and quantitative data.
Many authors in the eld of environmental science advocate standardization
of the modelling process, summarized to as `Good Modelling Practice', to
enhance transparency of environmental computer models [
In recent years signi cant progress has been made in the semantic
annotation of scienti c models, data sets, and publications. Many tools and techniques
are avaliable to connect measurements and terms to the identity of observable
entities they quantify [18{21]. A higher level of abstraction that is being
investigated is the semantic annotation of scienti c practice as a whole. The open
provenance model, PROV, 1 helps scientists to document and process
provenance information to ensure reproducibility of their analyses [
Several tools and techniques can be used to annotate tabular data. The Data
Cube vocabulary 2, for example, provides a means for publishing statistical data
as linked data with associated metadata in order to support interpretation and
reproducibility. Existing conversion systems like RDF123 [
In the above described annotation methods the spreadsheets themselves remain largely black-boxes. As a consequence, we may miss out on valuable information on the developers' understanding and interpretation of the system of interest. However, related work also shows that there are plenty solutions to the issue of representing scienti c tabular data. As such these studies provide useful tools and information that can be used as a starting point for present study. The general research question we wish to answer in our study is the following: To what extent can the underlying knowledge level model of an environmental computational spreadsheet be made explicit? We re ne this question into two more speci c subquestions. 1. How can the underlying knowledge level model of an environmental computational spreadsheet be adequately described? 1 W3C Provenance Working Group, http://www.w3.org/2011/prov/ 2 Data Cube, http://www.w3.org/TR/vocab-data-cube/ 3 Anzo, http://www.cambridgesemantics.com 5 6
An adequate description of the underlying knowledge level model is de ned as a description that { agrees with the views of the original developers of the spreadsheets. { can be understood and applied by the original developers of the spreadsheets and their peers. { allows representation of domain concepts, their hierarchical and property relations, and the computational relations that exist between these concepts 2. What are the requirements for a methodology for constructing the underlying knowledge level model of an environmental computational spreadsheet? When we apply our methodology to an environmental computational spreadsheet, we expect that the resulting constructed knowledge level model is an adequate description of the underlying knowledge level model.
We did two case studies on an existing environmental computer model,i.e., a spreadsheet model that enables policy analyses concerning the Dutch energy system .
In the rst case study [
Our case studies are all scienti c spreadsheet models of existing research projects from the domain of environmental science. We have access to the actual spreadsheets and corresponding datasets, as well as to the publications describing the models and analyses. Furthermore, we have personal contact with the model developers and users.
We develop our methodology based on the in-depth, qualitative analysis of one case study. We will manually analyze the layout of the spreadsheet tables, as well as the formulas connecting the spreadsheet cells. We determine to what extent the observed patterns provide insight in the semantics of the content of the tables, and record our ndings in heuristics. Spreadsheet terms can be matched automatically with concepts of external vocabularies on domain concepts, and on quantitative tabular data. We combine this matching with our layout heuristics to recognize the concepts in the spreadsheets and their interrelations. In addition, we will automatically trace the dependencies between spreadsheet cells through formulas and analyze the resulting networks using techniques for network analysis. We combine these analyses with our heuristics on formulas to construct the calculation work ow in the spreadsheets.
Research question 1 is studied by focusing on the performance of our method in each case study. The di erent steps in our methodology of constructing the knowledge level model are performed manually by the original developers, and their results are compared with results from our semi-automatic method. We test the applicability of the constructed model by using it to connect concepts from the spreadsheets, with concepts from corresponding publications, or data sets. In a separate user study we will test to what extent peers are able to understand and apply the constructed knowledge level model.
Research question 2 is studied by focusing on the di erent techniques that are used to describe the knowledge level model. The use of external vocabularies is evaluated by determining how many of the spreadsheet terms could be matched, and how relevant these matches are. We also determine which properties of these vocabularies in uence this matching. The use of network analysis techniques is evaluated by determining to what extent these techniques are able to recognize the important variables, as indicated by the original developers, in the calculation work ow. We determine which properties of the spreadsheets in uence the performance of our method. 8
In order to test our hypothesis we will formulate measurable de nitions on what it means for original developers and peers to understand and apply the constructed knowledge level model. Possible indicators we could use are, for example, { the number of concepts, relations and variables that occur both in the constructed model and in the manual analysis of the original developers. { the number of connections that can be made from the spreadsheet to corresponding publications and datasets. 9
We think our approach is likely to succeed as it is targeted at existing environmental computational spreadsheets. We expect that studying the patterns in these spreadsheets will provide us useful insights on environmental modeling. We also see several promising external developments. Firstly, there is a growing awareness of both the importance of open source code and data, and the importance of methods to provide corresponding credits to modelers and data providers. Besides, there is an increasing availability of external domain vocabularies.
This PhD research is now at the half way stage. Current work is an extension of
our rst case study (section 6) and involves the development of a semi-automatic
method for de ning the concepts and interrelations in spreadsheets. We use the
external vocabularies AGROVOC [27] and OM[
This publication was supported by the Data2Semantics project in the Dutch national program COMMIT. Guus Schreiber and Jan Top (supervisors), Paul Groth and Lora Aroyo are acknowledged for providing useful comments and suggestions.