Validation Experiment Protocols

This page gives protocols for the different VALUE experiments, i.e., mostly technical details on the cross validation, the data sources and the data format. For the rationale behind every experiment please refer to 

Douglas Maraun, Martin Widmann, José M. Gutierrez, Sven Kotlarski, Richard E. Chandler, Elke Hertig, Joanna Wibig, Radan Huth, Renate A.I. Wilcke (2015) VALUE ‐ A Framework to Validate Downscaling Approaches for Climate Change Studies, Earth's Future, 3(1), 1-14. Doi: 10.1002/2014EF000259

Please refer to the terms and conditions for details about the data policy (including publication of the validation results and the downscaled data on the VALUE portal). The validation results generated by the VALUE validation protal for each method can be shared in three different modes: private use only, sharing the results with VALUE members for purely scientific interest, or  publish them in a public mode according to a Creative Commons by-license, similar to open access journals. The downscaled data can be published separately (to participate in core VALUE publications, publication in public mode is mandatory for both the validation results and the data); redistribution of the raw data is not allowed. Every contributor is free to choose how many stations/regions to downscale and in which experiments to participate. The more, the more rigorously a method is tested, of course.

A special issue presenting the most relevant results is planned, based on the data uploaded prior to 30 September 2015 (deadline for experiments 1a and 1b). Key papers - aiming to present pan European results - will be organised top-down. Contribution to these papers requires a substantial contribution to the downscaling, and is subject to approval by the lead author. Additional papers (e.g., with regional foci or on special topics) can - coordinated with the VALUE steering group - be designed via a grass-root approach. A link to organise these papers will follow soon.   

EXPERIMENTS:

• EXPERIMENT 1(a): Perfect Predictor, Station Data. Deadline: 31/12/2015 (closed)
• EXPERIMENT 1(b.1): Perfect Predictor, Gridded data (E-OBS, closest gridboxes to the stations). Deadline: 31/12/2015 (closed)
• EXPERIMENT 1(c): Perfect Predictor, Nested Station Data (spatial aspects). Deadline: 19/2/2016 (closed)
• EXPERIMENT 1(d): Perfect Predictor, Station Data (multi-variable aspects). Deadline: 19/2/2016 (closed)
• [NEW] EXPERIMENT 2(a): GCM Predictor, Station Data (trend sensitivity). Deadline: 30/04/2022 (open)
• [NEW] EXPERIMENT 2(b): GCM Predictor, Gridded data (EURO-CORDEX domain). Deadline: Planned

EXPERIMENT 1(a): Perfect Predictor, Station Data (ECA&D)

Time Range and Cross Validation

01.01.1979 - 31.12.2008

Sub-periods for 5-fold cross validation:
1979 - 1984; 1985 - 1990; 1991 - 1996; 1997 - 2002; 2003 - 2008 

Repeat the calibration/validation process of the downscaling method five times, considering each of the superiods (in sucession) as the test set and the remaining four subperiods for calibration (e.g. the first test set would be 1979-1984 with the downscaling method calibrated using 1985 - 2008; the second test set would be 1985-1990 with the method calibrated using 1979-1984 and 1991-2008; and so on).  The resulting five cross-validated test sub-series should be combined into one series spanning 01.01.1979 - 31.12.2008. 

Predictors

Predictors are to be taken from ERA-Interim and can be downloaded from the ECMWF:

• Surface variables;
• Variables at pressure levels.

A standard pre-selection of commonly used predictors is available from the User Data Gateway (UDG) of the  Santander MetGroup (this dataset is used by the statistical downscaling portal). Registration (and acceptance of the ECMWF data policy) is required. Subsets of this dataset can be 1) downloaded in a user-friendly form using a web browser or 2) remotely accessed from R (more information here; also in pdf). The same predictors from several CMIP5 Earth System Models will be also available for succeeding experiments.

For those groups applying bias correction techniques, a dataset (text file) with the ERA-Interim data (precip and temp) for the closest gridboxes for the 86 stations is available in the VALUE datasets page). This dataset has been prepated to facilitate the work to these groups. 

Additionally, those groups applying bias correction techniques should use a second predictor dataset, obtained from a 0.11º resolution RCM (KNMI-RACMO22E) driven by ERA-Interim (taken from the EURO-CORDEX dataset). In order to minimize the workload for this experiment, this data has been processed and prepared following the same ASCII format as the observations, considering the nearest gridboxes to the 86 stations (VALUE_RACMO011_86_v2.zip; see VALUE datasets page for details). The results for this predictor dataset should be uploaded into the VALUE validation portal indicating the experiment Experiment_1a_RCM. Note that Experiment_1a should be used in the standard case (when the predictors are taken from ERA-Interim). 

Predictands

Daily station data for the 86 stations selected from ECA&D across Europe (VALUE_ECA_86_v2.zip). This dataset is available for download in the VALUE datasets page (new version with some corrections of outliers and inconsistencies is available since 17/4/2014).

Data Upload

Contributions should be submitted to the VALUE validation portal indicating the following experiments: Experiment_1a (predictors from ERA-Interim); Experiment_1a_RCM (predictors from the KNMI RCM driven by ERA-Interim; only for Bias Correction methods).

Data need to be uploaded as ASCII files, separately for each variable. Data for different stations are combined into one file. For deterministic methods, one file is required. For stochastic methods, 100 realisations from the underyling (potentially) time-varying pdf are required, each realisation in a single file, and all combined into a single zipped archive. 

The format of each ASCII file is identical to that of the predictand dataset, and consist of a header and data rows for each date. The first column contains the dates (YYYYMMDD), all other columns data for each downscaled station. Data are comma-separated, missing values are denoted as NaN by default (however any other missing code can be chosen and indicated in the validation portal in the uploading phase).

Header row:
YYYYMMDD, station_id of first downscaled station,..., station_id of last downscaled station

Data rows:
date, data for first downscaled station,..., data for last downscaled station

station_id refers to the code of the station as described in the file stations.txt file of the VALUE_ECA_86 dataset (see the description of the data format for more details).

EXPERIMENT 1(b.1): Perfect Predictor, Gridded data (E-OBS, closest gridboxes to the stations)

Time Range, Cross Validation, Predictors and Data Format

Same as Experiment_1a

EXPERIMENT 1d: Perfect Predictor, station data (multi-variable aspects)

Time Range, Cross Validation, Predictors and Data Format

Same as Experiment_1a. Moreover, a dataset (text file) with the ERA-Interim data (precip, temp, wind, humidity and cloud cover) for the closest gridboxes for the 12 stations is available in the VALUE datasets page). This dataset has been prepated to facilitate the work groups applying bias correction techniques.