By Lena Boysen (MPI, Germany), Colin Jones and Alberto Muñoz (UNIVLEEDS, UK).
In May 23-24 2017, the European Commission initialised two events involving CRESCENDO in Brussels: The Project interim review after 18 months of improving our Earth System Models (ESMs), and a separate Modelling workshop with the title: Evaluating climate and Earth system models at the process level, co-organised by CRESCENDO in partnership with other three H2020 projects: PRIMAVERA, Blue-Action and APPLICATE. Both meetings were held at the headquarters of EASME (EU Executive Agency for Small and Medium-sized Enterprises).
Our 1st Project interim review was organised in line with the article 22.1.2 of the Grant Agreement in May 23. Two external reviewers were present together with Commission Officers with the aim to assess the following aspects of CRESCENDO:
- the degree to which the work plan has been carried out and whether all deliverables were completed;
- whether the objectives are still relevant and provide scientific or industrial breakthrough potential;
- how resources were planned and used in relation to the achieved progress, and if their use respected the principles of economy, efficiency and effectiveness;
- the management procedures and methods of the action;
- the beneficiaries’ contributions and integration within the action;
- the expected potential scientific, technological, economic, competitive and social impact, and plans for using and disseminating results.
The review began with an overview of CRESCENDO (Colin Jones, UNIVLEEDS) followed by presentations about the current status, exciting results and ambitious plans of the different working packages. Specifically, the reviewers had the chance to question and comment on five topics dealing with the development and evaluation of biogeochemical process and aerosol modeling in the ocean (presented by Laurent Bopp, CNRS-IPSL), the atmosphere (by Ken Carslaw, UNIVLEEDS) and on land (by Lena Boysen, MPI) as well as the development of a new suite of socio-economic scenarios and the coupled model evaluation tool ESMValTool (by Veronika Eyring, DLR).
Following this review meeting and based on the review report drafted by the expert(s), the Agency has considered the project implementation was satisfactory stressing how the project has fully achieved its objectives and milestones for this period and excellent progress has been made throughout the initial startup phase of this project: the impact of the project closely follows the targets of the DoA, with already major outcomes in 6 of the 8 identified areas: 4 concern scientific activities and 2 target dissemination and exploitation of the results.
Later in the day after the review meeting, around two dozen of attendees from the four H2020 organizing projects (CRESCENDO, PRIMAVERA, Blue-Action and APPLICATE) plus members of other FP7 invited projects (BAACHUS, STRATOCLIM, DACCIWA, IS-ENES2, ICE-ARC, IMPREX) joined the Workshop on ESM evaluation: Evaluating climate and Earth system models at the process level (online workshop site). During one and a half days, projects aimed to synthesize common methods and underpin activities where we will likely find useful collaborations and that might then include:
1 – Identifying key process-level performance metrics/diagnostics with regards to :
- coupled physical climate models and Earth system models (additional biogeochemical and aerosol-chemistry processes in the latter).
- per domain (terrestrial, marine, atmospheric, cryosphere).
- per region (e.g. Arctic, but maybe teleconnections/drivers from tropics).
- per climate variable, or variables wrt co-variability of processes and interactions.
- per simulation/prediction/projection type and timescale of importance.
1)a. Shared/coordinated development/testing of evaluation methods/performance metrics.
1)b. Shared/coordinated implementation in one or more community evaluation tools.
2 – Identifying and sharing suitable evaluation data (observations, reanalysis, process models etc):
- In particular data that the projects are generating/improving or using it to constrain models.
3 – Identifying suitable (possibly shared) model simulations and experiment protocols:
- At least awareness of simulations planned/completed, since probably unlikely to change already planned ones.
4 – Shared/coordinated analysis of model simulations and understanding of process-level successes and problems:
- Focus on common key processes.
- Common model biases and their impacts (and how to fix them).
- Can we envision some common outcome of assessing the relative impact/importance of our different foci.
5 – Shared interest in model development and outcomes;
- At least awareness of the key model development areas that may be of common interest or impact common issues.
Soon three foci were summarised to increase the productivity and efficiency across the projects, which also apply to CRESCENDO’s tasks and aims:
- The need for common language (e.g. definitions for metric, constraints, diagnostics etc.) and more observations for validating models;
- An overview across projects (topics, time and spatial scales, methods etc.) to find partners for knowledge exchange and to reduce workload (i.e. avoiding the reproduction of similar methods and increasing expert input);
- The benefits of long-term support of tools/methods/data to guarantee their maintenance, distribution, correct application and survival post-CRESCENDO.
For CRESCENDO itself, key issues seemed to be the need for, among others, evaluation tools regarding ocean biogeochemistry and subsurface processes as well as observational data on e.g. soil moisture or nutrient flow from rivers into the Arctic.
The workshop concluded with the plan to organise follow up workshops to bridge the reported gaps and to coordinate new opportunities across H2020 projects with focus on high-resolution modeling, process-level understanding and ensemble simulations. A summary of actions to take and suggestions that were raised during the meeting are as follows:
- It may be useful to draft a document with definitions of metric, diagnostic, diagnosis, constrains.
- It may be useful having a simple table describing for each project: the duration of the project, the regions of interest, the timescales of interest, the diagnostic of interest.
- It may be useful to have a catalogue of available tools for processing model output: ESMValTool, CIS, AQ4Seas, PAGO (for ocean) (e.g. public wiki where researchers can drop the description of these packages or describe what they liked or not. A Wiki or blog or similar could also help to streaming communication among projects).
- Thoughts should be put on how to integrate these tools or have them converging to plan investing on a long term solution.
- Each project should nominate on person responsible for ESMValTool development (not applicable for Blue-Action) to coordinate actions (TOP DOWN).
- There should a list of topics and identified scientists for each topic and projects that should be put in touch and for each topic one person per project should be responsible to develop software that is ESMVal-ready. Topics emerging:
- Tracking meso-scale, extra-tropical cyclones, storms, polar lows.
- Ocean diagnostics are vastly missing in ESMVal and common development can happen around physical ocean processes that are of use for explaining the biogeochemistry (e.g. upwelling).
Overall, these were two very exciting days loaded with insights from many Earth system modeling and evaluation perspectives guiding towards more efficient and coordinated collaborations across EU projects.
Presentations and list of attendees are available at the online workshop site (note the files are password protected) at https://ec.europa.eu/eusurvey/runner/H2020-climate-modelling. Get it touch ([email protected]) if you wish to access the files.