Topical Discussion Meeting - Verification and validation – from initial concept to operational service

Suzy Bingham (Met Office); Alexi Glover (ESA); Mark Dierckxsens (BIRA-IASB); Sean Elvidge (University of Birmingham); Federico Da Dalt (Rhea System GmbH for ESA); Gareth Dorrian (University of Birmingham)
Wednesday 20/11, 11:15-12:30

This Topical Discussion Meeting will focus on methodologies and results from the evaluation of space weather models, systems and forecasts. Rigorous discussion on this topic will be encouraged between modellers, application developers, service providers and end users, following a selection of highlight presentations.

Verification and validation of space weather models, systems and forecasts provide an understanding of performance of these elements in reproducing/predicting the environment during quiet to extreme space weather conditions. Assessment is of great importance to modellers, service providers and end users, to enable further improvement and to gain confidence in a service.

Over the past few years, activities in this area have gained momentum in Europe and internationally. Discussion will be encouraged particularly on results from coordinated model/forecast testing, and from operational model/forecast evaluation. We will welcome discussion on verification and validation techniques including frameworks in development or already in use which facilitate these efforts. Also, discussion on the challenges we face in developing a community-wide assessment of space weather forecasting capabilities. This TDM will build upon discussions within international activities such as COSPAR PSW1.

The goal of this TDM is to showcase progress in both individual and coordinated validation activities, including results of targeted validation campaigns, in order to inspire further progress towards international community consensus both (a) on addressing accuracy and suitability allowing the transition of models to operations and (b) providing continuous validation for products and information provided to users of space weather services.

Program

11:15 Introduction
* Recap of previous meetings, Suzy Bingham (Met Office)

11:16 Frameworks & coordinated validation
* ESA SWE activities & ESC campaigns, Alexi Glover (ESA) & Federico Da Dalt (Rhea System GmbH for ESA)
* COSPAR panel space weather (PSW) activity, Alexi Glover (ESA)
* Discussion: challenges & solutions towards community-wide consensus on continuous assessment of space weather predictive capabilities

11:40 Individual validation results - accuracy & suitability - allowing transition of models to operations
* EUHFORIA solar wind validation results, Jasmina Magdalenic (SIDC)
* Validation of Dynamic Atmospheric Shower Tracking Interactive Model Application (DYASTIMA), Pavlos Paschalis (University of Athens)
* Validation of Space Weather Modeling Framework (SWMF), Norah Kaggwa Kwagala (University of Bergen)
* Discussion: consensus on addressing accuracy & suitability to allow transition of models to operations

12:00 Long-term performance & continuous validation
* Tracking progress of models over a long period of time, Yihua Zheng (NASA/GSFC)
* Developing a single space weather model prediction index to monitor performance, Siegfried Gonzi (Met Office)
* Discussion: consensus on providing long-term performance & continuous validation of products to users

12:15 Validation approaches - managing expectations
* Limitations of Kp index forecasting, Yuri Shprits (GFZ-Potsdam)
* Limitations to CME arrival forecasting, Mateja Dumbovic (Hvar Observatory)
* Discussion: challenges & solutions in managing expectations of performance of products

12:30 Finish