Splinter - Space Weather Metrics, Verification & Validation

A. Glover, S. Bingham, P. Jiggens, P. Wintoft
Thursday 20th, 16:30 - 18:00


In order to provide reliable services to end-users, it is crucial to understand the strengths and potential limitations of the various elements underpinning those services. This includes the assumptions and algorithms on which models are based, and also the reliability of the associated infrastructure: e.g. data systems, space and ground-based measurement infrastructure. This also includes situations during extreme solar and geomagnetic conditions. At the present time, within the space weather community, prototype services frequently operate as capability demonstrators and a full verification of their ability to reproduce/predict elements of the space environment under a range of space weather conditions, from the moderate to the extreme, has yet to be completed. Forecast accuracy has recently been addressed by a number of separate activities, but as yet a community-wide consensus on how to address this question and provide relevant information supporting end-users, service developers and modellers themselves for the wide range of models and domains involved has not been reached.

This splinter encourages dialogue between modellers, service developers and stakeholders including service users. The meeting will review current activities and initiatives ongoing both in Europe and internationally and investigate potential applicability of well established processes already in place within the meteorological community. We discuss space weather verification & validation needs for the current generation of activities under development and in planning in Europe, together with opportunities for modellers and scientific community to further participate in the space weather endeavour.

Program

16:30Introduction
16:35Towards a verification framework for forecasts of different centres, Andy Devos, J. Andries, C. Verbeeck, D. Berghmans (STCE-ROB) Belgium.
16:45Verification of extreme event forecasts, Institute of Space Physics, Peter Wintoft, IRF, Sweden
16:50SWENET Index Quality Statistics & Database Assessment, Alexi Glover, ESA.
16:55Performance Verification of Solar Flare Prediction Techniques: Present Status, Caveats, and Expectations, Manolis Georgoulis, Academy of Athens, Greece
17:00The use of modified Taylor diagrams for comparing geophysical models, Matthew Angling, University of Birmingham, UK.
17:10Translating verification experience from Meteorology to Space Weather, Suzy Bingham, Met Office, UK.
17:20Lessons learned from CCMC-led community-wide model validation challenges. Outlook on international coordination of M&V activities, Maria Kuznetsova, CCMC , USA.
17:30Discussion.
18:00Finish.


Discussion topics will include:
  1. What metrics and validation techniques are required in the current space weather landscape?
  2. What are the key challenges currently in model and forecast benchmarking?
  3. What direction should the space weather community be taking?
  4. What actions can agencies and organisations take in order to support a wider space weather validation effort?
  5. How to establish agreed realistic model/service targets to encourage targeted development and prototyping?
  6. What targeted actions would encourage groups not currently involved to further participate in space weather validation activities?
Back to the splinter overview