Session - Transitioning space weather research to operations: learning from the NWP experience

David Jackson, Suzy Bingham, Mauro Messerotti

Operational numerical weather prediction (NWP) systems have a long heritage, and the process of applying research in observations, modelling, data assimilation and verification to continually improve these operational systems is well understood. By contrast, although there is also a long heritage of space weather research, operational space weather prediction is in its infancy. There is considerable scope to identify and integrate best practices from operational NWP systems into operational space weather systems. We welcome contributions covering all aspects of this topic, including for example: - real time acquisition of observations and adaptation of research-focused instruments to operational use - robustness, reliability and testing of space weather prediction models - data assimilation methods for space weather prediction - skill scores and near real time verification - system resilience (eg backup data streams, 24/7 operations) - the role of the WMO and other meteorological bodies in developing operational space weather systems.

Talks and First Class Posters
Thursday November 20, 09:00-13:00, room Mosane

Poster Viewing
Thursday November 20, 11:00-11:30, area in front of room Mosane

Talks and First Class Posters

1 Oral 9:00 am The ESA Virtual Space Weather Modelling Centre – Phase 1
      Poedts, S1; Lapenta, G1; Lani, A2; Deconinck, H2; Diet, F3; Ho, N-D3; Mihalache, N3; Heynderickx, D4; De Keyser, J5; Crosby, N5; Rodriguez, L6; Van der Linden, R6; Jiggens, P7; Hilgers, A7
      1KU Leuven; 2VKI; 3SAS; 4DHConsultancy; 5BISA; 6ROB; 7ESTEC
      The ESA ITT project (AO/1-6738/11/NL/AT) to develop Phase 1 of a Virtual Space Weather Modelling Centre had the following objectives and scope:  1. The construction of a long term (~10 yrs) plan for the future development of a European virtual space weather modelling centre consisting of a new ‘open’ and distributed framework for the coupling of physics based models for space weather phenomena;  2. The assessment of model capabilities and the amount of work required to make them operational by integrating them in this framework and the identification of computing and networking requirements to do so.  3. The design of a system to enable models and other components to be installed locally or geographically distributed and the creation of a validation plan including a system of metrics for testing results.    The consortium that took up this challenge involves: 1)the  Katholieke Universiteit Leuven (Prime Contractor, coordinator: Prof. S. Poedts); 2) the Belgian Institute for Space Aeronomy (BIRA-IASB); 3) the Royal Observatory of Belgium (ROB); 4) the Von Karman Institute (VKI); 5) DH Consultancy (DHC); 6) Space Applications Services (SAS).   The project started on May 14 2012, and was completed in August 2014. Phase 1A of the project entailed a review of relevant technologies and models, the user requirements, and the system design. Phase 1B focused on the development of the prototype system using High Level Architecture (HLA), the installation of example physics-base space weather models, and development of coupling scripts/toolkit and validation . The final report will be presented incl. the architecture decisions made, the High Level Architecture framework, the current models that are integrated as well as the model couplings that have been installed.  The prototype VSWMC will be demonstrated.   The VSWMC system is being developed under ESA's General Support Technology Programme (GSTP) and is intended for transition to an operational system as part of the ESA Space Situational Awareness (SSA) programme.
2 Oral 9:15 am Accelerating Implementation of Advanced Space Weather Models and Forecasting Systems in Operations.
      Kuznetsova, M1; Maddox, M1; Rastaetter, L1; Mays, L1; Wiegand, C1; Chulaki, A1; Macneice, P1; Mullinix, R1; Pulkkinen, A1; Shim, J  S1; Taktakishvili, A1; Zheng, Y1
      1NASA Goddard Space Flight Center
      The Community Coordinated Modeling Center (CCMC, http://ccmc.gsfc.nasa.gov) hosts an expanding collection of state-of-the-art space weather models developed by the international space science community. Over the years the CCMC acquired the unique experience in preparing complex models and model chains for operational environment and developing tools and systems for space weather analysis, forecasting and validation.  The presentation will focus on successes and challenges in assessment of operational potential of research models and transitioning space weather research to operations. We will discuss opportunities for international collaborative activities that will facilitate development of improved space environment prediction capabilities and accelerate implementation of advanced models and forecasting systems in space weather operations.
3 Oral 9:30 am Toward Integrated Real-Time Modeling System for Heliospheric Space Weather
      Odstrcil, D1; Jackson, B2; Jian, L3; Luhmann, J4; Mays, L5; Rouillard, A6; Taktakishvili, A7; Xie, H5; Yu, H-S2
      1GMU and NASA/GSFC; 2UCSD; 3UMd and NASA/GSFC; 4UCB; 5CUA and NASA/GSFC; 6IRAP; 7NASA/GSFC
      The WSA-ENLIL-Cone modeling system at NASA/CCMC/SWRC and NOAA/SWPC enables faster-than-real time simulations of corotating and transient disturbances. This system uses a “single-map” approach that provides unchanged corotating background solar wind during a 5-day long prediction at Earth. We will present results achieved within the HelioWeather project which aims to improve current capabilities. We have modified ENLIL to ingest the time series of coronal maps produced by WSA utilizing GONG magnetograms and by UCSD’s tomographic reconstruction of the interplanetary scintillation (IPS) detected by the radio arrays at STELab. This enhancement allows ENLIL to run in a time-dependent fashion that is more realistic. We also enhanced the modeling system by automatic detection of shock parameters that are magnetically connected to planets and spacecraft, and by calculating the synthetic white-light images and constructing the time-elongation plots (“J-maps”). These enhancements provide additional possibilities for validation of the modeling system. We simulated over 700 coronal mass ejections (CMEs) that were operationally fitted by NASA/SWRC in 2011-2013. We will compare these predictions with in-situ measurements of solar wind parameters by ACE and STEREO spacecraft and with the J-maps observed by STEREO Heliospheric Imagers. This work has been sponsored by the NASA-NSF Partnership for Collaborative Space Weather Modeling (PCSWM) Program,
6 Oral 09:45 am FMI’s Operative Space Weather Service in LUOVA 24/7 Natural Hazards Warning System
      Laitinen, T1; Tanskanen, E1; Viljanen, A1; Palmroth, M1; Kauristie, K1; Säntti, K1
      1Finnish Meteorological Institute
      The Finnish Meteorological Institute (FMI) operates a national early warning and information system on natural hazards, LUOVA. The service is directed to Finnish government and safety authorities and provides information on severe weather, flooding, earthquakes, and other potentially dangerous events in Finland and abroad. System is based on an expert network of hydrologists, meteorologists, oceanographers, seismologists and space research scientists producing and providing information on natural disasters on 24/7 bases. Finnish Meteorological Institute is acting as a coordinator between actors and end-users and is responsible of operating the 24/7 monitoring and information dissemination system. Space weather alerts, warnings and predictions have been provided for LUOVA customers since spring 2013.   LUOVA exemplifies the synergies that can be found between established tropospheric weather services and new developing space weather services. Integration of space weather warnings to LUOVA system shows how research expertise has been transitioned to an operative service cost-effectively. The 24/7 duty service, whose cost exceeds the resources available to space weather services in Finland, is provided by meteorologists. They are supported by FMI space research scientists, who have prepared simplified monitoring tools, instructions and guidelines for the LUOVA operators. Research scientists also provide 24/7 consultation to the LUOVA operators in all major or exceptional space weather events.   In this presentation we will describe the LUOVA warning system in general and specifically the space weather warnings. We will also show browser-based monitoring and visualization tools used by LUOVA operators and educational material from the introduction courses provided to the on-duty meteorologists during the initialization period. We will illustrate the warning issuing process with examples including a solar radiation storm, a geomagnetic storm and the exceptional case of GOCE satellite re-entry.
4 Oral 10:00 am Real-Time Global Magnetospheric Simulation Predicts Space Weather Effects on GOCE Decay
      Laitinen, T1; Honkonen, I2; Palmroth, M1; Janhunen, P1
      1Finnish Meteorological Institute; 2Finnish Meteorological Institute / presently at NASA/GSFC
      The only European global magnetohydrodynamic simulation of the coupled magnetosphere-ionosphere system, GUMICS-4, has been used extensively for magnetospheric research at the Finnish Meteorological Institute (FMI) for two decades. Recently a new faster parallelized version of the code, GUMICS-5, has been developed with a view to employing it in operational space weather nowcasting. A fully automated chained simulation run system, similar to the ones used in numerical weather prediction, has also been developed. Once made fully operational, these codes will enable us to provide continuous faster-than-real-time simulations on numerous space weather effects. With ACE solar wind data as input, we reach a forecast time of approximately 15 to 20 minutes. The new code GUMICS-5 is currently in validation phase, and we expect to make the simulation system operational in 2015.  The simulation scheme was tested for the first time during the decay of ESA’s GOCE satellite in October-November 2013. During the 20-day period when the satellite gradually lost altitude due to air drag, several moderate magnetic storms occurred. Our preliminary results indicated ionospheric heating events which were estimated to cause a temporary 10–20 % increase in air density at GOCE’s altitude. GOCE orbital data shows simultaneous temporary acceleration of altitude loss with the same relative order of magnitude. This comparison indicates that space weather simulations have the potential to enhance orbit determination for satellites and space debris on low orbits through nowcasting of air drag variations.  Acknowledgements. We thank Prof. Dr. Heiner Klinkrad from ESA/ESOC for providing GOCE’s orbital data.
5 Oral 10:15 am The Nowcast Model for Low Energy (< 200 keV) Electrons in the Inner Magnetosphere
      Ganushkina, N1; Heynderickx, D2
      1Finnish Meteorological Institute; 2DH Consultancy BVBA, Leuven, Belgium
      We present the nowcast model for low energy (< 200 keV) electrons in the inner magnetosphere which was developed from the physics-based, research-oriented Inner Magnetosphere Particle Transport and Acceleration model (IMPTAM). The IMPTAM traces ions and electrons with arbitrary pitch angles from the plasma sheet to the inner L-shell regions with energies reaching up to hundreds of keVs in time-dependent magnetic and electric fields. The tracing of a  distribution of particles is conducted in the drift approximation  under the conservation of the first and second adiabatic invariants. Liouville's theorem is used to gain information of the entire distribution function. The operational version of IMPTAM works online under the completed SPACECAST project (http://fp7-spacecast.eu) and on-going SPACESTORM project. The nowcast model is driven by the real time solar wind and IMF parameters with 1 hour time shift for  propagation to the Earth's magnetopause, and by the real time Dst index. Real time geostationary GOES 13 or GOES 15 (whenever which available) MAGED data on electron fluxes in three energy channels (30-50 keV, 50-100 keV, 100-200 keV) are used for comparison and validation of IMPTAM running online. Working online near-real time nowcast of low energy electrons is very important tool and it provides highly valuable output. Low energy electron fluxes are very important to specify when hazardous satellite surface charging phenomena are considered. These fluxes constitute the low energy part of the seed population which is critically important for radiation belt dynamics. We demonstrate the model performance by analysing the metrics, as a descriptive statistics given by nRMSE (Normalized root-mean-square error), and as binary events including hit rates, false alarm rates and calculated Heidke Skill Scores.
1 Highlighted poster 10:30 AM Provision of Web Based Space Weather Services
      Burnett, C1; Bingham, S1
      1Met Office
      The E-poster will illustrate how the UK Met Office has approached the provision of space weather data and forecast services on its public facing website and through industry specific web pages.   Users of space weather forecasts, warnings, alerts and general information range from scientific experts in the field, government and industry partners who know enough to manage the impact of a space weather event,  to the general public, who may or may not, have heard of space weather.  The Met Office has a responsibility, as the UK risk owner, to raise the level of understanding about space weather, provide an operational forecast and warning service and help organisations be prepared for potential impacts.  The poster will explore the challenges the Met Office has faced in meeting the needs of diverse community of internet users and providing relevant data in a timely method suitable for their needs.
2 Highlighted poster 10:32 AM Improving Operational Geomagnetic Index Forecasting
      Billingham, L1; Kelly, G1
      1British Geological Survey
      Space weather prediction is moving from an era of pure curiosity-driven research into an era of 24/7 operations. The interest in space weather forecasts has never been greater, with society becoming ever more reliant upon technology and infrastructure which are potentially at risk. Amongst space weather hazards, geomagnetic storms are potential threats to power-grid infrastructure, communication systems and oil and gas operations.  Geomagnetic indices capture the severity of magnetic storms by summarising magnetic activity at spatially disparate locations.  They have become almost ubiquitous as parameterisations of storm-time magnetic conditions and are required inputs for radiation belt , ionospheric and neutral atmosphere models.  We present the first results from a study aiming to provide operational geomagnetic index prediction that is: robust and reliable, has high cadence, runs fast enough for real-time operations , and is accurate forecasting up to three days ahead. The predictive power of autoregressive and machine-learning techniques applied to combinations of solar, solar wind and geomagnetic data is investigated.  The predictions presented will ultimately form part of the British Geological Survey’s space weather forecast operations.
3 Highlighted poster 10:34 AM Ionospheric Storm Index over South African Region: Initial Attempt
      Tshisaphungo, M1; McKinnell, L-A1; Habarulema, J  B1
      1South African National Space Agency 
      The South African National Space Agency (SANSA) operates the Space Weather Regional Warning Center (RWC) for Africa.  The center exists within the SANSA Space Science Directorate located at Hermanus, South Africa. The main services offered by the Space Weather Center (SWC) include; High Frequency (HF) radio wave propagation predictions and a constant monitoring of the state of the ionosphere over the African region.  Under adverse space weather conditions, the ionosphere becomes more variant affecting operations of technological systems which depend on ionospheric radio waves propagation. These ionospheric variations can be observed in ionosonde and Global Positioning System (GPS) observations. However, practically an activity index which provides a quick and a proxy measure of the complex ionospheric behavior is valuable for operational space weather monitoring and forecasting. This paper presents the initial results and progress in developing an ionospheric index to be used over the African region.
4 Highlighted poster 10:36 AM Real-time Space Environment at MSU’s Space Monitoring Data Center
      Kalegaev, V1; Myagkova, I1; Barinova, W1; Bobrovnikov, S1; Dolenko, S1; Mukhametdinova, L1; Shiroky, V1; Shugay, Y1
      1Skobeltsyn Institute of Nuclear Physics / Moscow State University
      Space monitoring data center at Moscow State University provides operational information on radiation state of the near-Earth space. Internet portal http://swx.sinp.msu.ru/ provides access to the actual data characterizing the level of solar activity, geomagnetic and radiation conditions in the magnetosphere and heliosphere in the real time mode. Operational data coming from space missions (ACE, GOES, ELECTRO-L1, Meteor-M1) at L1, LEO and GEO and from the Earth’s surface are used to represent geomagnetic and radiation state of near-Earth environment. The models of space environment working in autonomous mode are used to generalize the information obtained from observations on the whole magnetosphere. Interactive applications and operational forecasting services are created on the base of these models. They automatically generate alerts on particle fluxes enhancements above the threshold values, both for SEP and relativistic electrons using data from LEO orbits. Special forecasting services give short-term forecast of SEP penetration to the Earth magnetosphere at low altitudes, as well as relativistic electron fluxes at GEO. Velocities of high-speed streams in solar wind on the Earth orbit are estimated with advance time of 3-4 days on the basis of automatic calculation of the areas of coronal holes detected on the images of the Sun obtained by the SDO satellite. By means of neural network approach, Dst and Kp indices online forecasting up to 1.5 hours ahead by the parameters of solar wind and the interplanetary magnetic field measured by ACE satellite, is carried out. Visualization system provides representation of experimental and modeling data in 2D and 3D.
5 Highlighted poster 10:38 AM Real-­time Ensemble Forecasting of Coronal Mass Ejections using the WSA-ENLIL+Cone Model
      Pulkkinen, A1; Mays, L2; Taktakishvili, A2; Kuznetsova, M1; Zheng, Y1; Odstricil, D3; MacNeice, P1; Rastaetter, L1
      1NASA; 2CUA; 3GMU
      Ensemble forecasting of coronal mass ejections (CMEs) provides significant information in that it provides an estimation of the spread or uncertainty in CME arrival time predictions. Real-time ensemble modeling of CME propagation in the heliosphere is performed by forecasters at the Space Weather Research Center (SWRC) using the WSA-ENLIL cone model available at the Community Coordinated Modeling Center (CCMC).  To estimate the effect of uncertainties in determining CME input parameters on arrival time predictions, a distribution of n (routinely n=48) CME input parameter sets are generated using the CCMC Stereo CME Analysis Tool (StereoCAT) which employs geometrical triangulation techniques. These input parameters are used to perform n different simulations yielding an ensemble of solar wind parameters at various locations of interest (satellites or planets), including a probability distribution of CME shock arrival times (for hits), and geomagnetic storm strength (for Earth-directed hits).  Ensemble simulations have been performed experimentally in real-time at the CCMC/SWRC since January 2013. We present the results of ensemble simulations for a total of 18 CME events in 2013, 13 of which were performed in real-time. For ensemble runs containing hits, one can check if the observed CME arrival was within the range of ensemble arrival time predictions. The average arrival time prediction was computed for each of the 15 ensembles predicting hits and using the actual arrival time, an average absolute error of 8.1 hours was found for all 15 ensembles, which is comparable to current forecasting errors. Some considerations for the accuracy of ensemble CME arrival time predictions include the importance of the initial distribution of CME input parameters, particularly the mean and spread. When the observed arrivals are not within the predicted range, this still allows the ruling out of prediction errors caused by tested CME input parameters. Prediction errors can also arise from ambient model parameters such as the accuracy of the solar wind background, and other limitations.   Additionally the ensemble modeling setup was used to complete a parametric event case study of the sensitivity of the CME arrival time prediction to free parameters for ambient solar wind model and CME. The parameter sensitivity study suggests future directions for the system, such as running ensembles using various magnetogram inputs to the WSA model.
7 Oral 11:30 am Transitioning Space Weather Models to Operations at the UK Met Office
      Bingham, S1; Burnett, C1; Jackson, D1; Gibbs, M1
      1UK Met Office
      The UK Met Office Space Weather Operations Centre (MOSWOC) provides a platform for running space weather models operationally.  WSA-Enlil & REFM are successfully running on the MOSWOC system and other models in the process of being implemented include DRAP and MIDAS.  When models are run operationally, this implies a robust infrastructure, with 24/7 support and provision of model results to aid the production of timely alerts, warnings & forecasts by expert space weather forecasters.  The processes and challenges involved in transitioning these models to operations are described in this presentation.  Many challenges are overcome to successfully implement such models, including re-writing code, acquiring real-time observations, running on supercomputers & managing firewalls.  Our experiences in running NWP systems at the Met Office is used to illustrate some of the differences between space weather research systems and operational NWP systems, and is used to point the way to future operational developments, such as real-time verification and intercomparison with model products from other centres.
8 Oral 11:45 am Space Weather Services Based on the Energetic Particle Telescope (EPT) Data
      Benck, S1; Cyamukungu, M1; Borisov, S1
      1Center for Space Radiations / Université Catholique de Louvain
      The forecast of energetic particle fluxes on time scales of hours to weeks, at a given position in space, can be achieved on the basis of experimentally determined particle lifetimes and on real-time measurements of contamination-free spectra. Such elaborated measurements can be performed by the Energetic Particle Telescope (EPT) that was launched on the 7th May 2013 onto a circular quasi-polar sun synchronous orbit at 820 km altitude and which provides real-time measurements of energy spectra of electrons (0.5 - 10 MeV), protons (9 - 300 MeV), and α- particles (38 - 1000 MeV). This presentation starts with a brief description of the EPT and its general performances. Then the EPT data exploitation plan will be presented (proton anisotropies, analysis of event-related particle flux enhancement and subsequent flux decay, comparison to Van Allen Probes data, AP8/AE8 and  IRENE model) with highlight on the development of space weather (SW) services based on measured flux lifetimes along with an example of forecast of electron and proton fluxes at LEO. Finally, a short description of the web site offering EPT based SW services will be given.
9 Oral 12:00 pm Real-time Acquisition of Plasmaspheric Electron Densities for Space Weather Predictions
      Lichtenberger, J1; Jörgensen, A2; Koronczay, D1; Ferencz, C1; Hamar, D1; Steinbach, P1; C., Mark3; Rodger, C4; Sannikov, D5; Cherneva, N5
      1Eötvös University; 2New Mexico Institute of Mining and Technology; 3British Antarctic Survey; 4University of Otago; 5Institute of Cosmophysical Research and Radio Wave Propagation
      The Automatic Whistler Detector and Analyzer Network (AWDANet)  is able to detect and analyze whistlers in quasi-realtime and can provide equatorial electron density data. The plasmaspheric electron densities  are key parameters for plasmasphere models in Space Weather related investigations, particularly in modeling charged particle accelerations and losses in Radiation Belts. The global AWDANet detects millions of whistlers in a year. The system has been recently completed with automatic analyzer capability in PLASMON (http://plasmon.elte.hu) project. It is based on a recently developed whistler inversion model, that opened the way for an automated process of whistler analysis, not only for single whistler events but for complex analysis of multiple-path propagation whistler groups. The network operates in quasi real-time mode since mid-2014,  fifteen  stations  provide equatorial electron densities that are used as inputs for a data assimilative plasmasphere model but thay can also be used directly in space weather research and models.
10 Oral 12:15 pm Detection of Space Weather Events in Magnetic Data
      Coïsson, P1; Heumez, B1; Telali, K1; Lalanne, Xavier1; Luc, T1; Luc, Ted1; Maury, V1; Chambodut, A2
      1Institut de Physique du Globe de Paris; 2Institut de Physique du Globe de Strasbourg
      The magnetic signature of space weather events allows to follow the evolution of magnetic storms from their commencement to the end of the recovery phase. The French Bureau Central du Magnetisme Terrestre manages  16 magnetic observatories distributed all over the world and collects data from for most of them in real time. Detection of sudden storm commencement is operational and is disseminated to subscribers through e-mail. This service will be developed further to provide additional information about the main characteristics of the undergoing storm. Another focus of research is the equatorial region. IPGP maintains a chain of magnetometers in West Africa, WAMNET, and a second one will be available by the end of 2014 in East Africa, with a station in Djibouti, coupled with the Addis Ababa observatory. A couple of magnetometers, one located below and one outside the equatorial electrojet can infer the ionospheric ExB drift, responsible of the development of the equatorial anomaly.  We present the result obtained from these chains of magnetometers and the related space weather products.
11 Oral 12:30 pm An operational real-time mid-latitude local disturbance index for space weather purposes
      Guerrero, A1; Cid, C1; Palacios, J1; Saiz, E1; Cerrato, Y1
      1University of Alcala
      Quantification of local geomagnetic disturbances plays an important role when predicting or evaluating potential impact that space weather events have on technology. However an accurate determination is not achieved applying systematic methods because several problems arise: procedures to determine magnetometer baselines are not standardized and the methods differ from different latitude locations and different purposes. Moreover, procedures to obtain the index in real-time differ from the procedures to obtain definitive output, usually because of their reliance on data which are not published immediately. On the other hand, the procedures at mid-latitude locations have to take into account the variability of the solar quiet daily curves. We present the 1-minute local disturbance index of Spain (mid-latitude) which is running in real-time mode to assess the relative importance of space weather disturbances for the Spanish National Power Grid Company (REE). The index, which uses data from the geomagnetic observatory of San Pablo-Toledo (SPT), offers with minimum delay  time from the measurements at SPT, the geomagnetic disturbance observed in the Spanish territory. The index is a space weather service offer by the University of Alcala (UAH-SWS) classifying geomagnetic disturbance events in three categories using traditional nomenclature (extreme, intense and moderate). We describe how we sort out the problems in order to offer an operational mid-latitude local disturbance index which gives real-time as well as historical data using the same procedure.
12 Oral 12:45 pm A Space Weather Index for the Radiation Field at Aviation Altitudes
      Meier, M1; Matthiae, D1
      1German Aerospace Center
      The additional dose contribution to the radiation exposure at aviation altitudes during Solar Particle Events (SPEs) has been a matter of concern for many years. After the Halloween storms in 2003 several airlines began to implement mitigation measures such as rerouting and lowering flight altitudes in response to alerts on the NOAA S-scale regarding solar radiation storms. These alerts are based on the integral proton flux above 10 MeV measured aboard the corresponding GOES-satellite which is operated outside the Earth’s atmosphere in a geosynchronous orbit. This integral proton flux has, however, been proved to be an insufficient parameter to apply to the radiation field at aviation altitudes without an accompanying analysis of the shape of the energy spectrum. Consequently, false alarms and corresponding disproportionate reactions ensued. Since mitigating measures can be quite cost-intensive, there has been a demand for appropriate space weather information among responsible airline managers for about a decade. Against this background, we propose the introduction of a new Space Weather index D, based on dose rates at aviation altitudes produced by solar protons during solar radiation storms, as the relevant parameter for the assessment of corresponding radiation exposure.  J. Space Weather Space Clim. 4 A13 (2014) DOI: http://dx.doi.org/10.1051/swsc/2014010