4-9 September 2022, Bonn, Germany
Enter zoom meeting


SPARK session
Dealing with Uncertainties

Dealing with Uncertainties

Weather forecasts have matured substantially in providing reliable probabilistic predictions, with a useful quantification of forecast uncertainties. Including this information in the communication of forecasts and warnings, and integrating it into downstream models and decision-making processes has become increasingly common practice.

Including uncertainties not only implies the interpretation of ‘raw’ uncertainty information in ensemble forecasts, their post-processing, and visualization, but also the integration of a wide range of non-meteorological aspects such as vulnerability and exposure data to estimate risk and the social, psychological and economic aspects which affect human decision-making.

In this session, we aim to support a holistic perspective on issues that arise when making use of uncertainty information of weather forecasts in decision processes and applications.
We encourage contributions that investigate the application and interpretation of uncertainty information along any of, but not limited to, the following questions:
- How does the quality of the final decision depend on forecast uncertainty and uncertainty from non-meteorological parts of the decision process?
- Where, along the chain from raw forecast uncertainty to the final decision, do the largest uncertainties arise?
- How is the uncertainty information (e.g., from ensemble prediction systems, multi-models etc.) propagated through the production chain up to the final decision?
- How can we tailor information about forecast uncertainty, and its representation, to a given user group, decision process or application?
- How is uncertainty represented best in a given case (e.g., as ensemble members, PDFs, or worst/best case) to reduce complexity and computational or cognitive cost?
- How can we identify the most suitable representation for different user-groups and decision processes?
- How can we incorporate vulnerability and exposure data in a risk-based decision framework?
- How can we evaluate and quantify the value of uncertainty information for decision making in different contexts?
- What strategies help the end-user to interpret the uncertainty in forecasts when making informed decisions?
- What are the benefits of impact-based or risk-based forecasts and warnings in decision-making (including for disaster risk reduction)?
- How can the interaction between scientists and end-users help to overcome reservations about uncertainty forecasts?
- How to apply in weather communication evidence of sociological and psychological factors that affect the interpretation of forecast uncertainties?
- How do we convince weather service providers to include uncertainty information when faced with their concerns that people will not understand it or that it undermines confidence in their services?

Conveners: Nadine Fleischhut, Vanessa Fundel, Jelmer Jeuring, Bruno Joly, Mark A. Liniger, Ken Mylne, Anders Doksæter Sivle
| Mon, 05 Sep, 14:00–15:29 (CEST)|Room HS 7
| Attendance Mon, 05 Sep, 16:00–17:30 (CEST) | Display Mon, 05 Sep, 08:00–18:00|b-IT poster area

Mon, 5 Sep, 14:00–15:30

Chairpersons: Ken Mylne, Vanessa Fundel, Jelmer Jeuring

Intro by the conveners

Onsite presentation
Corinna Möhrlen et al.

The number one target since the Ukraine war has been shaking our world in February 2022 and shown how much impact energy dependencies have on our life, is to increase the amount of renewable energy in an unseen effort and pace. Without the tools to predict and act upon the enormous amounts of variable generation to come, the electric grid will be difficult to keep stable. On top of the energy crisis, we start seeing impacts of the climate crisis with increasing temperatures that change weather systems, causing more extreme weather events that again have impact on the generation pattern from wind and solar driven generation units. The associated uncertainty of (1) strongly increasing penetration of renewable energy generation and (2) increasing amount of extreme weather events call for forecasting tools that take these uncertainties into account and help system operators, traders and balance responsible parties to be able to act upon uncertainties rather than being surprised by them. Mitigating balancing costs and increasing security of supply will have to be at the forefront of everybody’s top priorities.

The IEA Wind initiative “Probabilistic Forecasting Games and Experiments” is a collaboration with the Max-Planck Institute for Human Development’s WEXICOM project. The objectives of this initiative are (1) to empirically investigate the psychology behind adoption or refusal when dealing with uncertainty forecasts and (2) to use the empirical results to understand how weather and generation forecast providers have to present and communicate uncertainty forecasts to end-users for them to be able to exploit the benefits of the enhanced information in their decision process. Although we simplify our experiments, they are designed as realistic scenarios for many decision-makers in the industry. We are encouraged from the participant’s feedback that this type of gamification of a problem is received well as exemplary applications for the use and adoption of probabilistic forecasts into decision processes. The learning-by-doing strategy in a safe environment may not reproduce the entire context of a decision-making problem in a specific operational environment, but on the other hand it provides a platform to test different ways of introducing people with a complex topic, train and teach awareness for the challenges and benefits that come with the advanced technologies.

There are still many open questions that we need to answer, such as how decisions depend on the structure of the decision context, the communication and graphical or textual presentation, and many more. In this context, we will present the results of our second experiment, which is also dealing with the decision-making in extreme wind cases that can lead to power shutdown of wind turbines, and hope with this to inspire a vivid discussion on the topic in the session.

How to cite: Möhrlen, C., Fleischhut, N., Escallon, A., and Giebel, G.: Behavioural decision-making Experiments in weather-driven Energy Systems, EMS Annual Meeting 2022, Bonn, Germany, 5–9 Sep 2022, EMS2022-158, https://doi.org/10.5194/ems2022-158, 2022.


Online presentation
Jens Grundmann et al.

Reliable warnings and forecasts of extreme precipitation and resulting floods are important for disaster mangers to initiate flood defence measures. Thus, disaster managers are interested in extended lead times, which can be obtained by employing forecast of numerical weather models as driving data for hydrological models. Especially for small catchments, warning and forecasting systems are challenging due to the short response time of the catchments and the uncertainty of meteorological forecasts. To portray the inherent uncertainty of weather model output, ensemble hydro-meteorological forecasts can be used.

By this contribution, we present our operational web-based demonstration platform for ensemble hydrological forecasting in small catchments of Saxony, Germany (http://howa-innovativ.hydro.tu-dresden.de/WebDemoLive/). We use the ICON/COSMO-D2-EPS product of the German Weather Service (DWD), which provides an ensemble of 20 members each three hours.

Each member is evaluated regarding specific extreme precipitation thresholds for predefined hydrological warning regions. If these thresholds are exceeded in a specific region, rainfall-runoff models for the associated catchments are started to propagate the meteorological uncertainty into the resulting runoff, followed by statistical post processing and visualization. Forecasts are updated each hour if new precipitation observations by the radar product RADOLAN-RW of the DWD is available and they are processed for lead times up to 27 hours.

Different options for the visualization of the uncertainty information in precipitation and resulting runoff were discussed and evaluated by a series of workshops with locally responsible civil protection forces and water authorities. This leads to the current design of the web-based demonstration platform in an iterative process.  

The web-based demonstration platform is established for three pilot regions with different hydrological settings in Saxony, Germany. Besides layout and technical issues, results and experiences with the demonstration platform are presented for observed extreme events in the small pilot regions in 2018 and 2021.

How to cite: Grundmann, J., Six, A., and Philipp, A.: Communicating uncertainties for flood warning in small catchments using ensemble hydrological forecasting, EMS Annual Meeting 2022, Bonn, Germany, 5–9 Sep 2022, EMS2022-679, https://doi.org/10.5194/ems2022-679, 2022.


impulse 1: Bringing ensembles to the heart of Met Office operations, Ken Mylne and Nigel Roberts

impulse 2: How to communicate uncertainty in weather warnings to the public? Put it into perspective! Katja Schulze, Nathalie Popovic, and Nadine Fleischhhut

impulse 3: Factors behind local communities resilience to extreme weather events, Adam Choryński et al

impulse 4: Concept of Probabilistic Impact Based Forecasts for the Canadian Armed Forces, David Dégardin et al.


Onsite presentation
Ken Mylne and Nigel Roberts

“2022 marks the 30th anniversary of the first operational ensemble forecasts at ECMWF and NCEP (USA), and also 10 years since the Met Office started running the convective-scale MOGREPS-UK ensemble for the UK. Ensembles are now central to the NWP strategies of many centres. There is extensive scientific evidence for the greater skill of probabilistic forecasts based on ensembles and the Met Office plans for its new supercomputer have high resolution ensembles as the core operational systems, with higher resolution deterministic models used for experimental purposes. We have already changed our Key Performance Indicators for forecast accuracy to measure ensemble performance. The IMPROVER post-processing system (see presentation by Moseley and Mylne) is designed to fully exploit convective-scale ensembles and provides a seamless blended probabilistic supply of forecast data to underpin multiple products and services.  Despite many years development and the scientific evidence for greater skill, fully exploiting the benefits of ensembles remains a major challenge, both for operational meteorologists (forecasters) and in products and services for the public and professional users. There have been some notable successes, but many forecasts remain highly deterministic for understandable reasons. To address this, alongside our ensemble-driven NWP approach, the Met Office is seeking to make much wider use of ensembles and probabilistic forecasts throughout its operations.  This will involve close collaboration between scientists, operational staff and services, for example in operational testbed experiments where scientists and forecasters work alongside each other to test new capabilities and examine potentially new working practices. This talk will review early progress in this approach and plans for future research and development.”

How to cite: Mylne, K. and Roberts, N.: Bringing ensembles to the heart of Met Office operations, EMS Annual Meeting 2022, Bonn, Germany, 5–9 Sep 2022, EMS2022-481, https://doi.org/10.5194/ems2022-481, 2022.

Onsite presentation
Katja Schulze et al.

Communicating the uncertainty underlying weather warnings appears promising. Information about uncertainty can increase confidence in the forecast and support the decision-making process (Joslyn and LeClerc, 2013; LeClerc and Joslyn, 2015; Fundel et al., 2019). However, probabilities can be misunderstood if they are not appropriately communicated (Murphy, 1980; Gigerenzer et al., 2005; Budescu et al., 2014). Uncertainty information in weather warnings might also reduce threat perception and intention to act (Taylor 2021; Schulze & Voss, in prep.).

As part of the WEXICOM project (Weather warnings: from EXtreme event Information to COMunication and action), we conducted a representative experimental online survey in Germany (n=1 721). In close collaboration with Nathalie Popovic (MeteoSwiss), we investigated how the public perceives weather warnings with different formats of uncertainty information. The study tested three combinations of numerical and verbal uncertainty information in weather warnings (numerical probability & verbal probability vs. numerical probability & verbal frequency vs. numerical frequency & verbal frequency). Moreover, we tested all three conditions with and without additional explanation about the probability level at which the weather service typically issues a warning. All combinations were tested for three probability levels (20%; 40%; 60%) in two weather scenarios (hurricane force gale; thunderstorm with extremely heavy rainfall).

We find that low probabilities decreased the perceived risk and warning response – in line with numerous findings in other domains that low probability events are often not taken seriously. Interestingly, the format in which probabilities were communicated did not make a difference: The warning response was independent of whether probability information was given in numerical or verbal form. What was striking, however, was that providing information about the low probability level at which weather services typically issue a warning improved the perception of the warning — especially at low probability levels. Thus, the main problem for the public might not be understanding the uncertainty information but interpreting its magnitude in the context of weather warnings. Here, a simple reference point can help to put low probabilities into perspective.

How to cite: Schulze, K., Popovic, N., and Fleischhhut, N.: How to communicate uncertainty in weather warnings to the public? Put it into perspective!, EMS Annual Meeting 2022, Bonn, Germany, 5–9 Sep 2022, EMS2022-626, https://doi.org/10.5194/ems2022-626, 2022.

Onsite presentation
Adam Choryński et al.

In recent years one may notice an increasing number of extreme weather events occurrence, as well as their rising intensity. This becomes a threat for local communities and brings challenges for institutions responsible for risk management. Larger towns seem to have a different potential to deal with these risk, due to their resources (material, institutional, human). Large cities are also researched more often because of their larger exposure. On the other hand a research focus on extreme weather events resilience in small communities is lacking. Smaller towns also experience the effects of extreme meteorological events, and even if losses are not as high as in larger towns, on the local level they are they are very severe.

In this paper we analyze the issue of extreme weather events resilience in eight small municipalities from the Wielkopolska region in Poland. Main aim of this research is to study how local resilience arrangements formed in the studied municipalities acts during an another significant extreme meteorological event. Municipalities focus mostly on emergency services actions, but they differ in case of the second component of local resilience arrangements. These municipalities that were strongly affected by extremes concentrate more on structural – hard measures. Municipalities that have not faced so severe extreme weather events in the past, built resilience with a strong component of creativity resilience, where soft solutions related to education, training, organization of local risk management system complement emergency services activities.

This research is based on 40 in-depth interviews with local stakeholders, meteorological data, as well as information on firefighters operations since 2010, municipal budgets information. In order to analyze which factors affect resilience of local communities to extreme weather events based on analyzed case studies and which influence is rather limited qualitative comparative analysis (QCA) approach has been applied.

Acknowledgements: This research has been funded by the National Science Center of Poland under the grant number: 2018/31/B/HS4/03223

How to cite: Choryński, A., Jeran, A., Matczak, P., and Pińskwar, I.: Factors behind local communities resilience to extreme weather events, EMS Annual Meeting 2022, Bonn, Germany, 5–9 Sep 2022, EMS2022-411, https://doi.org/10.5194/ems2022-411, 2022.

Onsite presentation
David Dégardin et al.

Environment and Climate Change Canada’s National Programs and Applied Development (NPAD) group, which provides support to the Canadian Armed Forces (CAF), began in 2015, work toward probabilistic impact based forecasts, in order to meet the CAF's needs, for environmental forecasting (meteorological and oceanographic) and the related impacts on their operations. Operations led by CAF are multidisciplinary by nature, cover a wide variety of geographical areas and can span variable durations. They are prepared in advance and are refined as additional information becomes available. These aspects constitute a challenge regarding the development of a decision-support tool that meets the diverse requirements associated with these missions from the initial planning to the final deployments. Both the Meteorological Service of Canada (MSC) transformation initiative and the “data centric approach” promoted by the Canadian Forces Weather and Oceanographic Service (CFWOS) have inspired this situational awareness project, called Consolidated Weather Impact Chart (CWIC). Based on the concept of a suite of systems providing a seamless datasets, post-processing of deterministic and probabilistic models outputs in their respective fields of excellence. A first stage, relying on the Canadian Global Deterministic Prediction System (GDPS) has been operationally implemented to create deterministic “on demand” consolidated weather impact charts and deterministic “impact-grams” for any location. As a second stage, the inclusion of probabilistic outputs aimed to provide to the user a tool with information of likelihood and impact for a specific mission/operation. In order to offer a probabilistic version of CWIC, its conceptual development was inspired by the both National Severe Weather Warning Service weather impact matrix developed by the UK Met Office, which combines likelihood and impacts and the Extreme Forecast Index (EFI) formulated by ECMWF, which characterizes the abnormality of forecasted events with respect to the model-climate. Resulting charts have highlighted its significant capability as a decision support tool for a wide spectrum of customers’ needs due to the integration of adjustable thresholds. A third stage extended the application of the matrix concept in order to provide operational forecasters with standard weather element depictions based on the wealth of information provided by both the Canadian Global and Regional Ensemble Prediction Systems (GEPS/REPS). This brings consistency with information provided by the probabilistic version of CWIC and supports meteorologists in interpreting and conveying risks of impactful weather to the CAF. In addition, probabilistic “impact-grams” relay the uncertainty related to the suggested weather scenario. This presentation aims to expose and share concepts in order to stimulate feedback, discussions and future collaboration.

How to cite: Dégardin, D., Turcotte, M.-F., Lebel, M.-A., Hawkins, M., Smith, H., Murphy, C., Harris, R., and Sustersich, J.: Concept of Probabilistic Impact Based Forecasts for the Canadian Armed Forces, EMS Annual Meeting 2022, Bonn, Germany, 5–9 Sep 2022, EMS2022-719, https://doi.org/10.5194/ems2022-719, 2022.

Supporters & sponsors