Risk aversion and decision making using ensemble forecasts

Posted on June 6, 2017 by Marie-Amélie Boucher

by Marie-Amélie Boucher and Vincent Boucher

Assessing the value of forecasts is a very popular topic among the HEPEX community. The assessment of forecast value is highly dependent on the purpose served by the forecasts. For the specific problem of decision-making related to flood mitigation, Murphy (1976, 1977) proposed the use of the cost-loss ratio framework. The vast majority of papers related to the assessment of forecast value for flood mitigation adopt this framework, so one could think that everything is pretty much solved… except that the cost-lost ratio has very important flaws!

One such flaws is the fact that the cost-loss ratio assumes that the decision maker is risk neutral. Risk neutral individuals only care about the expected outcome, and disregard the spread of the distribution of outcomes. Risk neutral individuals are very rarely encountered in real life. Indeed, most of us are risk averse. That is, for the same expected outcome, we prefer less risky distributions. This is (among other things) why we buy insurance. Informally, most people dislike risk and would be willing to spend resources (e.g. money) in order to reduce the amount of risk faced.

Utility theory

Economists (and statisticians and mathematicians) have been studying those issues for a long time, and came up with many models and concepts which could be used in hydrology, as an alternative to the cost-loss ratio. The central framework is based on “utility theory”. To put it’s development a bit in a context, here is a (very) pseudo-historical adaptation of a conversation between Nicolau I Bernoulli and his cousin Daniel [1]:

It didn’t happen exactly like that, but the general idea is preserved. The game that Nicolau is referring to is now known as the St-Petersburg paradox and was first described in a letter. The first person to present it more formally was Daniel Bernoulli in 1738. He was also the first to suggest the idea of risk aversion and exposed (without any mathematics) that different persons faced with the same decision problem and the same information could take different decisions, because of different preferences.

It is only much later, in 1944, that the concept of risk-based individual preferences was formalized as a mathematical theory, by John von Neumann and Oskar Morgenstern [2]:

Utility theory is not perfect (and indeed, many generalizations and extensions exist), but it has at least two advantages over the cost-loss ratio:

  1. It allows for decisions to consider a finite number or a continuum of decisions (e.g. “protect” vs “don’t protect” for floods, or alternatively, “amount spent in protection”).
  1. It allows to account explicitly for the decision maker’s level of risk aversion in the assessment of forecasts value.

In hydrology

Krzysztofowicz suggested using utility theory in hydrology as early as in 1986. It is not a miraculous solution to the problem of assessing the value of forecasts for flood mitigation, but it would be an improvement over the cost-loss ratio. It can explain real-life behaviors that the cost-loss ratio cannot. This was the case in our recent application for the assessment of forecasts value on the Montmorency River in Canada (Matte et al. 2017). Other examples of accounting for risk-aversion for decision-making in hydrology and meteorology include Shorr (1966), and Cerdá and Quiroga Gómez (2008).

References:

  • Bernoulli D. (1738) Specimen Theoriae Novae de Mensura Sortis, Commentarii academiae scientiarum imperialis Petropolitanae, 5, 175-192.
  • Cerdá Tena E. and Quiroga Gómez S. (2008) Cost-Loss Decision Models with Risk Aversion, Working paper no. 01, Instituto Complutense de Estudios Internaciolales, 28 pages.
  • Krzysztofowicz R. (1986) Expected utility, benefit, and loss criteria for seasonal water supply planning, Water Resources Research, 22(3), 303-312.
  • Matte S., Boucher M-A, Boucher V. and Fortier-Filion T-C (2017) Moving beyond the cost-loss ratio, economic assessment of streamflow forecasts for a risk-averse decision maker, Hydrology and Earth System Sciences (Accepted)
  • Murphy A.H. (1977) Value of climatological, categorical and probabilistic forecasts in cost-loss ration situation, Monthly Weather Review, 105(7), 803-816
  • Murphy A.H. (1976) Decision-making models in cost-loss ratio situation and measures of values of probability forecasts, Monthly Weather Review, 104(8), 1058-1065
  • Shorr B. (1966) The cost/loss utility ratio, Journal of Applied Meteorology, 5(6), 501-803.
  • von Neumann J. and Morgenstern O. (1944) Theory of games and economic behavior, vol. 60, Princeton University Press Princeton, 625 pages.

Figure captions:

[1] The original pictures were taken from these websites: http://www.famous-mathematicians.com/daniel-bernoulli/ (right) and http://www2.stetson.edu/~efriedma/periodictable/html/Bi.html (left). The schematic drawing of D. Bernoulli’s blood pressure experiment apparatus was taken from https://plus.maths.org/content/daniel-bernoulli-and-making-fluid-equation

[2] The original pictures were taken from these websites: http://www.tinbergen.nl/oskar-morgenstern-and-john-von-neumann-shelby-white-and-leon-levy-archives-center/ (left) and http://www.karbosguide.com/books/pcarchitecture/chapter02.htm (right)

Posted in decision making, economic value | 2 Comments

The elusive optimal decision rule and the impact of forecasting

Posted on May 23, 2017 by Maria-Helena Ramos

Contributed by Micha Werner, IHE Delft and Deltares

This winter just passed I was faced with the kind of dilemma of the type I am sure many find all too familiar. I was the designated driver one weekend for my daughter’s hockey team away match.

Parents take turns to drive to away matches, and so I was all set to head off with four excitedly chatting thirteen year-olds. At breakfast, I saw on the internet that KNMI, the Dutch national meteorological agency had earlier that morning issued an amber alert. It had been well below zero for a couple of days and the roads were cold. A warm front from the West was now pushing its way through, and there was a high risk of black ice, leading to very slippery roads and potentially dangerous situations.

All non-essential trips were strongly discouraged, particularly on motorways in the west of the country that had been as yet little used that Sunday morning. The warning indicated that the risks would diminish by 9:30. I had to leave by 9 to make it on time and my route included a section of motorway. What was I to do? Should I heed the warning? Was this trip essential or was it non-essential?

Having professionally dabbled with the concept of costs and losses, the decision seemed obvious. The risks were high, and the consequences of a potential accident severe. So the obvious rational decision was not to go. But there were also consequences to not going. The match would need to be called off if the four girls I had with me did not show. Would the opposing team claim victory for a no-show? That could be embarrassing, and moreover could knock our team out of the top of the pool. Would other parents decide not to go; or rather would they go and be let down by my decision to stay at home?

A furtive discussion on our team WhatsApp group ensued. Finally an agreement was reached. Yes, I explained to a concerned mother, yes I had invested in winter tyres. The decision was made. We were off.

Making decisions based on warning information

This little story reflects the difficulty of making decisions based on warning information. Even when provided with quite specific information such as I received that morning, which not only targeted the road type but also provided specific details as to the timing. This is the level of detail that impact based warnings set out to achieve. Information contained in the forecast that is personally relevant to the consumer of the forecast. And yet making the decision that would seem obvious on paper is not easy in practice.

There has been some research on the process of an individual making a decision based on an advisory of flooding, or severe weather, or other (natural) hazards.

  • Does the warning apply to me? Do I trust it?
  • Am I inherently risk averse, or rather risk acceptant?
  • How is my decision influenced by the potential losses?
  • And in making a decision are these losses balanced against the cost of taking measures?

A decision making “serious-game”: the Shopkeepers Dilemma Game

The latter question listed above is the reasoning that underlies the rational decision making principle. It was also the question what was asked participants in a decision making “serious-game” we played at the 2016 European Geophysical Union during the Ensemble Hydro-meteorological Forecasting session. Some readers may also recall playing the game at the 2016 HEPEX conference in Quebec City, Canada, or at the Royal Meteorological Society Conference in Manchester, England. It was also played with a group of students in a flood management course at IHE Delft.

Micha Werner and his poster at EGU 2017

This year’s EGU saw a poster presenting the results, and a paper will further explore the details but is still in preparation (I had of course intended to write the paper earlier, and may have found more time to do so if only I had heeded more warnings and stayed at home, instead of going to weekend sports matches with my kids!)

In the game, three different shop-keepers were presented with a series of seven forecasts, each providing participants with the forecast probability that they and their shop could be flooded. They were asked to make one of three decisions for each forecast; choosing between taking no action; raising temporary defences on the embankment between their shop and the river; or moving their inventory. Except for doing nothing, all actions came at a cost. But flooding also caused a loss.

What was also important was that if the shop could stay open for business then a profit could be made. Obviously the shop could not stay open for business if flooded, or if the decision to move the inventory had been taken. However, if the demountable defences were raised, and these were not subsequently overtopped, then it was business as usual and a profit could be made.

The three shops selected in the game had quite different costs and losses:

  • One shop sold Ferraris, with significant losses when flooded, as well as high costs when moving the inventory. But profits when the shop stayed open for business were also high.
  • The second type of shop was a grocery store, with lower losses when flooded; but also lower costs to move inventory, and lower profits when staying open for business.
  • The third shop sold gravestones. Losses due to flooding gravestones were low; and lower or equal to the cost of raising the defences or moving the inventory. Of course when raising the embankments the shop could stay open for business if the embankments were not overtopped, making a modest profit.

At each step, after the decision on the bulletin was made, the actual outcome of the event was shown and participants asked to tally their results. The winner was the participant who had the lowest expected expense at the end of the game, due either to taking measures, which at times were taken in vain when the forecast flood did not materialise, or due to losses incurred when either no action, or inadequate action was taken.

The game material and publications are publicly available here in the Resources Page of the Hepex Portal, together with other games and lectures provided by the community.

A couple of interesting patterns

Reviewing the results of the 215 participants that played the game across the four sessions revealed a couple of interesting patterns. I will leave the full assessment of those results to the full paper, but some interesting thoughts came forward that may be worth mentioning.

  1. Choices made by those selling Ferraris were more or less as expected. They were keen to act on the forecasts; and the most likely to take action in response to the forecasts, with a clear relationship between the probability of forecasting and the inclination to take action. The forecast was of clear value to them.
  2. This was also more or less the case for those with grocery stores, though they did not take action quite as often.
  3. What was interesting though was the behaviour of those selling gravestones. Though these was little need to respond to the forecasts for them, as the cost of unnecessary action was very high compared to the losses when no action was taken, the gravestone sellers seemed overly keen to respond to the forecasts. Their behaviour was more or less the same as the grocery store participants. It would appear that they were prepared to take the risk of getting it wrong despite the cost, preferring to invest to avoid being flooded.  Such behaviour is often considered to be risk adverse, influencing decisions to be made that stray from the rule that would be prescribed by the rational cost-loss theory.

But not all decisions made in risky situations are necessarily risk adverse. The quite well known prospect theory, first described by Kahneman and Tversky (1979) outlines how decision makers may be risk seeking, or rather how they may be risk averse, depending on how information informing the risk based decision is framed.

Generally, in the game it seemed that the prospect of being flooded caused the decision to take protective action to be taken more often than necessary. This could also be considered loss aversion. People are prepared to invest in taking measures, just to avoid the displeasure of being flooded, which given the disgruntled mutters of participants in the room when flooded after deciding not to take action, clearly gives rise to negative emotions.

The role of emotion in the framing of a decision

Emotion is clearly an important factor in the framing of a decision, as outlined by Druckman and Mcdermott (2008). They go on to discuss how different emotions may influence decisions. Maybe a bit too much to discuss in a blog post that started short but just kept growing; and definitely a topic that is increasingly out of reach of a hydrologist such as myself. However, they also shed some light on my little dilemma. Clearly, the negative emotion connected to not going to the match, as well as my own preference confidence led to the taking of what was a risk-seeking decision.

Emotions may also lead to paralysis in decision making, resulting in no decision being made. That reminded me of a discussion I recently had with a certain Dr Stephen Hussey. He leads an NGO called the Dabane Trust that works with communities in Southern Zimbabwe. What he has found somewhat confounding is that despite clear and acknowledged signs of oncoming drought conditions provided by a seasonal forecast product, that it was very difficult to change what was considered to be normal behaviour by the communities. They could easily plant a less drought sensitive crop than the traditional maize; but they had always planted maize and changing that was not easily done.  Changing behaviour could perhaps be done if a champion for change could be found in the community, but otherwise it proved to be very difficult.

Of course I need not tell the reader that the true value of any forecast will depend on how that forecast is used to influence behaviour and take protective action. Impact based forecasting has recently emerged as a step towards personalising forecasts and through this influencing what an individual should do to get themselves and their property out of harm’s way in the face of natural hazards. That is no doubt a good thing, but the impact of a forecast ultimately depends on the behavioural change that it influences.

This calls to my mind for branching out to research on behavioural change, working with psychologists, social scientists, and economists, as well as many others; on thinking how to frame forecast information. We cannot do this as hydrologists alone. I myself always like to find analogies from my own experiences in decision making in my personal life, wondering how this may help explain the behaviour of recipients of forecasts.

There were no real issues with the roads that Sunday morning. We left a bit earlier than planned, took caution and arrived well on time. That day the team did reasonably; of the two matches they had to play they won one, and lost the other.

Some game-addicted members of the Hepex community at EGU 2017 (from left to right): Micha Werner, Hannah Cloke, Louise Arnal, Andy Wood, Louise Crochemore, Maria-Helena Ramos and Ilias Pechlivanidis). The poster in the back can be found here.

References

Posted in decision making, forecast communication, forecast users | 1 Comment

A platform: Community of Users on Secure, Safe and Resilient Societies

Posted on May 10, 2017 by Florian Pappenberger

The “Community of Users on Secure, Safe and Resilient Societies” (CoU) is an efficient platform of exchanges among different actors of different branches of security and crisis management. The CoU initiative came as a response to the complexity of the security and crisis management policy framework and the wide variety of research capacity-building and training initiatives that led to a lack of awareness about policies and/or research project outputs by users, namely policy-makers, scientists, industry/SMEs and practitioners (e.g. civil protection units, medical emergency services and police departments).

There is an High-Level Practitioners Event, taking place on 15 May 2017 [see program here] – or watch the live stream.

WHO ARE THE USERS?

Fields concerned by security, safety and resilience for societies are themselves scattered into many different disciplines and sectors.

WHY AND HOW PARTICIPATING?

Registered members [register here] have obviously different interests in being part of the network, in particular keeping up to date with policy and research developments, information sharing among different actors and networking. Most of the information sharing / update is carried out on line without the necessity to attend meetings, although meetings may also be useful for networking purposes and synergy building.

About attending meetings in person, the following considerations may apply:

You are a Policy-maker

Interactions may be of relevance to you in plenary sessions to share general information about your respective policy areas and get feedback from other actors. Participation in thematic workshops will allow to get in-depth insights into the areas concerned.

As an example, the CoU network involves UN representatives dealing with fight against crime and terrorism (UNICRI), disaster risk reduction (UN-ISDR), and transboundary industrial accidents (UN/ECE), EU Policy DGs such as DGs HOME, ECHO, SANTE, GROW, ENV, CLIMA, ENER, MOVE, TAXUD, as well as the JRC.

In addition, EU and intergovernmental agencies such as FRONTEX, EUROPOL, EEAS, EDA, are also involved. Moreover, representations at Member State’s level are through Ministries of Defence, Interior, Foreign Affairs, Civil Protection, Environment, Research and Industry, as well as Agencies and Regional Authorities.

Your are a Scientist

Representatives of research projects will find possible interest in getting information on policies in the respective areas to better understand needs related to technical implementation of various policies. Scientific findings may be discussed among different actors / users in the CoU framework, and project synergies are strongly encouraged, including the possibility to hold demonstrations.

You are an Industry (including SMEs) representative

Similarly to scientists, you will have possible interest in getting information on policies in the respective areas to better understand needs related to technical implementation of various policies, as well as meeting research project representatives and share with different actors views about innovation needs and perspectives.

You are a Practitioner

First responders, i.e. fire brigades, emergency services, police forces, civil protection units, military units, laboratories, etc. will find an interest to attend for getting updated information about EU policy and research developments and express their views about the tools, technologies, etc. development and their possible involvement in research and demonstrations.

Your are NGOs or Consultant

The CoU provides a large range of policy and research information that are of potential usefulness to NGOs, Civil Society Organisations, consultancy companies, etc.

General public

Interest to participate in person is less obvious for the general public owing to the highly technical level of discussions, but some events might be tailor-made to citizens, which is envisaged in the future.

Posted in activities | Leave a comment

EGU Twitter page

Posted on April 24, 2017 by Maria-Helena Ramos

This week, the HEPEX traveling circus will descend on Vienna, home of the annual convention of the European Geosciences Union. HEPEX-ers will be tweeting their way through the conference – and these Tweets will be assembled in below Twitter stream. Feel free to add your thoughts, ideas and observations – just make sure they’re summarized in 160 characters max – and don’t forget to include the #hepex tag!


Posted in Unclassified | 1 Comment

What will happen next week at EGU 2017 in hydrological forecasting?

Posted on April 19, 2017 by Maria-Helena Ramos

The EGU 2017 Annual General Assembly will take place next week, from 23–28 April 2017 in Vienna. Once again, researchers, professors, early career scientists and practitioners interested in any of the fields covered by geosciences, including hydrological sciences, atmospheric sciences and natural hazards, will be together to present and discuss their work.

The meeting program includes about 1,000 sessions and over 17,500 abstracts, of which over 200 contributions will be presented in 8 scientific and operational-focused sessions organized by the sub-division on Hydrological Forecasting.

Here you can find a list of what will be presented in relation to hydrological forecasting and of some main events to guide you through the week in Vienna:

On the first day, after the first morning coffee break, join us at the flash floods session, which will start with 6 oral presentations at 10:30 and will display 21 posters at the end of the day in Hall A:

  • 10:30–12:00 / Room 2.31: Flash floods and associated hydro-geomorphic processes – Learn more about flash flood characterization with an X-Band weather radar in the Eastern Mediterranean region, the geomorphic response associated to four large floods in Northern Italy, flash flood risk assessment in France and Germany, or flash flood forecasting and early warning systems over Europe.
  • POSTER SESSION 17:30–19:00 / Hall A: Come to discuss with poster presenters and meet colleagues of the Hydrological Forecasting sub-division.

On the second day, we do not have specific sessions organized by the Hydrological Forecasting sub-division, but several other sub-divisions are offering a rich program. Just as examples, see, for instance:

This is the day of our operational forecasting and warning system PICO session (as already mentioned in a previous blog post by Mike Cranston).

  • 08:30–12:00 / PICO Spot A: Operational forecasting and warning systems for natural hazards – This year, we have 28 interactive presentations at PICO screens. Several applications will be presented and we will also have the “traditional” game presentation, which, this year, was prepared by Louise Arnal et al.: Pathways to designing and running an operational flood forecasting system: an adventure game. Just come and try it yourself!
NEW!!   Short course on Hydrological Forecasting

It is co-organized by HEPEX and the Early Career Scientists (ECS), convened by Shaun Harrigan. The course will be given by Marie-Amélie Boucher and Jan Verkade. Check the blog post recently published for more details. Attendance is open to everybody and will be on the basis of first come, first served!

Wed, 26 Apr, 17:30–20:00 at Room -2.91

A long day with several Hydrological Forecasting sessions is waiting for you on Thursday:

The Division meeting for Hydrological Sciences (HS) will be convened by Elena Toth, on Thu, 27 Apr, 12:15–13:15 / Room B. It is the opportunity  to learn more about the way sessions related to Hydrological Sciences  are organized at the EGU Assembly. You’re all welcome!

Scientific sessions continue in the afternoon:

  • POSTER SESSION 17:30–19:00, will be mainly in Hall A, but also in Hall X4 (post-processing) and Hall X3 (coupled systems) for the co-organized sessions above.
  • Another tradition in Vienna: the HEPEX social gathering @ EGU. As in last year, it will be co-organized with partners of the IMPREX H2020 project. It will take place on Thursday evening at 8pm (restaurant will be confirmed at EGU). Since reservations have to be made in advance, and places are limited, please, contact Louise Arnal or Rebecca Emerton before Monday 24 April if you want to join us.

Last day of the EGU Assembly and again a full day of presentations:

  • 08:30–12:00 / Room 2.95: From sub-seasonal forecasting to climate projections: predicting hydrologic extremes and servicing water managers, with presentations on society vulnerability to extremes and the use of seasonal forecasts to improve water resources management.
The  meeting of the Sub-Division on Hydrological Forecasting will be convened by MH Ramos, on Fri, 28 Apr, 12:15–13:15 / Room 2.83
It is open to everybody. Come and join us, notably if you want to meet colleagues or get more involved in the organization of sessions and short courses related to hydrological forecasting at EGU in 2018.

Scientific sessions continue in the afternoon:

  • 13:30–17:00 / Room 2.95: Ensemble hydro-meteorological forecasting. Our traditional HEPEX session will be opened by Andy Wood’s talk on “Over-The-Loop ensemble streamflow forecasting in US watersheds”. It will be followed by talks on ensemble calibration, sensitivity analysis, ensemble nowcasting, and the use of satellites and tweets in flood disaster management. After a coffee break, you will have the opportunity to learn more about two projects, EDgE and GloFAS-Seasonal, and on recent progress on the skill of seasonal hydrological forecasts.

POSTER SESSION 17:30–19:00 in Hall A. NOTE: If you played last year’s game on Weighing costs and losses with Micha Werner (or also if you didn’t), you may be curious to learn more about the results that he will show in his poster. Come to the poster Hall A to get information and exchange ideas on the topic.

As you can see there is plenty to see and do in Vienna. See you there!

Posted in activities, announcements-events, meetings | Leave a comment

Understanding public responses to flood warnings

Posted on April 12, 2017 by Maria-Helena Ramos

Contributed by Michael Cranston, RAB Consultants, Scotland

The Winter 2015 floods in the United Kingdom and Ireland led to severe and widespread flooding for many communities.  The introduction of storm naming by the Met Office and Met Eireann the same year has since led to the names of Desmond, Frank and Eva becoming synonymous with the flooding that was experienced during that winter. Storm Desmond (5 December) brought major flooding to Cumbria and southern Scotland with a new 24-hour UK rainfall record of 341.4 mm being recorded at Honister Pass in Cumbria. Storm Eva (24 December) brought further flooding to parts of northern England, whilst Storm Frank (29/30 December) brought severe damage to communities in Newton Stewart and Ballater.

Severe flooding from the River Dee in Ballater (Source: Evening Express)

This series of major floods led to a UK Government review on National Flood Resilience which is summarised by Louise Arnal in this HEPEX article on what we should learn from the winter 2015 floods.  However, the UK has seen significant investment in improved flood forecasting in the 10 years post-Pitt and this account of the short and medium range forecasts ahead of Storm Frank highlights that flood forecasting tools and models are starting to support much earlier warning of significant flooding impacts.

Flood alerts and warnings in Scotland are now sent to over 25,000 members of the public since the Scottish Environment Protection Agency (SEPA) introduced the new service in 2011.  During the Winter floods, SEPA’s service saw 300,000 individual messages being issued to alert and warn of potential flooding.  Following this, SEPA, through the Centre of Expertise for Waters (CREW), subsequently commissioned research to understand public responses to warnings and to assess how effective the flood warning service is for reducing the impacts of flooding.

A research team at the School of Social Sciences the University of Dundee were commissioned to undertake this project in 2016. The primary method of collating information on public response was through a web-based questionnaire of registered customers of the service.  This survey was designed to assess associations between multiple customer characteristics, including location, type of message received, prior experience of flooding, risk awareness, and demographics.  1341 customers provided a response to the survey invitation, and crucially 1290 of those provided a postcode to allow for geocoding of their responses.  The information provided through the questionnaire response was further enhanced through several community group meetings held by the research team.

The research to understand public responses to flood warnings involved consultation with members of the public and community groups across Scotland, including this taken in the Speyside

The research results are starting to deliver some interesting findings around the public response to flood warnings.  It is clear from the results that the service is valued by those at risk of flooding with 66% of the respondents satisfied with the warnings they are provided.  Many take mitigating actions, of note: 62% of those who said flooding of land was important to them subsequently moved livestock on receipt of a warning; 71% of those that stated they had bought these measures, deployed property level protection; and 42% of all respondents at some time removed vehicles on the receipt of a warning.

Less effective elements of the service focus on those members of the public that are not engaged with the service, possibly the ‘flood unaware’ and elements that are not locally specific enough to encourage action.  For example, the least content of customers are those that do not receive regular messages. Also, those who receive regional Flood Alerts are less likely to act given the lack of locally specific information, which is typically misunderstood by members of the public as a part of the service that they consider should offer more local relevant flooding information. This theme was one of several that were explored further during the community workshops with some referring to the need for local reference points in warnings, a discussion which featured as part of the flood memory and historical references HEPEX article last year.

Work continues to conclude this research which will be published by CREW this summer with recommendations for future development of the flood warning service.  However, a summary of the results will be provided at the forthcoming EGU General Assembly as a PICO presentation as part of the Hydrological Sciences Division, in co-organization with the Natural Hazards Division, on a session of the Hydrological Forecasting Sub-Division on operational forecasting and warning for natural hazards.

This work will be presented as part of the PICO session on operational forecasting and warning systems for natural hazards: PICO spot A, Wednesday 26th April between 08:30 and 12:00 (see programme here)

This work would not have been possible without the efforts of the Dundee University research team of Dr. Alistair Geddes, Dr. Andrew Black and Alice Ambler and the guidance of SEPA’s Project Manager Cordelia Menmuir.

Michael Cranston is an Honorary Research Fellow with the School of Social Sciences at Dundee University and a consultant in flood early warning with RAB Consultants in Scotland.  Prior to this he was an operational flood forecaster and Manager of the Flood Forecasting and Warning team at SEPA.

Posted in decision making, floods, forecast communication, forecast users, operational systems | Leave a comment

A short course on Hydrological Forecasting: interview with Marie-Amélie Boucher and Jan Verkade

Posted on April 4, 2017 by Jan Verkade

Contributed by Maria-Helena Ramos (Irstea) and Shaun Harrigan (CEH)

This year Hepex is joining the Young Hydrologic Society (YHS) to offer a short course on hydrological forecasting during the EGU General Assembly in Vienna.

Shaun Harrigan (CEH), representative of the YHS, and I have been excited about this first experience and invite you to check the short description of the course’s content.

The course will take place on Wednesday 26 April, from 17:30-20:00, in room -2.91 (Brown Level -2 – Basement floor). It will be given by two active members of the Hepex community:

Marie-Amélie Boucher is a professor at the University of Quebec at Chicoutimi (UQAC). She has written several Hepex blog posts as a columnist in 2015 and a general contributor: “In Canada, I teach disciplines such as hydrology, hydraulics and fluid mechanics. My main research interests revolve mostly around ensemble forecasting and include multi-model forecasting, data assimilation, pre and post-processing and assessing the socio-economic value of forecasts.”

Jan Verkade is a researcher in hydrology as well as a developer of flood forecasting systems at Deltares and a flood forecaster for the Rijkswaterstaat River Forecasting Service in the The Netherlands. He is one of the creators of the Hepex Portal in 2013 and since then has been helping to manage it: “My research interests focus on hydrologic forecast, warning and response systems. I have ample experience in the development of operational, real-time hydrological forecasting systems and in applications of research findings in projects around the world. I have also aged matured grown up greyed considerably since this photo was taken.”

Can you tell us a little about what you are preparing for this course in Vienna?

JV: There will be some focus on the elements that distinguish realtime forecasting from modeling, including realtime data feeds, management of model states and state updating runs, data assimilation, primary and secondary data validation and forecast verification. Also, we will not strictly limit ourselves to “forecasting” and will give some attention to the use of forecasts: communication thereof, use in decision making, etc.

MAB: I won’t reveal too much: people have to come to the course to know its content in greater detail! However, I can tell you that we are tailoring the course to people who already have some background in hydrological modelling. We will assume that participants already know how to set up a model (any model) that runs in continuous mode, for instance, and that they have already done or worked with model simulations. Obviously, those who read the HEPEX blog are probably more than okay with that! So, this course is for anyone with some background in hydrology, and a participant does not need to be a forecaster to join us.

Why should an Early Career Scientist (or even an established researcher or flood forecaster) take the opportunity to attend this course?

MAB: Simply because hydrological forecasting is valuable for society for multiple reasons (see this and this and that, for instance), and because it is fun!

JV: Realtime forecasting is a discipline in itself but, despite its obvious relevance to society, receives relatively little attention in university curriculae. Here’s one of very few opportunities to learn more about it!

And one final question: what was your worst experience ever with hydrological forecasting (model, data, real-time forecasting…)?

JV: Over the years I have come across many issues related to pretty much all aspects of realtime forecasting. Many pertain to automated data feeds (precipitation gauges, numerical weather predictions) where something was wrong but, for one reason or another, the issue escaped detection by validation procedures. Biblical floods resulted – on screen, in any case. Others related to physical processes not included in the numerical models: a particular rain-on-snow event springs to mind. Yet others related to the handling of ‘states’. There was always lots of opportunity to learn!

MAB: Well, nothing dramatic, really! I crave the collaboration with operational forecasters and sometimes I hear about some bad experiences from them, but they are not my own! My research is always performed in hindcast mode (I’m a huge fan of the TIGGE data portal, for instance!), which probable makes things easier. Still, I definitively have a “least preferred hydrological model” when running my forecasting experiments. I guess we all have one, depending on personal experiences. From my experience, the implementation of this particular model requires disproportionate efforts considering the quality of the results it gives (compared, for instance, to simpler models). It also has those obscure error messages. I especially dislike the one that simply says that “[model’s name] is going to close now”. But I still have to work with it sometimes (I won’t detail why…), so maybe I’ll eventually learn one day to appreciate it (or to improve it!).

Thank you, Marie-Amélie and Jan, for your answers and your availability to prepare the EGU course!

For those willing to attend the course, you just need to be in Vienna for the EGU and show up in room -2.91 (Brown Level -2 – Basement floor) on Wed 26 April. Attendance will be on the basis of first come, first served!

See you there!

Posted in announcements-events, early-career scientists, forecast techniques, interviews | 2 Comments

Breaking news: Hydrology is the oldest of all scientific disciplines

Posted on April 1, 2017 by McFools Family

Posted on April 1, 2017 by Dr Professor McFools

The general belief until recently has been that hydrology is a very young discipline — i. e. Homo Hydro is a quite young scientific species — but it is time to debunk this myth. Scientists from the Grand Old Duke University revealed in this issue of Environment (impact factor: highish) that they had found fossils believed to be from a previously unknown early hydrological model ancestor — one that could have used simplified functions and may have even figured out how to make certain primitive predictions without ensemble members.

Scientists believe that mysterious episodes of knowledge exchange between otherwise isolated groups of our Homo hydro (sub species modellus) ancestors may have paved the way for parameter similarity evolution. The recent discovery also suggests that outbursts of multi-objective calibration and Bayesian uncertainty post-processing, which might have (eventually) greatly enhanced model performance, might also have killed hydrologic physics and broken the links between weather prediction and water resources management.

Figure 1. Reconstruction of what is believed to be the first ever recorded hydrological model

At this point we cannot exclude the theory that the homo hydro also paired with the homo mathematicus. This combination seem to be better suited to survive in a world with growing publish-or-perish pressure, but may also be the reason why the ancient wisdom was lost for many centuries in what can been named “the great drought”. During this period, only very spurious attempts were made to explain the physical processes behind the hydrological cycle (Figure 1), and for very long only the Nilometer was thought of as the earliest sign of hydrological predictions.

Evidence that calibrated specimens of hydrological models have spread even to the most remote areas of our planet like dandelions across a lawn had already been found in a recent research carried out by the team of Prof. McFools: “My feeling is that the application-focused purposes of an ancient community on hydrologic ensemble prediction have been the substrate for the birth of real-time techniques that could only be supported by these ‘physics-killer’ types of models“, the reconstitution of which is believed to depict the prototypical hydrological model of the early years.

“It’s an excellent case of a transitional forecasting approach discovery from a critical time period in the evolution of natural hazard risk reduction“, say scientists, who are now also confident that what seems to have been called ‘the routing function’ can now be coupled to the real world of weather predictions and re-establish the long-time missing links.  This should pave the way for refocusing on very applied tasks such as reservoir management.

N:\gebhyd\2_Leitung\M_Zappa\hepex_columns_2015\mcfools\bath_flood.png

Figure 2. Inspiration for single reservoir model and benchmark for hydropower management forecasting

Furthermore, the community realizes it must go back to fundamental laws and theoretical bases of hydrology, including a renewed commitment to rigorous and methodical development and testing.

N:\gebhyd\2_Leitung\M_Zappa\hepex_columns_2015\mcfools\einzellinear_speicher.jpg

Figure 3. How real world (right) inspires and deliver proofs of model assumptions (left)

“Our aim is to put physics back in hydrological forecasting. The fact that it has features that so clearly ally with our numerical (physically based) models helps us narrow the transition of the large-scale atmospheric and land-surface conditions, and suggests that the transition itself can be relatively approached by much simpler functions than what the current hydrological ensemble prediction community is trying to convince us that should be done” said D. McFools Jr, a graduate student member of the team that discovered the modelling remains.

“A great deal is at stake here” — he added. “I’ll be presenting my recent research findings in the next HEPEX workshop in Melbourne and I expect that the HEPEX community will praise my work and finally recognize the efforts my family has been doing for more than three years now (see here, here, and here ).”

Reference: McFools Jr, D., 2017. Has the Missing Link Been Found? International Journal of Creative Alternative Facts, 4 (17), pages 456-456.5

Posted in April fools! | Leave a comment

How suitable is quantile mapping for post-processing GCM precipitation forecasts?

Posted on March 28, 2017 by QJ Wang

Contributed by QJ Wang, University of Melbourne*

Poster session during the HEPEX Workshop at SMHI in Norrköping, Sweden.

Back in September 2015 at the highly successful HEPEX Seasonal Hydrological Forecasting Workshop at SMHI in Norrkoping, Sweden, I heard a number of presentations and saw posters on the use of quantile mapping for post-processing or downscaling GCM precipitation forecasts.

While quantile mapping was well known to be highly effective in bias correction, I was concerned that some of its limitations might not have been apparent to some people.

After discussing with Andy Wood and Maria-Helena Ramos at the workshop, I left the workshop with the idea of doing a piece of work to demonstrate both the effectiveness and limitations of the quantile mapping method.

Back in Melbourne, my colleagues at CSIRO, led by Tony Zhao and James Bennett, enthusiastically took on the research task. With inputs also from Andy and Helena, we recently published the results in Journal of Climate. The paper is simply titled: How suitable is quantile mapping for post-processing GCM precipitation forecasts? Read the abstract

In brief, quantile mapping is shown, as previously known, to be highly effective for bias correction. However, if there is still an ensemble spread reliability problem after bias correction, quantile mapping cannot fix up the problem. Think of a limiting case. The ensemble members are all identical and therefore have no spread and we know the forecast is not perfect. Applying quantile mapping will not introduce even an ounce of spread to the ensemble.

When past forecasts are not correlated with observations in any way for a particular situation, we should not be insisting on using the erroneous forecasts, and should instead revert to climatology forecasts. Take this point further with another limiting case. Past forecasts are found to be negatively correlated with observations for a particular situation. Applying quantile mapping cannot reverse or remove the negative correlation, as the order of values after quantile mapping does not change.

The good news is that there are alternatives to the quantile mapping method. In the paper, we demonstrated that the Bayesian joint probability method was effective in achieving bias correction, making forecast ensemble spread reliable, and steering the forecasts to climatology when the raw forecasts had no underlying skill.

If you are interested in the topic, take a look at the paper here. The authors would love to hear your thoughts.

Finally, may I use this opportunity to let you know that I recently made a career change by joining the University of Melbourne as a professor of hydrological forecasting. Leaving the fantastic water forecasting team in CSIRO that I built and loved was one of the most difficult decisions I had to make. I look forward to continuing to work with my colleagues at CSIRO and indeed the HEPEX community to advance research and practice of ensemble hydrological forecasting. My new email contact is quan.wang@unimelb.edu.au.

* with inputs from Tony Zhao, James Bennett, Andrew Schepen, Andy Wood, David Robertson, and Maria-Helena Ramos, after several discussions in the past months.

Posted in postprocessing, seasonal prediction | 2 Comments

Developing and running an ensemble prediction system – Interview with Jutta Thielen-del Pozo

Posted on March 21, 2017 by Maria-Helena Ramos

Posted by Maria-Helena Ramos (IRSTEA)

I met Jutta Thielen-del Pozo in 2005, when I went to the JRC in Ispra, Italy, to take a post-doctoral position. I had been doing research in hydrology for some time, but I didn’t know anything about ensemble predictions when I joined her team. They were developing the European Flood Awareness System (EFAS) and the first tests using weather ensemble predictions to issue EFAS warnings were just starting. I was fortunate enough to count with Jutta’s patience and amiability to get involved in the research being done within EFAS and introduced to the HEPEX community.

This is Jutta, during the Hepex workshop in Toulouse in 2009

Jutta is now Head of the Scientific Development Unit at the European Commission since July 2016, a challenging position as the units aims at stimulating novel and trans-displinary research within the JRC through a Centre of Advanced Studies, Exploratory Research, and novel instruments such as Science and Art and Collaborative Doctoral  Partnerships between JRC and higher education institutions.

Her new position has in no way reduced her historic enthusiasm for HEPEX and its community. She acted as a HEPEX co-chair from 2007 to 2012, and much of what this initiative is today is thanks to her pushing things forward, and adding people and ideas to the group.

At the JRC, Jutta was the EFAS project leader for 10 years, from 2003-2012. She acquired a broad experience in designing and running an ensemble prediction system. Since many of us are also dealing with these issues, I have asked her some questions. Here are her answers.

MHR: In your opinion, what are the crucial choices one has to make when starting the design of an ensemble flood prediction system?

JT: First of all, I think it is crucial to understand what the system is supposed to deliver and what the expectations are. Is it a small watershed with typically flash floods where decision makers have little time to act or is it a larger river with a comparatively slower response time? Is it an urban watershed? Is the watershed entirely within one’s own administrative boundaries or are others involved? Is the decision making pathway, from information to action in case of floods, long or short? How are the vulnerability and the coping capacity of society in case of flooding?

Depending on these answers, the required lead times and acceptable accuracies of the forecasts can be determined, which, in return, will determine which input data are required and what type of models and forecasting system is most suitable. For example, for flash flood prone regions, radar data blended with nowcasting and short term forecasts will be essential, whereas for riverine flood prone areas it is rather medium-range to monthly forecasts that will be more useful.

However, I would say that although input data and process modelling are key elements for a successful ensemble flood forecasting system, equally important is a communication and training strategy, so that the results are properly communicated and understood at any point in the forecasting chain. Only then users will understand that uncertainties exist; that they are acceptable at longer lead times if at the same time they provide longer time to act; and that uncertainties can and will be reduce as the events draw nearer and more data becomes available. This is particularly true today with the plethora of data sources available, including satellite and social media.

Generally, the HEPEX community has shown that whatever type of flood, ensemble prediction systems (EPS) tend to provide longer warning times for the decision making and yield more robust results. EPS give forecasters and decision makers more time to take different preparedness actions, play through different scenarios and therefore allow both preparedness and response teams to act more decisively during the crisis.

MHR: And what are the main difficulties someone should expect when running an ensemble flood prediction system in real-time?

JT: Real time operational forecasting always puts enormous pressure on not having the process chain interrupted – and there can be manifold reasons for this. There are the technical issues on site to ensure a 24/24 business continuity (power supply, redundancy of the systems in case one or more systems fails, sufficient storage at any time of the processing chain, availability of nodes for processing…) I remember an incident of the early EFAS days when a process got into an endless loop because a simple line of code was accidently deleted, which then started filling the hard disk. That was a nightmare at the time!

Then there are software issues, e.g. in case of necessary updates not having been sufficiently tested and resulting in hiccups or termination of processes. One particular aspect of ensemble prediction systems is that it involves a lot of input and output data and, for different reasons, not all files or all ensemble members may be available.  Therefore the process chain must be able to cope with time delays, partial availability of data and files, without flawing the analysis and visualisation of files.

I think another important aspect is that, during a crisis, forecasters are often asked additional questions which then need to be answered under enormous time pressure. Therefore the system should be designed in a way that additional information and in depth analysis can be extracted at any time in an easy way without slowing down the actual forecasting capacity.

MHR: How do you see the future of HEPEX? Any topics we should focus on or new directions to consider?

JT: I think HEPEX has already taken a good direction by involving end users and different stakeholders from the beginning. Yet much of the HEPEX activity remains directed towards scientists, forecasters and civil protection. I think social media now opens up new doors to keep involving the public more directly in flood forecasting. Posting pictures while flooding is going on is an obvious way and already taking place, but possibly this could be even more integrated. There is always the prejudice that “the public don’t understand probabilities”, but perhaps it is time to put this to the test and use the outcome to the advantage of the ensemble prediction systems?

Although HEPEX is open to everybody it is not yet a full global community and mostly restricted to those scientists and end users that have the technical power to run large and complex models and data sets. Yet, flooding takes the biggest toll in those regions where such capabilities are not available. And we know that flooding disproportionally affects the poorer communities. I would therefore like to see HEPEX trying to involve also end-users and scientists from those regions where computing power is limited, and finding solutions to introduce the value of ensemble prediction there too in order to have a fairer distribution of warning information globally.

MHR: And a last question: what are your working challenges now as Head of the Scientific Development Unit at the JRC?

JT: Since my new unit targets Scientific Development in the JRC in general, it is not focused on a particular topic. Its aim is rather to become the incubator for new research not yet part of the JRC’s portfolio, to stimulate exploratory research and to use Art and Science to connect JRC scientists across the JRC and with society. Such programmes are really important – in fact, if you remember, EFAS and the ensemble prediction system benefited in 2005 from the exploratory research programme at the time.

This is exciting and challenging at the same time. In order to address the changes in sciences and changes in society, my unit fosters collaboration between natural science and technology with the social sciences and humanities. We will be involving experts at the science-policy interface. This is opening up new ways of thinking and challenges us to think across disciplines. In this sense, I hope that with my work I can continue to contribute to HEPEX and stimulate new research questions.

 Thank you, Jutta, for your time and contribution!

Posted in floods, interviews, operational systems, social participation | Leave a comment