Assessing the value of forecasts is a very popular topic among the HEPEX community. The assessment of forecast value is highly dependent on the purpose served by the forecasts. For the specific problem of decision-making related to flood mitigation, Murphy (1976, 1977) proposed the use of the cost-loss ratio framework. The vast majority of papers related to the assessment of forecast value for flood mitigation adopt this framework, so one could think that everything is pretty much solved… except that the cost-lost ratio has very important flaws!
One such flaws is the fact that the cost-loss ratio assumes that the decision maker is risk neutral. Risk neutral individuals only care about the expected outcome, and disregard the spread of the distribution of outcomes. Risk neutral individuals are very rarely encountered in real life. Indeed, most of us are risk averse. That is, for the same expected outcome, we prefer less risky distributions. This is (among other things) why we buy insurance. Informally, most people dislike risk and would be willing to spend resources (e.g. money) in order to reduce the amount of risk faced.
Economists (and statisticians and mathematicians) have been studying those issues for a long time, and came up with many models and concepts which could be used in hydrology, as an alternative to the cost-loss ratio. The central framework is based on “utility theory”. To put it’s development a bit in a context, here is a (very) pseudo-historical adaptation of a conversation between Nicolau I Bernoulli and his cousin Daniel :
It didn’t happen exactly like that, but the general idea is preserved. The game that Nicolau is referring to is now known as the St-Petersburg paradox and was first described in a letter. The first person to present it more formally was Daniel Bernoulli in 1738. He was also the first to suggest the idea of risk aversion and exposed (without any mathematics) that different persons faced with the same decision problem and the same information could take different decisions, because of different preferences.
Utility theory is not perfect (and indeed, many generalizations and extensions exist), but it has at least two advantages over the cost-loss ratio:
- It allows for decisions to consider a finite number or a continuum of decisions (e.g. “protect” vs “don’t protect” for floods, or alternatively, “amount spent in protection”).
- It allows to account explicitly for the decision maker’s level of risk aversion in the assessment of forecasts value.
Krzysztofowicz suggested using utility theory in hydrology as early as in 1986. It is not a miraculous solution to the problem of assessing the value of forecasts for flood mitigation, but it would be an improvement over the cost-loss ratio. It can explain real-life behaviors that the cost-loss ratio cannot. This was the case in our recent application for the assessment of forecasts value on the Montmorency River in Canada (Matte et al. 2017). Other examples of accounting for risk-aversion for decision-making in hydrology and meteorology include Shorr (1966), and Cerdá and Quiroga Gómez (2008).
- Bernoulli D. (1738) Specimen Theoriae Novae de Mensura Sortis, Commentarii academiae scientiarum imperialis Petropolitanae, 5, 175-192.
- Cerdá Tena E. and Quiroga Gómez S. (2008) Cost-Loss Decision Models with Risk Aversion, Working paper no. 01, Instituto Complutense de Estudios Internaciolales, 28 pages.
- Krzysztofowicz R. (1986) Expected utility, benefit, and loss criteria for seasonal water supply planning, Water Resources Research, 22(3), 303-312.
- Matte S., Boucher M-A, Boucher V. and Fortier-Filion T-C (2017) Moving beyond the cost-loss ratio, economic assessment of streamflow forecasts for a risk-averse decision maker, Hydrology and Earth System Sciences (Accepted)
- Murphy A.H. (1977) Value of climatological, categorical and probabilistic forecasts in cost-loss ration situation, Monthly Weather Review, 105(7), 803-816
- Murphy A.H. (1976) Decision-making models in cost-loss ratio situation and measures of values of probability forecasts, Monthly Weather Review, 104(8), 1058-1065
- Shorr B. (1966) The cost/loss utility ratio, Journal of Applied Meteorology, 5(6), 501-803.
- von Neumann J. and Morgenstern O. (1944) Theory of games and economic behavior, vol. 60, Princeton University Press Princeton, 625 pages.
 The original pictures were taken from these websites: http://www.famous-mathematicians.com/daniel-bernoulli/ (right) and http://www2.stetson.edu/~efriedma/periodictable/html/Bi.html (left). The schematic drawing of D. Bernoulli’s blood pressure experiment apparatus was taken from https://plus.maths.org/content/daniel-bernoulli-and-making-fluid-equation
 The original pictures were taken from these websites: http://www.tinbergen.nl/oskar-morgenstern-and-john-von-neumann-shelby-white-and-leon-levy-archives-center/ (left) and http://www.karbosguide.com/books/pcarchitecture/chapter02.htm (right)