27 January 2010

Karen Clark on Short-Term Hurricane Loss Predictions

Karen Clark and Co. have an interesting new report out which evaluates the performance of short-term hurricane predictions issued by the catastrophe modeling industry. In short, they are not doing so well, as the image above indicates, with predicted losses far exceeding actual losses. Here is an excerpt from the report:
Given all of the uncertainties, near term projections do not have sufficient credibility to be used for important insurance applications such as product pricing and establishing solvency standards. In the case of pricing catastrophe exposure, the insurer or reinsurer is faced with the challenge of settling on a specific price for a specified time period for an exposure that has a highly uncertain expected value. While the near term models might be a useful tool for adding insight with respect to the potential range of expected outcomes for the upcoming policy period, the actual results of the last four years indicate that relying exclusively on the near term models to determine a rate can bring an extra level of instability and volatility to an already challenging pricing exercise. Individual insurers and reinsurers should instead consider the complete range and likelihood of possible outcomes in determining product pricing, taking into account the need for both stability and responsiveness in setting a strategy for pricing their products.
The perspective expressed by Karen Clark and Co. is quite similar to my own views, expressed in a paper published in 2009. Our website remains down, due to a concerted attack to deny access, but for anyone interested I'd be happy to email a copy of the paper, the title and abstract appear below.
United States hurricane landfalls and damages: Can one- to five-year predictions beat climatology?
Pielke, Roger A.
Environmental Hazards, Volume 8, Number 3, 2009 , pp. 187-200(14)

This paper asks whether one- to five-year predictions of United States hurricane landfalls and damages improve upon a baseline expectation derived from the climatological record. The paper argues that the large diversity of available predictions means that some predictions will improve upon climatology, but for decades if not longer it will be impossible to know whether these improvements were due to chance or actual skill. A review of efforts to predict hurricane landfalls and damage on timescales of one to five years does not lend much optimism to such efforts in any case. For decision makers, the recommendation is to use climatology as a baseline expectation and to clearly identify hedges away from this baseline, in order to clearly distinguish empirical from non-empirical justifications for judgements of risk.
Also, I participated in an AM Best roundtable discussion of catastrophe risk with insurance experts several weeks ago. The issue of catastrophe models was a part of the conversation. You can read a transcript of the discussion here. That is me below at the roundtable, talking about continued growth of losses based on our work.


  1. Roger, what is "a concerted attack to deny access"?

  2. -1-Roddy

    This sort of thing:


    It's been down for almost a week.

  3. You must be doing something right to be attacked in this way.

  4. Thank you. Why would anyone do that? Not a terribly amusing target surely.

  5. Thanks for the sentiments. I do feel bad for my colleagues and our collected students who suffer the consequences. There is no way to know, but I suspect it has to do with my writings. One reason i moved my blogging off campus.

    Anyway, let's get back to cat models ;-)

  6. Predicting seasonal hurricane numbers with much accuracy seems like a lost cause. One can give certain estimates based on factors that, on average, increase or decrease the likelihood of storms. Total hurricane estimates are always going to be in the +/- 3 storm or so range though. The factors that go into an individual storm forming and then becoming a hurricane are too diverse and variable. We've only recently had the tools that can estimate when a tropical wave has increased chances of becoming a storm. Still, waves that look favorable often can't form a closed circulation and sometimes what seems like nothing turns into a storm in a couple of days.

    The number of factors to consider only increase when trying to predict landfall. It takes a lot to come together for a storm to be able to traverse the Atlantic without recurving or running into unfavorable conditions. Trying to predict damage and cost impacts is even more impossible. A shift of 20 miles in a storm's landfall can make billions of dollars of difference.

    I think we can make some broad estimates based on climatology. There's trends in the numbers based on various weather conditions around the world such as ENSO state, but there's high variability.

    The only thing for sure is that the more we build on the east and gulf coasts, the higher our risk will be. We've been relatively lucky over the last 50 years. Just imagine if Andrew had made landfall 20 miles further north. Miami would have seen a direct hit. Or if Katrina would have hit 20 miles further west. New Orleans would have seen a few categories higher wind damage along with the flooding. Or if Hugo had made a path closer to Charleston or Myrtle Beach. Also, when a hurricane actually threads the needle and hits NYC from the sea, the damage totals will be very high.

  7. There is apparently an amazongate now.

    Amazonian forests were risk from global warming and would likely be replaced by "tropical savannas" if temperatures continued to rise.

    This claim is backed up by a scientific-looking reference but on closer investigation turns out to be yet another non-peer reviewed piece of work from the WWF. Indeed the two authors are not even scientists or specialists on the Amazon: one is an Australian policy analyst, the other a freelance journalist for the Guardian and a green activist.

    The WWF has yet to provide any scientific evidence that 40% of the Amazon is threatened by climate change -- as opposed to the relentless work of loggers and expansion of farms.

  8. Hurricane numbers, like all small number events, follow Poisson statistics over the short term (lambda, the average number of storms/year, may vary over the longer term). That means the variance is the number and the standard deviation is the square root of the number. So if the average is 10 storms in a year then the 90% confidence interval is 5 to 15 storms. For an average of 5/year, the interval is 1 to 9. Over the short term then, going with the average over the last few decades will likely be as or more skillful than a pre-season prediction by other means.

  9. Mr. Pielke:

    Lets assume for a moment that forecasters have the statistical probability of hurricanes exactly correct and that the statistical damage value is exactly correct.

    The hurricane subject area aka possible damage area is a known. The hurricane subject area is currently broken up into geographic governmental entities known as states. Roughly the state of Texas along the coast eastward ending in Connecticut.

    In each of these state entities, within the subject area, we have homogeneous exposure units concentrated in a narrow geographic area. Private welfare systems aka private insurance is based on homogenous exposure units spread over a wide geographic area. In other words, violated is an axiom within private welfare systems, by concentrating exposure in a narrow geographic area.

    Insurance is a retained state right i.e. regulated state by state. Insurers pricing occurs state by state.

    However, to solve for the insurance axiom violation of homogeneous exposure units concentrated in a narrow geographic area, the state entities within the subject area need to create one larger exposure area which is basically the subject area. In other words, the coastal areas of the many and several states needs to become one large exposure area.

    Not a federal plan, rather a syndicate of the states. The states with a hurricane exposure take their coastal counties and create one insurance territory aka exposure area. We then have a much wider geographic area with homogeneous exposure units. This wider geographic area is then priced for base private insurance and re-insurance. Its an arbitrary insurance territory with the states dropping state line regulatory issues and acquiescing to the syndicate's regulatory/coverage decisions with the syndicate made up of the states.

    In summary, you have to find a way in the hurricane subject area to overcome the violation of the axiom, within private welfare systems, that homogeneous exposure units need spread over a wide geographic area.

  10. OT "Policy Question"

    What would happen if US, Russian Federation and Australia agreed to 'coal production caps'?

    In my simple mind, creating an artificially scarce good benefits the producers of those goods. Hence, "Big Coal" in the US would benefit from 'climate legislation' thru increased profitability of existing operations.