15 June 2009

The Black Box of Risk

I have an op-ed in Tuesday's Financial Post (Canada) titled, "The Black Box of Risk." Here is the conclusion:
Efforts to create or impose certainty when certainty does not exist can be dangerous to our welfare. Consequently we have to be ever vigilant that our ability to engage in sophisticated modelling does not outstrip our ability to effectively use the results of those models in making decisions.
Read the whole thing here.


  1. Compare those crafting and using the financial models to those crafting and using the climate models. The financial industry paid for the very best statistical and analytical minds available. Climate scientists, on the other hand, have been admonished for their failure to use the services of people with a better handle on statistics.

    The financial modelers had extraordinary incentives pro and con to get it right. We are seeing the consequences of their mistakes. Climate modelers have NO downside consequence, if they are wrong. In fact, their personal circumstances are enhanced, if the models err on the side of gloom and doom.

    Given all that, Roger's admonition to be ever vigilant isn't nearly strong enough. Until the climate models have been verified and have been analyzed by outside auditors who turn them inside out until satisfied, only a complete fool would base any policy on their output.

  2. Both economic and climate systems share a fundamental characteristic - they are historical entities. Statistical models serve well when the subject is the behavior of atoms or molecules - these systems are essentially ahistorical. Chlorine gas in a jar will behave the same whether you observe it today or a billion years from now. Economic and climate systems are peturbed occasionally in ways that simple physical systems never suffer. As recently as 1990, what Nobel Prize-winning economist could have predicted the effect of the personal computer on enterprise worldwide? Where will the next major war occur, and what effect will it have on commodity prices? These things can only be modeled at a level so imprecise that the uncertainty make any results worthless. And so with climate modelling. Why was Greenland green during the Medieval period? What cause drove temperatures up for long enough to grow trees and allow agriculture? And why then, and not five hunderd years later? Or not at all? On a geological time-scale, change is the norm. On the scale of human history, unexplained perturbations have come and gone. In this case, the one thing you know is that there are things you don't know. And that means that you don't know how to model rare events with any comfidence. Garbage in, garbage out. The rest is hubris.

  3. I think there is a fundamental difference between climate modeling and financial modeling that should be mentioned. Financial models are written to provide guidance for investment decisions. As such, a model of business risk can only be accurate prior to investments made based on that model because the investment changes the initial conditions in some small way. However, if everyone is running the same models on the same initial assumptions and makes investment risk decisions based on the results, the cumulative investments make big changes to the initial conditions and the projected risk becomes inaccurate. This is what happened in the subprime fiasco. The banks that did the initial analysis and investments to high risk borrowers were correct in their risk assesment because they could count on the value of the mortgaged property to rise and investments could always be recovered through a sale. However, easy money with subprime borrowing inflated prices beyond their real values so the investment risk rose exponentially as the property sale outlet to recover lent mortgage money disappeared. There was not a "black swan", it was as predictable as an airplane stall when the pilot pull back on the yoke too sharply. Climate modeling is different in that human actions taken on the basis the results really don't change the initial conditions, at least on a large global scale. (I'll grant that the initial conditions can change on their own through natural processes.) The climate, being much more complex and poorly understood, is probably not modeled very well but at least most action on the results will not create a stall and crash that we've seen in the financial system.