31 January 2012

Economic Models: Caveat Utilitor!

The Australia Institute has released a very nice report by Richard Denniss titled, The use and abuse of economic modelling in Australia: Users' guide to Tricks of the Trade (PDF).  The essay illustrates its critique with several recent cases related to claims about jobs in the mining industry, the poker machine industry and as a consequence of the carbon tax.

Here is an excerpt:
Economic modelling has, for many people involved in Australian policy debates, become synonymous with the process of serious policy development. Proponents of policy change that are armed with economic modeling are often taken more seriously than those with 20 years experience working on the same problem. The modelling result that suggests tens of thousands of jobs will be lost or created often trumps logic or experience that suggests such claims are nonsensical.

This is not to suggest that modelling has no role to play in policy debates. It can and it does often make a useful contribution, but the fact that it sometimes can should not be confused with the conclusion that it always will. Indeed, in recent times some of the claims based on 'economic modelling' that has been made in debates such as the likely impact of poker machine reform or the introduction of a carbon price can only be described as nonsense.

The problem has become, however, that in an era in which segments of the media no longer have the time or inclination to examine claims before they are reported bad economic modelling is preferred by many advocacy and industry groups to good economic modelling for three main reasons:

1. it is cheaper
2. it is quicker
3. it is far more likely to yield the result preferred by the client

That said, bad economic modelling is relatively easy to identify if readers are willing to ask themselves, and the modeller, a range of simple questions. Indeed, it is even easier to spot when the modeler can't, or won't, answer such simple questions.
Economic models, like all models, can be very useful. But they can also be used in ways that are misleading or just plain wrong. Denniss provides some good advice for recognizing the difference.


  1. Modelling has become the PowerPoint presentation of the scientific world.

    I think Pielke, Sr.'s classification of models as hypotheses is correct. They are conceptual guides which may direct our efforts. They are to be proven and should not be confused with empirical evidence. Their value is commensurate with the completeness of the system's characterization and representation. When used as a premise for policy prescriptions, they should be generally considered to increase risk, and evaluated accordingly.

  2. Pity Richard is not able to see the same problems are inherent in climate models!

  3. Many economic models aka econometrics have long been pointed to as merely exercises in confirmation bias.

  4. -1 n.n.,

    Matt Briggs has a series of posts about this over the last few days, starting here. It's in reference to statistics, but it gets at the core of how we should be thinking about models in a way that's applicable to broader types of models.