10 March 2011

It is Always the Media's Fault


Last summer NCAR issued a dramatic press release announcing that oil from the Gulf spill would soon be appearing on the beaches of the Atlantic ocean.  I discussed it here.

Here are the first four paragraphs of that press release:
BOULDER—A detailed computer modeling study released today indicates that oil from the massive spill in the Gulf of Mexico might soon extend along thousands of miles of the Atlantic coast and open ocean as early as this summer. The modeling results are captured in a series of dramatic animations produced by the National Center for Atmospheric Research (NCAR) and collaborators.

The research was supported in part by the National Science Foundation, NCAR’s sponsor. The results were reviewed by scientists at NCAR and elsewhere, although not yet submitted for peer-review publication.

“I’ve had a lot of people ask me, ‘Will the oil reach Florida?’” says NCAR scientist Synte Peacock, who worked on the study. “Actually, our best knowledge says the scope of this environmental disaster is likely to reach far beyond Florida, with impacts that have yet to be understood.”

The computer simulations indicate that, once the oil in the uppermost ocean has become entrained in the Gulf of Mexico’s fast-moving Loop Current, it is likely to reach Florida's Atlantic coast within weeks. It can then move north as far as about Cape Hatteras, North Carolina, with the Gulf Stream, before turning east. Whether the oil will be a thin film on the surface or mostly subsurface due to mixing in the uppermost region of the ocean is not known.
A few weeks ago NCAR's David Hosansky who presumably wrote that press release, asks whether NCAR got it wrong.  His answer?  No, not really:
During last year’s crisis involving the massive release of oil into the Gulf of Mexico, NCAR issued a much-watched animation projecting that the oil could reach the Atlantic Ocean. But detectable amounts of oil never made it to the Atlantic, at least not in an easily visible form on the ocean surface. Not surprisingly, we’ve heard from a few people asking whether NCAR got it wrong.

These events serve as a healthy reminder of a couple of things:

*the difference between a projection and an actual forecast
*the challenges of making short-term projections of natural processes that can act chaotically, such as ocean currents
What then went wrong?
First, the projection. Scientists from NCAR, the Department of Energy’s Los Alamos National Laboratory, and IFM-GEOMAR in Germany did not make a forecast of where the oil would go. Instead, they issued a projection. While there’s not always a clear distinction between the two, forecasts generally look only days or hours into the future and are built mostly on known elements (such as the current amount of humidity in the atmosphere). Projections tend to look further into the future and deal with a higher number of uncertainties (such as the rate at which oil degrades in open waters and the often chaotic movements of ocean currents).

Aware of the uncertainties, the scientific team projected the likely path of the spill with a computer model of a liquid dye. They used dye rather than actual oil, which undergoes bacterial breakdown, because a reliable method to simulate that breakdown was not available. As it turned out, the oil in the Gulf broke down quickly due to exceptionally strong bacterial action and, to some extent, the use of chemical dispersants.

Second, the challenges of short-term behavior. The Gulf's Loop Current acts as a conveyor belt, moving from the Yucatan through the Florida Straits into the Atlantic. Usually, the current curves northward near the Louisiana and Mississippi coasts—a configuration that would have put it on track to pick up the oil and transport it into open ocean. However, the current’s short-term movements over a few weeks or even months are chaotic and impossible to predict. Sometimes small eddies, or mini-currents, peel off, shifting the position and strength of the main current.

To determine the threat to the Atlantic, the research team studied averages of the Loop Current’s past behavior in order to simulate its likely course after the spill and ran several dozen computer simulations under various scenarios. Fortunately for the East Coast, the Loop Current did not behave in its usual fashion but instead remained farther south than usual, which kept it far from the Louisiana and Mississippi coast during the crucial few months before the oil degraded and/or was dispersed with chemical treatments.

The Loop Current typically goes into a southern configuration about every 6 to 19 months, although it rarely remains there for very long. NCAR scientist Synte Peacock, who worked on the projection, explains that part of the reason the current is unpredictable is “no two cycles of the Loop Current are ever exactly the same." She adds that the cycles are influenced by such variables as how large the eddy is, where the current detaches and moves south, and how long it takes for the current to reform.

Computer models can simulate the currents realistically, she adds. But they cannot predict when the currents will change over to a new cycle.

The scientists were careful to explain that their simulations were a suite of possible trajectories demonstrating what was likely to happen, but not a definitive forecast of what would happen. They reiterated that point in a peer-reviewed study on the simulations that appeared last August in Environmental Research Letters. 
So who was at fault?  According to Hosansky it was those dummies in the media:
These caveats, however, got lost in much of the resulting media coverage.
Another perspective is that having some of these caveats in the press release might have been a good idea.

8 comments:

  1. NCAR got it wrong.

    Likely and could possibly have different meanings.

    Whether the caveats were buried in the text is irrelevant when communicating to the public.

    Relying on the arguments used by deceptive car dealers that the actual terms and conditions of the sale were buried in the fine print puts 'science' on the same plane as 'car sales'.

    Science as an institution is in trouble when the nations premier scientific institutions start using arguments that Billy Joe Bob of Billy Joe Bob's Auto Emporium use.

    ReplyDelete
  2. Putting caveats into the press release would have been useful. As would have very publicly announcing, as soon as it was possible to tell, that the projections were wrong.

    The hiding behind "projections" versus "forecasts" is dishonest, because they almost certainly would have taken credit if their "projections" had come true.

    ReplyDelete
  3. It appears a lot of these caveats were placed in the press release.

    I don't have a huge problem with this other than I think creating press releases for projections this uncertain is not a great idea. It's just asking for nature to prove the fancy computer model wrong in public view. I'm guessing others think the publicity while the oil spill was big news was worth it. Hopefully they learn from the differences between the various model runs and what really happened and make improvements to their model.

    ReplyDelete
  4. Aggressive self-promotion plus a media dominated by spreading fear and hype for profit meets and reality is the victim.
    Nothing new in this, really.

    ReplyDelete
  5. How do you know when they're lying? Beyond the 'lips moving' business, the disingenuous slip-sliding between 'forecast' and 'projection.' When they want you to trust them, it's a forecast. When they're proved wrong, it's a projection. Just another 'trick' to 'hide the failure.'

    Yes, I know I sound bitter. I always reserve the right to lose patience with scoundrels.

    ReplyDelete
  6. Their big mistake was making a forecast...er...cough,cough... projection on too short a timeframe. Typically, you allow enough time to comfortably publish dozens of papers, apply for additional grants, and get the requisite promotions before you can be proven right or wrong.

    I'm surprised they made the rookie mistake of projecting a catastrophe that could actually be evaluated within the timeframe of their working careers.

    ReplyDelete
  7. Their defense amounts to that they only said "might". However they also said "our best evidence" and "likely".

    May I point out though that yet again the predictions of the skeptics turned out to be correct regarding the strong bacterial action breaking down the oil. I thought that was hugely optimistic but I was wrong.

    From this episode then should we conclude that we can consider all press released projections to be only the pessimimistic worst case scenarios, with the more optimistic projections being left on the cutting room floor?

    ReplyDelete
  8. It might have been helpful if they had included an estimate of the dilution their model predicted.

    In any case, people inside the oil industry were saying things like "We know it'll be biodegraded quickly, but we can't say that in public".

    Nobody knowledgeable who didn't have a political axe to grind, or a grant to apply for, was surprised at how quickly the bacteria did their bacteria thing.

    ReplyDelete