Writing in Nature this week, Dan Sarewitz reflects on his recent participation on the BPC
Geoengineering Climate Remediation task force and why efforts to achieve consensus in science may leave out some of the most important aspects of science. Here is an excerpt:
The very idea that science best expresses its authority through
consensus statements is at odds with a vibrant scientific enterprise.
Consensus is for textbooks; real science depends for its progress on
continual challenges to the current state of always-imperfect knowledge.
Science would provide better value to politics if it articulated the
broadest set of plausible interpretations, options and perspectives,
imagined by the best experts, rather than forcing convergence to an
allegedly unified voice.
Yet, as anyone who has served on a consensus committee knows, much
of what is most interesting about a subject gets left out of the final
report. For months, our geoengineering group argued about almost every
issue conceivably related to establishing a research programme. Many
ideas failed to make the report — not because they were wrong or
unimportant, but because they didn't attract a political constituency in
the group that was strong enough to keep them in. The commitment to
consensus therefore comes at a high price: the elimination of proposals
and alternatives that might be valuable for decision-makers dealing with
complex problems.
Some consensus reports do include dissenting views, but these are
usually relegated to a section at the back of the report, as if
regretfully announcing the marginalized views of one or two malcontents.
Science might instead borrow a lesson from the legal system. When the
US Supreme Court issues a split decision, it presents dissenting
opinions with as much force and rigour as the majority position. Judges
vote openly and sign their opinions, so it is clear who believes what,
and why — a transparency absent from expert consensus documents. Unlike a
pallid consensus, a vigorous disagreement between experts would provide
decision-makers with well-reasoned alternatives that inform and enrich
discussions as a controversy evolves, keeping ideas in play and options
open.
Not surprisingly, Dan and I have come to similar conclusions on this subject. Back in 2001 in Nature I wrote (
PDF):
[E]fforts to reduce uncertainty via ‘consensus science’ — such as scientific assessments — are misplaced. Consensus science can provide only an illusion of certainty. When consensus is substituted for a diversity of perspectives, it may in fact unnecessarily constrain decision-makers’ options. Take for example weather forecasters, who are learning that the value to society of their forecasts is enhanced when decision-makers are provided with predictions in probabilistic rather than categorical fashion and decisions are made in full view of uncertainty.
As a general principle, science and technology will contribute more effectively to society’ needs when decision-makers base their expectations on a full distribution of outcomes, and then make choices in the face of the resulting — perhaps considerable — uncertainty.
In addition to leaving behind much of the interesting aspects of science, in my experience, the purpose of developing a "consensus" is to to quash dissent and end debate. Is it any wonder that policy discussions in the face of such a perspective are a dialogue of the like minded? In contrast, as Sarewitz writes, "a vigorous disagreement between experts would provide
decision-makers with well-reasoned alternatives that inform and enrich
discussions as a controversy evolves, keeping ideas in play and options
open."
14 comments:
Consensus in action!
by email from another in Blogger purgatory ...
"“Science might instead borrow a lesson from the legal system”. My knowledge here would be of the European legal system rather than the US system but I think that it is a very interesting idea. Not only is the dissenting opinions presented with as much force and rigour as the majority position but the rule is also that “newest” judge goes first when the relevant case is discussed among the judges. This is of course to test the poor man/woman but also to prevent “false” consensus and avoid some stern old judge quashing dissent and ending debate by forcefully expressing their view from the outset, scaring the less experienced judge into submission. The dissenting views are often of great value for later cases and only very lazy lawyers to not read them ....;o)
Kind regards,
Hannah"
There is your tag -- Consensus = lazy, at best (fraudulent, at worst)
Consensus generally favors the status quo, as that is the position with the most vested interests. As Dan articulates well, consensus is antithetical to good science.
Consensus also degrades good policy advocacy, for much the same reason. I have often watched environmental advocates decide strategy through consensus, guaranteeing that the dullest ideas prevail.
"In addition to leaving behind much of the interesting aspects of science, in my experience, the purpose of developing a "consensus" is to to quash dissent and end debate."
I assume that references your experience with reaching consensus in debates about science?
As it reads, it could, possibly, be with reference to reaching consensus as a general principle.
I'll just point out that in my experience - outside the domain of scientific debates - the "purpose" of reaching consensus is diametrically opposed to quashing dissension and ending debate - but instead to allow dissent to be openly explored in such a way that it doesn't necessarily impede progress - by incorporating dissenting perspectives into final resolutions.
I see nothing inherent about scientific debates that would necessarily imply a differing outcome of consensual processes than in other areas of debate. That isn't to say, however, that in any particular process of debate, consensual processes might not be poorly carried out.
Perhaps you could elaborate what distinguishes scientific debate in that regard from debates in other areas?
Have you ever been to a Quaker meeting, Roger?
Don't know if this got lost - so I'll repost.
Roger - you might be interested in viewing this example of "consensus" that has gone viral in the rightwing blogosphere. The clip is from what took place when Congressman John Lewis showed up at an Occupy Wall Street event in Atlanta.
Many commenters have an averse reaction to the mechanics of this process - the repeating of what people say (a process to ensure that everyone hears the comments), and the voting methods (used to expedite voting and allow for the full participation of the hearing impaired), but many go further to assert that Lewis' voice was suppressed, and further it has been asserted that the unfolding events showed racism among the OWS crowd.
The context is that Lewis showed up at the event, and some of the participants wanted Lewis to speak even though it wasn't on the pre-determined agenda. It had previously been decided that an agenda can only be changed if the group reaches a consensus on the change.
Here's what I see:
1) An orderly process by which everyone in a large group hears a variety of opinions.
2) An orderly process by which everyone in a large group participates in decision-making in relatively short order.
http://www.youtube.com/watch?v=3QZlp3eGMNI
Please note that the person who speaks up in opposition to Lewis speaking first praises Lewis' contributions to society.
Also please note Lewis' body language as he listens to the opposition to him speaking being voiced.
Now Fox News has a post up saying that Lewis was "stunned" by the events. Take a look at Lewis speaking about the events and see what you think:
http://www.youtube.com/watch?v=_XCLBNkBvL0&feature=youtu.be
Fox News spins this event as a "Lord of the Flies" type of self-governance that leads to oppression. To me, that proves that people can spin consensus processes in any way they choose. That doesn't mean that their characterizations are accurate.
http://www.foxnews.com/politics/2011/10/10/occupy-atlanta-protesters-use-assembly-rules-to-prevent-rep-lewis-from-speaking/
A scientific renaissance? Perhaps. Recalling the true spirit of science? Definitely.
When confronted with an incompletely characterized [open] system; with a hypothesis supported through limited, circumstantial evidence; we should respond with actions commensurate to our confirmed knowledge and evidenced forecast of risk.
The efforts of some to quell dissent through consensus is reminiscent of humanity's darker days. Consensus is a social construct. The scientific equivalent is confirmation. The former may be acceptable in matters of philosophy, but it should not be accepted in scientific endeavors.
Anyway, thank you, Dr. Pielke, for defying the consensus, so that we may continue the process of understanding our world. There is no value to be gained for humanity, if we prematurely restrict our observations to the most obvious, or most desirable, sets of speculation.
Let the evidence and arguments be presented. Demand that individuals with a hypothesis respond to contrary evidence and arguments. This is especially critical when an enterprise is funded through involuntary exploitation (e.g., taxes), if we hope to mitigate corruption of authoritarian interests.
-5- Joshua,
Having watched the video, what really struck me was that initially, there seemed to be a lot of support for allowing Rep Lewis to speak then and there. I got the feeling that the "moderator" or whatever didn't agree with that, and did his best to steer people away from the disruption of their agenda.
The vocal displeasure in the outcome was certainly evidence that whatever happened wasn't consensus. It's possible that a majority of the people there were in favor of the outcome, and it was a legitimate consensus. We can only see what the camera saw.
I think it would have been pretty easy, though, for a different moderator to drive a different conclusion to consensus. Of course, the need for a moderator just highlights the contradiction of some people being more equal than others.
MIC CHECK!
Not only the science, but the formal independent verification and validation (IV&V) of the global climate models (GCMs) can use this SCOTUS-style approach too.
For all software, not just the GCMs, the cost of IV&V must be weighed against its benefits (largely risk reduction/confidence building). Therefore, it is *always* necessary, based on the nature and usage of the software, to tailor the IV&V requirements and processes. (The IV&V appropriate for scientific work is not the same as that appropriate for policy decisions.)
But tailoring is subjective. So we can encounter stakeholder-related issues such as 1) there is incomplete understanding of the nature of IV&V or 2) there is ethically/politically/financially motivated obfuscation.
I've used this SCOTUS-style technique when presenting the "consensus" IV&V process/requirements tailoring to potentially contentious stakeholders for their acceptance. Obfuscation is more difficult using this approach.
Also, quality hinges on its corrective action processes. By putting forward a diversity of opinion, later necessary corrective actions become easier. Yet the documented majority tailoring opinion provides the justification for moving forward in face of risk.
Matt -
That's an interesting conjecture. I agree that the moderator had a predisposition to not have Lewis speak. But the reason for that seemed to be the pre-existing agenda and from what I understand, an agreement that the agenda should only be changed if there were consensus - which there wasn't.
I don't see any reason to infer that the "moderator" didn't want to hear what Lewis had to say, so much as that he was committed to the consensus-based process. Even if a majority wanted to hear Lewis speak, there wasn't an consensus.
That might be a downside for an agenda-driven process as opposed to a majority vote-driven process. But bringing it back to the questions of whether there is a compatibility between consensus viewpoint and science, I doubt that many people would argue that policies should be based on science that is determined by majority votes (as opposed to consensus).
If that were the case, then we would probably have an even less deliberative process than we currently have, don't you think (given the likely majority viewpoint of scientists on AGW?
Perhaps better is to have essentially what we have: a combination - an evaluation of the science that is loosely based on consensus evaluation that then undergoes a policy development process that is filtered through majority voting - albeit indirectly so in via our highly imperfect (and subject to disproportionately influenced) electoral processes.
In lieu of a consensus-based process of evaluating science as a guide to policy development, what should we have?
-10- Joshua,
I think the analogy between the two is whether either science or the Occupy Atlanta folks really had consensus for anything. Who says there is a consensus? Does anyone disagree (that there is a broad consensus, not with its conclusions)?
I think knowing how broad any particular bit of science is supported is a reasonable thing to consider when making policy. If only we had that on any sort of "environmental" issue.
Climate seems to be worse than many fields (viz. Climategate, changing the meaning of peer review literature, etc). I've heard rumors that someone around these parts has some sort of theory about honest brokering. That sounds like a good thing. Faking or forcing consensus seems antithetical to that to me.
Joshua said... 5
I'll try to illustrate Roger's point with an anecdote
Many years ago as a young child. We were returning from a family vacation in Maine to Connecticut. I saw a sign that said 'Hartford, CT..turn here'
The consensus among my parents and siblings was that the sign I saw was for Hartford, VT not Hartford, CT and that I had simply made a mistake of confusing the V for a C.
An hour and a half later, when it became clear beyond a reasonable doubt that we were headed in the wrong direction, I found myself sitting in the front seat of the car, trying to guide my father back to the point where I had seen the sign that said 'Hartford..turn here'.
Recording the dissenting view and placing it in full view helps later in the event that the consensus view ends up wrong.
This is why the Supreme Court always publishes the 'dissenting' opinions. The possibility that they were 'wrong' always exists.
I agree with everything Dan Sarewitz says about how consensus can undermine vigorous debate, and I agree that its usually better for advisory committees to articulate a range of opinions. But that doesn't mean "consensus is for textbooks." If the possibility of consensus -- or at least some kind of agreement -- were entirely absent, there would be no reason to discuss at all. Discussion without some possibility of agreement, however limited or remote, may be self-expression or advocacy, but it's not discussion.
Roger, I think this an important issue for crisis management as well as policy, with the financial crisis starting 2007 in mind.
I have blogged at http://djmarsay.wordpress.com/2011/10/11/the-voice-of-science-lets-agree-to-disagree-nature/ and commented on the Nature site.
Where I would disagree with you is in your implicit assumption that conventional probability theory is adequate. I go along with Keynes, that it isn't always. My wider blog is on this and related topics. Regards.
Post a Comment