As Maria said in the Sound of Music, “let’s start at the very beginning, it’s a very good place to start”. In order to talk about the Guidelines, let’s first deconstruct the President’s memo- sentence by sentence, equipped with a handy online dictionary. I have an earnest belief that unclear concepts cannot constitute a firm foundation for sound public policy.
(1) Science and the scientific process must inform and guide decisions of my Administration on a wide range of issues, including improvement of public health, protection of the environment, increased efficiency in the use of energy and other resources, mitigation of the threat of climate change, and protection of national security.I would argue that the words “and guide” decisions of my Administration strays a bit from what we currently think the role of science and policy should be.
Let’s just use Merriam-Webster online for consistency, GUIDE:
1. to act as a guide to : direct in a way or courseSo if we take this sentence literally, the authors are using a word usually defined as including the concept of direction; even “to a particular end.”
2a : to direct, supervise, or influence usually to a particular end
b : to superintend the training or instruction of
Sarewitz describes some of the problems with this concept in his Issues piece here.
(2) The public must be able to trust the science and scientific process informing public policy decisions.This is a laudable goal, but the fact (as described in my Conveyor Belt post here) is that the people involved in developing policy do not have a voice in framing the research questions, for the most part. And putting together the research snippets developed to inform policy decisions is not, in and of itself, science. Then there is the absence of QA/QC in many research projects putatively designed to inform policy. Finally, if we really cared about peer review for research important to policy, we would monitor who does the reviews, pay people to do it, and make public the review comments and replies.
(3) Political officials should not suppress or alter scientific or technological findings and conclusions.Hopefully, non-political officials wouldn’t do this either. However, I think this might be a potential minefield – is not doing what the professional wants “suppressing the findings” or just disagreeing with how they should be used in informing policy?
(4) If scientific and technological information is developed and used by the Federal Government, it should ordinarily be made available to the public.Good on that. I would also say that their papers in journals should be available for free to taxpayers.
(5) To the extent permitted by law, there should be transparency in the preparation, identification, and use of scientific and technological information in policymaking.This seems generally like a good idea. In my agency, we have been required to document how we use “the best available science” for certain decisions, and it seems to work fairly well.
(6) The selection of scientists and technology professionals for positions in the executive branch should be based on their scientific and technological knowledge, credentials, experience, and integrity.This seems pretty straightforward- except I’m not sure that we have ever considered “integrity” as a selection criterion and it seems a bit out of place interjected here; especially since the “integrity” idea seemed originally to be about political employees not listening to scientists. I also wonder how the Office of Personnel Management would feel about “integrity” and the merit promotion concept. Let’s turn to the Merriam-Webster definition:
Definition of INTEGRITYI guess I’m kind of lost here.. hopefully it would be good to have all employees exhibit integrity- but which moral values? How would you measure them? If you are going to call a reference and ask them about someone’s “integrity”, it seems to me that you should have a good idea exactly what you mean. Otherwise, you could get colorful stories about their activities following imbibing of certain beverages.
1: firm adherence to a code of especially moral or artistic VALUES : INCORRUPTIBILITY
2: an unimpaired condition : SOUNDNESS
3: the QUALITY or state of being complete or undivided : COMPLETENESS
I turned to Wikipedia here and my neurons just about experienced meltdown. When scientist terminology meets philosophy terminology there’s always a high potential for generalized fuzziness.
What is the difference between general old integrity and “scientific” integrity? And if politicals ignoring the “science” are the problem, why are we going after the science and technology professionals?
We may see in the next post, where we move on to the principles articulated in the President’s memo.
6 comments:
(6) The selection of scientists .... and integrity.
Not to worry
I'm sure Charlie Rangel and Pat Robertson are both available to serve as official judges of 'integrity', of course Senator Inhoffe has some ideas on integrity as well. :)
How about this: if it's wrong, don't do it?
Do you really need bureaucratic gobbledygook to know right from wrong?
Broad documents like this do not typically have the detailed kinds of explanations that I think would satisfy Sharon. Those would typically be placed in a set of regulations, an OMB circular (assuming there are revisions to the testimony review process), or specific agency integrity guidelines.
"And if politicals ignoring the “science” are the problem, why are we going after the science and technology professionals?"
This isn't about politicals 'ignoring' the science, it's about making it harder for politicals to manipulate the science and the research they use in forming policy (and easier to detect the inevitable instances of same). At least for the Executive Branch. Congress is not bound by this.
As for going after science and technology professionals, I don't see it. If the points are followed, then the science and technology professionals relied on for scientific and technological advice would be those whose work follows accepted practices and procedures and would not provide concerns over the quality of their output or their methods.
This document is intended for instances in which science and/or scientific research will be used in making policy decisions. I look at point 1 as describing what policy areas where this might happen in this Administration.
Point 2 is critical, and might address some of the confusion surrounding this whole thing. The target here is the science and scientific research used. That is what the Administration wants to make sure the public can trust.
A side issue is that a lot of research is used to inform policy that wasn't conducted with that in mind. What Sharon writes comes across as an interest in limiting the research used to inform policy to only that research which is conducted to inform policy. I think that would restrict way too much information unnecessarily (for instance, I think this would eliminate nearly all National Academies studies). While directed research is great and useful, it can't always be conducted on the same timeline as the policy questions being answered. Additionally, there's an implication that only research conducted specifically to inform policy is capable of doing so. That's a linear model, and they never tell the whole story.
Points 3, 4 and 5 are tightly linked, IMO. By disclosing the scientific findings (4), it's much easier to confirm instances where altering or suppression (3) of those findings takes place. Issues related to the government doing something the scientists didn't like or didn't recommend comes to Point 5. Point 5 is and will be tricky, as it could make it easier to see where science and science advice was ignored, and possibly why. The U.K. governments (at least recently) have been mum on offering explanations when they differ with their science advisors (certainly where their drugs policy is concerned), and I don't expect the U.S. to be much different here. Had the Interior Department followed points 4 and 5, perhaps the problems they had with the moratorium recommendations could have been avoided.
As to what is meant by scientific integrity, I think that it's the second definition - soundness and of an unimpaired condition. The memo (and the subsequent recommendations) have focused mainly on processes, which to me says second definition. Have the proper processes been followed in the relevant experiments/interviews/research? Were there conflicts (and were they disclosed) - either on the part of the researchers or in the use and/release of information by agencies? Were proper procedures followed? Did the research conform to expected practices in its conduct, both in the lab and with relevant ethical bodies (IRBs, financial disclosure, registering controlled agents, etc.). If a scientific field has a developed code of conduct, great. But not all of them have (particularly if their work does not involve research on or with human subjects).
David- You raise a number of interesting points.
First, you said
"Broad documents like this do not typically have the detailed kinds of explanations that I think would satisfy Sharon. Those would typically be placed in a set of regulations, an OMB circular (assuming there are revisions to the testimony review process), or specific agency integrity guidelines."
My point was that the objective of the guidelines was not clear to me, what we are trying to improve or stop doing by issuing this guidance. I typed "scientific integrity definition" into Google and got examples of
"Designation for reports by the United States office of research integrity, identifying questionable research published in articles or books. Notification of the questionable data is carried in the nih guide for grants and contracts."
In contrast, let's use as an example the Executive Order on environmental justice.
"each Federal agency shall make achieving environmental justice part of its mission by identifying and addressing, as appropriate, disproportionately high and adverse human health or environmental effects of its programs, policies, and activities on minority populations and low-income populations in the United States and .."
Notice that it clearly states what needs to be "identified" and "addressed."
In some of my old jobs, I would likely be one of the ones asked to draft some agency guidelines- and I would be operating from a lack of clarity on the intention. Moving it down a level or two doesn't clarify, it causes unclear ideas to spread into an uncoordinated hodgepodge across agencies, at a much greater cost to the taxpayer.
Second, you said
"This isn't about politicals 'ignoring' the science, it's about making it harder for politicals to manipulate the science and the research they use in forming policy (and easier to detect the inevitable instances of same)."
I am concurrently reading Kellow's "Science and Public Policy" book for our virtual book club on this blog, which makes an interesting counterpoint to this discussion. What it makes clear is that neither scientists nor politicals can be trusted not to "manipulate" the "science" to promote their own policy or self-aggrandizement agendas. But I still don't see how anything in the memo directly helps that- except for transparency requirements.
Third, you said
"Point 2 is critical, and might address some of the confusion surrounding this whole thing. The target here is the science and scientific research used. That is what the Administration wants to make sure the public can trust."
But people don't trust research, for a host of good reasons that are more structural than personality-driven. Again, the Kellow book is illustrative of some of the issues. You have to make the science trustworthy AND open the doors of how it is used to public scrutiny. Otherwise you are requiring the American public is buying a variety of research "pigs in pokes" and then requiring that information to be used in policy. (cont'd)
Reply Part II
David, you said
"What Sharon writes comes across as an interest in limiting the research used to inform policy to only that research which is conducted to inform policy. I think that would restrict way too much information unnecessarily (for instance, I think this would eliminate nearly all National Academies studies). While directed research is great and useful, it can't always be conducted on the same timeline as the policy questions being answered. Additionally, there's an implication that only research conducted specifically to inform policy is capable of doing so."
My point was simply that if research is initiated based on other framings than those accepted by policymakers, the utility of that research is not best decided by scientists, the agenda- driven ones of which have an obvious and triple conflict of interest (I am important, my research is important to fund, my research is important to decisions in the real world). I am not a science policy scholar, but I do know of the process of joint fact-finding for environmental conflict resolution- where that kind of discussion takes place openly among stakeholders, scientists, and people who are experts (such as practitioners). Application of a particular piece of science to policy is rightfully be a shared responsibility of policymakers (including stakeholders), a variety of disciplines of scientists, and other experts, open to public discussion and comment.
As a fan of transparency, in the past, I have found NAS studies to be too secretive for my taste. Perhaps this has improved?
I agree with what you said about scientific wholeness following scientific rules and perhaps then, scientific integrity should be about making sure people are following the scientific rules. But again, reading the Kellow book, it appears that peer review is necessary but not sufficient to ensure scientific quality. I'll discuss that more in my next post.
Sharon,
As to your first response:
I can't argue that the plain text of either the March 2009 memo or the recently released recommendations (are we still talking just about the former?) lacks a clear statement of goals aside from their conception of scientific integrity. I'm trying to surmise it from the context of the March memo and the associated incidents that got some scientists riled up to begin with.
The argument for pushing the specifics down to the agency level (well, one of them) would be to account for variations in processes, and kinds of science, utilized in various agencies. Do you think a more detailed policy can be generalized across the government? To what level of detail?
I'm not sure what else besides transparency could mitigate concerns over politicization of scientific findings. Arguably most of the March memo is about transparency - mainly on the science and scientists being utilized rather than the policy decision-making processes. There's an assumption that if conventional scientific procedures are followed that the resulting research would be scientifically sound. That seems to be the part you have the biggest concern about. This memo (and the recommendations) are probably not where that fight should happen. I think that fight would have more success closer to the ground, as it will be individual researchers and reviewers that will need to be convinced.
To your second response:
It's not stated in this memo, but it seems to me implied that scientists are not the final arbiters of the utility of research to policy. The concerns of scientists on this issue seems to me more focused on misrepresentation of scientific findings than on getting a chance to decide policy. Much research utilized to inform policy will have been conducted with no knowledge of how to inform policy or intent to inform policy. Can we realistically expect the kind of peer review (essentially a second round of it) and QA/QC that you would like to see, as it would have to be done retroactively?
My claim that the Academies studies would not be used comes not from a transparency concern (though that could be argued), but from the fact that the vast majority of those studies are not original, peer-reviewed research (though they will rely on it in their work). A fair number of scientifically-inclined advisory bodies may fall into the same basket (PCAST among them).
Post a Comment