The American Association for the Advancement of Science (AAAS) has been a major player on the scientific scene since its founding in 1848[1].
For 165 years AAAS has been a force for (1) advances in knowledge and understanding that are the hallmark of science, and (2) science policy. Its journal Science has developed into one of the world’s most prestigious peer-reviewed publications. Science is often researchers’ first choice for reporting on work deemed to be truly ground-breaking and of real significance (as opposed to more pedestrian progress).
In recent years, AAAS has added other peer-reviewed publications as well as a number of on-line vehicles for passing along news of interest to the community. One of the latter is ScienceInsider, which reported late last week that Lamar Smith, the new chair of the House Committee on Science, Space and Technology, is proposing new criteria for NSF grantmaking. Here are some excerpts:
The new chairman of the House science committee has drafted a bill that, in effect, would replace peer review at the National Science Foundation (NSF) with a set of funding criteria chosen by Congress. For good measure, it would also set in motion a process to determine whether the same criteria should be adopted by every other federal science agency…
…ScienceInsider has obtained a copy of the legislation, labeled “Discussion Draft” and dated 18 April, which has begun to circulate among members of Congress and science lobbyists. In effect, the proposed bill would force NSF to adopt three criteria in judging every grant. Specifically, the draft would require the NSF director to post on NSF’s website, prior to any award, a declaration that certifies the research is:
1) “…in the interests of the United States to advance the national health, prosperity, or welfare, and to secure the national defense by promoting the progress of science;
2) “… the finest quality, is groundbreaking, and answers questions or solves problems that are of utmost importance to society at large; and
3) “…not duplicative of other research projects being funded by the Foundation or other Federal science agencies.”
NSF’s current guidelines ask reviewers to consider the “intellectual merit” of a proposed research project as well as its “broader impacts” on the scientific community and society.
Two weeks ago, Republicans on the science committee took to task both John Holdren, the president’s science advisor, and Cora Marrett, the acting NSF director, during hearings on President Obama’s proposed 2014 science budget . They read the titles of several grants, questioned the value of the research, and asked both administration officials to defend NSF’s decision to fund the work…
Normally, this is where I insert the line, “there’s more, but this gives the flavor.” But in this case, that’s not so. The flavor of the fuller ScienceInsider piece is rather more negative than these excerpts would suggest. There’s the hint that time-honored peer-review will be replaced by this process and there are quotes from players that give the events a partisan cast.
The fuller reporting is surely accurate. Much of Washington business is partisan and becoming more so. Science and engineering, once considered non-partisan and in the national interest, are being sucked into the vortex.
Nevertheless, scientists should be slow to be defensive… and not be quick to take offense. The first criterion above is pretty much verbatim in the language establishing the National Science Foundation, an independent federal agency created by Congress in 1950 “to promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense…” The second two criteria make sense on the merits. There seems little to argue here. Who can be against high-quality, relevant science, or needless duplication? History also encourages us not to be overwrought. Scientists were initially skeptical when the GPRA requirements came in, but since then they seem to have been internalized into accepted ways of doing business…and by some lights the added requirements have been beneficial.
If there is reason for caution, it should come based on other grounds. Here are two; you may be able to add others.
To begin, every evidence suggests that if our Nation wants to be the indispensable nation for the 21st century despite our small population (4% of the global total), we must continue to lead the world in innovation. But we continue to require more and more administrative work from scientists. Increasingly our best and brightest, especially our early career scientists, are spending larger fractions of their time on proposal writing (as success rates decline, the process lengthens, and competition heats up) and the red tape associated with grants management. While valuable and necessary, this work comes at a cost; it reduces the amount of time and energy available for conduct of the research itself. Our Nation and our funding agencies should be innovative in reducing the number of hurdles and levels of review needed to ensure high quality research, not adding to such barriers.
Second, the distinction between (1) unnecessary duplication and (2) the redundancy needed to spur competition, verify results, speed the uptake of new advances, and protect against single-point failures is both a fine one and inherently subjective. During the Manhattan Project, for example, three competitive approaches to obtaining enriched uranium were run simultaneously, full bore. All three contributed. Given that the Axis powers were racing to develop the atomic bomb, should we have delayed commitment to any one of the three until after we’d run pilot projects to determine a single winner? In mapping the human genome, the country funded both a government brute-force technique and a private-sector approach focusing on preliminary research to reduce the cost of sequencing. Both contributed to U.S. leadership in bio-technology today. In a world where first-to-market conveys such powerful, long-term advantage, it may be dangerous to force another layer of approval (and accompanying delay) on grant-funding. Much of the key to past U.S. research success and current U.S. technological dominance lies in our willingness to invest in multiple approaches to scientific challenges.
[1] Readers of this blog might be interested in a little of the history, because of its origins in the Earth sciences. The AAAS grew out of the Association of American Geologists and Naturalists, founded in 1840, which originally consisted of ten geologists associated with the state geological surveys of the time and a handful of others. As early as 1851, the AAAS already had hundreds of members. Alexander Dallas Bache, a descendant of Benjamin Franklin and Superintendent of the U.S. Coastal Survey (a NOAA progenitor) was the president.
Just a quick follow-up comment on this post. I meant to add, but didn’t… that it appears the present NSF approach to peer-review of proposals will continue pretty much as it has for many years… that what is being proposed here are additional steps. I meant to add further that given the competitive nature of science, and the inherent nature of peer review, that these additional steps…Congressman Smith’s conditions that the research be of high quality, and not be redundant…are themselves redundant. They’ll add very little. That’s because peer reviewers are quickly dismissive of low-quality proposals and proposals that do no more than duplicate work already underway. Certification by the NSF director will in effect accomplish little more than affirm this.
Bill:-
I agree with your analysis, but it begs two larger questions:
• What is the proper role of the legislative branch in science?
• How should we decide what to fund?
Historically, Congress has played three roles in science – overseeing the spending by the executive branch, directing funding to areas or institutions Congress felt were being short-changed by existing funding mechanisms, and constraining how the executive branch disbursed funds for science.
In this light, questioning the funding for specific projects seems a legitimate exercise for a Congressional committee. Representative Smith’s draft bill fits that third role that Congress plays. However, history provides way too much evidence that when Congress meddles in the Executive Branch’s business like this it turns out badly (e.g., some of the worst abuses that led to the housing bubble were caused by Congressional direction to Fannie Mae and Freddy Mac). How can we expect people who’ve never run anything to provide worthwhile guidance about how a program should be run?
However, it’s the second question I posed, and Congress’s second role, that I find most interesting. By and large, Congress has used earmarks to fulfill this role. The academic mantra is that earmarks are “bad” because they flout peer review. Yet these same academics loudly proclaim the need for – and value of – diversity in the broader society. Earmarks are the only proven way to achieve the sort of diversity in research that academics claim to desire in the broader society. How many minority students, how many students in institutions of higher education in poorer states, were able to have a research experience only because their Congressional delegation directed funding to their schools?
Conversely, how much of the peer-reviewed research funding goes to the same researchers (and institutions) year after year? Is this because their work is consistently innovative and world class, or is it because they have become part of the “club?” While the investigations of Climategate uncovered no illegal behavior, they did reveal serious attempts to use the peer review process to prevent publication (and thus reduce the chances of funding) of work that contradicts the established orthodoxy.
I’m not trying to argue for or against peer review of proposals. I am trying to suggest that peer review alone may not be the healthiest model, or the one that best serves the interests of our country. Whatever anyone’s opinions, we all should periodically question how and what we’re funding.
Thanks, John,
I especially appreciate your point with respetc to how Congress can weigh in on science priorities; they really have few alternative avenues available. And you’re right that peer review is by no means a panacea.