After writing the post “Economics of peer review at NSERC”, I decided to email the author and direct his attention to the blog discussions on his paper. I was thrilled by Gordon’s rapid response and provision of the paper in question:
Gordon, R., & Poulin, B. (2009). Cost of the NSERC Science Grant Peer Review System Exceeds the Cost of Giving Every Qualified Researcher a Baseline Grant Accountability in Research, 16 (1), 13-40 DOI: 10.1080/08989620802689821
Gordon has also become active in the blog discussions of the paper in: A blog around the clock, Genomicron, Pyrenaemta, and Sandwalk. There is also more discussion on Bora Zivkovic’s (A blog around the clock) newest post on the topic and friend feeds, here and here, although Gordon hasn’t been present yet. One of the most interesting things about this discussion (to date) is the observation that most, if not all, of the posters and commentators didn’t read the paper prior to publishing their opinions. (I acknowledged in my post that I was commenting solely on the abstract, while many others did not.) I am not really planning to talk about the web 2.0 and social media implications of this further, but I think that it is a very interesting point to note.
One of the commentators on Bora Zivkovic’s original blog on the topic, his brother Marko in fact, wrote:
“ . . . I want to address something I have seen in the comments here (as well as at Lawrence Moran’s Sandwalk blog) that I think is highly pernicious. I find it really sad that scientists themselves would interiorize the highly insulting view that were we not forced to write grants we would just slip into crappy, useless research. This is precisely the attitude implied by the granting agencies: if we don’t slap ridiculous amount of self-justification and scrutiny onto you, you will take the money and run to the Bahamas. I think it is highly damaging to science to consider its practitioners as extrinsically motivated (some sort of Skinner’s pigeons who perform only when given reinforcement).
I would say that science types are by and large a self-selected bunch who obviously are not motivated by mere monetary rewards (if they were they would have turned their supposedly powerful brains to more lucrative endeavors such as business, law, etc.). Second, they pass through a long and arduous apprenticeship that culminates in a PhD. By then they should have soaked up the scientific ethos thoroughly. In addition, throughout our career trajectories we are scrutinized most carefully — when we get the job, when we are promoted, when we publish, etc. The article points out, and I think quite rightly, that scientists are judged more rigorously at all these points and that the one area where they should be trusted is precisely when they are embarking on new research.
. . . There will be plenty of “quality control” later on in the process (publishing, etc.). We as scholars/researchers/scientists are intrinsically motivated to search for truth, or the best approximation to it, with all we have and in the best way we could conceive of. We are the types who would do it even if we had just a pen and a piece of paper. We are like Picasso who supposedly said that if he were put in a solitary cell with nothing to draw with he would paint by spitting on the walls.”
I suppose that I am guilty of propagating this viewpoint, however, I think that it is dangerous to generalize about any population. Yes, the vast majority of science types (or academic types, for that matter) are highly motivated and curiosity-driven – I have met and worked with many shining examples of these individuals. Unfortunately, I have also crossed paths with some less savory types and had to clean up their messes. While they are the minority, the damage they wreak is not inconsequential and it has the unfortunate side effect of catching and staying in the public psyche – à la Scott S. Reuben (for example).
The reality is that incidents of research and budgetary fraud have created an environment where administrative controls are more prevalent. The explosion of administrivia is a direct consequence of additional reporting requirements and strict enforcement rules required by funding agencies. I don’t believe that there is a precedent, but many institutions live in fear of losing all agency money because of the whims of one shyster.
Attend a CAURA (Canadian Association of University Research Administrators) meeting and you’ll hear some shocking stories of incidents stretching the gamut of research fraud, grad student mistreatment, creative budgeting, ethics, and safety issues. The controls won’t stop the malicious fraudsters, but the burdens they place on the academic community as a whole are there for a reason. Sorry, trust isn’t always possible.
How does this relate back to the paper? Gordon and Poulin suggest that NSERC set aside a proportion of their funding portfolio (i.e. the Discovery grant budget) to provide baseline grants to all researchers. They do not advocate abolishing peer review altogether, but indicate that it would be more cost effective and stimulate more innovative research if each qualified applicant was given a seed grant to spend at their discretion (~$30,000).
There are many aspects of the paper that I agree with and some that I disagree with – it is definitely a thought provoking read. While I can see the benefit of such a plan, I am having a tough time thinking around the complexities of implementation.
What complexities? Well, obviously if you wiped out a significant proportion of the peer review process at NSERC, there would be cost efficiencies (i.e. smaller travel budgets for reviewers) and you certainly wouldn’t need as many program officers (i.e. a leaner NSERC); then those monies could be redirected to fund more research. The challenge comes in at the institutional level – how to administer the funds. Most institutions use grant applications to determine the level of risk that the research plan entails and therefore ensure that appropriate safety checks are put in place. Canadian institutions are more relaxed about budgetary adherence (especially for NSERC grants) versus their US counterparts. I suppose that a one or two page research and budget plan would address this adequately. I suspect that there would be a trickle down of administrative controls and an offloading of that cost line from the federal budget (of course, this might be offset by an increase in indirect costs funding).
Also, if everyone who asked for money got money, how do you handle stagnant researchers? Who decides to take the money away? Researchers have to voluntarily decide that they no longer want (!) to do research and forgo baseline funding? I think that this would create a huge swell of research emeriti and thereby increase the teaching burdens of new faculty by reducing the number of tenure-track teaching positions available. I think that this alone would negate the “savings” in research efficiency by avoiding the grant writing step for this funding.
It also strikes me that this would be a circular model: competitive hiring of a tenure-track faculty member, receipt of a baseline grant, publish, teach, service, promotion, baseline continues . . . et al. etc. . . . tenure achieved, baseline continues. Does baseline ever change? What happens if promotion stalls – does the baseline stop? What happens if this model is applied and found ineffective – how would you ever repeal funding for everyone – I can already feel the groundswell of backlash.
I’m not saying it couldn’t work, but it would require a big paradigm shift not only for researchers, but also for the upper echelon and worker bee levels of administrators to make it happen.
