DG2010 – evaluation changes
In the past, successful returning applicants were somewhat assured of a fairly steady level of funding. NSERC rules allowed for up to a 50% reduction in award for returning applicants, but this has rarely been applied. Generally, the expectation is that if you are successful, you will be funded at either the same or a slightly higher level than your previous Discovery award. Times sure have changed.
The DG2008 pooled success rate was 71%. The DG2009 success rate declined to 63.5% and more telling, the variation in award level scatter plot actually shows scatter. (See this post for a link to the statistical analysis.) DG2009 marked the first step of a phased change in the evaluation process for Discovery grants. (See this post for more discussion on these changes and links to official FAQs.) To briefly recap, the Grant Selection Committees (GSCs) are no more. They have been replaced with Evaluation Groups (EGs). The EG implementation is new for this competition, however, DG2010 will be the second year of implementing the funding level ‘binning’ system (a method for scoring applications based on their budget request, as either high, low or normal.)
This year will also be the first year for the separation of the evaluation and funding level decision steps. (Last year the GSCs implemented binning and made the funding decisions – this year the EGs get to scorecard and a separate group sets the funding level.) Theoretically, this new system will reduce cronyism by funding applications based purely on relative score, however, I suspect that the phase-in of this process will be a slow and somewhat painful experience for applicants used to the current system.
While the creation of EGs seems like a big deal, it will probably be the easiest change to swallow as it will allow for more flexible review of applications. The more difficult element for applicants will be building a budget that does not damage their chances for application success by scoring too high or too low on the evaluation indicator scorecard. The next few posts will be dedicated to providing suggestions on how to build a normal (?) budget.
[...] DG2010 – Evaluation Changes reviewed some of the changes to the evaluation process for the Discovery grant competition. I have had a few comments on this post via email regarding a misunderstanding of how the binning process will be applied. I am working on clarifying my understanding so that I can post a refined view of how this will work (or did work last year). [...]