NSF CAREER proposal--shredded!

I found out today that my NSF CAREER award proposal was declined. This was completely expected, and as I mentioned in my previous blog I have been looking forward to the constructive feedback that I know I need. Whether consciously or not, though, I now realize I was unrealistically hoping that I would receive mostly positive feedback and a few easy things to improve upon. Thus, my first read-through this afternoon of the detailed reviews was very surprising and wickedly demoralizing. Let me try to think of some other emotions: bewilderment, depression, frustration... OK that's enough. I think the most demoralizing aspect was that a response shared by many reviewers was that the proposal was poorly written and confusing. This must be true, given that the sentiment was shared, but as of now, I have no idea how to overhaul the proposal to correct for this, and I'm dreading having to figure that out.

At this point (5 hours later), I have carefully re-read all of the feedback, and my emotions are much more tempered, and a couple new ones have emerged: hope, gratitude. The gratitude is for the many scientists (5 reviewers, plus program manager, plus panel members) who clearly invested a huge amount of time in reading (and re-reading and deciphering my bad writing) my proposal and composing very detailed and thoughtful feedback. I truly feel badly and frustrated that I can't thank these reviewers personally, due to their anonymity. Really, the only thing I can do is to do my best to follow all of the advice they have given and correct all my mistakes with the resubmission this summer...and I plan on doing this.

Following advice of Jean-Claude Bradley, Cameron Neylon, and others, I have posted most of the potentially interesting and non-confidential parts of the proposal on Scribd. I am new to the site, but I think this is a link to the proposal. WARNING: reading the proposal has been deemed difficult and perhaps a waste of time by at least 5 respected scientists!

I am not going to post the reviewers comments, since I think they had the expectations that I would not share them. I will give a brief summary here.

Overall

• Two panels reviewed the proposal. Both panels ranked it low priority (approximately 50th percentile or worse). Ouch! One panel was the Molecular Genetics panel and the other was a special interdisciplinary panel between Mathematical and Physical Sciences and Molecular and Cellular Biosciences for interdisciplinary proposals such as this.
  
• I have feedback from 5 anonymous reviewers. I don't know how they fit into the two panels, or whether it's a 3rd source of feedback for the program manager. 4 reviewers understood the impact (in my opinion), whereas one reviewer seemed to hate my writing and didn't seem to understand because of that.
  
• All panels and individual reviewers had significant concerns about the proposal. There was no way this thing could have been funded without some kind of empty senate seat under my control.
• Involvement with OpenWetWare and open science was noticed and cited as a big positive for the broader impacts.
  

Shared Major Negative eedback

• Dearth of preliminary data / overly ambitious / seriously concerned about feasibility of any aspect of the proposal. This is very reasonable and somewhat expected. The students have been working hard on this project, we already have some significant preliminary data (related to "Shotgun DNA Mapping") and I expect a lot more before July of this year. I had sent in an update about the shotgun DNA mapping results in October, but based on reviewer feedback, I think that the program manager did not send this update to anyone. Another way I will address this issue is to re-organize the proposal (somehow) to make the feasibility more obvious.
• Poorly written and confusing. This sentiment was pretty much shared by all reviewers and even included in one panel summary. I have re-read the proposal many times since July and never really considered that it was so confusing. I was modeling the format after a couple funded CAREER awards that I had seen (in other fields, though). Well, clearly I need to improve this somehow and I don't have any good ideas. I feel bad that I caused so much trouble and wasted so much time of the thoughtful reviewers. One of the reviewers basically admitted to giving up, which I don't really blame him, given the comments from the others. I feel like I've been told that my baby is ugly and just can't see it. BTW, my kids are fantastically cute.
• Too many specific aims / third aim not well developed. I can see that this is true. One thing I stupidly didn't realize is that the reviewers basically expect NIH-style things such as "specific aims." I had actually suspected it would be viewed negatively if made it look too "NIH-ey." I will think a lot more about this, but right now what I'm thinking is that I should focus the science solely on Pol II transcription (ditch the DNA damage repair), and then reorganize the aims to be (1) shotgun DNA mapping (2) unzipping in vitro Pol II complexes and (3) mapping pol II and nucleosomes on native chromatin. This would be organized from most feasible to least feasible.
  

Shared Positive Feedback

• Most of the reviewers specifically mentioned that involvement with OpenWetWare was a big positive for the broader impacts. This was really good to see, and perhaps the most notable thing to people who happen to read this blog.
• All reviewers agreed that the impact would be very high (if successful). Further, I think some were even intrigued / excited about the idea. Many of the reviewers understood and seemed to agree with a main point I was trying to sell: that the combination of single-molecule analysis with yeast genetics is a potentially powerful and unique tool for chromatin biology.

I suppose this positive feedback is actually sufficient, if I can successfully correct the very serious flaws. After all, trying to look at things optimistically, I'd be in a much worse situation if the reviewers had thought the proposal were very well written and feasible, but lacking potential impact.

Overall, writing this blog entry has been therapeutic and also constructive. And if you do happen to skim any of the proposal, I welcome harsh criticism--I can stomach it!

SJK Note added Feb. 5, 2010:  I did end up submitting again in 2009.  The quick report is that I rewrote it in NIH style, and improved writing thruoghout.  The reviewers liked the science, but lacked confidence in me as a PI, due to lack of preliminary results.  Fair criticism that I think I can correct this year.  Most importantly: the panel LOVED open science as broader impacts.  Here're some links for my 2009 NSF CAREER proposal:

• FriendFeed thread discussing broader impacts feedback.
• Slide 6 of this presentation shows some of the positive reviews of open science from panel.
• Link to 2009 proposal on Scribd.