Saturday, December 27, 2014

Three observations about anonymity in peer review


Friday, December 26, 2014

Posting comments on papers

For many years, people have wondered why most online forums for comments result in hundreds of comments, but even the most exciting scientific results lead to the sound of crickets chirping. Lots of theories as to why, like fear of scientific reprisal or fear of saying something stupid or lack of anonymity.

Perhaps. But I wonder if part of it is just that it feels… incongruous to post comments on scientific papers. To date, I have posted exactly two comments on papers. My first owed its genesis (I think) to the fact that I had just read something about how nobody comments on papers, and so I was determined to post a comment on something. And it was a nice paper on something I found interesting and so I wanted to say something. I just now wrote my second comment. It was on this AWESOME paper (hat tip to Sri Kosuri) comparing efficiency of document preparation using Word vs. LaTeX (verdict: LaTeX loses, little surprise to me). Definitely something I found interesting, and so I somehow felt the urge to comment.

And then, as I started writing my comment, something just felt… wrong. Firstly, the process was annoying. I had to log in to my PLOS account, which I of course forgot all the details of. Then, as I was leaving my comment, I noticed a radio button at the bottom to say whether I had a competing interest. The whole process was starting to feel a whole lot more official than I had anticipated. Suddenly, the relatively breezy and light-hearted nature of my comment felt very out of place. It’s just very hard to escape the feeling that any commentary on a scientific paper must be couched in the stultifying language and framework of the typical peer review, which is just so different than the far more informal commentary than you get on, for instance, blog posts. And heaven forbid if you actually posted a joke or something like that.

I feel like part of the reason nobody comments is that publishing a paper seems like a Very Serious Business™, and so any writing or commentary associated with it seems like it should be just as serious. Well, I agree that publishing a paper is a very tedious business, but I think making scientific discourse a bit more lighthearted would be a good thing overall. And who knows, one side-effect could be that maybe someone might actually read the paper for a change!

Tuesday, December 23, 2014

Fortune cookies and peer review

Ever play that game where you take the fortune from a fortune cookie and then add “in bed” to the end of it for a funny reinterpretation? I’ve found it works pretty well if you just replace “in bed” with “in peer review”. Behold (from some recent fortune cookies I got):

Look for the dream that keeps coming back. It is your destiny in peer review.

Wiseness makes for oneself an island which no flood can overwhelm in peer review.

Ignorance never settles a question in peer review.

In the near future, you will discover how fortunate you are in peer review.

Every adversity carries with it the seed of an equal or greater benefit in peer review.

You will find luck when you go home in peer review.

Also reminds me of the weirdest fortune I ever got: “Alas! The onion you are eating is someone else’s water lily.” Not sure exactly what that means, in peer review or otherwise…

Saturday, December 20, 2014

Time-saving tip–make a FAQ for almost anything

One of the fundamental tenets of programming is DRY: Don’t Repeat Yourself. If you find yourself writing the same thing multiple times, you’re creating a problem in that you have to maintain consistency if you ever make a change, and you’ve had to write it twice.

In thinking about what I have to do in my daily life, a lot of it also involves repetitive tasks. The most onerous of these are requests for information that require somewhat length e-mails or what have you. Yet, many times, I end up answering the same questions over and over. Which brings up a solution: refer to a publicly available FAQ.

I first did this for RNA FISH because I was always getting similar questions about protocols and equipment, etc. So I made this website, which I think has been useful both for researchers looking for answers and for me in terms of saving me time writing out these answer for every person I meet.

I also recently saw a nice FAQ recently (can’t find the link, darn!) where someone had put together a letter of recommendation FAQ. As in, if you want a letter of recommendation from this person, here’s a list of details to provide and a list of criteria to determine whether they would be able to write a good one for you.

Another senior professor I met recently said that she got sick of getting papers from her trainees that were filled with various errors. So she set up a list of criteria and told everyone that she wouldn’t look at anything that didn’t pass that bar. Strikingly, she said that the trainees actually loved it–it made a nice checklist for them and they knew exactly what was expected of them.

I think all of these are great, and I think I might make up such documents myself. I’m also thinking of instituting an internal FAQ for our data management in the lab. Any other ideas?

Sunday, December 14, 2014


I found this article by Solomon Snyder to be informative:







Saturday, December 13, 2014

The Shockley model of academic performance

I just came across a very interesting post from Brian McGill about William Shockley’s model for why graduate student performance varies so much. Basically, the point is that being successful (in this case, publishing papers) requires clearing several distinct hurdles, and thus requires the following skills:
  1. ability to think of a good problem
  2. ability to work on it
  3. ability to recognize a worthwhile result
  4. ability to make a decision as to when to stop and write up the results
  5. ability to write adequately
  6. ability to profit constructively from criticism
  7. determination to submit the paper to a journal
  8. persistence in making changes (if necessary as a result of journal action).
Now, as Brian points out, if you were 50% better at all of these (not way beyond the norm, but just a little bit better), then your probability of succeeding in your assigned task (which is the product of the individual probabilities) is roughly 25 times better. This is huge! And it’s also to me a reason for great hope. The reason is that if, alternatively, being 25 times better required being 25 times better at any one particular thing, then it seems to me that it would require at least some degree of unusually strong innate ability in that one area. Like, if it was all about writing fast, then someone who was a supernaturally fast writer would just dominate and there’s nothing you could really do to improve yourself to that extent. But 50%? I feel like I could get 50% better at a lot of things! And so can you. Here are some thoughts I had about creativity, writing with speed, execution and rejection, and there are tons of other ways to get better at these things. Note that by this model, by far the most important quality in a person is the ability to reflect on their strengths and weaknesses and improve themselves in all of these categories.

I think this multiplicative model becomes even more interesting when you talk about working together with people in a lab. One point is that establishing a lab culture in which everyone pushes each other in all regards is critical and will have huge payoffs. Practically, this means having everyone buy in to what we collectively think of as a worthwhile idea, how we approach execution, how to write, what our standards of rigor are, and sharing stories of rejection and subsequent success through perseverance. This also provides some basis for the disastrous negative consequences of having a toxic person in lab: even if the effects on each other person in the lab in all or even some of these qualities are small, in aggregate, it can have a huge effect.

The other point is delegation strategy. It’s clear that in this model, one must avoid bottlenecks at all costs. This means that if you are unable to do something for reasons of time or otherwise and the person you are working with is also unable to do that task, things are going to get ugly. The most obvious case is that most PIs have only a limited capacity (if any) to actually work on a project. So if a trainee is unable to work on the project, nothing will happen. Next most obvious case is inability to write. If the trainee is unable to write and you as a PI have no time or desire to write, papers will not get written, period. Deciding how much time to invest in developing a trainee’s skills to shore up particular weaknesses is a related but somewhat different matter, and one that I think depends on the context.

This model also maybe provides some basis for the importance of “grit” or resilience or motor or drive or whatever it is you want to call it. These underlie those items on the list that are the hardest to change through mentorship. If someone just doesn’t have an ability to work on a project, then there’s not a whole lot you can do about it. If someone does not have the determination to do all the little things required to finish a project or to stick to it in the face of rejection, it will be hard to make progress, and there’s not much that you can do to alleviate these deficiencies as a mentor. I think many PIs have made this realization, and I have often gotten the advice that the most important thing they look for in a person is enthusiasm and drive. I would add to this being open to reflection and self-improvement. Everything else is just gravy.

Sunday, November 23, 2014

The most annoying words in scientific discourse






Now exactly what do these images mean? Speaking of which…


Paradigm. Need I say more?





[Updated to include Paradigm, forgot that one.]

Saturday, November 22, 2014









Sunday, November 16, 2014

A week in my e-mail life





Here's what I found:


Zooming in on the short time-scale:


  1. Spam
    1. Personal. e.g. table of contents.
    2. TOC. Tables of contents of various journals.
    3. Science. ResearchGate, Nature Publishing Group
  4. Teaching. Students asking for help or whatever.



Message 1:
Вы руководитель от Вас внезапно ушел бухгалтер!
Вас предали? Вы подставлены? Завтра налоговая?


Message 2:

Subject: Лучший Новогодний подарок - безопасность ваша и ваших близких!

Message 3:
Subject: Your  Account Was Banned
This is a joke :)

Than trying to work mounted on clumsy, long webfeet by the
ecriture artiste which the french writers that hears. Similarly,
employing the eye, it is a moment without devoting his heart
upon mahadeva. Towards the abode of bhishma, casting aside


Message 4:
Subject: Mandy - 100% results.







Sunday, November 9, 2014

My favorite quote about LaTeX




Saturday, November 8, 2014

“Edge of Tomorrow” and the case for better education




Friday, November 7, 2014

My water heater is 100% efficient (in the winter)




Thursday, November 6, 2014

Why are papers important for getting faculty positions?







Monday, November 3, 2014

Why don’t bioinformaticians learn how to run gels?









Friday, October 24, 2014

The eleven stages of academic grief


  1. Revise
  2. Resubmit
  3. Rejected
  4. Revise
  5. Resubmit
  6. Rejected
  7. Revise
  8. Resubmit
  9. Rejected

Thursday, October 16, 2014

What makes a scientist creative?






Sunday, October 12, 2014

Disabled Google Plus comments

Hi there readers,



Saturday, October 11, 2014

What have I learned since being a PI?




















Friday, October 3, 2014

A proposal for controlling the amount of paperwork





Sunday, September 28, 2014

Sigma's getting with it on Twitter!


Here's our lab's Sigma Selfie, starring cholesterol and calcium chloride:

Sunday, September 14, 2014











Saturday, September 13, 2014

Greatest molecular biologist of all time?






Sunday, August 31, 2014

My new favorite website

I had a couple posts earlier about the Fermi paradox inspired largely by a post on the website It's now officially my favorite website. Check out some of these other great posts, many characterized by mind-blowing depictions of scale:

Plus other fun stuff about baby names, trips to Japan and Russia, etc. Anyway, great site!

Saturday, August 30, 2014

Do I want molecular biology to turn into physics?

I just read this little Nature News feature about the detection of low-energy neutrinos produced during the fusion reactions that power the Sun. I don't know much nuclear physics, but the article says that fusion in the sun involves the fusion of two hydrogen molecules into deuterium, and the conversion of one proton into a neutron leads to the production of low energy neutrinos, but nobody has been able to detect them because their low energy leads to their signal getting swamped out.

The experiment itself sounds incredible:
The core of the Borexino experiment features a nylon vessel containing 278 tonnes of an ultrapure benzene-like liquid that emits flashes of light when electrons are scattered by neutrinos. The liquid was derived from a crude-oil source nearly devoid of radioactive carbon-14, which can hide the neutrino signal. The detector fluid is surrounded by 889 tonnes of non-scintillating liquid that shields the vessel from spurious radiation emitted by the experiment's 2,212 light detectors.
That is some BIG science! Clearly an experimental triumph.

But it's also a triumph of theory. Theory in physics is so amazing. Can't tell you how many articles about physics have some version of the following line:
While the detection validates well-established stellar fusion theory, future, more sensitive versions of the experiment could look for deviations from the theory that would reveal new physics.
Not only is theory qualitatively right in physics, it's usually provides quantitative predictions with an accuracy that seem utterly preposterous from the perspective of a molecular biologist:
Borexino can measure the flux of low-energy neutrinos with a precision of 10%. Future experiments could bring that down to 1%, providing a demanding test of theoretical predictions and thus potentially uncovering new physics.
For fellow quantitative biologists, it is beyond our wildest fantasies to have strongly predictive theories and models like these. Usually, we're pretty happy if we can get the sign of the prediction correct! That said, there is also something troubling in these articles, which are all the little hints like "more sensitive... could... reveal new physics". I think the operative word is "could". Mostly, it feels like our understanding of many fundamental physical processes is so deep and so accurate that there aren't many surprises left out there (except for that whole dark matter/energy thing...).

I think I'm overall really happy that biology has surprises coming out all the time that make us reconsider our basic understanding of how things work. I do think that molecular biology does suffer a bit from "let's find the latest molecule that breaks the dogma" syndrome instead of focusing more effort on systematizing what we do know, but for the most part, I love the energy that comes from the huge amount of unknowns in the field and the huge challenge that comes with the effort to make molecular biology as predictive physics. Of course, there's no reason to believe a priori that such predictivity is possible. But that's what makes it fun!

Saturday, August 23, 2014

Is academia really broken? Or just really hard?

(Second contrarian post in a row. Need to do some more positive thinking!)

Scarcely a day goes by when I don’t read something somewhere on the internet about how academia is broken. Usually, this centers around peer review of papers, getting an academic job, getting grants and so forth. God knows I’ve contributed a fair amount of similar internet-fodder myself. And just for the record, I absolutely do think that many of the systems that we have in place are deeply flawed and could do with a complete overhaul.

But what do all these hot-button meta-science topics have in common? Why do they engender such visceral reactions? I think they are all about the same basic underlying issue, namely competition for limited resources (spots in high impact journals, academic jobs, grant funding). I think we can and should fix the processes by which these resources are apportioned. But there’s also no getting around the fact that there are limited resources, and as such, there will be a large number of people dissatisfied with the results no matter what system we choose to use.

Take peer review of papers. Colossal waste of time, I agree. Personally, the best system I can envision is one where everyone publishes their work in PLOS ONE or equivalent with non-anonymous review (or, probably better, no review), then “editors” just trawl through that and publish their own “best of” lists. I’m sure you have a favorite vision for publishing, too, and I’m guessing it doesn’t look much like the current system–and I applaud people working to change this system. In the end, though, I anticipate that even if my system was adopted, everyone (including me) would still be complaining about how so and so hot content aggregator is not paying attention to their own particular groundbreaking results they put up on bioRxiv. The bottom line is that we are all competing for the limited attentions of our fellow scientists, and everyone thinks their own work is more important than it probably is, and they will inevitably be bummed when their work is not recognized for being the unique and beautiful snowflake that they are so sure it is. Groundbreaking, visionary papers will still typically be under-recognized at the time precisely because they are breaking new ground. Most papers will still be ignored. Fashionable and trendy papers will still be popular for the same reason that fashionable clothes are–because, umm, that’s the definition of fashion. Politics will still play a role in what people pay attention to. We can do pre-publication review, post-publication review, no review, more review, alt-metrics, old-metrics, whatever: these underlying truths will remain. It’s worth noting that the same sorts of issues are present even in fields with strong traditions of using pre-print servers and far less fetishization of publishing in The Glossies. I think it's the fear and heartbreak associated with rejection by one's peers (either by reviewers or by potential readers) that is the primary underlying motivation for people to consider alternative approaches to publishing–it certainly is for me. We should definitely consider and implement alternatives, but I think it's worth considering that the anguish that comes from nobody appearing to appreciate your work will always be present because other people's attention is a limited and precious resource that we are all fighting for. [Update 8/25: same points made here and here by Jeremy Fox]

For trainees, the other “great filter event” they probably experience is getting a faculty position. Yes, the system is probably somewhat broken (in particular with gender/racial disparities that we simply must address), although compared to peer review of papers, search committees are far more deliberate in their decision making, precisely because the stakes are so much higher. Yes, we can and should encourage and support students considering other career paths. I guess what I’m saying is that even if everyone went into science with their eyes wide open, with all the best mentoring in the world, the reality is that there are more dreamers than dream jobs available. That means many people who feel like they deserve such a position (and certainly many of them do) are not going to get one. And they probably won’t be happy about it.

(Sidebar about career path stuff: to be frank, most of the trainees I’ve met are pretty realistic about their chances of getting a faculty position and have many other plans they are considering as well, and so I think some of the “I’m not getting support and advice about other career choices” meme is overblown, especially these days. We can blame the “system” for somehow making it seem like doing something other than academics is a failure, and there is definitely some truth to that. At the same time, I think it’s fair to say that many people do a PhD because being a scientist was a long-held dream from childhood, and so if we’re being totally honest, at least some of the sense of failure comes from within. It’s a lot easier to say abstractly that we should be realistic with trainees and manage expectations and so forth than to actually look someone in the eye and tell them to their face that they should give up on their dream. I agree that this is the sort of hard stuff PIs should do as part of their jobs–I’m just saying it’s not as easy as it is sometimes made out to be. And yes, I’ve personally experienced both sides of this particular coin.)

Look, nobody likes this stuff. Rejecting is about as much fun as being rejected, and I FULLY support all efforts to make our scientific processes better in every possible way. All I’m saying is that even the best, most utopian system we can think of will suffer from inequities, politics, fashions, etc. because that is just human nature. The current systems are currently largely run by scientists, after all, and so we really have nobody to blame but ourselves. I realize it’s much easier to blame Spineless Editor From Fancy Journal, Nasty Reviewer with a Bone to Pick, Crusty Old Guy on the Hiring Committee, or Crazy Grant Reviewer with a Serious Mental Health Issue, and I’ve for sure blamed all those people myself when I have failed at something. Maybe I was right, or maybe I was wrong. I’m pretty sure it’s mostly a rationalization that lets me keep my chin up in what can sometimes be a fairly demoralizing line of work. Science is a human endeavor. It will be as good and as bad as humans are. And when the chips are down and there’s not enough to go around, that can bring out both the best and the worst in us.