Negative results and dodgy papers: keep quiet or publish?

Negative results are very rarely published in the literature. After all, the literature is bursting with new positive results and we don’t have enough time to read all of these, let alone papers describing what doesn’t work. Negative results are dull—who would want to read anything in the Journal of Negative Results?

Up until recently I haven’t had a problem with the status quo. I’m afraid the following discussion is a bit vague because I’m (still) not sure about how much detail I can go into my work, but please bear with me.

I came across a paper published this year which describes the effect of doing something quite specific in a synthesis on nanoparticle shape. Do the thing, get a particular nanoparticle shape (usually quite challenging to obtain); stop doing the thing, you get another shape (easy to obtain). I was quite excited because if it worked it would get around a major barrier to my desired nanoparticles.

I repeated the reaction exactly as the paper described, but it didn’t work.

I repeated the reaction in a flow reactor as it would make it easy to intensify the “thing”. According to the paper, this should definitely give the desired nanoparticles because the morphology selectivity/yield is directly proportional to the intensity of the “thing”. But it still didn’t work.

I’ve now given up on the reaction and moved on to something else. But that my results will not be published means that someone else could also waste a lot of time and money—on equipment, reagents, electron microscopy—repeating the experiment.

What can I do? I think I have three options:

Option 1: Do nothing.

I’ve already made it clear that I don’t like this option. I’m fairly sure the paper is wrong. It bugs me that it exists without some kind of mark against it.

Option 2: Email the authors.

I’m not too keen on this either. I suspect that my email would be ignored. Plus, I would rather any discussion happened in the open, which brings me on to…

Option 3: Blog about it (and possibly email the authors telling them that I blogged about it).

I feel uneasy about this. Could it be perceived as confrontational? Would I get a reputation as a troublemaker? I feel like it is the proper, scientific and open thing to do, but in reality it is absolutely not the done thing. I suspect most researchers would go for option one and do nothing. I could be right and the paper is wrong, but I’d be very happy to be proven wrong and get the reaction working.

What you think? Keep quiet, email or blog? Any other suggestions are welcome.

9 thoughts on “Negative results and dodgy papers: keep quiet or publish?”

  1. There is a fourth option, which would be to send a comment to the journal in question. Depending on the journal this would force the authors to respond, and depending on the outcome of the discussion the comment and response may be published.

    It’s a difficult one though – in my field of optics, if I can’t replicate something I tend to thing it’s down to my lack of skill! Maybe with a more protocol based experiment this is less of an issue, but it’s difficult to prove a negative.

    Without knowing all the details it’s difficult to give specific advice, but I’d hope your supervisor can give you something definitive.

  2. Isn’t there a chance that emailing the authors might lead them to give you some extra information that would help you replicate the results? I don’t know your field, but for many areas, even if you think you’ve put everything completely clearly in a paper/report, there can still be assumptions that seem so obvious that you forget to include them.

    Of course, if you didn’t get any useful response, the other paths are still open.

  3. I’ve lost count of the times this has happened, although mainly it’s the case of a terrible yield that was reported as >90% in the literature. However, I’ve also seen the other side of the coin. We had a reaction that worked perfectly, then stopped for no reason. Turned out it only worked with ultrapure solvent, and the solvent from our still was stopping it dead. There was also a case where the authors “forgot” to mention that the reaction had to be done in the dark. That detail did come out after sending them a polite email, so maybe it’s worth a try. Ultimately, go with what your PI is happy with, they’re the ones who get the stick if you piss someone off!

  4. Thanks for all of your comments.

    I now think the best thing to do would be to send them an email with a brief summary of my results. Kate and Polly, hopefully you’re right and it is something trivial missing from the paper.

    “Ultimately, go with what your PI is happy with, they’re the ones who get the stick if you piss someone off!”

    Absolutely, it’s something I always bear in mind when talking about work online. Going to check that they’re happy with me emailing beforehand.

    I’ll post any developments!

  5. Something to consider (from an astrophysics blogger who’s been at this a while):

    If you choose option 3, what you’ll basically be doing is drawing public attention to a scientifically obscure and — if your assessment is correct — dubious result.

    Do you really want to be plucking what you perceive as dodgy science out of obscurity, just to go and tell people to forget about it and that it’s problematic? I have done this before, back in the early stages of my blog, and I look back on that time with regret.

    It’s another thing if it’s a highly popularized result or a very famous person making some noise with it, or if it’s getting lots of attention and would benefit from a public skewering. I would encourage you to think about the type of impact you’d like to have before you go ahead and blog about what you perceive is lazy or dodgy — but barely noticed — science.

  6. Just came across your blog.

    I’d definitely avoid option 3.

    The reaction is question was probably someone’s entire PhD’s worth; out of 100 attempts they probably got one flukey good yield and that’s all they needed to publish. It’s just what happens I guess. Colleagues of mine have emailed the corresponding author in the past and have gotten a response.

Comments are closed.