Mistakes in Science

While I am happy about the coverage the OPERA faster-than-light neutrinos have gotten, I'm not looking forward to the eventual conclusion. The result will likely be overturned and be due to an uncertainty that wasn't well understood. Will that be reported on? Will the public understand? I know how hard it is to get undergraduate physics majors to understand error bars and error analysis, so I just don't think Fox News will give it a fair coverage. Either it will be "Physicists were wrong" or.... silence.

Biomedicine is a bit different. The rules are less firm, so unexpected experimental results get published much more often. But they also have bigger consequences. Recently a lot of attention has been paid to whether a certain virus causes Chronic Fatigue. One group has experimental results that it does, but many groups have not been able to get the same results. The result? People with CF have paid money for tests (and possibly treatments) that are ineffective.

While CF isn't a disease with many treatment options, some types of cancer are. If a study showed that Treatment A works very well, patients will be given Treatment A instead of Treatments B or C. What if the study was wrong? Patients may die who would have responded well to treatments B or C. A consoling point is that it takes a lot of work to get a treatment to the clinic - many papers that show Treatment A is effective. But what if mistakes get made?

The ethical violations of someone fabricating research are fairly clear. But what about unintentional mistakes? They happen. I've found mistakes in my own work, far after I thought I had a "final" result. I've found mistakes in published papers. Usually these things are small - though really, the big ones stand out. I dread the day that I find a mistake in my own work - after it has been published. The best thing we can do, as scientists, is to remember that the entire enterprise is meant to be like a conversation. We share ideas and data with each other in order to get feedback and improve our work. We must give reasonable feedback, be open to criticism, and be very honest.

The public has a right to expect us to not fabricate research, but they must understand we are only human.