In the wake of a journal's retraction of the Séralini GMO study due to "inconclusive" findings, Prof Jack Heinemann shares his top recommendations for the retraction of other "inconclusive" papers.
Let's give the scientific literature a good clean up
Prof Jack Heinemann
INBI, the Centre for Integrated Research in Biosafety via Newsvine, Dec 24 2013
http://biosafetycooperative.newsvine.com/_news/2013/12/24/22038524-lets-give-the-scientific-literature-a-good-clean-up
Food and Chemical Toxicology took the unusual step in November 2013 to retract a paper (Séralini et al., 2012) that it had published a year before. The paper’s strengths and limitations for measuring potential adverse effects of a kind of genetically engineered corn have been systematically evaluated (Meyer and Hilbeck, 2013). It did not have falsified data, was not plagiarized, nor was it found to be incorrect. Instead, the paper was found to be conclusively inconclusive by a post hoc review.
I think that determination was not wrong. After all, the authors said things like “need further investigations”. Those kinds of giveaways are never seen in good papers. Good papers are known to be good from the moment that they are published. They are never controversial. They are never wrong and never will be wrong. And they are always liked by the right people and the people who are right.
This retraction inspired me to review my collection of old photocopied papers and clean away all relics of my misspent youth reading and saving papers that should never have soiled my hands or clouded my mind. Oh the guilt for having condemned certain kinds of molecules to bear the likeness of those papers for so many years, hidden in a dark cabinet in a messy office.
"Why stop there?" I thought. Why not expunge the scientific literature itself of these unworthy scraps? I will share with you some of my top recommendations for retraction. This is far from a comprehensive list. You are welcome to browse my list. Afterward, perhaps you will stick around and I can tell you what I learned.
#1: The first paper to show that genes were made of DNA
Many thought that Oswald Avery and coworkers were unfairly denied the Nobel Prize for their work in 1944 (Avery et al., 1944) showing that the substance called DNA (deoxyribonucleic acid) could cause heritable changes in a bacterium (Deichmann, 2008). The paper is seen as seminal in the path to associating DNA with the stuff of genes.
Nobel Laureate and Avery contemporary Gunter Stent said of the work that “‘Avery’s discovery was widely discussed and yet it could not be appreciated for eight years...The then-current view of the molecular nature of DNA…made it well nigh inconceivable that DNA could be the carrier of hereditary information’” (recounted in Deichmann, 2008).
(Note to self: eight years? First sign that this is not a good paper.)
In framing their discovery, Avery et al. issued a caveat that held faith with the evidence. When they said that uptake of DNA into bacteria resulting in a heritable change was caused by DNA, they carefully qualified their finding to acknowledge that it was possible for “minute amounts of some other substance adsorbed to it or so intimately associated with it as to escape detection” and for that contaminant to be the cause of the trait and its heritability (Avery et al., 1944).
(Equivocal; second sign that this is not a good paper.)
The contents of this paper are ubiquitous in undergraduate textbooks used to teach microbiology and molecular biology. The paper is used as an icon of setting up careful and meaningful experimental controls. However, the results of Avery et al. were not conclusive in the day. So I nominate it for retraction from the scientific record.
#2: The first two papers describing the structure of DNA and how it might replicate
In their famous papers in April (Watson and Crick, 1953b) and May (Watson and Crick, 1953a) of 1953 reporting the structure of DNA and the implications of this structure, Watson and Crick expressed several fundamental qualifications on their data. For example, as late as the May paper they indicated that their X-ray crystallographic data reported in the April paper might not be definitive.
What made the data so profoundly interesting were the genetical implications because by the time of the Watson and Crick series of papers, there had already accumulated several independent experimental demonstrations of the relevance of nucleic acids, particularly DNA, to heritability of traits (Avery et al., 1944, Hershey and Chase, 1952). The most profound genetical implication was that the structure of the molecule could explain in biochemical terms how gene replication might happen.
“It has not escaped our notice that the particular pairing that we have postulated immediately suggests a possible copying mechanism for the genetic material” they said (Watson and Crick, 1953b). Nevertheless, it would be another five years to verify this claim linking speculation on DNA’s structure to experimental evidence of how that structure participated in its replication (Meselson and Stahl, 1958). The Watson and Crick model (Watson and Crick, 1953a) for how DNA replication might work was remarkably prescient, but it was not conclusive in the day and thus I nominate the paper be retracted from the scientific record.
#3: The first (actually not) paper to show that genes transfer from bacteria to plants
The suggestion that DNA (i.e., genes) might transfer from bacteria to plants and animals was controversial for many years (Ferguson and Heinemann, 2002). Historically some of the strongest evidence was provided by an interaction between the pathogenic bacterium Agrobacterium tumefaciens and plants which resulted in a disease called crown gall (Heinemann, 1991). There were accumulating lines of evidence that gene transfer occurred between bacteria and plants from the late 1960s to the middle 1970s (Drlica and Kado, 1975, Stroun et al., 1970, Stroun et al., 1977). However, an experiment in 1974 failed to demonstrate the existence of DNA of bacterial origin in plant gall tumors (Chilton et al., 1974). The team that first reported that DNA from A. tumefaciens was not found in plant galls also later reported not being able to replicate other studies providing positive detections of A. tumefaciens-derived DNA in the galls (Chilton et al., 1977). But wait! The team later did detect DNA from the bacterium in plant gall cells (Chilton et al., 1977).
The confirmation of DNA transfer from a bacterium to a plant became the basis for plant genetic engineering for decades, and remains part of the history of most genetically modified plants in commercial use. Consequently, this work attracted for the team leader the World Food Prize in 2013. Unfortunately though, both the Chilton et al. (1974) and the Chilton et al. (1977) papers report some results that were either wrong or inconclusive. I therefore suggest that the papers be retracted from the scientific record.
#4: The paper that showed that prions from cattle can’t infect brains of other species
In 1995 one of the UK’s leading laboratories on prion diseases published the results of an ongoing study with relevance to the question of whether a prion could transmit from cattle to humans (Anonymous, 1995).
The study used mice that were genetically modified to express a human version of a protein that changes in conformation when humans develop Creutzfeldt-Jacob disease (CJD). This is the human form of a spongiform encephalopathy. Mice exposed to proteins extracted from cattle demonstrating symptoms of “Mad Cow Disease”, otherwise known as bovine spongiform encephalopathy (BSE), showed no statistically significant increase in occurrence of the disease. This suggested that the agent causing BSE was not transmissible to humans from cattle (Collinge et al., 1995).
The journal Nature also published a news item wherein it reported a conflict between the authors and other scientists, the latter claiming that the results were inconclusive (Anonymous, 1995). Their argument was that the disease can have an indefinite incubation period between exposure and symptoms and the animals may not have been old enough to show symptoms at the time of publication (Hope, 1995). Moreover, the incubation period for cross-species disease, as was being monitored in the mice, was already known to be longer still. There was also the possibility that their results could be false negatives because of interference by the mouse version of the protein (Heinemann et al., 2013). The relevant control group of mice expressing only the human form of the protein had been started much later and results were not available at the time Collinge et al. chose to publish.
It later would be shown that the infectious particle in cattle could transmit to humans (and to GM mice), with the Collinge group among those credited with providing convincing evidence (Almond and Pattison, 1997). In an obscure addendum to the 1995 article appearing two years later in Nature, the group announced “We also included the interim results of challenge of mice expressing only human prion protein…with BSE; these mice had remained disease-free 264 days post-inoculation. We have now found that from 489 days onwards, some of these…mice developed a prion disease” (Collinge et al., 1997).
The results of the 1995 study were inconclusive at the time of publication but the paper persists in the scientific literature. I nominate it for retraction.
What have I learned?
First, that no service would be done to science or future scholars removing these classic papers from the literature. They are icons for different reasons. Some were revolutionary and under-appreciated (in their time). Some were revolutionary and timely. Others were state of the art but failed to follow the right clues. We have something to learn from all of them. Not all of what we learn from a scientific paper is about the science itself.
Second, conclusivity is in some aspects missing from all of these papers, and nearly absent in some of them. That does not make them irrelevant to the story, to science, and to learning about how science works. Conclusivity is a goal of an impoverished imagination.
In saying this I am not suggesting that some scientific works are not better than others. It is to suggest that at the forefront of discovery and investigation there will be great uncertainty. Some of it will be in the form of experimental error, some from human limits of perception, some from chance. Valid science stands the test of time. Give it that.
Third, some science is more controversial than other science. The closer that science gets to commercial interests, or conflicts with what lots of other people believe, the louder the shouts. The historical record of not retracting papers, some of which were inconclusive and outright wrong in important ways, shows that retraction is a political decision when not based on misconduct or error.
Fourth, what these papers teach us is that getting it wrong, or getting less than definitive results, is not uncommon. It is neither good nor bad but the unavoidable collateral of trying. When we fail to acknowledge this in science, and in teaching, we mislead the public and our students into thinking that answers are obvious and the science is unambiguous. Neither is true.
Finally, a retraction will be seen by some as something that it is not. They will take it as vindication that those who disagree with the paper were right. Retraction based on failure to be conclusive does not show this. It only shows how vulnerable scientific communication is to politics.
References
Almond, J. and Pattison, J. (1997). Human BSE. Nature 389, 437-438. Anonymous (1995). BSE results 'may quell panic', but caution still needed. Nature 378, 759. Avery, O. T., MacLeod, C. M. and McCarty, M. (1944). Studies on the chemical nature of the substance inducing transformation of Pneumococcal types. J. Exp. Med. 79, 137-158. Chilton, M.-D., Currier, T. C., Farrand, S. K., Bendich, A. J., Gordon, M. P. and Nester, E. W. (1974). Agrobacterium tumefaciens DNA and PS8 DNA not detected in crown gall tumors. Proc. Natl. Acad. Sci. USA 71, 3672-3676. Chilton, M.-D., Drummond, M. H., Merlo, D. J., Sciaky, D., Montoya, A. L., Gordon, M. P. and Nester, E. W. (1977). Stable incorporation of plasmid DNA into higher plant cells: the molecular basis of crown gall tumorigenesis. Cell 11, 263-271. Collinge, J., Palmer, M. S., Sidle, K. C. L., Hill, A. F., Gowland, I., Meads, J., Asante, E., Bradley, R., Doey, L. J. and Lantos, P. L. (1995). Unaltered susceptibility to BSE in transgenic mice expressing human prion protein. Nature 378, 779-783. Collinge, J., Palmer, M. S., Sidle, K. C. L., Hill, A. F., Gowland, I., Meads, J., Asante, E., Bradley, R., Doey, L. J. and Lantos, P. L. (1997). Addendum: Unaltered susceptibility to BSE in transgenic mice expressing human prion protein. Nature 389, 526. Deichmann, U. (2008). Antimicrobial resistance in Escherichia coli and Pseudomonas aeruginosa from Intensive Care Units in The Netherlands, 1998‚Äì2005. HSPS 34, 207-232. Drlica, K. A. and Kado, C. I. (1975). Crown gall tumors: are bacterial nucleic acids involved? Bacteriol. Rev. 39, 186-196. Ferguson, G. C. and Heinemann, J. A. (2002). Recent history of trans-kingdom conjugation. In Horizontal Gene Transfer, M. Syvanen, and C. I. Kado, eds. (London and San Diego, Academic Press), pp. 3-17. Heinemann, J. A. (1991). Genetics of gene transfer between species. Trends Genet. 7, 181-185. Heinemann, J. A., Agapito-Tenfen, S. Z. and Carman, J. A. (2013). A comparative evaluation of the regulation of GM crops or products containing dsRNA and suggested improvements to risk assessments. Environ Int 55, 43-55. (Open Access) Hershey, A. D. and Chase, M. (1952). Independent functions of viral protein and nucleic acid in growth of bacteriophage. J. Gen. Physiol. 36, 39-56. Hope, J. (1995). Mice and beef and brain diseases. Nature 378, 761-762. Meselson, M. and Stahl, F. W. (1958). The replication of DNA in Escherichia coli. Proc. Natl. Acad. Sci. USA 44, 671-682. Meyer, H. and Hilbeck, A. (2013). Rat feeding studies with genetically modified maize - a comparative evaluation of applied methods and risk assessment standards. Environ Sci Eur 25, 33. Séralini, G.-E., Clair, E., Mesnage, R., Gress, S., Defarge, N., Malatesta, M., Hennequin, D. and de Vendômois, J. S. (2012). Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize. Food Chem. Toxicol. 50, 4221–4231. Stroun, M., Anker, P. and Auderset, G. (1970). Natural release of nucleic acids from bacteria into plant cells. Nature 227, 607-608. Stroun, M., Anker, P., Maurice, P. and Gahan, P. (1977). Circulating nucleic acids in higher organisms. Int. Rev. Cytol. 51, 1-48. Watson, J. D. and Crick, F. H. C. (1953a). Genetical implications of the structure of deoxyribonucleic acid. Nature 171, 964-967. Watson, J. D. and Crick, F. H. C. (1953b). Molecular structure of nucleic acids. Nature 171, 737-738.