Cracks in the Foundation

How data fabrication, misrepresentation and irreproducibility are wasting research efforts and undermining our confidence in scientific literature

Fraud. Misrepresentation. Plagiarism. In the science community, words such as these are rare, and usually only spoken in a susurrus behind closed doors (see Box 1 for definitions). Many scientists are uncomfortable with the reality that misconduct occurs. Understandably, fraud and misrepresentation are perhaps the antitheses of science – a system that is defined by the derivation of general truths and laws, and the establishment of knowledge of the physical world through experimental observation.

Scientific literature, however, is not infallible. Whether deliberate or accidental, the knowledge base that we rely upon contains errors or misrepresented data that sends many chasing ghosts and results in countless hours and millions of dollars in research funding being wasted every year. Unfortunately, it is undeniable that data misrepresentation is on the rise.

Fraud and misrepresentation
A recent study published in PNAS by Fang et al. in 2012 showed that out of the 2,047 articles marked as retracted in PubMed, 67% were due to misconduct which encompassed fraud, plagiarism and duplication (see figure). According to the study, the frequency of retractions for fraud has been on the rise since the 1990s, with the greatest increase seen in the past decade.

Some individuals had retracted multiple articles, including anesthesiologist Joachim Boldt, previously of the Ludwigshafen Clinical Center in Germany, who now has a total of 88 retractions for fraud. The current retraction record however, is held by Yoshitake Fujii, another anesthesiologist with 172 articles retracted.

The details of these retractions are available on Retraction Watch, a blog created by Ivan Oransky and Adam Marcus (http://retractionwatch.wordpress.com). Oransky and Marcus shed light on some interesting cases of fraud including the story of Hyung-In Moon, a researcher at Dong-A University in South Korea, who invented fictional reviewers with corresponding e-mails so he could review his own article submissions.

Moon’s dishonesty was only suspected when the journal editor of Enzyme Inhibition and Medicinal Chemistry noticed that peer reviews for Moon’s article were being received within 24 hours. The editor contacted Moon, who admitted to reviewing his own articles, and twenty-four articles were eventually retracted from journals operated by the publisher Informa Healthcare in 2012. The editor-in-chief of Informa Healthcare also resigned at approximately the same time the retraction notices were published. Moon however, is still listed as a researcher with his university.

I have always been amused by the term scientific integrity, because I would have thought that each part of it implies the other. -Sydney Brenner

A recent high profile case of misconduct in Canada involved Zhiguo Wang, a former researcher at the Montreal Heart Institute and University of Montreal. Wang withdrew two papers in the Journal of Biological Chemistry (JBC) in 2011, and was dismissed from both institutions following an investigation showing that Wang “deviated from…ethical standards of proper scientific conduct and his responsibilities as a researcher.” A total of six of Wang’s papers have now been retracted. Although JBC did not detail the reason for the retractions, visual analysis of one paper reveals partial duplication of an image for two separate experiments (see figure below). In the midst of the allegations, Wang returned to China and his current position is unknown.

In most cases, misconduct is dealt with internally by the respective academic institution, and few researchers suffer any serious consequences for fraud. One rare example involving unusual repercussions for scientific fraud involves Dr. Eric Poehlman, a previously tenured faculty member at the University of Vermont.

In 2006, Poehlman pleaded guilty to “lying on a federal grant application and admitted to fabricating more than a decade’s worth of scientific data on obesity, menopause and aging, much of it while conducting clinical research…” Poehlman initially denied the allegations in 2001 and used a litany of excuses to explain the discrepancies in data. After a protracted investigation by the University of Vermont and prosecution by the United States Department of Justice in 2005, Poehlman admitted to committing fraud and received just one year in prison, which is still highly unusual for any scientist caught committing fraud.

Reproducibility

A key aspect of the scientific method is independent reproducibility. In an editorial published in Nature last year, Glenn Begley and Lee Ellis revealed that only 6 out of 53 (a shocking 11%) of selected ‘landmark’ studies in cancer research were reproducible by a group of scientists at Amgen. In their editorial, Begley and Ellis suggest that the low rates of reproducibility were not unique to Amgen. They cite another study by a group at Bayer Healthcare in Germany published in 2011 that found that only 25% of selected, published preclinical studies could be validated.

Many of the landmark studies evaluated by the Amgen team were published in high-impact journals, and individually had hundreds of citations. Unfortunately, due to non-disclosure agreements signed by Amgen scientists and the authors of the original studies, Begley and Ellis were not permitted to identify the specific studies that were irreproducible. As a result, significant research time and investment continue to be squandered on secondary studies based on data and conclusions made from publications with erroneous data.

Plagiarism

Plagiarism is another form of misconduct in science. Most cases of plagiarism involve duplication of data and text from an original publication, and the resubmission of data, most often in a different journal.

In academic settings, web-based services such as TurnItIn.com are commonly used to detect possible instances of plagiarism in student essays and reports. However, most journals do not typically perform automated checks for plagiarism in submitted research articles.

Harold Garner, a Professor at the Virginia Bioinformatics Institute at Virginia Tech, has made some significant strides in detecting cases of plagiarism in scientific literature. In 2006, Garner and his group described a text similarity search algorithm called eTBLAST. Garner and colleagues built upon eTBLAST to create an automated, online database of highly similar citations called Deja vu (freely searchable at http://dejavu.vbi.vt.edu/dejavu/). Based on abstracts indexed within PubMed, Deja vu automatically flags those that exceed a set similarity threshold. Flagged entries are manually curated to confirm or disprove duplicate entries that may also suggest plagiarism.

Some records in Deja vu represent cases where the authors have published the same article in different journals. Other Deja vu records show articles that share common authors who used highly similar writing styles and word organization between different publications. However, many research articles included in the database do not share common authors. Of these articles, some are blatant examples of plagiarism (see figure).

Using results from Deja vu, Garner wrote an article in Science in 2009 in which his group identified over 150 cases of potential plagiarism with no overlapping authors and an average full-text similarity of 86%. Based on these data and subsequent internal investigations by the respective journal editors, 56 of the duplicate articles were eventually retracted.

Addressing the issues

For some of these problems, solutions exist and are straightforward to implement. For others, practical ways of solving these issues are not readily available.

Calling attention to fraud or misrepresentation, for example, can be a dangerous endeavor. Whistleblowers not only face an uphill battle against the accused, but may also jeopardize their own careers in the process. Charles McCutchen, a physicist at the National Institutes of Health in Bethesda, Maryland, and author of many articles on scientific fraud and research ethics, wrote:

In protesting scientific fraud, the whistleblower soon realizes that he or she will have few allies. The biomedical science establishment has taken the position that fraud is very rare, and will use almost any means to maintain that illusion” (letter to Brian Martin, 1989).

Many external factors may contribute to the alarming increase in fraud, but the pressure to publish in high-impact journals can be a major driving force. Securing funding and tenure frequently hinges upon a successful track record that is mainly evaluated by the quality and quantity of publications. These rewards can become great motivators for misrepresentation and fraud. The disincentives are few, and only in extreme cases are there serious repercussions. Solving this would require a complete redesign of the system.

Reproducibility is also a difficult problem to address. Ideally, results that are robust should be reproducible in a variety of experimental systems. However, small variances in growth conditions or animal systems can eliminate minor, yet significant, observable differences. The peer review process is frustrating enough that imposing additional reproducibility requirements would only add to the burden that many authors carry in publishing their results.

Ultimately, all scientists – from trainees to investigators to reviewers and editors – must be rigorous in their analysis of data and vigilant for possible fraud, misrepresentation or plagiarism. Journals should perform automated checks for data or text duplication by using databases such as Deja vu and any available software to detect image manipulation. Retraction notices should be announced with transparency, rather than the often vague and obscure statements that journals issue to avoid embarrassment or potential libel charges. Overall, these changes have the potential to improve the quality of scientific literature and ensure the continued integrity of scientific knowledge.