NEW YORK (Reuters Health) – A semiautomated fact-checking tool, the Seek & Blastn tool, has found numerous errors in nucleotide-sequence reagents in biomedical research publications, researchers report.
These reagents are short DNA or RNA sequences used in laboratory techniques such as gene knockdown or polymerase chain reaction.
“I was surprised by the sheer number of incorrect reagents that we found: 91 different reagents from only 77 papers,” said Dr. Jennifer A. Byrne of The Children’s Hospital at Westmead’s molecular oncology laboratory and The University of Sydney, in Westmead, Australia.
“This suggests that there could be many more incorrect reagents still to discover in the literature,” she told Reuters Health by email.
Up to 50% of published preclinical research results have been estimated to be incorrect, and this has led to the development of fact-checking systems for research publications, Dr. Byrne and her colleagues note in PLOS ONE, online March 1.
In biomedical research, they add, most incorrect results are believed to derive from the incorrect use of material standards and experimental reagents.
Dr. Byrne and colleagues developed Seek & Blastn (S&B) to facilitate the identification of publications where the claimed status of a nucleotide sequence does not match its verified status and used it to screen two bodies of literature.
The tool involves three steps: identification and extraction of nucleotide sequences from text together with the associated claim status of targeting or non-targeting; blastn analysis (i.e., using a nucleotide query to search a nucleotide database); and then fact-checking to confirm or deny the usage claim associated with each extracted nucleotide sequence.
Of the two text corpora used to test the tool, Corpus P included 342 nucleotide sequences and Corpus U, with papers from Corpus P, PubMed, and Google Scholar, included 1,522 nucleotide sequences.
Overall, S&B identified 11% of Corpus P sequences and 30% of Corpus U sequences as having errors in sequence and/or status extraction and identified errors in gene identification in 25.4% of Corpus P and 56.8% of Corpus U sequences.
Based on S&B and subsequent manual analyses, there were 91 incorrectly identified nucleotide-sequence reagents, including 26 incorrectly identified RNA-targeting sequences and 65 incorrectly identified PCR primers.
S&B incorrectly flagged only one Corpus P paper (for a precision rate of 97.4%) and incorrectly flagged seven Corpus U papers (for a precision rate of 79.5%).
“We write to authors to let them know that errors have been identified, and then we contact the publishing journals and outline our concerns about each paper,” Dr. Byrne said. “In many cases, journals have published more than one paper with errors, in which case we can bundle reports together. Some journals have been very responsive, and a growing number of papers have either been retracted, corrected, or the subject of expressions of concern.”
“However,” she said, “it can take a long time for other journals to respond. In some cases, we have been writing to particular journals for over 2 years without it becoming clear whether the journal is willing to investigate or not.”
“We need to collectively invest in the health of the literature, or we won’t have the evidence base that we need to translate research to patients in future,” Dr. Byrne said.
Co-author Dr. Cyril Labbe of the University Grenoble Alpes, France, told Reuters Health by email, “It seems to me that, when journals care (often they don’t), they have to take a stand to decide if these errors are just sloppiness (then a correction, expression of concern would be needed) or if they are a symptom of made-up/invented experiments (then a retraction may be needed). The decision has to be taken by the editor.”
“To me, generally speaking, the key of this problem lies in the management of science,” he said. “Imposing strict bureaucratic targets like ‘you must publish at least x publications every y years’ is, by itself, an incitation to fraud.”
“It seems best if scientists are able to investigate without the fear of failing, without having the obligation to sort out success stories,” Dr. Labbe said. “And for science to best benefit the most people, not the fewest, it needs transparency, independence, the ability to fail and to contest results without fear.”
Dr. Byrne added that physicians need “to recognize the importance of reading the literature critically and not automatically believing everything that they read in the scientific literature (unfortunately).”
Seek & Blastn is freely accessible at http://scigendetection.imag.fr/TPD52/.
SOURCE: https://bit.ly/2HNlhL5
PLoS ONE 2019.
