Eight years ago, a team of researchers launched a project to carefully repeat early and influential lab experiments in cancer research. The Reproducibility Project as it is called recreated 50 experiments. Now, it reports that about half of the experiments did not produce the results reported originally.
The reproducibility of experiments and confirmation of results is central to wide acceptance of scientific claims.
“The truth is we fool ourselves. Most of what we claim is novel or significant is no such thing,” said Dr. Vinay Prasad. He is a cancer doctor and researcher at the University of California, San Francisco and was not involved in the project.
Most scientists believe that the strongest findings come from experiments that can be repeated with similar results.
But there is little reason for researchers to share methods and data so others can confirm the work, said Marcia McNutt. She is president of the National Academy of Sciences. Researchers lose respect in the scientific community if their results do not hold up to careful study, she added.
This Jan. 24, 2006 file photo shows a laboratory mouse looking over the gloved hand of a technician at the Jackson Laboratory, in Bar Harbor, Maine. (AP Photo/Robert F. Bukaty, File)
For the project, the researchers tried to repeat experiments from cancer biology papers. The papers had appeared in major scientific publications, including Nature and Cell, from 2010 to 2012.
Overall, 54 percent of the original findings failed to meet conditions set by the Reproducibility Project. The team’s study appears in the journal eLife. The nonprofit eLife receives support from the Howard Hughes Medical Institute, which also supports The Associated Press Health and Science Department.
Among the studies that did not hold up was one that claimed a link between a stomach bacteria and colon cancer. Another claimed a drug had shrunk tumors in mice. And, a third was a mouse study of a possible prostate cancer drug.
This is the second major study by the Reproducibility Project. In 2015, they found similar problems when they tried to repeat experiments in psychology.
Study co-writer Brian Nosek of the Center for Open Science said it can be wasteful to move forward without first doing the work to repeat findings.
The researchers tried to limit differences in how the cancer experiments were carried out. Often, they could not get help from the scientists who did the original work. They could not get answers to questions such as which kinds of mice to use or where to find specially engineered tumor cells.
Michael Lauer is a deputy director of research at the National Institutes of Health (NIH). He said, “I wasn’t surprised, but it is concerning that about a third of scientists were not helpful, and, in some cases, were beyond not helpful.”
Lauer added that the NIH plans to require data-sharing among organizations to which it provides support in 2023.
Dr. Glenn Begley is a bio-technology advisor and former head of cancer research at drugmaker Amgen. Ten years ago, he and other scientists at Amgen reported even lower rates of confirmation when they tried to repeat published cancer experiments.
Cancer research is difficult, Begley said. He added that “it is very easy for researchers to be attracted to results that look exciting and provocative, results that appear to further support their favorite idea as to how cancer should work, but that are just wrong.”
I’m Jonathan Evans.
Carla K. Johnson reported on this story for the Associated Press. Jonathan Evans adapted this story for Learning English. Caty Weaver was the editor.
______________________________________________
Words in This Story
originally – adv. in the beginning; when something first happened or began
novel – adj. new and different from what is already known
significant – adj. large enough to be noticed or have an effect
journal – n. a magazine that reports on things of special interest to a particular group of people
provocative – adj. causing discussion, thought, argument, etc.