Microbes May Take Some of the Blame for the Reproducibility Crisis

For the past several years, science has been plagued by what some consider a reproducibility crisis—the idea that many published studies don’t yield the same results when repeated by other scientists, or sometimes even by the same scientists. Surveys and studies have suggested myriad reasons such as unreliable reagents and cell lines, selective statistics, a bias for publishing studies with positive results, and genetic differences in animal models.

Andreas Bäumler, a microbiologist and immunologist at the University of California, Davis, and his team recently published evidence in Nature Microbiology last month (March 25) that, at least in some mouse studies, the problem may come down to the bacteria in the model’s gut. They found that mice of the same strain but purchased from different vendors had different susceptibilities to infection during experiments, a phenomenon that could be explained by varying microbiomes.

“It’s basically a very well-done study of something that people have known for a while and have done in various bits and pieces,” says Vincent Young, a microbiologist at the University of Michigan Medical School who was not part of the study. “There are some cases where, if the results are somewhat variable, you think you’ve controlled for everything. This is another thing that you need to look at to make sure that you’ve accounted for it. . . . Don’t ignore it anymore.”

Bäumler and his team study Salmonella typhimurium, the bacteria responsible for salmonella infections, and how it interacts with its host in the gut using mouse models. During experiments, when a group of animals is exposed to a pathogen, not all of them will get sick and of those that do, the disease will take varying tolls.

In an effort to minimize these differences and make his work as reproducible as possible, Bäumler has made a point of using mice that are genetically similar. But that doesn’t eliminate all the variability. He wondered if the differences originated in the gut, specifically, microbial differences in genetically similar mice coming from different vendors.

To find out if that was the case, his group purchased the same strain of mice from four companies and conducted a series of experiments to see if their microbiota differed and how that might influence infection.

“Once you know why things are variable, then it becomes quite easy to progress and to make experiments that are reproducible. The hard step is to find out what on earth causes the variation,” says Bäumler.

See “The Great Big Clean-Up”
He and his team exposed the mice to S. typhimuriumand noted that the ones that got sick often came from the same vendor, while the ones that were resistant came from the other firms.

To see if the inconsistency had to do with differing gut microbes, the group conducted fecal transplants—taking samples of the microbial community from the mice and transferring them to germ-free mice that had their microbiomes wiped out. As expected, the mice that received transplants from the resistant mice also resisted infection, whereas the mice that received their transplants from the susceptible donors got sick.

When the team sequenced the animals’ microbial RNA, nothing stood out. “It was clear that it was the microbiota,” Bäumler says. “[So] we tried to resequence everything, but that didn’t give us any clues of who was responsible.”

The team tried cohousing the susceptible mice with mice from another vendor for 14 days, which is known to lead to microbial sharing. Some of the susceptible mice became resistant to infection, but not all of them. Again, the scientists sequenced the microbiota in both the mice that resisted infection and those that didn’t and noticed that a certain group of bacteria, Enterobacteriaceae—which is the family of bacteria that S. typhimuriumbelongs to—had been transferred to the resistant mice but not the susceptible ones. “That was the first hint,” Bäumler says.

Building on that insight, the team isolated just the Enterobacteriaceae from the resistant mice and gave those to the susceptible mice. It worked; the previously vulnerable animals now resisted infection.

S. typhimuriumand an Enterobacteriaceae the team found—E. coli—have very similar metabolisms. They both require oxygen to flourish, unlike many other bacteria in the gut. In the final step of the experiment, the researchers knocked out an enzyme the E. coli use to respire oxygen, which led to an expansion of S. typhimurium.

While Young points out that knocking out this enzyme could have had other unseen effects, Bäumler says the expansion of the pathogen indicates that the two species had been competing for the same nutrients.

Bäumler emphasizes that the result will be important for helping scientists minimize variability in their studies.

“That’s where the silver lining in some of these issues really lays. . . . Yes, this is an issue for reproducibility. Yes, we all, if we’re doing mouse research, we need to be aware of these things and very cognizant of these variables,” says Aaron Ericsson, a microbiologist and pathologist at the University of Missouri College of Veterinary Medicine and coinvestigator at the Mutant Mouse Resource and Research Center, which offers services sequencing both genes and microbiomes of mice for other laboratories. “But then also we need to take a step back and go, ‘Well hey, this is really interesting. What might this tell us about this disease? Can we exploit this somehow therapeutically or diagnostically?’”