Incentives Vs. Information: Why the Best Science isn't Always Published

[Image Courtesy of Wikimedia Commons]

In a world of viral hits and the need for more clicks, scientific methodology should be the last thing affected.

However, the need for popular and cool research has even tampered with defining success in the sciences. Papers and findings get awards which lead to grant money which leads to more papers, and the cycle continues. However, the cycle can quickly spiral out of control, resulting in unintended consequences.

“The problem that we face is that the incentive system is focused almost entirely on getting research published, rather than on getting research right,” says Brian Nosek, a psychologist at the University of Virginia in Charlottesville in an interview with Science News.

“Real evaluation of scientific quality is as hard as doing the science in the first place,” Nosek says. “So, just like everyone else, scientists use heuristics to evaluate each other's work when they don't have time to dig into it for a complete evaluation.”

Paul Smaldino, a cognitive scientist at University California Merced, put that theory to the test to determine if poor science wins out. Smaldino and Richard McElreath at the Max Planck Institute for Evolutionary Anthropology in Leipzig created a simulation of a scientific "ecosystem." Just like natural selection, the simulated labs that best met "success" parameters reproduced. The ones that didn't simply died out.

Smaldino and McElreath discovered that when success was defined as publishing a lot of new research, the labs that thrived were the ones with "low effort" - sloppy. The labs producing high-quality, thoughtful science didn't publish enough to stay alive.

They even tried to rule out duplicating or double-checking findings of other groups. But low-quality research still thrived.

“The most successful are still going to low effort,” he explained, “because not everyone gets caught.”

But does it really happen, or are Smaldino and McElreath's findings simply off? John Ioannidis, a methods researcher at Stanford University, said their research makes sense:

“Once you have bad practices they can propagate and ruin the scientific process and become dominant. And I think there’s some truth to it, unfortunately."

Is there one traceable element to all of this? Sadly, no. Marc Edwards and Siddhartha Roy with the Department of Civil and Environmental Engineering wrote that the incentives/reward structure developed rapidly over the last 50 years. PhD graduates align themselves in positions with financial backing rather than academia largely for the perks. Edwards and Roy also noted that as adjunct professors become more popular with universities, colleges don't hire full-time professors as often as they're not as costly as their part-time counterparts.

For the research pair, slacking off on research methods for the sake of incentives could lead to devastating results:

"If a critical mass of scientists become untrustworthy, a tipping point is possible in which the scientific enterprise itself becomes inherently corrupt and public trust is lost, risking a new dark age with devastating consequences to humanity. Academia and federal agencies should better support science as a public good, and incentivize altruistic and ethical outcomes, while de-emphasizing output."

As with any issue, the first step is to identify that there is an issue. Edwards said that's easier said than done especially given the protective nature of scientists regarding their research:

“We are taught to be cowards when it comes to addressing these issues, because the personal and professional costs of revealing these problems is so high....I don’t want to wake up someday and realize I’m in a culture akin to professional cycling, where you have to cheat to compete.”

So what's the ultimate solution? Incentives for good research methodology, not only results. Scientists will have to be rewarded for their thoughtful research practices and reporting plans, rather than simply rewarding cool or newsworthy results.

Best-selling author Dan Pink talked about monetary incentives and other ways to effectively motivate people in a 2010 TED Talk. You can watch an animated version of that speech below.

Via Science News

SEE ALSO: Cyber Attacks Stun Amazon, Twitter, Spotify; More Attacks Noted