2:13 PM
Apr 12, 2018
 |  BioCentury  |  Regulation

Raising the bar

Why John Ioannidis says the norm for p-values should shift to 0.005

John Ioannidis has become one of the most-cited living scientists by telling his colleagues that a lot of what they think they know is wrong. His 2005 paper, “Why most published research findings are false,” has provoked vigorous debates in academic journals across scientific disciplines and on barstools around the globe.

Ioannidis, the C.F. Rehnborg Chair in Disease Prevention at Stanford University, sparked another round of debate recently by publishing a proposal to lower p-value thresholds for statistical significance to 0.005 from the 0.05 level in common use today. The shift would move about 30% of research results from statistically significant to merely suggestive, he estimates.

In his paper, Ioannidis states that the most common misconception is that a p-value of, for example, 0.02 (2%) means that there is a 2% chance the null hypothesis is correct -- i.e. that the drug is as effective as placebo -- and a 98% chance the alternative is correct -- that the drug is more effective than placebo.

This interpretation is only correct if a number of criteria for designing and implementing the experiment are met, and they rarely are.

Raising the bar on p-values would be a practical move, a relatively easy step to take to increase rigor. But it should be considered an interim step, rather than a solution to the underlying issues, according to Ioannidis. He contends that the ultimate goal should be adoption of more appropriate statistical tools.

This needs to start with critical thinking about the design of experiments and clinical trials. Scientists should perform fewer experiments, and those they do conduct should be much larger, Ioannidis says.

A clear-eyed view of statistics would likely result in broader use of Bayesian methods, as well as wider reporting and emphasis on effect sizes and confidence intervals, he believes.

He also advocates for more widespread adoption of best practices for conducting studies and reporting results, which can root out bias and facilitate better data-based decision-making.

BioCentury spoke with Ioannidis about his proposals. An edited version of the conversation follows.

BioCentury: In your recent paper, you said that p-values are misinterpreted, over-trusted and misused. How?

John Ioannidis: The most common misinterpretation is that p-values are equated with truth or non-truth statements. People literally take the p-value and translate it to a probability that something is correct. A p-value of 0.03 is translated as a 3% chance of being wrong and 97% chance the answer would be correct, which is completely misleading, completely wrong.

Over-trust is a problem because even if someone knows how to read p-values, they often make a lot of assumptions. They assume, for example,...

Read the full 2158 word article

User Sign in

Trial Subscription

Get a 4-week free trial subscription to BioCentury

Article Purchase

$150 USD
More Info >