The value of p-value

In the latest issue of Nature there is a piece by Regina Nuzzo about how un-reliable p-values are in interpreting statistical tests. She describes how researchers use the p-value entirely differently than how Fisher intended it to be used when he first published it in the 1920s. His significance, censu Nuzzo, was a mere means to test whether a result is worth pursuing. It has become the gold standard in scientific work to test for the 0.05 significance level without providing information about the feasibility of the hypothesis being tested, or on the strength of the effect being reported. Without that information, the p-value is not very informative and in many cases might be misleading. This paper can be related to a long historic feud among statisticians regarding the pros and cons of frequentist statistics versus, for example, Bayesian inference techniques.

The take home message from Nuzzo’s and other papers on the subject, is that it’s not enough to use p-value, one should always look for additional ways to test the rigor of one’s results, and to describe them in a way that renders them highly reproducible.

Advertisements

4 comments

  1. Thanks for bringing our attention to this Rafi. I’ve hyper-linked the Nuzzo paper. 😉

    In case you missed it, there was a paper in PNAS late last year addressing similar issues (http://www.pnas.org/content/110/48/19313.long). I haven’t had time to read it very carefully but I think it essentially says that the magical 0.05 threshold should be much lower; partly because some of the issues Nuzzo brings up in her first figure.

      1. Of course it’s possible to ignore p-value altogether, but I guess that the transition would be smoother if people reported both p-value and effect size (with confidence intervals as well).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s