Making error detection easier – and more automated: My guest post about statcheck on Retraction Watch

November 2015

I wrote a guest post for Retraction Watch about the development of Sacha’s and my R package “statcheck”, which automatically extracts statistical results and recomputes p-values.

Read it here: http://retractionwatch.com/2015/11/17/making-it-easier-and-more-automated-to-find-errors-a-guest-post-from-the-co-developer-of-statcheck/

“The prevalence of statistical reporting errors in psychology (1985-2013)” published at Behavior Research Methods

October 2015

In this paper we use the automated procedure “statcheck” to extract over 250.000 p-values from 30.000 psychology articles and check whether they are consistent.

We find that half of the articles contain at least one inconsistency, and 1 in 8 articles contains a gross inconsistency that affects the statistical conclusion. The prevalence of inconsistencies seems to be stable over time.

The article is Open Access and available here: http://link.springer.com/article/10.3758/s13428-015-0664-2

“The prevalence of statistical reporting errors in psychology (1985-2013)” accepted for publication at Behavior Research Methods

September 2015

In this manuscript we use the R package statcheck (by Sacha Epskamp & me) to examine the prevalence of statistical reporting errors in 8 major psychology journals from 1985 to 2013. We find that half of all articles contains at least one inconsistency and 1 in 8 articles contains a grossly inconsistent p-value that could have changed the statistical conclusion. We find no evidence that the prevalence of inconsistencies is increasing over the years.

You can find the post-print here: PDF

The Replication Paradox is now published!

June 2015

My latest paper “The replication paradox:  Combining studies can decrease accuracy of effect size estimates” is now published in Review of General Psychology. You can find a postprint of the paper here. The full reference is:

Nuijten, M. B., Van Assen, M. A. L. M., Veldkamp, C. L. S., & Wicherts, J. M. (2015).The replication paradox: Combining studies can decrease accuracy of effect size estimates. Review of General Psychology, 19(2), 172-182. http://dx.doi.org/10.1037/gpr0000034

The R package statcheck is now on CRAN!

November 2014

Statcheck is an R package that Sacha Epskamp and I wrote together. Statcheck extracts statistics and recomputes p-values. Extremely handy to check your own papers for accidental slip-ups in the result section, but it can also be used to estimate error prevalence across a wide range of scientific articles. So far, statcheck can only read results that are reported exactly in APA style. Note that in order to scan PDF files, you need to have the program pdf-to-txt installed and on your path variable. Furthermore, if you want to run statcheck on a Mac, you need to install XQuartz. For more info, see the project page.