In our new preprint we investigated the validity of statcheck. Our main conclusions were:

• statcheck’s sensitivity, specificity, and overall accuracy are very high. The specific numbers depended on several choices & assumptions, but ranged from:
  • sensitivity: 85.3% – 100%
  • specificity: 96.0% – 100%
  • accuracy: 96.2% – 99.9%
• The prevalence of statistical corrections (e.g., Bonferroni, or Greenhouse-Geisser) seems to be higher than we initially estimated
• But: the presence of these corrections doesn’t explain the high prevalence of reporting inconsistencies in psychology

We conclude that statcheck’s validity is high enough to recommend it as a tool in peer review, self-checks, or meta-research.