Sample sizes and effect sizes are negatively correlated in meta-analyses: Evidence and implications of a publication bias against nonsignificant findings

Academic Article

Abstract

  • Meta-analysis involves cumulating effects across studies in order to qualitatively summarize existing literatures. A recent finding suggests that the effect sizes reported in meta-analyses may be negatively correlated with study sample sizes. This prediction was tested with a sample of 51 published meta-analyses summarizing the results of 3,602 individual studies. The correlation between effect size and sample size was negative in almost 80 percent of the meta-analyses examined, and the negative correlation was not limited to a particular type of research or substantive area. This result most likely stems from a bias against publishing findings that are not statistically significant. The primary implication is that meta-analyses may systematically overestimate population effect sizes. It is recommended that researchers routinely examine the n-r scatter plot and correlation, or some other indication of publication bias and report this information in meta-analyses. © 2009 National Communication Association.
  • Authors

    Published In

    Digital Object Identifier (doi)

    Author List

  • Levine TR; Asada KJ; Carpenter C
  • Start Page

  • 286
  • End Page

  • 302
  • Volume

  • 76
  • Issue

  • 3