Limitations of meta-analysis in the evaluation of the effects of psychological therapy

Available meta-analyses have rather consistently shown that behavioral treatments are significantly more effective than alternative forms of psychotherapy. Nevertheless, the limitations of meta-analysis have thus far served more to obfuscate than to clarify the evaluation of therapy outcome. The practice of including methodologically poor studies in meta-analyses is criticized. Different attempts to counter this criticism are reviewed and found wanting. The many statistical, conceptual, and clinical problems inherent in deriving useful units of analysis (effect sizes) for the meta-analysis are highlighted. Finally, the various descriptive and potentially predictive functions of meta-analysis are discussed and compared with the contributions of conventional reviews of the treatment literature. It is concluded that neither the superiority of the former nor the alleged inadequacy of the latter have been logically or empirically demonstrated.