Do family caps on welfare affect births among welfare recipients? Reconciling efficacy and effectiveness estimates of impact through a blended design strategy

This paper proposes a blended experimental and quasi-experimental research strategy which we believe will help improve the external validity of evaluations of welfare reform and other social policies. We draw upon data from New Jersey's imposition of a family cap, where welfare benefits instead of increasing remained the same for women who conceive and bear a child while on public assistance. We directly compare estimates of the policy's effectiveness on the state's welfare population using (a) a point-in-time sample randomly assigned to experimental and control group conditions; (b) an initial cohort sample similarly assigned; and (c) a before and after policy implementation of the entire welfare caseload. We find that these estimates can differ dramatically, reflecting sensitivity to a time-induced heterogeneity we have labeled the Heraclitus effect. We discuss how multiple estimates of effectiveness can be used to bound expectations of a welfare reform policy impact when population dynamics data are available.