Table A1.1 Literature search: Inclusion Criteria
Inclusion criteria Rationale
Focus Cost-benefit analyses of one or more early childhood interventions
Cf. the research questions formulated in Chapter 1.
Study design Cost-benefit analysis including a description of costs and benefits and reporting a CB ratio or IRR.
Primary benefits obtained from study designs: social experiments, randomised controlled trials (RCT), quasi-experiments, natural experiments (regression
discontinuity design, difference-in-difference, IV), systematic reviews, meta-analytic designs, reviews and impact evaluations.
Primary benefits must be estimated or reported.
We include benefit analyses that conduct a full cost-benefit analysis.
We include economic evaluations that sought to compare at least two interventions (or an intervention with treatment as usual) in terms of their costs and benefits.
Primary benefits must be estimated or reported in order to monetise benefits.
Reviews and impact evaluations may contain cost-benefit analyses or methodological discussion although this is not the primary research objective.
Publication year Studies published from January 2008 to December 2017 (both months included)
A large review exists based on previous studies (Karoly 2008). Our aim is to expand this review with recent publications.
Publication type Cost-benefit analyses, cost-savings analyses, journal papers, working papers, technical reports, web appendixes, reports, policy reports, books and book chapters
We are interested in the grey-area literature (such as working papers, technical appendixes and guidelines) to identify the best practices and methodological developments in cost-benefit analyses of early childhood interventions.
Fields We search in the fields of social sciences:
economics, public policy, psychology and sociology.
We exclude development economics, health economics and medicine.
Our focus is social programmes not clinical experiments.
Interventions Interventions and programmes aimed at early childhood development, cognitive or non-cognitive development, social and emotional learning (SEL), mental health, literacy and language, school readiness, school attendance, academic attainment, risky behaviour, personal skills, life skills, and parent training and adult-child interactions.
Interventions that are implemented at home or in centre-based institutions (family centre, child care, day care nursery school), preschools, kindergarten, schools (primary or secondary; K-12), after-school activities, and youth and mentor programmes (e.g.
crime prevention programmes).
Our aim is to identify state-of-the-art methods. Thus, the specific aim of the intervention is of less importance. This means that our search is very broad and that we may have to spent additional resources on the initial screening of papers to identify those meeting our methodological quality standards.
We exclude papers on school and accountability reforms, teenage pregnancy prevention programmes and employment programmes. We also exclude papers on medical treatments.
Institutional context
Developed countries We exclude cost-benefit analyses from developing economics. This is partly due to resource constraints and partly due to our main research objective, which is to describe best-practice methods for cost-benefit analyses of programmes and child outcomes of interest in Scandinavian public policy.
Publication language
Studies written in English, Danish Swedish or Norwegian.
This is a resource constraint. In addition to studies written in English, which is the preferred publication language in Denmark, we only search studies in Danish, Swedish and Norwegian.
We expect studies of particular interest for cost-benefit analyses of Scandinavian early childhood programmes to be published in one of these languages.
Note: This table reports the inclusion criteria formulated for the electronic database searches
110
Table A1.2 Strategies for Searching the Literature Search of electronic databases (no. of studies) Description
Academic Search Premier (548)
EconLit (348)
ERIC (275)
IDEAS/RePEc (37)
Social Science Citation Index (332)
SocINDEX (77)
Sociological Abstracts (216)
Rapport (1)
Various (4) (Danish National Research Database)
Systematic literature search by VIVE’s Librarian.
The following combinations of search terms are used:
a) ”Cost” and ”benefit”
b) ”Early childhood intervention”, ”education intervention”,
”youth intervention”
In the databases Econlit, Sociological Abstracts and RePEc, we used the following search strings:
ab(benefit OR cost OR cost-benefit OR internal rate OR rate of return)
AND
ab(early childhood OR preschool OR kindergarten OR daycare OR nursery OR child care OR literacy OR language OR education OR K-12 OR school OR youth OR mentor OR vocational training OR training OR preparatory OR parenting OR family OR home-visiting OR center-based)
AND
ab(intervention OR program OR investment OR development OR training)
The search resulted in a total of 1838 studies.
Manual search of websites
Google and Google Scholar Searching for Danish cost-benefit analyses.
Searching for research organisations doing cost-benefit analyses of early childhood interventions (see below).
Searching Google Scholar for studies from the electronic database search to identify similar or related studies.
Search of research centres and organisations
RAND Labour and Population
Center for Benefit-Cost studies in Education (CBCSE)
Society for Benefit-Cost Analysis (SBCA)
Washington State Institute for Public Policy (WSIPP) Benefit-Cost Analysis Database
NBER Working Paper Series
World Bank, Impact Development Blog
J-PAL
Evans School Benefit-Cost Analysis Center Research and Projects
Brookings Institute (see the SGM model; Sawhill et al. 2014)
Searching list of publications and other public available resources (e.g. templates or description of common standards).
Searching for methodological discussions and references (e.g. in policy briefs or blogs).
111
The Heckman Equation Project Database:
www.theheckmanequation.org
MacArthur Foundation Database
Center on the Developing Child at Harvard University
Department of Health and Social Care (England)
CPB Netherlands Bureau for Economic Policy Analysis
Scandinavian:
The national board of health in Denmark (Socialstyrelsen)
The national board of health and welfare in Sweden (Socialstyrelsen)
NUBU – Nasjonalt utviklingssenter for barn og unge (Norway)
The Copenhagen Consensus Center (Denmark)
Footnote chasing
References in Karoly (2008)
References in Aos et al. (2004)
References in Heckman et al. (2010)
References in Belfield et al. (2015)
References in reviewed articles
References in non-reviewed articles
References from scholars who are active in the field
Searching additional references for specific methods and techniques applied in the cost-benefits analyses.
Conferences and workshops
Society for Benefit-Cost Analysis 11th Annual Conference & Meeting
Searching for latest methodological advances and directions in the field.
Applications of cost-benefit analyses in other fields.
Communications with scholars active in the field.
Note: This table describes our search for literature and methodological discussions from various sources.
112