{'fraction_non_alphanumeric': 0.024990760891881902, 'fraction_numerical': 0.00863138011743933, 'mean_word_length': 4.9962574481705815, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Using Household Grants to Benchmark the Cost Effectiveness of a USAID Workforce Readiness Program

We use a randomized experiment to compare a workforce training program to cash transfers in Rwanda. Conducted in a sample of poor and underemployed youth, this study measures the impact of the training program not only relative to a control group but relative to the counterfactual of simply disbursing the cost of the program directly to beneficiaries. While the training program was successful in improving a number of core outcomes (productive hours, assets, savings, and subjective well-being), cost-equivalent cash transfers move all these outcomes as well as consumption, income, and wealth. In the head-to-head costing comparison cash proves superior across a number of economic outcomes, while training outperforms cash only in the production of business knowledge. We find little evidence of complementarity between human and physical capital interventions, and no signs of heterogeneity or spillover effects.


Introduction
The demographic dividend in Sub-Saharan Africa is a double-edged sword. A young population provides an opportunity to benefit from the many productive years ahead while bearing a limited burden of dependency from older generations-but not if young people are unable to find productive employment (Fox et al., 2016). In spite of gains in formal educational attainment, youth unemployment rates remain high; for example while 40 percent of Rwanda's population is between the ages of 14-30, 65 percent of these youth are unemployed. This raises the prospect of both a lost generation of opportunity, and the political risks that accompany a large, unemployed, urban, young population (Bongaarts, 2016). Hence, it is critical to understand the barriers in physical and human capital that prevent youth from being fully productive.
In spite of this pressing need, policymakers have limited access to evidence-based interventions with a track record of effectiveness. This is not to say that active labor-market interventions have not been studied; for example, a recent review discusses nine randomized evaluations from developing countries (McKenzie, 2017). Despite some signs of success in generating employment (Alfonsi et al., 2019;Diaz and Rosas, 2016), the impacts of programs aimed at lifting human capital have been variable and less impressive than hoped in terms of labor and income benefits.
A systematic review by Kluve et al. (2017) finds labor market interventions to have positive effects on employment and income, but these impacts are small and highly variable across studies. At the same time, the costs of relaxing capital constraints are falling due to the widespread availability of mobile money in the developing world. A large literature finds that unconditional cash transfers are invested in durables (Haushofer and Shapiro, 2016), productive assets (Blattman et al., 2018;Gertler et al., 2012), and microenterprises (De Mel et al., 2012), suggesting that cash may be a reasonable alternative in delivering economic livelihood assistance to youth. Given that the literature has long recognized both 'money and ideas' may serve as constraints to the productivity of young entrepreneurs (Giné and Mansuri, 2014), rigorous comparative cost-effectiveness research across these different modalities is sorely needed, as well as a better understanding of potential complementarities between them.
This study addresses these challenges by undertaking an exercise in cash benchmarking: the direct comparison of in-kind-to cash-transfer programs in a single experimental setting. As an applied-science exercise, such a study is a form of comparative cost-effectiveness analysis; it compares the returns to alternative forms of programming on a pre-defined set of outcomes. And it can answer this counterfactual question subject to a distributional constraint, seeking to hold the value of programming per beneficiary constant across modalities. 1 Such cash-benchmarking exercises also inform a basic-science question, by lifting distinct constraints to individual employment outcomes. Similar efforts include Ahmed et al. (2016) who compare BRAC's ultra-poor programming to cash, or Karlan et al. (2014) who examine the comparative impact of the relaxation of credit and risk constraints in agriculture. In the context of youth livelihoods, training programs 1 and cash grants each move alternative potential constraints to productive employment-skills and liquidity, respectively. One way of conceiving of the value of this benchmarking activity is that for any given outcome, our design allows us to cast the opportunity cost of skills improvement in pecuniary terms, despite the fact that these skills cannot be bought on the market. This allows us not only to determine the benefit generated by an increment of skills improvement, but also to calculate the counterfactual cost of generating the same benefit by relaxing financial (rather than human capital) constraints. The inclusion of a combined arm allows us to study complementarities, asking if the returns to relaxing capital constraints improve when human capital constraints have also been relaxed. 2 We study this question using an individually randomized trial with 1,848 underemployed Rwandan youth to understand how a 'standard' package of training, soft skills, and networking interventions compares not only to an experimental control group but to an additional arm that receives household grants intended to be of equal cost to the donor-a cash benchmark. The study follows poor, underemployed youth aged 15-30 who expressed interest in participating in the training program. The core program is called Huguka Dukore/ Akazi Kanoze which means 'Get Trained and Let's Work/Work Well Done' in Kinyarwanda (abbreviated henceforth as HD); it follows USAID's strategy on workforce readiness and skills training and was implemented by Education Development Center, Inc. (EDC). The benchmarking cash transfer program was implemented by GiveDirectly (GD), a US-based nonprofit that specializes in making unconditional household grants via mobile money. USAID also uses cash transfers in its programming in a number of dimensions, so this study compares two different means through which it could attempt to deliver benefits. 3 These two treatments are compared to a control group, namely a set of individuals that receive neither program, and a combined arm that receives both. Our study provides a methodology incorporating randomization of transfer amounts and ex-post regression cost adjustment that can achieve this benchmarking objective in a general way. In a penultimate 'Value for Money' section we show how our study design can be used to conduct either cost equivalence comparisons, or to do cost effectiveness analysis by comparing benefit-cost ratios across arms.
The Huguka Dukore program is a particularly attractive candidate for a benchmarking evaluation. It is a five-year project (2017)(2018)(2019)(2020)(2021) aiming to provide 40,000 vulnerable youth with employability skills in 19 (of 30 total) districts nationwide. Targeting youth from poor households with less than secondary education, with an emphasis on women and youth with disabilities, HD offers multiple program pathways including: i) employment preparation; ii) individual and cooperative youth microenterprise start-up; iii) business development for existing microenterprises. HD is based on a predecessor Akazi Kanoze program, which operated in the country from 2012-2017, and which was evaluated as successful in an RCT led by EDC (Alcid, 2014). It is a carefully designed and intensive training program, and it is backed by rigorous research heading in to this study.
The Government of Rwanda places a high priority on such programs: Priority Area 1 of the "Economic Transformation Pillar" in its seven-year plan for the period 2017-2024 includes the key strategic intervention to "support and empower youth and women to create businesses through entrepreneurship and access to finance" (Republic of Rwanda, 2017, p. 3). Further, training programs of this sort are widespread across the developing world: Blattman and Ralston (2015) estimate that the World Bank alone spends almost a billion dollars annually on skills training programs. In spite of their prevalence, however, the cost-effectiveness of such programs is far from certain. Reviewing evidence on active labor market programs that operate on the supply side of the labor market, McKenzie (2017) finds that employment and earnings impacts are modest, with costs averaging 50 times the monthly income gain. And indeed, in its Future Drivers of Growth report, produced jointly with the the World Bank, the Government of Rwanda raises the possibility that "for a significant portion of the population who will continue creating their own jobs, capitalcentric programs may be more effective and cheaper to implement than simple training programs" (Government of Rwanda and World Bank Group, 2019, p. 81). Our study seeks to resolve this uncertainty by direct comparison.
The momentum for benchmarking has built as numerous studies have shown meaningful impacts of cash transfers on important life outcomes in the short term, such as child nutrition (Aguero et al., 2006;Seidenfeld et al., 2014), schooling (Skoufias et al., 2001), mental health (Baird et al., 2013;Samuels and Stavropoulou, 2016), teen pregnancy and HIV (Baird et al., 2011), microenterprise outcomes (De Mel et al., 2012), consumer durables (Haushofer and Shapiro, 2016), and productive assets (Gertler et al., 2012). The evidence on the long-term impacts of cash transfers is more mixed (Blattman et al., 2018), but some studies have found substantial impacts (Aizer et al., 2016;Balboni et al., 2019;Barham et al., 2014;Fernald et al., 2009;Hoynes et al., 2016). 4 The largest extant literature on benchmarking is based on the comparison of cash aid to food aid (Ahmed et al., 2016;Cunha et al., forthcoming;Hidrobo et al., 2014;Hoddinott et al., 2014;Leroy et al., 2010;Schwab et al., 2013), which has uncovered a fairly consistent result that food aid leads to a larger change in total calories while cash aid leads to an improvement in the diversity of foods consumed. Efforts to benchmark more complex, multi-dimensional programs to cash include BRAC's Targeting the Ultra-Poor program (Chowdhury et al., 2016), microfranchising (Brudevold-Newman et al., 2017), and graduation programs (Sedlmayr et al., 2017).
The randomized controlled trial proceeded in four steps. First, EDC's three local implementing partners within the study ran recruiting workshops drawing more than 2,000 eligible individuals who expressed interest in participating in HD. From this group we then conducted baselines, checked eligibility, and recruited an experimental study sample of 1848 individuals who consented to the lottery and the baseline survey and were included in the randomization. These individuals come from 328 villages in non-urban parts of the three districts of Rwamagana, Muhanga and Nyamagabe. The HD-imposed eligibility criteria for their training of vulnerable youth consist of (a) ages ranging from 16-30, and (b) between 6-12 years of education. Because of the conditions placed on GiveDirectly by the Rwandan government, we further strictly limited eligibility to (c) households registered in Ubudehe poverty status 1 or 2 (the poorest). In addition, in order to provide a study that has compliance rates with the HD training that are as high as possible, we further restricted eligibility to those who (d) expressed interest in participating in the employment and entrepreneurship readiness training. These individuals were recruited at a first 'orientation' meeting at which the local HD implementers and the survey firm (Innovations for Poverty Action, or IPA) recorded sufficient information to enroll them and to subsequently perform baseline surveys at the household (there were no refusals to the household survey). Second, IPA collected baseline data, implementing survey instruments that collected information both at the household level and at the individual beneficiary level for study participants. Third, IPA conducted a series of 13 public lotteries at the sector level, overseen by sector-and local-level officials, at which individuals were assigned to four main arms and a control, to be treated accordingly by implementers. Finally, the study collects baseline and midline (18 month) indicators across a range of economic, psychological, and business-related outcomes to measure comparative impacts.
We costed both programs in detail prior to, and after, the intervention period, following Levin et al. (2017). The ex-ante costing exercise was used to identify the approximate total cost of the HD intervention, as well as the estimated overhead costs to GiveDirectly of providing household grants in this context. The ex-ante costing of HD arrived at a per-beneficiary cost of $464.25. We then randomized transfer amounts at the individual level in the cash arm across four possible transfer amounts. These amounts were chosen to provide informative benefit/cost comparisons across two different margins: HD vs cash, and small versus large cash transfer amounts. Incorporating GiveDirectly's operating costs, the amount actually received by households that generates the same expected cost to USAID as HD is $410.65. The comparison between these two arms therefore provides a straightforward window on expected cost-equivalent impacts. Because we anticipated that the exact numbers from the ex-post costing exercise would differ from the ex-ante exercise, we randomized two bracketing cash transfer arms which transfer $317 and $503 to households. Thus even our smallest transfer is providing individuals with 167% of annual average per capita income.
In the end the HD program turned out to be less expensive than expected; the final ex-post costing figure of $332.27 is used to regression-adjust outcomes across transfer amounts to arrive at a comparison between HD and cash estimated at the exact cost-equivalent amount from the perspective of the donor, USAID. Because of the low costing number, an adjustment which was intended to be an interpolation over GD transfer costs ends up being an extrapolation to a value 16% lower than the smallest GD arm's cost.
The study also features a combined arm that receives both the middle GD transfer amount and HD training. The inclusion of this arm permits a classic test for complementarities between human capital and financial interventions. Finally, the larger cash transfer arm was an amount chosen by the cash implementer as maximizing their own cost-effectiveness (transferring $750, and costing $846.71 which turns out to be almost exactly the cost of the combined arm). The inclusion of this arm provides a statistically high-powered way of examining how benefit/cost ratios shift as the transfer amount rises. This opens up a different type of comparative cost effectiveness question: would the net benefit from cash transfers be maximized by concentrating large payments on a few individuals, or by spreading out smaller transfers to more people? And if more money is to be invested over the basic transfer, should it be in the form of additional physical capital, or is training then more effective?
Our results show that at 18 months after baseline, on average 15 months after being offered HD programming and at least 3 months after any training would have ended, Huguka Dukore has delivered real benefits. While there is no overall improvement in employment rates, the HD arm sees an increase in productive hours, productive asset values more than double, average savings increase by 60%, and subjective well-being is higher. In addition, the HD arm performs a half a standard deviation better on a test of business knowledge built against the course curriculum, showing that the program clears the basic bar of having created real learning.
The cash transfer arm, on average 14 months after transfers took place, sees improvements across a broad range of economic and psychological outcomes. These impacts prove surprisingly invariant to the transfer amount variation present in this study, suggesting that even our lower transfers clear a barrier that generates real benefit to households. With the exception of the improvement that HD generates on business knowledge, cash improves every outcome that HD improves, generally with a greater magnitude, and in addition drives monthly income, household-and individual-level consumption, livestock value, and overall wealth, to higher levels.
Consequently, when we conduct our pre-specified comparative impact analysis, we find costequivalent cash to generate significantly larger benefits for income, productive and livestock assets, individual consumption, and subjective well-being, while HD is more effective at generating the human capital benefit of business knowledge. We find no evidence of complementarity between human and physical capital interventions; if anything the combination appears to do worse than we would expect by adding up the individual impact of the two programs. Significant negative complementarities are present for productive hours of work and for subjective well-being. In summary, then, at the cost of around $330 per beneficiary where the core comparison is done, cash moves a set of economic and psychological outcomes more than HD, HD is better at improving business knowledge, and both of the efforts we engaged in to increase the amount spent above this (whether through more cash or through adding HD) had disappointing returns.
We then look for signs of heterogeneity in impacts across gender, as well as baseline consumption, risk aversion, and local employment rates. Overall we see little meaningful heterogeneity, suggesting that both interventions have a relatively consistent effect across richer and poorer, male and female, and across local labor market conditions. We then exploit the random variation generated by the lotteries in the intensity of treatment at the village level to look for evidence of spillovers. 5 The only evidence we find for spillovers are that compliance with HD, which overall in the sample is 85%, is improved by having more people in your village assigned to HD as well. Looking at our primary outcomes we find no evidence that spillovers from any treatment or to any treatment group are present, suggesting that by this measure at least the study is internally valid.
These results illustrate the complexity of the comparisons created by this type of benchmarked design. Considered relative to the control, HD has been successful in moving some of the key welfare indicators the program is geared towards, and has strongly improved the core metric of learning.
Even in a comparative sense, it is impressive that a program that made no material transfer to its beneficiaries could generate improvements in asset values half as large, and improvements in savings two thirds as large, as a program that gave them hundreds of dollars. Nonetheless when we compare the programs in a head-to-head way it becomes clear that at least at 18 months from baseline, cash is outperforming the training program across a set of indicators likely to be central to beneficiary economic welfare (individual consumption, income, livestock wealth, and subjective well-being). In something of a challenge to ever-more complex bundled programs, we find that each of these interventions has a distinct set of benefits that operate independently, and little is gained by providing them together.
In the remainder of the paper, we provide details of the experimental design, survey structure, and costing exercise in Section 2, and then present the results of the study in Section 3. Section 5 concludes.

Interventions
Huguka Dukore: Employment and entrepreneurship readiness training Huguka Dukore is a five-year activity providing 40,000 vulnerable youth with increased opportunities for wage and self-employment through a suite of interventions that includes market relevant work readiness training, employability skills training, work based learning, internship opportunities, links to employment and entrepreneurship training at the youth level. The program builds on lessons learned from EDCs prior work in this area through the Akazi Kanoze Youth Livelihoods Project (henceforth AK).
Over the life of the project, HD will prepare 21,000 new youth for employment with Rwandan employers, with an additional 2,000 alumni receiving middle management training. It is assisting 13,000 new HD participants to start their own microenterprise, while supporting 4,000 youth (2,000 new and 2,000 AK alumni) with an existing microenterprise to grow their business, linking 15,000 youth to financial services. Finally, HD provides support to its 30 local Implementing Partners (IPs) to improve their job placement rates. It is important to note that the full HD program includes a number of higher-level interventions, including training front-line providers, organizational capacity building in workforce development systems, and facilitating linkages between businesses, government, and local NGOs. Because our intervention studies the cross-individual variation within communities in which HD is working, we measure only the youth-level components of the intervention and not these more systematic dimensions.
The HD program consists of a number of separate modules which are taken serially over the course of a year. The first of these is 'Work Ready Now!', consisting of eight sub-modules (Personal Development, Interpersonal Communication, Work Habits and Conduct, Leadership, Health and Safety at Work, Worker and Employer Rights and Responsibilities, Financial Fitness, and Exploring Entrepreneurship). This module is taken by all students as the lead-in to the HD training, and consists of 10 five-day weeks of full-day training.
From here students choose the additional modules and the sector of work in which they receive additional training, and the curriculum splits according to the nature of formal employment opportunities in local markets. In more urban areas students would then move on to a Technical and Vocational Training (TVET) module, Transition to Work programming, and Work Based Learning Services. Because our study areas are almost exclusively rural, HD instead encourages students to focus on self-employment, meaning that the next module of HD would be the 'Be Your Own Boss' training, which is an entrepreneurship curriculum that is tailored to the specific interests and opportunities in a specific cohort of students, and lasts another 10 weeks. After this point HD students are typically placed in an internship or apprenticeship position with a local entrepreneur working in the selected sector. During this interval students have regular check-ins with their trainers. Within a year of the initiation of training students are considered 'graduates' of HD. 6 Because the curriculum involves several components of choice (whether to pursue vocational or small business training, the sector in which to be trained), our experimental analysis will treat HD as a single intervention of which this choice is an integral component. We provide an analysis of the determinants of participation in various components of the potential HD curriculum.

GiveDirectly: Household grants program
To benchmark the impact of the HD program on cash, we worked with GiveDirectly, a US-based 501(c)3 Non-Profit organization. GiveDirectly specializes in sending mobile money transfers directly to the mobile phones of beneficiary households to provide large-scale household grants in developing countries including Kenya, Uganda, and Rwanda. GiveDirectlys typical model has involved targeting households using mass-scale proxy targeting criteria such as roof quality. GiveDirectly builds an in-country infrastructure that allows them to enroll and make transfers to households while simultaneously validating via calls from a phone bank that transfers have been received by the correct people and in a timely manner. Their typical transfers are large and lump-sum, on the order of $1,000, and the organization provides a programatically relevant counterfactual to standard development aid programs, because it has a scalable business model that would in fact 6 Additional components of the broader HD curriculum include assisting students with access to finance through assistance in the formation of Savings and Internal Lending Communities and access to bank financing, and the use of a job matching resource that maintains a list of open positions and attempts to match graduates to them. These components of HD were not operative in the study districts at the time that we ran this evaluation. be capable of providing transfers to the tens of thousands of households that are served by the HD program.
Since eligibility did not condition on having a cellphone, during the enrollment process individuals who did not themselves own a cell phone provided a number belonging to a trusted family member or friend, and transfers were sent to them through this intermediary. The payments were made to beneficiaries in two installments two months apart, with the first payment comprising 40 percent of the total to be paid to the beneficiary, and the second payment completing the transfer.
After each payment is made, staff in the GiveDirectly call center team in Kigali contact every recipient to verify that payments have been received.
In terms of implementation timing, GD orientation commenced immediately after lotteries to notify youth randomized to receive a household grant and introduce them to the program. The value of household grants was not to be disclosed until the GD Treatment step below. GD Treatment (where transfer values will be disclosed to recipients) did not commence anywhere until the lotteries have been conducted everywhere in the district so as to avoid emphasizing the cash treatment prior to the completion of recruitment.

The Combined Arm
The Combined arm received both treatments (GD Middle plus HD). Both interventions were received at the same time as others in their same sector, meaning that they typically started the HD treatment several months before they would receive the household grant from GD.

Enrollment criteria
A detailed timeline showing the evolution of the study is presented in Figure B.1. The study recruits youth from 13 geographic 'sectors' in the districts of Rwamagana, Muhanga and Nyamagabe. 7 Study participants had to be eligible for Huguka Dukore, to attend an informational session about Huguka Dukore, to enroll in a lottery to determine participation in that program following that informational setting, and to be traceable to a residence in a village in the sector where they were recruited. Attendance in person at the public lottery was not required for program enrollment.
The study enrolled in its sample all individuals who met criteria for treatment by Huguka Dukore in the study sectors. More detail on sample recruitment and the conduct of the lotteries is provided in the Appendix. Table A.1 shows the process by which we moved from the original oversubscription universe to the final sample of 1848 individuals deemed as fully eligible who were recruited into the study and randomized. Of 2,275 individuals who attended an orientation meeting and signed up for HD, 1,967 were found based on administrative review to meet the eligibility criteria. A further 119 could not be located either in the village of their stated residence, or were found to be resident outside the sector entirely, and consequently were deemed ineligible for intervention and the study.
There were no survey refusals at baseline, so our study sample reflects the full population of individuals who were assigned to treatments. The final study sample therefore consists of the universe of all individuals who met the enrollment criteria for Huguka Dukore, who attended an information session; who agreed at that information session to be included in the assignment lottery; who were found resident in the relevant sector at baseline.
Demographic and employment characteristics (the latter of which will be defined in greater detail in Section 2.7 below) of the study participants are detailed in Table 2. Consistent with Huguka Dukore's 'soft' targeting criteria, the sample is 59 percent female, with an average age of 23.5, (among the random sample assigned to control). They have an average of 7.6 years of education, and typically live in households of approximately five individuals.
Although Huguka Dukore seeks to bolster employment opportunities for underemployed youth, it does not employ a hard criterion regarding employment for eligibility. Consequently, it is not unusual for individuals to report that they are employed: 33 percent of (control-group) respondents reported being employed at baseline, using a definition that excludes agricultural work on a farm belonging to their own household (see Section 2.7 for more details). By endline the employment rate had risen to 48% in the control group.
Nonetheless, individuals in the study population are quite poor. 32 percent reside in households that the Government of Rwanda categorizes as Ubudehe I-its lowest socio-economic category, denoting a condition of 'extreme poverty'. Median consumption per adult equivalent is 5,879 RWF per month, which in 2018 PPP terms translates to a consumption level of USD 0.66 per day.

Assignment protocol
The allocation of these study households to treatment was undertaken on a randomized basis across eligible, interested individuals using a public lottery. A public lottery was selected as the assignment mechanism given the very large sums of money being transferred and the desire by all parties to the research to ensure that the assignment was considered to be fair and impartial by the research subjects.
Lotteries were conducted at the sector level in each of the 13 sectors in the study, and the proportions assigned to each treatment were fixed at each lottery. This results in a fairly standard 'blocked' randomization structure across the 13 blocks in the study. Participants drew their own treatment status as tokens of different colors from a sack, where each token corresponded to a given treatment arm and the number of tokens in the hat was determined by IPA according to the number of participants with fixed proportions assigned to each treatment.
The detailed protocol for the lottery is as follows: 1. Beneficiaries did not have to be physically present at the lottery to be included in the study.
2. We explicitly recognized the right of EDC/HD to eliminate from eligibility any individuals who they feel, for whatever reason, was not serious about the program and that they did not believe will fully enroll in HD if selected. 3. Detailed information about GD was not provided prior to the lottery, but GD was described in detail at the lottery and every effort was made to preserve the separate identity of HD and GD so as not to provoke confusion about the broader HD program. All information provided at the lottery was given to everyone, and there was not an attempt to separate groups and give private information. 4. A representative of both GD and HD (or its local partner) were present at every lottery. 5. Individuals were notified whether they have been assigned to the GD, the combined arm or the HD arm at the time of the lottery.
6. Individuals assigned to GD received a variety of colors which correspond to different transfer amounts. This means that the random assignment to GiveDirectly simultaneously randomly assigned individuals to the different transfer size amounts. The exact financial amounts were not discussed at the time of the lotteries. GD explained that youth randomized to GD would be contacted soon after the lottery to orient them to the program, and visited at their place of residence to undertake the enrollment process. Table 1 shows the outcome of the lottery process, giving the number of individuals assigned to each of the treatment arms within each lottery, as well as overall.
The assignment of individuals to the main study arms was as follows: Household grants were randomized at the individual level over four transfer amounts. The value of the first transfer amount was made equivalent to the total cost of providing HD to each beneficiary, which is $452.47. Less GD's own associated costs of delivery, this means that an amount of $410.19 was actually transferred to households in this arm to make them cost-equivalent to USAID. Because we did not know the true per-capita cost of HD with certainty beforehand, we randomize GD transfer amounts to two additional values that bracket this expected cost. The bracketing amounts are derived by supposing that the number of beneficiaries for the year two tranche of HD funding nationwide might vary between 8,000 and 12,000 beneficiaries, meaning that the per-capita cost would vary between $377.05 and $565.58. Again netting out GD's costs of making transfers, that means that households in these arms actually receive $317.34 and $503.04, respectively (note that because we costed each GD transfer amount separately and because many of GD's costs are fixed at the individual level, the fraction of total cost that is operating cost declines as the transfer amount increases). The fourth transfer amount was designed to maximize the benefit-cost ratio of household grants, and transferred $750 to beneficiaries. Note : This table gives the number of study individuals assigned to each treatment arm in each of the 13 sectors within which lotteries were conducted. The lotteries were blocked so that fixed fractions of individuals are assigned to each arm.
In the first phase of lotteries, comprising 792 study participants-we randomized purely at the individual level, as the study design did not anticipate multiple enrollees from the same household.
In fact, the 792 participants in the first tranche of lotteries comprised 732 unique households.
This resulted 34 households in which individuals in the same household were assigned to different treatments (at the level of the major arms of the study). Having recognized this issue, we altered the protocol in the second phase of lotteries and assigned treatment at the household level. To reflect this issue we cluster standard errors at the household level.
Given the public nature of the lottery assignment, the study was not blinded either to participants or to the survey firm. The study is not a pipeline design, and to avoid expectancy biases we made it clear to the subjects at the time of the lottery that there would be no subsequent treatment by these implementers in the area.

Program Participation
Compliance with GiveDirectly treatment was nearly perfect. One individual in the middle GD arm was found to be ineligible by Ubudehe status and was not treated, and one individual assigned to the lower GD arm actually received the upper GD treatment. For GD the ITT is therefore effectively the average treatment effect.
As anticipated, HD was most successful in achieving participation in its initial 10-week training (Work Ready Now). 86% of the full HD treatment group (both HD-only and combined arms) were counted as enrolled according to the contractual definition (attending the end of the first week of WRN training). This is the rate that the costing exercise uses since it alone determines the amount paid from USAID to the local implementing partner. But we can use institutional data from the HD program to examine participation in more detail.
Retention during the course of WRN is high; 79% of the overall sample completes this 10week training program, which focuses on general workforce readiness. 8 69% of the of the HD sample complete the Build Your Own Business class (which is focused on entrepreneurship and selfemployment); 13 individuals who did not take WRN did then go on to enroll in BYOB. Finally, the technical training component of the HD intervention provides focused vocational instruction in a specific sector. 48% completed the Technical Training component of the program. In the combined arm, participation with each of these components is about 5 pp higher than in the HD-only arm.
Again, participation in Technical Training is not strictly confined to individuals who participated in any of the previous combinations of treatments. 9 To understand the factors that determine selection into the various components of HD, we regress participation in each component of the program on a battery of baseline characteristics, pooling the entire HD treatment group (HD only and combined). The results are presented Table  A.3. In general compliance is relatively similar across observed beneficiary characteristics; older individuals are slightly more likely to complete the entrepreneurship training but not technical training. There is a modest, negative association between working more at baseline and the likelihood of completing each stage of training; each additional hour of productive time use at baseline is associated with a reduction of approximately 0.4 percentage points in completion at all stages.
Conversely, higher debt stocks at baseline are associated with greater completion rates. Both of these effects appear to be primarily driven by initial compliance, with measured attributes having little ability to predict subsequent compositional changes.

Instruments
Because we were interested in understanding both individual-level and household-level impacts, we used two distinct instruments within each round of data collection. A household survey was administered to the household head, and a beneficiary survey was administered to the beneficiary.
For beneficiaries who lived on their own or who headed their own household, these instruments coincided.
We provide an overview of the contents of each instrument in Table A In addition to this midline, we also intend to return to the field 36 months after baseline to conduct a longer-term follow-up survey, providing an eventual window into longer-term impacts.
Data collected then will be used to revisit the longer-term evolution of these interventions on the lives of the beneficiaries.

Attrition
We attempted to follow up with all study beneficiaries 18 months after baseline. The tracking protocol for the post-treatment round was designed around the individual beneficiary, following him or her to whatever the relevant household was at that time (rather than tracking the baseline household). The interventions studied in this trial have the possibility of inducing migration; consequently it was particularly important to have a strategy to address attrition. Our tracking strategy proceeded in two phases. First, we attempted to track all individuals who were still residing in any study district or in Kigali. Once we had completed this exercise we were left with 122 baseline individuals who we had not yet found. We then randomly sampled half of these individuals (blocking on treatment status), and began an 'intensive tracking' phase that spent substantial resources to track them wherever they had gone, including migrating out of the country, and survey them. This exercise resulted in IPA finding and surveying all 60 living beneficiaries in the intensive tracking sample (one had passed away). Given this remarkable rate of contact, we have an unusual situation where we should be able to convincingly correct for attrition by simply giving the intensive tracking sample weights of 2.

13
To verify that the data weighted in this way recovers the missing potential outcomes, we should establish whether the intensive sample that we drew was representative of the universe of early attritors. We can analyze this by a balance test of the intensive tracking sampling across the baseline outcome for all the early attritors. The sample for this is small (122) but in Table A.4 we find no evidence of systematic problems with this sampling (2 outcomes out of 20 unbalanced with pvalues below 0.10 prior to correction for the False Discovery Rate or FDR, and none significant once we have corrected). These two pieces of evidence-representative sampling in intensive tracking and near-perfect tracking rate-suggest that we have an endline sample that is uniquely representative of the randomized universe.

Balance
The next step is then to establish whether the attrited and reweighted sample used for analysis was balanced at baseline. To ask this question, we estimate a balance table using baseline outcomes but only for the attrited endline sample, and with the weights, blocking, and clustering used in the endline analysis. This makes the balance test mimic the impact analysis we will run as closely as possible; these results are presented in Table 2. The experiment appears well balanced (note that this is also the case if we simply use the full unweighted baseline sample), with rates of rejection consistent with random noise and none of the joint F-tests of all treatments indicating imbalance.
We therefore proceed to the analysis of impacts with confidence that the study is internally valid.

Costing at Scale
The costing exercise in the study utilized the 'ingredients method' (for more discussion, see Dhaliwal and Tulloch, 2012;Levin and McEwan, 2001;Levin et al., 2017;Walls et al., 2019). The policy question is asked from the perspective of the donor (in this case, USAID): the policy objective is to achieve the highest benefit-cost ratio per intended beneficiary for each dollar that is spent on a program. Operating expenditures in the implementation chain are an inherent part of these costs, and so the lower transactions costs in getting mobile money to the beneficiary play an important role in their potential attractiveness. We conducted two different costing exercises at two moments in time. The ex-ante exercise, which was based on projected budgets and staffing costs, was used to predict the cost at the time of the study design, and to choose the ranges over which the lower GiveDirectly transfer amounts would be randomized. Then, a rigorous ex-post costing exercise was conducted for both programs after the fact, using actual budgets and expenditures.
Since the HD program is eventually to cover twenty-three districts (e.g. much larger than the study population only) we attempt to cost the full national program (not just the study sample), inclusive of all direct costs, all indirect in-country management costs including transport, real estate, utilities, and the staffing required to manage the program, and all international operating costs entailed in managing the HD program. Because we do not want differences in scale to drive 14 Notes: Table presents control means and standard deviations; regression coefficients and standard errors for associated comparisons, and p-value for a test of the hypothesis that all arms pool. Regression-based comparisons and associated hypothesis tests based on a regression with block indicators. * * * , * * , and * denote statistical significance at the 1, 5, and 10 percent levels, respectively. All continuous variables winsorized at top and bottom 1 percent. Inverse hyperbolic sine transformation taken for monthly income, household consumption, beneficiary expenditure, savings, debt, and wealth variables.
differential costs per beneficiary, we asked GiveDirectly to artificially scale up their operations and provide us with numbers reflecting the costs per beneficiary if they were running a national-scale program across eight districts, including 40,000 beneficiary households like HD. This is the relevant scale for a USAID program officer contemplating commissioning a program to move the outcomes studied. Beneficiary identification costs, incurred partly by the survey firm and partly by HD, are calculated on a per-head basis and added to the costs of both implementers equally. 10 We costed each GD arm separately, asking what the operating costs would have been if GD had run a national program at the scale of HD giving only transfers of that amount. Operating costs as a percentage of the amount transferred decline with transfer amount for GD because fixed costs represent a large share of their total costs. This allows us to conduct the benefit/cost comparisons at scale, rather than having the artificial, multi-amount environment of the study contaminate the costing exercise across arms.

Differential Compliance
Given that the Intention-to-Treat is the heart of the experimental analysis, we construct the 'cost per study subject' that corresponds to the spend on the sample over which the ITT is estimated.
The raw costing returns the cost per beneficiary, but less than this is spent per study subject to deliver the ITT if compliance is less than 1. Both implementers face a relatively simple relationship between cost and compliance. For GD, individuals not treated cost nothing. Similarly for HD, their rules stipulate that they pay sub-IPs a fixed amount based on enrollment at the end of the first of WRN training to then follow through and offer all appropriate subsequent classes in the curriculum.
These costs are almost exclusively based on offering the courses and do not scale sharply with class size. Hence, we consider all costs as 'averted' for non-compliers, and for each arm we calculate the ITT-comparable cost by multiplying the compliance rate times the cost per beneficiary. Table 3 shows the evolution of the costing analysis. As described above, the ex-ante costing exercise arrived at a figure of $464.25 per HD beneficiary, with bracketing costs of $377.03 and $571.74. GD took this number and applied their cost structure to it for a program scaled to 40,000 beneficiaries, and arrived at an ex-ante cost-equivalent transfer of $410.65 to be actually delivered to beneficiaries.

Final Costing Numbers
The bracketing cash arms received $317 and the upper arm $503, and the 'huge' arm received $750, the amount that GD believed would maximize benefit-cost.
Then, based on the ex-post costing exercise, we recalculate USAID costs applying the more accurate costing figures to the sums actually transferred. These figures show that HD was less expensive than anticipated, and GD operating costs were slightly higher than anticipated. This means that the effect USAID spend per beneficiary was only $388.32, while the spending for the per study individual, an amount similar to the GD large arm. These are the numbers used in the Cost Equivalent table. In sum, our study ends up with even the smallest of the GD cost-equivalent arms transferring somewhat too much to be directly comparable to HD, but the GD Large arm providing a very close cost counterfactual to the Combined arm. 11 It is important to remember in looking at our results, then that the GD arms cost more than HD, and only through the linearity assumption in our cost-equivalence comparison can we recover the exact benchmarking amount. An implication for future work is that using wider brackets may be reasonable given the considerable uncertainty we have uncovered moving from ex-ante to ex-post cost estimates. Note: The first column shows the ex-ante costing data on which study was designed; the core number is the HD cost around which the GD actual transfer amounts in column 2 were designed. Column 3 shows the results of the ex post costing exercise. Column 4 provides the share of spending that did not reach the beneficiaries either in cash or in direct training and materials costs. Column 5 shows the compliance rates, and since all costs are averted for non-compliers then the final column shows the final cost per study subject for each arm that are the basis of the cost-equivalent comparisons.

Overall ITT Impacts
The data from the study are analyzed consistent with the design being a multi-arm, householdrandomized program. Let the subscript i indicate the individual, h the household, and b the randomization block (lottery groups within which the randomization was conducted). For outcomes observed both at baseline (Y ihb0 ) and at endline (Y ihb1 ), we conduct ANCOVA analysis including the baseline outcome, fixed effects for the sector-level assignment blocks within which the randomization was conducted µ b , as well as a set of baseline control variables selected from the baseline data on the basis of their ability to predict the primary outcomes, denoted by X ihb0 . Base regressions to estimate the Intention to Treat Effect include indicators for the HD treatment T HD ihb , a vector of indicators for each of the three GD 'small' treatment values, T GDS1 ihb , T GDS2 ihb , and T GDS3 ihb , an indicator for the GD 'large' treatment T GDL ihb , and an indicator for the combined arm Block-level fixed effects,µ b , are included to account for the block randomization of the study. Standard errors will be clustered at the household level because the second tranche of treatment was assigned at the household level. Following the 'post-double-LASSO' procedure of Belloni et al.
(2014b), a set of covariates were selected using a LASSO algorithm on the control data; further details of this procedure are provided in Appendix C. For outcomes that are collected at endline only, we cannot include the lagged outcome to run the ANCOVA regression, and so use the simple cross-sectional analog to Equation (1).
To mitigate risks of false discovery across multiple outcomes and treatments, we report Anderson's 2008 False Discovery Rate to adjust p-values within each of the four relevant families (primary outcomes and the three families of secondary outcomes outlined in Section D.2.2), ensuring that the false discovery rate at the family level is controlled at five percent. This follows the procedures described in our Pre-Analaysis Plan (PAP).
Tables 4 and 5 present the results of this analysis, with the five primary outcomes in the rows of the table, and in the columns we include the core treatment arms of the study: HD, each of the three smaller GD arms that were designed to be cost equivalent, the large cash transfer arm (GD Large), and the arm that receives both the medium GD cash transfer amount and HD (Combined).
Appendix Tables A.5 and A.6 present the more parsimonious specification that pools the three smaller GD transfer amounts into one arm.
Because each treatment is measured with a dummy variable the outcomes here should be inter-  Note: The six columns of the table provide the estimate on dummy variables for each of the treatment arms, compared to the control group. The five primary outcomes are in rows. Regressions include but do not report the lagged dependent variable, fixed effects for randomization blocks, and a set of LASSO-selected baseline covariates, and are weighted to reflect intensive tracking. Standard errors are (in soft brackets) are clustered at the household level to reflect the design effect, and p-values corrected for False Discovery Rates across all the outcomes in the table are presented in hard brackets. Stars on coefficient estimates are derived from the FDR-corrected p-values, *=10%, **=5%, and ***=1% significance. Reported p-values in final three columns derived from F -tests of hypotheses that cost-benefit ratios are equal between: (a) GD Lower and HD; (b) GD Lower and GD Large; and (c) GD Large and Combined treatments. Employed is a dummy variable for spending more than 10 hours per week working for a wage or as primary operator of a microenterprise. Productive hours are measured over prior 7 days in all activities other than own-farm agriculture. Monthly income, productive assets, and household consumption are winsorized at 1% and 99% and analyzed in Inverse Hyperbolic Sine, meaning that treatment effects can be interpreted as percent changes. Beginning with Table 4, we see that none of the programs were successful in driving the core outcome of employment rates. Relative to a control group employment rate of 48%, neither HD, cash, nor the combination saw improvements of more than 5 percentage points. Given that the minimum detectable effect of our study (which can be calculated by multiplying the standard error times 1.96) was between 6 and 10 pp, these modest improvements are not close to being significant.
Moving to the continuous measure of the number of productive hours worked in a week, we see more promising impacts. Here, HD was successful in driving a 3 hour increase off a base of 18.4, an improvement of 16%. Over the 18 months since baseline the trend in the control group shows substantial increases in employment rates (from 33 to 48%) and productive hours (from 11 to 19 hours per week), and treatment effects relative to this large secular improvement are modest. 12 The GD transfers appear to have had a non-linear effect on productive time use; the middle transfer has a strong effect (an increase of 6.5 hours per week) but both smaller and larger transfers are insignificant. As the transfer amount increases the productive labor increase disappears, with the Large arm achieving an insignificant 1.1 hour improvement. This is the first evidence of an apparent non-monotonicity in the impact of cash, suggesting that moderate transfers induce a complementary increase in labor inputs, but once transfers become sufficiently large they begin to shift the opportunity cost of leisure, discouraging labor inputs. The Combined arm, far from showing signs of a complementarity between the two programs, suggests that when individuals receive both cash and training there is no corresponding improvement in labor inputs, while had they received either alone this benefit would have been observed. In the following subsection we provide a more detailed analysis of the employment impacts of the different programs.
Next we consider the primary household outcomes of monthly income, productive assets, and consumption per capita. All of these outcomes are measured as the inverse hyperbolic sine, meaning that impacts can be interpreted as percent changes (like a log, but capable of handling zeroes). 13 Beginning with the impact of HD, we see a 31% increase in income (not significant), no meaningful change in consumption, but an impressive and highly significant surge in productive assets; these rise to be 151% higher than the control group mean. For a program that has made no material transfers to the beneficiaries, this improvement is substantial and impressive. When we look at the impact of cash transfers, we replicate the results of many other studies, showing that cash can lead to substantial improvements in these indicators. Measuring outcomes roughly 14 months after the transfers were made, we find the GD treatments leading to a doubling of monthly income, again non-monotonic with intermediate transfers appearing to be most effective. There is a quadrupling of productive asset values, and household consumption increases by 20-36 percent. For the Large arm (whose value received is 80% larger than the GD Middle arm) we see outcomes that are in general very similar to the smaller arms; slightly smaller impacts on income, essentially identical impacts on productive asset values, and slightly larger effects on consumption. So again, these results are consistent with the idea that the Large arm induces a smaller productive effect than the Main arm leading to less earned income (see penultimate column of Table 4  Moving then to the secondary outcomes of the study, in Table 5 we present three families of outcomes: beneficiary welfare, household wealth, and cognitive and non-cognitive skill development.
Beginning with beneficiary welfare, we have two ways of measuring mental health. The first of these is a composite of the answers to two simple Likert-scaled questions about subjective well-being, one on happiness and one on life satisfaction. For this outcome, we see every arm improving subjective well-being, with effects for any arm receiving cash that are more than twice as large as the HD arm.
Our second measure of mental health is built around reporting on a set of potential mental health issues over the past two weeks, including stress, ability to concentrate, losing sleep, confidence, and feelings of worthlessness. Interestingly, none of the interventions improve this measure, suggesting that we are seeing improvements in general life satisfaction but no decrease in specific negative symptoms of poor mental health. Our final measure of beneficiary welfare is the inverse hyperbolic sine of the consumption of the specific beneficiary (as opposed to the primary outcome of household consumption). Here we see no improvements under HD, and increases of approximately 50% over the control outcome for any arm that receives cash (again larger for the middle-sized transfers).
Next we examine measures of household wealth. Perhaps unsurprisingly, cash transfers move these outcomes powerfully. Once again we see an impact that appears to be relatively homogeneous and somewhat non-monotonic in all arms that receive cash transfers, with net non-land wealth more than doubling in all arms but the smallest, livestock wealth tripling, and savings doubling. In Figure   B. show that any arm who received HD (HD or Combined) has about a half of a standard deviation improvement in their performance on this test, a very sizeable effect. GD does not move this score at all. The business attitudes measure we use captures attitudes toward entrepreneurship; while none of the interventions significantly improve it they all lead to an increase of about 0.1 SD, and prior to the correction for false discovery the smaller GD arms (which saw a surge in productive hours) had an effect on this measure significant at the 10% level. The improvement in business knowledge for HD participants does not translate into an increase in entrepreneurship.
In Tables A.5 and A.6 we present a more parsimonious specification that pools the three smaller GD arms into one; this analysis was pre-specified due to a concern about a potential lack of power when the smaller GD arms are analyzed separately. In the event we have many significant results for the disaggregated analysis, indicating that power is not a concern, and so the specification pooling these arms simply provides an overview of the average effects across these smaller arms.

Detailed Analysis of Employment Effects
Given the central importance of employment in this study, we now seek to dig deeper into the differential impact of the programs. More nuance can be provided for these results in a number In Appendix Table A.7 we present a more detailed analysis using the five disaggregated categories in which we asked the underlying time use questions. These report the hours beneficiaries worked in the previous week in agricultural wage labor, non-agricultural wage labor, nonagricultural enterprises, agricultural self-employment, or agricultural processing and trading. Interestingly, the HD treatment induces an increase in non-agricultural wage labor of an estimated 6 percentage points (against a control-arm counterfactual of 30 percent), as well as a modest increase in agriculture-related self-employment, while the impacts of cash transfers are concentrated in non-agricultural enterprise and self-employment labor, at the expense of agricultural (and, at high transfer values, other forms of) wage labor.
Next, we delve into the source of the dissonance between our binary measure of employment, which shows no impact, and the continuous measure of productive hours, which does. The obvious suspect here is the specific threshold used to define 'employed'. To examine this, we vary this threshold continuously from 5 hours per week to 40 hours per week, and present a visualization of employment rates across thresholds and across treatment arms. We use the disaggregated labor categories analyzed in Table A.7, which are presented in the Appendix in decreasing order of their importance to total employment. First, Figure B.4 presents effects on non-agricultural wage labor (the category with the most overall time use). In general, the treatments effects represent similar vertical shifts across all the employment thresholds, implying that the binary estimate of employment effects would not be sensitive to threshold. HD exerts a positive effect, and GD Large a negative effect, on wage employment. Non-agricultural self employment, as shown in Figure B.5, responds little to HD, and to cash amounts in a relatively monotonic way (bigger transfers mean bigger impacts). Agricultural wage employment, which is both arduous and lowpaid, appears to demonstrate an income effect in being depressed by any kind of cash transfer ( Figure B.6). 15 Agricultural self-employment is increased by all the treatments, and it is here that the non-monotonicity in transfer amounts is strongest, with GD Main seeing a larger effect than GD Large ( Figure B.7). Finally, agricultural processing and trading appears to be a sector that HD encourages individuals to leave, helping to explain smaller total productive hours effects despite the surge seen in non-ag wage employment and agricultural self-employment ( Figure B.8). Given the general invariance of the impacts to the thresholds used, this suggests that the dissonance arises from the fact that the treatment effects on productive hours are occurring among those who would anyways have been counted as employed.
Finally, we can exploit a different source of data, which is a series of phone calls made by IPA to all of the study participants over the period from November 2017 (around the end of the HD intervention) to May 2019 (a few months before the midline). The core purpose of these phone calls was to acquire tracking information for the in-person survey, but we block randomized the month in which we called each individual, and during the call asked the same time use questions that would be included in the midline. This survey was successful in reaching 1,797 of our study subjects (97%), and given the randomization by month provides a clean way of estimating the time path of treatment effects. In Figure  Summarizing this analysis, we see reasonable labor market effects of the HD program in a number of dimensions. We would not expect productive time use to improve during the intervention, and our midline survey appears to come about six months after the labor market impacts emerge.
HD focuses on job training and on helping beneficiaries start agricultural enterprises, and it is successful in both these endeavors. Because most of the individuals entering this work are leaving agricultural wage employment or agricultural trading (which we also count as employment), the headline employment rate does not shift substantially. Cash transfers, on the other hand, particularly strongly drive individuals away from agricultural wage work and into self-employment (both agricultural and non-agricultural). So these two interventions have the effect of pushing participants down somewhat different paths towards income generation.

Cost-Equivalent Benchmarking
We pre-specified a regression adjustment strategy for benchmarking HD at an exactly cost-equivalent level using the ex-post costing data from both programs. First, begin with the total GD donor cost per study subject within each transfer amount arm, denoted by t c (this is the final column in Table   3). Subtract from this number the benchmarked HD cost per household C from the same column, and denote the difference t c − C = τ c ; this is the deviation (positive or negative) of each GD arm from the benchmarked HD cost. Set τ c to zero in the control and HD arms. We can then re-run regression (1) above omitting the combined arm, and controlling for a linear term in τ c , a dummy for either treatment, and a dummy for receiving HD: In this specification T ihb is a dummy variable indicating that individual i in household h of randomization block b was assigned to any treatment (HD or GD). Subject to the assumption of linear transfer amount effects, the slope coefficient τ c captures impacts arising from deviations in GD cost from HD cost, the coefficient δ effectively gives the impact of GD at the cost of HD, and the dummy variable δ HD provides a direct benchmarking test: the differential impact of HD over GD at the same cost per eligible. We impose the simple linear functional form to preserve as much statistical power as possible for the core cost-equivalent benchmarking comparison, although it is straightforward to make this more flexible. Table 6 shows the cost-equivalent benchmarking results for primary outcomes, and Table 7 for secondaries. Beginning with the third column we linearize across transfer arms and test the marginal effect of receiving an additional $100 from GD. Remarkably, across all 17 outcomes we do not have a single case where we can detect a significant slope across transfer amounts; indeed 26 more than half of the transfer slope coefficients are negative. 16 The takeaway seems to be that this study features transfer amounts large enough to have cleared an impact threshold beyond which additional amounts of money do not lead to meaningfully better outcomes. For the purposes of cost adjustment, this result also indicates that the pre-specified way of using a linear transfer amount control to adjust the GD impacts will not have a strong effect on the predicted cost-equivalent outcome.
The second column, Cost-equivalent GD impact, reports the coefficient on an indicator for receiving any treatment. This estimates the effect of GD at the cost-equivalent level; here we see as significant the outcomes already shown to be moved by GD (hours worked, income, productive assets, consumption, subjective well-being, livestock wealth, and savings). In this pooled specification and with regression adjustment we now get a small effect of cost-equivalent cash on business attitudes, significant at the 10% level.
The first column, Differential impact of HD, presents the core of the comparative cost effectiveness exercise that this study was built to conduct. Using the linearized cost adjustment it gives the differential impact of HD relative to a cash transfer of precisely equal amount. The results indicate that HD leads to somewhat lower monthly income and substantially fewer productive assets than a cost-equivalent cash transfer. In terms of secondary outcomes, HD also does significantly worse at driving subjective well-being, beneficiary consumption, and livestock wealth, and of all the outcomes in the study only for the Business Knowledge index does HD do significantly better than cash.
Appendix Figure B.11 provides graphical interpretation for our cost equivalent comparison, using subjective well-being and business knowledge as outcomes. In this figure the grey circle plots the Control in outcome/cost space, the four black circles represent the GD arms, and the black diamond gives the actual HD outcome. The cost adjustment strategy first fits a linear regression of the outcome by transfer amount in the GD arms (solid black line), then extrapolates this line to the predicted value that would have obtained had the GD transfers exactly equalled the ex-post HD cost (hollow circle). Mapping the picture to the regression estimands, then, the third column gives the slope of the line, the second column gives the vertical differential between the predicted hollow circle and the control, and the core cost equivalent test measures the vertical difference between the hollow circle and the black diamond. This last quantity is the core purpose of our comparative exercise, the difference between what we actually observed in HD and what we predict we would have observed in GD at the exact same cost. For subjective well-being, despite an improvement in HD over the control we see the predicted cash outcome being more than twice as a far above the control. For business knowledge, it is clear that no amount of money generates an effect resembling HD. Similar figures for all primary outcomes, along with a side-by-side comparison to a more standard cost-effectiveness approach, are included in Figure 2 and discussed in Section 4.
These results illustrate the value of the double counterfactual created by the cash benchmarking design. Compared to what would have happened in the absence of the program HD is successful, leading to meaningful improvements in a number of the core outcomes it was designed to move.
Considering the cost of the program, however, and comparing to what would have happened if this cost had been distributed directly to beneficiaries, the picture is less rosy. Given that the direct distribution of these costs would have led to a surge in consumption and investment, the hurdle for success is raised, and we find the HD program falling short across most outcomes. Unless policymakers had a strong preference for the specific human capital built by HD and measured in our Business knowledge index, over the 18 month time horizon the benefits of cash would dominate. It is nonetheless impressive that HD managed to generate meaningful improvements in productive assets, savings, and subjective well-being without having made any material transfers to beneficiaries. Note: This table uses a linear adjustment of primary outcomes for program cost to compare HD and GD at exactly equivalent costs. The Transfer value column estimates the marginal effect of spending an extra $100 through cash transfers. The Cost-equivalent GD impact column is estimated as a dummy for either HD or GD treatment, and estimates the impact of cash at the exact cost of HD. The Differential impact of HD column then estimates the differential effect of HD above cash at this benchmarked cost. Regressions include but do not report the lagged dependent variable, fixed effects for randomization blocks, and a set of LASSO-selected baseline covariates, and are weighted to reflect intensive tracking. Standard errors are (in soft brackets) are clustered at the household level to reflect the design effect, and p-values corrected for False Discovery Rates across all the outcomes in the table are presented in hard brackets. Stars on coefficient estimates are derived from the FDR-corrected p-values, *=10%, **=5%, and ***=1% significance. Employed is a dummy variable for spending more than 10 hours per week working for a wage or as primary operator of a microenterprise. Productive hours are measured over prior 7 days in all activities other than own-farm agriculture. Monthly income, productive assets, and household consumption are winsorized at 1% and 99% and analyzed in Inverse Hyperbolic Sine, meaning that treatment effects can be interpreted as percent changes. Note: This table uses a linear adjustment of secondary outcomes for program cost to compare HD and GD at exactly equivalent costs. The Transfer value column estimates the marginal effect of spending an extra $100 through cash transfers. The Cost-equivalent GD impact column is estimated as a dummy for either HD or GD treatment, and estimates the impact of cash at the exact cost of HD. The Differential impact of HD column then estimates the differential effect of HD above cash at this benchmarked cost. Regressions include but do not report the lagged dependent variable, fixed effects for randomization blocks, and a set of LASSO-selected baseline covariates, and are weighted to reflect intensive tracking. Standard errors are (in soft brackets) are clustered at the household level to reflect the design effect, and p-values corrected for False Discovery Rates across all outcomes within each family are presented in hard brackets. Stars on coefficient estimates are derived from the FDR-corrected p-values, *=10%, **=5%, and ***=1% significance.

Complementarities
Next, we present a canonical statistical analysis of complementarities. To do this, we use only the control, the HD arm, the middle GD transfer, and the combined arm (who received HD and the middle GD transfer). This sets up a standard 2x2 design that cross-cuts the two treatments.
Secondly, we redefine the treatment dummies so that we include one control for 'any HD' (HD or combined), one for 'any GD' (GD or combined), and a dummy for the combined arm. Using this approach, the 'combined' arm dummy now measures not the difference relative to the control but instead whether there is an additional impact from the combination of HD and GD that is greater than what would be expected by adding together the independent HD and GD effects. It is therefore a direct test of complementarities; whether the whole of the combined arm is something different than the sum of the two parts.
Across the board, the evidence in Tables 8 and 9 suggests that far from finding positive complementarities, the whole appears to be less than the sum of the parts. The sign on the core test (column 3) is negative for all primary outcomes and for two thirds of secondary outcomes. The complementarity is significantly negative for productive hours and subjective well-being.
Complex, multi-dimensional programs are often justified on the grounds that poverty presents a range of constraints, meaning that individuals are unable to benefit unless more than one constraint is relaxed at once. Here we find that the effort to reduce human capital and physical capital barriers simultaneously generates no additional benefit. Outcomes that are driven by cash are not further helped by HD, outcomes driven by HD are not further helped by cash, and for two of our key outcomes the combination is actually worse than what we would expect from the independent effect of the two programs. On the other hand, by chance we have a circumstance where the cost of the Combined arm is very similar to the GD Large arm, and we also do not find evidence that the increase in cash transfer amounts from the Main to the Large transfer amounts is worthwhile.
The combined message is then that both efforts to increase expenditure per household from $300 to $750 were not justified, and more moderate expenditure amounts would dominate in a standard cost-benefit sense. Notes: HD, GD indicators defined as taking value of one if individual is in either the corresponding arm or the combined arm. Complementarity column reports the differential effect of being in the combined arm, compared to the sum of HD and GD impacts. Among individuals assigned to GD, only the mid-sized transfer arm is included in this analysis. Regressions include but do not report the lagged dependent variable, fixed effects for randomization blocks, and a set of LASSO-selected baseline covariates, and are weighted to reflect intensive tracking. Standard errors are (in soft brackets) are clustered at the household level to reflect the design effect, and p-values corrected for False Discovery Rates across all the outcomes in the table are presented in hard brackets. Stars on coefficient estimates are derived from the FDR-corrected p-values, *=10%, **=5%, and ***=1% significance. Notes: HD, GD indicators defined as taking value of one if individual is in either the corresponding arm or the combined arm. Complementarity column reports the differential effect of being in the combined arm, compared to the sum of HD and GD impacts. Among individuals assigned to GD, only the mid-sized transfer arm is included in this analysis. Regressions include but do not report the lagged dependent variable, fixed effects for randomization blocks, and a set of LASSO-selected baseline covariates, and are weighted to reflect intensive tracking. Standard errors are (in soft brackets) are clustered at the household level to reflect the design effect, and p-values corrected for False Discovery Rates across all the outcomes in the table are presented in hard brackets. Stars on coefficient estimates are derived from the FDR-corrected p-values, *=10%, **=5%, and ***=1% significance.

Analysis of Heterogeneity
Our pre-analysis plan indicates four baseline dimensions over which we would look for signs of heterogeneity. These are gender, household consumption per capita, risk aversion (measured by the choices in a Binswanger-Eckel-Grossman lottery) and local labor market conditions (the employment rate within each of the 67 'cells' of the study). 17 Following the PAP, we use a standard interaction between treatment indicators and baseline characteristics to study heterogeneity across these baseline covariates, and examine only primary outcomes. The covariates are demeaned prior to interaction so that the uninteracted coefficient should be interpreted as impact at the mean of the interaction variable. To avoid interpretation issues arising from co-linearity, we omit both the baseline outcome (ANCOVA) and also the LASSO-selected covariates that are included but not reported for all the prior regressions.
Results of subgroup analyses by gender, risk, consumption, and cell-level employment shares are presented in Appendix Tables A.8, A.9, A.10, and A.11, respectively. Overall, we find very little evidence of meaningful heterogeneity in the impact of the program across these four dimensions.
Gender itself has a huge effect (female beneficiaries are less likely to be employed, put in fewer productive hours, and have lower levels of income and assets), but none of the interventions affect women in a manner significantly different than men. It is important to recognize, however, that power starts to become a greater concern as we split the study into smaller cells. Reading point estimates, we see that in fact HD does lead to a 9 pp increase in employment rates among men, while women see a 3 pp deterioration in employment rates during HD. These effects are not significant, however, because the minimum detectable effect on HD among men is 10 pp, and on the gender interaction is 14 pp. Table A.9 shows the risk averse being somewhat better at translating the interventions into income, with few other differences. Encouragingly, Table A.10 shows impacts that are relatively invariant to baseline consumption, indicating that both cash and HD are equally effective for the very poor (the GD Large arm has slightly larger benefits, and the Combined arm slightly larger benefits, for those with higher consumption at baseline). Given that many related programs have had an easier time creating benefits for the non-poor, this suggests that both of the interventions studied here should be considered good candidates for heavily poverty-targeted programs. Baseline employment rates, studied in Table A.11, are not only not driving the impacts of the program but appear to be completely uncorrelated with outcomes overall. Finally, based on feedback from the Rwanda USAID mission, we included age as a dimension of heterogeneity, and again in Table A.12 find no evidence of differences. The bottom row of all of these tables shows the p-value on F-tests of joint significance across the four interaction terms (note these are based on the non-FDR-adjusted significance rates); we present 25 such omnibus tests and find two of them to be significant at the 10% level, fewer than we would expect by random chance. Hence, taken 17 We filed our initial PAP with the AEA and then subsequently submitted to the Journal of Development Economics' pre-registry facility. In the time between these two submissions we came to realize that the data from the Convex Time Budget exercise we conducted at baseline (Andreoni and Sprenger, 2012) had not produced meaningful results. As a result we dropped the analysis of heterogeneity using discount rates and hyperbolicity derived from the CTB, and present here only the heterogeneity tests included in the later PAP filed with the JDE. together the analysis of heterogeneity suggests programs that are having consistent and similar effects across different types of beneficiaries and across local labor market conditions. We look for spillovers both on program participation and on primary outcomes. While in principle there may be externalities of each program at several levels of contact, we focus on spillovers that are local, in the sense that they occur between individuals who reside in the same village at baseline. 18 The reason for doing so is both substantive-this is plausibly the level at which such interactions are most salient-as well as practical: since the randomization is blocked at the sector level, and provides no variation in treatment saturation at that or higher levels.

Spillovers on Compliance
Using this notation, we can represent the three types of spillover analysis conducted, in increasing order of complexity. First, we analyze whether there are spillover effects on the rate at which individuals choose to participate in Huguka Dukore (this question is not interesting for GD because compliance is so close to universal). A major concern during the study design phase was that the assignment of one's peers to cash would discourage participation with HD. To do this, we use only the HD arm and estimate the following linear probability model: where P W ivb is a measure of individual i's participation in treatment w ∈ {HD, Combined}. Table 10 illustrates that the density of GD treatment in a village does not drive compliance.
The first column analyzes compliance within the standalone HD arm, and the second column within the Combined arm. The point estimates on the GD saturation rates are zero or positive, and never significant. Instead, this table shows that HD compliance is driven by the Huguka Dukore treatment saturation; the point estimate in the standalone HD arm is significant at the 5% level and suggests that as an HD participant goes from having no-one else in the village treated to everyone else in the village treated, we can expect compliance to increase by 27 pp. Since HD in the absence our of experiment would naturally attempt to treat 100% of the willing and eligible individuals considered in this study, that means that our study features a compliance rate that may be slightly too low relative to the rate that would naturally occur. However, since the costing estimate is multiplied times the observed compliance rate and the ITT is similarly a function of observed compliance in our study, compliance falls out of the benefit/cost comparisons that we make (it is in both the numerator and the denominator). This suggests that our estimates are still likely to be meaningful, absent substantial Essential Heterogeneity in the sense of Heckman and Vytlacil (2005).
Next, we can take our study of spillovers to the primary outcomes used in the study. We first use a more parsimonious specification that looks for average spillover effects from each type of treatment saturation to the other members of the village. Table 11 conducts this analysis and uncovers no evidence of spillovers in outcomes; not only are none of the saturation rates for any of the three treatments significant, but the sign of the coefficients alternates signs across outcomes for all three treatment saturations.
For outcome Y ivb1 , we modify the specification of equation used to estimate ITT effects (equation 1) as follows: Finally, we use the full model from our PAP that allows for the estimation of spillovers both from each treatment arm, and on to each treatment arm: In equation (5), the three coefficients in δ 2 provide a test of the spillover effects of each of the three treatments onto control individuals, and the nine coefficients in the vector δ 3 test for whether the spillover effects from the saturations of any of the three treatments on to individuals directly within-regression false discovery corrections we do not have a single significant spillover effect in the control, or differential effect for any treatment, across any of the 60 spillover tests performed here. Using unadjusted p-values we find 4 of these comparisons significant at the 10% level and none at the 5% level, in line with random chance.   Table analyzes spillover effects of the three main treatments (HD, GD Main, and GD Large) on the five primary outcomes. The first three rows are dummy variables for own treatment status, and the next three are the saturation rates for the three treatments among others in the village, so measure the marginal effect of going from no-one else treated to everyone else treated. Regressions include but do not report the lagged dependent variable, fixed effects for randomization blocks, and a set of LASSO-selected baseline covariates, and are weighted to reflect intensive tracking. Standard errors are (in soft brackets) are clustered at the household level to reflect the design effect, and p-values corrected for False Discovery Rates across all the outcomes in the table are presented in hard brackets. Stars on coefficient estimates are derived from the FDR-corrected p-values, *=10%, **=5%, and ***=1% significance. Bottom row is the p-value on an F-test of the joint significance of the three saturation terms. It therefore appears that we can quite simply conclude that our study did not generate detectable within-village spillover effects on outcomes. While our blocked lotteries do not allow us to study the kinds of cross-village General Equilibrium (GE) effects generated in Egger et al. (2019), the size of our transfers are tiny relative to local GDP and it appears unlikely that we would have generated meaningful local GE effects. We do not see the kind of zero-sum diversionary treatment effects uncovered by job-training programs in more formalized labor markets (Crépon et al., 2013), presumably because a) our overall impact on employment was small, and b) the informal labor

Treatment status
Notes: Each panel presents predicted outcomes under each of the four main treatment arms (Control, HD, GD Main, and GD Large), as the saturation level of a specific active treatment arm changes. Rows correspond to the outcomes of employment, productive hours, and the inverse hyperbolic sine of monthly incomes, productive assets, and household consumption per adult-equivalent, respectively. Horizontal shaded bands highlight one standard deviation above and below the control mean. Columns illustrate effects of variation in saturation rates in HD, GD-main, and GD-large, respectively. All predicted outcomes evaluated at means of covariates used in Equation 5.
38 markets in this context do not demonstrate a hard capacity constraint in the same way. The invariance of outcomes to HD saturations suggests that the response of HD compliance to HD saturations is not generating a study ITT that is driven by the lower treatment saturations induced by the lottery. Comfortingly, the overall takeaway is that our analysis remains internally valid and there is no need to attempt to correct for spillovers.

Tracing cash flows
In this question, we ask a question specific to the cash-transfer arms of our study: How have participants used the funding they received? Tracing these cash flows facilitates understanding in several ways. First, variation in the use of funds by the amount of cash received, and by the provision of HD training alongside cash in the combined arm, may help us to understand the mechanisms by which these arms deliver distinct impacts. Second, the extent to which we can account for the full values of transfers received sheds light on whether there might be 'missing' dimensions of impact not accounted for in our evaluation. And third, the extent to which observed patterns of spending are consistent with a simple spend-down of cash grants-and not a set of investments likely to deliver future income gains-is indicative of the sustainability of the impacts of cash.
Accounting for cash flows requires us to model both the inflows and the outflows induced by each active treatment arm. Clearly, the core of the induced inflow is the value of cash itself received by the beneficiary. But we also need to account for other income gains that each intervention can cause. To do so, we estimate each arm's impacts on beneficiary income and on transfers received by the beneficiary household, and we include these as inflows. We then compare these impacts on inflows with estimated impacts on expenditures, where our expenditure measures are a mixture of flows-household consumption, and transfers and loans made to other households-as well as stocks-savings, debt, livestock, and other productive assets. Flow measures vary in their recall periods, with income and consumption measured over comparatively short recall periods (as analyzed in Section 3.1, these are constructed as estimates for the month prior to the follow-up survey) and transfers measured over a 12-month recall period.
We value impacts on each of these dimensions by multiplying estimated proportional impacts of each program, from the specification in equation (1), by the average level of the corresponding outcome at follow-up in the control group. We emulate the results of Section 3.1 for household transfers in Appendix Table A.20. Together with the ITT results on primary and secondary outcomes, and combined with estimates of control means, this allows us provide the estimated financial impact of each program on both inflows and outflows.
The results of this exercise are presented in Table 12. In Panel A of that table, we estimate impacts of each cash-transfer arm on households' total income. This comprises not only the direct value of transfers received, but also induced increases in beneficiary income and in transfers received from other households. Since beneficiary income is measured over a short recall, we need to make an assumption about its time path, and so we extrapolate over the 12 months since the cash of livestock, other productive assets, savings, and debt. Our total outflow measure is constructed as the sum of the financial value of impacts on each of these categories, with impacts on debt entering negatively. We then construct the share of cash inflows that can be accounted for by these measured dimensions as the ratio of total treatment-induced outflows to total treatment-induced inflows for each arm.
We draw three basic conclusions from this exercise. First, the fact that induced outflows constitute a large share of induced inflows in general suggests that our measures of financial assets and expenditures are relatively complete, at least as far as is relevant to the impacts of these transfers.
Second, expenditure patterns do exhibit some differences across arms. For example, investments in livestock and in other productive assets do not rise proportionally with the value of transfers.
Further, comparison between expenditure patterns in the Middle and Combined arms-for which cash transfer values are equal-suggests that there are modest differences in the application of cash induced by HD training. The Combined treatment seems to divert the flow of cash to livestock and productive assets to a greater extent than the Middle arm, at the expense, in part, of debt repayment. While these investments have not delivered increases in income at the time of follow-up, it suggests that there remains a possibility that the Combined arm will cause differences in incomes as these investments deliver returns over the long term. 20 Third, while our ability to draw inferences about the sustainability of cash-transfer impacts beyond the follow-up period is necessarily speculative, there are two features of this exercise that point to sustainability. First, the consumption impacts of the cash transfers are smaller than their income impacts in absolute terms. And to the extent that a rapidly declining consumption path over the period prior to follow-up period would have been indicative of a lack of sustainability, we note that under the current (generous) assumption about total impacts on inflows, there is little unaccounted-for expenditure that could have been part of a downward-sloping consumption pattern over time.

Value for Money
There are multiple ways that one can pose that most basic question in cost effectiveness: how can policy spending achieve the greatest effect? Our study is designed to emphasize one comparison, namely the cost equivalent one: if a comparable amount of money is to be spent per beneficiary across programs, which achieves the greatest benefit? This approach holds both the beneficiary pool and the spend per beneficiary fixed, and asks about comparative effectiveness.
A related but different question can be asked if one is willing to concentrate spending on a subset of the beneficiary pool. This is comparative cost effectiveness: how can money be spent to create the largest total benefit across the pool for a fixed overall budget? In Tables 4 and 5 when we test for differences in the ratio of the effect sizes to the cost of the arm, this is the question we are asking.
A visual comparison between the cost equivalence and the cost effectiveness approaches is provided in Figure 2. In the left-hand column of figures, we illustrate the cost-equivalent comparison 20 The differences in investments induced appear too small to have created meaningful differences in income over the period studied. For example, if the differential livestock investment in the Combined arm versus the Middle arm, of $262-$218=$44, was undertaken immediately after the transfer and paid an annuity value of 5 percent, the resulting difference in incomes would have been just over $2 in total over the year since transfers occurred.
for the five primary outcomes. Here, the question is focused on a specific point on the x-axis, namely the cost of HD (the black diamond), and the GD arms are pooled to estimate one counterfactual, which is the predicted value at the HD cost. The Control need not even be included to execute this comparison.
The right-hand column of Figure 2 plots the identical outcomes by arm, but uses them to instead represent the cost effectiveness question in a visual way. The term value for money can be thought of as the relationship between the vertical axis in these figures (the value generated in terms of improved outcomes) over the horizontal axis (the money spent). Hence, the slope of the line between the Control outcome and the outcome in each active treatment arm in this space is a direct representation of benefit/cost across arms. The steepest slope has the highest value for money, assuming that the donor is willing to be flexible regarding how the number of people treated and therefore the cost per person. 21 In our study, the cost-equivalent question can be posed at one cost by design (the ex-post cost of HD, compared to the extrapolated benefit of GD at that cost), and at another cost by accident (because the Combined and HD Large arms turn out to have virtually identical costs).
The comparative cost-effectiveness question, on the other hand, is easily posed across costs since it essentially compares the gradient of additional money at each modality and total cost. Comparative cost effectiveness is therefore an attractive way to horse-race the GD arms against each other.
Given these two potential approaches, what does our study say about value for money? Beginning with the central cost-equivalent comparison between HD and cash, the main results of the paper suggest that for the $332 price tag of HD, one could produce significantly better outcomes via cash transfers across productive assets, beneficiary consumption, and livestock wealth, and for this amount of money HD is superior only at producing business knowledge. This result can help policymakers with different objectives think about which type of intervention will best produce the outcomes that they want to see.
Appendix Tables A.18 and A.19 provide estimates of cost effectiveness for primary and secondary outcomes, respectively, by dividing the ITT estimates by the cost of the arm in hundreds of dollars, and testing for the differences across arms (this is the slope coefficient visually represented in the right-hand panel of Figure 2). Seen in comparative cost effectiveness terms, the GD Middle transfer appears to be superior overall. It has the highest benefit/cost ratio across four of the five primary outcomes, losing only to the GD Small arm on productive asset value. Among secondary outcomes, GD Middle wins or ties in terms of cost effectiveness for subjective well-being, beneficiary consumption, and net household wealth. HD has higher cost effectiveness in driving savings, and for business knowledge.
When we make the cost-equivalent comparison between the Combined arm and the GD Large arm, we are asking the following question: given that we have already spent the $494 dollars to Figure 2: Cost Equivalence versus Cost Effectiveness deliver the GD Main arm, is it then better to spend another $350 to deliver HD on top of that cash, or should that additional spending be used to amplify the cash transfer? Here, the answers look quite different. There is now no outcome for which the GD Large arm is significantly better than Combined, and the Combined arm continues to demonstrate HD's advantage at producing business knowledge. Why these divergent results? The answer appears to be evidence of diminishing marginal benefits of cash; because HD is effective at driving many of the economic outcomes and additional cash is having a weaker effect, we find these two interventions to have similar costequivalent benefits even for outcomes on which cash 'beat' HD starting from zero. Overall, the fact that in general the less expensive interventions produce superior outcomes in cost-effectiveness terms says that we uncover no evidence for the idea that concentrating spending per person above the levels seen in the GD Middle arm is justified.

Conclusions
This study undertakes an exercise increasingly called for in recent years: namely, a direct comparison of in-kind development aid to cash, in a study design that uses variation in cash transfer values to identify cost-equivalent comparisons. The comparison program, Huguka Dukore, is a well-established workforce readiness intervention, and the beneficiary group is one in which both interventions have straightforward pathways to generate long-term improvements in welfare.
Our findings on the impacts of Huguka Dukore contribute to the evidence base on supply-side active labor market programs, of which it is typical in design. Huguka Dukore appears successful in increasing beneficiaries' productive time use, as well as the stock of assets productive that they accumulate. 18 months after program assignment, its participants report higher levels of subjective well being, demonstrate improved business knowledge, and have increased their stocks of savings considerably. HD has no significant effects on employment rates or beneficiary incomes. Given a control-group endline average income of approximately USD 17.45 per month, our point estimates on HD's impacts on beneficiary income imply that approximately 61 months of earnings at this rate would be required to pay back the realized costs to beneficiaries, consistent with supply-side labor market policies elsewhere (McKenzie, 2017). But looking beneath the surface, we do see exploratory evidence of shifts to microenterprise and even non-agricultural wage labor, which may portend long-term labor-market impacts.
On the other hand, cash transfers to this population appear to have moved a wide range of outcomes, including productive hours, incomes, productive assets, and household consumption.
These impacts are substantial for beneficiaries and provide a meaningful return on the costs of intervention: for example, the cost to USAID of the middle transfer would be recuperated in beneficiary income impacts alone after approximately 26 months. Secondary measures including subjective well-being, household wealth, and savings are all meaningfully moved by these transfers.
Several lessons about the design mix of transfer programming are evident: there is no evidence of complementarities between these cash-transfer impacts and the skills provided by HD, and increases 44 in transfer size above the middle transfer amount of $411 included in our study do not appear justified in cost-effectiveness terms.
It is worth noting a number of study limitations. First, the final costing number is substantially lower than anticipated, meaning that the cost equivalent analysis must extrapolate to a cost lower than anything observed in the GD transfer arms. Second, in order to achieve the benchmarking we confine both implementers to somewhat unnatural sample selection rules. An organization providing cash transfers would never have a reason to target only individuals who express interest in a training program. For HD, the study constrained them to study only poor individuals (they do not normally use Ubudehe status as a targeting criterion). We examine only a first phase of implementation of HD, and we miss any environmental benefits to the employment landscape caused by HD's capacity building and job placement work. Also, because we induced HD to treat at a lower intensity than they normally would (they would typically have treated all the individuals in our study), we may not have captured the effect of a program running at greater intensity.
Nonetheless, the internal variation in our sample suggests that these issues would generate limited bias in our study: outcomes are flat across transfer amount in GD (costing error), impacts are homogeneous across a range of beneficiary characteristics (differential targeting rules), and we find no evidence of saturation effects (HD treatment intensity). It therefore appears likely that our study has reasonable external validity.
The comparative evaluation of cash and in-kind modalities in this study-together with experimentally induced variation in cash transfer sizes-allows us to speak to the cash-benchmarking question increasingly called for in recent years. How would beneficiary outcomes change if a standard and widespread development intervention were simply distributed to the beneficiaries in the form of a mobile money transfer? Proponents of cash transfers have suggested that they should be considered the 'index funds' of international development, meaning a benchmark to which other programs are compared (Blattman and Niehaus, 2014). We estimate that at transfer values that are cost-equivalent from the donor perspective, the impacts of cash transfers exceeds that of HD by a statistically significant margin on two of five primary outcomes: monthly income and productive assets. On the other hand, given the substantially superior performance of HD at producing the human capital measured by the business skills index, we can provide an exact exchange rate that quantifies the tradeoffs policymakers must be willing to make. Focusing on the comparative impacts on productive assets versus business skills, to prefer HD at 18 months policymakers must be willing to forego a 200% increase in beneficiary productive assets in order to obtain a 0.5 SD increase in human capital.
This approach to cash benchmarking holds donor expenditure per beneficiary constant-in effect, restricting comparisons of in-kind programs to cash transfers that could reach an equivalent number of beneficiaries with equal-sized transfers. This need not be the optimal intensity of transfers. A donor who seeks to maximize an additive social welfare function in any of the outcomes considered, for example, would prefer to choose programming that maximizes the benefit-cost ratio.
Summarizing across outcomes, the middle of the cash transfer sizes considered in this study seems 45 to perform best by this metric.
One of the most surprising results of our study is the lack of complementarities between GD and HD. It is worth noting, however, that we designed the Combined arm to provide the cleanest test of structural complementarities between physical and human capital from the beneficiary side, which required that we implemented each arm precisely as in the standalone case. In reality, however, these two programs could be interwoven in a number of deeper ways. At the very least, regular programming intended to add capital over training would typically only do so at the end of the training, while our Combined arm received their cash midway through HD. More fundamentally, one might think about making the cash component conditional on participation in HD, which we did not do. So our results should not be taken to mean that it is impossible to design cash and training programs in a complementary manner, but rather than simply providing them together does not automatically generate a whole greater than the sum of the parts. Given the many complex, expensive, and multi-dimensional programs pushed by the donor community, this result is worth paying attention to.
The lessons of this study add to the evidence base not just on specific programs, but on the relative importance of the underlying constraints they seek to address. HD does appear to have been effective in alleviating a human capital constraint: it improves business knowledge, though non-cognitive dimensions such as aspirations and beneficiaries' self-efficacy appear to have been harder to move. In this context, the returns to raising business knowledge appear not to have been as impactful as the returns to alleviating liquidity constraints through cash transfers. And the liquidity constraints faced by these individuals appear not to be exceedingly large, given that the returns to increases in cash-transfer size are modest. While our prior cash-benchmarking study in Rwanda (McIntosh and Zeitlin, 2019) targeted households with malnourished children, it also focused on Rwandans within Ubudehe 1 and 2, and so provides an interesting point of comparison to the results here. That study provided evidence that transfer sizes greater than USD 150 are required to induce changes in productive outcomes in this setting, but here we find transfers larger than USD 400 have limited additional value. This helps to identify the 'sweet spot' for cash in this context. Moreover, this study shows that the impact of human-capital and liquidity improvements flow through different channels, in particular as the HD program does move beneficiaries in the direction of wage employment, whereas cash transfers support self-employment. Importantly, these results are at 18-months only. The long-run impact of the mechanisms induced by each of these programs remains an important question.
46 transfers on nutrition: The South African Child Support Grant," 2006.
Ahmed, AU, JF Hoddinott, S Roy, E Sraboni, WR Quabili, and A Margolies, "Which    Note: Intention to treat pooling the three smaller cash transfer amounts into a single arm, GD Main. Regressions include but do not report the lagged dependent variable, fixed effects for randomization blocks, and a set of LASSO-selected baseline covariates, and are weighted to reflect intensive tracking. Standard errors are (in soft brackets) are clustered at the household level to reflect the design effect, and p-values corrected for False Discovery Rates across all the outcomes in the table are presented in hard brackets. Stars on coefficient estimates are derived from the FDR-corrected p-values, *=10%, **=5%, and ***=1% significance. p-values in final three columns derived from F -tests of hypotheses that benefit-cost ratios are equal between (a) GD Main and HD; (b) GD Main and GD Large; and (c) GD Main and Combined. Employed is a dummy variable for spending more than 10 hours per week working for a wage or as primary operator of a microenterprise. Productive hours are measured over prior 7 days in all activities other than own-farm agriculture. Monthly income, productive assets, and household consumption are winsorized at 1% and 99% and analyzed in Inverse Hyperbolic Sine, meaning that treatment effects can be interpreted as percent changes.  Notes: Panel A presents impacts on indicators for employment of any hours in the corresponding activity type in the preceding week. Panel B presents impacts on an indicator for overall employment, using the reported threshold for minimum hours. Regressions include but do not report an indicator for lagged employment status, fixed effects for randomization blocks, and a set of LASSO-selected baseline covariates, and are weighted to reflect intensive tracking. Standard errors are (in soft brackets) are clustered at the household level to reflect the design effect, and p-values corrected for False Discovery Rates across outcomes in each panel are presented in hard brackets. Stars on coefficient estimates are derived from the FDR-corrected p-values, *=10%, **=5%, and ***=1% significance.  Table presents tests for heterogeneity of treatment effects by Gender. Uninteracted coefficients in the first four rows give the treatment effect of the program on men, and the next four rows test for the differential effect between women and men. Standard errors are (in soft brackets) are clustered at the household level to reflect the design effect, and p-values corrected for False Discovery Rates across all the outcomes in the table are presented in hard brackets. Stars on coefficient estimates are derived from the FDR-corrected p-values, *=10%, **=5%, and ***=1% significance. p-value in the last row from an F-test on whether treatments have a jointly differential effect by gender.       Notes: Each column describes the direct and spillover effects of a specific treatment on Monthly Income (IHS); all results in the table are from a single estimation. Saturation mean and standard deviation correspond to the distribution of saturation rates for the treatment in question. Regressions include but do not report the lagged dependent variable, fixed effects for randomization blocks, and a set of LASSO-selected baseline covariates, and are weighted to reflect intensive tracking. Standard errors are (in soft brackets) are clustered at the household level to reflect the design effect, and p-values corrected for False Discovery Rates across all the outcomes in the table are presented in hard brackets. Stars on coefficient estimates are derived from the FDR-corrected p-values, *=10%, **=5%, and ***=1% significance. p-value in the last row corresponds to a test for whether the treatment in question has interference effects on any arm, including control. Notes: Each column describes the direct and spillover effects of a specific treatment on Productive Assets (IHS); all results in the table are from a single estimation. Saturation mean and standard deviation correspond to the distribution of saturation rates for the treatment in question. Regressions include but do not report the lagged dependent variable, fixed effects for randomization blocks, and a set of LASSO-selected baseline covariates, and are weighted to reflect intensive tracking. Standard errors are (in soft brackets) are clustered at the household level to reflect the design effect, and p-values corrected for False Discovery Rates across all the outcomes in the table are presented in hard brackets. Stars on coefficient estimates are derived from the FDR-corrected p-values, *=10%, **=5%, and ***=1% significance. p-value in the last row corresponds to a test for whether the treatment in question has interference effects on any arm, including control. Notes: Each column describes the direct and spillover effects of a specific treatment on Household Consumption (IHS); all results in the table are from a single estimation. Saturation mean and standard deviation correspond to the distribution of saturation rates for the treatment in question. Standard errors are (in soft brackets) are clustered at the household level to reflect the design effect, and p-values corrected for False Discovery Rates across all the outcomes in the table are presented in hard brackets. Stars on coefficient estimates are derived from the FDR-corrected p-values, *=10%, **=5%, and ***=1% significance. p-value in the last row corresponds to a test for whether the treatment in question has interference effects on any arm, including control.

Appendix C Selection of control variables
In our pre-analysis plan, we state that control variables for the primary specification "will be selected on the basis of their ability to predict the primary outcomes". In doing so, we seek to build on recent developments that balance the challenge of using baseline data to select variables that will reduce residual variance in equation (1) with the danger that researcher freedom in the selection of control variables can lead to p-hacking, in which right-hand-side variables are selected specifically on the basis of the statistical significance of the coefficient of interest (Card and Krueger, 1995;Casey et al., 2012), thereby invalidating inference.
To balance these concerns, we adapt the post-double-selection approach set forth in Belloni et al. (2014b, henceforth BCH). BCH advocate a two-step procedure in which, first, Lasso is used to automate the selection of control variables, and second, the post-Lasso estimator (Belloni et al., 2012) is used to estimate the coefficients of primary interest in Equation (1), effectively using Lasso as a model selection device but not imposing the shrunken coefficients that results from the Lasso estimates directly. Belloni et al. (2014b) demonstrate that this approach not only reduces bias in estimated treatment effects better than alternative approaches-less a concern given the successful randomization in our experiment-but that it may improve power while retaining uniformly valid inference.
In the first stage, model selection is undertaken by retaining control variables from the union of those chosen either as predictive of the treatment assignment or of the outcome. This model selection stage can be undertaken after residualizing to account for a set of control variables that the authors have a priori determined below in the model, as in Belloni et al. (2014a). In our case, we retain block fixed effects, lagged values of the outcome, and lagged values of (the inverse hyperbolic sine of) household wealth in all specifications, per our pre-analysis plan.
We modify the BCH approach for application to a randomized experiment in three ways. First, again following (Jones et al., 2019), for each outcome we choose the Lasso penalty parameter that minimizes the 10-fold cross-validated mean squared error. Second, to ensure that chance differences in the leverage of observations across different covariate sets do not lead to different conclusions about the (relative) impacts of treatment across different outcomes (Young, 2019), we take the union of covariate sets selected to be predictive of the five primary outcomes of the study, and use these as controls for all outcomes. And third, we modify the heteroskedasticity-robust Lasso estimator of Belloni et al. (2012) to incorporate sampling weights consistent with our design. 22 The set of potential covariates is determined as follows: • Baseline values of all primary outcomes, including the individual components of the employment status, productive time use, monthly income variables outlined in Section 2.7; • Baseline values of all secondary outcomes, • Baseline values of all dimensions of heterogeneity pre-specified in Section 3.4.
22 Specifically, we up-weight observations in our 'intensive tracking' endline sample by the inverse of the fraction of not-initially-reached individuals in the follow-up survey who were then assigned to intensive tracking.

81
• The number of study participants (in any arm of the study) in an individual's village, which is defined as the measure of network 'degree' for each individual in the spillover analysis of Section 3.5.
All variables are normalized prior to inclusion in the selection routine, to have mean zero and variance of one in the baseline sample. We include squares of all continuous variables and all pairwise interactions among the potential covariates above, and between the potential covariates above and the set of variables that force the routine to include without penalty To ensure that sample size is not affected by the choice of covariates, we impute values of zero for all variables in the potential covariate list, and for each potential covariate we include an indicator for whether such an imputation was undertaken among the list of potential covariates to be fed into the BCH first-stage selection procedure.
2. Announcement to the community in public places (churches, community halls) or a meeting to engage potential beneficiaries. At this point only the HD program wsa described to beneficiaries, and with only general language about the household grants arm. Guiding language: "We are pleased to be able to bring programming to this community that seeks to improve the livelihoods of vulnerable youth. To this end, we are requesting the names and contact details of youth meeting the following criteria: ¡insert eligibility criteria here¿.
Participating youth should be willing and interested to join an employment skills program, called HD, that will provide training and work experience to participants." 3. Screening of youth by the selection committee which produced the final list of potential beneficiaries that was passed to local implementing partners (IPs).
4. Invitation of potential beneficiaries to an orientation meeting. The language of this invitation reflected the fact that potential beneficiaries were not guaranteed places in HD, and might be randomly allocated to a different program or the control. Guiding language for official communication: "We have determined that you are eligible for the Huguka Dukore program. There may be more eligible individuals than Huguka Dukore will treat this year, so you are not yet guaranteed a place, though some of those not treated by Huguka Dukore will be supported by another NGO. To find out more about the Huguka Dukore program and to take the next step toward this opportunity, please attend an orientation meeting at XXX on YYY date."

5.
Orientation and awareness meeting with selected youth by local IPs at which they are given further explanation about the program. In HDs other districts, these orientation meetings convey information about the scope of that program, under a presumption that those who participate in the orientation meeting can have a place in HD should they choose to take it up.
6. Description of the lottery for program assignment. The lottery is described during this meeting with reference to another intervention providing livelihoods assistance that will also be determined by the lottery. Guiding language: Today you have learned more about the Huguka Dukore program. This is one of two programs that are being delivered by distinct NGOs, in coordination with Sector and District officials, both of which seek to improve livelihoods for vulnerable youth. If you decide that you are interested in participating in one of these programs, there is one more step in the selection process. To participate, you must attest that you have the time and interest required to participate in Huguka Dukore. Your name will then be entered into a pool of applicants. There will be a public meeting in which a lottery will be used to determine which of these applicants receives a place in HD. You may attend this meeting if you wish, but you do not have to do so in order to gain a place. Not all whose names are entered into the lottery will be placed in HD. Some of those who participate in the lottery will be passed to a second NGO, which provides assistance to individuals seeking to improve their livelihoods. Those who receive a place in either program will be contacted directly by the relevant organization after the lottery. To gain access to either program, you must participate in this lottery. If you are willing to participate, please provide your name and contact details in writing. Prior to the lottery, you may be contacted by an independent research organization called Innovations for Poverty Action, who are conducting a survey of potential beneficiaries. You do not have to participate in this survey in order to gain access to our program, and participation will not affect your chances of enrollment. However, we would be grateful for your willingness to participate in an interview with IPA, which will help us to understand the design and impacts of our work.
7. Registration for the lottery assignment. To correctly reflect the lottery process to participants, they were told when asked to enroll in the study that it is a lottery in which you will have a chance of receiving HD, a chance of receiving assistance from a different organization that gives household grants, and a chance that you do not receive either program." Individuals who do not choose to register for the study will not be excluded from receiving HD if they are eligible & choose to participate.

Appendix D.2.1 Defining Primary outcomes
For each of the outcomes defined below, we provide a definition, followed by an explanation of how that measure will be constructed from survey data. Survey questions either begin with a 'B-' for the beneficiary instrument or a 'H-' for the household instrument, followed by the two-digit section number, followed by 'q' and the question number. These refer to the beneficiary and household instruments, respectively. 23 There are five primary outcomes: