View in EDS HTML Full Text PDF Full Text

Foundational Learning Program Evaluation and Dropouts: Are Dropouts a Heterogeneous Group?

Saved in:
Bibliographic Details
Title: Foundational Learning Program Evaluation and Dropouts: Are Dropouts a Heterogeneous Group?
Language: English
Authors: David Gray, Louis-Philippe Morin
Source: Education Economics. 2025 33(2):198-217.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 20
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Descriptors: Foreign Countries, Career and Technical Education, Job Skills, Employment Potential, Program Evaluation, Dropouts, Career Readiness, Labor Force Development, Educational Attainment, Barriers, Employment Programs
Geographic Terms: Canada
DOI: 10.1080/09645292.2024.2309282
ISSN: 0964-5292
1469-5782
Abstract: We analyze the characteristics and outcomes of a Canadian foundational learning program's dropouts and compare them with those of the completers. We find significant heterogeneity within dropouts along two dimensions: when they drop out and why. Individuals whose characteristics have been historically associated with greater labour market barriers, and those with lower employability skills are more likely to complete the program. Individuals who face fewer barriers tend to leave at an early stage, while individuals without a high school degree tend to drop out later. Conditional on education, higher employability-skill participants are more likely to leave and return to school.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1486002
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFN--8kAyVGDuc0sMNZgdiFAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDJ78leQr2jhpfget2AIBEICBmyfvweBFhjqCz43kg5vUbuxxMHF2WR7RE5t8MW2iSpUn3SVcwGVp7Zxt8UcjvdbAIPScpQ9TyvQJwIXTFiAqKpsVjFxcYoaJzlj0v6crEmu1ZB3U_3oPZXKOUwpzVym7ZBxvHK6EvQvJwnEbTx3WOnitLvD6TTf54KqhLAA9rkqqZqdlUJhzrP6PhMBPsxuofbqTW3lXl2hMxvoj
Text:
  Availability: 1
  Value: <anid>AN0183842095;ede01apr.25;2025Mar21.03:44;v2.2.500</anid> <title id="AN0183842095-1">Foundational learning program evaluation and dropouts: are dropouts a heterogeneous group? </title> <p>We analyze the characteristics and outcomes of a Canadian foundational learning program's dropouts and compare them with those of the completers. We find significant heterogeneity within dropouts along two dimensions: when they drop out and why. Individuals whose characteristics have been historically associated with greater labour market barriers, and those with lower employability skills are more likely to complete the program. Individuals who face fewer barriers tend to leave at an early stage, while individuals without a high school degree tend to drop out later. Conditional on education, higher employability-skill participants are more likely to leave and return to school.</p> <p>Keywords: Active labour market policies; dropouts; policy evaluation</p> <hd id="AN0183842095-2">1. Introduction</hd> <p>Empirical evaluation of the causal impact of active labour market policies (ALMPs) is challenging due to the lack of suitable and reliable data (Heckman, Lalonde, and Smith [<reflink idref="bib16" id="ref1">16</reflink>]). Two particular aspects of data deficiencies are the high dropout and attrition rates that are likely to be present and non-random in these programs.[<reflink idref="bib1" id="ref2">1</reflink>] Furthermore, most ALMP data lack detailed information on who the dropouts are, why they dropped out, and their employability skills. These gaps render it difficult to have an idea of (<reflink idref="bib1" id="ref3">1</reflink>) the likely sign of biases when estimating the effects of the program, and (<reflink idref="bib2" id="ref4">2</reflink>) how ALMPs could be redesigned and/or better targeted at specific groups to reduce attrition. Consequently, the question regarding how to treat non-completers' information when estimating the impact of ALMPs remains complicated and unresolved (see McCall, Smith, and Wunsch [<reflink idref="bib21" id="ref5">21</reflink>] for a more detailed discussion).</p> <p>This paper examines the <emph>Foundations Workplace Skills Program</emph> (FWSP), a foundational learning program involving literacy and essential skills (LES) training delivered in the Canadian province of British Columbia. We analyze the characteristics and outcomes of FWSP dropouts. The central theme is the heterogeneity of this group along two different dimensions: the stage at which they drop out of the program and the destination outcome.</p> <p>We focus our analysis on three questions. First, what factors differentiate dropouts from participants who completed the program? Second, what are the predictors of the choices to either complete the program, drop out at an early stage, or drop out at a later stage? Third, what are the predictors of the alternative destinations, such as returning to school, finding a job, or further job search planning outside of the program? To our knowledge, this is the first study dealing with the phenomenon of non-completion in a Canadian context and one of the first regarding a foundational learning program for any country.</p> <p>We use the rich information in the FWSP's administrative data set to address the three questions above. It contains information on dropouts' initial levels of employability skills, as measured by <emph>Test of Workplace Essential Skills</emph> (TOWES) scores, their patterns of participation through the phases of the program, and their outcomes (e.g. whether or not the individual found a job or returned to school), all of which are related to the primary reason why dropouts left the program. This is one of the few studies to exploit such information in the evaluation literature. We also observe post-treatment (i.e. second try) TOWES test scores for the rather small minority (19%) of individuals who did complete the program.</p> <p>We first compare the characteristics of dropouts and completers upon enrolling in the FWSP and find that the two groups differ along important dimensions, most crucially on their initial TOWES scores in document use, numeracy, and reading. These differences in scores provide information beyond the typically observed participants' characteristics, such as educational attainment, age, and immigrant status. The median initial scores of dropouts are around (or above) the top quartile initial scores of individuals who completed the program. Given that the TOWES is intended to indicate employability skills, this finding alone suggests that the test can serve as a diagnostic measure for screening purposes.</p> <p>Regarding the predictors of the event of dropping out, our regression results suggest that females, individuals who are not native English speakers, and university graduates are less likely to do so. In contrast, individuals with higher initial TOWES scores but without a high-school diploma are more likely to drop out. Estimates from a multinomial logit model of the stage at which a participant dropped out indicate that women, immigrants, those with less than a HS education, and university graduates are less likely to drop out early, while those with higher TOWES scores are more likely to do so. Those with less than a high school education and those with lower TOWES scores are more likely to drop out at a late stage. Based on a multinomial logit model of the destination outcome, we find that clients with higher TOWES scores are more likely to return to school, and those with higher levels of education are less likely to do so. These findings suggest that dropping out should not necessarily be interpreted as a 'bad' or wasteful outcome, especially since one of the main goals is to facilitate finding work.</p> <p>Another component of our empirical analysis reveals patterns among the participants regarding their progress, or lack thereof, through the different phases of the program. These findings, combined with an examination of the observable characteristics of the dropouts and with regression results, suggest that the dropouts are a heterogeneous group. Information on the predictors of dropping out and on the choices to exit can indicate to program designers and policymakers who is more likely to complete a learning program and, therefore, whom to target (e.g. should they target a more vulnerable population?) when recruiting participants, especially in cases where the programs are likely to be over-subscribed or where the fixed costs of training programs are considerable.</p> <p>In addition to the targeting challenge, our findings provide indications for designing ALMPs customized to the needs of specific groups. Specifically, we find suggestive evidence that FWSP's overall dropout rate could be reduced by targeting more individuals with lower expected employability skills (e.g. high-school dropouts), and adjusting the program to meet their specific needs.</p> <p>Small-scale programs of this type are likely to continue to be implemented in the future, often in affiliation with post-secondary institutions such as community colleges. Indeed, in their survey of economic returns to education, Dickson and Harmon ([<reflink idref="bib12" id="ref6">12</reflink>]) conclude that targeted actions at the local level (as opposed to those having a standardized, 'universal' policy design) might become the norm. We thus view our findings as applicable to learning about the workings of similar interventions.</p> <p>The literature on the predictors of dropping out from adult training is somewhat scarce.[<reflink idref="bib2" id="ref7">2</reflink>] Papers that are directly related to ours include (de Crombrugghe, Espinoza, and Heijke [<reflink idref="bib11" id="ref8">11</reflink>]; Paul [<reflink idref="bib26" id="ref9">26</reflink>]; Waller [<reflink idref="bib27" id="ref10">27</reflink>]). The first two study the choice of dropping out from a German program, while the last one examines it from a Peruvian one. Researchers might have some pre-intervention information regarding their demographic and socio-economic characteristics. However, they rarely have access to post-intervention information, such as the reasons for which participants dropped out or the timing of that decision - information which we observe in our study. As in other studies, we observe some relevant pre-treatment information, but we also gauge the incidence of partial treatment (e.g. whether they participate in a significant portion of the program) and investigate whether the decision to drop out correlates with employability skils.</p> <p>Our paper can be related to the literature on a broader set of dynamics, including selection into treatment and dynamics within the treatment process itself. Studies such as Biewen et al. ([<reflink idref="bib5" id="ref11">5</reflink>]), Fitzenberger, Osikominu, and Paul ([<reflink idref="bib15" id="ref12">15</reflink>]) and Dalla-Zuanna and Liu ([<reflink idref="bib10" id="ref13">10</reflink>]) consist of evaluations that incorporate some of these processes into estimated treatment effects on employment and earnings within a choice-theoretic framework.[<reflink idref="bib3" id="ref14">3</reflink>] Like our study, Lacroix and Brouillette ([<reflink idref="bib20" id="ref15">20</reflink>]) is a Canadian application, although the nature of the intervention (a wage subsidy) and the clientele (social assistance recipients) are different. While those authors are concerned with program non-completion, which they observe and treat as exogenous, their primary focus is an extremely low take-up rate that they model. What they label 'early termination' is estimated to have a slightly favourable (negative) impact on transitions into social assistance and a positive impact on transitions out of it. The framework for our study is more straightforward, as we do not analyze entries but only dropouts. We do not consider any alternative interventions; instead, we investigate both the timing and the choice of participants' outside options.</p> <p>As suggested by Crépon and den Berg ([<reflink idref="bib9" id="ref16">9</reflink>]) and Escudero ([<reflink idref="bib14" id="ref17">14</reflink>]), improving aspects of the delivery and implementation of ALMP interventions can lead to a more efficient allocation of services and to better labour market outcomes, particularly in the case of relatively low-skilled individuals such as the ones that comprise our population.</p> <p>The rest of the paper is organized as follows. The next two sections detail the FWSP and the administrative data exploited in this paper. Section 4 presents our empirical strategies to investigate the differences between completers and non-completers and the predictors of dropping out of the program. Section 5 presents our findings, and Section 6 concludes.</p> <hd id="AN0183842095-3">2. The foundations workplace skills program (FWSP)</hd> <p>The Foundations Workplace Skills Program (FWSP) began in 2006 in Surrey, British Columbia, Canada. It is somewhat unique in that it blends two broad types of programming: foundational learning (i.e. developing literacy and essential skills) and labour-market-related learning intended to strengthen 'work readiness.' The first component can be categorized as general human capital development aiming to improve the participant's productivity. The second one involves job search assistance aiming to facilitate the matching process, the importance of which Altmann et al. ([<reflink idref="bib2" id="ref18">2</reflink>]) emphasize in their study. They find that the provision of information regarding employment prospects and labour market conditions has a positive impact on job search outcomes.</p> <p>The program was funded through the Labour Market Development Agreement between Human Resources and Skills Development Canada (HRSDC) and the provincial government.[<reflink idref="bib4" id="ref19">4</reflink>] Like the majority of ALMPs in Anglo-Saxon countries, FWSP participation is voluntary. The targeted clientele pays no fees and consists of unemployed individuals of any working age. Many of them are immigrants or displaced workers who are deficient in any of the nine essential skills articulated by the Conference Board of Canada.[<reflink idref="bib5" id="ref20">5</reflink>] The FWSP aims to diagnose and fill gaps in foundational skills tailored to clients' career aspirations. The stated program objectives are to assist clients in the following endeavours:</p> <p></p> <ulist> <item> developing an awareness, validating, and confirming (literacy, document use and numeracy) skill levels using the standardized Test of Workplace Essential Skills (TOWES) to provide direction for planning the appropriate next steps to gaining employment</item> <p></p> <item> understanding and articulating their skills in relation to the world of work</item> <p></p> <item> acquiring basic essential skills required for success in work, learning, and life</item> <p></p> <item> achieving long-term labour market attachment</item> </ulist> <p>It markets itself as being distinct from the formal schooling/training system by being customized according to the needs and attributes of individual participants, many of whom feel uncomfortable with modern information and communication technology.</p> <p>All candidates are referred by external caseworkers from various social service agencies within the community that are typically funded by the provincial government of British Columbia.[<reflink idref="bib6" id="ref21">6</reflink>] This intake process is very different from those utilized by active labour market interventions in countries such as Germany, where participation is required for eligibility for social insurance payments (Card et al. [<reflink idref="bib7" id="ref22">7</reflink>]; Card, Kluve, and Weber [<reflink idref="bib8" id="ref23">8</reflink>]). While participation is voluntary, in their evaluation of this program, Palameta, Dowie et al. ([<reflink idref="bib24" id="ref24">24</reflink>]) and Palameta, Nguyen et al. ([<reflink idref="bib25" id="ref25">25</reflink>]) suggest that case-managed clients sometimes feel accountable to some extent for their degree of participation, as they might have to rely on the case-managers network in the future. Those authors argue that some clients might take greater agency by virtue of entering through that channel.</p> <p>Although a few of these referrals are not accepted, such as those deemed to have very low literacy skills, the recruitment and initiation process is centered on the event of 'taking a skills test,' which is the TOWES examination. The TOWES is designed to measure employability skills in three domains: document use, numeracy and reading. The questions and problems on the questionnaire refer to authentic workplace documents or tasks. For example, one could be asked to extract information from a table (document use), to compute simple additions using an income tax form (numeracy), and to find the main idea of a passage in a trade manual (reading).</p> <p>The FWSP differentiates itself from other LES interventions in several ways. It targets job seekers rather than employed workers or individuals who are not in the labour force. It is not focused on the acquisition of credentials. In contrast to LES interventions delivered in a workplace setting, which usually focus on aligning training with vocational tasks and job performance indicators within a single sector or even a single occupation, the FWSP is contextualized to what the client and the staff view as a set of relevant and feasible occupations without the involvement of potential employers. As described in Palameta, Nguyen et al. ([<reflink idref="bib25" id="ref26">25</reflink>]),</p> <p>Few employment programs have used the LES framework to assess occupation-specific skill gaps among the unemployed, and as a result, there has been a lack of targeted services focused on occupation-oriented skills upgrading for job seekers[<reflink idref="bib7" id="ref27">7</reflink>]...Its focus on integrating LES assessment and upgrading within an occupation-targeted career path using occupational relevant materials is unique to training models for the unemployed (page 5).</p> <p>In addition, the FWSP target population is situated more at the periphery of the labour market than is the case for some of the unemployed in the sense that they often face significant barriers to finding a job (e.g. limited education and work experience, lack of job-hunting skills, and limited language skills).[<reflink idref="bib8" id="ref28">8</reflink>]</p> <p>The intervention is structured according to three phases, each of which is viewed as a separate subprogram. While they are coordinated and integrated, their structure is sequential but not strictly hierarchical, as certain participants can advance directly from phase one to phase three (in our data, only a tiny proportion of participants actually do so). A description of them is as follows:</p> <p></p> <ulist> <item> Phase one: The objective is to execute the initial assessment by taking the TOWES exam for the first time and to instill in them awareness of the importance of literacy and essential skills (LES). This phase lasts no longer than 6 hours.</item> <p></p> <item> Phase two: The objective is to develop a portfolio of the participant's inventory of essential skills with an eye on matching them with the skills that are required for 'target occupations' to which the candidate could aspire.[<reflink idref="bib9" id="ref29">9</reflink>] It typically involves 30 hours of classroom learning and workshops that expose participants to all nine of the essential and employability skills, and it instills in them the notion of transferability of skills and knowledge. The curriculum, which was developed by Douglas College, is centered on 'career adaptability' skills and knowledge. More specifically, these components are labelled (i) career planning, (ii) career decision-making self-efficacy, (iii) job search clarity, and (iv) job search self-efficacy. It is thought to be most suitable for those with relatively high skills and almost work-ready but perhaps perceiving some type of educational barrier.</item> <p></p> <item> Phase three: The objective is to enhance foundational skills on an individualized and customized basis as they relate to the targeted occupations. This phase utilizes occupationally-relevant learning materials. The activities include improving oral communication and working on a team, targeting skills deficits, executing software and internet applications available for career planning, learning how to learn with an eye on establishing a career, learning how to avoid skill loss, and providing supplementary counselling. It culminates in retaking the TOWES exam. The duration varies greatly over a range of 25 to 200 hours. On average, clients spent approximately 6 to 8 weeks in training at an average of 25 hours per week.[<reflink idref="bib10" id="ref30">10</reflink>] Some who were recommended by program staff at the end of phase two declined to continue. In contrast, some drop out right before this stage because staff do not find them suitable.</item> </ulist> <p>Each phase ends with a follow-up period of 12 weeks duration, which is a relatively short interval to observe outcomes. Furthermore, this 12-week period often overlaps with entry into a subsequent phase, which implies that on a cumulative basis, there are fewer than 36 weeks of follow-up for a participant who has completed all three phases.</p> <hd id="AN0183842095-4">3. Data</hd> <p>We have access to the FSWP administrative data for participants who began the program between 2006 and early 2012. The data are not structured along a chronological timeline but rather around the three phases that are described above. They are collected by program staff on an ongoing basis. The data contain information on participants' initial three TOWES scores (for document use, numeracy and reading), post-program TOWES scores (for those who completed the program), and demographic and socio-economic information. An attractive feature of the data is that, for each participant, we observe the outcome of each phase (conditional on having started that phase). For example, after phase one, we know the proportion of individuals who entered phase two, found a job, returned to school,[<reflink idref="bib11" id="ref31">11</reflink>] returned to career planning/search outside the program, or did not complete the phase.[<reflink idref="bib12" id="ref32">12</reflink>] Such information will help us investigate the timing and the reasons for dropping out.</p> <p>The FSWP began collecting socioeconomic and demographic information in June 2007, which marks the beginning of our sample. We restrict our sample to individuals aged 15-64 (only six participants were aged over 64) and individuals who had enough time to complete the program by the time the data were collected. The last observations recorded in our data set are dated February 2012. We discarded 116 individuals who started phase 1 on October 1st, 2011 or later. We chose this as the cutoff date because the median program duration time for completers is 95 days, and the 75th percentile is close to four months (112 days). Since we investigate participants' progress from phase one through phase three, we discarded nine participants who started the program in phase 2. Out of this sample of 1,733, we omitted 120 participants with missing information regarding their age, gender, mother tongue or baseline TOWES scores.</p> <hd id="AN0183842095-5">4. Methodology</hd> <p>We begin by describing the differences in the baseline characteristics between participants who completed the program and those who dropped out. Simple descriptive statistics will allow us to test whether the two groups are balanced in terms of demographic and socio-economic characteristics as well as in terms of pre-program TOWES scores. In Section 5.2, we proceed to investigate the potential predictors of dropping out of the program by estimating different specifications of the following regression model:</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>D</mi><mrow><mi>iym</mi></mrow></msub><mo>=</mo><mi>α</mi><mo>+</mo><mi>λ</mi><msub><mi>S</mi><mi>i</mi></msub><mo>+</mo><mrow><msub><mi mathvariant="bold-italic">X</mi><mrow><mi>i</mi></mrow></msub><mi mathvariant="normal">Γ</mi></mrow><mo>+</mo><msub><mi>ρ</mi><mi>y</mi></msub><mo>+</mo><msub><mi>π</mi><mi>m</mi></msub><mo>+</mo><msub><mi>ϵ</mi><mrow><mi>i</mi></mrow></msub><mo>,</mo></math> </ephtml> (<reflink idref="bib1" id="ref33">1</reflink>)</p> <p>where</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>D</mi><mrow><mi>iym</mi></mrow></msub></math> </ephtml> is a binary variable equal to 1 if the participant <emph>i</emph> who started the program in month <emph>m</emph> of year <emph>y</emph> dropped out of the program.</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>S</mi><mi>i</mi></msub></math> </ephtml> is the (average) TOWES score of individual <emph>i</emph> at the beginning of the program, while</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>X</mi><mrow><mi>i</mi></mrow></msub></math> </ephtml> contains the participants' socio-demographic characteristics (i.e. age, sex, immigrant status, mother tongue, and educational attainment, as presented in Table 1).</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>ρ</mi><mi>y</mi></msub></math> </ephtml> and</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>π</mi><mi>m</mi></msub></math> </ephtml> are year and month fixed effects meant to capture the effect that labour-market conditions and potential program improvements could have on drop-out decisions.[<reflink idref="bib13" id="ref34">13</reflink>] Here, we are interested in knowing whether observable characteristics (and initial TOWES scores) can predict the likelihood of dropping out. The results from estimating Equation (<reflink idref="bib1" id="ref35">1</reflink>) could be of particular interest to policymakers thinking of implementing a similar program (Heckman and Smith [<reflink idref="bib17" id="ref36">17</reflink>]).</p> <p>Table 1. Descriptive Statistics of Completers and Dropouts.</p> <p> <ephtml> <table><thead valign="bottom"><tr><td /><td>(1)</td><td>(2)</td><td>(3)</td><td>(4)</td></tr><tr><td>Variable</td><td>Whole Sample</td><td>Completers</td><td>Dropouts</td><td>Diff. (2)–(3)</td></tr></thead><tbody><tr><td><bold>A. Gender</bold></td></tr><tr><td>Female</td><td char=".">0.56</td><td char=".">0.72</td><td char=".">0.52</td><td char=".">0.20<p><graphic href="cede_a_2309282_ilm0006.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td><bold>B. Age</bold></td></tr><tr><td>Age 15–24</td><td char=".">0.10</td><td char=".">0.07</td><td char=".">0.11</td><td char=".">−0.04<p><graphic href="cede_a_2309282_ilm0007.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td>Age 25–34</td><td char=".">0.29</td><td char=".">0.30</td><td char=".">0.28</td><td char=".">0.02</td></tr><tr><td>Age 35–44</td><td char=".">0.31</td><td char=".">0.31</td><td char=".">0.31</td><td char=".">0.00</td></tr><tr><td>Age 45–54</td><td char=".">0.23</td><td char=".">0.26</td><td char=".">0.23</td><td char=".">0.03</td></tr><tr><td>Age 55–64</td><td char=".">0.07</td><td char=".">0.06</td><td char=".">0.07</td><td char=".">−0.01</td></tr><tr><td><bold>C. Language & Nationality</bold></td></tr><tr><td>Native English Speaker</td><td char=".">0.56</td><td char=".">0.29</td><td char=".">0.63</td><td char=".">−0.34<p><graphic href="cede_a_2309282_ilm0008.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td>Immigrant</td><td char=".">0.43</td><td char=".">0.65</td><td char=".">0.38</td><td char=".">0.27<p><graphic href="cede_a_2309282_ilm0009.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td><bold>D. Educational Attainment</bold></td></tr><tr><td>Less than High School</td><td char=".">0.20</td><td char=".">0.12</td><td char=".">0.23</td><td char=".">−0.11<p><graphic href="cede_a_2309282_ilm0010.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td>High School</td><td char=".">0.34</td><td char=".">0.30</td><td char=".">0.35</td><td char=".">−0.05</td></tr><tr><td>Some College</td><td char=".">0.03</td><td char=".">0.03</td><td char=".">0.04</td><td char=".">−0.01</td></tr><tr><td>College Graduate</td><td char=".">0.19</td><td char=".">0.18</td><td char=".">0.19</td><td char=".">−0.01</td></tr><tr><td>Some University</td><td char=".">0.01</td><td char=".">0.01</td><td char=".">0.01</td><td char=".">−0.00</td></tr><tr><td>University Graduate</td><td char=".">0.22</td><td char=".">0.37</td><td char=".">0.19</td><td char=".">0.18<p><graphic href="cede_a_2309282_ilm0011.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td /><td>(0.09)</td><td>(0.08)</td><td>(0.09)</td><td char=".">−0.01<p><graphic href="cede_a_2309282_ilm0012.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td><bold>E. Normalized Pre-Treatment Test Scores</bold></td></tr><tr><td>Document-Use Score</td><td char=".">0.00</td><td>−0.46</td><td char=".">0.11</td><td char=".">−0.57<p><graphic href="cede_a_2309282_ilm0013.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td /><td>(1.00)</td><td>(0.74)</td><td>(1.02)</td><td char=".">−0.29<p><graphic href="cede_a_2309282_ilm0014.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td>Numeracy Score</td><td char=".">0.00</td><td>−0.32</td><td char=".">0.08</td><td char=".">−0.40<p><graphic href="cede_a_2309282_ilm0015.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td /><td>(1.00)</td><td>(0.82)</td><td>(1.02)</td><td char=".">−0.21<p><graphic href="cede_a_2309282_ilm0016.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td>Reading Score</td><td char=".">0.00</td><td>−0.44</td><td char=".">0.11</td><td char=".">−0.54<p><graphic href="cede_a_2309282_ilm0017.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td /><td>(1.00)</td><td>(0.68)</td><td>(1.04)</td><td char=".">−0.36<p><graphic href="cede_a_2309282_ilm0018.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td>Average Score</td><td char=".">0.00</td><td>−0.45</td><td char=".">0.11</td><td char=".">−0.56<p><graphic href="cede_a_2309282_ilm0019.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td /><td>(1.00)</td><td>(0.67)</td><td>(1.03)</td><td char=".">−0.36<p><graphic href="cede_a_2309282_ilm0020.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td><bold>F. Attrition</bold></td></tr><tr><td>Started Phase II</td><td char=".">0.42</td><td char=".">0.97</td><td char=".">0.29</td><td char=".">0.68<p><graphic href="cede_a_2309282_ilm0021.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td>Started Phase III</td><td char=".">0.30</td><td char=".">1.00</td><td char=".">0.13</td><td char=".">0.87<p><graphic href="cede_a_2309282_ilm0022.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td>Took 2nd TOWES</td><td char=".">0.19</td><td char=".">1.00</td><td char=".">0.00</td><td char=".">–</td></tr><tr><td>Observations</td><td>1,613</td><td>313</td><td>1,300</td><td /></tr></tbody></table> </ephtml> </p> <p>1 Notes: Standard deviations in parentheses.</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mi /><mrow><mo>∗</mo></mrow></msup></math> </ephtml> significant at the 10% level;</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mi /><mrow><mo>∗</mo><mo>∗</mo></mrow></msup></math> </ephtml> significant at the 5% level;</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math> </ephtml> significant at the 1% level.</p> <p>In Section 5.3, we estimate a multinomial logit model to investigate further the potential heterogeneity within dropouts. Our data are rich enough to investigate whether participants who drop out of the program early (i.e. before entering phase three) are similar to those who completed a significant portion of the program. If individuals enter the program hoping to find a job, we could expect individuals receiving job offers they deem reasonable during the program to drop out. If that is the case, we could see that people who have historically faced fewer barriers to the labour market drop out earlier.</p> <p>Dropouts also quit the program for various reasons, some of which could be associated with a good outcome. For example, some participants sometimes quit the program because they found a job. However, other outcomes, such as going back to school or going back to job search/planning (outside of the FWSP), are harder to assess. Our data also allow us to investigate the predictors of these different choices made by dropouts.</p> <p>The estimated model when investigating the predictors of the dropout timing and reason for dropping out takes the following form:</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>P</mi><mo>(</mo><msub><mi>Y</mi><mrow><mi>iym</mi></mrow></msub><mo>=</mo><mi>j</mi><mo fence="false">|</mo><mrow><msub><mi mathvariant="bold-italic">X</mi><mrow><mi>i</mi></mrow></msub></mrow><mo>,</mo><mi>y</mi><mo>,</mo><mi>m</mi><mo>)</mo><mo>=</mo><mrow><mfrac><mrow><mi>exp</mi><mo>(</mo><msub><mi>α</mi><mi>j</mi></msub><mo>+</mo><msub><mi>λ</mi><mi>j</mi></msub><msub><mi>S</mi><mi>i</mi></msub><mo>+</mo><mrow><msub><mi mathvariant="bold-italic">X</mi><mrow><mi>i</mi></mrow></msub><msub><mtext fontfamily="times">β</mtext><mrow><mi>j</mi></mrow></msub></mrow><mo>+</mo><msub><mi>ρ</mi><mrow><mi>jy</mi></mrow></msub><mo>+</mo><msub><mi>π</mi><mrow><mi>jm</mi></mrow></msub><mo>)</mo></mrow><mrow><mn>1</mn><mo>+</mo><munderover><mo>∑</mo><mrow><mi>h</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>J</mi></mrow></munderover><mi>exp</mi><mo>(</mo><msub><mi>α</mi><mi>h</mi></msub><mo>+</mo><msub><mi>λ</mi><mi>h</mi></msub><msub><mi>S</mi><mi>i</mi></msub><mo>+</mo><mrow><msub><mi mathvariant="bold-italic">X</mi><mrow><mi>i</mi></mrow></msub><msub><mtext fontfamily="times">β</mtext><mrow><mi>h</mi></mrow></msub></mrow><mo>+</mo><msub><mi>ρ</mi><mrow><mi>hy</mi></mrow></msub><mo>+</mo><msub><mi>π</mi><mrow><mi>hm</mi></mrow></msub><mo>)</mo></mrow></mfrac></mrow><mo>,</mo></math> </ephtml> (<reflink idref="bib2" id="ref37">2</reflink>)</p> <p>where <emph>j</emph> represents the different options chosen by the participant (e.g. complete the program, drop early, or drop late, in the case of the timing decision). We include 'Complete the program' as a possible choice that we will use as a baseline outcome in order to have a clearer interpretation of the potential choice predictors. The regressors used in the multinomial logit model are the same as in Equation (<reflink idref="bib1" id="ref38">1</reflink>).</p> <hd id="AN0183842095-6">5. Findings</hd> <p></p> <hd id="AN0183842095-7">5.1. Are completers and dropouts similar?</hd> <p>Table 1 contains the descriptive statistics that are broken down into those who completed the FWSP and those who did not. The first column lists the values for the entire population. The right-most column contains the differences in the values of the sub-population means along with the corresponding level of statistical significance.</p> <p>Most participants (56 percent) are females, 43 percent are immigrants, and slightly more than half are native speakers. About 60 percent are aged between 25 and 44 years. Participants are by no means poorly educated: 45 percent have some post-secondary education experience, and 22 percent have graduated from a university. Panel E presents the normalized TOWES scores for all phase one participants. Panel F of Table 1 highlights the very high incidence of dropping out (and thus attrition with respect to post-program TOWES scores); only 19 percent of participants took the TOWES a second time, which constitutes official completion of the FWSP.</p> <p>The most remarkable differences between those who completed the intervention (column 2) and those who did not (column 3) involve the attributes of gender, language, immigrant status, and TOWES scores. Whereas women comprise 72 percent of the completers, they account for only 52 percent of the dropouts. Native speakers dominate the group of non-completers, whereas the reverse is true for completers. Almost two-thirds of the completer group are immigrants, while only 38 percent of the non-completers are immigrants. The patterns for educational attainment are noticeable at both the highest and the lowest levels. The share of those with less than a high-school diploma is higher among the non-completers, while the share of university graduates is much higher among the completers.</p> <p>Panel E shows that dropouts had much higher initial TOWES scores than completers: completers scored one-third to one-half of a standard deviation below the average TOWES score. Importantly, TOWES scores hint that there is greater heterogeneity with respect to employability skills within dropouts. The standard deviations of all four TOWES scores are significantly smaller for completers than for dropouts.</p> <p>Figure 1 shows that the TOWES scores distributions differ greatly across the two groups, even after taking into account observable characteristics differences. More precisely, the residuals plotted in Figure 1 are obtained after regressing the normalized TOWES scores on the age, education-attainment, mother-tongue, and immigrant status categorical variables. The completer-dropout distribution differences in Figure 1 can therefore be interpreted as performance differences that are not explained by the observable characteristics.[<reflink idref="bib14" id="ref39">14</reflink>] In all cases, the distribution of scores of those who took the TOWES only once (and eventually dropped out) dominates. Formal statistical tests (the Kolmogorov-Smirnov test) for the equivalence of the two distributions handily reject the null hypothesis (results not shown) of the absence of a difference. Whereas the distributions for the group who took the test twice (completers) appear to be bell-shaped, the distributions for the dropouts are left-skewed (especially for the unconditional normalized TOWES scores (Online Appendix Figure A.1).</p> <p>Graph: Figure 1. Residualized Normalized TOWES Scores by Subject and Participant Type. Note: These figures plot the residuals from regressing the normalized TOWES scores on the age, education-attainment, mother-tongue, and immigrant status categorical variables.</p> <p>Table 2 contains the results of a more in-depth comparative analysis of the TOWES raw scores. We observe the initial scores for the entire population; in panel A.1 for the 313 completers, and in panel A.2 for the 1,300 dropouts. In all three domains, the median scores of the non-completers exceed those of the completers by approximately the same magnitude. The same pattern is found at other points of the distribution, such as the top and the bottom quartiles. As a comparison, the dropouts' median scores are close to, if not above, the completers' top-quartile score. Panel B of Table 2 displays the post-treatment test scores, which are available only for the completers. Comparing these final scores with those of the initial (and only) scores of the non-completer group (panel A.2) indicates that, with the exception of the bottom quartile, the latter group fares slightly better. Even after having completed the program, completers' second-attempt TOWES scores are still lower than dropouts' first-attempt scores.</p> <p>Table 2. Pre- and Post-Treatment TOWES Scores.</p> <p> <ephtml> <table><thead valign="bottom"><tr><td /><td>Document Use</td><td>Numeracy</td><td>Reading</td></tr></thead><tbody><tr><td><bold>A. Initial Scores</bold></td></tr><tr><td>A.1. Completers (took test twice)</td></tr><tr><td>Mean</td><td>212.2</td><td>252.6</td><td>236.7</td></tr><tr><td>Standard Deviation</td><td>32.8</td><td>38.5</td><td>32.2</td></tr><tr><td>Top Quartile</td><td>237</td><td>278</td><td>256</td></tr><tr><td>Median</td><td>212</td><td>251</td><td>238</td></tr><tr><td>Bottom Quartile</td><td>189</td><td>227</td><td>218</td></tr><tr><td>Observations</td><td>313</td><td>313</td><td>313</td></tr><tr><td>A.2. Dropouts (took test once)</td></tr><tr><td>Mean</td><td>237.8</td><td>271.6</td><td>262.5</td></tr><tr><td>Standard Deviation</td><td>45.5</td><td>48.4</td><td>49.3</td></tr><tr><td>Top Quartile</td><td>273</td><td>307</td><td>302</td></tr><tr><td>Median</td><td>242.5</td><td>275</td><td>270</td></tr><tr><td>Bottom Quartile</td><td>208</td><td>239</td><td>226</td></tr><tr><td>Observations</td><td>1,300</td><td>1,300</td><td>1,300</td></tr><tr><td><bold>B. Post-Treatment Scores</bold></td></tr><tr><td>Mean</td><td>235.4</td><td>270.4</td><td>254.8</td></tr><tr><td>Standard Deviation</td><td>33.8</td><td>39.3</td><td>36.2</td></tr><tr><td>Top Quartile</td><td>257</td><td>299</td><td>279</td></tr><tr><td>Median</td><td>236</td><td>272</td><td>255</td></tr><tr><td>Bottom Quartile</td><td>214</td><td>243</td><td>230</td></tr><tr><td>Observations</td><td>313</td><td>313</td><td>313</td></tr></tbody></table> </ephtml> </p> <p>In summary, the typical completer could be characterized as a relatively educated (in formal terms) immigrant female whose native tongue is not English and scored one-third to one-half of a standard deviation below the average TOWES score. Interestingly, except for the educational attainment level, these characteristics (e.g. immigrant status, female, and non-native speaker) are historically associated with greater barriers to the labour market.</p> <p>In terms of consequences regarding the estimation of the effect of the program on, say, finding a job, the basic descriptive statistics presented above highlight the fact that completers and dropouts differ significantly in terms of observable socio-economic and demographic characteristics. This suggests they may also differ in terms of unobservables (Altonji, Elder, and Taber [<reflink idref="bib3" id="ref40">3</reflink>], [<reflink idref="bib4" id="ref41">4</reflink>]; Oster [<reflink idref="bib23" id="ref42">23</reflink>]). Consequently, discarding dropouts or assuming they did not receive the treatment are likely to yield very different results when estimating the impact of the program (especially given the high share of dropouts). This concern is reinforced by the fact that one of the primary outcomes of interest in assessing a program's performance is gaining employment, and that these two groups of participants differ on dimensions that are associated with the probability of finding a job. In particular, given that the TOWES scores measure employability skills, albeit imperfectly, it would not be surprising if dropouts were more likely than completers to find work after the training had they not dropped out.</p> <p>The difference in initial TOWES scores between completers and dropouts that we do observe would also make the estimation of the program's impact on these scores problematic. As mentioned above, we observe post-treatment TOWES scores for completers only, and dropouts have higher initial TOWES scores after controlling for observables. Imagine that participants who were 'lucky' on the initial TOWES attempt – that is, they randomly received higher scores than what was indicative of their true ability – are more likely to drop out of the program. Completers would then be composed of more 'unlucky' participants than the original sample of TOWES test takers. Subsequently on the second test attempt, the observed improvement could be solely due to a 'reversion to the mean' effect. In summary, the initial TOWES score differences between completers and dropouts uncovered here suggest that interpreting the TOWES score improvement as the effect of the program completion would require heroic assumptions.</p> <hd id="AN0183842095-8">5.2. What are the predictors of dropping out?</hd> <p>While the distribution differences presented in Figure 1 do not have a 'causal' interpretation, they do suggest that the choice to drop out is not the result of a random shock. Table 3 presents (based on a linear probability model) parameter estimates for Equation (<reflink idref="bib1" id="ref43">1</reflink>), which is our regression equation for the predictors of dropping out.[<reflink idref="bib15" id="ref44">15</reflink>] Specifications (<reflink idref="bib1" id="ref45">1</reflink>)–(<reflink idref="bib3" id="ref46">3</reflink>) concentrate on the links between socio-demographic characteristics and the likelihood to drop out, while (<reflink idref="bib4" id="ref47">4</reflink>) adds normalized average TOWES scores as a predictor for the dropping-out decision.[<reflink idref="bib16" id="ref48">16</reflink>] The results are, for the most part, in line with the findings based on our comparisons of the descriptive statistics (see Table 1). Females and participants who are not native speakers are less likely to drop out of the program (irrespective of the specification we examine). Importantly, we find that participants with higher initial TOWES scores are more likely to drop out. The TOWES' parameter estimate is not small: a one-standard-deviation increase in the average initial TOWES score is associated with a 6 percentage-point (p.p.) increase in the probability of dropping out. Once we control for the TOWES score, we can see that the parameter estimates for some predictors change significantly (e.g. for Native Speaker, Immigrant, Less than HS, and University Grad.), suggesting that these predictors are correlated with TOWES scores. In particular, we find that university graduates (high school dropouts) are 6.8 p.p. less (4.3 p.p. more) likely to drop out. The finding that those with university diplomas were more likely to complete the program was unexpected. We conjecture that this is a reflection of the fact that many of these individuals obtained their degrees in foreign countries and might lack confidence that they will be recognized by the labour market without supplemental training.</p> <p>Table 3. Participants' Initial Characteristics and Dropout Rate.</p> <p> <ephtml> <table><thead valign="bottom"><tr><td /><td>(1)</td><td>(2)</td><td>(3)</td><td>(4)</td></tr></thead><tbody><tr><td>Female</td><td char=".">−0.121<p><graphic href="cede_a_2309282_ilm0026.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td><td char=".">−0.081<p><graphic href="cede_a_2309282_ilm0027.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td><td char=".">−0.082<p><graphic href="cede_a_2309282_ilm0028.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td><td char=".">−0.073<p><graphic href="cede_a_2309282_ilm0029.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td /><td>(0.019)</td><td>(0.019)</td><td>(0.019)</td><td>(0.019)</td></tr><tr><td>Age 15–24</td><td char=".">0.041</td><td char=".">0.000</td><td char=".">−0.003</td><td char=".">0.007</td></tr><tr><td /><td>(0.031)</td><td>(0.031)</td><td>(0.031)</td><td>(0.031)</td></tr><tr><td>Age 25–34</td><td char=".">−0.006</td><td char=".">−0.011</td><td char=".">−0.014</td><td char=".">−0.021</td></tr><tr><td /><td>(0.026)</td><td>(0.025)</td><td>(0.025)</td><td>(0.025)</td></tr><tr><td>Age 45–54</td><td char=".">−0.022</td><td char=".">−0.026</td><td char=".">−0.028</td><td char=".">−0.030</td></tr><tr><td /><td>(0.027)</td><td>(0.027)</td><td>(0.027)</td><td>(0.026)</td></tr><tr><td>Age 55–64</td><td char=".">0.024</td><td char=".">0.001</td><td char=".">−0.005</td><td char=".">−0.007</td></tr><tr><td /><td>(0.038)</td><td>(0.036)</td><td>(0.036)</td><td>(0.036)</td></tr><tr><td>Native Speaker</td><td char="." /><td char=".">0.138<p><graphic href="cede_a_2309282_ilm0030.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td><td char=".">0.123<p><graphic href="cede_a_2309282_ilm0031.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td><td char=".">0.083<p><graphic href="cede_a_2309282_ilm0032.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td /><td char="." /><td>(0.032)</td><td>(0.033)</td><td>(0.033)</td></tr><tr><td>Immigrant</td><td char="." /><td char=".">−0.071<p><graphic href="cede_a_2309282_ilm0033.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td><td char=".">−0.062<p><graphic href="cede_a_2309282_ilm0034.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td><td char=".">−0.034</td></tr><tr><td /><td char="." /><td>(0.031)</td><td>(0.031)</td><td>(0.031)</td></tr><tr><td>Less than HS</td><td char="." /><td char="." /><td char=".">0.018</td><td char=".">0.043<p><graphic href="cede_a_2309282_ilm0035.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td /><td char="." /><td char="." /><td>(0.023)</td><td>(0.024)</td></tr><tr><td>Some College</td><td char="." /><td char="." /><td char=".">0.033</td><td char=".">−0.001</td></tr><tr><td /><td char="." /><td char="." /><td>(0.048)</td><td>(0.046)</td></tr><tr><td>College Grad.</td><td char="." /><td char="." /><td char=".">0.022</td><td char=".">0.009</td></tr><tr><td /><td char="." /><td char="." /><td>(0.027)</td><td>(0.027)</td></tr><tr><td>Some University</td><td char="." /><td char="." /><td char=".">0.020</td><td char=".">0.006</td></tr><tr><td /><td char="." /><td char="." /><td>(0.078)</td><td>(0.080)</td></tr><tr><td>University Grad.</td><td char="." /><td char="." /><td char=".">−0.046</td><td char=".">−0.068<p><graphic href="cede_a_2309282_ilm0036.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td /><td char="." /><td char="." /><td>(0.032)</td><td>(0.032)</td></tr><tr><td>Norm. Ave. Initial TOWES</td><td char="." /><td char="." /><td char="." /><td char=".">0.060<p><graphic href="cede_a_2309282_ilm0037.gif" content-type="Graph" /><math xmlns="http://www.w3.org/1998/Math/MathML"><msup xmlns=""><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math></p></td></tr><tr><td /><td char="." /><td char="." /><td char="." /><td>(0.010)</td></tr><tr><td>Month and Year F.E.</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td></tr><tr><td>Observations</td><td>1,613</td><td>1,613</td><td>1,613</td><td>1,613</td></tr></tbody></table> </ephtml> </p> <p>2 Notes: The dependent variable is a dummy variable equal to 1 if the participant dropped out of the program and 0 otherwise. The specifications are estimated by OLS (Linear Probability Model). Native Speaker is a dummy variable equal to 1 if the participant's mother tongue is English, and zero otherwise. Ave. Norm. Initial TOWES is the normalized average initial (document-use, numeracy and reading) TOWES score. U.R. is the participant's age-specific (e.g. age 25–34) provincial unemployment rate of the month that he/she entered the program. The omitted categories are Age 35–44 for age, and HS graduates for educational attainment. Heteroskedasticity-robust standard errors are in parentheses.</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mi /><mrow><mo>∗</mo></mrow></msup></math> </ephtml> significant at the 10% level;</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mi /><mrow><mo>∗</mo><mo>∗</mo></mrow></msup></math> </ephtml> significant at the 5% level;</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mi /><mrow><mo>∗</mo><mo>∗</mo><mo>∗</mo></mrow></msup></math> </ephtml> significant at the 1% level.</p> <hd id="AN0183842095-9">5.3. Why and at which phase do participants drop out?</hd> <p>We now examine the structure of the drop-out process from the FWSP, starting from the initial population of participants by characterizing the sequence of the flows. Table 4 presents the outcome distribution of participants at each phase as well as the distribution for their 'final' outcome.[<reflink idref="bib17" id="ref49">17</reflink>] It is important to note that, in the case of most participants, for reasons explained above related to the structure of the data, we have to treat all reported outcomes as final except for the event of moving on to the next phase. For example, if someone found a job by the end of phase one, we do not observe her/him during phases two and three. This pattern may seem natural for an outcome such as finding a job or going back to school, which are typically considered to be final outcomes. For this particular program, however, this is also the case for outcomes that one might normally consider as being intermediate, such as returning to career planning/search outside the program, which we treat as a mode of dropping out. Nevertheless, the information presented in Table 4 sheds some light on the potential issue related to partial treatment that dropouts represent.</p> <p>Table 4. Flows Through the Individual Phases.</p> <p> <ephtml> <table><thead valign="bottom"><tr><td>Outcomes</td><td>Proportion</td></tr></thead><tbody><tr><td><bold>A. Phase 1 (<italic>n</italic> = 1,613)</bold></td></tr><tr><td>Entered Phase 2</td><td char=".">0.422</td></tr><tr><td>Found a Job</td><td char=".">0.095</td></tr><tr><td>Returned to School</td><td char=".">0.177</td></tr><tr><td>Returned to Career Planning/Search</td><td char=".">0.136</td></tr><tr><td>Entered Phase 3</td><td char=".">0.019</td></tr><tr><td>Incomplete</td><td char=".">0.150</td></tr><tr><td><bold>B. Phase 2 (<italic>n</italic> = 681)</bold></td></tr><tr><td>Entered Phase 3</td><td char=".">0.652</td></tr><tr><td>Found a Job</td><td char=".">0.062</td></tr><tr><td>Returned to School</td><td char=".">0.104</td></tr><tr><td>Returned to Career Planning/Search</td><td char=".">0.069</td></tr><tr><td>Incomplete</td><td char=".">0.113</td></tr><tr><td><bold>C. Phase 3 (<italic>n</italic> = 479)</bold></td></tr><tr><td>Found a Job</td><td char=".">0.132</td></tr><tr><td>Returned to School</td><td char=".">0.509</td></tr><tr><td>Returned to Career Planning/Search</td><td char=".">0.198</td></tr><tr><td>Incomplete</td><td char=".">0.161</td></tr><tr><td><bold>D. Final Outcome (<italic>n</italic> = 1,613)</bold></td></tr><tr><td>Found a Job</td><td char=".">0.160</td></tr><tr><td>Returned to School</td><td char=".">0.371</td></tr><tr><td>Returned to Career Planning/Search</td><td char=".">0.224</td></tr><tr><td>Incomplete</td><td char=".">0.245</td></tr></tbody></table> </ephtml> </p> <p>3 Notes: 'Returned to Career Planning/Search' includes actively working with case manager, participating in career planning, or participating in job search. In phase 3 and Final outcomes, 'Returned to School' also includes a small number of participants who switched to another program (representing 5.9% of Phase 3 and 1.7% of all participants, respectively.</p> <p>For each of the three phases, we divided the participants' outcomes into five categories: (<reflink idref="bib1" id="ref50">1</reflink>) moved on to the next phase, (<reflink idref="bib2" id="ref51">2</reflink>) found a job, (<reflink idref="bib3" id="ref52">3</reflink>) returned to school, (<reflink idref="bib4" id="ref53">4</reflink>) returned to career planning/search outside the program,[<reflink idref="bib18" id="ref54">18</reflink>] or (<reflink idref="bib5" id="ref55">5</reflink>) had no clear reason for leaving the program. This last residual category (labelled 'Incomplete' in Table 4) involves exiting the FWSP, and it includes items such as not completing the phase, missing observations (no reporting), reporting 'no activity' or 'other.'[<reflink idref="bib19" id="ref56">19</reflink>]</p> <p>The first line of Panel A of Table 4 indicates that fewer than half of phase one participants commenced phase two. 9.5 percent found a job during or shortly after phase one, while 17.7 percent returned to school. An encouraging finding is that a significant fraction of the participants who did not continue on to phase two had what appear to be 'desirable' outcomes. In terms of the rate of job transition for these participants, the figure compares favourably to Brochu and Green ([<reflink idref="bib6" id="ref57">6</reflink>])'s findings for all of Canada, but there is a caveat to this comparison. Those authors find an (overall) average monthly hiring rate (conditional on previously being initially out of work) of 8.1 percent for the 1979-2008 period.[<reflink idref="bib20" id="ref58">20</reflink>] In our case, the information is obtained during a 12-week follow-up period, so we cannot assume that individuals who found a job did so within a month. Nevertheless, given that the 75 percent of individuals who entered phase two did so within 32 days of starting phase one (and 50 percent did so within 20 days), and if we assume that the 9.5 percent of individuals who found a job dropped out after finding a job (not before), then the evidence suggests that they found jobs fairly rapidly. A small number advanced directly to phase 3, and 13.6 percent 'returned to career/planning search,' presumably under the purview of their external care worker. A nontrivial fraction (15.0 percent) constitutes the 'incomplete' category, who left the program, and for whom no further information is available regarding their post-exit destination.</p> <p>Panel B indicates that 65.2 percent of the participants in phase two entered phase three, which is a much higher continuation rate than was the case between the first two phases. 6.2 percent found a job before the end of phase two, 10.4 percent returned to school, 6.9 returned to career planning/search, and 11.3 percent did not complete phase two. Panel C shows that 13.2 percent of phase three participants found a job before the end of phase three, while slightly over half returned to school. 16.1 percent did not complete phase three, and 19.8 percent of them left to work with an external case manager.</p> <p>These figures indicate that much of the dynamics as well as the dropping out occurred during phase one. Looking at the final, cumulative outcomes in panel D reveals that among those who eventually found a job, 59.3 percent (= 9.5/16.0 based on the full sample) did so during phase one. Among those who eventually returned to school, a little less than half (17.7/37.1) did so during phase one. Given that we have already seen that participants with higher initial TOWES scores tended not to continue in the program through phase three, these figures dovetail with the conjecture that participants with higher foundational skill levels may be finding jobs more easily or deciding to go back to school sooner. While these individuals could be classified as receiving little or no training given the nature of phase one, there remains a nontrivial proportion of dropouts who left the program during phases two and three. Thus, individuals who drop out later in the program receive partial treatment.</p> <hd id="AN0183842095-10">5.3.1. Are dropouts heterogeneous with respect to when they leave the program?</hd> <p>We now investigate whether there is evidence of significant heterogeneity across dropouts based on when they leave the program. We define 'early dropouts' as individuals who left the program before entering phase two, which corresponds to 57.8 percent of our observations. Recall that phase one is by far the shortest one of the program and the least costly for the FSWP to administer and deliver. We begin with a short descriptive analysis focused on TOWES performance and then move on to estimating the multinomial logit model presented in Equation (<reflink idref="bib2" id="ref59">2</reflink>).</p> <p>Figure 2 compares the TOWES score distribution of those who quit the program early to those who quit later. We also present the distribution for individuals who completed the program as a reference. Figure 2 suggests that the TOWES score distribution for dropouts presented in Figure 1 is composed of two significantly different distributions. Early dropouts did significantly better on average than participants who dropped out late, but the distribution of the former group also exhibits greater variability in terms of employability skills. Nevertheless, late dropouts did significantly better than completers.[<reflink idref="bib21" id="ref60">21</reflink>] Online Appendix Table A.1 also shows that the two groups differ in terms of socio-demographic characteristics in the same way completers and dropouts differ. Early dropouts are less likely than late dropouts to come from groups facing greater barriers to the labour market. Such descriptive evidence suggests the existence of significant heterogeneity within dropouts.</p> <p>Graph: Figure 2. Residualized Normalized TOWES Scores by Participant Program Progress. Note: These figures plot the residuals from regressing the normalized TOWES scores on the age, education-attainment, mother-tongue, and immigrant status categorical variables.</p> <p>Turning to the predictors of the different program stages at which people leave the program, Figure 3 presents the estimated average marginal effects from the multinomial logit model whose outcome is the stage at which the participants left the program (i.e. completed the program, dropped early, or dropped late). Full regression results are listed in Online Appendix Table A.2.[<reflink idref="bib22" id="ref61">22</reflink>] Since the estimates for completers are essentially the mirror image of those for dropouts in Table 3, we focus now on the predictors of early and late dropouts. Columns (<reflink idref="bib2" id="ref62">2</reflink>) and (<reflink idref="bib3" id="ref63">3</reflink>) of Figure 3 suggest that age does not predict when a participant will leave the program. All else equal, females and immigrants are less likely to drop out early (by 11.0 and 7.7 p.p., respectively). Individuals who have not completed high school are also 6.0 p.p. less (and 4.3 p.p. more) likely to drop out early (late). On the other hand, people with higher initial TOWES scores are significantly more (less) likely to drop out early (late). Such a finding highlights one of the main challenges faced by ALMPs when tailoring their program to the needs of specific groups: striking a balance between not appearing to be too elementary to appeal those with high employability skills while not being too challenging for those with lower skills. For example, if the intended beneficiaries were lower-skilled individuals, our findings suggest that the dropout rate could be reduced by targeting more individuals with lower expected employability skills (e.g. high-school dropouts), and tailoring the program to meet their needs.</p> <p>Graph: Figure 3. Multinomial Logit Model Estimated Average Marginal Effects for Program Progress. Note: The bands correspond to the 90% confidence intervals around the point estimates. If the interval lies outside of the vertical line, the null hypothesis that the average marginal effect equals zero is rejected. Estimates for 'Some College' and 'Some University' are not presented for clarity purposes as they are imprecise, and affect the scale of the x-axis. However, these variables are included in the model and their parameter estimates can be found in Online Appendix Table A.2.</p> <p>While these results suggest that individuals that could be facing more serious obstacles to entering the labour market stay in the program longer (either complete the program or drop out late), there is an exception: all else equal, university graduates are less likely to drop out of the program early. It turns out, however, that this finding is due to the fact that we control for individuals' TOWES scores, and university graduates tend to have higher TOWES scores. If we exclude the measure for these scores from the estimating equation, the parameter estimate for university graduates becomes smaller in magnitude and is no longer statistically significant (see Online Appendix Table A.4).[<reflink idref="bib23" id="ref64">23</reflink>] With this caveat, overall, the results presented in Figure 3 suggest that people facing fewer barriers to the labour market and with higher employability skills are more likely to leave the program early.</p> <hd id="AN0183842095-11">5.3.2. Are dropouts heterogeneous with respect to their post-program destination?</hd> <p>One interpretation of these findings is that, if the goal of participating in FWSP is to find employment, then individuals who can find work more easily will leave the program earlier. However, when looking at the destination of dropouts, the picture becomes more subtle. Figure 4 presents the TOWES score distribution by participant destination. Individuals who found a job performed worse than those who went back to school, and their score distribution is not statistically different from those who went back to career planning/search (outside of the FWSP) or those labelled 'incomplete.' The distribution for those who returned to school is statistically different from that of any other group (based on pairwise Kolmogorov-Smirnov tests). As in the case of the different stages people leave the program, dropouts seem to differ significantly in terms of employability skills based on their post-exit destination.</p> <p>Graph: Figure 4. Residualized Normalized TOWES Scores by Participant Destination. Note: These figures plot the residuals from regressing the normalized TOWES scores on the age, education-attainment, mother-tongue, and immigrant status categorical variables.</p> <p>Figure 5 presents the results from a multinomial logit model for which the outcomes are 'completing the program,' 'finding a job,' 'going back to school,' 'participating in job search/planning,' or leaving the program without giving a reason ('incomplete').[<reflink idref="bib24" id="ref65">24</reflink>] As in the case of when an individual leaves the program, there is not a clear link between age and the reason for dropping out. Females are less likely to quit the program for a job. In contrast with the conjecture that people facing fewer barriers to the labour market and with higher employability skills would leave the program early to start a job, the results in Column (<reflink idref="bib2" id="ref66">2</reflink>) suggest otherwise. Not surprisingly, individuals who already possess a college or university degree are less likely to go back to school (by 6.3 and 6.2 p.p., respectively). However, conditional on educational attainment, higher initial TOWES scores are associated with a greater probability of returning to school. Note that these findings are similar when we focus on the group of early dropouts. The main difference that is apparent in Online Appendix Table A.6, for which the estimating sample is restricted to early dropouts, is that early dropouts with higher TOWES scores are less likely to quit for a job, which is in line with the findings in Figure 4. While these results are not in line with an outside-option story (e.g. finding a job), it could be that higher TOWES scores send a positive signal to participants about their 'academic' abilities.</p> <p>Graph: Figure 5. Multinomial Logit Model Estimated Average Marginal Effects for Destination Outcome. Note: The bands correspond to the 90% confidence intervals around the point estimates. If the interval lies outside of the vertical line, the null hypothesis that the average marginal effect equals zero is rejected. Estimates for 'Some College' and 'Some University' are not presented for clarity purposes as they are imprecise, and affect the scale of the x-axis. However, these variables are included in the model and their parameter estimates can be found in Online Appendix Table A.3.</p> <hd id="AN0183842095-12">6. Conclusion</hd> <p>In this paper, we analyze one aspect of dynamic selection for a foundational learning program, namely the dropout decision. In addition to exploiting information on observable attributes of participants and their outcomes, we examine the patterns of heterogeneity within these clients with respect to the timing of the dropout choice as well as with respect to their destination outcomes. Overall, we find that individuals associated with groups who tend to face greater barriers to entering the labour market (e.g. females, immigrants, and non-native speakers) are more likely to complete the entire program. Importantly, individuals with higher TOWES scores, which is one indicator of employability, are more likely to drop out of the program. This variable yields information in addition to the observable characteristics of the completers, the late dropouts, and the early dropouts on the one hand, and between those who found a job, returned to school, or returned to their original case worker on the other hand.</p> <p>These differences have important consequences for estimating the impact of the program, whether one uses job-finding, returns to school, or the TOWES-score improvement as the outcome of interest. Our findings suggest that in cases for which the researchers have access to post-treatment outcomes for dropouts (and completers), they should pay particular attention to how they treat dropouts. Estimates for which dropouts are considered as treated, partially treated, or as part of the control group are likely to be significantly different, and analyzing them could be informative in regard to the robustness of their findings. When post-treatment outcome information is not available, and there is significant dropping out, as is common in adult learning and training programs, our findings highlight that any causal interpretation of outcome improvement of completers would require imposing assumptions of questionable validity.</p> <p>The design of this program could be modified in order to facilitate the evaluation of its impact. Unlike virtually all of the studies that we cite above, the degree of treatment is measured by the program stage rather than by the duration of time. There does exist a sharp dividing line between the low-cost, initial assessment phase, during which a majority of the dropouts have been determined, and the subsequent, more costly treatment phases. Control groups could be formed after phase one based on the observable characteristics of those treated individuals who are observed moving from phase one to phase two. That would reduce the degree of heterogeneity within the control group and render it more comparable to the treatment groups. If that were the point of departure for evaluations, measured subsequent attrition would be lower. More generally, we conclude that collecting (as much as possible) pre- and post-program information on program dropouts is essential to construct more solid evidence on the effectiveness of active labour market policies.</p> <p>The implications of our study go beyond the improvement of existing learning programs. We provide indications for designing ALMPs customized to the needs of specific groups. Specifically, we find suggestive evidence that FWSP's overall dropout rate could be reduced by either (<reflink idref="bib1" id="ref67">1</reflink>) targeting more individuals with lower expected employability skills, and adjusting the program to meet their specific needs, or (<reflink idref="bib2" id="ref68">2</reflink>) making the initial skills evaluation more challenging to make the program more appealing for individuals with higher employability skills.</p> <p>Finally, our findings have consequences for policymakers. In their survey of ALMPs, Crépon and den Berg ([<reflink idref="bib9" id="ref69">9</reflink>]) claim that the actual implementation of policies is an important yet understudied question, and that optimized targeting can lead to improved outcomes. At phase one, using the TOWES test appears to serve as an inexpensive screening mechanism that can direct less needy clients (facing relatively light barriers) towards productive alternatives outside of the program, and away from the more substantive but costly phases within the program. Resources could be allocated towards the provision of counselling services in phases two and three of this program. That indicator and other information, such as labour market histories or prior recourse to social insurance programs, can help predict future gains and dropout choices, both of which should be taken into consideration when recruiting participants. One curious yet potentially useful finding is that immigrants with university degrees have a relatively high incidence of completion. As suggested by Dalla-Zuanna and Liu ([<reflink idref="bib10" id="ref70">10</reflink>]) in the context of the US Job Corps training programs, targeting more aggressively individuals from groups who face greater barriers to labour market integration could simultaneously decrease dropout rates and direct more resources to individuals who are more likely to benefit from the program.</p> <hd id="AN0183842095-13">Supplemental Material</hd> <p>Supplemental data for this article can be accessed online at <ulink href="http://dx.doi.org/10.1080/09645292.2024.2309282">http://dx.doi.org/10.1080/09645292.2024.2309282</ulink>.</p> <hd id="AN0183842095-14">Acknowledgments</hd> <p>This paper emerged from plans to evaluate the FWSP intervention for which some results were circulated in a prior working paper entitled 'An analysis of a foundational learning program in BC: the Foundations Workplace Skills Program (FWSP) at Douglas College.' We gratefully acknowledge financing from Human Resources and Skills Development Canada. We have benefitted from the input of Pierre Brochu, Karen Myers, Craig Riddell, Arthur Sweetman, and Jean-Pierre Voyer. The assistance of Pam Tetarenko of Douglas College, who provided us with the data set and informed us about the program, was instrumental. All errors are our own.</p> <hd id="AN0183842095-15">Disclosure statement</hd> <p>No potential conflict of interest was reported by the author(s).</p> <ref id="AN0183842095-16"> <title> Notes </title> <blist> <bibl id="bib1" idref="ref2" type="bt">1</bibl> <bibtext> The literature generally makes a distinction between dropouts and attrition. For example, if a participant does not complete the program, but we still observe her outcome of interest (say, her employment status) after the program's end date, we would label this participant as a 'dropout.' If we do not observe the outcome of interest, we would label this participant as an 'attritor,' whether or not she completes the program. In this paper, we observe some post-treatment outcomes (e.g. employment status) but not others (e.g. post-treatment test scores) for individuals who did not complete the program. As such, our program 'non-completers' will be either 'attritors' or 'dropouts' depending on the outcome we are considering.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref4" type="bt">2</bibl> <bibtext> This scarcity contrasts with the literature on school dropouts, which is quite large. See, e.g. Agasisti, Bolzoni, and Soncin ([1]), Eegdeman et al. ([13]), and Hermann and Horn ([18]) and references therein.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref14" type="bt">3</bibl> <bibtext> Fitzenberger, Osikominu, and Paul ([15]) address the choice of program starting time and potentially positive and negative selection into dropout status. Both entries and exits from a German job training programs are endogenously modelled. Based on the same data set, Biewen et al. ([5]) take into account dynamic selection into different programs. Dalla-Zuanna and Liu ([10]), which is based on US data, deals with variation in time spent in treatment and the potential selection out of the primary intervention into alternative or additional programs. Mealli, Pudney, and Thomas ([22]) involves a long-term job training program in the UK. In their joint estimation of the outcomes of the duration of the training spell and the destination state, they observe early termination, but they do not model it. They determine that the impact of that event on the estimated transition probabilities into employment is negative relative to the event of completion, and they do not examine differences between completers and early terminators.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref19" type="bt">4</bibl> <bibtext> For an informative descriptive summary of Canada's rather complicated array of public training and skills development programs, see Jansen et al. ([19]).</bibtext> </blist> <blist> <bibl id="bib5" idref="ref11" type="bt">5</bibl> <bibtext> Reading text, document use, numeracy, writing, thinking skills, oral communication, continuous learning, working with others, and computer use.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref21" type="bt">6</bibl> <bibtext> Examples include SUCCESS (United Chinese Community Enrichment Services Society), ISS (Immigrant Services Society), Options Community Services, and PCRS (Pacific Community Resources Society).</bibtext> </blist> <blist> <bibl id="bib7" idref="ref22" type="bt">7</bibl> <bibtext> Those authors label that phenomenon 'the service gap'; addressing it is the crux of the FWSP.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref23" type="bt">8</bibl> <bibtext> Interviews with program officials indicated that the only alternative programming that existed at the time was some Basic Literacy programs.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref16" type="bt">9</bibl> <bibtext> The goal is not to develop an entirely new career path.</bibtext> </blist> <blist> <bibtext> A typical schedule might be 16 hours in group activities, 29 hours at home, and 26 hours in lab activities.</bibtext> </blist> <blist> <bibtext> While we do not observe the type of education pursued by those who returned to school, we can infer it using their career goal. The vast majority of them listed career goals requiring either community college (e.g. accounting clerks, health care assistants, licensed practical nurses) or professional training schools (e.g. carpenters, class 1 truck drivers, culinary artists).</bibtext> </blist> <blist> <bibtext> Since linking FSWP participants to their employment records is not feasible, we must rely on the information provided by the participants. When participants dropped out, they either provided that information to program staff directly, or to the case workers who originally referred him/her to the FWSP notified program staff. Unfortunately, no information in the data allows us to identify who provided the information (e.g. the participant or the case worker). This missing information should not affect the analysis of the stage at which participants leave the program, as the program administrators track this information. In particular, if individuals lied about having found a job, it would affect our descriptive statistics. Whether it would affect our regression analysis or not depends on whether the likelihood of falsely reporting finding a job is correlated with our predictors. As long as the likelihood of falsely reporting to have found a job is uncorrelated with our predictors, it should not affect our analysis. Alas, there is no way for us to investigate that. Finally, falsely reporting to have returned career planning/search outside the program or failing to complete a phase are not likely, as most of this information is collected by case workers and program administrators.</bibtext> </blist> <blist> <bibtext> Card, Kluve, and Weber ([8]) find that ALMPs have larger impacts in recessionary periods. We have investigated whether labour market conditions can also affect the probability of dropping out of the FWSP. Since age-specific unemployment rates and the month and year dummy variables are highly correlated, including the former as a control variable makes its parameter estimates imprecise. We therefore exclude the age-specific unemployment rate variable from our regressions.</bibtext> </blist> <blist> <bibtext> Plotting the unconditional normalized TOWES scores leads to the same conclusions (see Onlne Appendix Figure A.1).</bibtext> </blist> <blist> <bibtext> While the results presented here are based on a linear probability model, they are very similar (in terms of estimates and statistical significance) to those obtained from a probit or a logistic model (see Online Appendix Tables A.7 and A.8).</bibtext> </blist> <blist> <bibtext> We use the normalized averaged initial TOWES scores to control for TOWES performance given the high positive correlation between the subject-specific TOWES scores.</bibtext> </blist> <blist> <bibtext> As mentioned above, phase one consists of the initial assessment, phase two consists of 'portfolio development' and employability, and phase three deals with the enhancement of foundational skills applied to the work environment.</bibtext> </blist> <blist> <bibtext> Career planning/search includes individuals who work actively with a case manager, participating in career planning or in job search.</bibtext> </blist> <blist> <bibtext> The vast majority of observations in our 'Incomplete' category come from individuals who did not complete the phase. There are 8 missing observations, 6 'no activity,' and 14 'other' in phase one. In phase two, we have 7 missing observations, 0 'no activity,' and 3 'other.' Finally, there are 29 missing observations, 0 'no activity,' and 1 'other' in phase three.</bibtext> </blist> <blist> <bibtext> The hiring rate in Brochu and Green ([6]) is computed using two-month mini-panels constructed from Labour Force Surveys. Hence, these are month-to-month transitions.</bibtext> </blist> <blist> <bibtext> Pairwise Kolmogorov-Smirnov tests reject the assumptions that the distributions are equal at a 1% significance level.</bibtext> </blist> <blist> <bibtext> For the sake of expositional clarity, we do not present the estimates for 'Some College' and 'Some University' in Figure 3 as they are imprecise (due to the small number of observations) and affect the scale of the <emph>x</emph>-axis. They are listed in Online Appendix Table A.2.</bibtext> </blist> <blist> <bibtext> Online Appendix Table A.4 also shows that, if we exclude the control for TOWES scores, native speakers are more likely to drop out early. Again, this comes from the fact that these individuals have significantly higher TOWES scores.</bibtext> </blist> <blist> <bibtext> The estimated average marginal effects are also presented in Online Appendix Table A.3. As for the timing analysis, we do not present the estimates for 'Some College' and 'Some University' in Figure 5, but they can be found in Online Appendix Table A.3.</bibtext> </blist> </ref> <ref id="AN0183842095-17"> <title> References </title> <blist> <bibtext> Agasisti, Tommaso, Filippo Bolzoni, and Mara Soncin. 2023. " Would You Follow the Advice? The Effect of School Guidance on Students' Academic Success." Education Economics 1 – 16. https://doi.org/10.1080/09645292.2023.2239527.</bibtext> </blist> <blist> <bibtext> Altmann, Steffen, Armin Falk, Simon Jäger, and Florian Zimmermann. 2018. " Learning About Job Search: A Field Experiment with Job Seekers in Germany." Journal of Public Economics 164 : 33 – 49. https://doi.org/10.1016/j.jpubeco.2018.05.003.</bibtext> </blist> <blist> <bibtext> Altonji, Joseph G., Todd E. Elder, and Christopher R. Taber. 2005. " Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools." Journal of Political Economy 113 (1): 151 – 184. https://doi.org/10.1086/426036.</bibtext> </blist> <blist> <bibtext> Altonji, Joseph G., Todd E. Elder, and Christopher R. Taber. 2008. " Using Selection on Observed Variables to Assess Bias From Unobservables When Evaluating Swan-Ganz Catheterization." American Economic Review: Papers & Proceedings 98 (2): 345 – 350. https://doi.org/10.1257/aer.98.2.345.</bibtext> </blist> <blist> <bibtext> Biewen, Martin, Bernd Fitzenberger, Aderonke Osikominu, and Marie Paul. 2014. " The Effectiveness of Public-Sponsored Training Revisited: The Importance of Data and Methodological Choices." Journal of Labor Economics 32 (4): 837 – 897. https://doi.org/10.1086/677233.</bibtext> </blist> <blist> <bibtext> Brochu, Pierre, and David A. Green. 2013. " The Impact of Minimum Wages on Labour Market Transitions." The Economic Journal 123 (573): 1203 – 1235. https://doi.org/10.1111/ecoj.12032.</bibtext> </blist> <blist> <bibtext> Card, David, Pablo Ibarrarán, Ferdinando Regalia, David Rosas-Shady, and Yuri Soares. 2011. " The Labor Market Impacts of Youth Training in the Dominican Republic." Journal of Labor Economics 29 (2): 267 – 300. https://doi.org/10.1086/658090.</bibtext> </blist> <blist> <bibtext> Card, David, Jochen Kluve, and Andrea Weber. 2018. " What Works? A Meta Analysis of Recent Active Labor Market Program Evaluations." Journal of the European Economic Association 16 (3): 894 – 931. https://doi.org/10.1093/jeea/jvx028.</bibtext> </blist> <blist> <bibtext> Crépon, Bruno, and Gerard J. van den Berg. 2016. " Active Labor Market Policies." Annual Review of Economics 8 (1): 521 – 546. https://doi.org/10.1146/economics.2016.8.issue-1.</bibtext> </blist> <blist> <bibtext> Dalla-Zuanna, Antonio, and Kai Liu. 2019. "Understanding Program Complementarities: Estimating the Dynamic Effects of a Training Program with Multiple Alternatives." Institute of Labor Economics (IZA) Discussion Paper No. 12839.</bibtext> </blist> <blist> <bibtext> de Crombrugghe, Denis, Henry Espinoza, and Hans Heijke. 2010. "Determinants of Dropout Behaviour in a Job Training Programme for Disadvantaged Youth." ROA Research Memorandum 008, Maastricht University, Research Centre for Education and the Labour Market (ROA).</bibtext> </blist> <blist> <bibtext> Dickson, Matt, and Colm P. Harmon. 2011. " Economic Returns to Education: What We Know, What We Don't Know, and Where We Are Going—Some Brief Pointers." Economics of Education Review 30 (6): 1118 – 1122. https://doi.org/10.1016/j.econedurev.2011.08.003.</bibtext> </blist> <blist> <bibtext> Eegdeman, Irene, Ilja Cornelisz, Martijn Meeter, and Chris van Klaveren. 2023. " Identifying False Positives when Targeting Students At Risk of Dropping Out." Education Economics 31 (3): 313 – 325. https://doi.org/10.1080/09645292.2022.2067131.</bibtext> </blist> <blist> <bibtext> Escudero, Veronica. 2018. " Are Active Labour Market Policies Effective in Activating and Integrating Low-Skilled Individuals? An International Comparison." IZA Journal of Labor Policy 7 (4): 1 – 26.</bibtext> </blist> <blist> <bibtext> Fitzenberger, Bernd, Aderonke Osikominu, and Marie Paul. 2010. "The Heterogeneous Effects of Training Incidence and Duration on Labor Market Transitions." Institute of Labor Economics (IZA) Discussion Paper No. 5269.</bibtext> </blist> <blist> <bibtext> Heckman, James J., Robert J. Lalonde, and Jeffrey A. Smith. 1999. "The Economics and Econometrics of Active Labor Market Programs." In Handbook of Labor Economics, Vol. 3, edited by O. Ashenfelter and D. Card, 1865–2097. Amsterdam: Elsevier.</bibtext> </blist> <blist> <bibtext> Heckman, James J., and Jeffrey A. Smith. 1995. " Assessing the Case for Social Experiments." Journal of Economic Perspectives 9 (2): 85 – 110. https://doi.org/10.1257/jep.9.2.85.</bibtext> </blist> <blist> <bibtext> Hermann, Zoltán, and Dániel Horn. 2023. " The Effect of Decreased General Training on Skills and Dropout – Evidence From a Vocational School Reform in Hungary." Education Economics 31 (6): 725 – 734. https://doi.org/10.1080/09645292.2022.2160432.</bibtext> </blist> <blist> <bibtext> Jansen, Alix J., Linda A. White, Elizabeth Dhuey, Daniel Foster, and Michal Perlman. 2019. " Training and Skills Development Policy Options for the Changing World of Work." Canadian Public Policy 45 (4): 460 – 482. https://doi.org/10.3138/cpp.2019-024.</bibtext> </blist> <blist> <bibtext> Lacroix, Guy, and Dany Brouillette. 2011. " Assessing the Impact of a Wage Subsidy for Single Parents on Social Assistance." Canadian Journal of Economics 44 (4): 1195 – 1221. https://doi.org/10.1111/caje.2011.44.issue-4.</bibtext> </blist> <blist> <bibtext> McCall, Brian, Jeffrey A. Smith, and Conny Wunsch. 2016. "Government-Sponsored Vocational Education for Adults." In Handbook of the Economics of Education, Vol. 5, edited by Stephen Machin Eric A. Hanushek and Ludger Woessmann, 479–652. Amsterdam: Elsevier.</bibtext> </blist> <blist> <bibtext> Mealli, Fabrizia, Stephen Pudney, and Jonathan Thomas. 1996. " Training Duration and Post-Training Outcomes: A Duration-Limited Competing Risks Model." The Economic Journal 106 (435): 422 – 433. https://doi.org/10.2307/2235257.</bibtext> </blist> <blist> <bibtext> Oster, Emily. 2019. " Unobservable Selection and Coefficient Stability: Theory and Evidence." Journal of Business & Economic Statistics 37 (2): 187 – 204. https://doi.org/10.1080/07350015.2016.1227711.</bibtext> </blist> <blist> <bibtext> Palameta, Boris, Michael Dowie, Cam Nguyen, and David Gyarmati. 2016. "Foundations: Implementation and 12-Week Impacts of a Literacy and Essential Skills Intervention for Job Seekers." Technical Report, Social Research and Demonstration Corporation (SRDC).</bibtext> </blist> <blist> <bibtext> Palameta, Boris, Cam Nguyen, Taylor Shek wai Hui, and David Gyarmati. 2016. "Foundations: 12-Month Impacts of a Literacy and Essential Skills Intervention for Job Seekers." Technical Report, Social Research and Demonstration Corporation (SRDC).</bibtext> </blist> <blist> <bibtext> Paul, Marie. 2015. " Many Dropouts? Never Mind! -- Employment Prospects of Dropouts From Training Programs." Annals of Economics and Statistics 119/120): 235 – 267. https://doi.org/10.15609/annaeconstat2009.119-120.235.</bibtext> </blist> <blist> <bibtext> Waller, Marie. 2008. "Further Training for the Unemployed: What Can we Learn about Dropouts from Administrative Data?." FDZ Methodenreport 200804, Institute for Employment Research, Nuremberg, Germany.</bibtext> </blist> </ref> <aug> <p>By David Gray and Louis-Philippe Morin</p> <p>Reported by Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib16" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib21" firstref="ref5"></nolink> <nolink nlid="nl3" bibid="bib12" firstref="ref6"></nolink> <nolink nlid="nl4" bibid="bib11" firstref="ref8"></nolink> <nolink nlid="nl5" bibid="bib26" firstref="ref9"></nolink> <nolink nlid="nl6" bibid="bib27" firstref="ref10"></nolink> <nolink nlid="nl7" bibid="bib15" firstref="ref12"></nolink> <nolink nlid="nl8" bibid="bib10" firstref="ref13"></nolink> <nolink nlid="nl9" bibid="bib20" firstref="ref15"></nolink> <nolink nlid="nl10" bibid="bib14" firstref="ref17"></nolink> <nolink nlid="nl11" bibid="bib24" firstref="ref24"></nolink> <nolink nlid="nl12" bibid="bib25" firstref="ref25"></nolink> <nolink nlid="nl13" bibid="bib13" firstref="ref34"></nolink> <nolink nlid="nl14" bibid="bib17" firstref="ref36"></nolink> <nolink nlid="nl15" bibid="bib23" firstref="ref42"></nolink> <nolink nlid="nl16" bibid="bib18" firstref="ref54"></nolink> <nolink nlid="nl17" bibid="bib19" firstref="ref56"></nolink> <nolink nlid="nl18" bibid="bib22" firstref="ref61"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1486002
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Foundational Learning Program Evaluation and Dropouts: Are Dropouts a Heterogeneous Group?
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22David+Gray%22">David Gray</searchLink><br /><searchLink fieldCode="AR" term="%22Louis-Philippe+Morin%22">Louis-Philippe Morin</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Education+Economics%22"><i>Education Economics</i></searchLink>. 2025 33(2):198-217.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 20
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Career+and+Technical+Education%22">Career and Technical Education</searchLink><br /><searchLink fieldCode="DE" term="%22Job+Skills%22">Job Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Employment+Potential%22">Employment Potential</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Evaluation%22">Program Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Dropouts%22">Dropouts</searchLink><br /><searchLink fieldCode="DE" term="%22Career+Readiness%22">Career Readiness</searchLink><br /><searchLink fieldCode="DE" term="%22Labor+Force+Development%22">Labor Force Development</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Attainment%22">Educational Attainment</searchLink><br /><searchLink fieldCode="DE" term="%22Barriers%22">Barriers</searchLink><br /><searchLink fieldCode="DE" term="%22Employment+Programs%22">Employment Programs</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Canada%22">Canada</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/09645292.2024.2309282
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0964-5292<br />1469-5782
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: We analyze the characteristics and outcomes of a Canadian foundational learning program's dropouts and compare them with those of the completers. We find significant heterogeneity within dropouts along two dimensions: when they drop out and why. Individuals whose characteristics have been historically associated with greater labour market barriers, and those with lower employability skills are more likely to complete the program. Individuals who face fewer barriers tend to leave at an early stage, while individuals without a high school degree tend to drop out later. Conditional on education, higher employability-skill participants are more likely to leave and return to school.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1486002
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1486002
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/09645292.2024.2309282
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 20
        StartPage: 198
    Subjects:
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Career and Technical Education
        Type: general
      – SubjectFull: Job Skills
        Type: general
      – SubjectFull: Employment Potential
        Type: general
      – SubjectFull: Program Evaluation
        Type: general
      – SubjectFull: Dropouts
        Type: general
      – SubjectFull: Career Readiness
        Type: general
      – SubjectFull: Labor Force Development
        Type: general
      – SubjectFull: Educational Attainment
        Type: general
      – SubjectFull: Barriers
        Type: general
      – SubjectFull: Employment Programs
        Type: general
      – SubjectFull: Canada
        Type: general
    Titles:
      – TitleFull: Foundational Learning Program Evaluation and Dropouts: Are Dropouts a Heterogeneous Group?
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: David Gray
      – PersonEntity:
          Name:
            NameFull: Louis-Philippe Morin
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0964-5292
            – Type: issn-electronic
              Value: 1469-5782
          Numbering:
            – Type: volume
              Value: 33
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Education Economics
              Type: main
ResultId 1