View in EDS HTML Full Text PDF Full Text

The Speaking Proficiency Outcomes of Face-to-Face and Online Intensive Summer LCTL Programs

Saved in:
Bibliographic Details
Title: The Speaking Proficiency Outcomes of Face-to-Face and Online Intensive Summer LCTL Programs
Language: English
Authors: Dianna Murphy (ORCID 0000-0001-9716-0032), Sonya K. Sedivy (ORCID 0000-0002-5031-7837)
Source: Foreign Language Annals. 2024 57(4):872-899.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 28
Publication Date: 2024
Sponsoring Agency: Office of Postsecondary Education (ED)
Department of Education (ED)
Contract Number: P017A20001421
Document Type: Journal Articles
Reports - Research
Descriptors: Speech Communication, Second Language Learning, Second Language Instruction, Language Proficiency, Oral Language, Language Tests, Arabic, Sino Tibetan Languages, Turkic Languages, Indo European Languages, Dravidian Languages, Portuguese, Indonesian, Turkish, Urdu, Language Variation, Intensive Language Courses, Outcomes of Education, Uncommonly Taught Languages, Online Courses, COVID-19, Pandemics, Summer Programs, In Person Learning, Second Language Programs
Assessment and Survey Identifiers: ACTFL Oral Proficiency Interview
DOI: 10.1111/flan.12758
ISSN: 0015-718X
1944-9720
Abstract: This article is the first large-scale study to document the speaking proficiency outcomes of intensive programs in less commonly taught languages in US higher education. Speaking proficiency was measured by pre- and postprogram ACTFL Oral Proficiency Interviews (N = 484) in 14 languages: Arabic, Bengali, Brazilian Portuguese, Hindi, Indonesian, Kazakh, Persian, Tamil, Thai, Tibetan, Turkish, Urdu, Uyghur, and Uzbek. The intensive programs are usually taught face-to-face but were offered fully online during the COVID-19 pandemic. No significant difference in speaking proficiency outcomes, as measured by the ACTFL OPI, was found between the face-to-face and online formats. Students at all levels of instruction made significant gains in speaking proficiency, with greater gains made by students in Level 1 than at Levels 2-3, and with wide variation within instructional levels. The average speaking proficiency outcomes were Advanced Low (Level 3), Intermediate High (Level 2), and between Intermediate Low and Intermediate Mid (Level 1).
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1452841
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHeKFzqQs0iZAnAFqTe-1OtAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDG5DGGhP1-CWE7-F-AIBEICBm_O6r5GWFVfBvr4iodjXF6AbbJQXCHw7q2lsQmD4d2j0nAtcseWCAZNhufUq8-P8Co56eoU0D0cilYe8KYxLvTuJMg1wi4GEg4SjtZTQoJXo_GO6W1DljDDq2p2sTVQXt9Xy9Zb1dcRrHLYRJHO148SiIjsFMmT4hh3X9ns1eOwKDGK9y5IWQbMiEUDJBm2Wi30L3fzDFAT9lW5S
Text:
  Availability: 1
  Value: <anid>AN0181569079;fla01dec.24;2024Dec13.04:00;v2.2.500</anid> <title id="AN0181569079-1">The speaking proficiency outcomes of face‐to‐face and online intensive summer LCTL programs </title> <p>This article is the first large‐scale study to document the speaking proficiency outcomes of intensive programs in less commonly taught languages in US higher education. Speaking proficiency was measured by pre‐ and postprogram ACTFL Oral Proficiency Interviews (N = 484) in 14 languages: Arabic, Bengali, Brazilian Portuguese, Hindi, Indonesian, Kazakh, Persian, Tamil, Thai, Tibetan, Turkish, Urdu, Uyghur, and Uzbek. The intensive programs are usually taught face‐to‐face but were offered fully online during the COVID‐19 pandemic. No significant difference in speaking proficiency outcomes, as measured by the ACTFL OPI, was found between the face‐to‐face and online formats. Students at all levels of instruction made significant gains in speaking proficiency, with greater gains made by students in Level 1 than at Levels 2‐3, and with wide variation within instructional levels. The average speaking proficiency outcomes were Advanced Low (Level 3), Intermediate High (Level 2), and between Intermediate Low and Intermediate Mid (Level 1).</p> <p>Keywords: distance learning; oral proficiency (OPI and OPIc); post‐secondary/higher education; quantitative research</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/FLA/01dec24/flan12758-gra-0001.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="flan12758-gra-0001.jpg" title="." /> </p> <p></p> <hd id="AN0181569079-3">INTRODUCTION</hd> <p>Research to date on the proficiency outcomes of instructional programs in languages other than English (LOTEs) in US higher education has largely focused on more widely taught languages such as Chinese, French, German, Italian, Japanese, Russian, and Spanish. The present study contributes to this body of research by documenting the speaking proficiency outcomes of intensive summer programs in numerous so‐called "less commonly taught" languages (LCTLs)[<reflink idref="bib1" id="ref1">1</reflink>] for which there is little—if any—published research on proficiency outcomes in a US postsecondary education context, including Arabic, Bengali, Brazilian Portuguese, Hindi, Indonesian, Kazakh, Persian, Tamil, Thai, Tibetan, Turkish, Urdu, Uyghur, and Uzbek. The intensive summer LCTL programs in which the study is situated are typically held in‐person, face‐to‐face (F2F), but were taught fully online during the COVID‐19 global pandemic (Gacs et al., [<reflink idref="bib20" id="ref2">20</reflink>]; Hodges et al., [<reflink idref="bib24" id="ref3">24</reflink>]).[<reflink idref="bib2" id="ref4">2</reflink>] The present study used ratings from ACTFL Oral Proficiency Interviews (ACTFL OPIs) conducted over 5 years (2018–2022) to compare the speaking proficiency outcomes of the programs when offered F2F (2018–2019; 2022) and online in an emergency context (2020–2021); and to document the speaking proficiency outcomes of the 14 LCTL programs by language and level of instruction. In doing so, the study responds to calls for research on the proficiency outcomes of intensive language instruction (Collins & Muñoz, [<reflink idref="bib8" id="ref5">8</reflink>]), of online language instruction, using validated proficiency tests such as the OPI (Lin & Warschauer, [<reflink idref="bib30" id="ref6">30</reflink>]) and especially in LCTLs (Van Deusen‐Scholl, [<reflink idref="bib17" id="ref7">17</reflink>]), and for programs to share the results of speaking proficiency assessments as a "resource for the field" (Malone & Montee, [<reflink idref="bib38" id="ref8">38</reflink>], 984; see also Malone, [<reflink idref="bib36" id="ref9">36</reflink>]).</p> <hd id="AN0181569079-4">LITERATURE REVIEW</hd> <p>Following the initial publication of the provisional <emph>ACTFL Proficiency Guidelines</emph> and accompanying ACTFL OPI[<reflink idref="bib3" id="ref10">3</reflink>] in 1982 (Liskin‐Gasparro, [<reflink idref="bib32" id="ref11">32</reflink>]; Malone & Montee, [<reflink idref="bib38" id="ref12">38</reflink>]; Malone, [<reflink idref="bib37" id="ref13">37</reflink>]), researchers have used the ACTFL OPI and the Oral Proficiency Interview ‐ Computer (OPIc) to assess the speaking proficiency outcomes of LOTE programs in domestic postsecondary learning environments.[<reflink idref="bib4" id="ref14">4</reflink>] Environments included traditional undergraduate LOTE programs (Isbell et al., [<reflink idref="bib25" id="ref15">25</reflink>]; Magnan, [<reflink idref="bib35" id="ref16">35</reflink>]; Swender, [<reflink idref="bib53" id="ref17">53</reflink>]; Tschirner, [<reflink idref="bib55" id="ref18">55</reflink>]; Winke et al., [<reflink idref="bib57" id="ref19">57</reflink>]; Winke et al., [<reflink idref="bib58" id="ref20">58</reflink>]; Zhang et al., [<reflink idref="bib60" id="ref21">60</reflink>]), Language Flagship programs (Davidson & Garas, [<reflink idref="bib15" id="ref22">15</reflink>]; Davidson & Shaw, [<reflink idref="bib13" id="ref23">13</reflink>]; Davidson et al., [<reflink idref="bib10" id="ref24">10</reflink>]; Davidson, [<reflink idref="bib14" id="ref25">14</reflink>], [<reflink idref="bib11" id="ref26">11</reflink>]; Nugent & Slater, [<reflink idref="bib47" id="ref27">47</reflink>]), intensive immersion programs (Ke & Reed, [<reflink idref="bib26" id="ref28">26</reflink>]; Merrill et al., [<reflink idref="bib42" id="ref29">42</reflink>]; Miano et al., [<reflink idref="bib43" id="ref30">43</reflink>]; Rifkin, [<reflink idref="bib51" id="ref31">51</reflink>], [<reflink idref="bib52" id="ref32">52</reflink>]), and university‐sponsored language houses (Martinsen et al., [<reflink idref="bib39" id="ref33">39</reflink>]). With the notable exceptions of Kissau et al. ([<reflink idref="bib29" id="ref34">29</reflink>]) and Merrill et al. ([<reflink idref="bib42" id="ref35">42</reflink>]), there has been surprisingly little research using the ACTFL OPI or unofficial OPI to investigate the outcomes of online or distance LOTE courses (Van Deusen‐Scholl, [<reflink idref="bib17" id="ref36">17</reflink>]). In line with the focus of the present study, the brief literature review that follows is limited primarily to speaking proficiency measured by the ACTFL OPI or OPIc, or unofficial OPIs not conducted by Language Testing International, rated on either the ACTFL or Interagency Language Roundtable (ILR) scale; and to LOTE speaking proficiency research in domestic US higher education (i.e., in "at home" contexts of learning; Dewey, [<reflink idref="bib18" id="ref37">18</reflink>]; Freed et al., [<reflink idref="bib19" id="ref38">19</reflink>]). The literature review is organized by the types of instructional programs in which these studies were conducted: traditional undergraduate LOTE programs, Language Flagship programs, intensive domestic immersion programs, and online LOTE courses.</p> <hd id="AN0181569079-5">Speaking proficiency outcomes of traditional undergraduate LOTE programs</hd> <p>Numerous small‐scale, cross‐sectional studies have used OPI ratings to report on speaking proficiency outcomes in the undergraduate curriculum, often by level of instruction. Magnan ([<reflink idref="bib35" id="ref39">35</reflink>]), for example, found that for students of French (<emph>N</emph> = 40) at a large public university, the median speaking proficiency outcomes on the ACTFL scale were Intermediate Low to Intermediate Mid for first year (two semesters); Intermediate Mid for second year (four semesters); Intermediate High for third year (six semesters), and Advanced for fourth year (eight semesters). Reviewing similar studies for US collegiate students of German, Tschirner ([<reflink idref="bib55" id="ref40">55</reflink>]) reported speaking proficiency outcomes of Novice High or Intermediate Low after 1 year of instruction, Intermediate Low or Intermediate Mid after 2 years; Intermediate Mid or Intermediate High after 3 years, and Intermediate High to Advanced Low after 4 years, depending on whether the students had studied abroad (112‐113). Bernhart et al. ([<reflink idref="bib4" id="ref41">4</reflink>]) found that 70% of students of German reached Intermediate Mid in speaking in the first year; the remaining were assessed at Intermediate Low. In the second year, approximately 50% of the students of German in Bernhart ([<reflink idref="bib4" id="ref42">4</reflink>]) reached Advanced Low; the other 50% reached Intermediate High.[<reflink idref="bib5" id="ref43">5</reflink>]</p> <p>Large‐scale studies of proficiency outcomes in US postsecondary LOTE education to date, based on a small number of languages, have found that the majority of students do not reach the Advanced level in speaking. Swender ([<reflink idref="bib53" id="ref44">53</reflink>]) focusing on the speaking proficiency attainment of undergraduate LOTE majors, reported on the ACTFL OPI ratings of majors in French, German, Italian, Japanese, Mandarin Chinese, Russian, and Spanish (<emph>N</emph> = 501) from five liberal arts colleges. She found that overall, only 47% of the LOTE majors were rated above the ACTFL Advanced border, with the "greatest concentration of ratings (55.8%) in the Intermediate Mid‐Advanced Low range" (p. 523). More recently, three US universities (Michigan State University, the University of Minnesota, and the University of Utah), with funding from the Language Proficiency Flagship Initiative (LPFI), conducted extensive proficiency assessments of close to 9,000 students of Arabic, Chinese, French, German, Korean, Portuguese, and Russian at those universities. (See Gass & Winke, [<reflink idref="bib21" id="ref45">21</reflink>] for a description of the LPFI and of the purpose and scope of the grant funding.) Drawing on data from that study, Winke et al. ([<reflink idref="bib58" id="ref46">58</reflink>]) found that for university students of Arabic, Chinese, French, Portuguese, Russian, and Spanish, speaking was the weakest of all of modalities assessed (i.e., speaking, listening, and reading). In terms of final attainment, Winke et al. ([<reflink idref="bib58" id="ref47">58</reflink>]) found and that "with the exception of Spanish heritage learners, who consistently reached Advanced Low, the majority of the students tested after 4 years of university‐level coursework only attained between Intermediate Mid and Intermediate High proficiency in speaking" (p. 57).</p> <p>Other scholars have looked at LOTE student proficiency development over time, by semester of instruction. Isbell et al. ([<reflink idref="bib25" id="ref48">25</reflink>]), also drawing on data collected for the LPFI, used multiple administrations of the OPIc over the course of four semesters to investigate the speaking proficiency development of 814 students of Chinese, French, Russian, and Spanish. They found that on average, students gained approximately one‐third of an ACTFL sublevel in speaking each semester, after controlling for student backgrounds and learning experiences. There was no significant effect for language of study, which Isbell et al. ([<reflink idref="bib25" id="ref49">25</reflink>]) highlighted as a major result of the research: "differences in OPIc scores across languages were primarily reflective of differences in learner backgrounds and amount of instruction rather than of differences associated with the languages themselves" (p. 457). Similarly investigating student proficiency growth over time, Zhang et al. ([<reflink idref="bib60" id="ref50">60</reflink>]) used LPFI OPIc ratings of 1922 students of Chinese, French, Russian, and Spanish in the second, third, or fourth semester of instruction. They found that students gained an average of one ACTFL sublevel per year, "following a nonlinear developmental path that was mildly accelerating over time" (p. 832). Several variables explained the variation in the students' growth path, including language of study (e.g., students of Spanish outpaced students of Chinese and Russian in the rate of growth in speaking proficiency), prior experience with the language, interest in learning the language, and the importance of speaking to the students.</p> <hd id="AN0181569079-6">Language Flagship programs</hd> <p>The same federal agency that funded the LPFI, the National Security Education Program of the US Department of Defense, also spearheads The Language Flagship,[<reflink idref="bib6" id="ref51">6</reflink>] a national initiative to "create a pool of educated university graduates with demonstrated professional‐level language proficiency" (Nugent & Slater, [<reflink idref="bib47" id="ref52">47</reflink>], p. 10). Professional‐level proficiency is defined by The Language Flagship as ILR 3 (roughly equivalent to ACTFL Superior) or higher in speaking, and ILR 2+ (ACTFL Advanced High) or higher in reading and listening (Defense Language and National Security Education Office, [<reflink idref="bib16" id="ref53">16</reflink>]), which students reach by participating in articulated programs of domestic and overseas study. Domestic Language Flagship programs, designed for students in any major, offer options for intensive or accelerated language learning, opportunities for disciplinary learning in the language, individual or small‐group tutoring, and opportunities for language use and culture learning outside of formal courses. (Murphy et al., [<reflink idref="bib44" id="ref54">44</reflink>]). A focus on students' proficiency development and attainment is a hallmark of Language Flagship programs. Nugent and Slater ([<reflink idref="bib47" id="ref55">47</reflink>]), reporting on the final attainment of Language Flagship students in Arabic, Chinese, Hindi, Korean, Portuguese, Russian, Swahili, and Turkish, found that "by 2014, over 76% of the 101 Flagship students who completed the program that year had reached the goal of ILR 3, with over 95% of students across Flagship programs and languages reaching Level 2+ and above" (p. 23).[<reflink idref="bib7" id="ref56">7</reflink>]</p> <hd id="AN0181569079-7">Intensive LOTE domestic immersion programs</hd> <p>Intensive LOTE domestic immersion programs are found in Language Flagship programs, as an accelerated track within the traditional LOTE undergraduate curriculum, and as stand‐alone summer offerings for a national or international student audience. Dewey ([<reflink idref="bib18" id="ref57">18</reflink>]) defines domestic immersion programs as:</p> <p>settings where students study 'at home,' in programs that require many more hours per day of classroom instruction than are typical with normal academic‐year formal classroom settings, and where the goal of instruction is to teach the target language. In addition to providing significantly more hours of daily instruction in the L2, IM [immersion] programs promote language use outside of the classroom in a variety of ways (p. 304).</p> <p>Following this definition, intensive domestic immersion programs include the Middlebury Language Schools, which follow a "full immersion approach" (Urlaub, [<reflink idref="bib56" id="ref58">56</reflink>], p. 75) that supports students in honoring a pledge to only use the language of study for the duration of the program by integrating residential, dining, and extensive extracurricular activities in the program design. Intensive domestic immersion programs also include those that offer intensive instruction and co‐ and extracurricular activities, but do not require students to adhere to a language pledge, and therefore may not include a residential or daily shared dining component. Research on the proficiency outcomes of domestic immersion programs suggests that even very short programs may lead to measurable gains in students' proficiency. Miano et al. ([<reflink idref="bib43" id="ref59">43</reflink>]), for example, conducted pre‐ and postprogram unofficial OPIs with 51 students participating in a 2‐week immersion program in Spanish that required Intermediate‐level proficiency for admission. Miano et al. ([<reflink idref="bib43" id="ref60">43</reflink>]) found that students made significant gains in speaking proficiency in that short period of time, with preprogram ratings in the upper range of Intermediate Mid and postprogram ratings between Intermediate Mid and Intermediate High. In a longer immersion program that included a residential component, Ke and Reed ([<reflink idref="bib26" id="ref61">26</reflink>]) found that approximately 60% of students participating in a 9‐week domestic immersion Chinese program made observable gains in their speaking proficiency as measured by pre‐ and postprogram ACTFL OPIs. Rifkin, ([<reflink idref="bib51" id="ref62">51</reflink>], [<reflink idref="bib52" id="ref63">52</reflink>]) conducted unofficial pre‐ and postprogram OPIs with 55 students participating in the 9‐week Middlebury Russian School and found that students made gains in their speaking proficiency of between zero to three sublevels on the ACTFL scale over the course of the summer program. Comparing the proficiency outcomes of the Russian immersion program with those projected for traditional Russian programs, based on the immersion program's preprogram assessments and student self‐reports of prior Russian learning experiences, Rifkin ([<reflink idref="bib52" id="ref64">52</reflink>]) found an "immersion benefit" (p. 10), with students in the immersion program showing greater gains than students in traditional Russian programs, given the same number of instructional contact hours.</p> <hd id="AN0181569079-8">Online LOTE courses</hd> <p>Years later, Merrill et al. ([<reflink idref="bib42" id="ref65">42</reflink>]) used OPI and OPIc ratings to compare the speaking proficiency outcomes of that same immersion program, the Middlebury Russian School, when it was offered in the traditional F2F context, and when it was offered in an intensive online format during the COVID‐19 pandemic. When online during the COVID‐19 pandemic, the program had fewer weekly instructional hours and dramatically reduced opportunities for interaction in the language (Merrill et al., [<reflink idref="bib42" id="ref66">42</reflink>], p. 93). Despite these differences, Merrill et al. ([<reflink idref="bib42" id="ref67">42</reflink>]) found that the speaking proficiency outcomes of the F2F immersion and online intensive programs were "comparable" (p. 107). Kissau et al. ([<reflink idref="bib29" id="ref68">29</reflink>]) used the OPIc to investigate the speaking proficiency gains made by students enrolled in a semester‐long, asynchronous online Spanish course. (Unlike Merrill et al., [<reflink idref="bib42" id="ref69">42</reflink>], they did not compare gains made in an online course and F2F equivalent.) Kissau et al. ([<reflink idref="bib29" id="ref70">29</reflink>]) found that on average, students in the course, Spanish teacher candidates or Spanish majors, made significant gains in their speaking proficiency over the course of the semester, with eight of the 15 participants moving up at least one ACTFL sublevel.</p> <p>There are few other published studies using the OPI that report on the speaking proficiency outcomes of fully online LOTE courses or programs in either emergency or nonemergency contexts. Although there is a large and growing body of research on the effectiveness of computer assisted language learning for additional language learning (Grgurović et al., [<reflink idref="bib23" id="ref71">23</reflink>]; Plonsky & Ziegler, [<reflink idref="bib50" id="ref72">50</reflink>]), there are few studies that use standardized proficiency assessments (Link & Li, [<reflink idref="bib31" id="ref73">31</reflink>]; Van Deusen‐Scholl, [<reflink idref="bib17" id="ref74">17</reflink>]). The majority of studies comparing the outcomes of F2F and online language instruction are based on self‐perceptions or achievement (Lin & Warschauer, [<reflink idref="bib30" id="ref75">30</reflink>]; Link & Li, [<reflink idref="bib31" id="ref76">31</reflink>]). The small number of studies that do use standardized assessments to compare the speaking outcomes of F2F and online LOTE courses–all in Spanish–used assessments such as the Versant Test or STAMP Test and found no difference in the outcomes of the F2F and online courses (e.g., Blake et al., [<reflink idref="bib5" id="ref77">5</reflink>]; Burke Moneypenny & Aldrich, [<reflink idref="bib7" id="ref78">7</reflink>]); or found that the speaking proficiency outcomes of online courses offered during the COVID‐19 emergency were higher than F2F courses (Gleason et al., [<reflink idref="bib22" id="ref79">22</reflink>]).</p> <hd id="AN0181569079-9">Summary</hd> <p>With few exceptions, research to date on the speaking proficiency outcomes of domestic LOTE programs in US postsecondary contexts has focused on more widely taught languages and on F2F learning contexts. Few studies have used the ACTFL OPI or other standardized tests to document the proficiency outcomes of LOTE courses or programs, offered F2F or online in emergency or nonemergency contexts.[<reflink idref="bib8" id="ref80">8</reflink>] The present study contributes to research on the proficiency outcomes of postsecondary LOTE education in the United States by documenting the speaking proficiency outcomes of intensive summer programs in 14 LCTLs–Arabic, Bengali, Brazilian Portuguese, Hindi, Indonesian, Kazakh, Persian, Tamil, Thai, Tibetan, Turkish, Urdu, Uyghur, and Uzbek–expanding the research base on proficiency outcomes to include languages that are underrepresented in the research literature, and by comparing the speaking proficiency outcomes of those programs when offered F2F and when offered fully online, in the context of the COVID‐19 global crisis (Gacs et al., [<reflink idref="bib20" id="ref81">20</reflink>]).</p> <hd id="AN0181569079-10">RESEARCH QUESTIONS</hd> <p>This study addresses the following two research questions (RQs):</p> <p>RQ1. What is the difference, if any, in the speaking proficiency outcomes of intensive summer LCTL programs that were delivered F2F and those that were delivered online in the context of the COVID‐19 pandemic?</p> <p>RQ1.a. What is the difference, if any, in speaking proficiency outcomes of F2F and online intensive LCTL programs in terms of students' final <emph>attainment</emph>?</p> <p>RQ1.b. What is the difference, if any, in speaking proficiency outcomes of F2F and online intensive LCTL programs in terms of students' <emph>gains</emph>?</p> <p>RQ2. What are the speaking proficiency outcomes of intensive summer programs in different LCTLs and at different levels of instruction?</p> <p>RQ2.a. What are the speaking proficiency outcomes of intensive summer programs in different LCTLs and at different levels of instruction in terms of students' final <emph>attainment</emph>?</p> <p>RQ2.b. What are the speaking proficiency outcomes of intensive summer programs in different LCTLs and at different levels of instruction in terms of students' <emph>gains</emph>?</p> <hd id="AN0181569079-11">METHODS</hd> <p>To answer these RQs, this study employed a quantitative research methodology following a cross‐sectional design. The study was reviewed as exempt by the university's Institutional Review Board. The sections that follow describe the instructional context for the study, the participants, and the procedures for data collection and analysis.</p> <hd id="AN0181569079-12">Instructional context</hd> <p>Data for the study were from intensive summer programs in 14 languages offered through the Wisconsin Intensive Summer Language Institutes (WISLI) at the University of Wisconsin‐Madison in Summers 2018–2022. These programs met Dewey's ([<reflink idref="bib18" id="ref82">18</reflink>]) definition of "at‐home" immersion, but did not offer residential or shared dining options for students and did not have a language pledge like the Middlebury Language Schools. For these reasons, we classify the programs as intensive rather than immersion. Similar to Middlebury, however, WISLI offered intensive instruction in the summer term for a national and international student audience of graduate and undergraduate students, working professionals, and other nontraditional students, and includes co‐ and extracurricular activities in the program's design.</p> <p>WISLI regularly offered instruction in approximately 30 LCTLs at up to three levels of instruction, referred to in this article as Level 1, Level 2, and Level 3. In Summer 2020, there was an additional level of advanced Indonesian offered, referred to as Level 4. Students were placed in the appropriate course following local placement procedures; OPIs were not used for placement. Fourteen of the LCTLs offered through WISLI, those with relatively larger enrollments, were selected for inclusion in this study: Arabic, Bengali, Brazilian Portuguese,[<reflink idref="bib9" id="ref83">9</reflink>] Hindi, Indonesian, Kazakh, Persian, Tamil, Thai, Tibetan, Turkish, Urdu, Uyghur, and Uzbek. All WISLI programs offered the equivalent of a full academic year of instruction, with 20 weekly instructional contact hours in an 8‐week summer term, and with formal instruction supplemented by co‐curricular programming such as cultural activities, social get‐togethers, and guest speakers. In conjunction with the intensive summer programs in Hindi, Indonesian, Turkish, and Urdu, WISLI hosted federally funded Regional Flagship Language Initiatives (RFLIs) in those same languages. RFLI students, selected through the national Boren Scholarship and Fellowship competition, participated in the intensive summer domestic LCTL program, immediately followed by a semester abroad on a designated language‐intensive overseas program. RFLI students received additional instructional support, on top of the 20 weekly hours of instruction and co‐curricular programming for all WISLI students. That instructional support included additional cultural excursions, work with cultural learning materials designed for students preparing to study abroad, and 2 h per week of conversation practice with language partners.</p> <p>WISLI intensive summer programs are typically held in‐person, but during the COVID‐19 pandemic, in Summer 2020 and Summer 2021, they were offered fully online. When the intensive summer programs are held in‐person, classes meet for 4 h per day. When the programs were offered online during the COVID‐19 pandemic, the total number of instructional contact hours remained the same, through synchronous instruction and asynchronous learning activities. Synchronous instruction, enabled through videoconferencing platforms, was for a minimum of 2 h per day; asynchronous learning activities, as well as course materials and assessments, were delivered through the university's course management system. When online, WISLI continued to offer optional co‐curricular programming such as virtual city tours, film screenings, cooking demonstrations, and virtual guest speakers. Instructors of the online courses were provided with written guidelines on course design, online course templates in the university's course management system, virtual preprogram training led by pedagogy coordinators, and instructional technology tools, resources, and support.[<reflink idref="bib10" id="ref84">10</reflink>]</p> <hd id="AN0181569079-13">Participants</hd> <p>Participants (<emph>N</emph> = 484) were students enrolled in F2F (Summers 2018–2019; 2022; <emph>n</emph> = 229) or online (Summers 2020‐21; <emph>n</emph> = 255) intensive summer institutes in one of the 14 LCTLs. Table 1 displays the participants by language of study and by instructional delivery modality. (Percentages in tables in this article may not add up to 100 due to rounding).</p> <p>1 Table Language of study and mode of instruction (full sample, N = 484).</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th>Language of study</th><th>Mode of instruction</th><th align="left"><italic>n</italic></th><th align="left"><italic>%</italic></th></tr></thead><tbody valign="top"><tr><td>Arabic</td><td>F2F</td><td>14</td><td>2.89</td></tr><tr><td>Online</td><td>25</td><td>5.17</td></tr><tr><td>Bengali</td><td>F2F</td><td>8</td><td>1.65</td></tr><tr><td>Online</td><td>10</td><td>2.07</td></tr><tr><td>Brazilian Portuguese</td><td>F2F</td><td>8</td><td>1.65</td></tr><tr><td>Online</td><td>9</td><td>1.86</td></tr><tr><td>Hindi</td><td>F2F</td><td>67</td><td>13.84</td></tr><tr><td>Online</td><td>38</td><td>7.85</td></tr><tr><td>Indonesian</td><td>F2F</td><td>39</td><td>8.06</td></tr><tr><td>Online</td><td>30</td><td>6.20</td></tr><tr><td>Kazakh</td><td>F2F</td><td>2</td><td>0.41</td></tr><tr><td>Online</td><td>10</td><td>2.07</td></tr><tr><td>Persian</td><td>F2F</td><td>5</td><td>1.03</td></tr><tr><td>Online</td><td>23</td><td>4.75</td></tr><tr><td>Tamil</td><td>F2F</td><td>17</td><td>3.51</td></tr><tr><td>Online</td><td>7</td><td>1.45</td></tr><tr><td>Thai</td><td>F2F</td><td>12</td><td>2.48</td></tr><tr><td>Online</td><td>14</td><td>2.89</td></tr><tr><td>Tibetan</td><td>F2F</td><td>5</td><td>1.03</td></tr><tr><td>Online</td><td>7</td><td>1.45</td></tr><tr><td>Turkish</td><td>F2F</td><td>10</td><td>2.07</td></tr><tr><td>Online</td><td>29</td><td>5.99</td></tr><tr><td>Urdu</td><td>F2F</td><td>41</td><td>8.47</td></tr><tr><td>Online</td><td>40</td><td>8.26</td></tr><tr><td>Uyghur</td><td>F2F</td><td>1</td><td>0.21</td></tr><tr><td>Online</td><td>7</td><td>1.45</td></tr><tr><td>Uzbek</td><td>F2F</td><td>0</td><td>0.00</td></tr><tr><td>Online</td><td>6</td><td>1.24</td></tr></tbody></table> </ephtml> </p> <p>In Summers 2018–2020, the sample was comprised of all WISLI students in the targeted LCTLs who completed pre‐ and/or postprogram ACTFL OPIs as part of regular program assessment practices; data for those years is limited to ACTFL OPI ratings. For Summers 2021–2022, participants were WISLI students who consented to participate in the research by sharing their ACTFL OPI ratings and completing a background questionnaire. Sociodemographic information for participants was obtained through the background questionnaire and is thus only available for those Summer 2021–2022 participants who completed that study component (Table 2). Overall, the demographic profiles of participants displayed in Table 2 align with the population of WISLI students.</p> <p>2 Table Sociodemographic characteristics for 2021–2022 participants who completed a background questionnaire.</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th>Demographic characteristics</th><th align="left">2021–2022 Sample</th></tr><tr valign="bottom"><th><italic>n</italic></th><th align="left"><italic>%</italic></th></tr></thead><tbody valign="top"><tr><td>Gender</td><td /><td /></tr><tr><td>Man</td><td>55</td><td>43.65</td></tr><tr><td>Woman</td><td>65</td><td>51.59</td></tr><tr><td>Nonbinary</td><td>3</td><td>2.38</td></tr><tr><td>Not listed</td><td>2</td><td>1.59</td></tr><tr><td>Prefer not to answer</td><td>1</td><td>0.79</td></tr><tr><td>Student status</td><td /><td /></tr><tr><td>Undergraduate student</td><td>45</td><td>35.71</td></tr><tr><td>Graduate student</td><td>69</td><td>54.76</td></tr><tr><td>Working professional</td><td>3</td><td>2.38</td></tr><tr><td>Other</td><td>9</td><td>7.14</td></tr><tr><td>Age</td><td /><td /></tr><tr><td>17–21</td><td>36</td><td>28.57</td></tr><tr><td>22–25</td><td>41</td><td>32.54</td></tr><tr><td>26–30</td><td>33</td><td>26.19</td></tr><tr><td>31–35</td><td>11</td><td>8.73</td></tr><tr><td>36–40</td><td>3</td><td>2.38</td></tr><tr><td>40+</td><td>1</td><td>0.79</td></tr><tr><td>Did not answer</td><td>1</td><td>0.79</td></tr></tbody></table> </ephtml> </p> <p>1 <emph>Note</emph>: The 2021–2022 sample (<emph>n</emph> = 126) is comprised of participants for whom the researchers had, at a minimum, a postprogram OPI rating. The average age of participants for the 2021–2022 sample was <emph>M</emph> = 24.83 (SD = 5.06) years.</p> <hd id="AN0181569079-14">Procedures</hd> <p>Telephonic pre‐ and postprogram ACTFL OPIs were administered by Language Testing International as part of regular WISLI practices and at no cost to students. For students in Level 1 courses, preprogram ACTFL OPIs were typically administered only if required by program funders or per the terms of individual students' scholarships. For Summers 2018–2020, the researchers obtained pre‐ and postprogram ACTFL OPI ratings and limited student information from WISLI as existing assessment data. For Summers 2021 (online) and Summer 2022 (F2F), participants were recruited through a series of email solicitations sent by the researchers to all students (<emph>N</emph> = 415) of the 14 languages. Students who consented to participate in the study were asked to share their OPI ratings with the researchers and to complete a background questionnaire. In Summers 2021–2022, 159 students initially signed up to participate in the study, 139 of whom later participated in a postprogram ACTFL OPI and shared their ratings with the researchers.</p> <p>Proficiency outcomes were defined as comprised of both attainment, based only on postprogram ACTFL OPI ratings, and gains, based on changes in ACTFL OPI ratings from pre‐ to postprogram. For the analysis and to aid in the interpretation of gain scores, the ACTFL OPI ratings were coded on a scale of 1‐10 (Kenyon & Malabonga, [<reflink idref="bib27" id="ref85">27</reflink>]), as follows: Novice Low (<reflink idref="bib1" id="ref86">1</reflink>), Novice Mid (<reflink idref="bib2" id="ref87">2</reflink>), Novice High (<reflink idref="bib3" id="ref88">3</reflink>), Intermediate Low (<reflink idref="bib4" id="ref89">4</reflink>), Intermediate Mid (<reflink idref="bib5" id="ref90">5</reflink>), Intermediate High (<reflink idref="bib6" id="ref91">6</reflink>), Advanced Low (<reflink idref="bib7" id="ref92">7</reflink>), Advanced Mid (<reflink idref="bib8" id="ref93">8</reflink>), Advanced High (<reflink idref="bib9" id="ref94">9</reflink>), Superior (<reflink idref="bib10" id="ref95">10</reflink>). This same scale was used by Rifkin ([<reflink idref="bib52" id="ref96">52</reflink>]), Winke et al. ([<reflink idref="bib58" id="ref97">58</reflink>]), and Zhang et al. ([<reflink idref="bib60" id="ref98">60</reflink>]), among others. (See Tigchelaar ([<reflink idref="bib54" id="ref99">54</reflink>]) for a discussion of the use of this scale compared a weighted scale for the ACTFL OPI ratings.) The numeric values for the pre‐ and postprogram OPI ratings and responses from 2021 to 2022 participants to items on the background questionnaire were entered in a master Excel spreadsheet, and variables were created for program and student characteristics. These data were then imported into SPSS and data analysis was conducted using IBM SPSS Statistics 27.</p> <p>Both descriptive and inferential statistics were used to examine attainment and gains for participants who received online versus F2F instruction, as well as for participants enrolled at various course levels. To investigate attainment, any participant with a postprogram ACTFL OPI rating, regardless of whether they had a preprogram ACTFL OPI rating, was included in the analysis (<emph>N</emph> = 484). For analyses investigating gains, only participants with matched pre‐ and postprogram ACTFL OPI ratings were included (<emph>N</emph> = 219). As such, participants with matched data were included in both the attainment and the gains analyses. When descriptive statistics and frequencies for gain are reported, gain was operationalized as postprogram ACTFL OPI rating minus preprogram ACTFL OPI rating. As an example, a participant with a preprogram ACTFL rating of Novice High (coded as 3) and a postpostprogram rating of Intermediate Mid (coded as 5) would have a gain score of 2. Throughout, results are provided only if there were at least three participants in the (sub)group; any instances in which results are not reported due to small sample size are noted. Necessary model assumptions were checked before running any analyses. For all analyses, the normality assumption was assessed using the z‐test for skewness and kurtosis as described in Kim ([<reflink idref="bib28" id="ref100">28</reflink>]) and comparing computed z‐scores to the critical <emph>z</emph>‐value for the appropriate sample size (i.e., a critical <emph>z</emph>‐value of 1.96 for sample sizes less than 50 and a critical <emph>z</emph>‐value of 3.29 for sample sizes between 50 and 300). Similarly, for all analyses, Levene's test, using a significance value of <emph>p</emph> > .05, was used to assess homogeneity of variance. Specific to RQ1.b and RQ2.b, homogeneity of covariance matrices was evaluated, but because there were only two time points (pre‐ and post‐), the sphericity assumption was not assessed (Maxwell & Delaney, [<reflink idref="bib41" id="ref101">41</reflink>]). Any violations of assumptions are noted in the Results section; if nothing is noted, then all necessary assumptions were met.</p> <hd id="AN0181569079-15">RESULTS</hd> <p></p> <hd id="AN0181569079-16">RQ1: Difference in speaking proficiency outcomes of F2F and online intensive LCTL programs</hd> <p>This section provides results for RQ1: What is the difference, if any, in the speaking proficiency outcomes of intensive summer LCTL programs that were delivered F2F and those that were delivered online during the COVID‐19 pandemic? The results are presented first for speaking proficiency attainment (RQ1.a), followed by gains (RQ1.b).</p> <hd id="AN0181569079-17">RQ1.a. Speaking proficiency attainment in F2F and online intensive LCTL programs</hd> <p>To investigate RQ1.a, an independent samples t‐test was used to determine if there was a difference in postprogram ACTFL OPI ratings for the two instructional delivery modality groups, F2F and online. Results indicated that there was no significant difference on average postprogram ACTFL OPI ratings for individuals who received F2F instruction when compared to individuals who received online instruction during the COVID‐19 pandemic; <emph>t</emph>(<reflink idref="bib482" id="ref102">482</reflink>) = –1.77, <emph>p</emph> = .08, <emph>d</emph> = 0.16. See Table 3 for descriptive statistics for attainment in speaking proficiency for the two groups, F2F and online.</p> <p>3 Table Descriptive statistics for postprogram ACTFL OPI ratings by mode of instruction (N = 484).</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th>Mode of instruction</th><th align="left"><italic>n</italic></th><th align="left"><italic>M</italic></th><th align="left">SD</th><th align="left">Median</th><th align="left">Min</th><th align="left">Max</th></tr></thead><tbody valign="top"><tr><td>F2F</td><td>229</td><td>5.14</td><td>1.75</td><td>5</td><td>1</td><td>9</td></tr><tr><td>Online</td><td>255</td><td>5.43</td><td>1.76</td><td>5</td><td>1</td><td>10</td></tr></tbody></table> </ephtml> </p> <hd id="AN0181569079-18">RQ1.b. Speaking proficiency gains in F2F and online intensive LCTL programs</hd> <p>Table 4 shows average pre‐, post‐ and gain scores for F2F and online participants with matched data. To investigate whether there was a difference between the speaking proficiency gains made by students in F2F and online intensive summer programs (RQ1.b), a mixed (split‐plot) ANOVA, which allows for simultaneously testing the interaction between time (i.e., pre‐, post‐) and group (i.e., F2F and online), the main effect of time (within‐subjects), and the main effect of group (between‐subjects; Laerd Statistics, [<reflink idref="bib1" id="ref103">1</reflink>]; Maxwell & Delaney, [<reflink idref="bib41" id="ref104">41</reflink>]), was used.</p> <p>4 Table Average preprogram, postprogram, and ACTFL OPI gain scores by instructional delivery modality, matched sample (N = 219).</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th>Mode of instruction</th><th /><th align="left">Preprogram</th><th align="left">Postprogram</th><th align="left">Gain</th></tr><tr valign="bottom"><th align="left"><italic>n</italic></th><th align="left"><italic>M</italic></th><th align="left">SD</th><th align="left">Median</th><th align="left">Min</th><th align="left">Max</th><th align="left"><italic>M</italic></th><th align="left">SD</th><th align="left">Median</th><th align="left">Min</th><th align="left">Max</th><th align="left"><italic>M</italic></th><th align="left">SD</th><th align="left">Median</th><th align="left">Min</th><th align="left">Max</th></tr></thead><tbody valign="top"><tr><td>F2F</td><td>98</td><td>4.61</td><td>1.91</td><td>5</td><td>1</td><td>9</td><td>6.02</td><td>1.55</td><td>6</td><td>3</td><td>9</td><td>1.41</td><td>1.39</td><td>1</td><td>–1</td><td>5</td></tr><tr><td>Online</td><td>121</td><td>5.15</td><td>1.77</td><td>5</td><td>1</td><td>10</td><td>6.29</td><td>1.56</td><td>6</td><td>2</td><td>10</td><td>1.14</td><td>1.37</td><td>1</td><td>–2</td><td>6</td></tr></tbody></table> </ephtml> </p> <p>The interaction between instructional delivery modality and time was not significant; <emph>F</emph>(<reflink idref="bib1" id="ref105">1</reflink>, 217) = 2.03, <emph>p</emph> = .16, <ephtml> <math altimg="urn:x-wiley:0015718X:media:flan12758:flan12758-math-0001" display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msubsup><mi>η</mi><mi>p</mi><mn>2</mn></msubsup></mrow><annotation encoding="application/x-tex"> ${\eta }_{p}^{2}$</annotation></semantics></math> </ephtml>  = 0.01, indicating that the average change in ACTFL OPI scores from preprogram to postprogram did not differ significantly for students who received F2F and online instruction. The main effect of group was also not significant; <emph>F</emph>(<reflink idref="bib1" id="ref106">1</reflink>, 217) = 3.65, <emph>p</emph> = .06, <ephtml> <math altimg="urn:x-wiley:0015718X:media:flan12758:flan12758-math-0002" display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msubsup><mi>η</mi><mi>p</mi><mn>2</mn></msubsup></mrow><annotation encoding="application/x-tex"> ${\eta }_{p}^{2}$</annotation></semantics></math> </ephtml>  = 0.02. There was not a statistically significant difference on average ACTFL OPI ratings for those who participated in intensive F2F programs and those who participated in the intensive online programs, after collapsing across the two time points. The main effect of time, however, was significant; <emph>F</emph>(<reflink idref="bib1" id="ref107">1</reflink>, 217) = 184.23, <emph>p</emph> < .001, <ephtml> <math altimg="urn:x-wiley:0015718X:media:flan12758:flan12758-math-0003" display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msubsup><mi>η</mi><mi>p</mi><mn>2</mn></msubsup></mrow><annotation encoding="application/x-tex"> ${\eta }_{p}^{2}$</annotation></semantics></math> </ephtml>  = 0.46, with participants, on average, increasing from a preprogram ACTFL OPI rating of 4.88 (just below Intermediate Mid) to a postprogram ACTFL OPI rating of 6.16 (Intermediate High).</p> <hd id="AN0181569079-19">RQ2: Speaking proficiency outcomes of intensive summer LCTL programs by language and level of...</hd> <p>This section provides results for RQ2: What are the speaking proficiency outcomes of intensive summer programs in different LCTLs and at different levels of instruction? Given that there was no significant difference in the speaking proficiency outcomes of intensive LCTL programs that were delivered F2F or online, the two groups (F2F and online) were combined for the analyses presented below. The results are presented first for speaking proficiency attainment (RQ2.a), followed by gains (RQ2.b).</p> <hd id="AN0181569079-20">RQ2.a. Speaking proficiency attainment by language and level of instruction</hd> <p>Table 5 presents descriptive statistics for postprogram ACTFL OPI ratings by level of instruction, collapsing across languages.</p> <p>5 Table Descriptive statistics for postprogram ACTFL OPI ratings by level of instruction, full sample (N = 484).</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th>Level of instruction</th><th align="left"><italic>n</italic></th><th align="left"><italic>M</italic></th><th align="left">SD</th><th align="left">Median</th><th align="left">Min</th><th align="left">Max</th></tr></thead><tbody valign="top"><tr><td>Level 1</td><td>266</td><td>4.37</td><td>1.44</td><td>4</td><td>1</td><td>7</td></tr><tr><td>Level 2</td><td>119</td><td>5.93</td><td>1.29</td><td>6</td><td>3</td><td>9</td></tr><tr><td>Level 3</td><td>96</td><td>6.98</td><td>1.41</td><td>7</td><td>4</td><td>10</td></tr><tr><td>Level 4</td><td>3</td><td>7.67</td><td>0.58</td><td>8</td><td>7</td><td>8</td></tr></tbody></table> </ephtml> </p> <p>2 <emph>Note</emph>: Level 4 was comprised of students of Indonesian only.</p> <p>A one‐way analysis of variance (ANOVA) was used to test for differences in postprogram ACTFL OPI ratings by level of instruction. For this analysis, students enrolled in Level 4 courses were not included due to small sample size. There was a significant difference on average postprogram ACTFL OPI ratings among the three levels of instruction; <emph>F</emph>(<reflink idref="bib2" id="ref108">2</reflink>, 478) = 139.76, <emph>p</emph> < .001, <emph>η</emph><sups>2</sups> = 0.37, with 37% of the variation in postprogram ACTFL OPI ratings accounted for by differences in level of instruction. Follow‐up tests using a Bonferroni‐adjusted alpha level revealed that all pairwise comparisons were significant. On average, individuals enrolled in a Level 3 intensive program earned postprogram ACTFL OPI ratings of Advanced Low, compared to those enrolled in Level 2 programs, who earned ratings of Intermediate High, and those enrolled in Level 1 programs, who earned ratings between Intermediate Low and Intermediate Mid. Table 6 provides the frequency distribution of postprogram ACTFL OPI ratings for each level of instruction, collapsing across languages.</p> <p>6 Table Frequency distributions of postprogram ACTFL OPI ratings by level of instruction, full sample (N = 484).</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th>Postprogram ACTFL OPI rating</th><th align="left">Level 1</th><th align="left">Level 2</th><th align="left">Level 3</th><th align="left">Level 4</th></tr><tr valign="bottom"><th align="left"><italic>n</italic></th><th align="left"><italic>%</italic></th><th align="left"><italic>n</italic></th><th align="left"><italic>%</italic></th><th align="left"><italic>n</italic></th><th align="left"><italic>%</italic></th><th align="left"><italic>n</italic></th><th align="left"><italic>%</italic></th></tr></thead><tbody valign="top"><tr><td>NL</td><td>3</td><td>1.13</td><td>0</td><td>0.00</td><td>0</td><td>0.00</td><td>0</td><td>0.00</td></tr><tr><td>NM</td><td>28</td><td>10.53</td><td>0</td><td>0.00</td><td>0</td><td>0.00</td><td>0</td><td>0.00</td></tr><tr><td>NH</td><td>41</td><td>15.41</td><td>3</td><td>2.52</td><td>0</td><td>0.00</td><td>0</td><td>0.00</td></tr><tr><td>IL</td><td>66</td><td>24.81</td><td>11</td><td>9.24</td><td>4</td><td>4.17</td><td>0</td><td>0.00</td></tr><tr><td>IM</td><td>76</td><td>28.57</td><td>31</td><td>26.05</td><td>11</td><td>11.46</td><td>0</td><td>0.00</td></tr><tr><td>IH</td><td>27</td><td>10.15</td><td>36</td><td>30.25</td><td>17</td><td>17.71</td><td>0</td><td>0.00</td></tr><tr><td>AL</td><td>25</td><td>9.40</td><td>26</td><td>21.85</td><td>32</td><td>33.33</td><td>1</td><td>33.33</td></tr><tr><td>AM</td><td>0</td><td>0.00</td><td>8</td><td>6.72</td><td>19</td><td>19.79</td><td>2</td><td>66.67</td></tr><tr><td>AH</td><td>0</td><td>0.00</td><td>4</td><td>3.36</td><td>9</td><td>9.38</td><td>0</td><td>0.00</td></tr><tr><td>S</td><td>0</td><td>0.00</td><td>0</td><td>0.00</td><td>4</td><td>4.17</td><td>0</td><td>0.00</td></tr></tbody></table> </ephtml> </p> <p>3 <emph>Note</emph>: <emph>n</emph> = 266 for Level 1, <emph>n</emph> = 119 for Level 2, <emph>n</emph> = 96 for Level 3, and <emph>n</emph> = 3 for Level 4.</p> <p>Next, Table 7 shows the average speaking proficiency attainment by both language of study and level of instruction. Across languages, students' average speaking proficiency attainment increased with level of instruction, with a higher average postprogram ACTFL OPI rating for Level 2 courses than for Level 1, and a higher average ACTFL OPI rating for Level 3 courses than for Level 2. Within levels of instruction, the average postprogram ACTFL OPI ratings showed some variation. In Level 1 courses, students' speaking proficiency attainment ranged from Novice Mid/Novice High (Tamil) to Intermediate High (Uzbek). For Levels 2 and 3, average ACTFL OPI ratings ranged from Intermediate Low/Intermediate Mid (Tamil) to Advanced Mid (Uyghur) and from Intermediate High (Persian) to about Advanced Mid (Turkish), respectively.</p> <p>7 Table Descriptive statistics for postprogram ACTFL OPI ratings by language and level of instruction, full sample (N = 484).</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th>Language and Level of instruction</th><th><italic>n</italic></th><th><italic>M</italic></th><th>SD</th><th>Median</th><th>Min</th><th>Max</th></tr></thead><tbody valign="top"><tr><td>Arabic</td></tr><tr><td>Level 1</td><td>11</td><td>4.27</td><td>0.79</td><td>4</td><td>3</td><td>5</td></tr><tr><td>Level 2</td><td>13</td><td>5.54</td><td>0.66</td><td>5</td><td>5</td><td>7</td></tr><tr><td>Level 3</td><td>15</td><td>6.53</td><td>1.25</td><td>7</td><td>5</td><td>9</td></tr><tr><td>Bengali</td><td /><td /><td /><td /><td /><td /></tr><tr><td>Level 1</td><td>18</td><td>5.44</td><td>1.04</td><td>5</td><td>3</td><td>7</td></tr><tr><td>Brazilian Portuguese</td></tr><tr><td>Level 1</td><td>17</td><td>5.65</td><td>1.22</td><td>6</td><td>3</td><td>7</td></tr><tr><td>Hindi</td><td /><td /><td /><td /><td /><td /></tr><tr><td>Level 1</td><td>60</td><td>4.23</td><td>1.05</td><td>4</td><td>2</td><td>7</td></tr><tr><td>Level 2</td><td>29</td><td>5.97</td><td>1.02</td><td>6</td><td>4</td><td>8</td></tr><tr><td>Level 3</td><td>16</td><td>6.81</td><td>1.33</td><td>7</td><td>4</td><td>9</td></tr><tr><td>Indonesian</td><td /><td /><td /><td /><td /><td /></tr><tr><td>Level 1</td><td>39</td><td>5.31</td><td>1.34</td><td>5</td><td>3</td><td>7</td></tr><tr><td>Level 2</td><td>12</td><td>6.67</td><td>1.23</td><td>7</td><td>5</td><td>9</td></tr><tr><td>Level 3</td><td>15</td><td>7.53</td><td>1.13</td><td>7</td><td>5</td><td>10</td></tr><tr><td>Level 4</td><td>3</td><td>7.67</td><td>0.58</td><td>8</td><td>7</td><td>8</td></tr><tr><td>Kazakh</td></tr><tr><td>Level 1</td><td>4</td><td>5.75</td><td>0.96</td><td>5.5</td><td>5</td><td>7</td></tr><tr><td>Level 2</td><td>4</td><td>7.25</td><td>0.96</td><td>7.5</td><td>6</td><td>8</td></tr><tr><td>Level 3</td><td>4</td><td>7.50</td><td>2.08</td><td>7.5</td><td>5</td><td>10</td></tr><tr><td>Persian</td><td /><td /><td /><td /><td /><td /></tr><tr><td>Level 1</td><td>9</td><td>4.89</td><td>1.05</td><td>5</td><td>4</td><td>7</td></tr><tr><td>Level 2</td><td>8</td><td>5.13</td><td>0.83</td><td>5</td><td>4</td><td>6</td></tr><tr><td>Level 3</td><td>11</td><td>6.18</td><td>1.25</td><td>6</td><td>4</td><td>9</td></tr><tr><td>Tamil</td></tr><tr><td>Level 1</td><td>19</td><td>2.68</td><td>1.00</td><td>2</td><td>1</td><td>4</td></tr><tr><td>Level 2</td><td>5</td><td>4.40</td><td>2.07</td><td>4</td><td>3</td><td>8</td></tr><tr><td>Thai</td><td /><td /><td /><td /><td /><td /></tr><tr><td>Level 1</td><td>17</td><td>3.76</td><td>1.15</td><td>4</td><td>2</td><td>6</td></tr><tr><td>Level 2</td><td>3</td><td>5.67</td><td>1.15</td><td>5</td><td>5</td><td>7</td></tr><tr><td>Level 3</td><td>6</td><td>7.67</td><td>2.25</td><td>7.5</td><td>4</td><td>10</td></tr><tr><td>Tibetan</td></tr><tr><td>Level 1</td><td>11</td><td>3.45</td><td>1.37</td><td>3</td><td>2</td><td>6</td></tr><tr><td>Turkish</td></tr><tr><td>Level 1</td><td>21</td><td>4.10</td><td>1.48</td><td>4</td><td>1</td><td>7</td></tr><tr><td>Level 2</td><td>8</td><td>6.63</td><td>1.77</td><td>6</td><td>5</td><td>9</td></tr><tr><td>Level 3</td><td>10</td><td>7.80</td><td>0.92</td><td>7.5</td><td>7</td><td>9</td></tr><tr><td>Urdu</td></tr><tr><td>Level 1</td><td>33</td><td>3.45</td><td>1.20</td><td>4</td><td>1</td><td>6</td></tr><tr><td>Level 2</td><td>31</td><td>5.65</td><td>1.14</td><td>6</td><td>3</td><td>7</td></tr><tr><td>Level 3</td><td>17</td><td>6.53</td><td>1.28</td><td>6</td><td>4</td><td>9</td></tr><tr><td>Uyghur</td></tr><tr><td>Level 1</td><td>3</td><td>5.67</td><td>1.15</td><td>5</td><td>5</td><td>7</td></tr><tr><td>Level 2</td><td>3</td><td>8.00</td><td>1.00</td><td>8</td><td>7</td><td>9</td></tr><tr><td>Uzbek</td></tr><tr><td>Level 1</td><td>4</td><td>6.00</td><td>0.00</td><td>6</td><td>6</td><td>6</td></tr></tbody></table> </ephtml> </p> <p>4 <emph>Note</emph>: There was one student in Level 2 Tibetan, two students in Level 2 Uzbek, and two students in Level 3 Uyghur. Those data are not included due to small sample size. Indonesian was the only language with participants in a Level 4 course. There were no participants with postprogram ACTFL OPI ratings in Level 2 or Level 3 Bengali and Brazilian Portuguese. There were no Level 3 Tamil, Tibetan, and Uzbek participants with postprogram ACTFL OPI ratings.</p> <hd id="AN0181569079-21">RQ2.b. SpeakingSpeaking proficiency gains by language and level of instruction</hd> <p>Table 8 shows average pre‐ and postprogram ACTFL OPI ratings and gain scores for participants at Levels 1–3 of instruction, collapsing across languages. Greater gains in speaking proficiency were made by students at lower levels of instruction. Students enrolled in Level 1 intensive programs made the largest average gains, 2.63 ACTFL sub‐levels. Students in Level 2 courses made average gains in speaking proficiency of 1.26 ACTFL sub‐levels, followed by students in Level 3 courses, with average gains of less than one ACTFL sublevel, 0.84, in their speaking proficiency.</p> <p>8 Table Average preprogram, postprogram, and ACTFL OPI gain scores by level of instruction, matched sample (N = 219).</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th>Level of instruction</th><th align="left">Preprogram</th><th align="left">Postprogram</th><th align="left">Gain</th></tr><tr valign="bottom"><th align="left"><italic>n</italic></th><th align="left"><italic>M</italic></th><th align="left">SD</th><th align="left">Median</th><th align="left">Min</th><th align="left">Max</th><th align="left"><italic>M</italic></th><th align="left">SD</th><th align="left">Median</th><th align="left">Min</th><th align="left">Max</th><th align="left"><italic>M</italic></th><th align="left">SD</th><th align="left">Median</th><th align="left">Min</th><th align="left">Max</th></tr></thead><tbody valign="top"><tr><td>Level 1</td><td>27</td><td>2.00</td><td>1.24</td><td>2</td><td>1</td><td>5</td><td>4.63</td><td>1.25</td><td>5</td><td>2</td><td>7</td><td>2.63</td><td>1.42</td><td>3</td><td>0</td><td>6</td></tr><tr><td>Level 2</td><td>102</td><td>4.61</td><td>1.24</td><td>4.5</td><td>2</td><td>8</td><td>5.87</td><td>1.32</td><td>6</td><td>3</td><td>9</td><td>1.26</td><td>1.39</td><td>1</td><td>–2</td><td>6</td></tr><tr><td>Level 3</td><td>88</td><td>6.11</td><td>1.43</td><td>6</td><td>3</td><td>10</td><td>6.95</td><td>1.44</td><td>7</td><td>4</td><td>10</td><td>0.84</td><td>1.09</td><td>1</td><td>–1</td><td>4</td></tr></tbody></table> </ephtml> </p> <p>5 <emph>Note</emph>: Two individuals who were enrolled in Level 4 courses were excluded from the analysis due to small sample size.</p> <p>Similar to RQ1.b, a mixed (split‐plot) ANOVA was used to investigate whether there was a difference in speaking proficiency gains between participants at various levels of instruction. Two individuals enrolled in Level 4 courses were excluded from the analysis due to small sample size. There was observed skewness in preprogram ACTFL OPI scores for Level 1 using a critical <emph>z</emph>‐value of 1.96 (<emph>z</emph> = 2.91); however, ANOVA is generally robust to violations of normality (Laerd Statistics, [<reflink idref="bib1" id="ref109">1</reflink>]; Maxwell & Delaney, [<reflink idref="bib41" id="ref110">41</reflink>]).</p> <p>Average ACTFL OPI gains differed for the three instructional level groups as evidenced by a significant interaction effect; <emph>F</emph>(<reflink idref="bib2" id="ref111">2</reflink>, 214) = 20.10, <emph>p</emph> < .001, <ephtml> <math altimg="urn:x-wiley:0015718X:media:flan12758:flan12758-math-0004" display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msubsup><mi>η</mi><mi>p</mi><mn>2</mn></msubsup></mrow><annotation encoding="application/x-tex"> ${\eta }_{p}^{2}$</annotation></semantics></math> </ephtml>  = 0.16. The observed differences on average ACTFL OPI ratings between the three groups were larger at preprogram than at postprogram. For example, the average difference in preprogram ratings for Level 1 compared to Level 2 was 2.61. At postprogram, this difference decreased to 1.24. For Level 1 compared to Level 3, the difference in preprogram ACTFL OPI ratings was 4.11, compared to only 2.32 at postprogram. This trend was also evident for Level 2 compared to Level 3, with a preprogram difference of 1.5 compared to a postprogram difference of 1.08. In other words, the differences between instructional groups decreased after participating in such programs. Post hoc tests with a Bonferroni adjustment revealed that average ACTFL OPI gains for Level 1 students were significantly higher than average ACTFL OPI gains for both Level 2 and Level 3 students (<emph>p</emph> < .001). There was not a statistically significant difference in ACTFL OPI gains between Level 2 and Level 3 students (<emph>p</emph> = .07). In addition, tests for within‐subjects simple effects showed that, on average, ACTFL OPI ratings increased significantly from pre‐ to postprogram for all three course level groups (Level 1: <emph>t</emph>(<reflink idref="bib26" id="ref112">26</reflink>) = 9.63, <emph>p</emph> < .001, <emph>d</emph> = 1.85; Level 2: <emph>t</emph>(<reflink idref="bib101" id="ref113">101</reflink>) = 9.17, <emph>p</emph> < .001, <emph>d</emph> = 0.91; Level 3: <emph>t</emph>(<reflink idref="bib87" id="ref114">87</reflink>) = 7.22, <emph>p</emph> < .001, <emph>d</emph> = 0.77). Following Plonsky and Oswald's ([<reflink idref="bib49" id="ref115">49</reflink>]) recommendations for within‐subjects effects, these results range from small to large treatment effects, with the size of the effect decreasing with an increase in course level. Table 9 shows the frequency distribution of gain scores for each level of instruction.</p> <p>9 Table Frequency distributions of ACTFL OPI gains by level of instruction, matched sample (N = 219).</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th /><th align="left">Level of instruction</th></tr><tr valign="bottom"><th>ACTFL OPI gain score</th><th align="left">Level 1</th><th align="left">Level 2</th><th align="left">Level 3</th></tr><tr valign="bottom"><th align="left"><italic>n</italic></th><th align="left"><italic>%</italic></th><th align="left"><italic>n</italic></th><th align="left"><italic>%</italic></th><th align="left"><italic>n</italic></th><th align="left"><italic>%</italic></th></tr></thead><tbody valign="top"><tr><td>–2</td><td>0</td><td>0.00</td><td>1</td><td>0.98</td><td>0</td><td>0.00</td></tr><tr><td>–1</td><td>0</td><td>0.00</td><td>8</td><td>7.84</td><td>7</td><td>7.95</td></tr><tr><td>0</td><td>2</td><td>7.41</td><td>17</td><td>16.67</td><td>30</td><td>34.09</td></tr><tr><td>1</td><td>3</td><td>11.11</td><td>42</td><td>41.18</td><td>28</td><td>31.82</td></tr><tr><td>2</td><td>8</td><td>29.63</td><td>13</td><td>12.75</td><td>18</td><td>20.45</td></tr><tr><td>3</td><td>7</td><td>25.93</td><td>16</td><td>15.69</td><td>3</td><td>3.41</td></tr><tr><td>4</td><td>5</td><td>18.52</td><td>3</td><td>2.94</td><td>2</td><td>2.27</td></tr><tr><td>5</td><td>1</td><td>3.70</td><td>1</td><td>0.98</td><td>0</td><td>0.00</td></tr><tr><td>6</td><td>1</td><td>3.70</td><td>1</td><td>0.98</td><td>0</td><td>0.00</td></tr></tbody></table> </ephtml> </p> <p>6 <emph>Note</emph>: <emph>n</emph> = 27 for Level 1, <emph>n</emph> = 102 for Level 2, and <emph>n</emph> = 88 for Level 3. Two individuals who were enrolled a Level 4 course were excluded due to small sample size.</p> <p>Table 10 displays the average preprogram, postprogram, and ACTFL OPI gain scores by both language and level of instruction. Across languages, both pre‐ and postprogram ACTFL OPI ratings increased by level of instruction. In terms of gains, as level of instruction increased, average ACTFL OPI gain scores decreased. The one exception to this trend was for Turkish, where the average ACTFL OPI gain score for Level 1 was smaller than the average ACTFL OPI gain score for Level 2.</p> <p>10 Table Average preprogram, postprogram, and ACTFL OPI gain scores by level of instruction, matched sample (N = 219).</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th>Language and Level of instruction</th><th /><th>Preprogram</th><th>Postprogram</th><th>Gain</th></tr><tr valign="bottom"><th><italic>n</italic></th><th><italic>M</italic></th><th>SD</th><th>Median</th><th>Min</th><th>Max</th><th><italic>M</italic></th><th>SD</th><th>Median</th><th>Min</th><th>Max</th><th><italic>M</italic></th><th>SD</th><th>Median</th><th>Min</th><th>Max</th></tr></thead><tbody valign="top"><tr><td>Arabic</td></tr><tr><td>Level 2</td><td>12</td><td>4.33</td><td>0.78</td><td>4</td><td>3</td><td>6</td><td>5.50</td><td>0.67</td><td>5</td><td>5</td><td>7</td><td>1.17</td><td>0.83</td><td>1</td><td>0</td><td>3</td></tr><tr><td>Level 3</td><td>15</td><td>5.40</td><td>1.35</td><td>5</td><td>3</td><td>8</td><td>6.53</td><td>1.25</td><td>7</td><td>5</td><td>9</td><td>1.13</td><td>1.19</td><td>1</td><td>0</td><td>4</td></tr><tr><td>Hindi</td></tr><tr><td>Level 1</td><td>11</td><td>2.09</td><td>1.38</td><td>2</td><td>1</td><td>5</td><td>4.64</td><td>0.81</td><td>5</td><td>3</td><td>6</td><td>2.55</td><td>1.04</td><td>3</td><td>1</td><td>4</td></tr><tr><td>Level 2</td><td>28</td><td>4.36</td><td>1.10</td><td>4</td><td>2</td><td>6</td><td>5.96</td><td>1.04</td><td>6</td><td>4</td><td>8</td><td>1.61</td><td>1.23</td><td>1</td><td>0</td><td>5</td></tr><tr><td>Level 3</td><td>16</td><td>5.75</td><td>1.39</td><td>6</td><td>4</td><td>8</td><td>6.81</td><td>1.33</td><td>7</td><td>4</td><td>9</td><td>1.06</td><td>1.29</td><td>1</td><td>−1</td><td>3</td></tr><tr><td>Indonesian</td></tr><tr><td>Level 1</td><td>6</td><td>1.17</td><td>0.41</td><td>1</td><td>1</td><td>2</td><td>5.50</td><td>1.22</td><td>5</td><td>4</td><td>7</td><td>4.33</td><td>1.03</td><td>4</td><td>3</td><td>6</td></tr><tr><td>Level 2</td><td>12</td><td>4.75</td><td>1.54</td><td>4</td><td>3</td><td>8</td><td>6.67</td><td>1.23</td><td>7</td><td>5</td><td>9</td><td>1.92</td><td>1.56</td><td>2</td><td>−1</td><td>4</td></tr><tr><td>Level 3</td><td>11</td><td>6.73</td><td>1.19</td><td>7</td><td>4</td><td>9</td><td>7.45</td><td>1.29</td><td>7</td><td>5</td><td>10</td><td>0.72</td><td>0.90</td><td>1</td><td>0</td><td>3</td></tr><tr><td>Kazakh</td></tr><tr><td>Level 3</td><td>4</td><td>6.00</td><td>2.16</td><td>6.5</td><td>3</td><td>8</td><td>7.50</td><td>2.08</td><td>7.5</td><td>5</td><td>10</td><td>1.50</td><td>0.58</td><td>1.5</td><td>1</td><td>2</td></tr><tr><td>Persian</td></tr><tr><td>Level 2</td><td>8</td><td>4.63</td><td>0.92</td><td>5</td><td>3</td><td>6</td><td>5.13</td><td>0.83</td><td>5</td><td>4</td><td>6</td><td>0.50</td><td>1.31</td><td>0.5</td><td>−1</td><td>3</td></tr><tr><td>Level 3</td><td>10</td><td>6.00</td><td>1.33</td><td>5.5</td><td>5</td><td>9</td><td>6.10</td><td>1.29</td><td>6</td><td>4</td><td>9</td><td>0.10</td><td>0.74</td><td>0</td><td>−1</td><td>1</td></tr><tr><td>Tamil</td></tr><tr><td>Level 2</td><td>5</td><td>4.40</td><td>1.52</td><td>4</td><td>3</td><td>7</td><td>4.40</td><td>2.07</td><td>4</td><td>3</td><td>8</td><td>0.00</td><td>1.00</td><td>0</td><td>−1</td><td>1</td></tr><tr><td>Thai</td></tr><tr><td>Level 2</td><td>3</td><td>4.67</td><td>1.15</td><td>4</td><td>4</td><td>6</td><td>5.67</td><td>1.15</td><td>5</td><td>5</td><td>7</td><td>1.00</td><td>2.00</td><td>1</td><td>−1</td><td>3</td></tr><tr><td>Level 3</td><td>4</td><td>7.50</td><td>2.08</td><td>7.5</td><td>5</td><td>10</td><td>7.75</td><td>2.87</td><td>8.5</td><td>4</td><td>10</td><td>0.25</td><td>1.26</td><td>0</td><td>−1</td><td>2</td></tr><tr><td>Turkish</td></tr><tr><td>Level 1</td><td>3</td><td>3.00</td><td>2.00</td><td>3</td><td>1</td><td>5</td><td>4.67</td><td>2.08</td><td>4</td><td>3</td><td>7</td><td>1.67</td><td>1.53</td><td>2</td><td>0</td><td>3</td></tr><tr><td>Level 2</td><td>7</td><td>4.29</td><td>1.60</td><td>5</td><td>2</td><td>6</td><td>6.71</td><td>1.89</td><td>6</td><td>5</td><td>9</td><td>2.43</td><td>2.15</td><td>3</td><td>−1</td><td>6</td></tr><tr><td>Level 3</td><td>10</td><td>6.50</td><td>1.51</td><td>6</td><td>5</td><td>9</td><td>7.80</td><td>0.92</td><td>7.5</td><td>7</td><td>9</td><td>1.30</td><td>1.25</td><td>1</td><td>0</td><td>4</td></tr><tr><td>Urdu</td></tr><tr><td>Level 1</td><td>7</td><td>2.14</td><td>0.90</td><td>2</td><td>1</td><td>4</td><td>3.86</td><td>1.21</td><td>4</td><td>2</td><td>6</td><td>1.71</td><td>0.95</td><td>2</td><td>0</td><td>3</td></tr><tr><td>Level 2</td><td>23</td><td>5.00</td><td>1.41</td><td>5</td><td>2</td><td>8</td><td>5.61</td><td>1.20</td><td>6</td><td>3</td><td>7</td><td>0.61</td><td>0.99</td><td>1</td><td>−2</td><td>3</td></tr><tr><td>Level 3</td><td>16</td><td>6.13</td><td>1.15</td><td>6</td><td>5</td><td>8</td><td>6.63</td><td>1.26</td><td>6.5</td><td>4</td><td>9</td><td>0.50</td><td>0.82</td><td>1</td><td>−1</td><td>2</td></tr></tbody></table> </ephtml> </p> <p>7 <emph>Note</emph>: There were no participants with matched data in Level 1 Arabic or at any level of Bengali or Brazilian Portuguese. There was one student with matched data in Tibetan and one in Uzbek. There were two students with matched data in Level 4 Indonesian, one student in Level 2 Kazakh, one student in Level 2 Uyghur and two students in Level 3 Uyghur. None of these data were included in the table due to small sample sizes.</p> <hd id="AN0181569079-22">DISCUSSION</hd> <p></p> <hd id="AN0181569079-23">RQ1: Difference in speaking proficiency outcomes of F2F and online intensive LCTL programs</hd> <p>In the "crisis context" of the COVID‐19 global pandemic (Gacs et al., [<reflink idref="bib20" id="ref116">20</reflink>]), the fully online intensive LCTL programs in this study offered students access to language learning that they would not have had otherwise at a time when students had few other options (e.g., to enroll in a F2F US program or to study abroad). The study found that there was no difference in the speaking proficiency outcomes–as measured by the ACTFL OPI–between the 8‐week intensive LCTL programs that were offered fully online during the COVID‐19 emergency (Summers 2020 and 2021) and those that were offered in the traditional F2F format before (Summers 2018 and 2019) and after (Summer 2022) the crisis phase of the pandemic. This finding was true for all levels of instruction and for both final attainment and for gains: on average, students in both F2F and online intensive LCTL programs made significant gains in their speaking proficiency, with no significant differences between the two groups. These findings were similar to those of Merrill et al. ([<reflink idref="bib42" id="ref117">42</reflink>]), who found "comparable learning gains" (<reflink idref="bib107" id="ref118">107</reflink>) in a study that used OPI ratings to compare the speaking proficiency outcomes of a F2F domestic Russian immersion program with those of the same program when taught in a fully online intensive format during the pandemic.</p> <p>To the best of the authors' knowledge, the present study and Merrill et al. ([<reflink idref="bib42" id="ref119">42</reflink>]) are the only published studies in US higher education to date that compare the speaking proficiency outcomes of online and F2F <emph>intensive</emph> LOTE programs. The findings, however, are similar to results from studies that used measures other than the ACTFL OPI to compare the speaking proficiency outcomes of <emph>non‐intensive</emph> online and F2F LOTE courses, in both emergency and nonemergency contexts. Researchers found either no difference in the speaking proficiency outcomes (e.g., Blake et al., [<reflink idref="bib5" id="ref120">5</reflink>]; Burke Moneypenny & Aldrich, [<reflink idref="bib7" id="ref121">7</reflink>]), or slightly higher proficiency outcomes in online courses than in F2F ones (e.g., Gleason et al., [<reflink idref="bib22" id="ref122">22</reflink>]).</p> <p>It would be a mistake to interpret these findings to mean that the learning outcomes in the F2F and online contexts were identical, however. The ACTFL OPI, a "global assessment of functional speaking ability" (ACTFL, [<reflink idref="bib3" id="ref123">3</reflink>]), has been critiqued as a "blunt instrument" (Meredith, 1990, as cited in Malone, [<reflink idref="bib37" id="ref124">37</reflink>], p. 494), and proficiency tests more broadly, as Link and Li ([<reflink idref="bib31" id="ref125">31</reflink>], p. 3) emphasize, "may not fully capture learners' online language learning and development." In a recent study comparing the outcomes of F2F study abroad programs and fully online versions of those same programs offered during the pandemic, Davidson and Garas ([<reflink idref="bib15" id="ref126">15</reflink>]), for example, found qualitative differences in the speaking performance of students with similar or identical ACTFL OPI ratings, with "weaknesses" (p. 87) found in the performance of students who participated in the virtual programs (related to the use of socio‐pragmatic strategies and ease of speech, among other areas), when compared to F2F cohorts.[<reflink idref="bib11" id="ref127">11</reflink>] Moreover, this study focuses only on gains in speaking proficiency and not gains in other language modalities. More broadly, proficiency outcomes represent a subset of the many learning outcomes of educational programs in languages (Norris, [<reflink idref="bib46" id="ref128">46</reflink>]) and for intensive LCTL programs, the F2F context may provide for more than learning opportunities. In the case of WISLI, for example, the intensive summer LCTL programs are sites for academic and professional socialization and field‐building for languages and world regions that in many cases are represented by relatively small academic communities in the United States. Because many of the LCTLs in this study are offered at a relatively small number of US institutions (Looney & Lusin, [<reflink idref="bib33" id="ref129">33</reflink>]), some WISLI programs are run through formal consortia or other forms of interinstitutional partnerships, with students coming from those and other institutions from around the country to participate together in the intensive LCTL programs. The F2F LCTL programs thus serve as sites for developing academic and professional networks through interpersonal relationships, which may be difficult to replicate in online contexts.</p> <hd id="AN0181569079-24">RQ2: Speaking proficiency outcomes of intensive summer LCTL programs by language and level of...</hd> <p>The results of this study demonstrate the effectiveness of domestic intensive instruction in developing students' speaking proficiency, documented in other studies for intensive immersion programs (Ke & Reed, [<reflink idref="bib26" id="ref130">26</reflink>]; Merrill et al., [<reflink idref="bib42" id="ref131">42</reflink>]; Miano et al., [<reflink idref="bib43" id="ref132">43</reflink>]; Rifkin, [<reflink idref="bib51" id="ref133">51</reflink>], [<reflink idref="bib52" id="ref134">52</reflink>]). Similar to Rifkin, ([<reflink idref="bib51" id="ref135">51</reflink>], [<reflink idref="bib52" id="ref136">52</reflink>]), Magnan ([<reflink idref="bib35" id="ref137">35</reflink>]), Winke et al. ([<reflink idref="bib58" id="ref138">58</reflink>]), and others, this study mapped student proficiency levels on the ACTFL scale to the curriculum by level or year of instruction. Consistent with previous findings, the present study found that students' average speaking proficiency increased with the level of instruction. On average, students in the 8‐week intensive LCTL programs in this study reached between Intermediate Low and Intermediate Mid in speaking at the end of Level 1, Intermediate High at the end of Level 2, and Advanced Low at the end of Level 3. Similar to findings reported in Magnan ([<reflink idref="bib35" id="ref139">35</reflink>]) and Winke et al. ([<reflink idref="bib58" id="ref140">58</reflink>]), at all levels of instruction, there was wide variation in speaking proficiency attainment, resulting in, as Magnan ([<reflink idref="bib35" id="ref141">35</reflink>]) described it, bands of proficiency at each level of instruction with the "bands overlap[ping] from one level of study to the next" (p. 432). In this study, for example, seven ACTFL sub‐levels were represented by the postprogram ACTFL OPI ratings of Level 1 students, from Novice Low to Advanced Low; most Level 1 students reached Intermediate Low (24.81%) or Intermediate Mid (28.57%). Seven ACTFL sub‐levels were also represented in Level 2 students' postprogram ACTFL OPI ratings, from Novice High to Advanced High with 26.05% of Level 2 students reaching Intermediate Mid, 30.25% reaching Intermediate High, and 21.85% reaching Advanced Low. Park et al. ([<reflink idref="bib48" id="ref142">48</reflink>]), in a synthesis of SLA research on proficiency assessments, found that "institutional status" (p. 220), such as level of instruction, was commonly used by SLA researchers to estimate proficiency, in lieu of a formal proficiency assessment. The wide variation in the speaking proficiency of students by level of instruction in this and other empirical studies that use external measures of proficiency such as the ACTFL OPI to map student proficiency to levels of instruction, provide ample reason for not using level of instruction as a proxy for proficiency.</p> <p>Given the small size of sub‐groups, this study did not compare the speaking proficiency outcomes by language or provide frequency distributions of proficiency outcomes by language and level of instruction. From the descriptive statistics (Table 7), however, it is clear that the general finding of speaking proficiency increasing by level of instruction holds true for all of the languages in the study for which multiple levels of ACTFL OPI ratings were available. Table 7 also shows that the average final attainment in speaking proficiency, based on postprogram OPI ratings for Level 3, was quite high for some languages: between Advanced Low and Advanced Mid, for example, for Indonesian (<emph>M</emph> = 7.53, SD = 1.13), Kazakh (<emph>M</emph> = 7.50, SD = 2.08), Thai (<emph>M</emph> = 7.67, SD = 2.25), Turkish (<emph>M</emph> = 7.80, SD = 0.92), and—for the three students in Level 3 Uyghur—Advanced Mid (<emph>M</emph> = 8.00, SD = 1.00). The average final attainment in speaking proficiency in this study is comparable to that reported by Rifkin ([<reflink idref="bib52" id="ref143">52</reflink>]) for Middlebury Russian School Level 7 students, and higher than the average Intermediate Mid to Intermediate High speaking proficiency attainment for undergraduate LOTE programs found by Winke et al. ([<reflink idref="bib58" id="ref144">58</reflink>]). Domestic immersion and traditional academic year LOTE programs differ not only in the learning context, however; they also differ in the student populations. WISLI programs tend to have a relatively large proportion of graduate students, who are older than the traditional undergraduate population. Students participating in the intensive summer LCTL programs may also have scholarships or other incentives to study the language that differ from those available to students enrolled in academic year courses. These students, and individuals who choose to study a LCTL more broadly, may have specific reasons and motivations for studying the language that may differ from those of students who take courses in the more commonly taught languages (Brown, [<reflink idref="bib6" id="ref145">6</reflink>]; Magnan et al., [<reflink idref="bib34" id="ref146">34</reflink>]; Murphy et al., [<reflink idref="bib45" id="ref147">45</reflink>]).</p> <p>In terms of speaking proficiency gains, students at all levels of instruction in this study made significant gains in their speaking proficiency, with greater average gains by students in Level 1 (<emph>M</emph> = 2.63 sub‐levels of the ACTFL scale) than in Levels 2 (<emph>M</emph> = 1.26) and 3 (<emph>M</emph> = 0.84), and no significant difference in gains between Levels 2 and 3. These findings were not surprising, given other studies that show more rapid movement through lower levels of the ACTFL scale (e.g., Isbell et al., [<reflink idref="bib25" id="ref148">25</reflink>]; Magnan, [<reflink idref="bib35" id="ref149">35</reflink>]), captured visually by the "inverted pyramid" (ACTFL, [<reflink idref="bib2" id="ref150">2</reflink>], p. 4) depicting the ACTFL ratings. Similar to the wide variation found in attainment, this study found wide variation in students' gains in speaking proficiency. The majority of students in Level 1 courses made gains of two (29.63%) or three (25.93%) ACTFL sub‐levels; In Level 2, the largest group of students (41.18%) gained one ACTFL sublevel; and in Level 3, it was common for students not to make a measurable gain in speaking at all (34.09%), or to make a gain of one sublevel (31.82%). A small number of students made tremendous gains of four or more ACTFL sub‐levels; on the other end of the spectrum, a small number of students in Levels 2 and 3 showed negative gains, with lower ACTFL OPI ratings postprogram than at preprogram. On average, the speaking proficiency gains in this study were greater than those found by Zhang et al. ([<reflink idref="bib60" id="ref151">60</reflink>]), who found that students in traditional academic year LOTE programs gained an average of one ACTFL sublevel per year. In this study, there was also a wider range of gains by instructional level than those found by Rifkin ([<reflink idref="bib51" id="ref152">51</reflink>]), who reported gains of one to three ACTFL sub‐levels in speaking proficiency in the Middlebury Russian School. Isbell et al. ([<reflink idref="bib25" id="ref153">25</reflink>]) and Zhang et al. ([<reflink idref="bib60" id="ref154">60</reflink>]) looked at variables to explain differences in student proficiency growth, an area that merits exploration for this study's data set as well.</p> <p>This article shared findings related to both speaking proficiency attainment and gains by language and level of study (Tables 7 and 10), but given the small sizes of sub‐groups, did not make comparisons across languages. Moreover, the Hindi, Indonesian, Turkish, and Urdu programs in this study included RFLI students who received two additional hours of weekly conversation practice and other additional instructional support as part of their participation in the WISLI program and were preparing for a semester (or longer) of language learning‐focused in‐country or virtual study abroad. As a post hoc investigation, the researchers compared the speaking proficiency gains made by RFLI students and non‐RFLI students in those same language programs, and found that overall, RFLI students made greater average gains in their speaking proficiency than did non‐RFLI students. This trend was true across languages and levels, with the exception of Level 3 Indonesian. RFLI students in Advanced Indonesian had a higher average preprogram level of proficiency in speaking than did non‐RFLI students, however, so the relatively lower average gain score of RFLI students in that case may be explained in part by a ceiling effect. The average speaking proficiency outcomes of the intensive programs in the Hindi, Indonesian, Turkish, and Urdu programs would thus have been lower without the group of RFLI students.</p> <p>The small size of sub‐groups by language and level of instruction is a major limitation of this study, leading to results that are not generalizable. This limitation is a common one in research in instructional programs in LCTLs, however, given the relatively small size of many LCTL programs. The study also did not look at variables (e.g., students' background, their motivation for studying the language, or their language of study) that might explain differences in outcomes. Understanding the wide variation in student gains by looking at these and other variables merits further exploration. Future research might also employ qualitative measures to, for example, investigate the subjective experiences of students and instructors in the F2F and online learning environments, to complement the findings of the quantitative study presented.</p> <hd id="AN0181569079-25">CONCLUSION</hd> <p>Malone ([<reflink idref="bib36" id="ref155">36</reflink>], p. 312) argues that "understanding the proficiency levels and what is attainable during different learning sequences is crucial for language instructors to move their students to high levels of proficiency without expecting either too little or too much from them." This study is the first to document the speaking proficiency outcomes of intensive programs in numerous LCTLs using an external measure, the ACTFL OPI, providing initial benchmarks for similar programs when developing goals for their learners. With data from over 5 years and at three levels of instruction, findings from this study provide initial insights into what is attainable in terms of speaking proficiency in the 14 languages of this study, by adult learners in F2F or online intensive learning contexts. Overall, results from the study speak to the effectiveness of domestic intensive instruction in providing opportunities for US students of LCTLs to make significant gains in their speaking abilities over a relatively short period of time (i.e., 8 weeks). The study also revealed that within levels of instruction, there was great variation among students in their speaking proficiency glains, which is an area for further investigation.</p> <hd id="AN0181569079-26">ACKNOWLEDGMENTS</hd> <p>This study was a collaborative initiative of the University of Wisconsin‐Madison Language Institute and the Wisconsin Intensive Summer Language Institutes. Funding for the study was from a grant from the U.S. Department of Education International Research and Studies Program, an ACTFL Research Priorities grant, and the UW‐Madison Language Institute. The contents of this article were developed under grant # P017A200014‐21 from the U.S. Department of Education. However, those contents do not necessarily represent the policy of the U.S. Department of Education, and you should not assume endorsement by the Federal Government. The authors extend heartfelt gratitude to Dr. Jana Martin, UW‐Madison Language Institute, who assisted with the study design and with data collection, and oversaw the management of the grants that funded the study; Dr. Felecia Lucht, WISLI, who provided input on the study design and data collection procedures, coordinated with WISLI programs on data collection, and provided information about the WISLI programs for this article; Dr. Meg Malone, ACTFL, who provided expert guidance and support as an outside evaluator throughout the project; and Hadis Ghaedi, UW‐Madison Second Language Acquisition PhD Program, who provided input on the data collection. The authors would like to thank all WISLI students who participated in the study, and WISLI instructors and staff who assisted with data collection. We also thank Dan Davidson, American Councils for International Education, and Ed McDermott, National Security Education Program, for sharing information and references on proficiency research in Language Flagship Programs. Finally, we thank Dan Soneson, University of Minnesota, for sharing the background questionnaire used for the Language Proficiency Flagship Initiative.</p> <p>GRAPH: Supporting information</p> <ref id="AN0181569079-27"> <title> Footnotes </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> On its website, the National Council of Less Commonly Taught Languages (NCOLCTL) differentiates between French, German, Italian, and Spanish, studied by "approximately 91% of Americans who study foreign languages in our schools, colleges and universities" (https://ncolctl.org), and all other languages, which are classified as "less commonly taught." The Modern Language Association (Looney & Lusin, [33]) defines LCTLs somewhat differently: as all languages not in the top 15 nationally in terms of postsecondary enrollments (5). Following the NCOLCTL definition of LCTLs, all of the languages in this study would be considered LCTLs; following the MLA's definition, the same is true, with the exception of Arabic and Portuguese (Looney & Lusin, [33]).</bibtext> </blist> <blist> <bibl id="bib2" idref="ref4" type="bt">2</bibl> <bibtext> Hodges et al. ([24]) cautioned against comparing planned online instruction and emergency remote teaching, which Hodges et al. ([24]) define as "a temporary shift of instructional delivery to an alternate delivery mode due to crisis circumstances."</bibtext> </blist> <blist> <bibl id="bib3" idref="ref10" type="bt">3</bibl> <bibtext> The OPI was initially developed for US government purposes and later modified for use in educational contexts by ACTFL and the Educational Testing Service (Cox et al., [9]; Malone, [37]). The ACTFL OPI is a structured interview conducted by a certified tester and rated according to the scale in the <emph>ACTFL Proficiency Guidelines 2012‐Speaking</emph> or on the Interagency Language Roundtable (ILR) scale used by the US government (Malone & Montee, [38], 976). The ACTFL scale is comprised of five major levels—Novice, Intermediate, Advanced, Superior, and Distinguished—with sublevels of Low, Mid, and High for the lower three major levels (ACTFL, [3]).</bibtext> </blist> <blist> <bibl id="bib4" idref="ref14" type="bt">4</bibl> <bibtext> There is a large body of research on the proficiency outcomes of study abroad that is not reviewed here. See, for example, Davidson, ([14], [11]) and Mason et al. ([40]).</bibtext> </blist> <blist> <bibl id="bib5" idref="ref43" type="bt">5</bibl> <bibtext> See Norris & Pfeiffer (2003) and Zhang et al. ([60]) for reviews of studies that examine proficiency outcomes of lower‐level LOTE courses.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref51" type="bt">6</bibl> <bibtext> For more information about The Language Flagship, see thelanguageflagship.org.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref56" type="bt">7</bibl> <bibtext> There is no published research from domestic Language Flagship programs that maps student proficiency gains to the domestic curriculum. There are a number of studies, however, that report on student proficiency development and final attainment in Flagship overseas capstone programs (Davidson & Garas, [15]; Davidson & Lekic, [12]; Davidson & Shaw, [13]; Davidson et al., [10]; Davidson, [14], [11]; Nugent & Slater, [47]).</bibtext> </blist> <blist> <bibl id="bib8" idref="ref5" type="bt">8</bibl> <bibtext> As a reviewer of this article rightly noted, the costs associated with administering standardized tests may be prohibitive. Standardized tests, moreover, may not align well with the instructional goals or curriculum of a given program.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref83" type="bt">9</bibl> <bibtext> The intensive Brazilian Portuguese program is for students with 2–3 years of instruction in Spanish or another Romance language.</bibtext> </blist> <blist> <bibtext> For more information about the instructional plan for offering the intensive programs online during the pandemic, see <emph>Wisconsin Intensive Summer Language Institutes</emph> ([4]) <emph>Remote Synchronous & Asynchronous Teaching, Remote Language Instruction for Summer 2020</emph> (https://wisli.wisc.edu/wp-content/uploads/sites/239/2020/05/May%5f2020%5fWISLI%5fRemoteLanguageTraining.doc) and <emph>Program X Remote Synchronous & Asynchronous Teaching, Remote Language Instruction for Summer 2021</emph>(https://wisli.wisc.edu/wp-content/uploads/sites/239/2021/03/032421%5fWISLI%5fRemoteInstructionalPlan.pdf) (WISLI, 2020).</bibtext> </blist> <blist> <bibtext> Unlike the studies cited above, Davidson and Garas ([15]) found that on average, students in the virtual study abroad programs on average achieved one ACTFL sublevel lower in speaking than did students participating in the F2F study abroad programs.</bibtext> </blist> </ref> <ref id="AN0181569079-28"> <title> REFERENCES </title> <blist> <bibtext> Laerd Statistics. (n.d.). Mixed ANOVA using SPSS Statistics. https://statistics.laerd.com/spss-tutorials/mixed-anova-using-spss-statistics.php.</bibtext> </blist> <blist> <bibtext> ACTFL. (2012a). ACTFL oral proficiency interview familiarization manual. Author.</bibtext> </blist> <blist> <bibtext> ACTFL. (2012b). ACTFL proficiency guidelines 2012. Author. Retrieved June 6, 2023 from: https://<ulink href="http://www.actfl.org/uploads/files/general/ACTFLProficiencyGuidelines2012.pdf">www.actfl.org/uploads/files/general/ACTFLProficiencyGuidelines2012.pdf</ulink></bibtext> </blist> <blist> <bibtext> Bernhardt, E., Nissler, P., & Leffell, C. M. (2020). An integrated curriculum in collegiate‐level German studies with accompanying proficiency ratings. Die Unterrichtspraxis/Teaching German, 53 (1), 23 – 34.</bibtext> </blist> <blist> <bibtext> Blake, R., Wilson, N. L., Cetto, M., & Pardo‐Ballester, C. (2008). Measuring oral proficiency in distance, face‐to‐face, and blended classrooms. Language Learning & Technology, 12 (3), 114 – 127.</bibtext> </blist> <blist> <bibtext> Brown, A. V. (2009). Less commonly taught language and commonly taught language students: A demographic and academic comparison. Foreign Language Annals, 42 (3), 405 – 423.</bibtext> </blist> <blist> <bibtext> Burke Moneypenny, D., & Aldrich, R. (2016). Online and face‐to‐face language learning: A comparative analysis of oral proficiency in introductory Spanish. The Journal of Educators Online, 13, 105 – 174.</bibtext> </blist> <blist> <bibtext> Collins, L., & Muñoz, C. (2016). The foreign language classroom: Current perspectives and future considerations. The Modern Language Journal, 100 (S1), 133 – 147.</bibtext> </blist> <blist> <bibtext> Cox, T. L., Malone, M. E., & Winke, P. (2018). Future directions in assessment: Influences of standards and implications for language learning. Foreign Language Annals, 51 (1), 104 – 115.</bibtext> </blist> <blist> <bibtext> Davidson, D., Garas, N., & Lekic, M. (2017). Assessing language proficiency and intercultural development in the overseas immersion context. In D. Murphy, & K. Evans‐Romaine (Eds.), Exploring the U.S. language flagship program: Professional competence in a second language by graduation (pp. 156 – 176). Multilingual Matters.</bibtext> </blist> <blist> <bibtext> Davidson, D. (2015). The development of L2 proficiency and literacy within the context of the federally supported overseas language training programs for Americans. In J. Bown, & T. Brown (Eds.), To advanced proficiency and beyond: Theory and methods for developing superior second‐language ability (pp. 117 – 150). Georgetown University Press.</bibtext> </blist> <blist> <bibtext> Davidson, D., & Lekic, M. D. (2010). The overseas immersion setting as contextual variable in adult SLA: Learner behaviors associated with language gain to level‐3 proficiency in Russian. Russian Language Journal, 60 (1), 55 – 78.</bibtext> </blist> <blist> <bibtext> Davidson, D., & Shaw, J. (2019). A cross‐linguistic and cross‐skill perspective on L2 development in study abroad. In P. Winke, & S. M. Gass (Eds.), Foreign language proficiency in higher education (pp. 217 – 242). Springer.</bibtext> </blist> <blist> <bibtext> Davidson, D. E. (2010). Study abroad: When, how long, and with what results? New data from the Russian front. Foreign Language Annals, 43 (1), 6 – 26.</bibtext> </blist> <blist> <bibtext> Davidson, D. E., & Garas, N. (2023). What makes study abroad transformative? Comparing linguistic and cultural contacts and learning outcomes in virtual vs in‐person contexts. L2 Journal, 15 (2), 71 – 91.</bibtext> </blist> <blist> <bibtext> Defense Language and National Security Education Office (DLNSEO). (2022). 2022 National Security Education Program Annual Report. Retrieved August 18, 2023 from: https://dlnseo.org/sites/default/files/2022NSEPAnnualReport.pdf</bibtext> </blist> <blist> <bibtext> Van Deusen‐Scholl, N. (2015). Assessing outcomes in online foreign language education: What are key measures for success? The Modern Language Journal, 99 (2), 398 – 400.</bibtext> </blist> <blist> <bibtext> Dewey, D. P. (2004). A comparison of reading development by learners of Japanese in intensive domestic immersion and study abroad contexts. Studies in Second Language Acquisition, 26 (2), 303 – 327.</bibtext> </blist> <blist> <bibtext> Freed, B. F., Segalowitz, N., & Dewey, D. P. (2004). Context of learning and second language fluency in French. Studies in Second Language Acquisition, 26, 275 – 301.</bibtext> </blist> <blist> <bibtext> Gacs, A., Goertler, S., & Spasova, S. (2020). Planned online language education versus crisis‐prompted online language teaching: Lessons for the future. Foreign Language Annals, 53 (2), 380 – 392.</bibtext> </blist> <blist> <bibtext> Gass, S., & Winke, P. (2019). Proficiency testing in the U.S. context: An introduction. In P. Winke, & S. M. Gass (Eds.), Foreign language proficiency in higher education (pp. 3 – 13). Springer.</bibtext> </blist> <blist> <bibtext> Gleason, J., Cardone, R., & Bartlett, A. (2022). The impact of the pandemic on student Spanish language proficiency, EdArXiv Preprints : https://doi.org/10.35542/osf.io/g4qd8</bibtext> </blist> <blist> <bibtext> Grgurović, M., Chapelle, C. A., & Shelley, M. C. (2013). A meta‐analysis of effectiveness studies on computer technology‐supported language learning. ReCALL, 25 (2), 165 – 198.</bibtext> </blist> <blist> <bibtext> Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020). The difference between emergency remote teaching and online learning. Retrieved from: https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning</bibtext> </blist> <blist> <bibtext> Isbell, D. R., Winke, P., & Gass, S. M. (2019). Using the ACTFL OPIc to assess proficiency and monitor progress in a tertiary foreign languages program. Language Testing, 36 (3), 439 – 465.</bibtext> </blist> <blist> <bibtext> Ke, C., & Reed, D. J. (1995). An analysis of results from the ACTFL oral proficiency interview and the Chinese proficiency test before and after intensive instruction in Chinese as a foreign language. Foreign Language Annals, 28 (2), 208 – 222.</bibtext> </blist> <blist> <bibtext> Kenyon, D., & Malabonga, V. (2001). Comparing examinee attitudes toward computer‐assisted and other oral proficiency assessments. Language Learning & Technology, 5 (2), 60 – 83.</bibtext> </blist> <blist> <bibtext> Kim, H. Y. (2013). Statistical notes for clinical researchers: Assessing normal distribution (2) using skewness and kurtosis. Restorative Dentistry & Endodontics, 38 (1), 52 – 54. https://doi.org/10.5395/rde.2013.38.1.52</bibtext> </blist> <blist> <bibtext> Kissau, S., J. Davin, K., & Wang, C. (2019). Enhancing instructor candidate oral proficiency through interdepartmental collaboration. Foreign Language Annals, 52 (2), 358 – 372.</bibtext> </blist> <blist> <bibtext> Lin, C.‐H., & Warschauer, M. (2015). Online foreign language education: What are the proficiency outcomes? The Modern Language Journal, 99 (2), 394 – 397.</bibtext> </blist> <blist> <bibtext> Link, S., & Li, J. (2018). The online language learning imperative: Maximizing assessment practices to ensure student success. In S. Link, & J. Li (Eds.), Assessment across online language education (pp. 1 – 18). Equinox Publishing.</bibtext> </blist> <blist> <bibtext> Liskin‐Gasparro, J. E. (2003). The ACTFL proficiency guidelines and the oral proficiency interview: A brief history and analysis of their survival. Foreign Language Annals, 36 (4), 483 – 490.</bibtext> </blist> <blist> <bibtext> Looney, D., & Lusin, N. (2019). Enrollments in languages other than English in United States institutions of higher education, Summer 2016 and Fall 2016: Final report. Modern Language Association.</bibtext> </blist> <blist> <bibtext> Magnan, S., Murphy, D., & Sahakyan, N. (2014). Goals of collegiate learners and the standards for foreign language learning. Modern Language Journal Monograph Series, 98, S1t. https://onlinelibrary.wiley.com/toc/15404781/2014/98/S1</bibtext> </blist> <blist> <bibtext> Magnan, S. S. (1986). Assessing speaking proficiency in the undergraduate curriculum: Data from French. Foreign Language Annals, 19 (5), 429 – 438.</bibtext> </blist> <blist> <bibtext> Malone, M. (2019). Afterword and next steps. In P. Winke, & S. Gass (Eds.), Foreign language proficiency in higher education (pp. 309 – 318). Springer.</bibtext> </blist> <blist> <bibtext> Malone, M. E. (2003). Research on the oral proficiency interview: Analysis, synthesis, and future directions. Foreign Language Annals, 36 (4), 491 – 497.</bibtext> </blist> <blist> <bibtext> Malone, M. E., & Montee, M. J. (2010). Oral proficiency assessment: Current approaches and applications for post‐secondary foreign language programs. Language and Linguistics Compass, 4, 972 – 986.</bibtext> </blist> <blist> <bibtext> Martinsen, R. A., Baker, W., Bown, J., & Johnson, C. (2011). The benefits of living in foreign language housing: The effect of language use and second‐language type on oral proficiency gains. The Modern Language Journal, 95 (2), 274 – 290.</bibtext> </blist> <blist> <bibtext> Mason, L., Powers, C., & Donnelly, S. (2015). The Boren awards: A report of oral language proficiency gains during academic study abroad. A cumulative report over 15 years and 53 languages. Institute of International Education.</bibtext> </blist> <blist> <bibtext> Maxwell, S. E., & Delaney, H. D. (2004). Designing experiments and analyzing data: A model comparison perspective (2nd ed.). Lawrence Erlbaum Associates Publishers.</bibtext> </blist> <blist> <bibtext> Merrill, J., Dengub, E., & Pastushenkov, D. (2021). Language gains in intensive synchronous online and face‐to‐face Russian immersion programs: A comparison. Russian Language Journal, 71 (2), 87 – 120.</bibtext> </blist> <blist> <bibtext> Miano, A. A., Bernhardt, E. B., & Brates, V. (2016). Exploring the effects of a short‐term spanish immersion program in a postsecondary setting. Foreign Language Annals, 49 (2), 287 – 301.</bibtext> </blist> <blist> <bibtext> Murphy, D., Evans‐Romaine, K., Anishchenkova, V., & Jing‐Schmidt, Z. (2017). Laying the groundwork: Programmatic models in U.S. language flagship programs. In D. Murphy, & K. Evans‐Romaine (Eds.), Exploring the U.S. language flagship program: Professional competence in a second language by graduation (pp. 29 – 50). Multilingual Matters.</bibtext> </blist> <blist> <bibtext> Murphy, D., Magnan, S., Back, M., & Garrett‐Rucks, P. (2009). Reasons students take courses in less commonly taught languages. Journal of the National Council of Less Commonly Taught Languages, 7, 45 – 80.</bibtext> </blist> <blist> <bibtext> Norris, J. M. (2006). The issue: The why (and how) of assessing student learning outcomes in college foreign language programs. The Modern Language Journal, 90 (4), 576 – 583.</bibtext> </blist> <blist> <bibtext> Nugent, M., & Slater, R. (2017). The language flagship: Creating expectations and opportunities for professional‐level language learning in undergraduate education. In D. Murphy, & K. Evans‐Romaine (Eds.), Exploring the U.S. Language Flagship Program: Professional competence in a second language by graduation (pp. 9 – 28). Multilingual Matters.</bibtext> </blist> <blist> <bibtext> Park, H. I., Solon, M., Dehghan‐Chaleshtori, M., & Ghanbar, H. (2021). Proficiency reporting practices in research on second language acquisition: Have we made any progress? Language learning, 72 (1), 108 – 237.</bibtext> </blist> <blist> <bibtext> Plonsky, L., & Oswald, F. L. (2014). How big is "big"? Interpreting effect sizes in L2 research. Language learning, 64 (4), 878 – 912. https://doi.org/10.1111/lang.12079</bibtext> </blist> <blist> <bibtext> Plonsky, L., & Ziegler, N. (2016). The CALL‐SLA interface: Insights from a second‐order synthesis. Language Learning & Technology, 20 (2), 17 – 37.</bibtext> </blist> <blist> <bibtext> Rifkin, B. (2003). Oral proficiency learning outcomes and curricular design. Foreign Language Annals, 36 (4), 582 – 588.</bibtext> </blist> <blist> <bibtext> Rifkin, B. (2005). A ceiling effect in traditional classroom foreign language instruction: Data from Russian. The Modern Language Journal, 89 (1), 3 – 18.</bibtext> </blist> <blist> <bibtext> Swender, E. (2003). Oral proficiency testing in the real world: Answers to frequently asked questions. Foreign Language Annals, 36 (4), 520 – 526.</bibtext> </blist> <blist> <bibtext> Tigchelaar, M. (2019). Exploring the relationship between self‐assessments and OPIc ratings of oral proficiency in French. In P. Winke, & S. M. Gass (Eds.), Foreign language proficiency in higher education (pp. 153 – 173). Springer.</bibtext> </blist> <blist> <bibtext> Tschirner, E. (2018). Language testing: Current practices and future developments. Die Unterrichtspraxis/Teaching German, 51 (2), 105 – 120.</bibtext> </blist> <blist> <bibtext> Urlaub, P. (2019). When less becomes more: On the growth of less commonly taught languages in adult immersion programs. Journal of the National Council of Less Commonly Taught Languages, 26 (2), 73 – 86.</bibtext> </blist> <blist> <bibtext> Winke, P., Gass, S., & Heidrich, E. S. (2019). Modern‐day foreign language majors: Their goals, attainment, and fit within a twenty‐first century curriculum. In P. Winke, & S. M. Gass (Eds.), Foreign language proficiency in higher education (pp. 93 – 113). Springer.</bibtext> </blist> <blist> <bibtext> Winke, P., Zhang, X., Rubio, F., Gass, S., Soneson, D., & Hacking, J. (2020). The proficiency profile of language students: Implications for programs. Second Language Research and Practice, 1 (1), 25 – 64.</bibtext> </blist> <blist> <bibtext> Wisconsin Intensive Summer Language Institutes. (2020). Wisconsin Intensive Summer Language Institutes (WISLI) remote synchronous and asynchronous teaching. Author. Retrieved September 8, 2023 from: https://wisli.wisc.edu/wp-content/uploads/sites/239/2021/03/032421_WISLI_RemoteInstructionalPlan.pdf</bibtext> </blist> <blist> <bibtext> Zhang, X., Winke, P., & Clark, S. (2020). Background characteristics and oral proficiency development over time in lower‐division college foreign language programs. Language learning, 70 (3), 807 – 847.</bibtext> </blist> </ref> <aug> <p>By Dianna Murphy and Sonya K. Sedivy</p> <p>Reported by Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib20" firstref="ref2"></nolink> <nolink nlid="nl2" bibid="bib24" firstref="ref3"></nolink> <nolink nlid="nl3" bibid="bib30" firstref="ref6"></nolink> <nolink nlid="nl4" bibid="bib17" firstref="ref7"></nolink> <nolink nlid="nl5" bibid="bib38" firstref="ref8"></nolink> <nolink nlid="nl6" bibid="bib36" firstref="ref9"></nolink> <nolink nlid="nl7" bibid="bib32" firstref="ref11"></nolink> <nolink nlid="nl8" bibid="bib37" firstref="ref13"></nolink> <nolink nlid="nl9" bibid="bib25" firstref="ref15"></nolink> <nolink nlid="nl10" bibid="bib35" firstref="ref16"></nolink> <nolink nlid="nl11" bibid="bib53" firstref="ref17"></nolink> <nolink nlid="nl12" bibid="bib55" firstref="ref18"></nolink> <nolink nlid="nl13" bibid="bib57" firstref="ref19"></nolink> <nolink nlid="nl14" bibid="bib58" firstref="ref20"></nolink> <nolink nlid="nl15" bibid="bib60" firstref="ref21"></nolink> <nolink nlid="nl16" bibid="bib15" firstref="ref22"></nolink> <nolink nlid="nl17" bibid="bib13" firstref="ref23"></nolink> <nolink nlid="nl18" bibid="bib10" firstref="ref24"></nolink> <nolink nlid="nl19" bibid="bib14" firstref="ref25"></nolink> <nolink nlid="nl20" bibid="bib11" firstref="ref26"></nolink> <nolink nlid="nl21" bibid="bib47" firstref="ref27"></nolink> <nolink nlid="nl22" bibid="bib26" firstref="ref28"></nolink> <nolink nlid="nl23" bibid="bib42" firstref="ref29"></nolink> <nolink nlid="nl24" bibid="bib43" firstref="ref30"></nolink> <nolink nlid="nl25" bibid="bib51" firstref="ref31"></nolink> <nolink nlid="nl26" bibid="bib52" firstref="ref32"></nolink> <nolink nlid="nl27" bibid="bib39" firstref="ref33"></nolink> <nolink nlid="nl28" bibid="bib29" firstref="ref34"></nolink> <nolink nlid="nl29" bibid="bib18" firstref="ref37"></nolink> <nolink nlid="nl30" bibid="bib19" firstref="ref38"></nolink> <nolink nlid="nl31" bibid="bib21" firstref="ref45"></nolink> <nolink nlid="nl32" bibid="bib16" firstref="ref53"></nolink> <nolink nlid="nl33" bibid="bib44" firstref="ref54"></nolink> <nolink nlid="nl34" bibid="bib56" firstref="ref58"></nolink> <nolink nlid="nl35" bibid="bib23" firstref="ref71"></nolink> <nolink nlid="nl36" bibid="bib50" firstref="ref72"></nolink> <nolink nlid="nl37" bibid="bib31" firstref="ref73"></nolink> <nolink nlid="nl38" bibid="bib22" firstref="ref79"></nolink> <nolink nlid="nl39" bibid="bib27" firstref="ref85"></nolink> <nolink nlid="nl40" bibid="bib54" firstref="ref99"></nolink> <nolink nlid="nl41" bibid="bib28" firstref="ref100"></nolink> <nolink nlid="nl42" bibid="bib41" firstref="ref101"></nolink> <nolink nlid="nl43" bibid="bib482" firstref="ref102"></nolink> <nolink nlid="nl44" bibid="bib101" firstref="ref113"></nolink> <nolink nlid="nl45" bibid="bib87" firstref="ref114"></nolink> <nolink nlid="nl46" bibid="bib49" firstref="ref115"></nolink> <nolink nlid="nl47" bibid="bib107" firstref="ref118"></nolink> <nolink nlid="nl48" bibid="bib46" firstref="ref128"></nolink> <nolink nlid="nl49" bibid="bib33" firstref="ref129"></nolink> <nolink nlid="nl50" bibid="bib48" firstref="ref142"></nolink> <nolink nlid="nl51" bibid="bib34" firstref="ref146"></nolink> <nolink nlid="nl52" bibid="bib45" firstref="ref147"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1452841
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: The Speaking Proficiency Outcomes of Face-to-Face and Online Intensive Summer LCTL Programs
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Dianna+Murphy%22">Dianna Murphy</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0001-9716-0032">0000-0001-9716-0032</externalLink>)<br /><searchLink fieldCode="AR" term="%22Sonya+K%2E+Sedivy%22">Sonya K. Sedivy</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-5031-7837">0000-0002-5031-7837</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Foreign+Language+Annals%22"><i>Foreign Language Annals</i></searchLink>. 2024 57(4):872-899.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 28
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Office of Postsecondary Education (ED)<br />Department of Education (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: P017A20001421
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Speech+Communication%22">Speech Communication</searchLink><br /><searchLink fieldCode="DE" term="%22Second+Language+Learning%22">Second Language Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Second+Language+Instruction%22">Second Language Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Proficiency%22">Language Proficiency</searchLink><br /><searchLink fieldCode="DE" term="%22Oral+Language%22">Oral Language</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Tests%22">Language Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Arabic%22">Arabic</searchLink><br /><searchLink fieldCode="DE" term="%22Sino+Tibetan+Languages%22">Sino Tibetan Languages</searchLink><br /><searchLink fieldCode="DE" term="%22Turkic+Languages%22">Turkic Languages</searchLink><br /><searchLink fieldCode="DE" term="%22Indo+European+Languages%22">Indo European Languages</searchLink><br /><searchLink fieldCode="DE" term="%22Dravidian+Languages%22">Dravidian Languages</searchLink><br /><searchLink fieldCode="DE" term="%22Portuguese%22">Portuguese</searchLink><br /><searchLink fieldCode="DE" term="%22Indonesian%22">Indonesian</searchLink><br /><searchLink fieldCode="DE" term="%22Turkish%22">Turkish</searchLink><br /><searchLink fieldCode="DE" term="%22Urdu%22">Urdu</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Variation%22">Language Variation</searchLink><br /><searchLink fieldCode="DE" term="%22Intensive+Language+Courses%22">Intensive Language Courses</searchLink><br /><searchLink fieldCode="DE" term="%22Outcomes+of+Education%22">Outcomes of Education</searchLink><br /><searchLink fieldCode="DE" term="%22Uncommonly+Taught+Languages%22">Uncommonly Taught Languages</searchLink><br /><searchLink fieldCode="DE" term="%22Online+Courses%22">Online Courses</searchLink><br /><searchLink fieldCode="DE" term="%22COVID-19%22">COVID-19</searchLink><br /><searchLink fieldCode="DE" term="%22Pandemics%22">Pandemics</searchLink><br /><searchLink fieldCode="DE" term="%22Summer+Programs%22">Summer Programs</searchLink><br /><searchLink fieldCode="DE" term="%22In+Person+Learning%22">In Person Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Second+Language+Programs%22">Second Language Programs</searchLink>
– Name: SubjectThesaurus
  Label: Assessment and Survey Identifiers
  Group: Su
  Data: <searchLink fieldCode="SU" term="%22ACTFL+Oral+Proficiency+Interview%22">ACTFL Oral Proficiency Interview</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/flan.12758
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0015-718X<br />1944-9720
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This article is the first large-scale study to document the speaking proficiency outcomes of intensive programs in less commonly taught languages in US higher education. Speaking proficiency was measured by pre- and postprogram ACTFL Oral Proficiency Interviews (N = 484) in 14 languages: Arabic, Bengali, Brazilian Portuguese, Hindi, Indonesian, Kazakh, Persian, Tamil, Thai, Tibetan, Turkish, Urdu, Uyghur, and Uzbek. The intensive programs are usually taught face-to-face but were offered fully online during the COVID-19 pandemic. No significant difference in speaking proficiency outcomes, as measured by the ACTFL OPI, was found between the face-to-face and online formats. Students at all levels of instruction made significant gains in speaking proficiency, with greater gains made by students in Level 1 than at Levels 2-3, and with wide variation within instructional levels. The average speaking proficiency outcomes were Advanced Low (Level 3), Intermediate High (Level 2), and between Intermediate Low and Intermediate Mid (Level 1).
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1452841
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1452841
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/flan.12758
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 28
        StartPage: 872
    Subjects:
      – SubjectFull: Speech Communication
        Type: general
      – SubjectFull: Second Language Learning
        Type: general
      – SubjectFull: Second Language Instruction
        Type: general
      – SubjectFull: Language Proficiency
        Type: general
      – SubjectFull: Oral Language
        Type: general
      – SubjectFull: Language Tests
        Type: general
      – SubjectFull: Arabic
        Type: general
      – SubjectFull: Sino Tibetan Languages
        Type: general
      – SubjectFull: Turkic Languages
        Type: general
      – SubjectFull: Indo European Languages
        Type: general
      – SubjectFull: Dravidian Languages
        Type: general
      – SubjectFull: Portuguese
        Type: general
      – SubjectFull: Indonesian
        Type: general
      – SubjectFull: Turkish
        Type: general
      – SubjectFull: Urdu
        Type: general
      – SubjectFull: Language Variation
        Type: general
      – SubjectFull: Intensive Language Courses
        Type: general
      – SubjectFull: Outcomes of Education
        Type: general
      – SubjectFull: Uncommonly Taught Languages
        Type: general
      – SubjectFull: Online Courses
        Type: general
      – SubjectFull: COVID-19
        Type: general
      – SubjectFull: Pandemics
        Type: general
      – SubjectFull: Summer Programs
        Type: general
      – SubjectFull: In Person Learning
        Type: general
      – SubjectFull: Second Language Programs
        Type: general
      – SubjectFull: ACTFL Oral Proficiency Interview
        Type: general
    Titles:
      – TitleFull: The Speaking Proficiency Outcomes of Face-to-Face and Online Intensive Summer LCTL Programs
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Dianna Murphy
      – PersonEntity:
          Name:
            NameFull: Sonya K. Sedivy
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 0015-718X
            – Type: issn-electronic
              Value: 1944-9720
          Numbering:
            – Type: volume
              Value: 57
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Foreign Language Annals
              Type: main
ResultId 1