View in EDS HTML Full Text PDF Full Text

A Meta-Analysis of Effects of Automated Writing Evaluation on Anxiety, Motivation, and Second Language Writing Skills

Saved in:
Bibliographic Details
Title: A Meta-Analysis of Effects of Automated Writing Evaluation on Anxiety, Motivation, and Second Language Writing Skills
Language: English
Authors: Xiaoli Huang (ORCID 0009-0006-4782-0446), Wei Xu (ORCID 0000-0002-7224-1116), Fan Li (ORCID 0009-0008-6733-6795), Zhonggen Yu (ORCID 0000-0002-3873-980X)
Source: Asia-Pacific Education Researcher. 2024 33(4):957-976.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 20
Publication Date: 2024
Document Type: Journal Articles
Information Analyses
Descriptors: Writing Evaluation, Automation, Writing Attitudes, Anxiety, Computer Attitudes, Writing Skills, Motivation, Second Language Learning, Writing Achievement, Second Language Instruction
DOI: 10.1007/s40299-024-00865-y
ISSN: 0119-5646
2243-7908
Abstract: With the rapid advancement of information technologies, automated writing evaluation technologies have developed so fast that they can be applied to writing assessments. However, scanty studies have pooled the effects of automated writing evaluation on writing performance. Through a PRISMA protocol-based meta-analysis, this study concludes that automated writing evaluation technologies can significantly reduce anxiety such as writing anxiety and computer anxiety compared with those without the assistance of automated writing evaluation. Automated writing evaluation can also significantly improve writing motivation and second language (L2) writing skills. Educators and designers should highlight how to combine automated writing evaluation with human rating methods and maximize the improvements in L2 writing skills. In the future, artificial intelligence may be integrated into automated writing evaluation to develop advanced automated writing evaluation to address higher-level issues in writing practice.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1430873
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwEN_UmEDOWmj_-Qj5t9F-Y9AAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDKPtElaiAlHTPyEreAIBEICBmx5Sra4ObRwY4RGZe78WzA7FdCsqkOl10LhpsGFVYe0jP-fVlSv5AcDosFe8fu1CN8XJe6lIzbpfWov_DTg4eqBnE5JAKRVuZ5Zb2dRvCHxHjHAvioVv6fRnPvgSNP1chU64ury8ipXQepbkkPcUkbTXKFssMd1CaX1ygFmxs3oVVkQWdXjQ-_phz4feS12SqhlSnlt0eLDThukn
Text:
  Availability: 1
  Value: <anid>AN0178463698;[gchw]01aug.24;2024Jul18.05:38;v2.2.500</anid> <title id="AN0178463698-1">A Meta-analysis of Effects of Automated Writing Evaluation on Anxiety, Motivation, and Second Language Writing Skills </title> <p>With the rapid advancement of information technologies, automated writing evaluation technologies have developed so fast that they can be applied to writing assessments. However, scanty studies have pooled the effects of automated writing evaluation on writing performance. Through a PRISMA protocol-based meta-analysis, this study concludes that automated writing evaluation technologies can significantly reduce anxiety such as writing anxiety and computer anxiety compared with those without the assistance of automated writing evaluation. Automated writing evaluation can also significantly improve writing motivation and second language (L2) writing skills. Educators and designers should highlight how to combine automated writing evaluation with human rating methods and maximize the improvements in L2 writing skills. In the future, artificial intelligence may be integrated into automated writing evaluation to develop advanced automated writing evaluation to address higher-level issues in writing practice.</p> <p>Keywords: Automated writing evaluation; Anxiety; Motivation; L2 writing skills</p> <p>Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/s40299-024-00865-y.</p> <hd id="AN0178463698-2">Introduction</hd> <p>The fast-paced evolution of information technologies has led to remarkable innovations in writing evaluation, particularly in the new century. One such innovation that has gained immense popularity is the Automated Writing Evaluation (AWE) system. This technology has revolutionized the way writing tasks are evaluated, thanks to its remarkable efficiency and accuracy. Educators have increasingly embraced AWE systems as a replacement for traditional human evaluation. This shift can be attributed to several factors. Firstly, AWE systems often outperform human raters in terms of evaluation accuracy and consistency. These systems, powered by machine learning algorithms, can analyze student writing with a high degree of precision, identifying areas of strength and weakness with greater objectivity. Shermis and Hamner ([<reflink idref="bib68" id="ref1">68</reflink>], [<reflink idref="bib69" id="ref2">69</reflink>]) found that AWE systems demonstrated superior performance in writing evaluation, surpassing human raters in terms of evaluation accuracy, reliability, and efficiency.</p> <p>Moreover, AWE systems can significantly reduce the workload of educators. These systems can automatically score and grade writing tasks, enabling educators to quickly identify areas of improvement and provide timely feedback to students (Abbas et al., [<reflink idref="bib2" id="ref3">2</reflink>]). This not only saves educators' time and effort but also ensures that students receive feedback on their writing skills more efficiently. Multiple studies, such as those conducted by Weigle ([<reflink idref="bib82" id="ref4">82</reflink>]) and Rudner et al. ([<reflink idref="bib62" id="ref5">62</reflink>]), have demonstrated the operational effectiveness of AWE systems in scoring writing assignments and evaluating the quality of student writing. Additionally, AWE systems offer a high degree of consistency in writing evaluation. Unlike human raters, who may be subject to biases or inconsistent evaluation criteria, AWE systems use a standardized and objective approach to evaluate student writing (Shadiev et al., [<reflink idref="bib66" id="ref6">66</reflink>]). This consistency ensures fairness and objectivity in the evaluation process, providing students with a clear understanding of their writing strengths and areas for improvement.</p> <p>AWE has become increasingly popular in both theoretical and practical aspects of English writing, promising to revolutionize the way students learn and improve their writing skills. However, despite its widespread use, there is still a lack of consensus on whether AWE can effectively enhance second language (L2) writing proficiency, especially in specialized writing contexts (Sun & Fan, [<reflink idref="bib72" id="ref7">72</reflink>]). The impact of AWE on L2 writing skills is complex and multifaceted. While some studies suggest that AWE can provide valuable feedback and improve writing proficiency, others argue that it may not be as effective as human evaluation, especially in capturing the nuances of L2 writing. Furthermore, the role of psychological factors such as anxiety and motivation in determining the effectiveness of AWE is yet to be fully understood (Taskiran et al., [<reflink idref="bib75" id="ref8">75</reflink>]).</p> <p>Given the conflicting results and limited literature on the topic, there is an urgent need for further research into the influence of AWE on L2 writing skills, anxiety, and motivation. Such research could help us understand the optimal conditions for AWE use and identify the factors that may hinder or enhance its effectiveness (Wang & Troia, [<reflink idref="bib78" id="ref9">78</reflink>]). By doing so, we can move closer to answering the question of whether AWE can truly improve L2 writing proficiency and writing anxiety. Therefore, although AWE has gained popularity in English writing instruction, its impact on L2 writing skills remains enigmatic (Li, [<reflink idref="bib44" id="ref10">44</reflink>]). Future research is crucial to fill the gaps in our understanding and optimize the use of AWE in writing instruction. Only then can we truly evaluate the effect of AWE in enhancing students' L2 writing proficiency and reducing writing anxiety (Khoii & Doroudian, [<reflink idref="bib43" id="ref11">43</reflink>]; Sun & Fan, [<reflink idref="bib72" id="ref12">72</reflink>]).</p> <p>Scanty meta-analyses have been conducted to explore the impact of AWE on motivation, anxiety, and second language (L2) writing skills. While some studies have shown promise, there is still a need for more comprehensive research. A recent multi-level meta-analysis found that AWE technology can enhance vocabulary usage, produce significant effects on writing outcomes, and particularly benefit undergraduate students (Ngo et al., [<reflink idref="bib55" id="ref13">55</reflink>]). Additionally, another meta-analysis examined the measurements of the AWE effect on L2 writing skills and moderator variables (Mohsen, [<reflink idref="bib52" id="ref14">52</reflink>]). Despite these insights, there remains a scarcity of meta-analytic research that delves into the influence of AWE on anxiety, motivation, and L2 writing proficiency. Future research should aim to fill this gap and provide a more nuanced understanding of the role of AWE in writing instruction (Wei et al., [<reflink idref="bib81" id="ref15">81</reflink>]). By doing so, we can gain a deeper understanding of how AWE can best be utilized to support learners and improve their writing abilities.</p> <p>The meta-analytic study of the effects of AWE on motivation, anxiety, and second language (L2) writing skills is not only necessary but also highly meaningful. Firstly, AWE has become a popular tool in language learning contexts, providing learners with prompt feedback on their writing (Li et al., [<reflink idref="bib45" id="ref16">45</reflink>]). Despite its widespread use, however, there has been a lack of systematic research examining its impact on learners' motivation, anxiety, and L2 writing proficiency (Yu, [<reflink idref="bib88" id="ref17">88</reflink>], [<reflink idref="bib89" id="ref18">89</reflink>]). Meta-analysis offers a powerful means to aggregate and synthesize findings from multiple studies, providing a more comprehensive and nuanced understanding of the effects of AWE on these outcomes (Mohsen, [<reflink idref="bib52" id="ref19">52</reflink>]). By meta-analytically studying the effects of AWE, researchers can identify consistent patterns and trends in the data, revealing which aspects of writing are most influenced by this technology (Liu et al., [<reflink idref="bib50" id="ref20">50</reflink>]). Such information can greatly inform teaching practices, leading to more effective and targeted interventions that enhance learners' writing experiences and outcomes (Yu, [<reflink idref="bib88" id="ref21">88</reflink>], [<reflink idref="bib89" id="ref22">89</reflink>]).</p> <p>Moreover, understanding the impact of AWE on learners' anxiety is particularly important. Anxiety can significantly affect language learners' motivation and performance, and by understanding how AWE may either alleviate or exacerbate anxiety, teachers can better identify and address possible challenges that may arise during the learning process (Solhi et al., [<reflink idref="bib71" id="ref23">71</reflink>]; Zhai & Ma, [<reflink idref="bib91" id="ref24">91</reflink>]). Consequently, meta-analysis of the effects of AWE on motivation, anxiety, and L2 writing skills can contribute significantly to improved language teaching and learning worldwide. By providing a more comprehensive understanding of the effects of AWE and exploring moderators that may influence these effects, meta-analysis can help us to better hold the influence of this technology and optimize its use in language classrooms (Table 1).</p> <p>Table 1 Comparative objectives between previous and current studies</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Studies</p></th><th align="left"><p>Highlights</p></th></tr></thead><tbody><tr><td align="left"><p>Ngo et al. (<xref ref-type="bibr" rid="bibr55">2022</xref>)</p></td><td align="left"><p>English writing performance, usage of vocabulary, and benefits of AWE</p></td></tr><tr><td align="left"><p>Mohsen (<xref ref-type="bibr" rid="bibr52">2022</xref>)</p></td><td align="left"><p>AWE effects on L2 writing skills and moderating variables</p></td></tr><tr><td align="left"><p>This study</p></td><td align="left"><p>Effects of AWE on anxiety, motivation, and L2 writing skills</p></td></tr></tbody></table> </ephtml> </p> <p>Table 1 showcases the differences between literature and this study, revealing the significance of this study.</p> <p>Anxiety and motivation play a pivotal role in determining the success of second language (L2) writing outcomes. According to Ibishi et al. ([<reflink idref="bib39" id="ref25">39</reflink>]), these two affective factors significantly impact learners' writing performance and progress. Anxiety, in particular, can hinder learners' ability to express themselves effectively in L2, leading to decreased fluency and accuracy (Sun & Fan, [<reflink idref="bib72" id="ref26">72</reflink>]). High levels of anxiety can be debilitating, causing writers to procrastinate or even avoid writing altogether. This avoidance behavior can have a profound impact on their progress and overall writing proficiency. On the other hand, motivation acts as a driving force for learners to engage actively in writing activities and strive for improvement. Motivated learners are more likely to persevere through challenges, seek feedback, and apply new knowledge and skills to their writing (Woodworth & Barkaoui, [<reflink idref="bib85" id="ref27">85</reflink>]). They are also more inclined to take risks and experiment with new language structures, which is crucial for effective language development.</p> <p>A meta-regression analysis further highlights the importance of anxiety, motivation, and L2 writing skills in moderating the effect sizes regarding the effect of AWE on writing outcomes (z = 2.34, d = 0.02, 95%CI 0.095 ~1.087). This analysis reveals that when learners are anxious and lack motivation, the effectiveness of AWE may be compromised. Conversely, when learners are motivated and have strong writing skills, they are more likely to benefit from AWE and make significant progress in their writing. Therefore, it is essential for educators and practitioners to consider anxiety and motivation as integral components of L2 writing outcomes. By fostering a positive learning environment that addresses learners' anxieties and stimulates their motivation, we can promote effective language development and help learners achieve their goals in L2 writing.</p> <hd id="AN0178463698-3">Literature Review</hd> <p>Some key terms are defined to facilitate comprehension of this study. AWE refers to the use of computer-based tools that automatically assess and provide feedback on written compositions (Nazar & Renau, [<reflink idref="bib54" id="ref28">54</reflink>]). These tools employ natural language processing techniques to analyze writing samples and provide scores, comments, and suggestions to writers. Anxiety, in the context of writing, refers to the feelings of apprehension, unease, and nervousness that individuals may experience when engaging in writing tasks (Abdullah et al., [<reflink idref="bib3" id="ref29">3</reflink>]). This anxiety can be caused by various factors such as fear of evaluation, lack of confidence, and the pressure to perform well (Derakhshan et al., [<reflink idref="bib24" id="ref30">24</reflink>]). Motivation is the driving force that prompts individuals to engage in specific behaviors or activities. In the field of writing, motivation refers to the desire and energy that writers have to produce written work, whether it be for academic, professional, or personal reasons (Akbarzadeh et al., [<reflink idref="bib4" id="ref31">4</reflink>]; Derakhshan et al., [<reflink idref="bib21" id="ref32">21</reflink>], [<reflink idref="bib22" id="ref33">22</reflink>], [<reflink idref="bib23" id="ref34">23</reflink>]). Second language writing skills refer to the abilities and proficiency that individuals develop when learning to write in a language that is not their native language (Al-nakhle, [<reflink idref="bib7" id="ref35">7</reflink>]). This involves the use of grammar, vocabulary, sentence structure, and other linguistic elements to express ideas and communicate effectively in the target language.</p> <p>In the literature review, the primary focus on outcomes not directly related to writing skills stems from the broader scope of research interests and methodological limitations. It is important to note that L2 writing skills are often considered within the broader context of second language acquisition research, which explores many aspects such as learners' cognitive processes, affective factors, and social and cultural factors that influence language learning (Yu, [<reflink idref="bib88" id="ref36">88</reflink>], [<reflink idref="bib89" id="ref37">89</reflink>]). Therefore, the literature review prioritizes broader outcomes that provide a more comprehensive understanding of the complex nature of L2 writing development. Additionally, the meta-analysis aims to synthesize findings across multiple studies to identify patterns and trends across a range of outcomes not limited to writing skills alone. By exploring a broader range of outcomes, the review can contribute to a more comprehensive understanding of the overall L2 writing development process and identify areas that require further investigation (Mohsen, [<reflink idref="bib52" id="ref38">52</reflink>]). It is expected that the review first addresses the controversies surrounding L2 writing skills as a major outcome and then explores other outcomes directly or indirectly related to L2 writing skills to provide a more comprehensive overview of the field.</p> <p>Most studies reported positive effects of AWE on writing performance across the world. AWE could be used to evaluate writing and improve scoring efficiency, facilitating teachers' work. AWE could provide timely feedback for individuals and improve Turkish university students' L2 writing skills (Han & Sari, [<reflink idref="bib32" id="ref39">32</reflink>]). The AWE technology, i.e., Icouse and Icourse + Pigai could improve Chinese university students' English writing proficiency by enhancing their self-regulation (Han et al., [<reflink idref="bib33" id="ref40">33</reflink>]). Students tended to hold positive attitudes toward AWE although they might not be fully satisfied with the use of AWE. AWE could improve writing quality in terms of language organization although no significant effect of automated evaluation and the anxiety caused on writing performance was revealed (Sun & Fan, [<reflink idref="bib72" id="ref41">72</reflink>]). However, the effect of AWE might be prone to various factors such as educational contexts, learning goals, and writing limitations (Khoii & Doroudian, [<reflink idref="bib43" id="ref42">43</reflink>]). It is thus necessary to study the effects of AWE to a larger extent. The factors related to educational contexts included online, peer assessment, automated feedback, teacher feedback (Derakhshan et al., [<reflink idref="bib24" id="ref43">24</reflink>]), grammar feedback, and self-regulated learning. Learning goals included fluent writing, automated essay evaluation, and computer-mediated collaborative writing. Writing limitations included different levels of difficulty or complexity of writing tasks, the format and structure of written products, language barriers, and access to resources.</p> <hd id="AN0178463698-4">Anxiety</hd> <p>AWE has garnered increasing attention from scholars in recent years, yet the impact of this technology on writer anxiety remains an underexplored area of research. A recent randomized controlled trial conducted by Waer ([<reflink idref="bib76" id="ref44">76</reflink>]) among English majors at an Egyptian university shed light on this important issue. The study revealed that the use of AWE significantly reduced anxiety levels among Egyptian university students when it came to writing. This finding was particularly noteworthy as it suggested that AWE could be an effective tool in helping anxious students overcome their writing fears and improve their practice. Furthermore, the study found that AWE slightly enhanced the grammatical knowledge of English as a second language among those students who were not initially apprehensive about their writing abilities. These findings have important implications for educational practice, indicating that AWE could play a key role in fostering a positive writing environment that supports students' development and reduces anxiety. In conclusion, the potential benefits of AWE in reducing writer anxiety and enhancing writing skills among students are significant and worth further exploration.</p> <p>AWE tools, with their increasing prevalence in educational settings, can influence writers' anxiety levels. Technological advancements have brought about the widespread adoption of these tools, designed to provide timely feedback on student writing. Despite their convenience, concerns about the mental well-being of students using AWE have emerged (Abas et al., [<reflink idref="bib1" id="ref45">1</reflink>]). One significant concern is the potential for AWE to create undue pressure on writers, who may feel compelled to meet arbitrary standards set by the software. This pressure, in turn, may lead to increased anxiety and self-doubt, potentially hindering their writing development (Cao & Yu, [<reflink idref="bib14" id="ref46">14</reflink>]). It is imperative to carefully examine the effect of AWE on writers' anxiety to ensure that the technology does not cause undue stress. Understanding this relationship can inform the development of best practices in the use of AWE tools, fostering a positive learning environment that encourages creativity and critical thinking without adding to students' anxiety. To this end, future research should delve deeper into the intricacies of AWE's impact on writing anxiety, with the aim of informing humane and effective writing pedagogy and improving the power of technology to enhance, rather than hinder student learning experiences.</p> <hd id="AN0178463698-5">Motivation</hd> <p>AWE has attracted increasing attention from scholars in recent years due to its capacity to positively impact writing motivation. Numerous studies have reported that AWE programs can enhance students' writing motivation and mindsets by enriching their writing experiences and perceptions of writing activities. For instance, Yao et al. ([<reflink idref="bib87" id="ref47">87</reflink>]) found that AWE could complement writing activities and create a positive learning environment among Chinese university English majors. This positive academic environment, in turn, motivated students to engage more actively in writing tasks. Similarly, Kaya et al. ([<reflink idref="bib41" id="ref48">41</reflink>]) reported that feedback from the Fluent Writing Evaluation Rubric significantly improved primary school students' writing fluency in Turkey. The timely and specific feedback provided by AWE tools informed students about their weaknesses and strengths in writing, thus encouraging them to write with enhanced motivation. In conclusion, AWE not only reduces writer anxiety but also acts as a motivational tool that can foster a positive writing environment and improve students' writing practices.</p> <hd id="AN0178463698-6">L2 Writing Skills</hd> <p>Writing skills are fundamental to effective communication across a range of contexts, from academic papers to professional reports (Cheng, [<reflink idref="bib17" id="ref49">17</reflink>]). These skills involve the art of expressing thoughts and ideas clearly, utilizing precise grammar, punctuation, and vocabulary (Mohsen, [<reflink idref="bib52" id="ref50">52</reflink>]). Skilled writers can craft coherent sentences and paragraphs, ensuring a smooth flow and natural cadence, making their work both readable and impactful. Furthermore, they have a keen understanding of their audience, allowing them to choose appropriate language and format to convey information effectively. The development of these skills is ongoing, requiring regular practice, feedback, and a solid grounding in grammar, sentence structure, and critical thinking (Prichard, [<reflink idref="bib60" id="ref51">60</reflink>]). By continuously honing their writing abilities, individuals can communicate more effectively, whether in academic, professional, or even personal settings.</p> <p>Some studies argued that AWE could improve L2 writing skills through feedback. For instance, the feedback of AWE could improve academic L2 writing skills and provide effective suggestions for the improvement of writing quality (Liao Calvin et al., [<reflink idref="bib47" id="ref52">47</reflink>]). The AWE technology, i.e., Summary Street®, could guide students to write properly through timely feedback and improve middle school students' L2 writing skills using Latent Semantic Analysis (Franzke et al., [<reflink idref="bib27" id="ref53">27</reflink>]). The timely feedback and consistent scoring of AWE could facilitate Arabic language writing skills of children students in Saudi Arabia due to its higher rates of correct evaluation than human rating (Azmi et al., [<reflink idref="bib8" id="ref54">8</reflink>]). Timely feedback could provide constructive suggestions for writers and improve their L2 writing skills by reminding them of their writing errors or problems. Timely feedback could also make students aware of the importance of L2 writing skills and the progress of their writing tasks. Students might subconsciously improve their L2 writing skills with the assistance of AWE.</p> <p>AWE can enhance second language (L2) writing skills across various dimensions, including writing accuracy, autonomy, semantics, contexts, consistency, and collaboration. Wang et al. ([<reflink idref="bib79" id="ref55">79</reflink>]) emphasized that AWE could significantly improve writing accuracy and autonomy, thus enhancing L2 writing skills among university students in China. Furthermore, AWE has been shown to improve the assessment accuracy of writing semantics and contexts through the use of embedded neural words (Derakhshan et al., [<reflink idref="bib21" id="ref56">21</reflink>], [<reflink idref="bib22" id="ref57">22</reflink>], [<reflink idref="bib23" id="ref58">23</reflink>]; Tashu & Horváth, [<reflink idref="bib74" id="ref59">74</reflink>]). The AWE program, Criterion®, has been found to significantly improve writing scores, maintain essay scoring consistency, and reduce teachers' workloads (Hassanzadeh & Fotoohnejad, [<reflink idref="bib34" id="ref60">34</reflink>]). Additionally, computer-assisted collaborative writing and online assessment have been shown to improve L2 writing skills and enhance collaborative dynamics among university students in China (Zhang & Chen, [<reflink idref="bib92" id="ref61">92</reflink>]). Moreover, AWE has been found to improve English writing abilities compared to teacher evaluation, and students hold positive attitudes towards AWE (Wang, [<reflink idref="bib80" id="ref62">80</reflink>]). When combined with appropriate teaching strategies, AWE can effectively improve L2 writing skills among primary and middle school students (Nunes et al., [<reflink idref="bib56" id="ref63">56</reflink>]). Therefore, the integration of AWE into L2 writing instruction holds promise for enhancing students' writing abilities across multiple dimensions.</p> <hd id="AN0178463698-7">Conflicting Results</hd> <p>The analysis of AWE in terms of its effects on anxiety and motivation in language learning reveals a complex and intriguing picture. While some studies, such as Chen and Hwang's ([<reflink idref="bib16" id="ref64">16</reflink>]), indicate that AWE can be advantageous in reducing anxiety and enhancing motivation among learners, others like Link et al. ([<reflink idref="bib49" id="ref65">49</reflink>]) suggest that the quantity and quality of teacher feedback play a crucial role in these effects. Some findings even suggest opposite or non-significant impacts. This variance in outcomes can be attributed to multiple factors, including individual differences in learning styles, preferences, and language backgrounds (Parra & Calero, [<reflink idref="bib58" id="ref66">58</reflink>]; Alberto Benitez, [<reflink idref="bib5" id="ref67">5</reflink>]). Additionally, research by Wilson and Czik ([<reflink idref="bib83" id="ref68">83</reflink>]) has shown positive impacts of AWE on motivation and anxiety, while other studies, such as those by Alberto Benitez ([<reflink idref="bib5" id="ref69">5</reflink>]), have not found similar benefits. This suggests that the relationship between AWE and anxiety/motivation is not straightforward and could be influenced by multiple variables. These variables include the specific implementation of AWE, the nature of the writing tasks, and the learners' individual characteristics.</p> <p>Therefore, the conflicting results underscore the need for further research to explore the intricate relationships between AWE, anxiety, and motivation. Future studies should consider a wide range of factors, including individual learner characteristics, teaching styles, and the specific design and implementation of AWE systems. This expanded research will provide a deeper understanding of how AWE affects language learning outcomes and will help us optimize its use for maximum benefit. Additionally, it is crucial to consider the role of human interaction and feedback in tandem with AWE to ensure a balanced and comprehensive approach to language learning. By doing so, we can improve the capacity of AWE to enhance learning experiences and outcomes for language learners.</p> <p>The analysis of AWE has revealed conflicting results regarding its impact on writing skills. Some studies, such as Wilson and Czik ([<reflink idref="bib83" id="ref70">83</reflink>]), suggest that AWE can improve writing skills, with teachers reporting that it helps them provide timely feedback, leading to improved writing retention. However, other studies, including Huang and Renandya ([<reflink idref="bib37" id="ref71">37</reflink>]) and Gollnitz ([<reflink idref="bib30" id="ref72">30</reflink>]), argue that AWE technologies may not significantly improve L2 writing skills and quality due to high costs and despite their benefits. Furthermore, Gee ([<reflink idref="bib28" id="ref73">28</reflink>]) and Beach ([<reflink idref="bib9" id="ref74">9</reflink>]) suggest that AWE may not enhance students' writing motivation or quality. While university students in China have positive attitudes towards AWE technologies like Pigai, this does not necessarily translate into improved English writing skills (Huang & Renandya, [<reflink idref="bib37" id="ref75">37</reflink>]). Even in the United States, where the PEG Writing® AWE technology is widely used, studies have found that it does not significantly improve writing quality through timely feedback (Wilson & Czik, [<reflink idref="bib83" id="ref76">83</reflink>]), despite some reports of skill improvement (Wang, [<reflink idref="bib80" id="ref77">80</reflink>]). These conflicting results demonstrate it is necessary to explore the intricate relationships between AWE, anxiety, and motivation, considering factors such as individual learner characteristics, teaching styles, and the specific design and implementation of AWE systems.</p> <p>The analysis of AWE has generated mixed findings on its impact on writing skills. While some studies (e.g., Jansen et al., [<reflink idref="bib40" id="ref78">40</reflink>]) report positive effects of AWE on student writing skills and teachers' ability to provide timely feedback, other studies question these benefits. Guo and Wang ([<reflink idref="bib31" id="ref79">31</reflink>]), for instance, argues that despite their popularity, AWE technologies may not significantly improve L2 writing skills and quality due to high costs and other factors. A literature review reveals varying results from studies conducted in different regions. In Turkey, for example, research has shown differences in writing performances between AWE-assisted and non-AWE-assisted groups. These studies suggest the positive results of writing skills in Turkey (Sahin & Yagiz, [<reflink idref="bib63" id="ref80">63</reflink>]).</p> <p>Contrastingly, studies conducted in Iran indicate no significant differences in writing performances between AWE-assisted and non-AWE-assisted groups. These findings suggest that, in this particular context, the use of AWE may not lead to significant improvements in student writing skills among lower-level Iranian EFL learners (Biglari et al., [<reflink idref="bib12" id="ref81">12</reflink>]). Similarly, research in Egypt also reports no significant differences in writing performances between AWE-assisted and non-AWE-assisted groups (Waer, [<reflink idref="bib77" id="ref82">77</reflink>]). This suggests that, despite the widespread use of AWE technologies, they may not necessarily translate into measurable improvements in student writing in this region. These conflicting results underscore the need for further research to explore the intricate relationships between AWE, anxiety, and motivation, considering factors such as individual learner characteristics, teaching styles, and the specific design and implementation of AWE systems.</p> <p>In summary, the literature review presents a comprehensive analysis of existing research, revealing mixed results regarding the impact of AWE tools. While some studies indicate positive effects on anxiety reduction and motivation enhancement, others suggest limited benefits or even negative impacts. The literature also highlights the importance of considering contextual factors, such as learner characteristics and teaching methods, when evaluating the effectiveness of AWE. The findings of literature underscore the need for further research to address the remaining gaps and uncertainties, including the effects of AWE on anxiety, motivation, and second language writing skills and its potential to support different writing processes.</p> <p>The research design of our meta-analysis aimed to comprehensively evaluate the impact of AWE on anxiety, motivation, and second language writing skills. By systematically reviewing and analyzing existing studies, we aimed to identify patterns and trends in the relationship between AWE and these key affective and cognitive factors. Our approach was innovative in several ways. Firstly, we focused on a previously under-explored area of research, examining the role of AWE in addressing learners' anxieties and stimulating their motivation. Secondly, our meta-analysis employed rigorous statistical methods to pool and compare findings across studies, providing a more accurate and comprehensive understanding of the effects of AWE. Finally, our study contributed to the field by identifying gaps in the literature and providing recommendations for future research.</p> <hd id="AN0178463698-8">Research Questions and Hypotheses</hd> <p>Drawing from previous research and its sometimes conflicting results, the present study aims to provide a comprehensive overview and synthesis of previous findings. The current study hopes to provide clarity and consensus on the role of AWE in education, ultimately aiding educators in making informed decisions about its use in their classrooms.</p> <p>Based on the literature reviewed, we proposed the following research questions:</p> <p></p> <ulist> <item> Do learners assisted with AWE exhibit significantly higher L2 writing skills compared to those without AWE?</item> <p></p> <item> Does AWE assistance fail to significantly reduce anxiety in learners compared to those without AWE?</item> <p></p> <item> Do learners assisted with AWE show significantly higher motivation in L2 writing than those without AWE?</item> <p></p> <item> Are there significant differences in writing performances between AWE and non-AWE-assisted learners in China?</item> <p></p> <item> Are Turkish learners' writing performances comparable between AWE-assisted and non-AWE-assisted groups?</item> <p></p> <item> Is there any significant difference in writing performances between AWE and non-AWE-assisted learners in Iran?</item> <p></p> <item> Do Egyptian learners show similar writing performances between AWE-assisted and non-AWE-assisted groups?</item> <p></p> <item> Are there significant variations in writing performances between AWE and non-AWE-assisted learners in the USA?</item> </ulist> <p>These questions aim to explore the reasons behind the hypotheses, investigate the effectiveness of AWE in different regions, and understand factors influencing L2 writing outcomes. Consequently, the ultimate goal is to test the following null hypotheses:</p> <hd id="AN0178463698-9">H1</hd> <p>Learners assisted with AWE could not have significantly higher L2 writing skills than those without AWE.</p> <hd id="AN0178463698-10">H2</hd> <p>Learners assisted with AWE could not have significantly lower anxiety than those without AWE.</p> <hd id="AN0178463698-11">H3</hd> <p>Learners assisted with AWE could not have significantly higher motivation than those without AWE.</p> <hd id="AN0178463698-12">H4</hd> <p>There are no significantly different writing performances between AWE and non-AWE-assisted groups in China.</p> <hd id="AN0178463698-13">H5</hd> <p>There are no significantly different writing performances between AWE and non-AWE-assisted groups in Turkey.</p> <hd id="AN0178463698-14">H6</hd> <p>There are no significantly different writing performances between AWE and non-AWE-assisted groups in Iran.</p> <hd id="AN0178463698-15">H7</hd> <p>There are no significantly different writing performances between AWE and non-AWE-assisted groups in Egypt.</p> <hd id="AN0178463698-16">H8</hd> <p>There are no significantly different writing performances between AWE and non-AWE-assisted groups in the USA.</p> <hd id="AN0178463698-17">Methods</hd> <p></p> <hd id="AN0178463698-18">Literature Search</hd> <p>To ensure a thorough search of scientific documents, researchers conducted independent literature searches in multiple online databases, adhering to their unique search guidelines. They achieved commendable inter-rater reliability (k = 0.91). On June 12, 2023, they began by querying Web of Science with keywords related to automated writing and essay assessments, combined with control, experimental, and treatment group terms. This search yielded 10 results. Subsequently, they searched EBSCO, using the Abstract search field with similar keywords, retrieving 23 results. Then, they searched ScienceDirect, entering terms for control, experimental, and treatment groups in the "Find articles with these terms" field and terms for automated writing and essay assessments in the "Title, abstract or author-specified keywords" field, resulting in 122 articles. Finally, they searched Engineering Village, entering similar keywords in the Subject/Title/Abstract field, retrieving 30 articles. Through these sequential searches, the researchers were able to compile a comprehensive collection of relevant scientific documents.</p> <p>On June 12, 2023, the researchers expanded their search for relevant articles on the Taylor & Francis Online platform. They carefully crafted their search query, incorporating keywords such as "control group," "experimental group," "treatment group," "automated writing," "assessment," "essay," and "automatic writing evaluation." This meticulous approach was designed to ensure that they captured all relevant articles that could contribute to their research. The search on Taylor & Francis Online yielded 41 articles that were closely aligned with their research objectives. Encouraged by their initial findings, the researchers shifted their focus to the Sage database. They applied similar criteria to filter through the vast collection of articles available. The employment of this strategy yielded a remarkable outcome, leading to the acquisition of a substantial corpus of 774 articles. The researchers were thrilled with the results, as they now had a rich source of information to draw from and analyze for their ongoing investigation. These findings marked a significant milestone in their research, setting them on a promising path forward.</p> <hd id="AN0178463698-19">General Eligibility Criteria for Studies to be Included</hd> <p>The general eligibility criteria for studies to be included in our analysis were stringent and comprehensive, ensuring the quality and relevance of the articles considered. By applying these criteria, we were able to assemble a high-quality dataset of articles that would contribute to the reliability and validity of our research findings.</p> <p></p> <ulist> <item> Topic Relevance: Studies must focus on the use of AWE systems and their effects on anxiety, motivation, and/or second language writing skills.</item> <p></p> <item> Research Design: Quantitative studies, including randomized controlled trials, quasi-experimental studies, and correlational studies, are eligible. Qualitative studies and case studies may be included if they provide valuable insights into the topic.</item> <p></p> <item> Population: Studies must involve learners of a second language who use AWE systems for writing tasks. The participants' age, gender, proficiency level, and educational context should be clearly described.</item> <p></p> <item> Intervention: Studies must describe the use of an AWE system as part of the writing instruction or learning process. The specific AWE system used, its features, and how it was integrated into the learning environment should be detailed.</item> <p></p> <item> Outcomes: Studies must report on at least one of the following outcomes: changes in anxiety levels, motivation for writing, or improvements in second language writing skills. Quantitative measures of these outcomes should be provided.</item> <p></p> <item> Methodological Quality: Studies must demonstrate good methodological rigor, including clear research questions, appropriate sample sizes, reliable and valid measures, and appropriate statistical analysis.</item> <p></p> <item> Publication Status: Both published and unpublished studies may be included, provided they meet the other eligibility criteria and are accessible for review and data extraction.</item> <p></p> <item> Ethical Considerations: Studies must adhere to ethical standards, including informed consent, confidentiality, and protection of participant rights.</item> </ulist> <p>By applying these general eligibility criteria, the meta-analysis will ensure that only studies that are relevant, methodologically sound, and ethically responsible are included in the analysis. This will enhance the validity and reliability of the findings.</p> <hd id="AN0178463698-20">Inclusion and Exclusion of Literature</hd> <p>Researchers established both inclusion and exclusion criteria to filter the obtained results. The studies will be included if they (<reflink idref="bib1" id="ref83">1</reflink>) fall within the scope of the effects of AWE on anxiety, motivation, and L2 writing skills, (<reflink idref="bib2" id="ref84">2</reflink>) can provide enough data for meta-analyses such as means, standard deviations, and numbers of both control and experimental groups, (<reflink idref="bib3" id="ref85">3</reflink>) are rigidly designed and highly rated according to Standards for Reporting on Empirical Social Science Research in AERA Publications established by American Educational Research Association (https://<ulink href="http://www.aera.net/Portals/38/docs/12ERv35n6%5fStandard4Report%20.pdf">www.aera.net/Portals/38/docs/12ERv35n6%5fStandard4Report%20.pdf</ulink> retrieved on June 12, 2023), and (<reflink idref="bib4" id="ref86">4</reflink>) they are published in conferences and journals, as books, book chapters, technical reports, or white papers.</p> <p>They will be excluded if they (<reflink idref="bib1" id="ref87">1</reflink>) are duplicate studies, (<reflink idref="bib2" id="ref88">2</reflink>) are papers in the form of abstracts, tutorials, posters, or presentations, (<reflink idref="bib3" id="ref89">3</reflink>) cannot provide abstracts, (<reflink idref="bib4" id="ref90">4</reflink>) fall out of the scope of the effects of AWE on anxiety, motivation, and L2 writing skills, (<reflink idref="bib5" id="ref91">5</reflink>) belong to withdrawn publications, corrections, editorials, notes, short surveys, reference work entries, news, or datasets, (<reflink idref="bib6" id="ref92">6</reflink>) adopt too small sample sizes, (<reflink idref="bib7" id="ref93">7</reflink>) are poorly designed, (<reflink idref="bib8" id="ref94">8</reflink>) produce unconvincing results, or (<reflink idref="bib9" id="ref95">9</reflink>) cannot provide enough data for meta-analyses. Two raters included and excluded the literature according to both criteria. The inter-rater reliability reached a satisfactory level (k = 0.93). If both raters cannot reach an agreement on any selection, a third rater will make a decision. After the inclusion and exclusion processes, 22 results were included in the meta-analysis (Fig. 1).</p> <p>Graph: Fig. 1 A PRISMA flow diagram of systematic selection of literature</p> <hd id="AN0178463698-21">Data Coding</hd> <p>The variable <emph>L2 writing skills</emph> includes <emph>overall writing performance, writing quality scores, writing scores, writing proficiency, academic writing performance, writing performance, writing grades, fluent L2 writing skills, English essay writing scores, syntactic complexity, creative writing abilities, grammatical knowledge, scores of essays, writing quality,</emph> and <emph>comment numbers in written forms</emph>. The variable <emph>anxiety</emph> in writing assisted with AWE includes <emph>writing anxiety, computer anxiety, English writing anxiety, and errors</emph> made in writing. Researchers include <emph>errors</emph> made in writing as <emph>anxiety</emph> because learners with anxiety are prone to errors. The variable <emph>motivation</emph> includes <emph>writing motivation, self-regulation strategies,</emph> and <emph>writing self-efficacy</emph> since learners with strong self-regulation and self-efficacy tend to have strong motivation in writing activities (Yu, [<reflink idref="bib90" id="ref96">90</reflink>]).</p> <p>A meta-analysis was adopted to test the proposed null hypotheses. The meta-analysis aimed to pool the effect sizes and arrived at a more comprehensive conclusion regarding the effects of AWE on writing performances. Generally, researchers coded the included literature in view of primary outcomes and moderators (Benavides-Varela et al., [<reflink idref="bib11" id="ref97">11</reflink>]). Specifically, researchers coded the obtained data such as numbers of participants, sampling, means, standard deviations, subgroups, countries, journals, and treatments. The subgroups included comparative studies on motivation, anxiety, and L2 writing skills between the AWE-assisted group and the control group without the assistance of AWE (Table 2). The meta-analysis was conducted based on the protocol of the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) (Page et al., [<reflink idref="bib57" id="ref98">57</reflink>]) and the framework of Meta-Analytic Reporting Standards (MARS) (Kepes et al., [<reflink idref="bib42" id="ref99">42</reflink>]).</p> <p>Table 2 The details of included studies</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Author</p></th><th align="left"><p>Year</p></th><th align="left"><p>Subgroup</p></th><th align="left"><p>Country</p></th><th align="left"><p>Source</p></th><th align="left"><p>Sample</p></th></tr></thead><tbody><tr><td align="left"><p>Cheng et al.</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr18">2016</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>China</p></td><td align="left"><p>2016 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE)</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Franzke et al.</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr27">2005</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>USA</p></td><td align="left"><p>Journal of Educational Computing Research</p></td><td align="left"><p>Middle School students</p></td></tr><tr><td align="left"><p>Gollnitz</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr30">2010</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>USA</p></td><td align="left"><p>Oakland University dissertation</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Han & Sari</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr32">2022</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>Turkey</p></td><td align="left"><p>Computer Assisted Language Learning</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Han et al.</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr33">2021</xref></p></td><td align="left"><p>Motivation/</p><p>L2 writing skills</p></td><td align="left"><p>China</p></td><td align="left"><p>Frontiers in psychology</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Hassanzadeh & Fotoohnejad</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr34">2021</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>Iran</p></td><td align="left"><p>Journal of computer-assisted learning</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Hoffman</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr36">2019</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>USA</p></td><td align="left"><p>Journal of Applied Research in Higher Education</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Huang & Renandya</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr37">2020</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>China</p></td><td align="left"><p>Innovation in Language Learning and Teaching</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Kaya et al.</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr41">2020</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>Turkey</p></td><td align="left"><p>Education and Science</p></td><td align="left"><p>Elementary School Students</p></td></tr><tr><td align="left"><p>Khoii & Doroudian</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr43">2014</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>Iran</p></td><td align="left"><p>6<sup>Tth</sup> International Conference on Education and New Learning Technologies</p></td><td align="left"><p>Iranian advanced EFL learners</p></td></tr><tr><td align="left"><p>Li et al.</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr46">2020</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>China</p></td><td align="left"><p>Research-publishing</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Liao Calvin et al.</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr47">2019</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>China</p></td><td align="left"><p>8th International Congress on Advanced Applied Informatics, IIAI-AAI 2019</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Reynold et al.</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr61">2021</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>China</p></td><td align="left"><p>Asia–Pacific Education Researcher</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Sun & Fan</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr72">2022</xref></p></td><td align="left"><p>Anxiety/L2 writing skills</p></td><td align="left"><p>China</p></td><td align="left"><p>Studies in Educational Evaluation</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Turkben</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr73">2019</xref></p></td><td align="left"><p>Anxiety/motivation/L2 writing skills</p></td><td align="left"><p>Turkey</p></td><td align="left"><p>Eurasian Journal of Educational Research</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Waer</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr76">2021</xref></p></td><td align="left"><p>Anxiety/L2 writing skills</p></td><td align="left"><p>Egypt</p></td><td align="left"><p>Innovation in Language Learning and Teaching</p></td><td align="left"><p>Egytian university students</p></td></tr><tr><td align="left"><p>Wang</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr80">2022</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>China</p></td><td align="left"><p>Library Hi Tech</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Wang et al.</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr79">2013</xref></p></td><td align="left"><p>Anxiety</p></td><td align="left"><p>China</p></td><td align="left"><p>Computer Assisted Language Learning</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Wilson & Czik</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr83">2016</xref></p></td><td align="left"><p>Motivation</p></td><td align="left"><p>USA</p></td><td align="left"><p>Computer Applications in Education</p></td><td align="left"><p>Mixed</p></td></tr><tr><td align="left"><p>Yao et al.</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr87">2021</xref></p></td><td align="left"><p>Motivation</p></td><td align="left"><p>China</p></td><td align="left"><p>Assessing Writing</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Zhang & Chen</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr92">2022</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>China</p></td><td align="left"><p>Journal of Second Language Writing</p></td><td align="left"><p>University students</p></td></tr><tr><td align="left"><p>Ebadi et al.</p></td><td align="left"><p><xref ref-type="bibr" rid="bibr25">2023</xref></p></td><td align="left"><p>L2 writing skills</p></td><td align="left"><p>Iran</p></td><td align="left"><p>Computers in the Schools</p></td><td align="left"><p>University students</p></td></tr></tbody></table> </ephtml> </p> <p>Although most of the included studies conducted randomized controlled trials, some of them adopted different designs. For instance, some of them conducted both pre and post-tests and implemented within-group, between-group, or multiple-group comparisons (Khoii & Doroudian, [<reflink idref="bib43" id="ref100">43</reflink>]). In this case, researchers will compare post-tests since they can better present the effects after a given treatment. For those with multiple-group comparisons, researchers will select the group comparison that can best present the effects after the treatment. For those with different data from multiple comparisons, researchers may also combine means, standard deviations, and numbers of participants using scientific formulas (Morris & DeShon, [<reflink idref="bib53" id="ref101">53</reflink>]).</p> <p>In case there are multiple experimental groups, researchers will select the one that can best reflect the effect of AWE on L2 writing skills. For instance, the study conducted by Han et al. ([<reflink idref="bib33" id="ref102">33</reflink>]) contained two experimental groups involving the use of two technologies, i.e., Icourse (a learning management system) and Icourse + Pigai (an automatic writing evaluation system). Researchers selected the experimental group intervened by the latter because the latter technology belongs to an AWE tool, while the former does not.</p> <p>When there are two groups, each of which undergoes both pre and post-tests, researchers will collect the data from both post-tests because they are the results after experiments. For instance, Khoii and Doroudian ([<reflink idref="bib43" id="ref103">43</reflink>]) divided the participants into both experimental and control groups, which underwent both pre and post-tests. Researchers collected the data from the post-tests because they are the results after the experiment scored by AWE program My Access! in writing practice.</p> <hd id="AN0178463698-22">Risk Assessment and Management</hd> <p>Heterogeneity, publication bias, and stability of the estimates were assessed to minimize the risks in the study. Testing heterogeneity aims to determine whether a random-effect or fixed-effect model is adopted to run the meta-analysis. Researchers consider I<sups>2</sups>, Heterogeneity statistics, and values of <emph>z</emph> and <emph>p</emph> in the choice of models. The heterogeneity will be considered <emph>unimportant</emph> if the value of I<sups>2</sups> ranges from 0 to 40%, <emph>moderate</emph> if from 30 to 60%, <emph>substantial</emph> if from 50 to 90%, and <emph>considerable</emph> if from 75 to 100% (Higgins & Green, [<reflink idref="bib35" id="ref104">35</reflink>]). If the value I<sups>2</sups> is larger than 50%, a random-effect model will be adopted, while if it is smaller than 50%, a fixed model will be adopted. A sensitivity analysis in Stata was adopted to test the stability or robustness of the collected estimates. Begg (Begg & Mazumdar, [<reflink idref="bib10" id="ref105">10</reflink>]) and Egger's (Egger et al., [<reflink idref="bib26" id="ref106">26</reflink>]) tests were used to test the publication bias.</p> <hd id="AN0178463698-23">Calculating Effect Sizes</hd> <p>In this meta-analysis, the effect sizes were calculated using Cohen's d, following the protocol established by Morris and DeShon ([<reflink idref="bib53" id="ref107">53</reflink>]). This calculation determined the differences in means between experimental and control groups, dividing it by the standard deviations across both groups. As per Sedgwick and Marston ([<reflink idref="bib65" id="ref108">65</reflink>]), this measure gave us an accurate representation of the effect size. The interpretation of these effect sizes would be considered <emph>very small</emph> if it was approximately 0.1, <emph>small</emph> if it was about 0.2, <emph>medium</emph> if it was around 0.5, <emph>large</emph> if it was approximately 0.8, <emph>very large</emph> if it was about 1.2, and <emph>huge</emph> if it was approximately 2 (Sawilowsky, [<reflink idref="bib64" id="ref109">64</reflink>]). In addition, the researchers presented a comprehensive forest plot that included confidence intervals, effect sizes, and weights of the included estimates. This visual representation aided in the synthesis and evaluation of the collective data, enabling a more comprehensive understanding of the subject matter.</p> <hd id="AN0178463698-24">Results</hd> <p></p> <hd id="AN0178463698-25">Results of Publication Bias Tests</hd> <p>Publication bias, often referred to as the tendency for certain types of studies to be more likely to be published than others, is a significant concern in research. This bias can lead to a distorted representation of the true findings, as studies with positive or significant results tend to be more favored for publication than those with negative or non-significant outcomes. In essence, publication bias can introduce a significant distortion in the overall body of knowledge, leading to misleading conclusions. Fortunately, in the present study, Fig. 2 provides a visual representation of the distribution of effect sizes, indicating that the effect sizes are nearly symmetrically distributed along both sides of the middle line. This distribution suggests that there is no evidence of publication bias present in the included studies. Furthermore, statistical tests conducted by both Egger (t = 0.88, p = 0.386) and Begg (z = 0.63, p = 0.532) support this conclusion, indicating the absence of publication bias.</p> <p>Graph: Fig. 2 Tests of publication bias</p> <p>These encouraging results indicate that the studies included in the present analysis are representative of the entire spectrum of published and unpublished works in the field of AWE-assisted writing. This finding is particularly crucial, as it ensures the reliability and validity of the research outcomes. The absence of publication bias ensures that the included studies provide an unbiased representation of the existing evidence, thereby avoiding distortions that could potentially lead to misleading conclusions. Researchers can, therefore, confidently base their findings on the included studies, assured that they accurately reflect the entire body of knowledge in the area of AWE-assisted writing. This finding paves the way for more reliable and valid research in this important field.</p> <hd id="AN0178463698-26">L2 Writing Skills, Anxiety, and Motivation (H1-3)</hd> <p>When it comes to meta-analysis, selecting the appropriate model is crucial for obtaining accurate and reliable results. In the current study, the researchers took a careful approach to determine whether to use a fixed-effect model or a random-effect model. To make this decision, they first examined the heterogeneity of the estimates obtained from the various studies included in the meta-analysis. Heterogeneity refers to the variation in the results across studies due to differences in factors such as participant characteristics, study design, or measurement methods. If the estimates are homogeneous, indicating that the studies are similar and the results are consistent, a fixed-effect model is typically used. However, if there is significant heterogeneity, indicating that the studies vary widely in their findings, a random-effect model is more appropriate.</p> <p>In this study, the researchers used chi-squared tests to assess heterogeneity. The p-values obtained for anxiety (p = 0.015), motivation (p < 0.01), L2 writing skills (p < 0.01), and the overall results (p < 0.01) were all lower than 0.05, indicating the presence of heterogeneity in the estimates. This means that the studies included in the meta-analysis differed significantly in their findings. To further quantify the extent of heterogeneity, the researchers also calculated the I<sups>2</sups> statistic. The I<sups>2</sups> values for anxiety (I<sups>2</sups> = 71.2%), motivation (I<sups>2</sups> = 92.3%), L2 writing skills (I<sups>2</sups> = 93.6%), and the overall results (I<sups>2</sups> = 93.8%) were all relatively high, indicating significant heterogeneity in the estimates.</p> <p>Based on these findings, the researchers chose to adopt a random-effect model for the meta-analysis. The random-effect model assumes that the studies included in the meta-analysis are drawn from a population of studies and accounts for the variation across studies. This model provides a more conservative and reliable estimate of the overall effect, taking into account the heterogeneity among the studies. The results of the meta-analysis using the random-effect model are presented in a forest plot (Fig. 3). This plot visually illustrates the distribution of the effect sizes across studies and their associated confidence intervals. It allows researchers to quickly identify the magnitude and direction of the effect as well as the degree of overlap among the studies.</p> <p>Graph: Fig. 3 A forest plot of L2 writing skills, anxiety, and motivation assisted with AWE with a tutorial nature</p> <p>Figure 3 shows that AWE can significantly improve L2 writing skills and motivation, and significantly reduce anxiety. The forest plot depicted above features a distinct midline that represents the "no-effect" threshold. A horizontal line intersecting this line signifies a confidence interval, while the dot placed within this interval indicates the specific effect size. At the base of each variable, a diamond shape is visible, indicating the pooled effect size. In order to determine the significance of the effect sizes between the experimental and control groups, one must consider where the effect size and diamond fall relative to the no-effect line. If they cross this line, it indicates that there are no significant differences in effect sizes between the two groups. However, if the effect size is located on the left side of the line, it means that the values in the experimental group are significantly lower than those in the control group. Conversely, if it's on the right side, it indicates that the values in the experimental group are significantly higher. In this way, the forest plot serves as a visual tool to easily interpret and compare the collective data, allowing for a comprehensive understanding of the subject matter.</p> <p>The forest plot depicted in Fig. 3 offers a comprehensive view of the meta-analysis findings. Evidently, the diamond representing anxiety in the AWE-assisted group is situated notably to the left of the no-effect line, indicating a substantial reduction compared to the control group. Specifically, the effect size is estimated at d = − 0.80, with a confidence interval ranging from − 1.26 to − 0.33. This significant difference is further supported by a z-score of 3.35 and a p-value of 0.001, confirming that anxiety levels are lower in the experimental group.</p> <p>In contrast, the diamonds for motivation and L2 writing skills are situated to the right of the no-effect line, indicating positive effects in the experimental group. For motivation, the effect size is d = 0.87, with a confidence interval of 0.30 to 1.44. Similarly, for L2 writing skills, the effect size is d = 0.67, with a confidence interval of 0.34 to 1.00. Both of these positive effects are statistically significant, with z-scores of 2.99 and 3.98, respectively, and corresponding p-values of 0.003 and less than 0.01. Based on these results, researchers have rejected the first three null hypotheses, concluding that the AWE-assisted intervention has a significant impact on reducing anxiety and enhancing motivation and L2 writing skills compared to the control group.</p> <hd id="AN0178463698-27">Writing Performances in China (H4)</hd> <p>The effect sizes across different countries—China (I<sups>2</sups> = 92.9%, p < 0.01), Turkey (I<sups>2</sups> = 96.4%, p < 0.01), Egypt (I<sups>2</sups> = 94.9%, p < 0.01), Iran (I<sups>2</sups> = 94.0%, p < 0.01), and the USA (I<sups>2</sups> = 88.5%, p < 0.01)—were found to be significantly heterogeneous. Therefore, random-effect models were employed for the meta-analysis. The pooled effect size, as depicted by the diamond in Fig. 4, is situated to the right of the no-effect line, indicating a positive impact of AWE-assisted writing in China. Specifically, the mean score of the experimental group is significantly higher than that of the control group [d = 0.47, with a confidence interval of 0.13 to 0.81]. This significant difference is further supported by a z-score of 2.72 and a p-value of 0.007. Based on these findings, researchers rejected the fourth null hypothesis, concluding that AWE-assisted writing has a significant positive impact in China.</p> <p>Graph: Fig. 4 A forest plot of AWE-assisted writing across the world</p> <hd id="AN0178463698-28">Writing Performances in Turkey, Iran, Egypt, and the USA (H5-8)</hd> <p>The diamonds in Turkey [d = 1.26, (− 0.25 ~ 2.78), z = 1.63, p = 0.103], Egypt [d = − 0.09, (− 1.33 ~ 1.16), z = 0.14, p = 0.890], Iran [d = 1.14, (− 0.55 ~ 2.83), z = 1.33, p = 0.185], and the USA [d = 0.09, (− 0.31 ~ 0.48), z = 0.42, p = 0.674] all intersect the no-effect line, indicating no significant differences in writing performance between AWE-assisted and non-AWE-assisted groups in these countries. Based on these findings, researchers accepted the fifth, sixth, seventh, and eighth null hypotheses, stating that there are no significant differences in writing performance between AWE and non-AWE-assisted groups in Turkey, Iran, Egypt, and the USA. To provide a clear overview, researchers compiled a summary of the test results for the proposed null hypotheses in Table 3.</p> <p>Table 3 Results of hypothesis tests</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>N</p></th><th align="left"><p>Hypothesis</p></th><th align="left"><p>Result</p></th></tr></thead><tbody><tr><td align="left"><p>1</p></td><td align="left"><p>1. Learners assisted with AWE could not have significantly higher L2 writing skills than those without AWE</p></td><td align="left"><p>Rejected</p></td></tr><tr><td align="left"><p>2</p></td><td align="left"><p>2. Learners assisted with AWE could not have significantly lower anxiety than those without AWE</p></td><td align="left"><p>Rejected</p></td></tr><tr><td align="left"><p>3</p></td><td align="left"><p>3. Learners assisted with AWE could not have significantly higher motivation than those without AWE</p></td><td align="left"><p>Rejected</p></td></tr><tr><td align="left"><p>4</p></td><td align="left"><p>4. There are no significantly different writing performances between AWE and non-AWE-assisted groups in China</p></td><td align="left"><p>Rejected</p></td></tr><tr><td align="left"><p>5</p></td><td align="left"><p>5. There are no significantly different writing performances between AWE and non-AWE-assisted groups in Turkey</p></td><td align="left"><p>Accepted</p></td></tr><tr><td align="left"><p>6</p></td><td align="left"><p>6. There are no significantly different writing performances between AWE and non-AWE-assisted groups in Iran</p></td><td align="left"><p>Accepted</p></td></tr><tr><td align="left"><p>7</p></td><td align="left"><p>7. There are no significantly different writing performances between AWE and non-AWE-assisted groups in Egypt</p></td><td align="left"><p>Accepted</p></td></tr><tr><td align="left"><p>8</p></td><td align="left"><p>8. There are no significantly different writing performances between AWE and non-AWE-assisted groups in the USA</p></td><td align="left"><p>Accepted</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0178463698-29">Results of a Sensitivity Analysis</hd> <p>A sensitivity analysis was conducted to test the stability and reliability of the results. Researchers obtained Fig. 5 by entering. "Metaninf N M SD N0 M0 SD0, label (namevar = Author, yearvar = Year) random Cohen" into the command column in Stata/MP 14.0. As shown in Fig. 5, the middle line indicates the average effect sizes, while the lines on both sides refer to the confidence intervals including the lower and upper limits. A dot indicates an individual effect size of a study. All the dots are located within the lower and upper limits of confidence intervals if given studies are omitted. Therefore, the meta-analysis estimates are considered robust and stable.</p> <p>Graph: Fig. 5 Results of a sensitivity analysis</p> <hd id="AN0178463698-30">Discussion</hd> <p>AWE could reduce anxiety if properly designed and adopted in writing evaluation. In case students perceive that AWE can make writing easier and smoother, their anxiety during writing may be reduced since they can complete the writing tasks easily. Educators can require students to integrate AWE into their writing to reveal their weakness, improve their self-efficacy, and reduce their writing anxiety. AWE can remind students of the errors and help them develop their thoughts, leading to reduced anxiety and improving their writing confidence. With AWE, students may become increasingly independent since they can solve most of the writing issues assisted with AWE although the harshest problems may remain unsolved. With intermittent or continuous trials with AWE, their writing will become growingly easier, coupled with reduced anxiety.</p> <p>However, AWE could increase writing anxiety due to its complex characteristics. The writing anxiety might source from technological stress (Matsumura & Hann, [<reflink idref="bib51" id="ref110">51</reflink>]). Technological stress referred to computer anxiety or computer resistance (Yaghi & Abu-Saba, [<reflink idref="bib86" id="ref111">86</reflink>]). Users of computers tended to feel uncomfortable and nervous, especially when they felt their computer skills were not enough to operate the system and apply related skills to writing (Smith & Caputi, [<reflink idref="bib70" id="ref112">70</reflink>]). The complexity of the AWE system might have puzzled them and external variables such as the Internet connection speed might also make them difficult to handle the writing process assisted with AWE. Designers and educators should make every effort to make the AWE technologies easily perceived and used in writing so that their anxiety could be reduced.</p> <p>With AWE, students will be motivated to write since the technologies can promote their writing practice. Timely and effective feedback is essential to promote students' writing motivation since it can provide guidelines in a timely manner. AWE can replace teachers who always fail to provide timely and consistent feedback for students (Bitchener & Ferris, [<reflink idref="bib13" id="ref113">13</reflink>]). AWE technologies can provide suggestions on writing performance instantaneously either inside or outside the classrooms, which can hardly be realized by human raters. AWE can also provide individualized feedback, catering to specific requirements. Considering both benefits and drawbacks of AWE technologies, they can be used to complement teachers' evaluation of writing tasks rather than completely replace them. The combination of AWE and human teachers may be able to motivate students better to produce high-quality writing tasks than the merely AWE-assisted style (Woodworth & Barkaoui, [<reflink idref="bib85" id="ref114">85</reflink>]).</p> <p>AWE technologies can improve L2 writing skills, especially at the lower-level skills such as knowledge. English writing has been considered a tedious and frustrating job for students who tend to reject this harsh task. AWE can activate students to write by providing instant and diagnostic feedback (Hyland, [<reflink idref="bib38" id="ref115">38</reflink>]). The frequent feedback on writing can correct writing and provide suggestions on how to improve it, which can improve L2 writing skills gradually and subconsciously. AWE can provide feedback on lower-level skills such as grammar while human teachers can address higher-level knowledge such as organization, content, coherence, and cohesion. Students' revisions can be improved based on the mixed feedback, together with enhanced L2 writing skills (Link et al., [<reflink idref="bib49" id="ref116">49</reflink>]). Although AWE may not be able to improve higher-level L2 writing skills, the combination between AWE and human raters may exert a strong and positive influence on L2 writing skills.</p> <p>AWE has sparked controversy in education circles. On one hand, AWE can provide teachers with objective feedback and objective data on student writing, enabling them to pinpoint areas where students need improvement (Shadiev et al., [<reflink idref="bib66" id="ref117">66</reflink>]). This can be a powerful motivator, encouraging students to work harder and refine their writing skills. However, some argue that AWE can have the opposite effect, leading to increased anxiety as students worry about numerical scores and artificial intelligence's judgment of their work (Sun & Fan, [<reflink idref="bib72" id="ref118">72</reflink>]). They fear that AWE may encourage formulaic writing and stifle creativity, critical thinking, and overall writing skills (Passonneau, [<reflink idref="bib59" id="ref119">59</reflink>]). The key is finding the right balance between objective feedback and subjective evaluation to foster motivation and improve writing skills without causing undue anxiety.</p> <p>AWE systems have been increasingly used in recent years to assist in the evaluation of written work, especially in second language learning contexts (Wei et al., [<reflink idref="bib81" id="ref120">81</reflink>]). These systems provide instant feedback on various aspects of writing, such as grammar, vocabulary, sentence structure, and style. While AWE systems are designed to enhance the writing process, their effectiveness and impact on writing performance may vary across different cultures and educational systems (Aldosemani et al., [<reflink idref="bib6" id="ref121">6</reflink>]). In this discussion, we compare the writing performances of AWE-assisted and non-AWE-assisted groups in China, Turkey, Iran, Egypt, and the United States. It is important to note that these countries have distinct educational systems, languages, and cultural backgrounds, which may influence the use and effectiveness of AWE systems.</p> <p>In China, where English is taught as a foreign language, AWE systems have been widely adopted in universities and language institutes. These systems provide valuable feedback on English grammar and vocabulary usage, which is crucial for non-native speakers (Shadiev et al., [<reflink idref="bib66" id="ref122">66</reflink>]). Studies have shown that AWE-assisted students in China tend to improve their grammar and vocabulary usage more quickly than those without AWE assistance. However, some critics argue that overreliance on AWE systems may lead to a lack of creativity and individuality in writing (Chen et al., [<reflink idref="bib15" id="ref123">15</reflink>]).</p> <p>In Turkey, where Turkish is the primary language, AWE systems are used less frequently due to the limited availability of suitable systems for Turkish. However, in recent years, there has been a growing interest in using AWE for Turkish, especially in the context of English language learning (Taskiran et al., [<reflink idref="bib75" id="ref124">75</reflink>]). Similar to China, AWE-assisted students in Turkey may benefit from improved grammar and vocabulary usage, but there are also concerns about potential limitations in creativity and individuality (Crosthwaite et al., [<reflink idref="bib20" id="ref125">20</reflink>]).</p> <p>In Iran and Egypt, where English is also taught as a foreign language, the use of AWE systems may face similar challenges and opportunities as in China and Turkey. However, it is worth noting that the educational systems and cultural backgrounds in these countries may influence the implementation and effectiveness of AWE systems (Zhai & Ma, [<reflink idref="bib91" id="ref126">91</reflink>]). For example, in Iran, where the educational system is highly centralized, the integration of AWE systems into the curriculum may require more bureaucratic approval and coordination. In Egypt, where the educational system is more decentralized, there may be more flexibility in the use of AWE systems at the local level (Shehab et al., [<reflink idref="bib67" id="ref127">67</reflink>]).</p> <p>In the United States, where English is the primary language, AWE systems are less common in primary and secondary education but are widely used in higher education and professional writing contexts (Wilson et al., [<reflink idref="bib84" id="ref128">84</reflink>]). These systems provide valuable feedback on writing style, clarity, and coherence, which is crucial for academic and professional writing. However, in the United States, there is also a strong emphasis on creativity and individuality in writing, and some critics argue that the use of AWE systems may limit these aspects of writing (Gibbons & Li, [<reflink idref="bib29" id="ref129">29</reflink>]).</p> <p>To sum up, the use and effectiveness of AWE systems in different countries are influenced by various factors, including the educational system, language background, and cultural values. While AWE systems can provide valuable feedback and improve writing performance in terms of grammar, vocabulary, and style, they should be used cautiously to avoid limiting creativity and individuality in writing. It is thus important to consider the specific needs and challenges of each country when designing and implementing AWE systems.</p> <hd id="AN0178463698-31">Conclusion</hd> <p></p> <hd id="AN0178463698-32">Major Findings</hd> <p>Through a PRISMA protocol-based meta-analysis, this study concludes that AWE technologies can significantly reduce anxiety such as writing anxiety and computer anxiety compared with those without the assistance of AWE. AWE can also significantly improve writing motivation and L2 writing skills through the combination with human writing raters. Educators and designers should highlight how to combine both writing evaluation methods and maximize the improvements in L2 writing skills.</p> <hd id="AN0178463698-33">Future Research Implications</hd> <p>Although this study rigorously adhered to the PRISMA protocol, it is important to acknowledge its limitations. Firstly, due to the constraints of library sources, it was not possible to filter all relevant literature, potentially excluding valuable studies. Secondly, the scope of this research was limited to a subset of the available advanced AWE technologies, meaning that some potentially significant systems may have been overlooked.</p> <p>Many studies have indeed shown the positive impact of Pigai technologies on writing improvements, particularly in Chinese contexts (Han et al., [<reflink idref="bib33" id="ref130">33</reflink>]; Huang & Renandya, [<reflink idref="bib37" id="ref131">37</reflink>]). These systems, often referred to as AWE tools, have been found to effectively assist learners in enhancing their writing abilities by providing timely and personalized feedback.</p> <p>However, it is important to note that the majority of these studies have been conducted primarily in Chinese, limiting our understanding of how AWE tools might perform in other languages. Given the significant linguistic differences across various languages, it is crucial for future research to validate the effectiveness of Pigai technologies in different contexts. This will not only help us understand the cross-cultural applicability of these tools but also provide valuable insights into how they can be further optimized to meet the unique needs of learners from different backgrounds.</p> <p>Moreover, current AWE systems primarily focus on lower-level writing skills such as word collocation and grammar correction. While these are important aspects of writing, they do not fully encompass the complexity of higher-level skills such as argumentative strength, coherence, and logical flow. Future research should therefore aim to explore how AWE tools can be enhanced to provide feedback on these higher-level skills (Conijn et al., [<reflink idref="bib19" id="ref132">19</reflink>]). One promising direction is the integration of artificial intelligence (AI) into AWE systems. By leveraging the power of AI, advanced AWE tools could potentially evaluate writing more comprehensively, addressing issues such as coherence and argumentative quality (Lin & Yu, [<reflink idref="bib48" id="ref133">48</reflink>]).</p> <p>Additionally, it is crucial to consider the user acceptance of AWE tools. Despite their potential benefits, learners may be resistant to using these systems if they do not perceive them as useful or easy to use (Aldosemani et al., [<reflink idref="bib6" id="ref134">6</reflink>]). Future research should therefore investigate the factors that influence user perception of AWE tools' usefulness and ease of use (Yu, [<reflink idref="bib88" id="ref135">88</reflink>], [<reflink idref="bib89" id="ref136">89</reflink>]). By understanding these factors, we can design more effective AWE tools that are not only beneficial but also appealing to learners. This approach will ensure that AWE tools are not only effective in improving writing skills but also widely adopted and used by learners.</p> <p>Consequently, while AWE tools have shown promising potential in enhancing writing skills, there remain numerous areas that require further exploration. Cross-cultural applicability is one such area, as it is essential to understand how these tools can be adapted and utilized effectively across different languages and cultures. Additionally, the ability of AWE tools to address higher-level writing skills remains a crucial aspect that needs to be addressed. Furthermore, user acceptance is a crucial factor that determines the success of these tools. Future research that delves into these issues will provide a deeper understanding of capabilities AWE tools and how they can revolutionize writing education globally, ensuring that learners across all languages and cultures can benefit from them.</p> <hd id="AN0178463698-34">Author Contributions</hd> <p>Xiaoli Huang: Methodology, Writing-Original Draft, Revising and Editing, Xu Wei: Data Curation and Revising, Fan Li: Investigation and Editing; Zhonggen Yu: Conceptualization.</p> <hd id="AN0178463698-35">Funding</hd> <p>This work is supported by Key Research and Application Project of the Key Laboratory of Key Technologies for Localization Language Services of the State Administration of Press and Publication, "Research on Localization and Intelligent Language Education Technology for the 'Belt and Road Initiative" (Project Number: CSLS 20230012), and Special fund of Beijing Co-construction Project-Research and reform of the "Undergraduate Teaching Reform and Innovation Project" of Beijing higher education in 2020-innovative "multilingual + " excellent talent training system (202010032003).</p> <hd id="AN0178463698-36">Data Availability</hd> <p>We make sure that all data and materials support our published claims and comply with field standards.</p> <hd id="AN0178463698-37">Declarations</hd> <p></p> <hd id="AN0178463698-38">Conflict of interest</hd> <p>We have no conflicts of interest to declare that are relevant to the content of this article.</p> <hd id="AN0178463698-39">Ethical Approval</hd> <p>The study was approved by the institutional review board of Beijing Language and Culture University. All researchers can provide written informed consents.</p> <hd id="AN0178463698-40">Supplementary Information</hd> <p>Below is the link to the electronic supplementary material.</p> <p>Graph: Supplementary file1 (XLSX 14 KB)</p> <hd id="AN0178463698-41">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0178463698-42"> <title> References </title> <blist> <bibl id="bib1" idref="ref45" type="bt">1</bibl> <bibtext> Abas A, Arifin K, Ali MAM, Khairil M. A systematic literature review on public participation in decision-making for local authority planning: A decade of progress and challenges. Environmental Development. 2023; 46. 10.1016/j.envdev.2023.100853</bibtext> </blist> <blist> <bibl id="bib2" idref="ref3" type="bt">2</bibl> <bibtext> Abbas M, van Rosmalen P, Kalz M. A data-driven approach for the identification of features for automated feedback on academic essays. IEEE Transactions on Learning Technologies. 2023; 16; 6: 914-925. 10.1109/TLT.2023.3320877</bibtext> </blist> <blist> <bibl id="bib3" idref="ref29" type="bt">3</bibl> <bibtext> Abdullah ATH, Alsohbo KSI, Hassan I. Gender differences in written communication anxiety among Libyan Postgraduates in Malaysia. Arab World English Journal. 2022; 13; 3: 285-295. 10.24093/awej/vol13no3.18</bibtext> </blist> <blist> <bibl id="bib4" idref="ref31" type="bt">4</bibl> <bibtext> Akbarzadeh M, Tajadini M, Narafshan MH. Metacognitive awareness instruction: A mixed method study on high school EFL learners' writing development and intrinsic motivation. Apuntes Universitarios. 2020; 10; 4: 189-202. 10.17162/au.v10i4.504</bibtext> </blist> <blist> <bibl id="bib5" idref="ref67" type="bt">5</bibl> <bibtext> Alberto Benitez R. Perceptions on the automated assessment of written discourse. Enunciacion. 2019; 24; 2: 227-240. 10.14483/22486798.14311</bibtext> </blist> <blist> <bibl id="bib6" idref="ref92" type="bt">6</bibl> <bibtext> Aldosemani T, Assalahi H, Lhothali A, Albsisi M. Automated writing evaluation in EFL contexts: A review of effectiveness, impact, and pedagogical implications. International Journal of Computer-Assisted Language Learning and Teaching. 2023. 10.4018/IJCALLT.329962</bibtext> </blist> <blist> <bibl id="bib7" idref="ref35" type="bt">7</bibl> <bibtext> Al-nakhle H. The effectiveness of scenario-based virtual laboratory simulations to improve learning outcomes and scientific report writing skills. PLoS ONE. 2022; 17; 11. 10.1371/journal.pone.0277359</bibtext> </blist> <blist> <bibl id="bib8" idref="ref54" type="bt">8</bibl> <bibtext> Azmi AM, Al-Jouie MF, Hussain M. AAEE – Automated evaluation of students' essays in Arabic language. Information Processing & Management. 2019; 56; 5: 1736-1752. 10.1016/j.ipm.2019.05.008</bibtext> </blist> <blist> <bibl id="bib9" idref="ref74" type="bt">9</bibl> <bibtext> Beach R. The effects of between-draft teacher evaluation versus student selfevaluation on high school students' revising of rough drafts. Research in the Teaching of English. 1979; 13; 2: 111-119. 10.58680/rte197917848</bibtext> </blist> <blist> <bibtext> Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994; 50: 1088-1101. 10.2307/2533446</bibtext> </blist> <blist> <bibtext> Benavides-Varela S, Zandonella Callegher C, Fagiolini B, Leo I, Altoà G, Lucangeli D. Effectiveness of digital-based interventions for children with mathematical learning difficulties: A meta-analysis. Computers & Education. 2020; 157. 10.1016/j.compedu.2020.103953</bibtext> </blist> <blist> <bibtext> Biglari A, Izadpanah S, Namaziandost E. The effect of portfolio assessment on Iranian EFL learners' autonomy and writing skills. Education Research International. 2021; 2021: 4106882. 10.1155/2021/4106882</bibtext> </blist> <blist> <bibtext> Bitchener J, Ferris DR. Written corrective feedback in second language acquisition and writing. 2012; Routledge. 10.4324/9780203832400</bibtext> </blist> <blist> <bibtext> Cao W, Yu Z. Exploring learning outcomes, communication, anxiety, and motivation in learning communities: A systematic review. Humanities & Social Sciences Communications. 2023; 10; 1: 866. 10.1057/s41599-023-02325-2</bibtext> </blist> <blist> <bibtext> Chen, L, Leong, C. W, Feng, G, Lee, C. M, & Somasundaran, S. (2015). Utilizing Multimodal Cues to Automatically Evaluate Public Speaking Performance. 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), 394–400. Xi'an, China, 2015, pp. 394–400, https://doi.org/10.1109/ACII.2015.7344601</bibtext> </blist> <blist> <bibtext> Chen M-RA, Hwang G-J. Effects of experiencing authentic contexts on English speaking performances, anxiety and motivation of EFL students with different cognitive styles. Interactive Learning Environments. 2022; 30; 9: 1619-1639. 10.1080/10494820.2020.1734626</bibtext> </blist> <blist> <bibtext> Cheng, J. (2001). Writing for communication: A proposal for reform in teaching English writing in China (PQDT:65454716). Doctoral Dissertation of Xiamen University (People's Republic of China).</bibtext> </blist> <blist> <bibtext> Cheng, G, Law, E, & Wong, T. L. (2016). Investigating effects of automated feedback on EFL students' reflective learning skills. 2016 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE). 7–9 December 2016, Dusit Thani Bangkok Hotel, Bangkok, Thailand. https://doi.org/10.1109/tale.2016.7851798</bibtext> </blist> <blist> <bibtext> Conijn R, Martinez-Maldonado R, Knight S, Buckingham Shum S, Van Waes L, van Zaanen M. How to provide automated feedback on the writing process? A participatory approach to design writing analytics tools. Computer Assisted Language Learning. 2022; 35; 8: 1838-1868. 10.1080/09588221.2020.1839503</bibtext> </blist> <blist> <bibtext> Crosthwaite P, Ningrum S, Lee I. Research trends in L2 written corrective feedback: A bibliometric analysis of three decades of Scopus-indexed research on L2 WCF. Journal of Second Language Writing. 2022; 58. 10.1016/j.jslw.2022.100934</bibtext> </blist> <blist> <bibtext> Derakhshan A, Setiawan S, Ghafouri M. Modeling the interplay of Indonesian and Iranian EFL teachers' apprehension, resilience, organizational mattering, and psychological well-being. Iranian Journal of Language Teaching Research. 2024; 12; 1: 21-43. 10.30466/ijltr.2024.121416</bibtext> </blist> <blist> <bibtext> Derakhshan A, Karimpour S, Nazari M. Exploring the professional role identities of English for academic purposes practitioners: A qualitative study. International Review of Applied Linguistics in Language Teaching (IRAL). 2024. 10.1515/iral-2023-0126</bibtext> </blist> <blist> <bibtext> Derakhshan A, Molana K, Nazari M. I wonder who I am when my assessment ideas are not respected": A case-study inquiry into the role of institutional policies in English language teachers' assessment identity. Asia Pacific Journal of Education. 2024. 10.1080/02188791.2024.2328237</bibtext> </blist> <blist> <bibtext> Derakhshan A, Moradi F, Nazari M. Conceptualising the role of practice level in language teacher identity construction: An identities-in-practice study. International Journal of Applied Linguistics. 2024. 10.1111/ijal.12533</bibtext> </blist> <blist> <bibtext> Ebadi S, Gholami M, Vakili S. Investigating the effects of using Grammarly in EFL writing: The case of articles. Computers in the Schools. 2023; 40; 1: 85-105. 10.1080/07380569.2022.2150067</bibtext> </blist> <blist> <bibtext> Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. British Medical Journal. 1997; 315: 629-634. 10.1136/bmj.315.7109.629</bibtext> </blist> <blist> <bibtext> Franzke M, Kintsch E, Caccamise D, Johnson N, Dooley S. Summary Street®: Computer support for comprehension and writing. Journal of Educational Computing Research. 2005; 33; 1: 53-80. 10.2190/DH8F-QJWM-J457-FQVB</bibtext> </blist> <blist> <bibtext> Gee TC. Students' responses to teacher comments. Research in the Teaching of English. 1972; 6; 2: 212-221. 10.58680/rte197220151</bibtext> </blist> <blist> <bibtext> Gibbons J, Li M. Doctoral dissertations on second language writing and technology in the USA (2010–2019). Language Teaching. 2021; 54; 2: 273-291. 10.1017/S0261444820000403</bibtext> </blist> <blist> <bibtext> Gollnitz, D.L. (2010). Improvement of writing at Grades 10 and 11: Does automated essay scoring software help students improve their writing skills? Oakland University Doctiral Dissertation.</bibtext> </blist> <blist> <bibtext> Guo K, Wang D. To resist it or to embrace it? Examining ChatGPT's potential to support teacher feedback in EFL writing. Education and Information Technologies. 2023. 10.1007/s10639-023-12146-0</bibtext> </blist> <blist> <bibtext> Han T, Sari E. An investigation on the use of automated feedback in Turkish EFL students' writing classes. Computer Assisted Language Learning. 2022. 10.1080/09588221.2022.2067179</bibtext> </blist> <blist> <bibtext> Han YX, Zhao S, Ng LL. How technology tools impact writing performance, lexical complexity, and perceived self-regulated learning strategies in EFL academic writing: A comparative study. Frontiers in Psychology. 2021; 12. 10.3389/fpsyg.2021.752793</bibtext> </blist> <blist> <bibtext> Hassanzadeh M, Fotoohnejad S. Implementing an automated feedback program for a foreign language writing course: A learner-centric study Implementing an AWE tool in a L2 class. Journal of Computer Assisted Learning. 2021; 37; 5: 1494-1507. 10.1111/jcal.12587</bibtext> </blist> <blist> <bibtext> Higgins, J.P.T, & Green, S. (2021). Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration 2011. 2021 Available from https://handbook-5-1.cochrane.org/</bibtext> </blist> <blist> <bibtext> Hoffman B. The influence of peer assessment training on assessment knowledge and reflective writing skill. Journal of Applied Research in Higher Education. 2019; 11; 4: 863-875. 10.1108/JARHE-01-2019-0004</bibtext> </blist> <blist> <bibtext> Huang S, Renandya WA. Exploring the integration of automated feedback among lower-proficiency EFL learners. Innovation in Language Learning and Teaching. 2020; 14; 1: 15-26. 10.1080/17501229.2018.1471083</bibtext> </blist> <blist> <bibtext> Hyland, K. (1990). A genre description of the argumentative essay. RELC Journal, 21(1), 66-78. https://doi.org/10.1177/003368829002100105</bibtext> </blist> <blist> <bibtext> Ibishi A, Xhaferri G, Ivanovska B. Error analysis by using the prepositive complement-a study with Albanian-speaking German learners in North Macedonia. Folia Linsguistica Et Litteraria. 2023; 44: 103-130. 10.31902/fll.44.2023.6</bibtext> </blist> <blist> <bibtext> Jansen T, Meyer J, Fleckenstein J, Horbach A, Keller S, Moeller J. Individualizing goal-setting interventions using automated writing evaluation to support secondary school students' text revisions. Learning and Instruction. 2024; 89. 10.1016/j.learninstruc.2023.101847</bibtext> </blist> <blist> <bibtext> Kaya B, Ates S, Yildirim K, Rasinski T. Effect of feedback on Turkish fourth-grade elementary school students' fluent writing skills. Education and Science. 2020; 45; 201: 189-205. 10.15390/EB.2019.8413</bibtext> </blist> <blist> <bibtext> Kepes S, McDaniel MA, Brannick MT, Banks GC. Meta-analytic reviews in the organizational sciences: Two meta-analytic schools on the way to MARS (the Meta-Analytic Reporting Standards). Journal of Business and Psychology. 2013; 28; 2: 123-143. 10.1007/s10869-013-9300-2</bibtext> </blist> <blist> <bibtext> Khoii, R, & Doroudian, A. (2014). Automated Scoring of EFL Learners' Written Performance: A Torture of a Blessing? EDULEARN14. In: 6th International Conference on Education and New Learning Technologies. 7–9 July, 2014. Barcelona, Spain</bibtext> </blist> <blist> <bibtext> Li R. Still a fallible tool? Revisiting effects of automated writing evaluation from activity theory perspective. British Journal of Educational Technology. 2023; 54; 3: 773-789. 10.1111/bjet.13294</bibtext> </blist> <blist> <bibtext> Li W-Y, Kau K, Shiung Y-J. Pedagogic exploration into adapting automated writing evaluation and peer review integrated feedback into large-sized university writing classes. SAGE Open. 2023; 13; 4: 21582440231209087. 10.1177/21582440231209087</bibtext> </blist> <blist> <bibtext> Li W, Lu Z, Liu QFrederiksen K-M, Larsen S, Bradley L, Thouësny S. Syntactic complexity development in college students' essay writing based on AWE. CALL for widening participation: short papers from EUROCALL 2020. 2020; Research-publishing.net: 190-194. 10.14705/rpnet.2020.48.1187</bibtext> </blist> <blist> <bibtext> Liao Calvin, C. Y, Li, Y, & Cheng Hercy, N. H. (2019). Examining the Effects of Automatic Comment Classification on Comment Types in Peer Review for Graduate Students' Academic Writing. In: Proceedings - 2019 8th International Congress on Advanced Applied Informatics, IIAI-AAI 2019, p. 215-220. https://doi.org/10.1109/IIAI-AAI.2019.00050</bibtext> </blist> <blist> <bibtext> Lin YP, Yu ZG. Extending technology acceptance model to higher-education students' use of digital academic reading tools on computers. International Journal of Educational Technology in Higher Education. 2023. 10.1186/s41239-023-00403-8</bibtext> </blist> <blist> <bibtext> Link S, Mehrzad M, Rahimi M. Impact of automated writing evaluation on teacher feedback, student revision, and writing improvement. Computer Assisted Language Learning. 2022; 35; 4: 605-634. 10.1080/09588221.2020.1743323</bibtext> </blist> <blist> <bibtext> Liu M, Zhang LJ, Biebricher C. Investigating students' cognitive processes in generative AI-assisted digital multimodal composing and traditional writing. Computers & Education. 2024; 211. 10.1016/j.compedu.2023.104977</bibtext> </blist> <blist> <bibtext> Matsumura S, Hann G. Computer anxiety and students' preferred feedback methods in EFL writing. The Modern Language Journal. 2004; 88; 3: 403-415. 10.1111/j.0026-7902.2004.00237.x</bibtext> </blist> <blist> <bibtext> Mohsen MA. Computer-mediated corrective feedback to improve L2 writing skills: A meta-analysis. Journal of Educational Computing Research. 2022; 60; 5: 1253-1276. 10.1177/07356331211064066</bibtext> </blist> <blist> <bibtext> Morris SB, DeShon RP. Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods. 2002; 7; 1: 105-125. 10.1037/1082-989X.7.1.105</bibtext> </blist> <blist> <bibtext> Nazar R, Renau I. ESTILECTOR: An automatic evaluation system for academic writing in Spanish. Perspectiva Educacional. 2023; 62; 2: 37-59. 10.4151/07189729-Vol.62-Iss.2-Art.1427</bibtext> </blist> <blist> <bibtext> Ngo TTN, Chen HHJ, Lai KKW. The effectiveness of automated writing evaluation in EFL/ESL writing: A three-level meta-analysis. Interactive Learning Environments. 2022. 10.1080/10494820.2022.2096642</bibtext> </blist> <blist> <bibtext> Nunes A, Cordeiro C, Limpo T, Castro S. Effectiveness of automated writing evaluation systems in school settings: A systematic review of studies from 2000 to 2020. Journal of Computer Assisted Learning. 2022; 38; 2: 599-620. 10.1111/jcal.12635</bibtext> </blist> <blist> <bibtext> Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ-British Medical Journal. 2021; 372. 10.1136/bmj.n71</bibtext> </blist> <blist> <bibtext> Parra GL, Calero SX. Automated writing evaluation tools in the improvement of the writing skill. International Journal of Instruction. 2019; 12; 2: 209-226. 10.29333/iji.2019.12214a</bibtext> </blist> <blist> <bibtext> Passonneau, R. (2018). EAGER: Collaborative Research: Automated Instruction Assistant for Argumentative Essays. National Science Foundation, Cyberlearn & Future Learn Tech (Grant NUMBER: 1847853).</bibtext> </blist> <blist> <bibtext> Prichard JR. Writing to learn: An evaluation of the calibrated peer review program in two neuroscience courses. Journal of Undergraduate Neuroscience Education: JUNE: A Publication of FUN, Faculty for Undergraduate Neuroscience. 2005; 4; 1: A34-A39</bibtext> </blist> <blist> <bibtext> Reynolds BL, Kao CW, Huang YY. Investigating the effects of perceived feedback source on second language writing performance: A quasi-experimental study. Asia-Pacific Education Researcher. 2021; 30; 6: 585-595. 10.1007/s40299-021-00597-3</bibtext> </blist> <blist> <bibtext> Rudner LM, Garcia V, Welch C. An evaluation of IntelliMetricTM essay scoring system. The Journal of Technology, Learning and Assessment. 2006; 4; 4: 1-21</bibtext> </blist> <blist> <bibtext> Sahin I, Yagiz O. An exploratory investigation into the graduate supervisors' cognitions of academic writing in English (AWE). Journal of Language Teaching and Learning. 2024; 14; 1: 66-84</bibtext> </blist> <blist> <bibtext> Sawilowsky SS. New effect size rules of thumb. Journal of Modern Applied Statistical Methods. 2009; 8; 2: 597-599. 10.22237/jmasm/1257035100</bibtext> </blist> <blist> <bibtext> Sedgwick, P.M, & Marston, L. (2013). Meta-analyses: standardised mean differences. BMJ: British Medical Journal, 347. https://doi.org/10.1136/bmj.f7257</bibtext> </blist> <blist> <bibtext> Shadiev R, Feng Y, Zhussupova R, Huang Y-M. Intercultural competence development through a tele-collaborative project supported by speech-enabled corrective feedback technology. Journal of Computer Assisted Learning. 2023. 10.1111/jcal.12906</bibtext> </blist> <blist> <bibtext> Shehab, A, Elhoseny, M, & Hassanien, A. E. (2016). A hybrid scheme for automated essay grading based on LVQ and NLP techniques. ICENCO 2016 12th International Computer Engineering Conference (ICENCO), Cairo, Egypt, 2016, pp. 65-70,https://doi.org/10.1109/ICENCO.2016.7856447</bibtext> </blist> <blist> <bibtext> Shermis, M. D, & Hamner, B. (2012). Contrasting state-of-the-art automated scoring of essays: Analysis. Presented at the National Council of Measurement in Education, Vancouver, BC, Canada. https://doi.org/10.4324/9780203122761.CH19Corpus</bibtext> </blist> <blist> <bibtext> Shermis MD, Hamner BShermis MD, Burstein J. Contrasting state-of-the-art in machine scoring of essays. Handbook of automated essay evaluation. 2013; Routledge: 313-346. 10.4324/9780203122761</bibtext> </blist> <blist> <bibtext> Smith B, Caputi P. Cognitive interference in computer anxiety. Behaviour & Information Technology. 2001; 20: 265-273. 10.1080/01449290110069392</bibtext> </blist> <blist> <bibtext> Solhi M, Derakhshan A, Pawlak M, Ünsal B. Exploring the interplay between EFL learners' L2 writing boredom, writing motivation, and boredom coping strategies. Language Teaching Research. 2024. 10.1177/13621688241239178</bibtext> </blist> <blist> <bibtext> Sun B, Fan T. The effects of an AWE-aided assessment approach on business English writing performance and writing anxiety: A contextual consideration. Studies in Educational Evaluation. 2022; 72. 10.1016/j.stueduc.2021.101123</bibtext> </blist> <blist> <bibtext> Türkben, T. (2019). The effect of self-regulation based strategic reading education on comprehension, motivation, and self-regulation skills. International Journal of Progressive Education, 15(4), 27-46. https://doi.org/10.29329/ijpe.2019.203.3</bibtext> </blist> <blist> <bibtext> Tashu TM, Horváth T. A layered approach to automatic essay evaluation using word-embedding. Communications in computer and information science. 2019; Springer International Publishing: 77-94. 10.1007/978-3-030-21151-6_5</bibtext> </blist> <blist> <bibtext> Taskiran A, Yazici M, Erdem Aydin I. Contribution of automated feedback to the English writing competence of distance foreign language learners. E-Learning and Digital Media. 2024; 21; 1: 24-41. 10.1177/20427530221139579</bibtext> </blist> <blist> <bibtext> Waer H. The effect of integrating automated writing evaluation on EFL writing apprehension and grammatical knowledge. Innovation in Language Learning and Teaching. 2021. 10.1080/17501229.2021.1914062</bibtext> </blist> <blist> <bibtext> Waer H. The effect of integrating automated writing evaluation on EFL writing apprehension and grammatical knowledge. Innovation in Language Learning and Teaching. 2023; 17; 1: 47-71. 10.1080/17501229.2021.1914062</bibtext> </blist> <blist> <bibtext> Wang H, Troia GAA. Writing quality predictive modeling: Integrating register-related factors. Written Communication. 2023; 40; 4: 1070-1112. 10.1177/07410883231185287</bibtext> </blist> <blist> <bibtext> Wang YJ, Shang HF, Briody P. Exploring the impact of using automated writing evaluation in English as a foreign language university students' writing. Computer Assisted Language Learning. 2013; 26; 3: 234-257. 10.1080/09588221.2012.655300</bibtext> </blist> <blist> <bibtext> Wang Z. Computer-assisted EFL writing and evaluations based on artificial intelligence: A case from a college reading and writing course. Library Hi Tech. 2022; 40; 1: 80-97. 10.1108/LHT-05-2020-0113</bibtext> </blist> <blist> <bibtext> Wei P, Wang X, Dong H. The impact of automated writing evaluation on second language writing skills of Chinese EFL learners: A randomized controlled trial. Frontiers in Psychology. 2023; 14: 1249991. 10.3389/fpsyg.2023.1249991</bibtext> </blist> <blist> <bibtext> Weigle SC. Validation of automated scores of TOEFL iBT tasks against non-test indicators of writing ability. Language Testing. 2010; 27: 335-353. 10.1177/0265532210364406</bibtext> </blist> <blist> <bibtext> Wilson J, Czik A. Automated essay evaluation software in English Language Arts classrooms: Effects on teacher feedback, student motivation, and writing quality. Computers & Education. 2016; 100: 94-109. 10.1016/j.compedu.2016.05.004</bibtext> </blist> <blist> <bibtext> Wilson J, Huang Y, Palermo C, Beard G, MacArthur CA. Automated feedback and automated scoring in the elementary grades: Usage, attitudes, and associations with writing outcomes in a districtwide implementation of MI Write. International Journal of Artificial Intelligence in Education. 2021; 31; 2: 234-276. 10.1007/s40593-020-00236-w</bibtext> </blist> <blist> <bibtext> Woodworth J, Barkaoui K. Perspectives on using automated writing evaluation systems to provide written corrective feedback in the ESL classroom. TESL Canada Journal. 2020; 37; 2: 234-247. 10.18806/tesl.v37i2.1340</bibtext> </blist> <blist> <bibtext> Yaghi HM, Abu-Saba MB. Teachers' computer anxiety: An international perspective. Computers in Human Behavior. 1998; 14: 321-336. 10.1016/S0747-5632(98)00010-7</bibtext> </blist> <blist> <bibtext> Yao Y, Wang WJ, Yang XP. Perceptions of the inclusion of Automatic Writing Evaluation in peer assessment on EFL writers' language mindsets and motivation: A short-term longitudinal study. Assessing Writing. 2021; 50. 10.1016/j.asw.2021.100568</bibtext> </blist> <blist> <bibtext> Yu, M. (2019). The Effectiveness of Online Automatic Evaluation on College English Writing. In Y. Zhang, X. Zang, & T. Volodina (Eds.), Proceedings of the 6th International Conference on Education, Language, Art, and Inter-Cultural Communication (ICELAIC 2019) (Vol. 378, pp. 109–111). https://doi.org/10.2991/assehr.k.191217.024</bibtext> </blist> <blist> <bibtext> Yu Z. Identifying student satisfaction in the flipped English class enhanced with clickers. International Journal of Information and Communication Technology Education. 2019; 15; 4: 4. 10.4018/IJICTE.2019100103</bibtext> </blist> <blist> <bibtext> Yu Z. Learning outcomes, motivation, and satisfaction in gamified English vocabulary learning. SAGE Open. 2023. 10.1177/21582440231158332</bibtext> </blist> <blist> <bibtext> Zhai N, Ma X. Automated writing evaluation (AWE) feedback: A systematic investigation of college students' acceptance. Computer Assisted Language Learning. 2022; 35; 9: 2817-2842. 10.1080/09588221.2021.1897019</bibtext> </blist> <blist> <bibtext> Zhang M, Chen W. Assessing collaborative writing in the digital age: An exploratory study. Journal of Second Language Writing. 2022. 10.1016/j.jslw.2022.100868</bibtext> </blist> </ref> <aug> <p>By Xiaoli Huang; Wei Xu; Fan Li and Zhonggen Yu</p> <p>Reported by Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib68" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib69" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib82" firstref="ref4"></nolink> <nolink nlid="nl4" bibid="bib62" firstref="ref5"></nolink> <nolink nlid="nl5" bibid="bib66" firstref="ref6"></nolink> <nolink nlid="nl6" bibid="bib72" firstref="ref7"></nolink> <nolink nlid="nl7" bibid="bib75" firstref="ref8"></nolink> <nolink nlid="nl8" bibid="bib78" firstref="ref9"></nolink> <nolink nlid="nl9" bibid="bib44" firstref="ref10"></nolink> <nolink nlid="nl10" bibid="bib43" firstref="ref11"></nolink> <nolink nlid="nl11" bibid="bib55" firstref="ref13"></nolink> <nolink nlid="nl12" bibid="bib52" firstref="ref14"></nolink> <nolink nlid="nl13" bibid="bib81" firstref="ref15"></nolink> <nolink nlid="nl14" bibid="bib45" firstref="ref16"></nolink> <nolink nlid="nl15" bibid="bib88" firstref="ref17"></nolink> <nolink nlid="nl16" bibid="bib89" firstref="ref18"></nolink> <nolink nlid="nl17" bibid="bib50" firstref="ref20"></nolink> <nolink nlid="nl18" bibid="bib71" firstref="ref23"></nolink> <nolink nlid="nl19" bibid="bib91" firstref="ref24"></nolink> <nolink nlid="nl20" bibid="bib39" firstref="ref25"></nolink> <nolink nlid="nl21" bibid="bib85" firstref="ref27"></nolink> <nolink nlid="nl22" bibid="bib54" firstref="ref28"></nolink> <nolink nlid="nl23" bibid="bib24" firstref="ref30"></nolink> <nolink nlid="nl24" bibid="bib21" firstref="ref32"></nolink> <nolink nlid="nl25" bibid="bib22" firstref="ref33"></nolink> <nolink nlid="nl26" bibid="bib23" firstref="ref34"></nolink> <nolink nlid="nl27" bibid="bib32" firstref="ref39"></nolink> <nolink nlid="nl28" bibid="bib33" firstref="ref40"></nolink> <nolink nlid="nl29" bibid="bib76" firstref="ref44"></nolink> <nolink nlid="nl30" bibid="bib14" firstref="ref46"></nolink> <nolink nlid="nl31" bibid="bib87" firstref="ref47"></nolink> <nolink nlid="nl32" bibid="bib41" firstref="ref48"></nolink> <nolink nlid="nl33" bibid="bib17" firstref="ref49"></nolink> <nolink nlid="nl34" bibid="bib60" firstref="ref51"></nolink> <nolink nlid="nl35" bibid="bib47" firstref="ref52"></nolink> <nolink nlid="nl36" bibid="bib27" firstref="ref53"></nolink> <nolink nlid="nl37" bibid="bib79" firstref="ref55"></nolink> <nolink nlid="nl38" bibid="bib74" firstref="ref59"></nolink> <nolink nlid="nl39" bibid="bib34" firstref="ref60"></nolink> <nolink nlid="nl40" bibid="bib92" firstref="ref61"></nolink> <nolink nlid="nl41" bibid="bib80" firstref="ref62"></nolink> <nolink nlid="nl42" bibid="bib56" firstref="ref63"></nolink> <nolink nlid="nl43" bibid="bib16" firstref="ref64"></nolink> <nolink nlid="nl44" bibid="bib49" firstref="ref65"></nolink> <nolink nlid="nl45" bibid="bib58" firstref="ref66"></nolink> <nolink nlid="nl46" bibid="bib83" firstref="ref68"></nolink> <nolink nlid="nl47" bibid="bib37" firstref="ref71"></nolink> <nolink nlid="nl48" bibid="bib30" firstref="ref72"></nolink> <nolink nlid="nl49" bibid="bib28" firstref="ref73"></nolink> <nolink nlid="nl50" bibid="bib40" firstref="ref78"></nolink> <nolink nlid="nl51" bibid="bib31" firstref="ref79"></nolink> <nolink nlid="nl52" bibid="bib63" firstref="ref80"></nolink> <nolink nlid="nl53" bibid="bib12" firstref="ref81"></nolink> <nolink nlid="nl54" bibid="bib77" firstref="ref82"></nolink> <nolink nlid="nl55" bibid="bib90" firstref="ref96"></nolink> <nolink nlid="nl56" bibid="bib11" firstref="ref97"></nolink> <nolink nlid="nl57" bibid="bib57" firstref="ref98"></nolink> <nolink nlid="nl58" bibid="bib42" firstref="ref99"></nolink> <nolink nlid="nl59" bibid="bib53" firstref="ref101"></nolink> <nolink nlid="nl60" bibid="bib35" firstref="ref104"></nolink> <nolink nlid="nl61" bibid="bib10" firstref="ref105"></nolink> <nolink nlid="nl62" bibid="bib26" firstref="ref106"></nolink> <nolink nlid="nl63" bibid="bib65" firstref="ref108"></nolink> <nolink nlid="nl64" bibid="bib64" firstref="ref109"></nolink> <nolink nlid="nl65" bibid="bib51" firstref="ref110"></nolink> <nolink nlid="nl66" bibid="bib86" firstref="ref111"></nolink> <nolink nlid="nl67" bibid="bib70" firstref="ref112"></nolink> <nolink nlid="nl68" bibid="bib13" firstref="ref113"></nolink> <nolink nlid="nl69" bibid="bib38" firstref="ref115"></nolink> <nolink nlid="nl70" bibid="bib59" firstref="ref119"></nolink> <nolink nlid="nl71" bibid="bib15" firstref="ref123"></nolink> <nolink nlid="nl72" bibid="bib20" firstref="ref125"></nolink> <nolink nlid="nl73" bibid="bib67" firstref="ref127"></nolink> <nolink nlid="nl74" bibid="bib84" firstref="ref128"></nolink> <nolink nlid="nl75" bibid="bib29" firstref="ref129"></nolink> <nolink nlid="nl76" bibid="bib19" firstref="ref132"></nolink> <nolink nlid="nl77" bibid="bib48" firstref="ref133"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1430873
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: A Meta-Analysis of Effects of Automated Writing Evaluation on Anxiety, Motivation, and Second Language Writing Skills
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Xiaoli+Huang%22">Xiaoli Huang</searchLink> (ORCID <externalLink term="http://orcid.org/0009-0006-4782-0446">0009-0006-4782-0446</externalLink>)<br /><searchLink fieldCode="AR" term="%22Wei+Xu%22">Wei Xu</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-7224-1116">0000-0002-7224-1116</externalLink>)<br /><searchLink fieldCode="AR" term="%22Fan+Li%22">Fan Li</searchLink> (ORCID <externalLink term="http://orcid.org/0009-0008-6733-6795">0009-0008-6733-6795</externalLink>)<br /><searchLink fieldCode="AR" term="%22Zhonggen+Yu%22">Zhonggen Yu</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-3873-980X">0000-0002-3873-980X</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Asia-Pacific+Education+Researcher%22"><i>Asia-Pacific Education Researcher</i></searchLink>. 2024 33(4):957-976.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 20
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Information Analyses
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Writing+Evaluation%22">Writing Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Attitudes%22">Writing Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Anxiety%22">Anxiety</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Attitudes%22">Computer Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Skills%22">Writing Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Motivation%22">Motivation</searchLink><br /><searchLink fieldCode="DE" term="%22Second+Language+Learning%22">Second Language Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Achievement%22">Writing Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Second+Language+Instruction%22">Second Language Instruction</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s40299-024-00865-y
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0119-5646<br />2243-7908
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: With the rapid advancement of information technologies, automated writing evaluation technologies have developed so fast that they can be applied to writing assessments. However, scanty studies have pooled the effects of automated writing evaluation on writing performance. Through a PRISMA protocol-based meta-analysis, this study concludes that automated writing evaluation technologies can significantly reduce anxiety such as writing anxiety and computer anxiety compared with those without the assistance of automated writing evaluation. Automated writing evaluation can also significantly improve writing motivation and second language (L2) writing skills. Educators and designers should highlight how to combine automated writing evaluation with human rating methods and maximize the improvements in L2 writing skills. In the future, artificial intelligence may be integrated into automated writing evaluation to develop advanced automated writing evaluation to address higher-level issues in writing practice.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1430873
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1430873
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s40299-024-00865-y
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 20
        StartPage: 957
    Subjects:
      – SubjectFull: Writing Evaluation
        Type: general
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Writing Attitudes
        Type: general
      – SubjectFull: Anxiety
        Type: general
      – SubjectFull: Computer Attitudes
        Type: general
      – SubjectFull: Writing Skills
        Type: general
      – SubjectFull: Motivation
        Type: general
      – SubjectFull: Second Language Learning
        Type: general
      – SubjectFull: Writing Achievement
        Type: general
      – SubjectFull: Second Language Instruction
        Type: general
    Titles:
      – TitleFull: A Meta-Analysis of Effects of Automated Writing Evaluation on Anxiety, Motivation, and Second Language Writing Skills
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Xiaoli Huang
      – PersonEntity:
          Name:
            NameFull: Wei Xu
      – PersonEntity:
          Name:
            NameFull: Fan Li
      – PersonEntity:
          Name:
            NameFull: Zhonggen Yu
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 0119-5646
            – Type: issn-electronic
              Value: 2243-7908
          Numbering:
            – Type: volume
              Value: 33
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Asia-Pacific Education Researcher
              Type: main
ResultId 1