View in EDS HTML Full Text PDF Full Text

Validating Student AI Competency Self-Efficacy (SAICS) Scale and Its Framework

Saved in:
Bibliographic Details
Title: Validating Student AI Competency Self-Efficacy (SAICS) Scale and Its Framework
Language: English
Authors: Thomas K. F. Chiu (ORCID 0000-0003-2887-5477), Murat Çoban (ORCID 0000-0003-2415-5747), Ismaila Temitayo Sanusi (ORCID 0000-0002-5705-6684), Musa Adekunle Ayanwale (ORCID 0000-0001-7640-9898)
Source: Educational Technology Research and Development. 2025 73(4):2785-2807.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 23
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Education Level: Elementary Education
Secondary Education
Descriptors: Artificial Intelligence, Digital Literacy, Competence, Self Efficacy, Measures (Individuals), Test Validity, Elementary School Students, Secondary School Students, Interdisciplinary Approach, Decision Making, Data, Ethics, Design, Multimedia Materials, Factor Analysis, Gender Differences, Test Items, Test Construction, Delphi Technique
DOI: 10.1007/s11423-025-10512-y
ISSN: 1042-1629
1556-6501
Abstract: Nurturing student artificial intelligence (AI) competency is crucial in the future of K-12 education. Students with strong AI competency should be able to ethically, safely, healthily, and productively integrate AI into their learning. Research on student AI competency is still in its infancy, primarily focusing on theoretical and professional discussions, along with qualitative investigations. This two-stage study aims to propose an AI competency framework for students and confirm the reliability and validity of its scale--student AI competency self-efficacy (SAICS)--in K-12 education. In stage 1, we used a three-round Delphi study to propose the framework and its scale. The framework has eight dimensions: interdisciplinary learning with AI, assessment with AI, decision-making with AI, data, ethics and AI, designing AI, multimedia creation with AI, human-centric learning, and confidence with AI. Each dimension contains four items. In stage 2, we involved 448 students to validate the scale using confirmatory factor analysis and model comparisons. The analyses showed that the scale is consistent across male and female students. The SAICS scale comprises 32 items and addresses eight dimensions of AI competency. Researchers can use the framework and SAICS to design their interventions and correlational research associated with student AI competency. Teachers can use them to develop learning outcomes for AI-based learning activities, and policymakers can use them to establish national AI standards.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1483804
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwEOfv5ctHf3zODci0lNM2NIAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDE0LKGwXP6JfLIRiegIBEICBm98W8I2-AaObd82I6qe2OuCguN6FaPrrnrp_FxIuFAWOLgrRNoYy3Vw-ihHTiveGltQ4xKGHlZ_8XC7PKChpUSO1l5TKMC7hHZBzHgKEQHFVemxx_gNMc4MjxSlr9pW80FTStzAIXB_tJGxWdi8tzrSwoK4iCTsvNjyhgeeFQ_OJwqJux_DAl15Htu2O7lckVqzxNFZtFdK70kzU
Text:
  Availability: 1
  Value: <anid>AN0188049029;etr01aug.25;2025Sep22.01:49;v2.2.500</anid> <title id="AN0188049029-1">Validating student AI competency self-efficacy (SAICS) scale and its framework </title> <p>Nurturing student artificial intelligence (AI) competency is crucial in the future of K-12 education. Students with strong AI competency should be able to ethically, safely, healthily, and productively integrate AI into their learning. Research on student AI competency is still in its infancy, primarily focusing on theoretical and professional discussions, along with qualitative investigations. This two-stage study aims to propose an AI competency framework for students and confirm the reliability and validity of its scale—student AI competency self-efficacy (SAICS)—in K-12 education. In stage 1, we used a three-round Delphi study to propose the framework and its scale. The framework has eight dimensions: interdisciplinary learning with AI, assessment with AI, decision-making with AI, data, ethics and AI, designing AI, multimedia creation with AI, human-centric learning, and confidence with AI. Each dimension contains four items. In stage 2, we involved 448 students to validate the scale using confirmatory factor analysis and model comparisons. The analyses showed that the scale is consistent across male and female students. The SAICS scale comprises 32 items and addresses eight dimensions of AI competency. Researchers can use the framework and SAICS to design their interventions and correlational research associated with student AI competency. Teachers can use them to develop learning outcomes for AI-based learning activities, and policymakers can use them to establish national AI standards.</p> <p>Keywords: AI competency; Scale development; K-12 education; AI literacy; Delphi study; Education Curriculum and Pedagogy Specialist Studies In Education</p> <hd id="AN0188049029-2">Introduction</hd> <p>Artificial intelligence (AI) and education for all, moving from the professional to the mainstream, is a global initiative (Chiu, [<reflink idref="bib4" id="ref1">4</reflink>]; Touretzky et al., [<reflink idref="bib28" id="ref2">28</reflink>]). Understanding AI, including how it operates and the societal benefits it may provide, is the primary step toward a safe and healthy life in an AI-driven society. Most of us will live, learn, and work with AI, such as ChatGPT and Copilot, in the future at home, in schools, and in the workplace. Therefore, this education prepares youth for a future workforce that will require a greater level of AI literacy. A growing body of research suggests what AI literacy is and how to measure it (Chiu, [<reflink idref="bib6" id="ref3">6</reflink>]; Chiu et al., [<reflink idref="bib9" id="ref4">9</reflink>]; Kong et al., [<reflink idref="bib20" id="ref5">20</reflink>]; Laupichler et al., [<reflink idref="bib21" id="ref6">21</reflink>]; Long & Magerko, [<reflink idref="bib24" id="ref7">24</reflink>]; Zhou et al., [<reflink idref="bib39" id="ref8">39</reflink>]). Their findings have contributed to AI and education research and practice by providing insights into the expected learning outcomes and potential content for curricula. These studies focus on more mature learners or users, such as those in higher education (Kong et al., [<reflink idref="bib20" id="ref9">20</reflink>]; Long & Magerko, [<reflink idref="bib24" id="ref10">24</reflink>]) and non-AI experts (Laupichler et al., [<reflink idref="bib21" id="ref11">21</reflink>]). Their results were driven by the perspectives of university teaching staff, the opinions of researchers, and the literature review and might not be suitable for younger children, such as those in grades K–12, due to their developmental stage. Thus, more studies on defining and measuring AI for younger students from other participant perspectives, such as teachers, are needed.</p> <p>AI literacy places greater emphasis on skills, but AI competency acknowledges a wider range of knowledge, capabilities, and attitudes (Chiu et al., [<reflink idref="bib9" id="ref12">9</reflink>]; UNESCO, [<reflink idref="bib29" id="ref13">29</reflink>]). As stated by Falloon ([<reflink idref="bib13" id="ref14">13</reflink>]), the concept of competency needs ongoing adaptation to account for the progressive advancements in digital systems. This necessitates that individuals assess their present capabilities and needs and, if needed, review their learning to adapt to the dynamic digital landscape, possibilities, and challenges that are propelled by advancements in emerging technologies. Compared to non-AI technology, AI is a more disruptive and ever-evolving technology; engaging in updating its technical knowledge is required for student AI competency (Chiu, [<reflink idref="bib5" id="ref15">5</reflink>]; Chiu et al., [<reflink idref="bib9" id="ref16">9</reflink>]; Falloon, [<reflink idref="bib13" id="ref17">13</reflink>]; Liu et al., [<reflink idref="bib23" id="ref18">23</reflink>]; Zhou et al., [<reflink idref="bib39" id="ref19">39</reflink>]). Moreover, the current related studies mix the terms AI literacy, AI self-efficacy, and AI competency. More studies are needed to clarify these terms.</p> <p>This study aims to propose an AI competency framework for K-12 students based on teacher perspectives, as well as to suggest and validate its scales, i.e., the student AI competency self-efficacy (SAICS) scale. The findings are expected to assist researchers in determining the effectiveness of empirical studies aimed at improving student AI competency, as well as teachers in assessing their students' competency to apply AI in learning. Policymakers can use our findings to establish their national standards for AI competency. Overall, the findings would help us understand the key components of K-12 student AI competency.</p> <hd id="AN0188049029-3">Conceptual background</hd> <p></p> <hd id="AN0188049029-4">Measuring AI literacy</hd> <p>AI literacy is defined as the ability to critically assess AI technology, interact and communicate with AI in an efficient manner, and use AI as a tool at home, at work, and online (Long & Magerko, [<reflink idref="bib24" id="ref20">24</reflink>]). Digital literacy refers to the capacity to utilize and assess digital information, tools, and services appropriately and to incorporate this knowledge into lifelong learning processes (Bawden, [<reflink idref="bib3" id="ref21">3</reflink>]) and is viewed as a prerequisite of AI literacy (Long & Magerko, [<reflink idref="bib24" id="ref22">24</reflink>]). Individuals should grasp how to utilize computers to make sense of AI. Moreover, computational literacy is defined as the ability to explore, communicate, and express ideas through code (diSessa, [<reflink idref="bib12" id="ref23">12</reflink>]) and is not always required for AI literacy (Chiu, [<reflink idref="bib4" id="ref24">4</reflink>]; Long & Magerko, [<reflink idref="bib24" id="ref25">24</reflink>]). Coding skills are necessary for developing but not using AI. Users could effectively, ethically, and safely use AI in their daily lives without mastering coding skills. In addition, data literacy refers to the capacity to comprehend, interpret, engage with, evaluate, and debate data as part of a larger process of inquiry into the world (D'Ignazio, [<reflink idref="bib10" id="ref26">10</reflink>]) and is overlapped with AI literacy (Chiu, [<reflink idref="bib4" id="ref27">4</reflink>], [<reflink idref="bib6" id="ref28">6</reflink>]; Long & Magerko, [<reflink idref="bib24" id="ref29">24</reflink>]). It is because the key idea of AI is learning from data. Overall, having a clear idea of what "AI literacy" is can help with mainstream education and skill assessments related to AI.</p> <p>Literature suggests frameworks to capture constructs or items related to AI literacy and inform scholars to develop scales for AI education research. Long and Magerko ([<reflink idref="bib24" id="ref30">24</reflink>]) used an interdisciplinary literature review approach to present one of the first comprehensive frameworks, consisting of five major dimensions and 17 areas for AI literacy assessment. The dimensions are: What is AI? What can AI do? How does AI work? How should AI be used? How do people perceive AI? The suggested areas are technical and too many, which are more appropriate for students in higher education or employees in the workplace. For example, the area "understanding that agents are programmable" is not for younger students. Laupichler et al. ([<reflink idref="bib21" id="ref31">21</reflink>]) used a Delphi study to propose 38 self-reported items as an AI literacy scale. The scale has no dimensions, i.e., the whole set of items should be used for research or practice. This study is similar to Long and Magerko's ([<reflink idref="bib24" id="ref32">24</reflink>]), and its items are too specific and technical. Kong et al. ([<reflink idref="bib20" id="ref33">20</reflink>]) developed and used a 10-item AI literacy test (perceptions of their own levels of AI literacy) and a 17-item AI Empowerment Survey (sense of empowerment to deal with AI) to assess student AI learning. It did not present a whole set of instruments. Overall, the findings of these major studies have different views on how to measure AI literacy. All of them used self-reported items to assess participants' AI literacy. This self-reported approach was used in many related studies (Lin et al., [<reflink idref="bib22" id="ref34">22</reflink>]; Shih et al., [<reflink idref="bib26" id="ref35">26</reflink>]).</p> <p>However, a self-reported approach can only examine participants' perceptions of using and evaluating AI (i.e., AI self-efficacy) but not their abilities and skills (i.e., AI literacy). AI literacy should be measured by tests or examinations similar to international literacy assessment programs such as the Program for International Student Assessment and Trends in International Mathematics and Science Studies. This argument is also strongly supported by the study of the AI self-efficacy scale done by Wang and Chuang ([<reflink idref="bib30" id="ref36">30</reflink>]). The study developed and validated the self-efficacy scale consisting of four dimensions—assistance, anthropomorphic interaction, comfort with AI, and technological skills—using 317 adults. The format of the items (i.e., self-reported feeling, thinking) is similar to that in the studies discussed in the previous paragraph. Moreover, self-reported approaches have been studied in the majority of AI education studies on anxiety and attitude toward AI, as they focus more on how individuals feel and think. For example, AI attitude and AI for social good (Chiu et al., [<reflink idref="bib9" id="ref37">9</reflink>]), general attitudes towards AI (Schepman & Rodway, [<reflink idref="bib25" id="ref38">25</reflink>]), and AI anxiety (Wang & Wang, [<reflink idref="bib31" id="ref39">31</reflink>]). Given that we did not know what to include in AI education for the public or the mainstream, this is the most sensible course of action. Overall, both AI self-efficacy and AI literacy should be included in a more comprehensive assessment of student learning.</p> <hd id="AN0188049029-5">AI literacy and competency for school students</hd> <p>Self-efficacy concerns beliefs about their ability to know and apply; AI literacy concerns knowing, applying, and evaluating; and AI competency is about applying and evaluating the knowledge in a beneficial way (Falloon, [<reflink idref="bib13" id="ref40">13</reflink>]; Wang & Chuang, [<reflink idref="bib30" id="ref41">30</reflink>]; Zhou et al., [<reflink idref="bib39" id="ref42">39</reflink>]). These three concepts influence how to measure individuals' AI learning and understanding. In AI education empirical research, using self-efficacy is more popular. Two plausible explanations are (i) lack of reliable and validated instruments for AI literacy and competency and (ii) literacy and competency assessment depends on a research or teaching context (e.g., learning content in an intervention should be aligned with the tests). Overall, using self-efficacy in assessing student AI competency would be more effective.</p> <p>As we discussed, digital literacy is a prerequisite for digital competency (Long & Magerko, [<reflink idref="bib24" id="ref43">24</reflink>]). The move from AI literacy to competency is supported by various United Nations Educational, Scientific, and Cultural Organization (UNESCO) reports and Falloon's ([<reflink idref="bib13" id="ref44">13</reflink>]) study on digital competency. Falloon ([<reflink idref="bib13" id="ref45">13</reflink>]) clearly distinguished the difference between digital literacy and competency. He encouraged scholars and educators to have more comprehensive and broad-based conceptualizations than the current technical and literacy frameworks. It is because young people require ever-increasingly complex knowledge and abilities to operate in a variety of digitally mediated environments in an ethical, responsible, safe, and productive manner (Falloon, [<reflink idref="bib13" id="ref46">13</reflink>]). AI competency clearly involves more than understanding what AI is, how AI works, how to use AI, and what impact AI brings (Chiu et al., [<reflink idref="bib9" id="ref47">9</reflink>]). Given the fast and disruptive evolution of AI technology, this mindset is vital for AI learning. AI-literate users are able to leverage classic AI technologies in their daily lives in an effective and ethical manner, whereas AI-competent users are at ease with emerging AI technologies for specific purposes (Chiu, [<reflink idref="bib5" id="ref48">5</reflink>]; Chiu & Sanusi, [<reflink idref="bib8" id="ref49">8</reflink>]; Falloon, [<reflink idref="bib13" id="ref50">13</reflink>]; Liu et al., [<reflink idref="bib23" id="ref51">23</reflink>]; Zhou et al., [<reflink idref="bib39" id="ref52">39</reflink>]). For example, AI-literate students understand AI knowledge well in general, while AI-competent students can effectively use AI in their learning (project-based essay writing). To integrate AI into classrooms, it is advisable to embrace the concept of AI competency. The United Nations Educational, Scientific, and Cultural Organization (UNESCO) drafted an AI competency framework for K–12 students: "the knowledge, skills, and attitudes students should acquire to understand and actively engage with AI in a safe and ethical manner in school and beyond" (UNESCO, [<reflink idref="bib29" id="ref53">29</reflink>]). The framework consists of four primary components: a human-centric mindset, the ethics of AI, AI techniques and applications, and AI system design. Each component has three levels: understand, apply, and create. This framework is more about AI knowledge and understanding, focusing less on students' competency using AI in their learning. Chiu et al., ([<reflink idref="bib9" id="ref54">9</reflink>], 2025) took a different perspective and advocated that the definitions of AI competency should emphasize a specific context, i.e., K-12 students should be competent to use AI in their activities.</p> <hd id="AN0188049029-6">Teacher perspective for assessing younger student AI learning</hd> <p>AI competency is frequently employed to define the competency of non-engineering rather than engineering students (Laupichler et al., [<reflink idref="bib21" id="ref55">21</reflink>]; Long & Magerko, [<reflink idref="bib24" id="ref56">24</reflink>]; Wang & Chuang, [<reflink idref="bib30" id="ref57">30</reflink>]). Without obtaining formal professional training in AI, non-engineers are those who employ AI systems rather than build them. Non-engineering includes school students, who utilize AI applications for learning purposes. Therefore, AI and education for students from general education streams is therefore a global educational initiative (Chiu, [<reflink idref="bib5" id="ref58">5</reflink>]; UNESCO, [<reflink idref="bib29" id="ref59">29</reflink>]; Yau et al., [<reflink idref="bib36" id="ref60">36</reflink>]). Related studies on designing and creating AI learning activities for K–12 students were conducted worldwide in Europe, Korea, Hong Kong, China, and the United States. Teacher viewpoints were extensively included in the design of a number of these studies (Chiu, [<reflink idref="bib4" id="ref61">4</reflink>]; Kim et al., [<reflink idref="bib18" id="ref62">18</reflink>]; Williams et al., [<reflink idref="bib33" id="ref63">33</reflink>]; Yau et al., [<reflink idref="bib36" id="ref64">36</reflink>]). In comparison to academics, school teachers have a deeper understanding of younger students' learning and needs. However, most current studies on AI learning assessment are recommended by academics and focus on more senior learners or users. These may neglect teacher perspectives and sense-making (Chiu & Chai, [<reflink idref="bib7" id="ref65">7</reflink>]); however, teachers' beliefs and views will decide what AI learning and assessment look like. Hence, it is important to get the views of school teachers when designing assessment approaches for younger students (K–12 in this study).</p> <hd id="AN0188049029-7">Research gaps and significance of this study</hd> <p>Our literature review provides three major arguments to support the significance of this study. First, measuring AI learning should go beyond skills and abilities (literacy) due to its rapidly emerging and disruptive nature (Chiu, [<reflink idref="bib5" id="ref66">5</reflink>], [<reflink idref="bib6" id="ref67">6</reflink>]; Falloon, [<reflink idref="bib13" id="ref68">13</reflink>]; Liu et al., [<reflink idref="bib23" id="ref69">23</reflink>]). We suggest including more AI-based learning activities in the framework. Second, most current studies were designed for more mature participants, such as students in higher education and employees in the workplace, and took AI professional or academic perspectives (Laupichler et al., [<reflink idref="bib21" id="ref70">21</reflink>]; Long & Magerko, [<reflink idref="bib24" id="ref71">24</reflink>]; UNESCO, [<reflink idref="bib29" id="ref72">29</reflink>]; Wang & Chuang, [<reflink idref="bib30" id="ref73">30</reflink>]). Their findings were not appropriate for younger students and suggested that teachers' perspectives should be included in the research. Third, the three terms used for AI self-efficacy, literacy, and competency are mixed in the literature (Kong et al., [<reflink idref="bib20" id="ref74">20</reflink>]; Laupichler et al., [<reflink idref="bib21" id="ref75">21</reflink>]). Accordingly, this study significantly contributes to AI and education by clarifying AI competency by suggesting its assessment. Researchers and educators could use the findings to design their studies and evaluate their student learning in AI, respectively, to advance AI education research.</p> <hd id="AN0188049029-8">This study and method</hd> <p></p> <hd id="AN0188049029-9">Research goal</hd> <p>This study is to propose an AI competency framework for K-12 students from the point of view of school teachers and validate its measurement items. More specifically, we suggest the key components for student AI competency, followed by developing and validating a student artificial intelligence competency self-efficacy (SAICS) scale. Accordingly, the three main research questions are:</p> <p></p> <ulist> <item> RQ1:What are the dimensions and their self-reported items for student AI competency from a K-12 teacher perspective?</item> <p></p> <item> RQ2:Is the SAICS scale quantitatively validated?</item> <p></p> <item> RQ3:Are there any significant differences in the SAICS scale based on student gender?</item> </ulist> <p>We used a two-stage approach to answer the three RQs. In stage 1, we used a Delphi study to identify the key component of student AI competency (RQ1) and develop items for SAICS. This stage is to construct a set of student AI competencies. The competencies were rated for their importance by a group of experienced teacher panelists. In stage 2, we used quantitative analyses to validate the SAICS scale (RQ2 and RQ3). The items measuring the competency were validated using confirmatory factor analysis (CFA).</p> <hd id="AN0188049029-10">Stage 1: a Delphi study</hd> <p>A Delphi study is a widely recognized methodology used to address research questions by identifying a consensus viewpoint among a group of specialists. It facilitates participant introspection, enabling them to refine and review their opinions and ideas considering the anonymous perspectives expressed by others (Barrett & Heale, [<reflink idref="bib2" id="ref76">2</reflink>]). Three Delphi rounds are sufficient to establish an equilibrium where further rounds do not significantly alter the findings (Laupichler et al., [<reflink idref="bib21" id="ref77">21</reflink>]; Teixeira et al., [<reflink idref="bib27" id="ref78">27</reflink>]). Accordingly, we used a three-round process to answer the three research questions, which is supported by a similar study by Laupichler et al. ([<reflink idref="bib21" id="ref79">21</reflink>]).</p> <p>Every Delphi study has its own set of guidelines for attaining consensus on every topic (Barrett & Heale, [<reflink idref="bib2" id="ref80">2</reflink>]; Keeney et al., [<reflink idref="bib17" id="ref81">17</reflink>]). For example, it is easier to get all survey respondents to agree on three items than it is to get them to agree on twenty items. The process of developing agreement levels is intrinsically subjective and requires consideration of the questions posed, the range of possible options, and the size of the panel. According to the review study of Diamond et al. ([<reflink idref="bib11" id="ref82">11</reflink>]), the agreement levels ranged from approximately 50–95%. This study used 75% as the agreement level, similar to the study of Teixeira et al. ([<reflink idref="bib27" id="ref83">27</reflink>]). In each round, the participants gave their feedback on the items for AI literacy, self-efficacy, and self-reflective mindset. They may suggest removing or combining some of the items or dimensions and adding new items or dimensions. Moreover, the determination of sample size for Delphi research lacks unanimity; hence, no minimum panel size is established (Barrett & Heale, [<reflink idref="bib2" id="ref84">2</reflink>]; Chiu, [<reflink idref="bib5" id="ref85">5</reflink>]; Jorm, [<reflink idref="bib16" id="ref86">16</reflink>]). Prior studies employed ten to thirty-four experts as participants (Ahmadi et al., [<reflink idref="bib1" id="ref87">1</reflink>]; Barrett & Heale, [<reflink idref="bib2" id="ref88">2</reflink>]; Teixeira et al., [<reflink idref="bib27" id="ref89">27</reflink>]). The expert participants of this study were K–12 AI education professionals; that is, they possess extensive knowledge and expertise in developing AI learning activities for K–12 levels. To ensure that the participants are knowledgeable, the selection criteria are holding undergraduate or graduate degrees in computer science, technology, STEM, or engineering; (ii) having at least 3 years of formal experience teaching AI (classroom teaching); and (iii) having completed at least twelve hours of professional development on AI education. With the understanding that K–12 AI education is still in its infancy and that most related research started in 2019, having taught for 3 years is considered experienced.</p> <p>This Delphi study was conducted in Hong Kong, which started the first AI K–12 education project in August 2019. In Hong Kong, an official AI curriculum for seventh through ninth graders was developed in September 2023. Hong Kong schools taught AI using the learning materials and evaluation exam.</p> <p>We invited 40 teachers from a professional development program on AI for education to take part, but only 29 teachers met the requirements and agreed. They all used the official curriculum resources for their AI classes. Twenty-two of them were male, which represented the gender ratios of AI teachers (representative sampling). The average AI teaching experience is 2.8 years old, and the average age is 31 years old. The main research team included four AI and education scholars from different regions—Hong Kong, Africa, Finland, and Turkey—and three teacher-researchers on AI education.</p> <p>This stage intentionally developed key components of K-12 student competency and their items. First, to develop the initial materials for the Delphi Round 1, the research team compiled the items by reviewing the related studies. For the first set, the reviewed studies included the official curriculum and the studies of UNESCO ([<reflink idref="bib29" id="ref90">29</reflink>]), Chiu et al. ([<reflink idref="bib9" id="ref91">9</reflink>]), Long and Magerko ([<reflink idref="bib24" id="ref92">24</reflink>]), Wang and Chuang ([<reflink idref="bib30" id="ref93">30</reflink>]), and Wu and Chen ([<reflink idref="bib34" id="ref94">34</reflink>]).</p> <p>Figure 1 shows the process of our Delphi study. In Round 1, we distributed the lists to the teacher participants. The teachers gave qualitative comments on the dimensions and items. They also assessed if the items reflect student AI learning at the K-12 level. The participants were asked to revise, remove, combine, and add new items. After each round, the research team consolidated the participants' feedback to revise the dimensions and items. Items with substantial changes are considered new items in the next round. The items that reached the agreement level were marked as final. In Rounds 2 and 3, the teachers received the most updated lists of activities, together with information on the agreement level. They continued to give comments on the dimensions and items.</p> <p>Graph: Fig. 1 The process of our Delphi study</p> <hd id="AN0188049029-11">Stage 2: quantitative analyses to validate SAICS scale</hd> <p>Four hundred and forty-eight student participants from six Hong Kong secondary schools (Grades 7–12) from various socioeconomic backgrounds. They aged from 12 to 17 years old (M = 14.8, SD = 2.8; male: 222 and female: 226). The schools were chosen from three school-university partnership programs overseen by the corresponding author. The items used in this stage are the results from the Delphi student and were assessed on a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree).</p> <p>To respond to RQ2, the measurement model was evaluated using traditional fit indices and standards, such as Kline ([<reflink idref="bib19" id="ref95">19</reflink>]), which include the chi-square tests of fit, the comparative fit index (CFI), and the root mean square error of approximation (RMSEA) with a 90% confidence interval. To address RQ3, we used changes in CFI as indicated for comparing models to see if the TAICS scale is consistent across gender groupings (Zhan, [<reflink idref="bib37" id="ref96">37</reflink>]).</p> <hd id="AN0188049029-12">Results</hd> <p></p> <hd id="AN0188049029-13">Stage 1</hd> <p></p> <hd id="AN0188049029-14">Initial materials</hd> <p>Table 1 shows the draft for the research team's discussions. The draft was developed based on Chiu et al. ([<reflink idref="bib9" id="ref97">9</reflink>]), Long and Magerko ([<reflink idref="bib24" id="ref98">24</reflink>]), UNESCO ([<reflink idref="bib29" id="ref99">29</reflink>]), Wang and Chuang ([<reflink idref="bib30" id="ref100">30</reflink>]), and Wu and Chen ([<reflink idref="bib34" id="ref101">34</reflink>]). The research team wrote the initial draft by modifying the four dimensions and 22 items of Wang and Chuang ([<reflink idref="bib30" id="ref102">30</reflink>]) to develop initial material. We revised the wording to fit the younger student and made the sentence structure more consistent. For example, most of the revised items started with "I..." or "It is...". We changed "assistance" into "support for learning" and "anthropomorphic interaction" into "human-like interaction". We also added perceived ease of use (Wu & Chen, [<reflink idref="bib34" id="ref103">34</reflink>]) because they are strongly related to self-efficacy. Moreover, we modified eight items from the study of Grant et al. ([<reflink idref="bib14" id="ref104">14</reflink>]) to develop the items for assessing self-reflective mindset for the AI questionnaire. We also add five more dimensions, such as assessment with AI, creation with AI, critical thinking, perceived ease to use as well as data, ethics, and AI. The initial materials have 10 dimensions and 50 items for Delphi Round 1, see Table 2.</p> <p>Table 1 The first draft for the panel's discussion</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Items</p></th><th align="left"><p>Dimension</p></th></tr></thead><tbody><tr><td align="left"><p><italic>First draft for the panel's discussion</italic></p></td><td align="left" /></tr><tr><td align="left"><p>• Some AI technologies/products make learning easier</p><p>• I find that AI technologies/products are helpful for learning</p><p>• AI technologies/products are good aids for learning</p><p>• Using AI technologies/products makes learning more interesting</p><p>• I'm confident in my ability to learn simple programming of AI technologies/products if I were provided the necessary tools</p><p>• AI technologies/products help me to save a lot of time</p><p>• I find it easy to get AI technologies/products to do what I want them to do</p></td><td align="left"><p>Assistance</p></td></tr><tr><td align="left"><p>• I think the interactive process of AI technologies/products is very vivid, just like chatting with a real person</p><p>• I think the way that AI technologies/products express content when interacting is unique, just like a real person</p><p>• I think there is no difference between the dialogue method of AI technologies/products compared with the dialogue with real people</p><p>• I think the tone of AI technologies/products when interacting is the same as that of real people</p><p>• I feel that the way of expression of AI technologies/products in the interactive text is the same as that of real people</p></td><td align="left"><p>Anthropomorphic interaction</p></td></tr><tr><td align="left"><p>• When interacting with AI technologies/products, I feel very calm</p><p>• When interacting with AI technologies/products, I find it easy</p><p>• When interacting with AI technologies/products, I feel comfortable in my heart</p><p>• When interacting with AI technologies/products, I feel very peaceful</p><p>• When interacting with AI technologies/products, I feel very relaxed</p><p>• I can happily interact with AI technologies/products smoothly</p></td><td align="left"><p>Comfort with AI</p></td></tr><tr><td align="left"><p>• When using AI technologies/products, I am not worried that I might press the wrong button and cause risks</p><p>• When using AI technologies/products, I am not worried that I might press the wrong button and damage it</p><p>• When using AI technologies/products, there is nothing that I do not know why</p><p>• AI technologies/products jargon does not baffle me</p></td><td align="left"><p>Technological skills</p></td></tr><tr><td align="left" colspan="2"><p><italic>Self-reflective mindset</italic></p></td></tr><tr><td align="left"><p>• I frequently examine my feelings</p><p>• I frequently take time to reflect on my thoughts</p><p>• I often think about the way I feel about things</p><p>• It is important for me to evaluate the things that I do</p><p>• I am very interested in examining what I think about</p><p>• It is important to me to try to understand what my feelings mean</p><p>• I have a definite need to understand the way that my mind works</p><p>• It is important to me to be able to understand how my thoughts arise</p></td><td align="left"><p>Engagement in self-reflection</p><p>Needs for self-reflection</p></td></tr></tbody></table> </ephtml> </p> <p>Table 2 The initial materials for Round 1</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Items</p></th><th align="left"><p>Initial list</p></th></tr></thead><tbody><tr><td align="left"><p>• I know how to use AI technology to support my learning</p><p>• I know how to use AI technology for self-reflection in learning</p><p>• I know how to use AI technology to solve my learning difficulties</p><p>• I know how to choose the right AI technology to support my learning</p><p>• I find learning with AI technology more effective than without</p><p>• I find that AI technology is helpful for learning</p><p>• I believe AI technology can help me learn a topic by myself</p><p>• I believe AI technology can help me find additional questions for my learning</p></td><td align="left"><p>Support for learning</p></td></tr><tr><td align="left"><p>• I can use AI to check whether my solutions are correct</p><p>• I can use AI to generate questions for further learning</p><p>• I can use AI to get feedback to write essays</p></td><td align="left"><p>Assessment with AI</p></td></tr><tr><td align="left"><p>• I can use AI to create images for my learning (e.g., presentations, communications, assignments)</p><p>• I can use AI to create videos for my learning</p><p>• I can use AI to create audio for my learning</p></td><td align="left"><p>Creation with AI</p></td></tr><tr><td align="left"><p>• I have good critical thinking to learn with AI</p><p>• I am able tell whether AI can benefit my learning</p><p>• I could judge if an essay generated by AI is correct for my learning</p><p>• I can judge if an image/video essay generated by AI is correct for my learning</p></td><td align="left"><p>Critical thinking</p></td></tr><tr><td align="left"><p>• I think the interactive process of AI technology is very vivid, just like chatting with a real person</p><p>• I think the way that AI technology expresses content when interacting is unique, just like a real person</p><p>• I think there is no difference between the method of AI technology compared with the dialogue with real people</p><p>• I think the tone of AI technology when interacting is the same as that of real people</p><p>• I feel that the way of expression of AI technology in the interactive text is the same as that of real people</p></td><td align="left"><p>Human-like interaction</p></td></tr><tr><td align="left"><p>• I feel comfortable using AI technology in learning</p><p>• I feel confident using AI technology in learning</p><p>• I feel relaxed using AI technology in learning</p><p>• I feel calm using AI technology in learning</p><p>• I feel peaceful using AI technology in learning</p><p>• I can happily interact with AI technology smoothly</p></td><td align="left"><p>Comfort with AI</p></td></tr><tr><td align="left"><p>• When using AI technology, I am not worried that I might press the wrong button and cause risks</p><p>• When using AI technology, I am not worried that I might press the wrong button and damage it</p><p>• When using AI technology, there is nothing that I do not know why</p><p>• AI technologies/products jargon does not baffle me</p></td><td align="left"><p>Technological skills</p></td></tr><tr><td align="left"><p>• I find it easy to get AI technology to do what I want it to do</p><p>• I find AI technology helps me save a lot of time</p><p>• I find AI technology easy to use for learning</p><p>• I find it easy to learn AI technology</p><p>• I believe it is easy to become proficient in using AI for learning</p><p>• I believe the interaction with AI technology is clear and understandable</p></td><td align="left"><p>Perceived ease to use</p></td></tr><tr><td align="left"><p>• I know how to protect my personal data when using AI</p><p>• I understand that AI technology sometimes gives a biased output</p><p>• I understand that AI technology sometimes generates incorrect output</p><p>• I understand that AI technology can generate fake information</p></td><td align="left"><p>Data, ethics, and AI</p></td></tr><tr><td align="left"><p>• I frequently examine AI knowledge</p><p>• I frequently take time to reflect on my AI knowledge</p><p>• I often think about the way I use AI in learning</p><p>• It is important for me to evaluate the AI technology that I use</p><p>• I am very interested in examining how I use AI technology in learning</p><p>• It is important to me to try to understand the advantages and risks of the AI technology I use</p><p>• I have a definite need to understand how AI technology processes data</p></td><td align="left"><p>Engagement in self-reflection</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0188049029-15">Three Delphi rounds</hd> <p>Three Delphi rounds were completed by all the participants. Table 3 shows the results of Rounds 1, 2, and 3. Eight dimensions were revised and finalized from 10 dimensions in the initial lists. The labels of two dimensions remain the same, and they are "Assessment with AI (ASAI)" and "Data, ethics, and AI (DEAI)". Five of them were renamed: "Interdisciplinary learning with AI (ILAI)" instead of "Support for learning"; "Multimedia creation with AI (MCAI)" instead of "Creation with AI'; "Decision making with AI (DMAI)" instead of "Critical thinking"; "Human-centric learning (HCLG)" instead of "Human-like interaction"; and Designing AI (DGAI)" instead of "Technological skills". Three of them were integrated into one dimension: "Confidence with AI (CFAI)" instead of "Comfort with AI", "Perceived ease to use", and "Engagement in self-reflection". Each of the dimensions has 4 items. Therefore, the final list has eight dimensions and 32 items (see the ✓ in Table 3).</p> <p>Table 3 The first list and the results of Round 1, 2, and 3 in the self-efficacy questionnaire (self-reported items)</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" colspan="5"><p>Items for student AI competency</p></th></tr><tr><th align="left"><p>Items</p></th><th align="left"><p>Initial list</p></th><th align="left"><p>Round 1</p></th><th align="left"><p>Round 2</p></th><th align="left"><p>Round 3</p></th></tr></thead><tbody><tr><td align="left"><p>• I know how to use AI technology to support my learning</p></td><td align="left" rowspan="12"><p>Interdisciplinary learning with AI (Support for learning)</p></td><td align="left"><p>@</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• I know how to use AI technology for self-reflection in learning</p></td><td align="left"><p>@</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• I know how to use AI technology to solve my learning difficulties</p></td><td align="left"><p>@</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• I know how to choose the right AI technology to support my learning</p></td><td align="left"><p>@</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• I find learning with AI technology more effective than without</p></td><td align="left" /><td align="left"><p>@</p></td><td align="left" /></tr><tr><td align="left"><p>• I find that AI technology is helpful for learning</p></td><td align="left"><p>@</p></td><td align="left"><p>@</p></td><td align="left" /></tr><tr><td align="left"><p>• I believe AI technology can help me learn a topic by myself</p></td><td align="left" /><td align="left"><p>@</p></td><td align="left" /></tr><tr><td align="left"><p>• I believe AI technology can help me find additional questions for my learning</p></td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>✓ I can use AI to get different alternative information to solve real-life problems better</p></td><td align="left" /><td align="left"><p> + </p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can use AI to gain knowledge in a discipline I am unfamiliar with</p></td><td align="left" /><td align="left"><p> + </p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can use AI to conduct inquiry-based learning in other disciplines</p></td><td align="left" /><td align="left"><p> + </p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can use AI to find relevant information outside the classroom curriculum quickly and efficiently</p></td><td align="left" /><td align="left"><p> + </p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can use AI to check whether my solutions are correct</p></td><td align="left" rowspan="5"><p>Assessment with AI</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can use AI to generate questions for further learning</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>• I can use AI to get feedback to write essays</p></td><td align="left"><p>^</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>✓ I can use AI to get feedback to improve my ideas</p></td><td align="left"><p>^</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can use AI to get ideas to complete my learning tasks</p></td><td align="left" /><td align="left"><p> + </p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can use AI to create images for my learning (e.g., presentations, communications, assignments)</p></td><td align="left" rowspan="4"><p>Multimedia creation with AI (Creation with AI)</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can use AI to create videos for my learning</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can use AI to create audio for my learning</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can use AI to create slides for my learning</p></td><td align="left"><p> + </p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>• I have good critical thinking to learn with AI</p></td><td align="left" rowspan="6"><p>Decision making with AI (Critical thinking)</p></td><td align="left"><p>^</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• I am able tell whether AI can benefit my learning</p></td><td align="left"><p>^</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can ask for more resources based on the recommendations by AI for my learning</p></td><td align="left"><p>^</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can judge if resources (e.g., websites, videos) recommended by AI are appropriate for my learning</p></td><td align="left"><p>^</p></td><td align="left" /><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can judge if an essay generated by AI is correct for my learning</p></td><td align="left"><p>#</p></td><td align="left" /><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can judge if an image/video essay generated by AI is correct for my learning</p></td><td align="left"><p>#</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• I think the interactive process of AI technology is very vivid, just like chatting with a real person</p></td><td align="left" rowspan="9"><p>Human-centric learning (Human-like interaction)</p></td><td align="left"><p>@</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• I think the way that AI technology expresses content when interacting is unique, just like a real person</p></td><td align="left"><p>@</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• I think there is no difference between the method of AI technology compared with the dialogue with real people</p></td><td align="left"><p>@</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• I think the tone of AI technology when interacting is the same as that of real people</p></td><td align="left"><p>@</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• I feel that the way of expression of AI technology in the interactive text is the same as that of real people</p></td><td align="left"><p>@</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>✓ I believe that AI is a tool to enhance my learning experience, rather than just a means to finish assignments</p></td><td align="left"><p> + </p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can leverage AI to boost my understanding and skills, not merely to complete tasks</p></td><td align="left"><p> + </p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can ensure my essays are original when co-writing with AI</p></td><td align="left" /><td align="left"><p> + </p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I can collaborate with AI to create slides that are my ideas</p></td><td align="left" /><td align="left"><p> + </p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>• When using AI technology, I am not worried that I might press the wrong button and cause risks</p></td><td align="left" rowspan="8"><p>Designing AI (Technological skills)</p></td><td align="left"><p>@</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• When using AI technology, I am not worried that I might press the wrong button and damage it</p></td><td align="left"><p>@</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• When using AI technology, there is nothing that I do not know why</p></td><td align="left"><p>@</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• AI technologies/products jargon does not baffle me</p></td><td align="left"><p>@</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>✓ I know how to collect data for designing machine learning</p></td><td align="left"><p> + </p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I know how to create a machine learning application</p></td><td align="left"><p> + </p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I know how to categorize data for designing machine learning</p></td><td align="left"><p> + </p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I know how to evaluate my created machine learning</p></td><td align="left"><p> + </p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I know how to protect my personal data when using AI</p></td><td align="left" rowspan="5"><p>Data, ethics, and AI</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I understand that AI technology sometimes gives a biased output</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I understand that AI technology sometimes generates incorrect output</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>• I understand that AI technology can generate fake information</p></td><td align="left"><p>^</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>✓ I understand that I should not create fake information</p></td><td align="left"><p>^</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I feel comfortable using AI technology in learning</p></td><td align="left" rowspan="8"><p>Confidence with AI (Comfort with AI)</p></td><td align="left"><p>^</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I feel confident using AI technology in learning</p></td><td align="left"><p>^</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>• I feel relaxed using AI technology in learning</p></td><td align="left"><p>^</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• I feel calm using AI technology in learning</p></td><td align="left"><p>^</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• I feel peaceful using AI technology in learning</p></td><td align="left"><p>–</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>• I can happily interact with AI technology smoothly</p></td><td align="left"><p>–</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>✓ I am confident I will find new AI technology for learning</p></td><td align="left"><p>^</p></td><td align="left"><p> + </p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>✓ I find it easy to learn AI technology</p></td><td align="left"><p>^</p></td><td align="left"><p>#</p></td><td align="left"><p>#</p></td></tr><tr><td align="left"><p>• I find it easy to get AI technology to do what I want it to do</p></td><td align="left" rowspan="6"><p>(Perceived ease to use)</p></td><td align="left"><p>^</p></td><td align="left" rowspan="6" /><td align="left" rowspan="6" /></tr><tr><td align="left"><p>• I find AI technology helps me save a lot of time</p></td><td align="left"><p>–</p></td></tr><tr><td align="left"><p>• I find AI technology easy to use for learning</p></td><td align="left"><p>–</p></td></tr><tr><td align="left"><p>• I find it easy to learn AI technology</p></td><td align="left"><p>^</p></td></tr><tr><td align="left"><p>• I believe it is easy to become proficient in using AI for learning</p></td><td align="left"><p>^</p></td></tr><tr><td align="left"><p>• I believe the interaction with AI technology is clear and understandable</p></td><td align="left"><p>–</p></td></tr><tr><td align="left"><p>• I frequently examine AI knowledge</p></td><td align="left" rowspan="7"><p>(Engagement in self-reflection)</p></td><td align="left"><p>–</p></td><td align="left" rowspan="7" /><td align="left" rowspan="7" /></tr><tr><td align="left"><p>• I frequently take time to reflect on my AI knowledge</p></td><td align="left"><p>–</p></td></tr><tr><td align="left"><p>• I often think about the way I use AI in learning</p></td><td align="left"><p>–</p></td></tr><tr><td align="left"><p>• It is important for me to evaluate the AI technology that I use</p></td><td align="left"><p>–</p></td></tr><tr><td align="left"><p>• I am very interested in examining how I use AI technology in learning</p></td><td align="left"><p>^</p></td></tr><tr><td align="left"><p>• It is important to me to try to understand the advantages and risks of the AI technology I use</p></td><td align="left"><p>–</p></td></tr><tr><td align="left"><p>• I have a definite need to understand how AI technology processes data</p></td><td align="left"><p>–</p></td></tr></tbody></table> </ephtml> </p> <p>#Reach agreement level; @ removed; ^ combined or revised; + added ✓Indicates the final list that reaches agreement level</p> <hd id="AN0188049029-16">Stage 2</hd> <p></p> <hd id="AN0188049029-17">Internal consistency, variability and validity of SAICS scale</hd> <p>As shown in Table 4, TAISCS scores were internally consistent for all eight dimensions—ILAI, ASAI, DMAI, DEAI, DGAI, MCAI, HCLG, and CFAI. All Cronbach's alpha (CR) values are equal to or greater than 0.89; All dimensions were near to normal distributions (skewness and kurtosis between −1 and 1) and had sufficient variability (range and standard deviation); All dimensions had high factor loadings, ranging from 0.76 to 0.96 (> 0.80). The correlation matrix between SAICS scale scores supported the scale's convergent validity, with all correlated dimensions (r > 0.34; <emph>p</emph> <.05).</p> <p>Table 4 Descriptive statistics and correlation among all the items in measurement model</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Dimension</p></th><th align="left"><p>CR</p></th><th align="left"><p>Mean (SD)</p></th><th align="left"><p>1</p></th><th align="left"><p>2</p></th><th align="left"><p>3</p></th><th align="left"><p>4</p></th><th align="left"><p>5</p></th><th align="left"><p>6</p></th><th align="left"><p>7</p></th><th align="left"><p>8</p></th></tr></thead><tbody><tr><td align="left"><p>1. ILAI</p></td><td align="left"><p>.91</p></td><td align="left"><p>3.21 (1.09)</p></td><td align="left"><p>–</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>2. ASAI</p></td><td align="left"><p>.90</p></td><td align="left"><p>3.14 (1.09)</p></td><td align="left"><p>.54*</p></td><td align="left"><p>–</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>3. DMAI</p></td><td align="left"><p>.89</p></td><td align="left"><p>3.18 (1.05)</p></td><td align="left"><p>.42*</p></td><td align="left"><p>.51*</p></td><td align="left"><p>–</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>4. DEAI</p></td><td align="left"><p>.89</p></td><td align="left"><p>3.10 (1.03)</p></td><td align="left"><p>.52*</p></td><td align="left"><p>.50*</p></td><td align="left"><p>.49*</p></td><td align="left"><p>–</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>5. DGAI</p></td><td align="left"><p>.91</p></td><td align="left"><p>3.03 (1.08)</p></td><td align="left"><p>.38*</p></td><td align="left"><p>.46*</p></td><td align="left"><p>.43*</p></td><td align="left"><p>.44*</p></td><td align="left"><p>–</p></td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>6. MCAI</p></td><td align="left"><p>.90</p></td><td align="left"><p>2.98 (1.12)</p></td><td align="left"><p>.50*</p></td><td align="left"><p>.48*</p></td><td align="left"><p>.45*</p></td><td align="left"><p>.55*</p></td><td align="left"><p>.47*</p></td><td align="left"><p>–</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>7. HCLG</p></td><td align="left"><p>.96</p></td><td align="left"><p>3.07 (1.11)</p></td><td align="left"><p>.50*</p></td><td align="left"><p>.41*</p></td><td align="left"><p>.34*</p></td><td align="left"><p>.43*</p></td><td align="left"><p>.36*</p></td><td align="left"><p>.49*</p></td><td align="left"><p>–</p></td><td align="left" /></tr><tr><td align="left"><p>8. CFAI</p></td><td align="left"><p>.96</p></td><td align="left"><p>3.08 (1.05)</p></td><td align="left"><p>.46*</p></td><td align="left"><p>.35*</p></td><td align="left"><p>.40*</p></td><td align="left"><p>.40*</p></td><td align="left"><p>.35*</p></td><td align="left"><p>.41*</p></td><td align="left"><p>.50</p></td><td align="left"><p>–</p></td></tr></tbody></table> </ephtml> </p> <p>*<emph>p</emph> <.05</p> <hd id="AN0188049029-18">CFA</hd> <p>In the CFA, the fitness indices of the measured items revealed a good model fit regarding the measurement model's goodness-of-fit: x<sups>2</sups>/df = 2.39 (< 5.0); RMSEA = 0.056 (< 0.08); SRMR = 0.03 (< 0.05); PNFI = 0.81 (> 0.50); CFI = 0.95 (> 0.90) (Kline, [<reflink idref="bib19" id="ref105">19</reflink>]; Zhan, [<reflink idref="bib37" id="ref106">37</reflink>]). Figure 2 shows the path relations and coefficients between the items. These results show that the measurement model of SAICS scale has high validity and reliability scores (RQ2).</p> <p>Graph: Fig. 2 Confirmatory factor analysis of 32 items of the SAICS scale</p> <hd id="AN0188049029-19">Gender differences across SAICS scale dimensions</hd> <p>We examined the gender differences in SAICS (RQ3). A multiple-group CFA was performed to determine whether the measurement models differed significantly between gender (male vs. female) groups. A drop of 0.01 in CFI is regarded as suitable for detecting lack of invariance across several groups in structural equation modeling (Cheung & Rensvold, 2002); thus, changes in CFI are advised for comparing models with sufficiently large sample sizes (Zhan, [<reflink idref="bib37" id="ref107">37</reflink>]).</p> <p>Table 5 shows the results of a multiple-group CFA to determine whether the structure of the SAICS scale was consistent between male and female teachers. The results showed that the baseline model is a good fit, with RMSEA = 0.057, PNFI = 0.761, and CFI = 0.916. The invariant measurement weight model provides a good fit, with RMSEA = 0.056, PNFI = 0.781, and CFI = 0.916. Similarly, the invariant structural covariances model fits well, with RMSEA = 0.056, PNFI = 0.806, and CFI = 0.911. Thus, all the models are suitable for comparison. The model comparison analysis revealed that both models of invariant measurement weight (change in CFI = 0.000) and invariant structural covariances (change in CFI = 0.005) differed by less than 0.01 from the baseline model. These results demonstrate that the measuring model is consistent across male and female students.</p> <p>Table 5 Invariance tests across teachers of different genders</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Model</p></th><th align="left"><p>x<sup>2</sup>/df</p></th><th align="left"><p>RMSEA</p></th><th align="left"><p>PNFI</p></th><th align="left"><p>CFI</p></th><th align="left"><p>Change in CFI</p></th><th align="left"><p>Change in df</p></th></tr></thead><tbody><tr><td align="left" /><td align="left" colspan="6"><p>Invariance across male and female teachers</p></td></tr><tr><td align="left"><p>Baseline model</p></td><td char="." align="char"><p>2.425</p></td><td char="." align="char"><p>.057</p></td><td char="." align="char"><p>.761</p></td><td char="." align="char"><p>.916</p></td><td char="." align="char" /><td char="." align="char" /></tr><tr><td align="left"><p>Invariant measurement weight model</p></td><td char="." align="char"><p>2.382</p></td><td char="." align="char"><p>.056</p></td><td char="." align="char"><p>.781</p></td><td char="." align="char"><p>.916</p></td><td char="." align="char"><p>.000</p></td><td char="." align="char"><p>28</p></td></tr><tr><td align="left"><p>Invariant structural covariances model</p></td><td char="." align="char"><p>2.417</p></td><td char="." align="char"><p>.056</p></td><td char="." align="char"><p>.806</p></td><td char="." align="char"><p>.911</p></td><td char="." align="char"><p>.005</p></td><td char="." align="char"><p>60</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0188049029-20">Discussion</hd> <p>This two-stage study seeks AI-experienced professional teachers' opinions to create an AI competency framework for students and validate its SAICS scale. The results offer two major implications: an AI competency framework for students and a validated SAICS scale (see Table 6).</p> <p>Table 6 The final SAICS scale</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Dimension</p></th><th align="left"><p>Items</p></th></tr></thead><tbody><tr><td align="left"><p>Interdisciplinary learning with AI</p></td><td align="left"><p>✓ I can use AI to get different alternative information to solve real-life problems better</p><p>✓ I can use AI to gain knowledge in a discipline I am unfamiliar with</p><p>✓ I can use AI to conduct inquiry-based learning in other disciplines</p><p>✓ I can use AI to find relevant information outside the classroom curriculum quickly and efficiently</p></td></tr><tr><td align="left"><p>Assessment with AI</p></td><td align="left"><p>✓ I can use AI to check whether my solutions are correct</p><p>✓ I can use AI to generate questions for further learning</p><p>✓ I can use AI to get feedback to improve my ideas</p><p>✓ I can use AI to get ideas to complete my learning tasks</p></td></tr><tr><td align="left"><p>Multimedia creation with AI</p></td><td align="left"><p>✓ I can use AI to create images for my learning (e.g., presentations, communications, assignments)</p><p>✓ I can use AI to create videos for my learning</p><p>✓ I can use AI to create audio for my learning</p><p>✓ I can use AI to create slides for my learning</p></td></tr><tr><td align="left"><p>Decision making with AI</p></td><td align="left"><p>✓ I can ask for more resources based on the recommendations by AI for my learning</p><p>✓ I can judge if resources (e.g., websites, videos) recommended by AI are appropriate for my learning</p><p>✓ I can judge if an essay generated by AI is correct for my learning</p><p>✓ I can judge if an image/video essay generated by AI is correct for my learning</p></td></tr><tr><td align="left"><p>Human-centric learning</p></td><td align="left"><p>✓ I believe that AI is a tool to enhance my learning experience, rather than just a means to finish assignments</p><p>✓ I can leverage AI to boost my understanding and skills, not merely to complete tasks</p><p>✓ I can ensure my essays are original when co-writing with AI</p><p>✓ I can collaborate with AI to create slides that are my ideas</p></td></tr><tr><td align="left"><p>Designing AI</p></td><td align="left"><p>✓ I know how to collect data for designing machine learning</p><p>✓ I know how to create a machine learning application</p><p>✓ I know how to categorize data for designing machine learning</p><p>✓ I know how to evaluate my created machine learning</p></td></tr><tr><td align="left"><p>Data, ethics, and AI</p></td><td align="left"><p>✓ I know how to protect my personal data when using AI</p><p>✓ I understand that AI technology sometimes gives a biased output</p><p>✓ I understand that AI technology sometimes generates incorrect output</p><p>✓ I understand that I should not create fake information</p></td></tr><tr><td align="left"><p>Confidence with AI</p></td><td align="left"><p>✓ I feel comfortable using AI technology in learning</p><p>✓ I feel confident using AI technology in learning</p><p>✓ I am confident I will find new AI technology for learning</p><p>✓ I find it easy to learn AI technology</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0188049029-21">Eight dimensions in AI competency framework for students</hd> <p>The results of the Delphi study suggest eight dimensions in an AI competency framework for students. They are Interdisciplinary Learning with AI (ILAI), Assessment with AI (ASAI), Decision Making with AI (DMAI), Data, Ethics, and AI (DEAI), Designing AI (DGAI), Multimedia Creation with AI (MCAI), Human-Centric Learning (HCLG), and Confidence with AI (CFAI). The framework focuses on student capacity to use AI to support their learning ethically, healthily, and productively. DMAI, DEAI, DGAI, and HCLG are UNESCO's ([<reflink idref="bib29" id="ref108">29</reflink>]) AI competency framework for school students, and Chiu et al. ([<reflink idref="bib9" id="ref109">9</reflink>]); ILAI, ASAI, MCAI, and CFAI are aligned with the studies of Chiu et al. ([<reflink idref="bib9" id="ref110">9</reflink>]), Chiu ([<reflink idref="bib5" id="ref111">5</reflink>]), Lin et al. ([<reflink idref="bib22" id="ref112">22</reflink>]), Shih et al. ([<reflink idref="bib26" id="ref113">26</reflink>]), and Wang and Chuang ([<reflink idref="bib30" id="ref114">30</reflink>]). This framework considers the impact of generative AI technologies such as ChatGPT and other educational chatbots and includes interdisciplinary learning (ILAI), assessment (ASAI), and multimedia creation (HCAI), including text-to-image and text-to-video (Chiu, [<reflink idref="bib5" id="ref115">5</reflink>]; Chiu et al., [<reflink idref="bib9" id="ref116">9</reflink>]; Zhang & Zhang, [<reflink idref="bib38" id="ref117">38</reflink>]). It also emphasizes the importance of meaningful learning, remembering, and understanding when using AI in learning (HCLG) (Guo, [<reflink idref="bib15" id="ref118">15</reflink>]). Students are required to make their judgments on the outputs created or recommended by AI (DMAI). Designing AI applications (DGAI) and comprehending data and AI ethics (DEAI) can assist students in making informed learning decisions. This framework also emphasizes the emotional side; students should be confident in integrating AI into their learning (CFAI).</p> <hd id="AN0188049029-22">SAICS scale</hd> <p>Our findings showed that the SAICS scale, which we developed using a Delphi study in Stage 1, was reliable and valid. The SAICS scale was consistent among both female and male students. Our analyses demonstrated that the proposed eight-dimensional scale can measure student AI competency (see Table 3). The following provides detailed descriptions for each dimension.</p> <p></p> <ulist> <item> ILAI: This competency to use AI to support interdisciplinary learning (Chiu, [<reflink idref="bib5" id="ref119">5</reflink>]). AI can be a powerful tool for solving real-life problems by providing alternative information and perspectives. It helps in gaining knowledge in students' unfamiliar disciplines by offering tailored resources and explanations. AI supports inquiry-based learning across various fields, enabling deeper exploration and understanding. AI can quickly and efficiently find relevant information outside the classroom curriculum, making it easier to stay informed and up to date. Students should be able to enhance their problem-solving skills, broaden their knowledge base, and conduct thorough research in a more efficient manner, making AI an invaluable asset in interdisciplinary learning.</item> <p></p> <item> ASAI: This competency is used to support assessment (Chiu, [<reflink idref="bib5" id="ref120">5</reflink>]; Weng et al., [<reflink idref="bib32" id="ref121">32</reflink>]; Xia et al., [<reflink idref="bib35" id="ref122">35</reflink>]). Students should be able to use AI to verify the correctness of their solutions, ensuring they are on the right track. AI can generate questions to further their learning, helping them delve deeper into subjects. They can use AI to get feedback to refine and improve their ideas. They can use AI to get creative suggestions to complete their learning tasks, making their learning process related to revision more efficient and effective. By leveraging AI, students can enhance their understanding, receive personalized feedback, and explore unfamiliar knowledge, ultimately enriching their learning experience.</item> <p></p> <item> MCAI: This competency to use AI to create multimedia content (Zhang & Zhang, [<reflink idref="bib38" id="ref123">38</reflink>]). AI can enhance student learning by creating images for presentations, communications, and assignments. It can also generate videos and audio to support their learning tasks, making complex concepts more accessible. AI can help students create engaging slides for their learning tasks, ensuring their presentations are visually appealing and informative. Therefore, students should be able to produce high-quality, multimedia content that enriches their learning experience.</item> <p></p> <item> DMAI: This competency is to make the right decisions on AI outputs and recommendations for learning. Students should be able to evaluate whether the suggested websites, videos, and other materials are suitable for their learning needs. They can assess the accuracy and relevance of AI-generated essays and images/videos for their needs. They should be able to ensure that the content they use is appropriate and beneficial for their learning goals.</item> <p></p> <item> HCLG: This competency can be seen as awareness of using AI for meaningful learning, remembering, and understanding. Students should understand that AI is a powerful tool to enhance their learning experience, rather than just a means to finish assignments. They should use AI to boost their understanding and skills, not merely complete tasks. For example, when co-writing with AI, students ensure their essays remain original and reflect their own thoughts. Students can collaborate with AI to create slides that embody their ideas, making their presentations more engaging and personalized.</item> <p></p> <item> DGAI: This competency is to design AI applications. Students should be proficient in creating machine learning applications, which allows them to develop innovative solutions. They should understand the process of collecting data for machine learning, ensuring the data is relevant and accurate. They should also be able to evaluate their created machine learning models, assessing their performance and making necessary improvements. This competency will help students know how to choose the right AI technology to support their learning, ensuring it aligns with their learning goals.</item> <p></p> <item> DEAI: This competency is related to student understanding of data and AI ethics. Students should know how to protect their personal data when using AI, ensuring their privacy and security. They should understand that AI technology can sometimes produce biased or incorrect outputs and should be aware of the potential for AI to generate fake information. Recognizing these limitations, they are committed to using AI responsibly and ethically and understand the importance of not creating or spreading fake information. By being vigilant and informed, students can leverage AI effectively while maintaining integrity and accuracy in their work. This awareness helps students navigate the complexities of AI technology with confidence and responsibility.</item> <p></p> <item> CFAI: This competency concerns student confidence to use AI in their learning. Students should feel comfortable and confident using AI technology in their learning. They should find it simple to learn and adapt to new AI tools, making the integration of AI into their learning seamless and beneficial. This confidence and ease with AI technology empower students to explore and leverage its full potential in their learning.</item> </ulist> <hd id="AN0188049029-23">SAICS scale serves as a reference for research and practices</hd> <p>Student AI competency is linked to the successful integration of AI in education (Chiu, [<reflink idref="bib5" id="ref124">5</reflink>]; Chiu et al., [<reflink idref="bib9" id="ref125">9</reflink>]; UNESCO, [<reflink idref="bib29" id="ref126">29</reflink>]; Zhou et al., [<reflink idref="bib39" id="ref127">39</reflink>]). Students should be able to ethically, safely, healthily, and productively use AI to support their learning. The eight-dimension student AI competency and SAICS scale in this study allow K-12 education researchers and practitioners to methodically analyze student AI competency. They can be applied in the following proposed areas, among others.</p> <p></p> <ulist> <item> Researchers can use SAICS to reproduce and update the instruments used to evaluate their intervention and learning design for AI instruction in schools.</item> <p></p> <item> Teacher educators can introduce this framework and SAICS in their teacher education program, as fostering student AI competency is also one of the main responsibilities of teachers.</item> <p></p> <item> Teachers can utilize the framework and SAICS as learning objectives/outcomes to design AI learning activities. This is because assessment can influence learning design.</item> <p></p> <item> Policymakers can leverage the framework and SAICS to create national AI competency standards, inform school and public education programs, and design assessment and policy parameters to ensure effective AI integration in education.</item> </ulist> <hd id="AN0188049029-24">Conclusion, limitations and future research suggestions</hd> <p>This study proposes a student AI competency framework and its SAICS scale. It is crucial to recognize that AI education research is still in its infancy. The findings allow us to continue our work in this field. Moreover, AI technologies are projected to be integrated into K-12 education. The findings are critical in helping us understand what student AI competency should be and how to measure it. Moreover, future research should address the four limitations of this study when using and improving the framework and the SAICS scale, see the following.</p> <p></p> <ulist> <item> AI competency is linked to current literacies such as data, mathematics, science, and computation. This study focuses mostly on the level of AI competency. Future research should seek to provide a comprehensive picture by incorporating all relevant literacy factors.</item> <p></p> <item> The SAICS scale is reliable and valid but lacks specificity across disciplines, focusing solely on assessing student competency in learning. There might be variations in different disciplines. Future research on AI competency in learning specific subjects is necessary.</item> <p></p> <item> The SAICS scale's gender differences were the sole focus of this study. Future research should assess the impact of additional factors, including culture and socioeconomic background, on the SAICS scale.</item> <p></p> <item> This scale may not be appropriate for a non-Chinese population, as the participants were Chinese students. Future research should adapt the language to the specific environment and be validated in a variety of communities, including non-Chinese or multiracial groups in regions such as the Middle East, South Asia, the United States, Europe, the United Kingdom, and Australia.</item> </ulist> <hd id="AN0188049029-25">Acknowledgements</hd> <p>The authors thank you for the school teachers for participate this study.</p> <hd id="AN0188049029-26">Funding</hd> <p>This study is funded by General Research Grants (project code: 14610522).</p> <hd id="AN0188049029-27">Data availability</hd> <p>The datasets used for the current study are available from the corresponding author on reasonable request.</p> <hd id="AN0188049029-28">Declarations</hd> <p></p> <hd id="AN0188049029-29">Competing interests</hd> <p>There is no conflict of interests between the author and participants.</p> <hd id="AN0188049029-30">Ethical approval</hd> <p>This study got ethical clearance from the author's university.</p> <hd id="AN0188049029-31">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0188049029-32"> <title> References </title> <blist> <bibl id="bib1" idref="ref87" type="bt">1</bibl> <bibtext> Ahmadi A, Noetel M, Parker PD, Ryan R, Ntoumanis N, Reeve J. A classification system for teachers' motivational behaviours recommended in self-determination theory interventions. Journal of Education Psychology. 2023; 115; 8: 1158-1176. 10.1037/edu0000783</bibtext> </blist> <blist> <bibl id="bib2" idref="ref76" type="bt">2</bibl> <bibtext> Barrett D, Heale R. What are Delphi studies?. Evidence-Based Nursing. 2020; 23; 3: 68-69. 10.1136/ebnurs-2020-103303</bibtext> </blist> <blist> <bibl id="bib3" idref="ref21" type="bt">3</bibl> <bibtext> Bawden DLankshear C, Knobel M. Origins and concepts of digital literacy. Digital literacies: Concepts, policies and practices. 2008; Peter Lang: 17-32</bibtext> </blist> <blist> <bibl id="bib4" idref="ref1" type="bt">4</bibl> <bibtext> Chiu TKF. A holistic approach to artificial intelligence (AI) curriculum for K-12 schools. TechTrends. 2021; 65: 796-807. 10.1007/s11528-021-00637-1</bibtext> </blist> <blist> <bibl id="bib5" idref="ref15" type="bt">5</bibl> <bibtext> Chiu TKF. The impact of generative AI (GenAI) on practices, policies and research direction in education: A case of ChatGPT and Midjourney. Interactive Learning Environments. 2024; 32; 10: 6187-6203. 10.1080/10494820.2023.2253861</bibtext> </blist> <blist> <bibl id="bib6" idref="ref3" type="bt">6</bibl> <bibtext> Chiu, T. K. F. (2025). Student AI literacy and competency. In Empowering K-12 education with AI. Taylor spsampsps Francis. https://doi.org/10.4324/9781003498377-2</bibtext> </blist> <blist> <bibl id="bib7" idref="ref65" type="bt">7</bibl> <bibtext> Chiu TKF, Chai CS. Sustainable curriculum planning for artificial intelligence education: A self-determination theory perspective. Sustainability. 2020; 12; 14: 5568. 10.3390/su12145568</bibtext> </blist> <blist> <bibl id="bib8" idref="ref49" type="bt">8</bibl> <bibtext> Chiu TKF, Sanusi IT. Define, foster, and assess student and teacher AI literacy and competency for all: Current status and future research direction. Computers & Education Open. 2024; 6: 100182. 10.1016/j.caeo.2024.100182</bibtext> </blist> <blist> <bibl id="bib9" idref="ref4" type="bt">9</bibl> <bibtext> Chiu TKF, Ahmand Z, Ismail M, Sanusi IT. What are artificial intelligence literacy and competency? A comprehensive framework to support them. Computers & Education Open. 2024; 6. 10.1016/j.caeo.2024.100171100171</bibtext> </blist> <blist> <bibtext> D'Ignazio C. Creative data literacy: Bridging the gap between the data-haves and data-have nots. Information Design Journal. 2017; 23; 1: 6-18</bibtext> </blist> <blist> <bibtext> Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, Wales PW. Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. Journal of Clinical Epidemiology. 2014; 67; 4: 401-409. 10.1016/j.jclinepi.2013.12.002</bibtext> </blist> <blist> <bibtext> diSessa AA. Changing minds: Computers, learning, and literacy. 2000; MIT Press</bibtext> </blist> <blist> <bibtext> Falloon G. From digital literacy to digital competence: The teacher digital competency (TDC) framework. Educational Technology Research and Development. 2020; 68: 2449-2472. 10.1007/s11423-020-09767-4</bibtext> </blist> <blist> <bibtext> Grant AM, Franklin J, Langford P. The self-reflection and insight scale: A new measure of private self-consciousness. Social Behavior and Personality: An International Journal. 2002; 30; 8: 821-835. 10.2224/sbp.2002.30.8.821</bibtext> </blist> <blist> <bibtext> Guo X. Facilitator or thinking inhibitor: Understanding the role of ChatGPT-generated written corrective feedback in language learning. Interactive Learning Environments. 2024. 10.1080/10494820.2024.2445177</bibtext> </blist> <blist> <bibtext> Jorm AF. Using the Delphi expert consensus method in mental health research. The Australian and New Zealand Journal of Psychiatry. 2015; 49; 10: 887-897. 10.1177/0004867415600891</bibtext> </blist> <blist> <bibtext> Keeney S, Hasson F, McKenna H. Consulting the oracle: Ten lessons from using the Delphi technique in nursing research. Journal of Advanced Nursing. 2006; 53; 2: 205-212. 10.1111/j.1365-2648.2006.03716.x</bibtext> </blist> <blist> <bibtext> Kim S, Jang Y, Choi S, Kim W, Jung H, Kim S, Kim H. Analyzing teacher competency with TPACK for K-12 AI education. KI-Künstliche Intelligenz. 2021; 35; 2: 139-151. 10.1007/s13218-021-00731-9</bibtext> </blist> <blist> <bibtext> Kline RB. Principles and practice of structural equation modeling. 2015; The Guilford Press</bibtext> </blist> <blist> <bibtext> Kong SC, Cheung WMY, Zhang G. Evaluation of an artificial intelligence literacy course for university students with diverse study backgrounds. Computers and Education: Artificial Intelligence. 2021; 2: 100026</bibtext> </blist> <blist> <bibtext> Laupichler MC, Aster A, Raupach T. Delphi study for the development and preliminary validation of an item set for the assessment of non-experts' AI literacy. Computers and Education: Artificial Intelligence. 2023; 4: 100126. 10.1016/j.caeai.2023.100126</bibtext> </blist> <blist> <bibtext> Lin C-H, Yu C-C, Shih P-K, Wu LY. International Forum of educational Technology & Society STEM based artificial intelligence learning in general education for non-engineering Undergraduate students. Technology in Society. 2021; 24; 3: 224-237. 10.2307/27032867</bibtext> </blist> <blist> <bibtext> Liu C, Hou J, Tu YF, Wang Y, Hwang GJ. Incorporating a reflective thinking promoting mechanism into artificial intelligence-supported English writing environments. Interactive Learning Environments. 2021; 31; 9: 5614-5632. 10.1080/10494820.2021.2012812</bibtext> </blist> <blist> <bibtext> Long, D, & Magerko, B. (2020, April). What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI conference on human factors in computing systems (pp. 1–16). https://doi.org/10.1145/3313831.3376727</bibtext> </blist> <blist> <bibtext> Schepman A, Rodway P. Initial validation of the general attitudes towards artificial intelligence scale. Computers in Human Behavior. 2020; 1: 100014. 10.1016/j.chbr.2020.100014</bibtext> </blist> <blist> <bibtext> Shih PK, Lin CH, Wu LY, Yu CC. Learning ethics in AI-teaching nonengineering undergraduates through situated learning. Sustainability. 2021. 10.3390/su13073718</bibtext> </blist> <blist> <bibtext> Teixeira PJ, Marques MM, Silva MN, Brunet J, Duda J, Haerens L, La Guardia J, Lindwall M, Lonsdale C, Markland D. A classification of motivation and behavior change techniques used in self-determination theory-based interventions in health contexts. Motivation Science. 2020; 6; 4: 438-455. 10.1037/mot0000172</bibtext> </blist> <blist> <bibtext> Touretzky D, Gardner-McCune C, Seehorn D. Machine learning and the five big ideas in AI. International Journal of Artificial Intelligence in Education. 2023; 33; 2: 233-266. 10.1007/s40593-022-00314-1</bibtext> </blist> <blist> <bibtext> UNESCO (2024, May 18). AI competency frameworks for school students and teachers. The United Nations Educational, Scientific and Cultural Organization. https://<ulink href="http://www.unesco.org/en/digital-education/ai-future-learning/competency-frameworks">www.unesco.org/en/digital-education/ai-future-learning/competency-frameworks</ulink>.</bibtext> </blist> <blist> <bibtext> Wang YY, Chuang YW. Artificial intelligence self-efficacy: Scale development and validation. Education and Information Technologies, Advanced Online Publication. 2023. 10.1007/s10639-023-12015-w</bibtext> </blist> <blist> <bibtext> Wang YY, Wang YS. Development and validation of an artificial intelligence anxiety scale: An initial application in predicting motivated learning behavior. Interactive Learning Environments. 2022; 30; 4: 619-634. 10.1080/10494820.2019.1674887</bibtext> </blist> <blist> <bibtext> Weng X, Xia Q, Gu MM, Rajaram K, Chiu TKF. Assessment and learning outcomes for generative AI in higher education: A scoping review on current research status and trends. Australasian Journal of Educational Technology. 2024. 10.14742/ajet.9540</bibtext> </blist> <blist> <bibtext> Williams R, Ali S, Devasia N, DiPaola D, Hong J, Kaputsos SP, Jordan B, Breazeal C. AI+ ethics curricula for middle school youth: Lessons learned from three project-based curricula. International Journal of Artificial Intelligence in Education. 2023; 33; 2: 325-383. 10.1007/s40593-022-00298-y</bibtext> </blist> <blist> <bibtext> Wu B, Chen X. Continuance intention to use MOOCs: Integrating the technology acceptance model (TAM) and task technology fit (TTF) model. Computers in Human Behavior. 2017; 67: 221-232. 10.1016/j.chb.2016.10.028</bibtext> </blist> <blist> <bibtext> Xia Q, Weng XJ, Ouyan F, Lin T-J, Chiu TKF. A Scoping review on how generative artificial intelligence transform assessment in higher education. International Journal of Education Technology in Higher Education. 2024; 21: 40. 10.1186/s41239-024-00468-z</bibtext> </blist> <blist> <bibtext> Yau S, Chai CS, Chiu TKF, Meng H, King I, Yam Y. A phenomenographic approach on teacher conceptions of teaching artificial intelligence (AI) in K-12 schools. Education and Information Technologies. 2022; 28: 1041-1064. 10.1007/s10639-022-11161-x</bibtext> </blist> <blist> <bibtext> Zhan Y. Developing and validating a student feedback literacy scale. Assessment & Evaluation in Higher Education. 2022; 47; 7: 1087-1100. 10.1080/02602938.2021.2001430</bibtext> </blist> <blist> <bibtext> Zhang L, Zhang W. Integrating large language models into project-based learning based on self-determination theory. Interactive Learning Environments. 2024. 10.1080/10494820.2024.2446535</bibtext> </blist> <blist> <bibtext> Zhou X, Li Y, Chai CS, Chiu TKF. Defining, enhancing, and assessing artificial intelligence literacy and competency in K-12 education from a systematic review. Interactive Learning Environments, Advanced Online Publication,. 2025. 10.1080/10494820.2025.2487538</bibtext> </blist> </ref> <aug> <p>By Thomas K. F. Chiu; Murat Çoban; Ismaila Temitayo Sanusi and Musa Adekunle Ayanwale</p> <p>Reported by Author; Author; Author; Author</p> <p></p> <p>Thomas K. F. Chiu Thomas K. F. Chiu is an assistant professor of digital futures at The Chinese University of Hong Kong. His research interests include educational technology, AI, mathematics, STEM education, educational policy, and teacher education.</p> <p>Murat Çoban Murat Çoban is an assocaite professor at Ataturk University, and his research areas include Social Sciences and Humanities, Natural Sciences.</p> <p>Ismaila Temitayo Sanusi Ismaila Temitayo Sanusi is Postdoctoral Researcher at University of Eastern Finland, and his interest is democratizing ML and AI through K-12.</p> <p>Musa Adekunle Ayanwale Musa Adekunle Ayanwale is a senior postdoctoral research fellow at University of Johannesburg, and his research areas include assessment.</p> </aug> <nolink nlid="nl1" bibid="bib28" firstref="ref2"></nolink> <nolink nlid="nl2" bibid="bib20" firstref="ref5"></nolink> <nolink nlid="nl3" bibid="bib21" firstref="ref6"></nolink> <nolink nlid="nl4" bibid="bib24" firstref="ref7"></nolink> <nolink nlid="nl5" bibid="bib39" firstref="ref8"></nolink> <nolink nlid="nl6" bibid="bib29" firstref="ref13"></nolink> <nolink nlid="nl7" bibid="bib13" firstref="ref14"></nolink> <nolink nlid="nl8" bibid="bib23" firstref="ref18"></nolink> <nolink nlid="nl9" bibid="bib12" firstref="ref23"></nolink> <nolink nlid="nl10" bibid="bib10" firstref="ref26"></nolink> <nolink nlid="nl11" bibid="bib22" firstref="ref34"></nolink> <nolink nlid="nl12" bibid="bib26" firstref="ref35"></nolink> <nolink nlid="nl13" bibid="bib30" firstref="ref36"></nolink> <nolink nlid="nl14" bibid="bib25" firstref="ref38"></nolink> <nolink nlid="nl15" bibid="bib31" firstref="ref39"></nolink> <nolink nlid="nl16" bibid="bib36" firstref="ref60"></nolink> <nolink nlid="nl17" bibid="bib18" firstref="ref62"></nolink> <nolink nlid="nl18" bibid="bib33" firstref="ref63"></nolink> <nolink nlid="nl19" bibid="bib27" firstref="ref78"></nolink> <nolink nlid="nl20" bibid="bib17" firstref="ref81"></nolink> <nolink nlid="nl21" bibid="bib11" firstref="ref82"></nolink> <nolink nlid="nl22" bibid="bib16" firstref="ref86"></nolink> <nolink nlid="nl23" bibid="bib34" firstref="ref94"></nolink> <nolink nlid="nl24" bibid="bib19" firstref="ref95"></nolink> <nolink nlid="nl25" bibid="bib37" firstref="ref96"></nolink> <nolink nlid="nl26" bibid="bib14" firstref="ref104"></nolink> <nolink nlid="nl27" bibid="bib38" firstref="ref117"></nolink> <nolink nlid="nl28" bibid="bib15" firstref="ref118"></nolink> <nolink nlid="nl29" bibid="bib32" firstref="ref121"></nolink> <nolink nlid="nl30" bibid="bib35" firstref="ref122"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1483804
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Validating Student AI Competency Self-Efficacy (SAICS) Scale and Its Framework
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Thomas+K%2E+F%2E+Chiu%22">Thomas K. F. Chiu</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-2887-5477">0000-0003-2887-5477</externalLink>)<br /><searchLink fieldCode="AR" term="%22Murat+Çoban%22">Murat Çoban</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-2415-5747">0000-0003-2415-5747</externalLink>)<br /><searchLink fieldCode="AR" term="%22Ismaila+Temitayo+Sanusi%22">Ismaila Temitayo Sanusi</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-5705-6684">0000-0002-5705-6684</externalLink>)<br /><searchLink fieldCode="AR" term="%22Musa+Adekunle+Ayanwale%22">Musa Adekunle Ayanwale</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-7640-9898">0000-0001-7640-9898</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Educational+Technology+Research+and+Development%22"><i>Educational Technology Research and Development</i></searchLink>. 2025 73(4):2785-2807.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 23
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Digital+Literacy%22">Digital Literacy</searchLink><br /><searchLink fieldCode="DE" term="%22Competence%22">Competence</searchLink><br /><searchLink fieldCode="DE" term="%22Self+Efficacy%22">Self Efficacy</searchLink><br /><searchLink fieldCode="DE" term="%22Measures+%28Individuals%29%22">Measures (Individuals)</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+School+Students%22">Elementary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Secondary+School+Students%22">Secondary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Interdisciplinary+Approach%22">Interdisciplinary Approach</searchLink><br /><searchLink fieldCode="DE" term="%22Decision+Making%22">Decision Making</searchLink><br /><searchLink fieldCode="DE" term="%22Data%22">Data</searchLink><br /><searchLink fieldCode="DE" term="%22Ethics%22">Ethics</searchLink><br /><searchLink fieldCode="DE" term="%22Design%22">Design</searchLink><br /><searchLink fieldCode="DE" term="%22Multimedia+Materials%22">Multimedia Materials</searchLink><br /><searchLink fieldCode="DE" term="%22Factor+Analysis%22">Factor Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Gender+Differences%22">Gender Differences</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Construction%22">Test Construction</searchLink><br /><searchLink fieldCode="DE" term="%22Delphi+Technique%22">Delphi Technique</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s11423-025-10512-y
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1042-1629<br />1556-6501
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Nurturing student artificial intelligence (AI) competency is crucial in the future of K-12 education. Students with strong AI competency should be able to ethically, safely, healthily, and productively integrate AI into their learning. Research on student AI competency is still in its infancy, primarily focusing on theoretical and professional discussions, along with qualitative investigations. This two-stage study aims to propose an AI competency framework for students and confirm the reliability and validity of its scale--student AI competency self-efficacy (SAICS)--in K-12 education. In stage 1, we used a three-round Delphi study to propose the framework and its scale. The framework has eight dimensions: interdisciplinary learning with AI, assessment with AI, decision-making with AI, data, ethics and AI, designing AI, multimedia creation with AI, human-centric learning, and confidence with AI. Each dimension contains four items. In stage 2, we involved 448 students to validate the scale using confirmatory factor analysis and model comparisons. The analyses showed that the scale is consistent across male and female students. The SAICS scale comprises 32 items and addresses eight dimensions of AI competency. Researchers can use the framework and SAICS to design their interventions and correlational research associated with student AI competency. Teachers can use them to develop learning outcomes for AI-based learning activities, and policymakers can use them to establish national AI standards.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1483804
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1483804
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s11423-025-10512-y
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 23
        StartPage: 2785
    Subjects:
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Digital Literacy
        Type: general
      – SubjectFull: Competence
        Type: general
      – SubjectFull: Self Efficacy
        Type: general
      – SubjectFull: Measures (Individuals)
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Elementary School Students
        Type: general
      – SubjectFull: Secondary School Students
        Type: general
      – SubjectFull: Interdisciplinary Approach
        Type: general
      – SubjectFull: Decision Making
        Type: general
      – SubjectFull: Data
        Type: general
      – SubjectFull: Ethics
        Type: general
      – SubjectFull: Design
        Type: general
      – SubjectFull: Multimedia Materials
        Type: general
      – SubjectFull: Factor Analysis
        Type: general
      – SubjectFull: Gender Differences
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Test Construction
        Type: general
      – SubjectFull: Delphi Technique
        Type: general
    Titles:
      – TitleFull: Validating Student AI Competency Self-Efficacy (SAICS) Scale and Its Framework
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Thomas K. F. Chiu
      – PersonEntity:
          Name:
            NameFull: Murat Çoban
      – PersonEntity:
          Name:
            NameFull: Ismaila Temitayo Sanusi
      – PersonEntity:
          Name:
            NameFull: Musa Adekunle Ayanwale
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 1042-1629
            – Type: issn-electronic
              Value: 1556-6501
          Numbering:
            – Type: volume
              Value: 73
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Educational Technology Research and Development
              Type: main
ResultId 1