View in EDS HTML Full Text PDF Full Text

Investigating Assessment Types in an Online Climate Change Class: Moderating and Mediating Effects

Saved in:
Bibliographic Details
Title: Investigating Assessment Types in an Online Climate Change Class: Moderating and Mediating Effects
Language: English
Authors: April L. Millet (ORCID 0000-0001-7928-7945), Emre Dinç (ORCID 0000-0002-1838-4044), Timothy J. Bralower (ORCID 0000-0002-3503-859X)
Source: Educational Technology Research and Development. 2024 72(6):3075-3101.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 27
Publication Date: 2024
Sponsoring Agency: National Aeronautics and Space Administration (NASA)
National Science Foundation (NSF)
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Undergraduate Students, Test Format, Tests, Laboratory Experiments, Comparative Testing, Curriculum Based Assessment, Capstone Experiences, Climate, Knowledge Level, Concept Formation, Critical Thinking, Problem Solving, Creative Thinking, Discovery Processes, Academic Achievement, Learning Processes, Testing
DOI: 10.1007/s11423-024-10392-8
ISSN: 1042-1629
1556-6501
Abstract: This study aimed to examine the effect of four types of assessment on overall student success in an online college-level climate change course. Quizzes, midterms, lab assignments, and a capstone project as well as knowledge check questions were used to assess different aspects of student learning, consistent with Bloom's taxonomy hierarchy. Quizzes and midterms assess basic knowledge, including remembering and understanding concepts, laboratory assignments require students to analyze and integrate concepts, and the capstone allows students to evaluate their understanding and create new content. Binary logistic regression, multiple regression analysis, continuous-by-continuous interaction modeling, and path analysis were used to investigate the moderating and mediating effects of these assessment types. We found both direct and indirect positive interactions as well as one negative interaction. Positive interactions were identified between quiz and lab assignment achievement and between capstone achievement and lab assignment achievement. The total score for correctly answered knowledge check questions positively affected quiz and lab assignment achievements. The interaction between capstone project achievement and total score for correctly answered knowledge check questions showed a negative interaction. Finally, the total score for correctly answered knowledge-check questions had an indirect positive effect on overall student success in the course. Results show that different types of assessment in an online course are complementary and amplify student learning.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1452034
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwF2F8oz_YSNM7KYC6J323jNAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDABT83BiUNHVQPT1vAIBEICBm8g1rxJT3dheiBg03feCl2mBSVAVu8lC1BeR9KfAtTEHiqh5H1RzLR53R1GKWyeZutjl4C3CZlCIj2iy-nyYQJvpA3vWsp65lNn3t4nrNWi9zM7uGwc_PsD1a8RDZPtjmlrY1_fdw5AqxbBWE-YRrMoKPGhxBaSNJpjIruOuKhnCOnrg3xw2PX9ZYQ-c0TQAVXuxm1s9mXrwdN1L
Text:
  Availability: 1
  Value: <anid>AN0181464865;etr01dec.24;2024Dec09.03:53;v2.2.500</anid> <title id="AN0181464865-1">Investigating assessment types in an online climate change class: moderating and mediating effects: Investigating assessment types in an online climate change class: A. L. Millet et al </title> <p>This study aimed to examine the effect of four types of assessment on overall student success in an online college-level climate change course. Quizzes, midterms, lab assignments, and a capstone project as well as knowledge check questions were used to assess different aspects of student learning, consistent with Bloom's taxonomy hierarchy. Quizzes and midterms assess basic knowledge, including remembering and understanding concepts, laboratory assignments require students to analyze and integrate concepts, and the capstone allows students to evaluate their understanding and create new content. Binary logistic regression, multiple regression analysis, continuous-by-continuous interaction modeling, and path analysis were used to investigate the moderating and mediating effects of these assessment types. We found both direct and indirect positive interactions as well as one negative interaction. Positive interactions were identified between quiz and lab assignment achievement and between capstone achievement and lab assignment achievement. The total score for correctly answered knowledge check questions positively affected quiz and lab assignment achievements. The interaction between capstone project achievement and total score for correctly answered knowledge check questions showed a negative interaction. Finally, the total score for correctly answered knowledge-check questions had an indirect positive effect on overall student success in the course. Results show that different types of assessment in an online course are complementary and amplify student learning.</p> <p>Keywords: Assessment; E-learning; STEM; Online education; Retrieval practice; Education Specialist Studies In Education</p> <p>Copyright comment Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.</p> <hd id="AN0181464865-2">Introduction</hd> <p>Online education has become increasingly important before, during, and after the COVID-19 pandemic, accelerating the need for remote learning and making it a significant part of education systems. Online assessment has gained importance due to the shift to remote learning, making continued research and improvement of online assessment practices critical. Educators must develop effective strategies for assessing student performance and progress in an online learning environment, including creating reliable assignments that accurately measure students' learning. Therefore, it is essential to conduct research in online assessment to ensure learners' success in online courses.</p> <p>[The course Name: removed for blind review] is an entry-level undergraduate course that was developed in partnership with [The Partner Institution's Name: removed for blind review], the eLearning arm of [The University Name: removed for blind review]. The course is the core of both online certificate and minor programs in Earth Sustainability.</p> <p>The course provides students with instruction on climate science, the impacts on humans and natural ecosystems, as well as ways that humans can mitigate and adapt to climate change. This course includes a total of 12 modules, each requiring 1 week to complete. The course includes 6 weeks on the basis of Earth's climate and why it is changing, 5 weeks on the impact of climate change on the oceans, water, food supply, sea level, and ecosystems, and a final week on adaptation and mitigation. Students are assessed using four assessment types, including weekly quizzes, laboratory assignments, two non-cumulative midterms, and a six-part capstone assignment.</p> <p>The course has been taught every semester since Fall 2012 in a variety of formats. The first offering of the course was in-person with lectures and laboratories offered in the class period. This format was replaced by the development of an online platform [The Link for the online platform: removed for blind review] where both lecture and lab materials, and a capstone assignment, are presented, and the course subsequently transitioned to fully online and blended formats. In the fully online format, lectures, labs and capstone are asynchronous. In the blended format, the lab or capstone are held in person and the remainder of the course is online and asynchronous. There are no formal lectures; instead, most of the material is written in an engaging way for students to read at their own pace. These materials are supplemented by short daily video primers on relevant topics. The course website was created in Drupal, our content management system, using images, videos, and interactive elements from around the world using Creative commons licenses. Creative commons materials are used to decrease copyright conflicts (Kim, [<reflink idref="bib23" id="ref1">23</reflink>]) like the one we faced.</p> <p>Today, the course website has 233 pages of text that include images, videos, multimedia, knowledge checks, labs, and reading assignments. These pages have nearly 700 photos, graphics, charts, and tables spread out throughout the course. In addition to those visual elements, the course also utilizes 150 videos. Some videos are YouTube videos that have been embedded. Some of them were created by the instructor of the class and the Learning Design team. In addition to the text, images, and video, over 70 simple knowledge check questions with interactive content have been added. Interactive knowledge check questions enabled us to provide active learning to students and to create interactive and flexible learning along with automatic feedback (Singleton & Charlton, [<reflink idref="bib44" id="ref2">44</reflink>]; Wilkie et al., [<reflink idref="bib49" id="ref3">49</reflink>]).</p> <p>The instructor first taught the course fully online at [The University Name: removed for blind review] in 2013 and made it available for observation or download as an open education resource (OER) the same year. OERs are open source which allow access to and engagement with the content available either under a license or without copyright (Wiley et al., [<reflink idref="bib48" id="ref4">48</reflink>]). The course has been downloaded 56 times in 21 different countries. The majority of these have been initiated by 4-year colleges (see Fig. 1). The course, in its current form, has been taken by nearly 6000 [The University Name: removed for blind review] students during the last 10 years. Most students take the course in the fully online format. In addition, the content has been viewed by many more people worldwide based on the information from google analytics.</p> <p>Graph: Fig. 1 Infographics for the course information</p> <hd id="AN0181464865-3">Theoretical framework</hd> <p>The theoretical framework for the course design is based on Moore's three types of interaction in online education (Moore, [<reflink idref="bib58" id="ref5">58</reflink>]), the use of backward design for the design of learning (Wiggins & McTighe, [<reflink idref="bib47" id="ref6">47</reflink>]), the organization of assessment formats based on Bloom's taxonomy (Anderson et al., [<reflink idref="bib3" id="ref7">3</reflink>]), and the advantage of monitoring cognition under metacognition (Flavell, [<reflink idref="bib17" id="ref8">17</reflink>]).</p> <p>The course was designed to keep the students as active and engaged with the materials as possible. Each module consists of custom content with knowledge check questions embedded throughout. In addition, the instructor also assigns additional readings for most modules and laboratory assignments for eight out of twelve modules. Students are assessed using four assessment types, including weekly quizzes, laboratory assignments, two non-cumulative midterm exams, and a bi-weekly capstone assignment. The weekly quizzes and two non-cumulative midterm exams are aimed at testing student understanding of the content. The laboratory assignments are designed to give students the opportunity to interpret models and real data related to Earth's climate and how its change is impacting the oceans and land, stressing the analysis of large data sets and the complexity of the Earth System. The capstone assignment presents students with the opportunity to do research and synthesize different aspects of what they have learned. Students write six capstone entries for a project called [The Project Name: removed for blind review] that focuses on the impact of climate change on communities around the world. Each entry is 200–350 words. The strongest entries received invitations for publication in a compilation at: [The Link for the publication: removed for blind review] that is becoming a valuable resource on the impact of climate change across the planet viewed through students' eyes.</p> <hd id="AN0181464865-4">Module design</hd> <p>Each module has a consistent design. The module's introductory page has a video introducing the concepts of the module, a list of the goals and objectives for the module, and a Roadmap that lists the assignments for the lesson. This is followed by content pages that have pictures, graphics, tables, links to readings, videos, and embedded knowledge check questions labeled check your understanding. Finally, there is a summary with reminders on which assignments they need to complete. There is also a lab or a capstone page depending on whether a lab or capstone assignment is due in that module (see Fig. 2).</p> <p>Graph: Fig. 2 Design specifics on a sample module for water resources and climate change</p> <hd id="AN0181464865-5">Moore's three types of interaction</hd> <p>Student support is at the forefront of design considerations that help make students more engaged and successful in online courses (Liu et al., [<reflink idref="bib57" id="ref9">57</reflink>]; Schrum & Hong, [<reflink idref="bib40" id="ref10">40</reflink>]). In our course, students are supported in a variety of ways. We use Moore's Three Types of Interaction (Moore, [<reflink idref="bib58" id="ref11">58</reflink>]) as a guide to ensure that students are successful in online courses. The three types of interaction that Moore speaks of include learner-instructor, learner-learner, and learner-content interactions (see Table 1).</p> <p>Table 1 Student support provided in the course</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Interactions</p></th><th align="left"><p>Notes</p></th></tr></thead><tbody><tr><td align="left"><p>Student-faculty interactions</p></td><td align="left"><p>Daily videos, weekly emails, answer questions on yammer, and available for personal email exchanges</p></td></tr><tr><td align="left"><p>Student-content interactions</p></td><td align="left"><p>Video how-tos, feedback in knowledge check questions, practice tests, Help for using LMS</p></td></tr><tr><td align="left"><p>Student–student interactions</p></td><td align="left"><p>Yammer provides students with a way to engage with each other and help answer questions</p></td></tr></tbody></table> </ephtml> </p> <p>Learner-Instructor interactions include daily instructor-created video announcements and weekly emails to keep students engaged with the instructor in Canvas. Students can also initiate interaction by reaching out to the instructor at any time using email or the course's Yammer group. Yammer is a social media platform that enables students to interact with each other through synthesizing, interpreting, and sharing information related to course content as well as reflecting on and organizing the learning process (Borge & Goggins, [<reflink idref="bib54" id="ref12">54</reflink>]). The use of social media platforms, such as Yammer, for student support is an increasing trend in online course design decisions (Rotar, [<reflink idref="bib37" id="ref13">37</reflink>]). The instructor monitors the posts and answers student questions. Students can also answer their peer's questions in Yammer.</p> <p>Learner-Learner interactions in asynchronous online courses can be challenging to integrate because of how students participate in the class (Elizondo-Garcia & Gallardo, [<reflink idref="bib16" id="ref14">16</reflink>]; Singh et al., [<reflink idref="bib43" id="ref15">43</reflink>]). In this course, Learner-Learner interactions consist of students using Yammer for discourse about labs and responding to each other in the daily announcements in Canvas. In addition, students often self-organize themselves into study groups using a variety of technologies.</p> <p>Learner-Content interactions, as the core of understanding and learning in an online course, occur when students get the information directly from the learning materials provided (Singh et al., [<reflink idref="bib43" id="ref16">43</reflink>]). This type of interaction is supported through Canvas' built-in support mechanisms to help students be more successful. In addition to the Help resources in Canvas, how-to videos accompany each lab and all knowledge check questions in the course provide students with immediate feedback after they answer. This interaction is observed in learners' time spent on online course materials and resulted in a highly positive correlation with students' weekly quiz achievements (Zimmerman, [<reflink idref="bib51" id="ref17">51</reflink>]). In addition, every quiz, midterm and lab has a practice test that allows students to assess their preparation for the graded assignments, and if necessary to interact with the instructor about questions they have before they take the for-credit assignment.</p> <p>Because students work alone in the online class format, this three-prong approach is especially important when it comes to supporting them through completing the labs in the course. How-to videos help students learn the technology and answer any questions about difficult concepts.</p> <hd id="AN0181464865-6">Backward design and Bloom's taxonomy</hd> <p>Backwards design and Bloom's Taxonomy were used to design the course. Backward design first focuses on the desired outcomes for the course or lesson, followed by making assessment decisions of understanding, and then outlining objectives to reach the desired outcomes and constructing the most suitable learning activities (Wiggins & McTighe, [<reflink idref="bib47" id="ref18">47</reflink>]) (see Fig. 3). When using this approach, the learning designer works with the teaching faculty to create a solid framework that can be used to build the course (Rotar, [<reflink idref="bib36" id="ref19">36</reflink>]). The process starts with the end goal in mind. In this case, what students can do, know, or interpret at the end of each module and the course. The process starts by first identifying the learning outcomes of the course. Next, an assessment plan is created to evaluate students, however, assessment instruments are created later in the process. Similarly, the assignments and the activities that students will complete are planned next. Once a solid framework is developed, all efforts shift to creation.</p> <p>Graph: Fig. 3 Backward design modified from Wiggins McTighe (2005)</p> <p>Learning objectives are created for each lesson to communicate how students will reach the course outcomes by interacting with course content and media, and associated activities, assignments, and assessments. The learning objectives are tweaked if necessary. This strategy helps to keep the instructor and learning designer focused on the same goals with other positive impacts. For example, Reynolds and Kearns ([<reflink idref="bib34" id="ref20">34</reflink>]) developed a lesson planner tool adopting the backward design. The backward design planner tool that can also be adapted to course design has positive effects on time management, student engagement, content delivery, and frequent feedback in the college-level course.</p> <p>When designing the course, we took a holistic approach to selecting the types of assignments, large data sets, level of complexity, and assessments that would give students the best chance to be successful. We chose assessments that we knew would guide students through each level of Bloom's Taxonomy (see Fig. 4). There are four types of high-stakes assessments in the course. We wanted the course to be an active learning experience for the students. So, in addition to the labs and the capstone that we had planned, we integrated retrieval practice into the course by adding self-check questions on some content pages, and self-check knowledge-check quizzes as well as practice quizzes in Canvas. By using Bloom's Taxonomy as a guide, the assessments we designed help students move up the triangle toward higher-order learning (see Fig. 5). We used weekly quizzes (worth 20% of the final grade) and two midterm exams (worth 30% of the final grade) for knowledge acquisition to help students remember and understand information. We used labs (worth 30% of the final grade) to help students apply and analyze large Earth science data sets and run models that complement conceptual knowledge presented in the content. The labs emphasize the complexity of Earth Systems. A capstone project (worth 20% of the final grade) was developed to help students take all that knowledge and synthesize it to research and evaluate information about the location of their choosing to learn about and then create something new using what they learned.</p> <p>Graph: Fig. 4 Bloom's taxonomy as a guide for assessments in the course design</p> <p>Graph: Fig. 5 Assessment orders in Bloom's Taxonomy adapted from Anderson et al. ([<reflink idref="bib3" id="ref21">3</reflink>])</p> <p>The uses and advantages of each type of assessment have been discussed in the literature. For example, quizzes are the most preferred assessment type by instructors in online learning (Ametepe & Khan, [<reflink idref="bib2" id="ref22">2</reflink>]; Cullen et al., [<reflink idref="bib55" id="ref23">55</reflink>]; Means et al., [<reflink idref="bib29" id="ref24">29</reflink>]; Rocco, [<reflink idref="bib35" id="ref25">35</reflink>]) because weekly quizzes support learning and enhance performance in other types of assessments (El-Hashash, [<reflink idref="bib15" id="ref26">15</reflink>]). Quizzes are also praised for ensuring interaction, engagement, and motivation (Raes et al., [<reflink idref="bib32" id="ref27">32</reflink>]; Salas-Morera et al., [<reflink idref="bib60" id="ref28">60</reflink>]; Zimmerman, [<reflink idref="bib51" id="ref29">51</reflink>]). Quiz success positively affects academic success (Salas-Morera et al., [<reflink idref="bib60" id="ref30">60</reflink>]) and is positively correlated with the frequency of visiting the course website used for an online class (Ramos & Yudko, [<reflink idref="bib33" id="ref31">33</reflink>]). A recent study (d'Alessio et al., [<reflink idref="bib11" id="ref32">11</reflink>]) investigated if weekly time spent on an online geoscience course influences overall success. For students who spent three hours or less on the course content weekly, the time spent positively affected their overall success. For those spending more than that, time spent did not affect their overall success.</p> <p>Based on behaviorist theory, quizzes are resources used to learn the facts about the course content (Ally, [<reflink idref="bib1" id="ref33">1</reflink>]). According to Bonk and Zhang ([<reflink idref="bib10" id="ref34">10</reflink>]), some learners tend to grasp facts while learning; therefore, quizzes can be effective as a means to transmit and practice retrieval of facts. Note that the aim of integrating quizzes into a course is to encourage students to study course content regularly, learn basic information, and prepare them for higher cognitive activities (Freeman et al., [<reflink idref="bib18" id="ref35">18</reflink>]; Salas-Morera et al., [<reflink idref="bib60" id="ref36">60</reflink>]). Quizzes are an irreplaceable part of the assessment in online learning. For example, Radmila and Andrii ([<reflink idref="bib31" id="ref37">31</reflink>]) discussed online education models including massive online open courses (MOOC) by highlighting the tests in MOOCs. Radmila and Andrii provided a sample course from Harvard University, which was converted into a MOOC in 2014. During the course update from 2018 to 2019, quizzes and lab assignments were added as additional assessment materials.</p> <p>Lab assignments are as important as quizzes in learning for some courses. Considering cognitive strategies, learning <emph>how the processes are carried out</emph> rather than <emph>what occurs</emph> is the key to lab assignments bringing real-life situations for students to practice (Ally, [<reflink idref="bib1" id="ref38">1</reflink>]). Lab assignments based on large Earth datasets enable students to understand "how things work" (Bonk & Zhang, [<reflink idref="bib10" id="ref39">10</reflink>], p. 250), including the complexity of the Earth System, and this is the main goal of the labs in this course. In some labs, real-life scenarios are provided as simulations using web-based intuitive STELLA models that allow students to experiment with how the Earth works and how it is affected by different parameters, guiding them to generate contextual knowledge (Rule & Bajzek, [<reflink idref="bib59" id="ref40">59</reflink>]). For example, Maggioni et al. ([<reflink idref="bib28" id="ref41">28</reflink>]) presented a module for learning remote sensing in hydrology and practicing its applications with real-life data in an undergraduate course. Their aim was to provide students with opportunities to learn contextualized knowledge that is useful for students' future professions. Bursztyn et al. ([<reflink idref="bib54" id="ref42">54</reflink>]) conducted a study to investigate an online tool used for lab assignments in a geoscience course to practice and analyze a real-life model of strike and dip measurements. Students' and instructors' impressions were mainly positive, and online lab assignments had positive evaluations in measures such as active learning and learning effectiveness.</p> <p>The capstone project provides students with the opportunity to contextualize their understanding of the course material by doing their own research on communities (Ally, [<reflink idref="bib1" id="ref43">1</reflink>]; Kearns, [<reflink idref="bib22" id="ref44">22</reflink>]; Schoenbohm & McMillan, [<reflink idref="bib39" id="ref45">39</reflink>]) that are impacted by climate change. Contextual understanding generates personal meaning while assessing student learning authentically in the real-world context (Bonk & Zhang, [<reflink idref="bib10" id="ref46">10</reflink>]; Rocco, [<reflink idref="bib35" id="ref47">35</reflink>]). Students are in control of their learning and develop their understanding while working on capstone projects. According to the constructivist learning theory, students learn by actively participating in meaningful experiences, and these experiences allow the evaluation of student performances (Alomyan & Green, [<reflink idref="bib53" id="ref48">53</reflink>]; Kocadere & Ozgen, [<reflink idref="bib24" id="ref49">24</reflink>]; Zane, [<reflink idref="bib50" id="ref50">50</reflink>]). For example, Swanson et al. ([<reflink idref="bib46" id="ref51">46</reflink>]) integrated a capstone project assignment into an introductory environmental geoscience course to evaluate students' performance and learning while studying landfill siting exercises. The aim was to synthesize the previously learned information and apply it to the chosen and unfamiliar context. Similarly, Dere et al. ([<reflink idref="bib12" id="ref52">12</reflink>]) designed a capstone project assignment in an undergraduate geoscience course. Students applied their knowledge and considered how to solve an environmental problem. The capstone project assessed student learning while experiencing what a researcher does to deal with an environmental issue.</p> <hd id="AN0181464865-7">Metacognition</hd> <p>Various types of assessments are recommended to ensure the use of multiple learning strategies and elicit students' weaknesses and strengths in the online learning process (Arend, [<reflink idref="bib5" id="ref53">5</reflink>]; Bhebhe & Maphosa, [<reflink idref="bib9" id="ref54">9</reflink>]). In addition to the high-stakes assessments mentioned above, the course also uses optional low-stakes assessments known as Check Your Understanding or <emph>knowledge check</emph> questions throughout the course content. Knowledge check questions have an interactive nature that supports interaction, allow students to identify and remove misunderstandings, and can be tracked as a component of formative assessment (Authors, 2021; Lewis et al., [<reflink idref="bib26" id="ref55">26</reflink>]). We use these self-checks in the course as a type of retrieval practice. Retrieval practice is a strategy that enhances learning by asking students to retrieve information after learning something. Figure 6 below is an example of one of the questions that have been dispersed throughout the course content<emph>.</emph></p> <p>Graph: Fig. 6 Example knowledge check question in the course</p> <p>Knowledge check questions provide metacognitive knowledge, metacognitive regulation, and metacognitive experience (Flavell, [<reflink idref="bib17" id="ref56">17</reflink>]). Students check their knowledge and cognitive process (i.e., metacognitive knowledge) by consulting and responding to the check your understanding questions. Learning is monitored and evaluated (i.e., metacognitive regulation) by reviewing the answer and the feedback provided. Then, they experience their own cognitive learning (i.e., metacognitive experience) by adding or removing knowledge based on the interaction with the question. Accordingly, students' metacognitive knowledge coincides with being "aware of what and how they learn," and metacognitive regulation is about the "ability to plan, monitor, and evaluate their own learning" (Sebesta & Speth, [<reflink idref="bib41" id="ref57">41</reflink>], p. 2). Metacognition is a significant skill in student learning, particularly for learning something new and challenging, due to monitoring and regulating learning (Andrade, [<reflink idref="bib4" id="ref58">4</reflink>]; Azevedo et al., [<reflink idref="bib6" id="ref59">6</reflink>]; Sezgin-Memnun, [<reflink idref="bib42" id="ref60">42</reflink>]; Yan & Brown, [<reflink idref="bib61" id="ref61">61</reflink>]). Mainly, self-assessment questions are used to monitor learning and determine how an understanding is true or false, complete or incomplete (Andrade, [<reflink idref="bib4" id="ref62">4</reflink>]; Hartwig & Dunlosky, [<reflink idref="bib56" id="ref63">56</reflink>]; Kruger & Dunning, [<reflink idref="bib25" id="ref64">25</reflink>]). With these in mind, self-assessment, such as the knowledge check questions, enables students to engage with the course content with the advantages of the increase in grades, metacognitive skills, and academic performance (Hartwig & Dunlosky, [<reflink idref="bib56" id="ref65">56</reflink>]; Hwang et al., [<reflink idref="bib21" id="ref66">21</reflink>]; Yan & Brown, [<reflink idref="bib61" id="ref67">61</reflink>]). Therefore, students should be allowed to revisit self-assessment questions whenever they seek information, guidance, or timely feedback (Lewis et al., [<reflink idref="bib26" id="ref68">26</reflink>]; Sebesta & Speth, [<reflink idref="bib41" id="ref69">41</reflink>]).</p> <p>However, it is not a good strategy to use online non-graded assessment types, such as knowledge check questions, without backed-up pedagogies and design choices (Arend, [<reflink idref="bib5" id="ref70">5</reflink>]; Bhebhe & Maphosa, [<reflink idref="bib9" id="ref71">9</reflink>]). Knowledge check questions within retrieval practice were designed and integrated into the course by considering <emph>spacing</emph>, <emph>feedback</emph>, and <emph>desirable difficulties</emph> (Authors, 2021). Dunlosky et al. ([<reflink idref="bib13" id="ref72">13</reflink>]) argued that multiple self-assessment tests providing feedback with particular space between them are more beneficial for the performance on the final evaluation. Also, the frequency of attempting to answer self-assessment questions and the number of correctly answered questions are significantly and positively correlated with high performance on the final evaluation (Dunlosky et al., [<reflink idref="bib13" id="ref73">13</reflink>]). As seen, the non-graded knowledge check questions as additional practices supporting student learning enable actively participating in the learning process (Andrade, [<reflink idref="bib4" id="ref74">4</reflink>]; Bhebhe & Maphosa, [<reflink idref="bib9" id="ref75">9</reflink>]; Eddy & Hogan, [<reflink idref="bib14" id="ref76">14</reflink>]; Freeman et al., [<reflink idref="bib18" id="ref77">18</reflink>]; Kearns, [<reflink idref="bib22" id="ref78">22</reflink>]). Self-assessment questions can be scored, but those scores do not need to be aggregated in the final grade as in Lewis et al. ([<reflink idref="bib26" id="ref79">26</reflink>]). Contrarily, they can merely provide feedback as a source of external evidence to instructors to monitor students' learning and understanding (Lewis et al., [<reflink idref="bib26" id="ref80">26</reflink>]; Yan & Brown, [<reflink idref="bib61" id="ref81">61</reflink>]). In our study, we also scored correctly answered self-assessment knowledge check questions, but the given scores were not aggregated to the final grade in the online course.</p> <p>According to Kearns ([<reflink idref="bib22" id="ref82">22</reflink>]), knowledge check type questions like our Check Your Understanding questions allow instructors to asses student learning and provide feedback to correct misconceptions. By positioning these questions after certain topics and at the end of sections, we also use them to help students manage their cognitive load. When students choose to complete the questions, they pause for a minute to read and answer the question and then read over the feedback for the question before moving on to the next topic or section. Self-assessments allow students to pause and reflect on what they learned (Bonk & Zhang, [<reflink idref="bib10" id="ref83">10</reflink>]; Hwang et al., [<reflink idref="bib21" id="ref84">21</reflink>]) and to adjust their understanding through judgments based on the answer and the feedback provided (Ally, [<reflink idref="bib1" id="ref85">1</reflink>]; Andrade, [<reflink idref="bib4" id="ref86">4</reflink>]; Yan & Brown, [<reflink idref="bib61" id="ref87">61</reflink>]). Wilkie and their co-authors (2018) stated that to help learners gauge their understanding, it is helpful to provide activities such as single/multiple choice quizzes, fill in the blianks, and true/false questions at regular intervals. In addition, knowledge checks serve as preparation for learners to tackel larger exams and other assessments they may encounter in the future. (Wilkie et al., [<reflink idref="bib49" id="ref88">49</reflink>]). Self-assessment questions with delayed or immediate feedback create an interactive platform and scaffold students' learning and performance (Hwang et al., [<reflink idref="bib21" id="ref89">21</reflink>]; Lewis et al., [<reflink idref="bib26" id="ref90">26</reflink>]). In our case, we used a combination of multiple-choice, True/False questions, and multiple select questions. The questions we use are the same or similar to the questions that show up in the high-stakes quizzes removing the element of surprise that was also mentioned in Wilkie et al. ([<reflink idref="bib49" id="ref91">49</reflink>]).</p> <p>All of our Check Your Understanding questions, which are known as knowledge check questions, were created using H5P. H5P enables easy creation, sharing, and reusability of interactive HTML5 content using JavaScript (<emph>H5P</emph>, n.d.). H5P allows us to provide immediate feedback for each question so students can correct any faulty knowledge or misconceptions before they move on to the next topic or take the weekly quiz. Students can also answer the questions multiple times if they choose to prepare for a quiz or exam. We use the H5P module available for Drupal, which is our content management system, to create the knowledge check questions to allow us to track student usage.</p> <hd id="AN0181464865-8">Research questions</hd> <p>We ran a study to determine whether the knowledge checks embedded in the course content impacted student assessment scores in a positive manner. We also wanted to know which types of assessments were impacted the most. The study was run in five different courses. This study shares the results and findings of the largest of the five courses, Earth in the Future. The following research questions guided the study:</p> <p></p> <ulist> <item> Does time spent on knowledge check questions, scores for correctly answered knowledge check questions, frequency of answering knowledge check questions, quizzes, capstone, and lab assignments affect overall success in an online course?</item> <p></p> <item> Is there an interaction effect between knowledge check questions-related variables, quizzes, capstone, and lab assignments on success in an online course?</item> <p></p> <item> Do knowledge check questions-related variables affect quiz, capstone, and lab assignment achievement?</item> <p></p> <item> Is there an indirect effect of knowledge check questions-related variables on overall success in the course?</item> </ulist> <hd id="AN0181464865-9">Method</hd> <p></p> <hd id="AN0181464865-10">Participants and setting</hd> <p>The setting for this study was an online entry-level undergraduate Earth Science course offered at [The University Name: removed for blind review] both to resident students and students in its [The online campus name: removed for blind review]. The study setting covered various topics: climate change in the past, recent climate change, climate systems on Earth, general circulation models, global carbon cycle, ocean circulation and its impact on climate, ocean acidification, water resources and climate change, food supply and climate change, rising seas, terrestrial ecosystems in peril, and adaptation and mitigation. Participants were assessed through weekly quizzes with multiple choice questions, lab assignments with data and climate models, a capstone project through six checkpoints for the impact of climate change, and two midterms. Four hundred and thirty-six undergraduates were invited to the study. A total of two hundred and eighty undergraduates participated (i.e., n = 280, which accounts for 64% of the overall targeted group).</p> <hd id="AN0181464865-11">Measures</hd> <p>For the <emph>capstone assignment</emph>, six entries related to the topics discussed in the modules were required.</p> <p>There were eight <emph>lab exercises</emph>. The lowest lab exercise score was dropped while calculating the final score.</p> <p>There were twelve <emph>quizzes</emph> in the course. The lowest quiz score was dropped while calculating the final score.</p> <p>Knowledge check questions were provided as supplementary materials to support the learning process. Time spent, the scores for correctly answered, and the frequency of answering knowledge check questions were independent variables in addition to the quizzes, lab assignments, and capstone project. <emph>Time spent</emph> was considered at the level of none to 1 week (i.e., 7 days between modules). One point was given for correctly answering knowledge check questions; however, participants were informed that <emph>the scores for correctly answered knowledge check questions</emph> would not be included in the overall final grade. <emph>Frequency</emph> is the number of times that participants answered the knowledge check questions.</p> <hd id="AN0181464865-12">Data analysis</hd> <p>We used binary logistic regression analysis to detect the effects of knowledge check questions, quizzes, capstone, and lab assignments on student success. Dummy variables were created for logistic regression: successful (<reflink idref="bib1" id="ref92">1</reflink>) = final score is 90 or above and unsuccessful (0) = final score is below 90. Logistic regression can be used to predict new cases as well as to identify the relationships between independent variables and the outcome variable (Pituch & Stevens, [<reflink idref="bib30" id="ref93">30</reflink>]). A score of 90 or above is indicative of exceptional understanding and mastery of the course material, aligning with the rigorous performance benchmarks set by our institution. This high standard is not just a measure of proficiency but a mark of distinction, separating good performance from high achievement. It is important to note that our threshold is high, but our data set is large and robust enough to ensure the validity of our findings.</p> <p>While investigating the interaction effect on success in an online course, we used a continuous-by-continuous interaction model within linear regression with moderator variables. Moderator variable analyses help to understand the effect in-depth (Fritz & Arthur, [<reflink idref="bib19" id="ref94">19</reflink>]). Moderator variables moderate the relation between independent and outcome variables, and the effect of moderator and independent variables together is called interaction (Fritz & Arthur, [<reflink idref="bib19" id="ref95">19</reflink>]). To continue conducting a continuous-by-continuous interaction model within linear regression with moderator variables, we centered the means for each independent variable (new variable = old variable – mean of the old variable) and created an interaction term by multiplying centered means (i.e., the interaction term for quiz and capstone = centered quiz mean × centered capstone mean).</p> <p>To investigate the effects of time spent, scores for correctly answered, frequency of answering knowledge check questions on quizzes, capstone, and lab assignment achievements, we conducted multiple regression analyses. Multiple regression analysis is used for the investigation of an existing effect, measuring the magnitude of the effect, and predicting the effect (Rubinfeld, [<reflink idref="bib38" id="ref96">38</reflink>]).</p> <p>Path analysis is used to investigate the direct, indirect, and total effects of exogenous variables on endogenous variables in a hypothesized model (Lleras, [<reflink idref="bib27" id="ref97">27</reflink>]). In particular, we tested different models to investigate the indirect effect of knowledge check questions on overall success through path analysis. We created the model presented in this study based on previous literature and empirical tests, discussed the previous analyses, reported fit indices, provided the model's illustration, and discussed the results of path analysis with other analyses conducted (Stage et al., [<reflink idref="bib45" id="ref98">45</reflink>]).</p> <hd id="AN0181464865-13">Results</hd> <p></p> <hd id="AN0181464865-14">RQ1. Do time spent on knowledge check questions, scores for correctly answered knowledge chec...</hd> <p>A binary logistic regression was conducted to investigate the effects of time spent on knowledge check questions (M = 10,597.05, SD = 19,047.67), scores for correctly answered knowledge check questions (M = 24.19, SD = 15.28), frequency of answering knowledge check questions (M = 32.61, SD = 13.89) questions, quiz score (M = 84.74, SD = 10.27), capstone score (M = 95.18, SD = 13.60), and lab assignment score (M = 86.37, SD = 9.41) on the success in the online course. Omnibus test of model coefficients (χ2(<reflink idref="bib6" id="ref99">6</reflink>) = 266.872, p < 0.001) showed that the model was statistically significant. Hosmer and Lemeshow test (χ2(<reflink idref="bib8" id="ref100">8</reflink>) = 6.734, p = 0.566) indicated the fit of predictions made by the model through observed memberships to the unsuccessful and successful groups. The model explained 84.6% (Nagelkerke R2) of the variation in the success in the online course. The model's accuracy rate in the classification of unsuccessful and successful groups is 93.2%. The classification table (see Table 2) showed that the accuracy rate of predicting successful participants is 91.8% (sensitivity), and the accuracy rate of predicting unsuccessful participants is 94% (specificity) with this model. The positive predictive value is 89.11%, indicating all cases were correctly predicted as successful in the online course. The negative predictive value is 95.53%, indicating that all cases were correctly predicted as unsuccessful.</p> <p>Table 2 Classification table</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left" /><th align="left" colspan="3"><p>Predicted</p></th></tr><tr><th align="left" rowspan="2"><p>Observed</p></th><th align="left" /><th align="left" colspan="2"><p>success</p></th><th align="left" rowspan="2"><p>Percentage correct</p></th></tr><tr><th align="left" /><th align="left"><p>unsuccessful</p></th><th align="left"><p>successful</p></th></tr></thead><tbody><tr><td align="left" rowspan="2"><p>success</p></td><td align="left"><p>unsuccessful</p></td><td align="left"><p>171</p></td><td align="left"><p>11</p></td><td char="." align="char"><p>94.0</p></td></tr><tr><td align="left"><p>successful</p></td><td align="left"><p>8</p></td><td align="left"><p>90</p></td><td char="." align="char"><p>91.8</p></td></tr><tr><td align="left"><p>Overall percentage</p></td><td align="left" /><td align="left" /><td align="left" /><td char="." align="char"><p>93.2</p></td></tr></tbody></table> </ephtml> </p> <p>The cut value is 0.500</p> <p>Quiz, capstone, and lab assignment significantly predicted the success in the online course (see Table 3). An increase in quiz, capstone, and lab assignment scores was associated with an increased likelihood of being successful. When the capstone score increases one unit (when all other variables are constant), the final score increases by 0.257 points. When the quiz score increases one unit (when all other variables are constant), the final score increases by 0.453 points. When the lab assignment score increases one unit (when all other variables are constant), the final score increases by 0.350 points.</p> <p>Table 3 Logistic regression analysis results</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left" /><th align="left" /><th align="left" /><th align="left" /><th align="left" /><th align="left" /><th align="left" colspan="2"><p>95% C. I. for EXP(B)</p></th></tr><tr><th align="left" /><th align="left"><p>B</p></th><th align="left"><p>S.E</p></th><th align="left"><p>Wald</p></th><th align="left"><p>df</p></th><th align="left"><p>p</p></th><th align="left"><p>Exp(B)</p></th><th align="left"><p>lower</p></th><th align="left"><p>upper</p></th></tr></thead><tbody><tr><td align="left"><p>Capstone</p></td><td char="." align="char"><p>0.257</p></td><td char="." align="char"><p>0.075</p></td><td char="." align="char"><p>11.704</p></td><td align="left"><p>1</p></td><td align="left"><p> < 0.001</p></td><td char="." align="char"><p>1.294</p></td><td char="." align="char"><p>1.116</p></td><td char="." align="char"><p>1.499</p></td></tr><tr><td align="left"><p>Quiz</p></td><td char="." align="char"><p>0.453</p></td><td char="." align="char"><p>0.087</p></td><td char="." align="char"><p>27.392</p></td><td align="left"><p>1</p></td><td align="left"><p> < 0.001</p></td><td char="." align="char"><p>1.573</p></td><td char="." align="char"><p>1.328</p></td><td char="." align="char"><p>1.864</p></td></tr><tr><td align="left"><p>Lab assignment</p></td><td char="." align="char"><p>0.350</p></td><td char="." align="char"><p>0.073</p></td><td char="." align="char"><p>23.244</p></td><td align="left"><p>1</p></td><td align="left"><p> < 0.001</p></td><td char="." align="char"><p>1.419</p></td><td char="." align="char"><p>1.231</p></td><td char="." align="char"><p>1.636</p></td></tr><tr><td align="left"><p>Time spent</p></td><td char="." align="char"><p>0.000</p></td><td char="." align="char"><p>0.000</p></td><td char="." align="char"><p>0.081</p></td><td align="left"><p>1</p></td><td align="left"><p>1.000</p></td><td char="." align="char"><p>1.000</p></td><td char="." align="char"><p>1.000</p></td><td char="." align="char"><p>1.000</p></td></tr><tr><td align="left"><p>Scores</p></td><td char="." align="char"><p>-0.002</p></td><td char="." align="char"><p>0.029</p></td><td char="." align="char"><p>0.003</p></td><td align="left"><p>1</p></td><td align="left"><p>0.998</p></td><td char="." align="char"><p>0.998</p></td><td char="." align="char"><p>0.943</p></td><td char="." align="char"><p>1.057</p></td></tr><tr><td align="left"><p>Frequency</p></td><td char="." align="char"><p>0.039</p></td><td char="." align="char"><p>0.033</p></td><td char="." align="char"><p>1.397</p></td><td align="left"><p>1</p></td><td align="left"><p>1.040</p></td><td char="." align="char"><p>1.040</p></td><td char="." align="char"><p>0.975</p></td><td char="." align="char"><p>1.110</p></td></tr><tr><td align="left"><p>constant</p></td><td char="." align="char"><p>-99.388</p></td><td char="." align="char"><p>14.460</p></td><td char="." align="char"><p>47.243</p></td><td align="left"><p>1</p></td><td align="left"><p>0.000</p></td><td char="." align="char"><p>0.000</p></td><td char="." align="char" /><td char="." align="char" /></tr></tbody></table> </ephtml> </p> <hd id="AN0181464865-15">RQ2. Is there an interaction effect between knowledge check questions-related variables, quiz...</hd> <p>After determining the prediction rates of successful and unsuccessful groups and the significant variable indicating the effect on and the change in the final score, we conducted linear regression analyses with continuous variables (continuous-by-continuous) to investigate the existence of an interaction effect on success in the online course.</p> <hd id="AN0181464865-16">Interaction between quiz and lab assignment</hd> <p>This model explains 84.6% of the variation, F (<reflink idref="bib3" id="ref101">3</reflink>, 276) = 512.776, tolerance > 0.25, VIF < 5. There is a <emph>positive interaction</emph> between the quiz and the lab assignment. Therefore, the influence of the quiz on the overall success in the online course depends on the lab assignment score and vice versa. When the lab assignment score changes, the effect of the quiz on the overall success changes, and vice versa. The amount of change is + 0.004 (see Table 4).</p> <p>Table 4 Results of interaction analyses</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left" /><th align="left"><p>Unstandardized B</p></th><th align="left"><p>Standardized coefficients</p></th><th align="left" colspan="2"><p>t</p></th><th align="left"><p>p</p></th></tr></thead><tbody><tr><td align="left" rowspan="5"><p>Quiz & lab assignment interaction</p></td><td align="left"><p>Quiz*</p></td><td align="left"><p>0.465</p></td><td align="left"><p>0.531</p></td><td align="left" colspan="2"><p>15.335</p></td><td align="left"><p> < 0.001</p></td></tr><tr><td align="left"><p>Lab assign*</p></td><td align="left"><p>0.516</p></td><td align="left"><p>0.541</p></td><td align="left" colspan="2"><p>15.134</p></td><td align="left"><p> < 0.001</p></td></tr><tr><td align="left"><p>Interaction</p></td><td align="left"><p><bold>0.004</bold></p></td><td align="left"><p>0.123</p></td><td align="left" colspan="2"><p>4.068</p></td><td align="left"><p> < 0.001</p></td></tr><tr><td align="left"><p>constant</p></td><td align="left"><p>85.387</p></td><td align="left" /><td align="left" colspan="2"><p>388.656</p></td><td align="left"><p>0.000</p></td></tr><tr><td align="left" colspan="6"><p>R = 0.921, R<sup>2</sup> = 0.848, Adjusted R<sup>2</sup> = 0.846, Std. Error of the Estimate = 3.52323</p></td></tr><tr><td align="left" rowspan="5"><p>Capstone & lab assignment interaction</p></td><td align="left"><p>Capstone*</p></td><td align="left"><p>0.318</p></td><td align="left"><p>0.482</p></td><td align="left" colspan="2"><p>16.891</p></td><td align="left"><p> < 0.001</p></td></tr><tr><td align="left"><p>Lab assign*</p></td><td align="left"><p>0.630</p></td><td align="left"><p>0.660</p></td><td align="left" colspan="2"><p>26.137</p></td><td align="left"><p> < 0.001</p></td></tr><tr><td align="left"><p>Interaction*</p></td><td align="left"><p><bold>0.002</bold></p></td><td align="left"><p>0.085</p></td><td align="left" colspan="2"><p>2.975</p></td><td align="left"><p>0.003</p></td></tr><tr><td align="left"><p>constant</p></td><td align="left"><p>85.502</p></td><td align="left" /><td align="left" colspan="2"><p>429.636</p></td><td align="left"><p>0.000</p></td></tr><tr><td align="left" colspan="6"><p>R = 0.934, R<sup>2</sup> = 0.872, Adjusted R<sup>2</sup> = 0.870, Std. Error of the Estimate = 3.23569</p></td></tr><tr><td align="left" rowspan="5"><p>Capstone & total scores for knowledge check questions interaction</p></td><td align="left"><p>Capstone*</p></td><td align="left"><p>0.425</p></td><td align="left"><p>0.643</p></td><td align="left" colspan="2"><p>14.815</p></td><td align="left"><p> < 0.001</p></td></tr><tr><td align="left"><p>Scores*</p></td><td align="left"><p>0.146</p></td><td align="left"><p>0.249</p></td><td align="left" colspan="2"><p>6.493</p></td><td align="left"><p> < 0.001</p></td></tr><tr><td align="left"><p>Interaction</p></td><td align="left"><p><bold>-0.004</bold></p></td><td align="left"><p>-0.097</p></td><td align="left" colspan="2"><p>-2.274</p></td><td align="left"><p>0.024</p></td></tr><tr><td align="left"><p>constant</p></td><td align="left"><p>85.799</p></td><td align="left" /><td align="left" colspan="2"><p>250.178</p></td><td align="left"><p>0.000</p></td></tr><tr><td align="left" colspan="6"><p>R = 0.783, R<sup>2</sup> = 0.613, Adjusted R<sup>2</sup> = 0.608, Std. Error of the Estimate = 5.62204</p></td></tr><tr><td align="left" colspan="5"><p>Quiz & capstone interaction</p></td><td align="left" rowspan="9" colspan="2"><p>No interaction</p></td></tr><tr><td align="left" colspan="5"><p>Quiz & time spent on knowledge check questions interaction</p></td></tr><tr><td align="left" colspan="5"><p>Quiz & total scores for knowledge check questions interaction</p></td></tr><tr><td align="left" colspan="5"><p>Quiz & frequency of answering knowledge check questions interaction</p></td></tr><tr><td align="left" colspan="5"><p>Lab assignment & time spent on knowledge check questions interaction</p></td></tr><tr><td align="left" colspan="5"><p>Lab assignment & total scores for knowledge check questions interaction</p></td></tr><tr><td align="left" colspan="5"><p>Lab assignment & frequency of answering knowledge check questions interaction</p></td></tr><tr><td align="left" colspan="5"><p>Capstone & time spent on knowledge check questions interaction</p></td></tr><tr><td align="left" colspan="5"><p>Capstone & frequency of answering knowledge check questions interaction</p></td></tr></tbody></table> </ephtml> </p> <p>The amount of change as a result of interactions was highlighted using bold text *Centered mean</p> <p>Regression lines for the predicted value of success in the online course based on quiz and lab assignment scores were given in Fig. 7 and Fig. 8, respectively. The groups (i.e., buckets) were determined based on being unsuccessful (below 90) and successful (90 and above). The significance levels of simple slopes were analyzed by simple linear regression.</p> <p>Graph: Fig. 7 Regression lines for successful and unsuccessful groups based on their lab assignment success</p> <p>Graph: Fig. 8 Regression lines for successful and unsuccessful groups based on their quiz success</p> <p>The blue regression line (see Fig. 7) represents the unsuccessful group, and the red regression line represents the successful group in the lab assignment. For the unsuccessful group, the quiz effect is statistically significant on the success in the course, F (<reflink idref="bib1" id="ref102">1</reflink>, 163) = 280.348, p < 0.001. For the successful group in lab assignment, the quiz effect is statistically significant on the success the course, F (<reflink idref="bib1" id="ref103">1</reflink>, 113) = 242.982, p < 0.001. Since the slope is higher for the unsuccessful group in lab assignment, the quiz effect on success is stronger for those being unsuccessful in lab assignment.</p> <p>The blue regression line (see Fig. 8) represents the unsuccessful group, and the red regression line represents the successful group in the quiz. For the unsuccessful group in the quiz, the lab assignment effect is statistically significant on the overall success, F (<reflink idref="bib1" id="ref104">1</reflink>, 183) = 371.718, p < 0.001. For the successful group in the quiz, the lab assignment effect is statistically significant on the overall success, F (<reflink idref="bib1" id="ref105">1</reflink>, 93) = 99.041, p < 0.001. Since the slope is higher for the unsuccessful group in the quiz, the lab assignment effect on success is stronger for those who are unsuccessful in the quiz.</p> <hd id="AN0181464865-17">Interaction between capstone and lab assignment</hd> <p>This model explains 87% of the variation, F (<reflink idref="bib3" id="ref106">3</reflink>, 276) = 625.036, tolerance > 0.25, VIF < 5. There is a <emph>positive interaction</emph> between capstone and lab assignment. Therefore, the influence of the capstone on the overall success in the course depends on the lab assignment score and vice versa. When the lab assignment score changes, the effect of the capstone on the overall success changes, and vice versa. The amount of change is + 0.002 (see Table 4).</p> <p>Regression lines for the predicted value of success based on capstone and lab assignment scores are given in Fig. 9 and Fig. 10, respectively. The groups (i.e., buckets) were determined based on being unsuccessful (below 90) and successful (90 and above). The significance levels of simple slopes were analyzed by simple linear regression.</p> <p>Graph: Fig. 9 Regression lines for successful and unsuccessful groups based in their lab assignment success</p> <p>Graph: Fig. 10 Regression lines for successful and unsuccessful groups based on their capstone success</p> <p>The blue regression line represents the unsuccessful group, and the red regression line represents the successful group in the lab assignment (see Fig. 9). For the unsuccessful group in lab assignment, the capstone effect is statistically significant on the overall success, F (<reflink idref="bib1" id="ref107">1</reflink>, 163) = 219.433, p < 0.001. For the successful group in the lab assignment, the capstone effect is statistically significant on the overall success, F (<reflink idref="bib1" id="ref108">1</reflink>, 113) = 135.771, p < 0.001. Since the slope is higher for the unsuccessful group in the lab assignment, the capstone effect on overall success is stronger for those who are unsuccessful in the lab assignment.</p> <p>The blue regression line represents the unsuccessful group, and the red regression line represents the successful group in the capstone (see Fig. 10). For the unsuccessful group in capstone, the lab assignment effect is statistically significant on overall success, F (<reflink idref="bib1" id="ref109">1</reflink>, 34) = 78.213, p < 0.001. For the successful group in the capstone, the lab assignment effect is statistically significant on the overall success, F (<reflink idref="bib1" id="ref110">1</reflink>, 242) = 748.309, p < 0.001. Since the slope is higher for the unsuccessful group in the capstone, the lab assignment effect on success is stronger for those students unsuccessful in the capstone.</p> <hd id="AN0181464865-18">Interaction between capstone and scores</hd> <p>This model explains 60.8% of the variation, F (<reflink idref="bib3" id="ref111">3</reflink>, 276) = 145.513, tolerance > 0.25, VIF < 5. There is a <emph>negative interaction</emph> between the capstone and the scores for correctly answered knowledge check questions. Therefore, the influence of the capstone on the overall success in the course depends on the scores and vice versa. When the scores change, the effect of the capstone on the overall success changes, and vice versa. The amount of change is − 0.004 (see Table 4). The scores moderated the effect of capstone on success.</p> <p>Regression lines for the predicted value of success in the course based on the capstone and scores were given in Fig. 11. The groups (i.e., buckets) were determined based on being unsuccessful (below 90) and successful (90 and above) in the capstone. The significance levels of simple slopes were analyzed by simple linear regression.</p> <p>Graph: Fig. 11 Regression lines for successful and unsuccessful groups based on their capstone success</p> <p>The blue regression line represents the unsuccessful group, and the red regression line represents the successful group in the capstone (see Fig. 11). For the unsuccessful group in capstone, the total score effect is statistically significant on the overall success, F (<reflink idref="bib1" id="ref112">1</reflink>, 34) = 7.979, p < 0.01. For the successful group in the capstone, the total score effect is statistically significant on the overall success, F (<reflink idref="bib1" id="ref113">1</reflink>, 242) = 35.232, p < 0.001. Since the slope is higher for the unsuccessful group in the capstone, the effect of scores on student success is stronger for those being unsuccessful in the capstone.</p> <hd id="AN0181464865-19">RQ3. Do time spent on, scores for correctly answered, frequency of answering knowledge check...</hd> <p>After investigating the interaction effects (i.e., moderating the effect on overall success), we investigated the effects of time spent on, scores for, and frequency of knowledge check questions on the quiz, the capstone, and the lab assignment achievements. Multiple regression analyses (see Table 5) indicated that the scores for knowledge check questions were significantly affecting the quiz (F (<reflink idref="bib3" id="ref114">3</reflink>, 276) = 16.584, p < 0.01, B = 0.167, tolerance > 0.25, VIF < 5) and lab assignment (F (<reflink idref="bib3" id="ref115">3</reflink>, 276) = 12.226, p < 0.001, B = 0.198, tolerance > 0.25, VIF < 5) scores.</p> <p>Table 5 Multiple regression analyses results</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left"><p>Unstandardized coefficient B</p></th><th align="left"><p>Unstandardized coefficients Std. Error</p></th><th align="left"><p>Standardized coefficient B</p></th><th align="left"><p>t</p></th><th align="left"><p>p</p></th></tr></thead><tbody><tr><td align="left" colspan="6"><p>Quiz (R = 0.391, R<sup>2</sup> = 0.153, adj. R<sup>2</sup> = 0.144)</p></td></tr><tr><td align="left"><p> Time spent</p></td><td align="left"><p>5.249E-5</p></td><td char="." align="char"><p>0.000</p></td><td char="." align="char"><p>0.097</p></td><td char="." align="char"><p>1.648</p></td><td char="." align="char"><p>0.100</p></td></tr><tr><td align="left"><p> Scores</p></td><td align="left"><p>0.167</p></td><td char="." align="char"><p>0.061</p></td><td char="." align="char"><p>0.249</p></td><td char="." align="char"><p>2.759</p></td><td char="." align="char"><p>0.006</p></td></tr><tr><td align="left"><p> Frequency</p></td><td align="left"><p>0.085</p></td><td char="." align="char"><p>0.066</p></td><td char="." align="char"><p>0.115</p></td><td char="." align="char"><p>1.295</p></td><td char="." align="char"><p>0.197</p></td></tr><tr><td align="left"><p> Constant</p></td><td align="left"><p>77.361</p></td><td char="." align="char"><p>1.471</p></td><td char="." align="char" /><td char="." align="char"><p>52.579</p></td><td char="." align="char"><p> < 0.001</p></td></tr><tr><td align="left" colspan="6"><p>Lab assignment (R = 0.342, R<sup>2</sup> = 0.117, adj. R<sup>2</sup> = 0.108)</p></td></tr><tr><td align="left"><p> Time spent</p></td><td align="left"><p>-1.179E-5</p></td><td char="." align="char"><p>0.000</p></td><td char="." align="char"><p> − 0.024</p></td><td char="." align="char"><p> − 0.395</p></td><td char="." align="char"><p>0.693</p></td></tr><tr><td align="left"><p> Scores</p></td><td align="left"><p>0.198</p></td><td char="." align="char"><p>0.057</p></td><td char="." align="char"><p>0.321</p></td><td char="." align="char"><p>3.487</p></td><td char="." align="char"><p> < 0.001</p></td></tr><tr><td align="left"><p> Frequency</p></td><td align="left"><p>0.025</p></td><td char="." align="char"><p>0.062</p></td><td char="." align="char"><p>0.037</p></td><td char="." align="char"><p>0.403</p></td><td char="." align="char"><p>0.687</p></td></tr><tr><td align="left"><p> Constant</p></td><td align="left"><p>80.909</p></td><td char="." align="char"><p>1.377</p></td><td char="." align="char" /><td char="." align="char"><p>58.761</p></td><td char="." align="char"><p> < 0.001</p></td></tr><tr><td align="left" colspan="6"><p> Capstone (R = 0.215, R<sup>2</sup> = 0.046, adj. R<sup>2</sup> = 0.036)</p></td></tr><tr><td align="left"><p> Time spent</p></td><td align="left"><p>1.576E-5</p></td><td char="." align="char"><p>0.000</p></td><td char="." align="char"><p>0.022</p></td><td char="." align="char"><p>0.352</p></td><td char="." align="char"><p>0.725</p></td></tr><tr><td align="left"><p> Scores</p></td><td align="left"><p>0.099</p></td><td char="." align="char"><p>0.085</p></td><td char="." align="char"><p>0.111</p></td><td char="." align="char"><p>1.161</p></td><td char="." align="char"><p>0.247</p></td></tr><tr><td align="left"><p> Frequency</p></td><td align="left"><p>0.106</p></td><td char="." align="char"><p>0.093</p></td><td char="." align="char"><p>0.108</p></td><td char="." align="char"><p>1.140</p></td><td char="." align="char"><p>0.255</p></td></tr><tr><td align="left"><p> Constant</p></td><td align="left"><p>89.178</p></td><td char="." align="char"><p>2.068</p></td><td char="." align="char" /><td char="." align="char"><p>43.125</p></td><td char="." align="char"><p> < 0.001</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0181464865-20">RQ4. Is there an indirect effect of knowledge check questions-related variables on overall su...</hd> <p>Model fit indices (χ2 = 2.261, df = 1, p = 0.133; CFI = 0.998 > 0.9, TLI = 0.990 > 0.9, GFI = 0.996 > 0.9, RMSEA = 0.067 < 0.08) indicate a goodness of fit for the hypothesized model (see Fig. 12 for the final hypothesized model). In this model, all hypotheses were found significant (p < 0.001). The standardized indirect effect of the scores for correctly answered knowledge check questions on overall success was 0.353, meaning that when the scores for correctly answered knowledge check questions increased one-unit, overall success increased by 0.353. See Table 6 for the standardized regression coefficients of direct, indirect, and total effects.</p> <p>Graph: Fig. 12 The path model with standardized regression coefficients for direct and indirect effects</p> <p>Table 6 Standardized regression coefficients of direct, indirect, and total effects</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Hypotheses</p></th><th align="left"><p>Direct effect</p></th><th align="left"><p>Indirect effect</p></th><th align="left"><p>Total effect</p></th></tr></thead><tbody><tr><td align="left"><p>Knowledge check scores → lab assignment success</p></td><td char="." align="char"><p>0.341</p></td><td char="." align="char"><p>–</p></td><td char="." align="char"><p>0.341</p></td></tr><tr><td align="left"><p>Knowledge check scores → quiz success</p></td><td char="." align="char"><p>0.144</p></td><td char="." align="char"><p>0.227</p></td><td char="." align="char"><p>0.372</p></td></tr><tr><td align="left"><p>Knowledge check scores → overall success</p></td><td char="." align="char"><p>–</p></td><td char="." align="char"><p>0.353</p></td><td char="." align="char"><p>0.353</p></td></tr><tr><td align="left"><p>Quiz success → overall success</p></td><td char="." align="char"><p>0.497</p></td><td char="." align="char"><p>–</p></td><td char="." align="char"><p>0.497</p></td></tr><tr><td align="left"><p>Lab assignment success → quiz success</p></td><td char="." align="char"><p>0.666</p></td><td char="." align="char"><p>–</p></td><td char="." align="char"><p>0.666</p></td></tr><tr><td align="left"><p>Lab assignment success → overall success</p></td><td char="." align="char"><p>0.492</p></td><td char="." align="char"><p>0.331</p></td><td char="." align="char"><p>0.823</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0181464865-21">Discussion</hd> <p>After examining the results (see Table 7 for a summary), we identified three strong predictors of overall success. Of the three, quiz (the remember phase in Bloom) success was the strongest positive predictor for overall success. This result aligns with the importance of quiz, particularly weekly quizzes, in online learning (El-Hashash, [<reflink idref="bib15" id="ref116">15</reflink>]; Salas-Morera et al., [<reflink idref="bib60" id="ref117">60</reflink>]). Lab assignments (the application phase in Bloom) were a strong and positive predictor. Finally, the capstone (the create phase in Bloom) was both a significant and strong predictor of overall success in the course. The integration of the three assessment types designed in a way to represent different phases in Bloom's taxonomy has the potential to positively influence student achievement in the online course.</p> <p>Table 7 Summary of the results</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left"><p>Findings</p></th></tr></thead><tbody><tr><td align="left" rowspan="2"><p>RQ1</p></td><td align="left"><p>a. Quiz, capstone, and lab assignment scores significantly predicted the overall success level</p></td></tr><tr><td align="left"><p>b. The model with knowledge check questions related variables, quiz, capstone, and lab assignments predicted 93.2% of the participants' success classification. This model's sensitivity and specificity were high, as well</p></td></tr><tr><td align="left" rowspan="2"><p>RQ2</p></td><td align="left"><p>a. There were interaction effects between quiz & lab assignment (positive), lab assignment & capstone (positive), and capstone and scores for correctly answered knowledge check questions (negative)</p></td></tr><tr><td align="left"><p>b. As for those who are in the low success group of any of these variables (i.e., quiz, lab, capstone), they benefitted more from the moderator variable to increase the likelihood of being successful in an online course</p></td></tr><tr><td align="left"><p>RQ3</p></td><td align="left"><p>The total score given for the correctly answered knowledge check questions significantly and positively affected the quiz and lab assignment achievements, but not capstone achievement</p></td></tr><tr><td align="left"><p>RQ4</p></td><td align="left"><p>Total score for correctly answered knowledge check questions indirectly and positively affected overall success</p></td></tr></tbody></table> </ephtml> </p> <p>Scores were given for each correctly answered knowledge check question which we found had a positive effect on quiz, lab assignments, and overall success. Therefore, our result is more specific compared to the correlation between frequency of visiting course webpages and quiz success (Ramos & Yudko, [<reflink idref="bib33" id="ref118">33</reflink>]). Please note that students knew that the points earned on the knowledge check questions would not impact their final grade because we did not aggregate the scores given to correctly answered questions into the final grade. Lewis et al. ([<reflink idref="bib26" id="ref119">26</reflink>]) did not also add points awarded to the final grade in their study. The participants could have just skipped answering the questions and still received their two-point extra credit for participating in the study. So, students were triggered by intrinsic motivation (i.e., being successful) rather than extrinsic motivation (e.g., earning an extrinsic reward or avoiding punishment). The positive impact of correctly answered knowledge check questions on quizzes and lab assignments is a significant indicator of learner-content interaction in the course. Previous studies (e.g., Dunlosky et al., [<reflink idref="bib13" id="ref120">13</reflink>]; Zimmerman, [<reflink idref="bib51" id="ref121">51</reflink>]) indicated that the time spent on online course materials, frequency of checking learning materials, and the number of correctly answered supplementary questions affect student achievement and performance. Therefore, knowledge check questions, used for self-assessment, might create and support learner-content interaction by triggering intrinsic motivation for the sake of achievements in other assessments in an online course.</p> <p>In addition to the positive effects, we found that the knowledge check questions (i.e., self-check assessments) did not influence capstone success. There was also a negative interaction between capstone success and scores for correctly answered knowledge check questions. This could be due to the context of the two. Capstone projects generally aim to apply the knowledge learned and synthesized into the real-world context (e.g., Dere et al., [<reflink idref="bib12" id="ref122">12</reflink>]; Swanson et al., [<reflink idref="bib46" id="ref123">46</reflink>]). Contrarily, knowledge check questions in this course allow students to test whether they understand the topic recently presented before moving on to a new topic. The capstone project was developed to help students synthesize what they have learned to research and evaluate information about a location and then create something new from what they learned. Knowledge check question content and objectives could be added to the course and be aligned with the capstone project's objective and content to turn that negative interaction into a positive one.</p> <hd id="AN0181464865-22">Implications and recommendations</hd> <p></p> <hd id="AN0181464865-23">Design takeaways</hd> <p>There are a few key design recommendations for the application of knowledge check questions based on our experience combined with the findings presented above. The fundamentals of e-learning design encourage learning designers to start the design process by <emph>chunking</emph> or breaking the content into smaller pieces based on learning objectives. First, <emph>chunk</emph> the content into lessons or modules. Then <emph>chunk</emph> the lesson or module into smaller, more digestible topics. Plan to add individual knowledge check questions after topics, especially the more difficult ones or topics that serve as a foundation for other learning. Add knowledge check quizzes, a collection of related questions, at the end of a lesson or a module. To gain the biggest impact, align the knowledge checks with the learning objectives and, when possible, the question type as well. If students will see multiple choice questions on a quiz or exam, use that type of question. After the course is live, run reports regularly to determine if students are using them. The results can be used by the instructor to provide feedback to the students on their performance. Based on the results of the study, completing the knowledge checks showed a positive effect in performance in the class. Encouraging lower-performing students who did not complete the knowledge checks may positively impact their quiz and lab assignment scores. An added benefit of running reports is that the information gathered for the reports can also be used for formative assessment to determine where any adjustments may be needed with the course content.</p> <p>Based on the results of our study, the knowledge checks had a slightly negative interaction with the capstone. This result was not unexpected due to the nature of the capstone. Students were required to use higher-order learning to create their capstone assignments. When a course requires higher-order learning, knowledge checks may not be appropriate.</p> <p>After reviewing the results of the study, a few ideas have materialized that we will explore when we revise the course:</p> <p>At the time of the study, about half of the modules in the course had knowledge check questions in them. The practical reason for doing this study was to determine if adding knowledge check questions was worth the effort and expense it takes to create them. Our findings confirm that it is. The first action that we plan to take is writing and adding knowledge check questions to all the modules in the course.</p> <p>Moving forward, we plan to run a report after each semester to track whether students continue to use the knowledge check questions as well as to create a record for the instructor to use should he need it to counsel students on how to do better in the class. If he sees that students are not utilizing the questions, that might be an easy first step to doing better. Early in the semester, the instructor plans to stress how beneficial completing the knowledge checks is to each student's overall success. We may also investigate creating pre-module knowledge checks to help students better understand what they know and what they need to pay closer attention to as they work their way through the module.</p> <hd id="AN0181464865-24">Implications for the future</hd> <p>The strategy that we used is adaptable to any course. The use of knowledge checks, whether created using H5P, a different tool, or even asked in-person, will have a positive impact on overall student success for any course that has a knowledge acquisition requirement and quizzes as part of the assessment plan.</p> <p>When designing or revising courses, learning designers and faculty can consider using assessment types that have positive interactions to amplify their effect on overall success. The results of this study indicate that quiz and lab assignments as well as lab assignments and capstone project amplify the overall success. If these types of assignments are used in an online course, the combination of them would serve the purpose of increasing overall success.</p> <p>If higher order learning is the main assessment vehicle for a course, the types of knowledge checks that we used in this study will not have a positive impact on that type of assessment. It is best to try a different tactic or find a tool that allows for deeper analysis of student answers, something like an automated feedback system. Automated feedback systems provide suitable, personalized, and fast feedback based on students' answers (Barker, [<reflink idref="bib7" id="ref124">7</reflink>], [<reflink idref="bib8" id="ref125">8</reflink>]).</p> <hd id="AN0181464865-25">Limitations</hd> <p>This study has some limitations. First, it did not directly incorporate demographic data into the analysis. Including detailed demographic information encompassing race and gender would have allowed for a demographic comparison. Second, the study's specific context, which is being conducted within an online geoscience course focused on climate change, limits the generalizability of its findings. Future research could address these limitations by incorporating a broader demographic analysis across various disciplines. Third, although the study uses a limited range of assessment types, each represents different aspects of assessment approaches.</p> <hd id="AN0181464865-26">Conclusions</hd> <p>This study investigated the effect of various assessment types on the overall success of undergraduate students in an asynchronous online course covering climate change. In particular, the effects of quizzes, lab assignments, a capstone project, and knowledge check questions on overall success were examined. The results indicated that quizzes, lab assignments, a capstone project, and knowledge check questions were effective assessment strategies. Knowledge check questions were also effective on quizzes and lab assignment achievements.</p> <p>The results of this study support the design decisions of the online asynchronous geoscience undergraduate course. Learning designers, in any online course, can predict student achievements based on weekly quizzes, lab assignments, and a capstone project and support metacognitive processes via knowledge check questions. Different disciplines may have unique structures and requirements for their courses. For instance, lab assignments may not be a component of online courses in certain disciplines. Learning designers in other disciplines can also predict student achievements based on assessments such as quizzes, capstones, and discipline-specific knowledge check questions. The discipline specific assessments and their predictive nature, along with knowledge check questions, also lead to further research. A capstone project, which is a higher-level assessment type in Bloom's taxonomy, might require a different approach to checking knowledge as knowledge check questions provide just-in-time and delayed feedback.</p> <hd id="AN0181464865-27">Acknowledgements</hd> <p>The original course development costs were supported through a grant from The National Aeronautics and Space Administration (NASA) (TJB Co-PI). The revision to the course and development of four other courses in the series were supported by a National Science Foundation (NSF) grant (TJB Co-PI).</p> <hd id="AN0181464865-28">Funding</hd> <p>No funding was received to assist with the preparation of this manuscript.</p> <hd id="AN0181464865-29">Data availability</hd> <p>Data will be shared upon reasonable request.</p> <hd id="AN0181464865-30">Declarations</hd> <p></p> <hd id="AN0181464865-31">Competing interests</hd> <p>The authors report there are no competing interests to declare. The authors have no relevant financial or non-financial interests to disclose.</p> <hd id="AN0181464865-32">Ethical approval</hd> <p>This study was approved by the Pennsylvania State University's IRB (Study ID: STUDY00010309).</p> <hd id="AN0181464865-33">Informed consent</hd> <p>Consent was taken from the participants.</p> <hd id="AN0181464865-34">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0181464865-35"> <title> References </title> <blist> <bibl id="bib1" idref="ref33" type="bt">1</bibl> <bibtext> Ally MAnderson T. Foundations of educational theory for online learning. The theory and practice of online learning. 2008; Athabasca University Press: 15-44. 10.15215/aupress/9781897425084.003</bibtext> </blist> <blist> <bibl id="bib2" idref="ref22" type="bt">2</bibl> <bibtext> Ametepe JD, Khan N. Teaching physics during COVID-19 pandemic: Implementation and report of teaching strategies to support student learning. Physics Education. 2021; 56; 6. 10.1088/1361-6552/ac266f065030</bibtext> </blist> <blist> <bibl id="bib3" idref="ref7" type="bt">3</bibl> <bibtext> Anderson LW, Krathwohl DR, Airasian PW, Cruikshank KA, Mayer RE, Pintrich PR, Raths J, Wittrock MJ. A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. 2001Abridged; Longman</bibtext> </blist> <blist> <bibl id="bib4" idref="ref58" type="bt">4</bibl> <bibtext> Andrade HL. A critical review of research on student self-assessment. Frontiers in Education. 2019; 4: 1-13. 10.3389/feduc.2019.00087</bibtext> </blist> <blist> <bibl id="bib5" idref="ref53" type="bt">5</bibl> <bibtext> Arend BD. Course assessment practices and student learning strategies in online courses. Journal of Asynchronous Learning Networks. 2007; 11; 4: 3-13</bibtext> </blist> <blist> <bibl id="bib6" idref="ref59" type="bt">6</bibl> <bibtext> Azevedo R, Behnagh RF, Harley JM, Trevors GNaumovska L. Metacognition and self-regulated learning in student-centered learning environments. Theoretical foundations of learning environments. 20122; Routledge: 27</bibtext> </blist> <blist> <bibl id="bib7" idref="ref124" type="bt">7</bibl> <bibtext> Barker T. An automated feedback system based on adaptive testing: Extending the model. International Journal of Emerging Technologies in Learning (iJET). 2010; 5; 2: 11-14. 10.3991/ijet.v5i2.1235</bibtext> </blist> <blist> <bibl id="bib8" idref="ref100" type="bt">8</bibl> <bibtext> Barker T. An automated individual feedback and marking system: An empirical study. The Electronic Journal of E-Learning. 2011; 9; 1: 1-14https://eric.ed.gov/?id=EJ930249</bibtext> </blist> <blist> <bibl id="bib9" idref="ref54" type="bt">9</bibl> <bibtext> Bhebhe S, Maphosa CNaumovska L. An exploration of online assessment in institutions of higher learning. The impact of covid19 on the international education system. 2020; Proud Pen: 172-183. 10.51432/978-1-8381524-0-6_13</bibtext> </blist> <blist> <bibtext> Bonk CJ, Zhang K. Introducing the R2D2 model: Online learning for the diverse learners of this world. Distance Education. 2006; 27; 2: 249-264. 10.1080/01587910600789670</bibtext> </blist> <blist> <bibtext> Cullen MA, Stewart D, Stringer CBJensen K, S. NAckerud,. Accessing science through an online database: A comparison of student learning and engagement using library database readings as a textbook alternative. The evolution of affordable content efforts in the higher education environment: Programs, case studies, and examples. 2018; Libraries Publishing</bibtext> </blist> <blist> <bibtext> d'Alessio MA, Lundquist LL, Schwartz JJ, Pedone V, Pavia J, Fleck J. Social presence enhances student performance in an online geology course but depends on instructor facilitation. Journal of Geoscience Education. 2019; 67; 3: 222-236. 10.1080/10899995.2019.1580179</bibtext> </blist> <blist> <bibtext> Dere A, Engelmann C, White T, Wymore A, Hoffman A, Washburne J, Conklin MGosselin DC, Egger AE, Taber JJ. Implementing and assessing integrate critical zone science materials in an undergraduate geoscience program. Interdisciplinary teaching about earth and the environment for a sustainable future. 2019; Springer International Publishing: 123-144. 10.1007/978-3-030-03273-9_6</bibtext> </blist> <blist> <bibtext> Dunlosky J, Rawson KA, Marsh EJ, Nathan MJ, Willingham DT. Improving students' learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest. 2013; 14; 1: 4-58. 10.1177/1529100612453266</bibtext> </blist> <blist> <bibtext> Eddy SL, Hogan KA. Getting under the hood: How and for whom does increasing course structure work?. CBE—Life Sciences Education. 2014; 13; 3: 453-468. 10.1187/cbe.14-03-0050</bibtext> </blist> <blist> <bibtext> El-Hashash A. Weekly quizzes reinforce student learning outcomes and performance in biomedical sciences in-course assessments. Open Journal of Educational Research. 2022; 2; 4: 168-178. 10.31586/ojer.2022.273</bibtext> </blist> <blist> <bibtext> Elizondo-Garcia J, Gallardo K. Peer feedback in learner-learner interaction practices. Mixed methods study on an xMOOC. Electronic Journal of E-Learning. 2020; 18; 2: 122-135. 10.34190/EJEL.20.18.2.002</bibtext> </blist> <blist> <bibtext> Flavell JH. Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist. 1979; 34; 10: 906-911. 10.1037/0003-066X.34.10.906</bibtext> </blist> <blist> <bibtext> Freeman S, Haak D, Wenderoth MP. Increased course structure improves performance in introductory biology. CBE—Life Sciences Education. 2011; 10; 2: 175-186. 10.1187/cbe.10-08-0105</bibtext> </blist> <blist> <bibtext> Fritz MS, Arthur AMBraddick O. Moderator variables. Oxford research encyclopedia of psychology. 2017; Oxford University Press</bibtext> </blist> <blist> <bibtext> Hartwig MK, Dunlosky J. Study strategies of college students: Are self-testing and scheduling related to achievement?. Psychonomic Bulletin & Review. 2012; 19; 1: 126-134. 10.3758/s13423-011-0181-y</bibtext> </blist> <blist> <bibtext> Hwang W-Y, Hsu J-L, Shadiev R, Chang C-L, Huang Y-M. Employing self-assessment, journaling, and peer sharing to enhance learning from an online course. Journal of Computing in Higher Education. 2015; 27; 2: 114-133. 10.1007/s12528-015-9096-3</bibtext> </blist> <blist> <bibtext> Kearns LR. Student assessment in online learning: Challenges and effective practices. MERLOT Journal of Online Learning and Teaching. 2012; 8; 3: 198-208</bibtext> </blist> <blist> <bibtext> Kim M. The creative commons and copyright protection in the digital era: Uses of creative commons licenses. Journal of Computer-Mediated Communication. 2008; 13; 1: 187-209. 10.1111/j.1083-6101.2007.00392.x</bibtext> </blist> <blist> <bibtext> Kocadere SA, Ozgen D. Assessment of basic design course in terms of constructivist learning theory. Procedia—Social and Behavioral Sciences. 2012; 51: 115-119. 10.1016/j.sbspro.2012.08.128</bibtext> </blist> <blist> <bibtext> Kruger J, Dunning D. Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology. 1999; 77; 6: 1121-1134. 10.1037/0022-3514.77.6.1121</bibtext> </blist> <blist> <bibtext> Liu G, Yao J, Zhou Y. Does teacher and student-student support influence students' engagement in an online course?. 2Nd International Conference on Mental Health and Humanities Education. 2021; 10: 120. 10.2991/assehr.k.210617.052</bibtext> </blist> <blist> <bibtext> Lleras CKempf-Leonard K. Path analysis. Encyclopedia of social measurement. 2005; Elsevier Inc.: 25-30. 10.1016/B0-12-369398-5/00483-7; 3</bibtext> </blist> <blist> <bibtext> Maggioni V, Girotto M, Habib E, Gallagher MA. Building an online learning module for satellite remote sensing applications in hydrologic science. Remote Sensing. 2020; 12; 18: 3009. 10.3390/rs12183009</bibtext> </blist> <blist> <bibtext> Moore MG. Editorial: Three types of interaction. American Journal of Distance Education. 1989; 3; 2: 1-7. 10.1080/08923648909526659</bibtext> </blist> <blist> <bibtext> Pituch KA, Stevens JP. Applied multivariate statistics for the social sciences: Analyses with SAS and IBM's SPSS. 20166: NW; Routledge</bibtext> </blist> <blist> <bibtext> Radmila S, Andrii P. Online education models comprehensive analysis. International Scientific Journal Industry "4.0". 2020; 5; 3: 148-150</bibtext> </blist> <blist> <bibtext> Raes A, Vanneste P, Pieters M, Windey I, Van Den Noortgate W, Depaepe F. Learning and instruction in the hybrid virtual classroom: An investigation of students' engagement and the effect of quizzes. Computers & Education. 2020; 143. 10.1016/j.compedu.2019.103682103682</bibtext> </blist> <blist> <bibtext> Ramos C, Yudko E. "Hits" (not "Discussion Posts") predict student success in online courses: A double cross-validation study. Computers & Education. 2008; 50; 4: 1174-1182. 10.1016/j.compedu.2006.11.003</bibtext> </blist> <blist> <bibtext> Reynolds HL, Kearns KD. A planning tool for incorporating backward design, active learning, and authentic assessment in the college classroom. College Teaching. 2017; 65; 1: 17-27. 10.1080/87567555.2016.1222575</bibtext> </blist> <blist> <bibtext> Rocco S. Online assessment and evaluation. New Directions for Adult and Continuing Education. 2007. 10.1002/ace.249</bibtext> </blist> <blist> <bibtext> Rotar O. Stakeholder partnerships in online learning design: A systematic literature review. SSRN. 2021. 10.2139/ssrn.4011802</bibtext> </blist> <blist> <bibtext> Rotar O. Online student support: A framework for embedding support interventions into the online learning cycle. Research and Practice in Technology Enhanced Learning. 2022; 17: 2. 10.1186/s41039-021-00178-4</bibtext> </blist> <blist> <bibtext> Salas-Morera L, Arauzo-Azofra A, García-Hernández L. Analysis of online quizzes as a teaching and assessment tool. Journal of Technology and Science Education. 2012; 2; 1: 39-45. 10.3926/jotse.30</bibtext> </blist> <blist> <bibtext> Schoenbohm LM, McMillan M. Worldbuilding from tectonic first principles: Integrating and challenging undergraduate knowledge through a course project. Journal of Geoscience Education. 2022; 70; 1: 56-72. 10.1080/10899995.2021.1908810</bibtext> </blist> <blist> <bibtext> Schrum L, Hong S. From the field: Characteristics of successful tertiary online students and strategies of experienced online educators. Education and Information Technologies. 2002; 7; 1: 5-16. 10.1023/A:1015354423055</bibtext> </blist> <blist> <bibtext> Sebesta AJ, Speth EB. How should I study for the exam? Self-regulated learning strategies and achievement in introductory biology. CBE—Life Sciences Education. 2017; 16; 2: 1-12. 10.1187/cbe.16-09-0269</bibtext> </blist> <blist> <bibtext> Sezgin-Memnun D. A comparison of metacognitive awareness levels of future elementary teachers in Turkey and USA. Educational Research and Reviews. 2013; 8; 6: 277-288</bibtext> </blist> <blist> <bibtext> Singh GSB, Nor AA, Musa N. Online learning participation in an asynchronous environment: A case study in an online and distance learning university in Malaysia. ASEAN Journal of Open and Distance Learning. 2020; 12; 2: 18</bibtext> </blist> <blist> <bibtext> Singleton R, Charlton A. Creating H5P content for active learning. Pacific Journal of Technology Enhanced Learning. 2020; 2; 1: 13-14. 10.24135/pjtel.v2i1.32</bibtext> </blist> <blist> <bibtext> Stage FK, Carter HC, Nora A. Path analysis: An introduction and analysis of a decade of research. The Journal of Educational Research. 2004; 98; 1: 5-13. 10.3200/JOER.98.1.5-13</bibtext> </blist> <blist> <bibtext> Swanson SK, Bahr JM, Wilcox JD. A solid waste landfill siting exercise for an introductory environmental geology course. Journal of Geoscience Education. 2006; 54; 4: 458-463. 10.5408/1089-9995-54.4.458</bibtext> </blist> <blist> <bibtext> Wiggins G, McTighe J. Understanding by design. 20052; Association for Supervision and Curriculum Development</bibtext> </blist> <blist> <bibtext> Wiley D, Bliss TJ, McEwen MSpector JM, Merrill MD, Elen J, Bishop MJ. Open educational resources: A review of the literature. Handbook of Research on Educational Communications and Technology. 2014; Springer</bibtext> </blist> <blist> <bibtext> Yan Z, Brown GTL. A cyclical self-assessment process: Towards a model of how students engage in self-assessment. Assessment & Evaluation in Higher Education. 2017; 42; 8: 1247-1262. 10.1080/02602938.2016.1260091</bibtext> </blist> <blist> <bibtext> Zane TW. Performance assessment design principles gleaned from constructivist learning theory (part 1). TechTrends. 2009; 53; 1: 81-90. 10.1007/s11528-009-0242-5</bibtext> </blist> <blist> <bibtext> Zimmerman TD. Exploring learner to content interaction as a success factor in online courses. The International Review of Research in Open and Distributed Learning. 2012; 13; 4: 152. 10.19173/irrodl.v13i4.1302</bibtext> </blist> <blist> <bibtext> Alomyan, H, & Green, D. (2019). Learning theories: Implications for online learning design. Proceedings of the 3rd International Conference on E-society, E-education and E-technology, 126–130. https://doi.org/10.1145/3355966.3358412</bibtext> </blist> <blist> <bibtext> Borge, M, & Goggins, S. (2014). Towards the facilitation of an online community of learners: Assessing the quality of interactions in Yammer. Learning and becoming in practice: The International Conference of the Learning Sciences (ICLS), 1: 753–760</bibtext> </blist> <blist> <bibtext> Bursztyn, N, Riegel, H, Sajjadi, P, Masters, B, Zhao, J, Huang, J, Bagher, M. M, Oliver Wallgrun, J, & Klippel, A. (2021). Fostering geological thinking through virtual strike and dip measurements. 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 303–308. https://doi.org/10.1109/VRW52623.2021.00061</bibtext> </blist> <blist> <bibtext> H5P. (n.d.). Create, Share and Reuse Interactive HTML5 Content in Your Browser. Retrieved April 23, 2022, from https://h5p.org/</bibtext> </blist> <blist> <bibtext> Lewis, D, Trail, T, Srinivasan, S, Lee, S. J, & Lopez, S. (2010). Knowledge check questions: Best practices for use of this instructional strategy. In J. Herrington & C. Montgomerie (Eds.), Proceedings of ED-MEDIA 2010—World Conference on Educational Multimedia, Hypermedia & Telecommunications (pp. 2783–2788). Association for the Advancement of Computing in Education. https://<ulink href="http://www.learntechlib.org/primary/p/35034/">www.learntechlib.org/primary/p/35034/</ulink></bibtext> </blist> <blist> <bibtext> Means, B, Peters, V, Neisler, J, & Griffiths, R. (2020). STEM courses during the COVID pandemic: Lessons from Spring 2020. Digital Promise Global. https://digitalpromise.dspacedirect.org/bitstream/handle/20.500.12265/109/STEM%20Courses%20During%20COVID_Dec2020.pdf?sequence=1</bibtext> </blist> <blist> <bibtext> Rubinfeld, D. L. (2000). Reference guide on multiple regression. In Federal Judicial Center, Reference manual on scientific evidence (2nd ed, pp. 179–227).</bibtext> </blist> <blist> <bibtext> Rule, G. S, & Bajzek, D. M. (2005). Authentic learning and assessments: Major components in transforming superficial understanding into knowledge—Applications to introductory biochemistry. In G. Richards (Ed.), Proceedings of E-Learn 2005—World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (pp. 1497–1502). Association for the Advancement of Computing in Education (AACE). https://<ulink href="http://www.learntechlib.org/primary/p/21408/">www.learntechlib.org/primary/p/21408/</ulink></bibtext> </blist> <blist> <bibtext> Wilkie, S, Zakaria, G, McDonald, T, & Borland, R. (2018). Considerations for designing H5P online interactive activities. In M. Campbell, J. Willems, C. Adachi, D. Blake, I. Doherty, S. Krishnan, S. Macfarlane, L. Ngo, M. O'Donnell, S. Palmer, L. Riddell, I. Story, H. Suri, & J. Tai (Eds.), Proceedings of the 35th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE) (pp. 543–549). Australasian Society for Computers in Learning in Tertiary Education.</bibtext> </blist> </ref> <aug> <p>By April L. Millet; Emre Dinç and Timothy J. Bralower</p> <p>Reported by Author; Author; Author</p> <p></p> <p>April L. Millet April L. Millet earned a Master of Education from Penn State's Instructional Systems Program. She has spent nearly 30 years at Penn State career developing expertise in faculty development, educational technology, and creating residential, hybrid, and online courses. Her research focuses on student engagement and retrieval practice in online edication. She is currently an Associate Teaching Professor in the Dutton Institute of Teaching Learning Excellence at the Pennsylvania State University.</p> <p>Emre Dinç Emre Dinç is a PhD Candidate in Learning, Design, and Technology at Penn State. His research interests are STEM education, reasoning, and educational technology.</p> <p>Timothy J. Bralower Timothy J. Bralower is a Professor of Geoscience at Penn State University with 35 years of experience teaching a range of topics including paleontology, Earth history and marine geology at the undergraduate and graduate levels. His current focus is on climate change and natural hazards and their impacts on society, and he teaches large online classes to students at Penn State and around the world through the University's World Campus. His research addresses ancient climate change and mass extinction and he has over 170 publications covering these topics. He has conducted fieldwork in mountain belts around the world and participated in numerous marine expeditions including drilling in the Chicxulub crater. He is passionate about educating the next generation, making young people aware of the world around them and engaging them in STEM.</p> </aug> <nolink nlid="nl1" bibid="bib23" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib44" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib49" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib48" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib58" firstref="ref5"></nolink> <nolink nlid="nl6" bibid="bib47" firstref="ref6"></nolink> <nolink nlid="nl7" bibid="bib17" firstref="ref8"></nolink> <nolink nlid="nl8" bibid="bib57" firstref="ref9"></nolink> <nolink nlid="nl9" bibid="bib40" firstref="ref10"></nolink> <nolink nlid="nl10" bibid="bib54" firstref="ref12"></nolink> <nolink nlid="nl11" bibid="bib37" firstref="ref13"></nolink> <nolink nlid="nl12" bibid="bib16" firstref="ref14"></nolink> <nolink nlid="nl13" bibid="bib43" firstref="ref15"></nolink> <nolink nlid="nl14" bibid="bib51" firstref="ref17"></nolink> <nolink nlid="nl15" bibid="bib36" firstref="ref19"></nolink> <nolink nlid="nl16" bibid="bib34" firstref="ref20"></nolink> <nolink nlid="nl17" bibid="bib55" firstref="ref23"></nolink> <nolink nlid="nl18" bibid="bib29" firstref="ref24"></nolink> <nolink nlid="nl19" bibid="bib35" firstref="ref25"></nolink> <nolink nlid="nl20" bibid="bib15" firstref="ref26"></nolink> <nolink nlid="nl21" bibid="bib32" firstref="ref27"></nolink> <nolink nlid="nl22" bibid="bib60" firstref="ref28"></nolink> <nolink nlid="nl23" bibid="bib33" firstref="ref31"></nolink> <nolink nlid="nl24" bibid="bib11" firstref="ref32"></nolink> <nolink nlid="nl25" bibid="bib10" firstref="ref34"></nolink> <nolink nlid="nl26" bibid="bib18" firstref="ref35"></nolink> <nolink nlid="nl27" bibid="bib31" firstref="ref37"></nolink> <nolink nlid="nl28" bibid="bib59" firstref="ref40"></nolink> <nolink nlid="nl29" bibid="bib28" firstref="ref41"></nolink> <nolink nlid="nl30" bibid="bib22" firstref="ref44"></nolink> <nolink nlid="nl31" bibid="bib39" firstref="ref45"></nolink> <nolink nlid="nl32" bibid="bib53" firstref="ref48"></nolink> <nolink nlid="nl33" bibid="bib24" firstref="ref49"></nolink> <nolink nlid="nl34" bibid="bib50" firstref="ref50"></nolink> <nolink nlid="nl35" bibid="bib46" firstref="ref51"></nolink> <nolink nlid="nl36" bibid="bib12" firstref="ref52"></nolink> <nolink nlid="nl37" bibid="bib26" firstref="ref55"></nolink> <nolink nlid="nl38" bibid="bib41" firstref="ref57"></nolink> <nolink nlid="nl39" bibid="bib42" firstref="ref60"></nolink> <nolink nlid="nl40" bibid="bib61" firstref="ref61"></nolink> <nolink nlid="nl41" bibid="bib56" firstref="ref63"></nolink> <nolink nlid="nl42" bibid="bib25" firstref="ref64"></nolink> <nolink nlid="nl43" bibid="bib21" firstref="ref66"></nolink> <nolink nlid="nl44" bibid="bib13" firstref="ref72"></nolink> <nolink nlid="nl45" bibid="bib14" firstref="ref76"></nolink> <nolink nlid="nl46" bibid="bib30" firstref="ref93"></nolink> <nolink nlid="nl47" bibid="bib19" firstref="ref94"></nolink> <nolink nlid="nl48" bibid="bib38" firstref="ref96"></nolink> <nolink nlid="nl49" bibid="bib27" firstref="ref97"></nolink> <nolink nlid="nl50" bibid="bib45" firstref="ref98"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1452034
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Investigating Assessment Types in an Online Climate Change Class: Moderating and Mediating Effects
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22April+L%2E+Millet%22">April L. Millet</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0001-7928-7945">0000-0001-7928-7945</externalLink>)<br /><searchLink fieldCode="AR" term="%22Emre+Dinç%22">Emre Dinç</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-1838-4044">0000-0002-1838-4044</externalLink>)<br /><searchLink fieldCode="AR" term="%22Timothy+J%2E+Bralower%22">Timothy J. Bralower</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-3503-859X">0000-0002-3503-859X</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Educational+Technology+Research+and+Development%22"><i>Educational Technology Research and Development</i></searchLink>. 2024 72(6):3075-3101.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 27
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: National Aeronautics and Space Administration (NASA)<br />National Science Foundation (NSF)
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Format%22">Test Format</searchLink><br /><searchLink fieldCode="DE" term="%22Tests%22">Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Laboratory+Experiments%22">Laboratory Experiments</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Testing%22">Comparative Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Curriculum+Based+Assessment%22">Curriculum Based Assessment</searchLink><br /><searchLink fieldCode="DE" term="%22Capstone+Experiences%22">Capstone Experiences</searchLink><br /><searchLink fieldCode="DE" term="%22Climate%22">Climate</searchLink><br /><searchLink fieldCode="DE" term="%22Knowledge+Level%22">Knowledge Level</searchLink><br /><searchLink fieldCode="DE" term="%22Concept+Formation%22">Concept Formation</searchLink><br /><searchLink fieldCode="DE" term="%22Critical+Thinking%22">Critical Thinking</searchLink><br /><searchLink fieldCode="DE" term="%22Problem+Solving%22">Problem Solving</searchLink><br /><searchLink fieldCode="DE" term="%22Creative+Thinking%22">Creative Thinking</searchLink><br /><searchLink fieldCode="DE" term="%22Discovery+Processes%22">Discovery Processes</searchLink><br /><searchLink fieldCode="DE" term="%22Academic+Achievement%22">Academic Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+Processes%22">Learning Processes</searchLink><br /><searchLink fieldCode="DE" term="%22Testing%22">Testing</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s11423-024-10392-8
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1042-1629<br />1556-6501
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This study aimed to examine the effect of four types of assessment on overall student success in an online college-level climate change course. Quizzes, midterms, lab assignments, and a capstone project as well as knowledge check questions were used to assess different aspects of student learning, consistent with Bloom's taxonomy hierarchy. Quizzes and midterms assess basic knowledge, including remembering and understanding concepts, laboratory assignments require students to analyze and integrate concepts, and the capstone allows students to evaluate their understanding and create new content. Binary logistic regression, multiple regression analysis, continuous-by-continuous interaction modeling, and path analysis were used to investigate the moderating and mediating effects of these assessment types. We found both direct and indirect positive interactions as well as one negative interaction. Positive interactions were identified between quiz and lab assignment achievement and between capstone achievement and lab assignment achievement. The total score for correctly answered knowledge check questions positively affected quiz and lab assignment achievements. The interaction between capstone project achievement and total score for correctly answered knowledge check questions showed a negative interaction. Finally, the total score for correctly answered knowledge-check questions had an indirect positive effect on overall student success in the course. Results show that different types of assessment in an online course are complementary and amplify student learning.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1452034
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1452034
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s11423-024-10392-8
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 27
        StartPage: 3075
    Subjects:
      – SubjectFull: Undergraduate Students
        Type: general
      – SubjectFull: Test Format
        Type: general
      – SubjectFull: Tests
        Type: general
      – SubjectFull: Laboratory Experiments
        Type: general
      – SubjectFull: Comparative Testing
        Type: general
      – SubjectFull: Curriculum Based Assessment
        Type: general
      – SubjectFull: Capstone Experiences
        Type: general
      – SubjectFull: Climate
        Type: general
      – SubjectFull: Knowledge Level
        Type: general
      – SubjectFull: Concept Formation
        Type: general
      – SubjectFull: Critical Thinking
        Type: general
      – SubjectFull: Problem Solving
        Type: general
      – SubjectFull: Creative Thinking
        Type: general
      – SubjectFull: Discovery Processes
        Type: general
      – SubjectFull: Academic Achievement
        Type: general
      – SubjectFull: Learning Processes
        Type: general
      – SubjectFull: Testing
        Type: general
    Titles:
      – TitleFull: Investigating Assessment Types in an Online Climate Change Class: Moderating and Mediating Effects
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: April L. Millet
      – PersonEntity:
          Name:
            NameFull: Emre Dinç
      – PersonEntity:
          Name:
            NameFull: Timothy J. Bralower
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 1042-1629
            – Type: issn-electronic
              Value: 1556-6501
          Numbering:
            – Type: volume
              Value: 72
            – Type: issue
              Value: 6
          Titles:
            – TitleFull: Educational Technology Research and Development
              Type: main
ResultId 1