View in EDS HTML Full Text PDF Full Text

A Reflective Thinking-Promoting Approach to Enhancing Graduate Students' Flipped Learning Engagement, Participation Behaviors, Reflective Thinking and Project Learning Outcomes

Saved in:
Bibliographic Details
Title: A Reflective Thinking-Promoting Approach to Enhancing Graduate Students' Flipped Learning Engagement, Participation Behaviors, Reflective Thinking and Project Learning Outcomes
Language: English
Authors: Chen, Mei-Rong Alice, Hwang, Gwo-Jen (ORCID 0000-0001-5155-276X), Chang, Yu-Ying
Source: British Journal of Educational Technology. Sep 2019 50(5):2288-2307.
Availability: Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Peer Reviewed: Y
Page Count: 20
Publication Date: 2019
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Graduate Students, Reflection, Blended Learning, Learner Engagement, Student Participation, Active Learning, Student Projects, Outcomes of Education, Instructional Effectiveness
DOI: 10.1111/bjet.12823
ISSN: 0007-1013
Abstract: Although flipped learning has been recognized as being a potential approach enabling students to learn at their own pace before the class and facilitating in-depth peer-to-peer and student-to-teacher interactions in the class, it remains a challenge to promote students' active learning in the before-class stage, which could significantly affect their in-class engagement and learning performance. In this study, a reflective thinking-promoting approach is proposed to facilitate students' learning design project performance, technology-enhanced active engagement, and their reflective thinking and participation in the before-class stage of flipped learning. A quasi-experiment was conducted on a flipped Digital Learning course of a Master's program in a university to evaluate the effects of the approach on students' learning design performance, engagement, reflective thinking and participation. A total of 19 students (7 male and 12 female) were in the experimental group learning with the reflective thinking-promoting approach, while 19 (4 male and 15 female) were in the control group learning with the conventional flipped learning approach. The results indicated that the proposed approach significantly enhanced not only the students' learning design project outcomes and reflective thinking, but also their engagement and participation in the before-class stage of flipped learning.
Abstractor: As Provided
Entry Date: 2019
Accession Number: EJ1225719
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHBzGRCOz50bt7OJOcZxG9lAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDOEUpksVspZg510jFgIBEICBmwttsZXD4FjMuGUrbsb_xtr310ysI4I2SYdzqENZfK8gIHYkqRqZPY87UtolkWX2f4X4Ll3zmynogBQCrtVGvcu5jr5JsmfDPBVPBtWDFEK__iFw5JvKJhnlBoTMkTh93icfeVSBSYX55pmDcNxCOn_gQHtS4hNQqz1l6HRlI-3LXSTPU6twDdZ-R5aRRBUJhnviG9Y3qUrdnHP8
Text:
  Availability: 1
  Value: <anid>AN0138203574;58i01sep.19;2019Aug24.05:29;v2.2.500</anid> <title id="AN0138203574-1">A reflective thinking‐promoting approach to enhancing graduate students' flipped learning engagement, participation behaviors, reflective thinking and project learning outcomes </title> <sbt id="AN0138203574-2">Introduction</sbt> <p>Although flipped learning has been recognized as being a potential approach enabling students to learn at their own pace before the class and facilitating in‐depth peer‐to‐peer and student‐to‐teacher interactions in the class, it remains a challenge to promote students' active learning in the before‐class stage, which could significantly affect their in‐class engagement and learning performance. In this study, a reflective thinking‐promoting approach is proposed to facilitate students' learning design project performance, technology‐enhanced active engagement, and their reflective thinking and participation in the before‐class stage of flipped learning. A quasi‐experiment was conducted on a flipped Digital Learning course of a Master's program in a university to evaluate the effects of the approach on students' learning design performance, engagement, reflective thinking and participation. A total of 19 students (7 male and 12 female) were in the experimental group learning with the reflective thinking‐promoting approach, while 19 (4 male and 15 female) were in the control group learning with the conventional flipped learning approach. The results indicated that the proposed approach significantly enhanced not only the students' learning design project outcomes and reflective thinking, but also their engagement and participation in the before‐class stage of flipped learning. Practitioner NotesWhat is already known about this topic Flipping learning is an effective teaching approach that shifts the lecture time to the before‐class stage and hence teachers have more time to conduct learning activities to promote students' higher order thinking as well as to deal with individual students learning problems.Students' learning experience, motivation and belief could be the factors that guide students towards engagement and participation in content and help them learn new skills.Engaging students in reflective thinking is an important and challenging issue. It provides students with an opportunity to scrutinize their own learning and hence make progress.What this paper adds A reflective thinking‐promoting approach into flipped learning is proposed to facilitate students' flipped learning engagement and participation behaviors as well as their project performance and reflective thinking.In addition to promoting students' learning outcomes, the results indicated that the proposed approach provides promising results on the technology‐enhanced active learning experience and participation in online learning in the before‐class stage of flipped learning.Implications for practice and/or policy Via monitoring students' online before‐class progress, instructors can recognize the factors that affect students' learning, adjust or differentiate their instruction and even provide students with more opportunities or with additional support to meet students' needs for learning.The link between the video lectures and the classroom activities can be examined in future research to perceive the influence of video lectures on students' participation behaviors in‐class activities.Forming reflective thinking skill is important, but attainable; it needs students' engagement and participation in time and effort.</p> <p>Flipped learning is an evolutionary educational approach that combines lecture‐based and interactive teaching methods (Johnson & Renner, [<reflink idref="bib41" id="ref1">41</reflink>]; Strayer, [<reflink idref="bib65" id="ref2">65</reflink>]). It reverses the instructional focus of teachers, helping students to foster lower level thinking skills (ie, remembering and understanding) at home and developing their higher order thinking skills (ie, analyzing, evaluating and creating) in the classroom (Anderson, Krathwohl, & Bloom, [<reflink idref="bib2" id="ref3">2</reflink>]; Bergmann & Sams, [<reflink idref="bib9" id="ref4">9</reflink>]). In recent years, millions of videos of online educational resources from subject experts on a wide range of topics have been made freely available online. This ever‐increasing access to lecture capture technology has made flipped learning much easier. Flipped learning is no longer a buzzword in education but a tendency and even a must (Bishop & Verleger, [<reflink idref="bib11" id="ref5">11</reflink>]; Hoffman, [<reflink idref="bib34" id="ref6">34</reflink>]; Lage, Platt, & Treglia, [<reflink idref="bib47" id="ref7">47</reflink>]). Many agree that it enhances learning practice and makes full use of the class time, focusing on student‐centered interactive activities and individual scaffolding (Bergmann & Sams, [<reflink idref="bib9" id="ref8">9</reflink>]; Tam, [<reflink idref="bib67" id="ref9">67</reflink>]).</p> <p>Among various educational objectives, reflective thinking skills involve complex judgmental skills such as critical thinking and problem solving (Burton, [<reflink idref="bib14" id="ref10">14</reflink>]). These are indispensable skills in today's world of technologies and information abundance. Therefore, it is believed that flipped learning must be able to help students attain these skills in all classrooms, at all education levels and in all subjects. To acquire reflective thinking in flipped learning, students need to fully participate and be engaged in designed pre‐class and in‐class activities and experience the process of remembering, understanding, analysis, synthesis, evaluation and finally conceptual change. This process is recognized as the development of reflective thinking (Atkins & Murphy, [<reflink idref="bib5" id="ref11">5</reflink>]; Hong & Choi, [<reflink idref="bib35" id="ref12">35</reflink>]), a natural form of learning from experience and reflection. Moreover, it is an active, persistent and careful consideration (Porntaweekul, Raksasataya, & Nethanomsak, [<reflink idref="bib55" id="ref13">55</reflink>]). If students can be aware of and govern their learning by actively participating in reflective thinking, that is, if they can assess what they know, what they need to know and how they bridge that gap in learning contexts, learning will occur. Therefore, in this study, we aimed to examine the effect of a reflective thinking‐promoting approach on students' learning design performance and reflective thinking compared with conventional flipped learning. Meanwhile, students' technology‐enhanced active engagement and participation in the learning management system (hereafter LMS) were scrutinized. Further, we hoped to build up a systematic module, leading to more replicable and sustained flipped classrooms. Four research questions are addressed:</p> <p></p> <ulist> <item> Does the reflective thinking‐promoting approach enhance students' learning design performance?</item> <p></p> <item> Does the reflective thinking‐promoting approach strengthen students' engagement?</item> <p></p> <item> Does the reflective thinking‐promoting approach boost the students' reflective thinking?</item> <p></p> <item> How does the reflective thinking‐promoting approach boost the students' participation in the before‐class stage of flipped learning?</item> </ulist> <hd id="AN0138203574-3">Literature review</hd> <p></p> <hd id="AN0138203574-4">Flipped learning</hd> <p>The flipped classroom refers to the teaching mode which reverses the traditional instruction by delivering teachers' lectures at the pre‐class stage in the form of instructional videos or other media to enable teachers to have more time in the class to help students do exercises and solve the learning problems they encounter (Bergmann & Sams, [<reflink idref="bib9" id="ref14">9</reflink>]). In addition, the Flipped Learning Network (https://flippedlearning.org/) has stated the importance of conducting effective "flipped learning" by taking four components (ie, flexible environment, learning culture, intentional content and professional educator) into account. Researchers have further indicated the need to employ effective learning strategies in flipped classrooms rather than only paying attention to the development of pre‐class instructional videos to draw on learners' active learning and engagement (Bishop & Verleger, [<reflink idref="bib11" id="ref15">11</reflink>]). Jensen, Kummer, and Godoy ([<reflink idref="bib40" id="ref16">40</reflink>]) defined that active learning is using additional technology, teaching materials and peer instruction for meaningful learning experience. Instead of teaching, the focus should be on how to involve students in autonomous learning, that is "the involvement of students in activities and thinking about activities." In addition, Hung's study ([<reflink idref="bib36" id="ref17">36</reflink>]) indicated that active learning involves students in doing activities and in thinking about the information, they are learning. In her study, the experimental group participants had also invested more out‐of‐class study time in demonstrating their learning engagement. Many educators and teachers have supported this revolution (Bergmann & Sams, [<reflink idref="bib9" id="ref18">9</reflink>]; Butt, [<reflink idref="bib15" id="ref19">15</reflink>]; Hamdan, McKnight, McKnight, & Artfstrom, [<reflink idref="bib32" id="ref20">32</reflink>]; Hwang, Lai, & Wang, [<reflink idref="bib38" id="ref21">38</reflink>]), whereas some have raised doubts about its effects (Hung, [<reflink idref="bib37" id="ref22">37</reflink>]; Johnson & Renner, [<reflink idref="bib41" id="ref23">41</reflink>]; Strayer, [<reflink idref="bib65" id="ref24">65</reflink>]), and some consider it time‐consuming and tedious to prepare video recordings and in‐class activities. Nevertheless, it is believed that two major aspects can make flipped learning effective and practical: one is in‐class activities that can trigger students' engagement and the other is top‐quality educational videos of an optimal duration that suits students' attention span.</p> <p>The premise of flipping a class is the combination of digitally based lectures as pre‐class tasks and learner‐centered activities in reserved class time (Hoffman, [<reflink idref="bib34" id="ref25">34</reflink>]; Tam, [<reflink idref="bib67" id="ref26">67</reflink>]). Because of its blended features, the flipped classroom approach involves digital platforms, that is, learning management systems, also known as LMSs (eg, Moodle and 1know) and real face‐to‐face human interaction (Bishop & Verleger, [<reflink idref="bib11" id="ref27">11</reflink>]). In the traditional teacher‐centered learning theories, learners watch the instructional videos or read the assigned materials via the LMS out of class without any limitation of time and space; in contrast, in class, aside from assessing learners' retention and understanding of the subject matter, teachers, based on learners' diversity and on their knowledge and experiences, implement a learner‐centered learning approach to have interactive classroom activities (Soliman, [<reflink idref="bib64" id="ref28">64</reflink>]).</p> <p>However, flipping a class does not really contribute to flipped learning. Flipped learning requires certain kinds of action (Hamdan <emph>et al</emph>., [<reflink idref="bib32" id="ref29">32</reflink>]; Hwang, Lai, & Wang, [<reflink idref="bib38" id="ref30">38</reflink>]). The Flipped Learning Network presented an effective flipped learning model with four components: flexible environment, learning culture, intentional content and professional educator (Hung, [<reflink idref="bib37" id="ref31">37</reflink>]). There are still some implications worth restating. First of all, the learning environment has to remain flexible enough to create individual work areas, small group work spaces, and whole class mingling and demonstration stations (Bergmann & Sams, [<reflink idref="bib9" id="ref32">9</reflink>]). Next, a wide variety of learning experiences, approaches and academic‐support strategies shift from the teacher‐centered to learner‐centered orientation, focusing on one‐to‐one differentiated instruction and immediate feedback in a comfortable learning culture (Bergmann & Sams, [<reflink idref="bib9" id="ref33">9</reflink>]). A scaffolding effect on cognitive and social development occurs at any moment (Hamdan <emph>et al</emph>., [<reflink idref="bib32" id="ref34">32</reflink>]). Moreover, the video content is not simply "add‐on" homework; it helps learners to build up a solid foundation for in‐class activities and to develop their conceptual understanding and procedural fluency (Seaboyer, [<reflink idref="bib60" id="ref35">60</reflink>]).</p> <p>Over the past decade, numerous studies have shown a sequence of positive results, including learners' academic performance (Bergmann & Sams, [<reflink idref="bib9" id="ref36">9</reflink>]; Sergis, Sampson, & Pelliccione, [<reflink idref="bib61" id="ref37">61</reflink>]), perceptions of engagement and learning skills (Elmaadaway, [<reflink idref="bib24" id="ref38">24</reflink>]), satisfaction (Bergmann & Sams, [<reflink idref="bib9" id="ref39">9</reflink>]), self‐regulation (Lai & Hwang, [<reflink idref="bib49" id="ref40">49</reflink>]; Sun, Wu, & Lee, [<reflink idref="bib66" id="ref41">66</reflink>]), preferences (Bates & Galloway, [<reflink idref="bib8" id="ref42">8</reflink>]), technology acceptance (Kissi, Nat, & Armah, [<reflink idref="bib44" id="ref43">44</reflink>]) and so on. Nevertheless, the core value of flipped learning, that is, higher‐order thinking skills, is less discussed and less satisfactorily resolved. Flipped learning, not just revolving around lecture‐based learning to attain lower thinking skills, that is, understanding and memorization, also encourages learners to develop higher order thinking skills in class: to apply, synthesize and even create knowledge (Seaboyer, [<reflink idref="bib60" id="ref44">60</reflink>]). Therefore, a sound, comprehensive flipped learning module will be outlined in this paper.</p> <hd id="AN0138203574-5">Technology‐enhanced active engagement and participation</hd> <p>Learning effectiveness is determined by students' optimal engagement, and this engagement is achieved by factors that promote learning (Chuang, Weng, & Chen, [<reflink idref="bib19" id="ref45">19</reflink>]; Shernoff, Csikszentmihalyi, Schneider, & Shernoff, [<reflink idref="bib63" id="ref46">63</reflink>]). Students' optimal engagement is their self‐awareness of and their commitment to their own learning (Andrusyszyn & Yankou, [<reflink idref="bib3" id="ref47">3</reflink>]). Chen and Wu ([<reflink idref="bib16" id="ref48">16</reflink>]), for instance, indicated that in learners' experience, motivation and belief are the factors that guide learners towards engagement in content and help them learn new skills. The former is derived from the motive for learners' actions, willingness and goals, whereas the latter is a firmly held opinion related to their learning strategies, academic performance and motivation (Chen & Pajares, [<reflink idref="bib17" id="ref49">17</reflink>]). In fact, two personal traits, namely motivation and belief, were examined and were found to be influential in learning in a flipped classroom context (Chuang <emph>et al</emph>., [<reflink idref="bib19" id="ref50">19</reflink>]). On the other hand, researchers (Ainley & Ainley, [<reflink idref="bib1" id="ref51">1</reflink>]; Shernoff & Csikszentmihalyi, [<reflink idref="bib62" id="ref52">62</reflink>]), have particularly pointed out that learning engagement is characterized by crucial factors such as learners' concentration, interest and enjoyment. That is, in a powerful learning engagement, learners will be at a high energy level to engage in their learning and ultimately they have positive outcomes or results (Fink, [<reflink idref="bib27" id="ref53">27</reflink>]). However, with the rapid development and popularization of Web 2.0 and computer technology, these digital native students are faced with an unprecedented impact on their learning experience. According to Thompson ([<reflink idref="bib68" id="ref54">68</reflink>]) and Fong and Wang ([<reflink idref="bib28" id="ref55">28</reflink>]), digital natives' learning motivation drops very fast when they are involved in reading. Their attention and interest reduces even more in academic subjects (Koutropoulos, [<reflink idref="bib46" id="ref56">46</reflink>]; Prensky, [<reflink idref="bib56" id="ref57">56</reflink>]). Therefore, technology‐enhanced active learning has become particularly important.</p> <p>The concept of active learning refers to students' active engagement and participation in the learning process (Peng, Wang, & Sampson, [<reflink idref="bib52" id="ref58">52</reflink>]). It seizes the idea of learning by doing and eventually leads students to knowledge construction and continuous learning (Argote & Miron‐Spektor, [<reflink idref="bib4" id="ref59">4</reflink>]; Pahl & Kenny, [<reflink idref="bib50" id="ref60">50</reflink>]). Technology is an effective tool that can facilitate the learning process and consecutively create an active environment for learners to build their knowledge, skills, experience and engagement. For this reason, a reflective analysis of existing technology‐enhanced active learning engagement becomes crucial in this present study.</p> <p>Learning engagement determines whether learners have learned throughout the course, but it is an individual attribute and needs to be examined (Felder & Brent, [<reflink idref="bib26" id="ref61">26</reflink>]; Ventura & Moscoloni, [<reflink idref="bib69" id="ref62">69</reflink>]). Based on the theory of Flow, proposed by Csikszentmihalyi ([<reflink idref="bib22" id="ref63">22</reflink>]), Schwarzenberg, Navon, Nussbaum, Pérez‐Sanagustín, and Caballero ([<reflink idref="bib59" id="ref64">59</reflink>]) set up a more thorough, comprehensive assessment model to measure learning experience. In this model, the constructs of learning experience were adopted from Shernoff, Csikszentmihalyi, Schneider, and Shernoff's ([<reflink idref="bib63" id="ref65">63</reflink>]) engagement, which consists of feedback, challenge, peer instruction, choice and enjoyment. In the construct of feedback, three dimensions are focused on: (<reflink idref="bib1" id="ref66">1</reflink>) the objectives and success, (<reflink idref="bib2" id="ref67">2</reflink>) self‐current performance and (<reflink idref="bib3" id="ref68">3</reflink>) task completeness. Challenge, slightly beyond one's current level of ability, has a discernible effect on academic performance. Peer instruction provides learners with an opportunity to interact and learn from each other; it has a positive impact on learning achievement. The perceived choice is related to one's satisfaction and autonomous learning. Enjoyment refers to the satisfaction with the expected outcomes of the task.</p> <p>In this study, we utilized Schwarzenberg and his colleagues' ([<reflink idref="bib59" id="ref69">59</reflink>]) experience model as a reference to investigate learners' engagement because it combines theories that describe the factors motivating learners and the conditions needed to generate the optimum engagement. With the help of the LMSs that monitor learners' participation in the before‐class stage of flipped learning, it is believed that technology‐enhanced active engagement most likely represents a learner's conceptions of how, when and where flipped learning does and can take place.</p> <hd id="AN0138203574-6">Reflective thinking</hd> <p>Reflective thinking (also known as RT) is rational thinking realized by mental discipline (Kok, [<reflink idref="bib45" id="ref70">45</reflink>]). It is often used synonymously with critical thinking, but unlike critical thinking which includes various thinking skills leading to satisfying outcomes, RT puts more emphasis on the process of making decisions or stating opinions about what has happened (Evans, [<reflink idref="bib25" id="ref71">25</reflink>]; Schön, [<reflink idref="bib58" id="ref72">58</reflink>]). RT provides learners with a structured opportunity to scrutinize their own learning (Verpoorten, Westera, & Specht, [<reflink idref="bib70" id="ref73">70</reflink>]). During a reflective activity, learners can develop reflective thinking skills by (<reflink idref="bib1" id="ref74">1</reflink>) relating new knowledge to previous understandings, (<reflink idref="bib2" id="ref75">2</reflink>) thinking in abstract and conceptual ways, (<reflink idref="bib3" id="ref76">3</reflink>) applying specific strategies in new tasks and (<reflink idref="bib4" id="ref77">4</reflink>) understanding their own ideas and thoughts (Hwang, Wu, & Ke, [<reflink idref="bib39" id="ref78">39</reflink>]).</p> <p>RT can help learners to employ thinking skills such as analysis, synthesis and evaluation to reach a conceptual change (Atkins & Murphy, [<reflink idref="bib5" id="ref79">5</reflink>]; Hong & Choi, [<reflink idref="bib35" id="ref80">35</reflink>]). It is especially a critical factor in problem solving (Kok, [<reflink idref="bib45" id="ref81">45</reflink>]; Wang, Yuan, Kirschner, Kushniruk, & Peng, [<reflink idref="bib71" id="ref82">71</reflink>]). Although previous studies pointed out that RT is often associated with post‐practice methods of experience recapture through self‐assessment, such as portfolios or learning diaries (Evans, [<reflink idref="bib25" id="ref83">25</reflink>]) or interactive activities (Hwang <emph>et al</emph>., [<reflink idref="bib39" id="ref84">39</reflink>]), whether flipping learning activates RT skills and engages learners in active reflection needs to be further explored.</p> <p>Many researchers privilege RT and believe it can promote reflection upon practice (Atkins & Murphy, [<reflink idref="bib5" id="ref85">5</reflink>]; Evans, [<reflink idref="bib25" id="ref86">25</reflink>]; Hong & Choi, [<reflink idref="bib35" id="ref87">35</reflink>]; Hwang <emph>et al</emph>., [<reflink idref="bib39" id="ref88">39</reflink>]; Kok, [<reflink idref="bib45" id="ref89">45</reflink>]), but not many of them can actually measure RT and tell whether people are meeting their goal of developing RT and further give explanations (Kember <emph>et al</emph>., [<reflink idref="bib43" id="ref90">43</reflink>]). Fortunately, Kember and his colleagues offered a prominent, detailed questionnaire for assessing different levels of reflective thinking. The questionnaire consists of four constructs, namely habitual action, understanding, reflection and critical reflection. Habitual action refers to learners' automatic performance with little conscious thought, whereas understanding means that learners can understand and comprehend a concept in academic learning. As for reflection, based on Dewey's definition ([<reflink idref="bib23" id="ref91">23</reflink>], p. 9), learners engage in intellectual and affective activities, raise questions, test the assumptions during the process of problem solving, and finally gain new understandings and appreciations. The last is a critical reflection, part of a higher order of thinking. It is a reasoning process which finally reaches a perspective transformation. Having the identities of scales, the next step is to draw up the effect of a reflective thinking‐promoting approach in flipped learning.</p> <hd id="AN0138203574-7">Reflective thinking‐promoting approach for flipped learning</hd> <p>The "reflective thinking‐promoting approach" proposed in this study emphasizes the guided "reflection" in a project after engaging students in the learning process of watch‐annotate, summarize‐question, discuss‐give feedback and reflect‐project by referring to Rath's ([<reflink idref="bib57" id="ref92">57</reflink>]) WSQ (Watch‐Summary‐Question) framework and Bloom's Taxonomy Matrix (Anderson <emph>et al</emph>., [<reflink idref="bib2" id="ref93">2</reflink>]) to serve as the theoretical framework of the flipped learning activities. The proposed approach is divided into watch‐annotate and summarize‐question in the pre‐class stage, discuss‐give feedback in the in‐class stage, and reflect‐project in the after‐class stage, with six cognitive process dimensions (remember, understand, apply, analyze, evaluate and create) and four knowledge dimensions (factual, conceptual procedural, knowledge and metacognitive) of Bloom's Taxonomy Matrix (Anderson <emph>et al</emph>., [<reflink idref="bib2" id="ref94">2</reflink>]), as shown in Figure. This is intended to enhance learners' reflective thinking with a positive learning experience.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/58I/01sep19/bjet12823-fig-0001.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="bjet12823-fig-0001.jpg" title="Structure of a reflective thinking‐promoting approach for flipped learning [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <p>Pardo <emph>et al</emph>. ([<reflink idref="bib51" id="ref95">51</reflink>]) stated that video annotation could be a beneficial strategy to help learners achieve their learning outcomes. In conventional flipped learning, video watching as a pre‐class activity is a less interactive and one‐way form of learning (Lai & Hwang, [<reflink idref="bib49" id="ref96">49</reflink>]). Fu and Hwang ([<reflink idref="bib29" id="ref97">29</reflink>]) stated that the adoption of technologies could promote learners' learning engagement, reflection and reflective thinking. Because of more recent advanced technologies, considering the "1know" system (<ulink href="http://www.1know.net">http://www.1know.net</ulink>) as an LMS example as mentioned earlier, it can offer time‐stamped annotation features, whereby learners can watch course videos and take notes. Meanwhile, learners can also pause, rewind, re‐watch the videos. In addition, a reflective thinking‐promoting approach was adopted not only to watch‐annotate but also to summarize question as a series of tasks before class, and then the learners discussed the question selected by the instructor and then provided feedback in class. The purpose is to help the learners engage in reflective thinking based on Bloom's Taxonomy Matrix. Accordingly, they were given a chance to develop reflective thinking skills (see Figure).</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/58I/01sep19/bjet12823-fig-0002.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="bjet12823-fig-0002.jpg" title="Illustrative example of the flipped learning management system [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <p>To evaluate the effect of incorporating the reflective thinking‐promoting approach into flipped learning on learner' learning performance, technology‐enhanced active engagement, reflective thinking and participation in the before‐class stage of flipped learning, the reflective thinking‐promoting approach was implemented in an online learning management system (LMS), the 1know system, to monitor learners' progress, as shown in the color‐coded task list in Figure. The colors help to keep learners alert and more interested in their own learning progress as well as their peers', encouraging them to learn.</p> <p>One of the most substantial advantages of using the LMS is that it can monitor learners' learning progress on pre‐class tasks, note taking and behavior (time spent on each task), but it can also gather data on students' online learning behavior. It can also offer time‐stamped annotation features whereby students can watch course videos and take notes. In addition, it allows students to propose questions (as shown in Figure).</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/58I/01sep19/bjet12823-fig-0003.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="bjet12823-fig-0003.jpg" title="Interface of question‐posing after watching the instructional video lecture [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <hd id="AN0138203574-11">Experimental design</hd> <p>To evaluate the effectiveness of the innovative flipped learning approach, an experiment was conducted on a digital learning course to evaluate the learning design performance, learner engagement, reflective thinking and participation in the before‐class stage of flipped learning with two different flipped learning approaches. The objectives of the Analysis and Applying Strategies of Digital Learning Literature course are to foster students' literature reading and analyzing competences as well as their abilities of applying digital learning strategies to educational settings and analyzing the learning effectiveness.</p> <hd id="AN0138203574-12">Participants</hd> <p>A quasi‐experimental design with a digital survey was carried out in an Analysis and Applying Strategies of Digital Learning Literature course at a technology university in northern Taiwan. A total of 38 graduate students (11 male and 27 female) participated in this study, of whom 19 (7 male and 12 female) were in the experimental group learning with the reflective thinking‐promoting approach and 19 (4 male and 15 female) were in the control group learning with the conventional flipped learning approach. In order to avoid the influence of different teachers on the experimental outcomes, the two classes were instructed by the same senior professor. The students in the experimental group learned with the reflective thinking‐promoting approach incorporated into the flipped learning approach whereas those in the control group learned with the conventional flipped learning approach. During the pre‐class learning activity, both groups were assigned to watch seven videos, take notes, write a summary and post a question, while the control group students were assigned similar activities, but note‐taking and question‐posing were not obligatory, so it was an option for them.</p> <hd id="AN0138203574-13">Experimental process</hd> <p>In this study, the Analysis and Applying Strategies of Digital Learning Literature course was held for 3 hours a week over a period of 18 weeks. According to the aim of the graduate course, it is to prepare graduate students for advanced research, particularly for the graduate thesis and doctoral dissertation (Austin, [<reflink idref="bib7" id="ref98">7</reflink>]; Boote & Beile, [<reflink idref="bib12" id="ref99">12</reflink>]). Based on the objective of Analysis and Applying Strategies of Digital Learning Literature course, 10 journal papers were assigned as course materials and seven instructional videos (as in Table) providing a starting point for students to cultivate their reading and research skills. Students were instructed to study various digital learning strategies (eg, peer assessment, project‐based learning and problem‐based learning) via literature review and analysis, and to analyze the subjects' learning performances and patterns from various aspects by applying the strategies to practical educational settings. Journal readings were examples related to the learning design and research design methods introduced in the instructional videos, such as learning strategies and pedagogical theories. In class, the students were guided to implement learning design and research design as well as to analyze the data (eg, learning logs) provided by the instructor using the learning strategies and analysis methods learned in the pre‐class stage. The instructional videos were developed by an experienced professor who had had more than 20 years' experience teaching the course, and were reviewed by another experienced educational technology instructor and two flipped learning experts recognized by FLGlobal.</p> <p>Example of course content and videos</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Week</th><th align="left">Journal papers and instructional video links</th><th align="left">Time</th></tr></thead><tbody><tr><td align="left">11</td><td align="left">Paper 5: Improving learning achievements, motivations and problem‐solving skills through a peer assessment‐based game development approach</td><td align="left"> </td></tr><tr><td align="left">Video1: Peer assessment <ext-link href="https://www.youtube.com/watch?v=tAUqRwRgdEI%26feature=youtu.be" title="https://www.youtube.com/watch?v=tAUqRwRgdEI&feature=youtu.be" /></td><td align="left">7:34</td></tr><tr><td align="left">12</td><td align="left">Paper 6: Effects of different online peer‐feedback approaches on students' performance skills, motivation and self‐efficacy in a dance course</td><td align="left"> </td></tr><tr><td align="left">Paper 7: A concept map‐embedded educational computer game for improving students' learning performance in natural science courses</td></tr><tr><td align="left">Video 2: Video sharing <ext-link href="https://www.youtube.com/watch?v=9d-EHp8qgbc%26feature=youtu.be" title="https://www.youtube.com/watch?v=9d-EHp8qgbc&feature=youtu.be" /></td><td align="left">9:32</td></tr><tr><td align="left">13</td><td align="left">Paper 8: Effects of an integrated concept mapping and web‐based problem‐solving approach on students' learning achievements, perceptions and cognitive loads</td><td align="left"> </td></tr><tr><td align="left">Video 3: Digital Media Design Principle of Design—cognitive load 3‐1 <ext-link href="https://www.youtube.com/watch?v=gLj7Z-InR8M%26feature=youtu.be" title="https://www.youtube.com/watch?v=gLj7Z-InR8M&feature=youtu.be" /></td><td align="left">16:32</td></tr><tr><td align="left">Cognitive Theory of Multimedia Learning 3‐2 <ext-link href="https://www.youtube.com/watch?v=a7-eEDVRpvs%26feature=youtu.be" title="https://www.youtube.com/watch?v=a7-eEDVRpvs&feature=youtu.be" /></td><td align="left">11:28</td></tr><tr><td align="left">14</td><td align="left">Paper 9: Influences of an inquiry‐based ubiquitous gaming design on students' learning achievements, motivation, behavioral patterns, and tendency towards critical thinking and problem solving</td><td align="left"> </td></tr><tr><td align="left">Video 4: Problem‐based learning 1 <ext-link href="https://youtu.be/BJv%5feNe74I0" /></td><td align="left">08:49</td></tr><tr><td align="left">Problem‐based learning 2 <ext-link href="https://www.youtube.com/watch?v=qB%5f6rFK2-sg%26feature=youtu.be" title="https://www.youtube.com/watch?v=qB%5f6rFK2-sg&feature=youtu.be" /></td><td align="left">16:42</td></tr><tr><td align="left">Inquiry‐based learning <ext-link href="https://youtu.be/RnwSKzl9PiE" /></td><td align="left">14:04</td></tr></tbody></table> </ephtml> </p> <p>In this course, the participants gave graded reports during the class to display their knowledge and to present all aspects of their original research ideas to fellow researchers in the field. The class was carried out in the read‐and‐present format. That is, the professor had a list of journal articles on the syllabus for each week related to the strategies of digital learning; each participant studied one of the articles and then gave an in‐class report. Each participant was asked to summarize the objectives and findings of the journal article during his or her report, and to discuss ideas and provide their own reflections on the readings with other participants. During the 3‐hour class, a report was conducted by one or two participants for around 30 minutes, and then the professor provided feedback on the students' reports and discussed some other issues during the remaining time. After each report, the participants were assigned to discuss and answer the questions from the pre‐class tasks while 1 hour was allotted to answer a question in class. The participants were given a week to complete the pre‐class tasks. Both the experimental and control groups watched the pre‐class video lectures before class and finished the assigned tasks prior to the in‐class group discussion sessions. The students in the experimental group generated open‐ended questions based on the instructional videos related to the topics of the lecture, such as, "What kind of teaching strategies can be used to increase germane cognitive load and promote schema properties?" and "What teaching methods can be used to transform short‐term memory into long‐term memory?" These questions were posted by the students of the experimental group after they watched the instructional videos, and were then selected and edited by the professor. On some occasions (eg, when some important issues were overlooked by the students), the professor could propose questions as well. Then, both the experimental group and the control group students discussed these questions in class. The professor explained and clarified any confusion about the video lectures after their discussion.</p> <p>Figure shows the experimental design of this study. The experiment was conducted on the four units of an Analysis and Applying Strategies of Digital Learning Literature course, which aims to teach the graduate students the analysis and applying strategies of digital learning literature. Before the experiment, the two groups of students spent 3 weeks on the concepts of flipped classroom strategies. Pre‐learning assessments for evaluating their knowledge of collecting digital learning literature and reporting on a digital learning paper was administered. Following that, the participants in the experimental group were required to watch seven videos, take notes, write a summary, and post a question before class meetings; on the other hand, those in the control group were required to do similar activities except for taking notes and posting a question before class meetings. In the pre‐class tasks, the students in the experimental group and control group were required to watch videos and then write a summary on the LMS platform before the commencement of the class. The videos were embedded with a few short questions to enhance students' understanding of that topic. The duration for each pre‐class video was kept to a maximum of 16:42 minutes and an average of 12:05 minutes because of the constraint of students' concentration time (Brame, [<reflink idref="bib13" id="ref100">13</reflink>]). Researchers have suggested that the duration of the video lectures in flipped learning should be class grade multiplied by 1.5 in minutes (Atwa, Din, & Hussin, [<reflink idref="bib6" id="ref101">6</reflink>]; Bergmann & Sams, [<reflink idref="bib9" id="ref102">9</reflink>]), with a maximum video length of around 20 minutes for older students (Handke & Franke, [<reflink idref="bib33" id="ref103">33</reflink>]; Phillips & Trainor, [<reflink idref="bib54" id="ref104">54</reflink>]).</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/58I/01sep19/bjet12823-fig-0004.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="bjet12823-fig-0004.jpg" title="Experiment procedure [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <p>The pre‐class video was uploaded, and the pre‐class tasks were assigned each week in the LMS platform. The pre‐class tasks are the summary of the assigned journal papers, and summary and annotation of the video lecture, as shown in Figure. The students in the experimental group and control group were required to summarize and submit what they had remembered and comprehended concerning the video's content. However, in the experimental group, the students were required to post a question regarding what they had watched from the video lectures. The instructor would select 10 frequently asked questions and have students in both the experimental group and the control group discuss them as one of the in‐class tasks. After all the pre‐class and in‐class learning activities, the students designed an e‐learning research project, presented it and filled out the post‐questionnaires for measuring their learning outcomes. The process was carried out to activate the students' metacognitive knowledge for more learning to take place. It helps students experience the cognitive process of Bloom Taxonomy Matrix's create level.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/58I/01sep19/bjet12823-fig-0005.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="bjet12823-fig-0005.jpg" title="A list of pre‐class tasks [Colour figure can be viewed at wileyonlinelibrary.com]" /> </p> <p></p> <hd id="AN0138203574-16">Measuring instruments</hd> <p>In this study, the students' learning design performance, technology‐enhanced active engagement, reflective thinking and participation in the before‐class stage of flipped learning were assessed. Learning design performance was evaluated based on the students' learning design project reports. The pre‐project report aimed to evaluate the students' basic knowledge of designing digital learning activities. The students were required to present reports during the class to display their knowledge and perspectives of their learning design, including what they had learned from the literature and their own ideas of learning design. The post‐learning design report aimed to assess the students' concepts of applying and innovative and effective strategies for digital learning. The total score of both reports was 100. Two experienced instructors rated the students' reports based on a rubric consisting of four dimensions, that is, correctness, completeness, innovation and effectiveness. The two instructors' ratings were found to have the substantial agreement of inter‐rater consistency.77 Cohen's Kappa (Cohen, [<reflink idref="bib20" id="ref105">20</reflink>]).</p> <p>The technology‐enhanced active student engagement questionnaire was adopted from the learning experience assessment scale developed by Schwarzenberg <emph>et al</emph>. ([<reflink idref="bib59" id="ref106">59</reflink>]). The questionnaire consisted of 18 items in five subscales, namely feedback (six items), challenge (three items), peer instruction (three items), choice (three items) and enjoyment (three items), with a 5‐point Likert scale. The Cronbach's alpha values of the subscales are,.90,.76,.84,.73 and.60 respectively.</p> <p>The questionnaire of reflective thinking was adopted based on the questionnaire to measure the level of reflective thinking developed by Kember <emph>et al</emph>. ([<reflink idref="bib43" id="ref107">43</reflink>]). It consists of 16 items, including four dimensions: habitual action, reflection, critical reflections and understanding. The Cronbach's alpha values of the four dimensions were.80,.87,.91 and.81 respectively.</p> <p>The time spent viewing video lectures and the words posted in the assigned tasks were collected for analysis of participation behaviors in the before‐class stage of flipped learning. There were seven video lectures (total 5,081 seconds, average 726 seconds) and 17 tasks published in LMS. During the 4‐week flipped learning, participants in the experimental group and control group watched video lectures and posted their summaries and reflections on the video lectures as the pre‐class tasks. In order to identify students' participation behaviors in the before‐class stage of flipped learning, the data of all participants' time spent (in seconds) watching the video lectures and words (Chinese characters) were analyzed.</p> <hd id="AN0138203574-17">Experimental results</hd> <p>There were three kinds of data collected, namely (<reflink idref="bib1" id="ref108">1</reflink>) the students' scores of the pre‐reports and post‐reports, (<reflink idref="bib2" id="ref109">2</reflink>) the results of the technology‐enhanced active learning engagement questionnaire, (<reflink idref="bib3" id="ref110">3</reflink>) the results of the reflective thinking questionnaire and (<reflink idref="bib4" id="ref111">4</reflink>) Data in LMS of participants' time spent watching video lectures and words posted. SPSS software was used to analyze the data. The dependent variables were checked for normal distribution and homogeneity of variance before analysis.</p> <hd id="AN0138203574-18">Learning design performances</hd> <p>The one‐way ANCOVA was used to compare the two groups' learning design performances by adopting the reflective thinking‐promoting approach into the flipped learning model as an independent variable, while the posttest and pretest scores were respectively the dependent variable and covariate. After confirming the assumption of homogeneity of regression with <emph>F</emph> = .023 (<emph>p</emph> = .88 > .05), ANCOVA was performed, as shown in Table. A significant difference was found with <emph>F</emph> = 4.29 (<emph>p</emph> < .05), showing that the learning design performances of the two groups were significantly different because of the different flipped learning models. The result showed that the participants (adjusted mean = 89.05, Std. error = 1.48) who learned with the reflective thinking‐promoting approach incorporated into the flipped learning model outperformed the participants (adjusted mean = 84.73, Std. error = 1.48) who learned with the conventional flipped learning mode.</p> <p>ANCOVA result of the post scores on students' learning performance</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Group</th><th align="left"><italic>N</italic></th><th align="left">Mean</th><th align="left"><italic>SD</italic></th><th align="left">Adjusted mean</th><th align="left">Std. error</th><th align="left"><italic>F</italic></th><th align="left"><italic>η</italic><sup>2</sup></th></tr></thead><tbody><tr><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">89.05</td><td align="char" char=".">5.411</td><td align="char" char=".">89.05</td><td align="char" char=".">1.48</td><td align="char" char=".">4.29<xref ref-type="fn" rid="tfn1" /></td><td align="char" char=".">.106</td></tr><tr><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">84.73</td><td align="char" char=".">7.327</td><td align="char" char=".">84.73</td><td align="char" char=".">1.48</td><td align="char" char="."> </td><td align="char" char="."> </td></tr></tbody></table> </ephtml> </p> <p>1 * <emph>p</emph> < .05.</p> <hd id="AN0138203574-19">Technology‐enhanced active learning engagement</hd> <p>Table shows the <emph>t</emph>‐test result for the technology‐enhanced active learning engagement ratings of the two groups. The means and standard deviations of the ratings were 4.59 and.30 for the experimental group, and 4.36 and.35 for the control group. The ratings of the two groups were significantly different with <emph>t</emph> = 2.21 (<emph>p</emph> < .05). There was a significant difference between the two groups with <emph>d</emph> = .71 (Cohen, [<reflink idref="bib21" id="ref112">21</reflink>]). This implies that there was more positive effect on the learning engagement of the participants in the experimental group than on those in the control group.</p> <p>Independent sample t‐test results of the technology‐enhanced active learning experience</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Group</th><th align="left"><italic>N</italic></th><th align="left">Mean</th><th align="left"><italic>SD</italic></th><th align="left"><italic>t</italic></th><th align="left">Cohen's <italic>d</italic></th></tr></thead><tbody><tr><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">4.59</td><td align="char" char=".">.30</td><td align="char" char=".">2.21<xref ref-type="fn" rid="tfn2" /></td><td align="char" char=".">.71</td></tr><tr><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">4.36</td><td align="char" char=".">.35</td><td align="char" char=".">19</td><td align="char" char="."> </td></tr></tbody></table> </ephtml> </p> <p>2 * <emph>p</emph> < .05.</p> <p>We also performed <emph>t</emph>‐tests to assess the impact of specific dimensions. As shown in Table , it is recognized that peer instruction <emph>t</emph> = 2.65 (<emph>p</emph> < .05, <emph>d</emph> = .86) had more positive effect, with a significant difference, on the experimental group (<emph>M</emph> = 4.77 <emph>SD</emph> = .33) than on the control group (<emph>M</emph> = 4.33 <emph>SD</emph> = .64).</p> <p>Independent sample t‐test results of the technology‐enhanced active learning experience in each dimension</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Dimensions</th><th align="left">Group</th><th align="left"><italic>N</italic></th><th align="left">Mean</th><th align="left"><italic>SD</italic></th><th align="left"><italic>t</italic></th><th align="left">Cohen's <italic>d</italic></th></tr></thead><tbody><tr><td align="left">Feedback</td><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">4.67</td><td align="char" char=".">.39</td><td align="char" char=".">.86</td><td align="char" char=".">.30</td></tr><tr><td align="left"> </td><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">4.55</td><td align="char" char=".">.43</td><td align="char" char="."> </td><td align="char" char="."> </td></tr><tr><td align="left">Challenge</td><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">4.31</td><td align="char" char=".">.44</td><td align="char" char=".">1.64</td><td align="char" char=".">.51</td></tr><tr><td align="left"> </td><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">4.04</td><td align="char" char=".">.61</td><td align="char" char="."> </td><td align="char" char="."> </td></tr><tr><td align="left">Peer instruction</td><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">4.77</td><td align="char" char=".">.33</td><td align="char" char=".">2.65<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">.86</td></tr><tr><td align="left"> </td><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">4.33</td><td align="char" char=".">.64</td><td align="char" char="."> </td><td align="char" char="."> </td></tr><tr><td align="left">Choice</td><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">4.70</td><td align="char" char=".">.43</td><td align="char" char=".">.86</td><td align="char" char=".">.27</td></tr><tr><td align="left"> </td><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">4.58</td><td align="char" char=".">.46</td><td align="char" char="."> </td><td align="char" char="."> </td></tr><tr><td align="left">Enjoyment</td><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">4.39</td><td align="char" char=".">.47</td><td align="char" char=".">1.42</td><td align="char" char=".">.46</td></tr><tr><td align="left"> </td><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">4.18</td><td align="char" char=".">.44</td><td align="char" char="."> </td><td align="char" char="."> </td></tr></tbody></table> </ephtml> </p> <p>3 * <emph>p</emph> < .05.</p> <hd id="AN0138203574-20">Reflective thinking</hd> <p>The <emph>t</emph>‐test outcome of the comparison of reflective thinking for the two learning approaches is shown in Table. The mean of reflective thinking ratings for the experimental group was 4.57 (<emph>t</emph> = 3.34, <emph>p</emph> < .01, <emph>d</emph> = 1.11), which was significantly higher than that of the control group (mean = 4.16). Furthermore, the result of reflective thinking for the two groups represents a good effect size according to Cohen's ([<reflink idref="bib21" id="ref113">21</reflink>]) criteria, where a Cohen's <emph>d</emph> value greater than.8 is considered as a large effect size. This means that with the reflective thinking‐promoting approach incorporated into the flipped learning approach, the students achieved significantly higher reflective thinking than those in the conventional flipped learning approach. In other words, the reflective thinking‐promoting approach can successfully promote students' reflective thinking.</p> <p>Independent sample t‐test results of reflective thinking</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Group</th><th align="left"><italic>N</italic></th><th align="left">Mean</th><th align="left"><italic>SD</italic></th><th align="left"><italic>t</italic></th><th align="left">Cohen's <italic>d</italic></th></tr></thead><tbody><tr><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">4.57</td><td align="char" char=".">.35</td><td align="char" char=".">3.34<xref ref-type="fn" rid="tfn4" /></td><td align="char" char=".">1.11</td></tr><tr><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">4.16</td><td align="char" char=".">.39</td><td align="char" char="."> </td><td align="char" char="."> </td></tr></tbody></table> </ephtml> </p> <p>4 ** <emph>p </emph>< .01.</p> <p>In addition, it was found that three of four dimensions showed significant differences between the two groups, as shown in Table , in particular, for habitual action (<emph>t</emph> = 2.65, <emph>p</emph> < .05, <emph>d</emph> = .85), reflection (<emph>t</emph> = 3.09, <emph>p</emph> < .001, <emph>d</emph> = 1.50) and critical reflection (<emph>t</emph> = 2.51, <emph>p</emph> < .0001, <emph>d</emph> = .92). Such a finding could be further evidence that the students who learned with the reflective thinking‐promoting approach in the flipped learning approach showed better habitual action, reflection and critical thinking than those who learned with the conventional flipped learning approach.</p> <p>Independent sample t‐test results of the reflection between the two groups</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Dimensions</th><th align="left">Group</th><th align="left"><italic>N</italic></th><th align="left">Mean</th><th align="left"><italic>SD</italic></th><th align="left"><italic>t</italic></th><th align="left">Cohen's <italic>d</italic></th></tr></thead><tbody><tr><td align="left">Habitual action</td><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">4.67</td><td align="char" char=".">.35</td><td align="char" char=".">2.65<xref ref-type="fn" rid="tfn5" /></td><td align="char" char=".">.85</td></tr><tr><td align="left"> </td><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">4.24</td><td align="char" char=".">.62</td><td align="char" char="."> </td><td align="char" char="."> </td></tr><tr><td align="left">Reflection</td><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">4.61</td><td align="char" char=".">.38</td><td align="char" char=".">3.09<xref ref-type="fn" rid="tfn6" /></td><td align="char" char=".">1.50</td></tr><tr><td align="left"> </td><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">3.92</td><td align="char" char=".">.53</td><td align="char" char="."> </td><td align="char" char="."> </td></tr><tr><td align="left">Critical reflection</td><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">4.46</td><td align="char" char=".">.38</td><td align="char" char=".">2.51<xref ref-type="fn" rid="tfn7" /></td><td align="char" char=".">.92</td></tr><tr><td align="left"> </td><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">4.05</td><td align="char" char=".">.50</td><td align="char" char="."> </td><td align="char" char="."> </td></tr><tr><td align="left">Understanding</td><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">4.53</td><td align="char" char=".">.53</td><td align="char" char=".">.47</td><td align="char" char=".">.16</td></tr><tr><td align="left"> </td><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">4.43</td><td align="char" char=".">.72</td><td align="char" char="."> </td><td align="char" char="."> </td></tr></tbody></table> </ephtml> </p> <ulist> <item>5 * <emph>p</emph> < .05;</item> <item>6 ** <emph>p</emph> < .01;</item> <item>7 *** <emph>p</emph> < .001.</item> </ulist> <hd id="AN0138203574-21">Participation behaviors in the before‐class stage of flipped learning</hd> <p>In order to clearly understand the students' participation behaviors in the before‐class stage of flipped learning, students' average time spent (in seconds) on online video lectures and the words (Chinese characters) they posted was analyzed. Table shows the <emph>t</emph>‐test results of the comparison of the experimental group (Mean = 1217.72, <emph>t</emph> = 3.14, <emph>p</emph> < .01, <emph>d</emph> = 1.01) which spent significantly more time than the control group (Mean = 885.99) watching video lectures. In addition, the <emph>t</emph>‐test results also reveal that the experimental group (Mean = 1344.26, <emph>t</emph> = 5.66, <emph>p</emph> < .001, <emph>d</emph> = 1.84) posted significantly more words than the control group (Mean = 1005.68).</p> <p>Independent sample t‐test results for students' participation behaviors in the before‐class stage of flipped learning</p> <p> <ephtml> <table><thead valign="top"><tr><th align="left">Variable</th><th align="left">Group</th><th align="left"><italic>N</italic></th><th align="left">Mean</th><th align="left"><italic>SD</italic></th><th align="left"><italic>t</italic></th><th align="left">Cohen's <italic>d</italic></th></tr></thead><tbody><tr><td align="left">Time spent on videos</td><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">1217.72</td><td align="char" char=".">4.38.62</td><td align="char" char=".">3.14<xref ref-type="fn" rid="tfn8" /></td><td align="char" char=".">1.02</td></tr><tr><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">885.99</td><td align="char" char=".">139.01</td><td align="char" char="."> </td><td align="char" char="."> </td></tr><tr><td align="left">Words posted</td><td align="left">Experimental group</td><td align="char" char=".">19</td><td align="char" char=".">1344.26</td><td align="char" char=".">236.33</td><td align="char" char=".">5.66<xref ref-type="fn" rid="tfn9" /></td><td align="char" char=".">1.84</td></tr><tr><td align="left"> </td><td align="left">Control group</td><td align="char" char=".">19</td><td align="char" char=".">1005.68</td><td align="char" char=".">110.39</td><td align="char" char="."> </td><td align="char" char="."> </td></tr></tbody></table> </ephtml> </p> <ulist> <item>8 ** <emph>p</emph> < .01;</item> <item>9 *** <emph>p</emph> < .001.</item> </ulist> <hd id="AN0138203574-22">Discussion and conclusions</hd> <p>The study aimed to explore students' engagement and participation in active learning with learning management systems and to examine the effect of whether a reflective thinking‐promoting approach would enhance students' flipped learning design performance and participation. The implications of the findings are discussed as follows.</p> <p>In general, the findings provide evidence to support the reflective thinking‐promoting approach. Concerning research questions 1 and 3: This study aimed to measure the impact of the integrated reflective thinking‐promoting approach in flipped learning on the students' learning design performance and reflective thinking of graduate students in the fall 2018 Analysis and Applying Strategies of Digital Learning Literature course. In the experimental group, the participants showed a significantly higher level of learning design performance and reflective thinking, indicating that integrating the reflective thinking‐promoting approach in flipped learning has a positive impact on students' learning design performance and reflective thinking capacity. In particular, the results of three of four dimensions: habitual action, reflection and critical reflection, demonstrated significant differences between the experimental group and the control group.</p> <p>One of the most important pre‐class tasks in this study was question‐posing, and this task provided an instruction way for both teaching and learning. Participants in the experimental group were required to pose a question after watching each video lecture, and then the instructors would select 10 suitable questions and have participants in the experimental group and the control group discuss them in the class. Lan and Lin ([<reflink idref="bib48" id="ref114">48</reflink>]) mentioned that good quality question‐posing requires more reflective thinking, and suggested that engaging in the practice of question‐posing can foster students' cognitive development. In this study, reflective thinking tasks in flipped learning offered participants an organized approach to examine their own learning, so they had an opportunity to develop reflective thinking skills. Various studies have shown that promoting students' reflective thinking via activities is valuable for learning design performance (Bigge & Shermis, [<reflink idref="bib10" id="ref115">10</reflink>]; Cheng & Chau, [<reflink idref="bib18" id="ref116">18</reflink>]; Hwang, Wu, & Ke, [<reflink idref="bib39" id="ref117">39</reflink>]; Phan, [<reflink idref="bib53" id="ref118">53</reflink>]; Verpoorten, Westera, & Specht, [<reflink idref="bib70" id="ref119">70</reflink>]). This is consistent with a considerable amount of literature demonstrating the correlation of higher order skills with students' success in learning (Ghanizadeh, [<reflink idref="bib30" id="ref120">30</reflink>]; Kealey <emph>et al</emph>., [<reflink idref="bib42" id="ref121">42</reflink>]). Phan ([<reflink idref="bib53" id="ref122">53</reflink>]) further demonstrated that the effectiveness of the learning process is influenced by reflective thinking; likewise, learning achievement is influenced by reflective thinking. Reflective thinking fosters knowledge that can be applied to a variety of situations and contexts. Students have the capability to reflect initially, to acquire and then to generate the new knowledge, so they tend to be more successful academically.</p> <p>Flipped learning combines video lectures on LMSs as pre‐class content instruction and in‐class hands‐on activities that encourage students to practice, apply, reflect and master the content that they have learned from the pre‐class requirements (Hoffman, [<reflink idref="bib34" id="ref123">34</reflink>]; Tam, [<reflink idref="bib67" id="ref124">67</reflink>]). This blended approach provides flexibility for students who can work at their own pace and take on the responsibility for learning; however, it may not be easy for teachers to keep students more engaged with learning, which could lead to dissatisfaction in the end (Johnson & Renner, [<reflink idref="bib41" id="ref125">41</reflink>]; Strayer, [<reflink idref="bib65" id="ref126">65</reflink>]). Therefore, students' engagement and participation in active learning with the LMS becomes more important. The results in Table for question 2 present that, in the technology‐enhanced active engagement, the students with a reflective thinking‐promoting approach overall outperformed the students in the conventional flipped learning environment. As Chuang, Weng, and Chen ([<reflink idref="bib19" id="ref127">19</reflink>]) mentioned, students' optimal engagement that promotes learning depends on certain factors. In this study, taking notes and posting questions are such behaviors of learning engagement.</p> <p>In terms of detailed constructs of the engagement, the students with a reflective thinking‐promoting approach had a statistically significant difference in peer instruction, compared with the students in the conventional flipped learning environment. Students' learning engagement and active learning were examined in this study. Gilboy <emph>et al</emph>. (2015) claimed that learning engagement is students' commitment to and responsibility for learning as they contribute their time and effort to their learning production. Jensen, Kummer, and Godoy ([<reflink idref="bib40" id="ref128">40</reflink>]) defined that active learning is using additional technology, teaching materials and peer instruction for a meaningful learning experience. Instead of teaching, the focus should be on how to involve students in autonomous learning, that is, the involvement of students in activities and thinking about activities. In addition, Hung (2015) indicated that active learning involves students in doing activities and thinking about the information they are learning. In her study, the participants of the experimental group also invested more out‐of‐class study time to demonstrate their learning engagement. In this study, the participants in the experimental group increased their participation time in the before‐class stage of flipped learning, where they needed to take notes and reflect when raising questions. In class, they had deeper discussions on their own questions with peers and the instructor. The process of student engagement was the reason for their active learning (Pahl & Kenny, [<reflink idref="bib50" id="ref129">50</reflink>]). Lastly, it was found that the students in both groups had high values for feedback, challenge, peer instruction, choice and enjoyment (with means of more than four on the 5‐point Likert scale). Perhaps that is because the LMS can show students' color‐coded progress on each task list, which resulted in their commitment and enhanced their attentiveness to their own and their peers' learning progress.</p> <p>The present study examined students' engagement in active learning with learning management, and the findings were positive. Meanwhile, this is evidence that a reflective thinking‐promoting approach can enhance students' flipped learning design performance and participation. A reflective‐thinking promoting approach amplifies the core value of flipped learning, that is, higher‐order thinking. Students can activate their intuitive reflection and gain learning experience skills by analyzing, synthesizing and evaluating what they have learned when interacting with the video content instruction and discussions from their peers and instructor. In addition, with the LMSs that monitor students' progress, instructors can recognize the factors that affect students' learning, adjust or differentiate their instruction and even provide students with more opportunities or with additional support to meet students' needs for learning. Flipped learning with a reflective thinking‐promoting approach is one of the examples that encourages students to become more responsible for their learning and that aids them in learning their subject matter.</p> <p>On the other hand, there are several limitations and issues that need to be considered for future research. First, the sample size was insufficient, making it difficult to make generalizations. However, it was not easy to have a great number of graduate students in this present study. Therefore, we suggest that in future research, a variety of subjects can be considered as participants. Second, the time of the study was not long enough. If the time were longer, the results should be more substantial because reflective thinking and active learning are important, but they take time to become habits. Third, the activities carried out in the class, such as students' discussion and feedback, were not entirely assessed in this study. Overall, the most significant purpose of this study is to illustrate the fact that forming reflective thinking skills is both essential and attainable. Most importantly, it needs students' engagement, participation, time and effort.</p> <hd id="AN0138203574-23">Acknowledgements</hd> <p>This study is supported in part by the Ministry of Science and Technology of the Republic of China under contract number MOST‐105‐2511‐S‐011‐008‐MY3.</p> <hd id="AN0138203574-24">Statements on open data, ethics and conflict of interest</hd> <p>The data can be obtained by sending request e‐mails to the corresponding author.</p> <p>The participants were protected by hiding their personal information during the research process. They knew that the participation was voluntary and they could withdraw from the study at any time.</p> <p>There is no potential conflict of interest in this study.</p> <ref id="AN0138203574-25"> <title> References </title> <blist> <bibl id="bib1" idref="ref51" type="bt">1</bibl> <bibtext> Ainley, M., & Ainley, J. (2011). Student engagement with science in early adolescence: The contribution of enjoyment to students' continuing interest in learning about science. Contemporary Educational Psychology, 36 (1), 4 – 12.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref3" type="bt">2</bibl> <bibtext> Anderson, L. W., Krathwohl, D. R., & Bloom, B. S. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. New York, NY : Longman.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref47" type="bt">3</bibl> <bibtext> Andrusyszyn, M.‐A., & Yankou, D. (2004). Attention fatigue and the effect of debriefing in a web‐enhanced graduate nursing course. International Journal of Instructional Technology and Distance Learning, 1 (4), 1 – 7.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref59" type="bt">4</bibl> <bibtext> Argote, L., & Miron‐Spektor, E. (2011). Organizational learning: From experience to knowledge. Organization Science, 22 (5), 1123 – 1137.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref11" type="bt">5</bibl> <bibtext> Atkins, S., & Murphy, K. (1993). Reflections: A review of the literature. Journal of Advanced Nursing, 18, 1188 – 1192.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref101" type="bt">6</bibl> <bibtext> Atwa, Z. M., Din, R., & Hussin, M. (2018). Effectiveness of flipped learning in physics education on Palestinian high school students' achievement. Journal of Personalized Learning, 2 (1), 73 – 85.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref98" type="bt">7</bibl> <bibtext> Austin, A. E. (2002). Preparing the next generation of faculty: Graduate school as socialization to the academic career. The Journal of Higher Education, 73 (1), 94 – 122.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref42" type="bt">8</bibl> <bibtext> Bates, S., & Galloway, R. (2012). The inverted classroom in a large enrolment introductory physics course: A case study. Paper presented at the Proceedings of the HEA STEM learning and teaching conference.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref4" type="bt">9</bibl> <bibtext> Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day. Washington, DC : International Society for Technology in Education.</bibtext> </blist> <blist> <bibtext> Bigge, M. L., & Shermis, S. S. (1999). Learning theories for teachers (6th ed.). New York : Longman.</bibtext> </blist> <blist> <bibtext> Bishop, J. L., & Verleger, M. A. (2013). The flipped classroom: A survey of the research. Paper presented at the ASEE national conference proceedings, Atlanta, GA.</bibtext> </blist> <blist> <bibtext> Boote, D. N., & Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher, 34 (6), 3 – 15.</bibtext> </blist> <blist> <bibtext> Brame, C. J. (2016). Effective educational videos: Principles and guidelines for maximizing student learning from video content. CBE—Life Sciences Education, 15 (4), es6.</bibtext> </blist> <blist> <bibtext> Burton, E. (2010). High level thinking and questioning strategies. Cross Cultural Management, 17 (1), 5 – 9.</bibtext> </blist> <blist> <bibtext> Butt, A. (2014). Student views on the use of a flipped classroom approach: Evidence from Australia. Business Education & Accreditation, 6 (1), 33 – 43.</bibtext> </blist> <blist> <bibtext> Chen, C. H., & Wu, I. C. (2012). The interplay between cognitive and motivational variables in a supportive online learning system for secondary physical education. Computers & Education, 58, 542 – 550.</bibtext> </blist> <blist> <bibtext> Chen, J. A., & Pajares, F. (2010). Implicit theories of ability of grade 6 science students: Relation to epistemological beliefs and academic motivation and achievement in science. Contemporary Educational Psychology, 35, 75 – 87.</bibtext> </blist> <blist> <bibtext> Cheng, G., & Chau, J. (2013). Exploring the relationship between students' self‐regulated learning ability and their ePortfolio achievement. Internet and Higher Education, 17, 9 – 15.</bibtext> </blist> <blist> <bibtext> Chuang, H., Weng, C., & Chen, C. (2018). Which students benefit most from a flipped classroom approach to language learning? British Journal of Educational Technology, 49 (1), 56 – 68.</bibtext> </blist> <blist> <bibtext> Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20 (1), 37 – 46.</bibtext> </blist> <blist> <bibtext> Cohen, J. (1988). Set correlation and contingency tables. Applied Psychological Measurement, 12 (4), 425 – 434.</bibtext> </blist> <blist> <bibtext> Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York, NY : Harper Perennial.</bibtext> </blist> <blist> <bibtext> Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process. Boston, MA : D.C. Heath & Co Publishers.</bibtext> </blist> <blist> <bibtext> Elmaadaway, M. A. N. (2018). The effects of a flipped classroom approach on class engagement and skill performance in a Blackboard course. British Journal of Educational Technology, 49 (3), 479 – 491.</bibtext> </blist> <blist> <bibtext> Evans, J. S. B. (2010). Intuition and reasoning: A dual-process perspective. Psychological Inquiry, 21 (4), 313 – 326.</bibtext> </blist> <blist> <bibtext> Felder, R. M., & Brent, R. (2005). Understanding student differences. Journal of Engineering Education, 94 (1), 57 – 72.</bibtext> </blist> <blist> <bibtext> Fink, L. D. (2003). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco, CA : John Wiley & Sons.</bibtext> </blist> <blist> <bibtext> Fong, J., & Wang, F. L. (Eds.). (2007). Blended learning. Edinburgh, United Kingdom 15–17 August, 2007 Proceedings. City University of Hong Kong.</bibtext> </blist> <blist> <bibtext> Fu, Q.‐K., & Hwang, G.‐J. (2018). Trends in mobile technology‐supported collaborative learning: A systematic review of journal publications from 2007 to 2016. Computers & Education, 119, 129 – 143.</bibtext> </blist> <blist> <bibtext> Ghanizadeh, A. (2017). The interplay between reflective thinking, critical thinking, self‐monitoring, and academic achievement in higher education. Higher Education, 74 (1), 101 – 114.</bibtext> </blist> <blist> <bibtext> Gilboy, M. B., Heinerichs, S., & Pazzaglia, G. (2015). Enhancing student engagement using the flipped classroom. Journal of Nutrition Education and Behavior, 47 (1), 109 – 114.</bibtext> </blist> <blist> <bibtext> Hamdan, N., McKnight, P. E., McKnight, K., & Artfstrom, K. M. (2013). The flipped learning model: A white paper based on the literature review. Retrieved from the Flipped Learning Network website.</bibtext> </blist> <blist> <bibtext> Handke, J., & Franke, P. (2013). xMOOCs im virtual linguistics campus. In: Schulmeister R, Herausgeber. (Ed.), MOOCs‐massive open online courses: Offene Bildung oder Geschäftsmodell (pp. 101 – 126). Münster : Waxmann Verlag.</bibtext> </blist> <blist> <bibtext> Hoffman, E. S. (2014). Beyond the flipped classroom: Redesigning a research methods course for e‐instruction. Contemporary Issues in Education Research, 7, 51 – 62.</bibtext> </blist> <blist> <bibtext> Hong, Y., & Choi, I. (2015). Assessing reflective thinking in solving design problems: The development of a questionnaire. British Journal of Educational Technology, 46 (4), 848 – 863.</bibtext> </blist> <blist> <bibtext> Hung, H.-T. (2015). Flipping the classroom for English language learners to foster active learning. Computer Assisted Language Learning, 28 (1), 81 – 96.</bibtext> </blist> <blist> <bibtext> Hung, H. (2017). Design‐based research: Redesign of an English language course using a flipped classroom approach. TESOL, 51 (1), 180 – 192.</bibtext> </blist> <blist> <bibtext> Hwang, G., Lai, C., & Wang, S. (2015). Seamless flipped learning: A mobile technology‐enhanced flipped classroom with effective learning strategies. Journal of Computer in Education, 2 (4), 449 – 473.</bibtext> </blist> <blist> <bibtext> Hwang, G.‐J., Wu, P.‐H., & Ke, H.‐R. (2011). An interactive concept map approach to supporting mobile learning activities for natural science courses. Computers & Education, 57 (4), 2272 – 2280.</bibtext> </blist> <blist> <bibtext> Jensen, J. L., Kummer, T. A., & Godoy, P. D. D. M. (2015). Improvements from a flipped classroom may simply be the fruits of active learning. CBE—Life Sciences Education, 14 (1), ar5.</bibtext> </blist> <blist> <bibtext> Johnson, L. W., & Renner, J. D. (2012). Effect of the flipped classroom model on a secondary computer applications course: Student and teacher perceptions, questions, and student achievement. Louisville, KY : University of Louisville.</bibtext> </blist> <blist> <bibtext> Kealey, B. T., Holland, J., & Watson, M. (2005). Preliminary evidence on the association between critical thinking and performance in principles of accounting. Issues in Accounting Education, 20 (1), 33 – 49.</bibtext> </blist> <blist> <bibtext> Kember, D., Leung, D. Y. P., Jones, A., Loke, A. Y., McKay, J., Sinclair, K., ... Yeung, E. (2000). Development of a questionnaire to measure the level of reflective thinking. Assessment & Evaluation in Higher Education, 25 (4), 381 – 395.</bibtext> </blist> <blist> <bibtext> Kissi, P. S., Nat, M., & Armah, R. B. (2018). The effects of learning‐family conflict, perceived control over time and task‐fit technology factors on urban‐rural high school students' acceptance of video‐based instruction in flipped learning approach. Educational Technology Research and Development, 66 (6), 1547 – 1569.</bibtext> </blist> <blist> <bibtext> Kok, J. (2002). Reflective journal writing: How it promotes reflective thinking in clinical nursing education: A students' perspective. Curationis, 25 (3), 35 – 42.</bibtext> </blist> <blist> <bibtext> Koutropoulos, A. (2011). Digital natives: Ten years after. MERLOT Journal of Online Learning and Teaching, 7 (4), 525 – 538.</bibtext> </blist> <blist> <bibtext> Lage, M. J., Platt, G. J., & Treglia, M. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. The Journal of Economic Education, 31 (1), 30 – 43.</bibtext> </blist> <blist> <bibtext> Lan, Y.-F., & Lin, P.-C. (2011). Evaluation and improvement of student's question-posing ability in a web-based learning environment. Australasian Journal of Educational Technology, 27 (4), 581 – 599.</bibtext> </blist> <blist> <bibtext> Lai, C., & Hwang, G. (2016). A self‐regulated flipped classroom approach to improving students' learning performance in a mathematics course. Computers & Education, 100, 126 – 140.</bibtext> </blist> <blist> <bibtext> Pahl, C. & Kenny, C. (2008). The future of technology enhanced active learning – A roadmap. In M. L. Lytras, D. Gasevic, P. Ordonez de Pablos, & W. Huang (Eds.), Technology enhanced learning (pp. 348 – 375). Hershey, PA : IGI Global.</bibtext> </blist> <blist> <bibtext> Pardo, A., Mirriahi, N., Dawson, S., Zhao, Y., Zhao, A., & Gašević, D. (2015). Identifying learning strategies associated with active use of video annotation software. Paper presented at the Proceedings of the Fifth International Conference on Learning Analytics and Knowledge.</bibtext> </blist> <blist> <bibtext> Peng, J., Wang, M., & Sampson, D. (2017). Visualizing the complex process for deep learning with an authentic programming project. Journal of Educational Technology & Society, 20 (4), 275 – 287.</bibtext> </blist> <blist> <bibtext> Phan, H. P. (2009). Exploring students' reflective thinking practice, deep processing strategies, effort, and achievement goal orientations. Educational Psychology, 29 (3), 297 – 313.</bibtext> </blist> <blist> <bibtext> Phillips, C. R., & Trainor, J. E. (2014). Millennial students and the flipped classroom. ASBBS Proceedings, 21 (1), 519.</bibtext> </blist> <blist> <bibtext> Porntaweekul, S., Raksasataya, S., & Nethanomsak, T. (2016). Developing reflective thinking instructional model for enhancing students' desirable learning outcomes. Educational Research and Reviews, 11 (6), 238 – 251.</bibtext> </blist> <blist> <bibtext> Prensky, M. (2001). Digital natives, digital immigrants part 1. The Horizon, 9 (5), 1 – 6. https://doi.org/10.1108/10748120110424816</bibtext> </blist> <blist> <bibtext> Raths, D. (2014). Nine video tips for a better flipped classroom. The Education Digest, 79 (6), 15.</bibtext> </blist> <blist> <bibtext> Schön, D. A. (2017). The reflective practitioner: How professionals think in action. Abingdon : Routledge.</bibtext> </blist> <blist> <bibtext> Schwarzenberg, P., Navon, J., Nussbaum, M., Pérez‐Sanagustín, M., & Caballero, D. (2018). Learning experience assessment of flipped courses. Journal of Computing in Higher Education, 30 (2), 237 – 258.</bibtext> </blist> <blist> <bibtext> Seaboyer, J. (2013). The role of technology‐assisted assessment in fostering critical reading in undergraduate literary studies. International Computer Assisted Assessment Conference. UK : Computer Assisted Assessment.</bibtext> </blist> <blist> <bibtext> Sergis, S., Sampson, D. G., & Pelliccione, L. (2018). Investigating the impact of flipped classroom on students' learning experiences: A self‐determination theory approach. Computers in Human Behavior, 78, 368 – 378.</bibtext> </blist> <blist> <bibtext> Shernoff, D. J., & Csikszentmihalyi, M. (2009). Flow in schools: Cultivating engaged learners and optimal learning environments. In R. Gilman, E. S. Heubner, & M. J. Furlong (Eds.), Handbook of positive psychology in schools (pp. 131 – 145). New York, NY : Routledge.</bibtext> </blist> <blist> <bibtext> Shernoff, D. J., Csikszentmihalyi, M., Schneider, B., & Shernoff, E. S. (2003). Student engagement in high school classrooms from the perspective of flow theory. School Psychology Quarterly, 18 (2), 158 – 176.</bibtext> </blist> <blist> <bibtext> Soliman, N. A. (2016). Teaching English for academic purposes via the flipped learning approach. Procedia ‐ Social and Behavioral Sciences, 232, 122 – 129.</bibtext> </blist> <blist> <bibtext> Strayer, J. (2012). The flipped classroom: Turning the traditional classroom on its head. Retrieved from <ulink href="http://www.knewton.com/flipped-classroom/">http://www.knewton.com/flipped-classroom/</ulink></bibtext> </blist> <blist> <bibtext> Sun, J. C. Y., Wu, Y. T., & Lee, W. I. (2017). The effect of the flipped classroom approach to Open Course Ware instruction on students' self‐regulation. British Journal of Educational Technology, 48 (3), 713 – 729.</bibtext> </blist> <blist> <bibtext> Tam, M. (2000). Constructivism, instructional design, and technology: Implications for transforming distance learning. Educational Technology & Society, 3 (2), 50 – 60.</bibtext> </blist> <blist> <bibtext> Thompson, P. (2013). The digital natives as learners: Technology use patterns and approaches to learning. Computers & Education, 65, 12 – 33.</bibtext> </blist> <blist> <bibtext> Ventura, A. C., & Moscoloni, N. (2015). Learning styles and disciplinary differences: A cross‐sectional study of undergraduate students. International Journal of Learning and Teaching, 1 (2), 88 – 93.</bibtext> </blist> <blist> <bibtext> Verpoorten, D., Westera, W., & Specht, M. (2011). Infusing reflective practice in eLearning courses—Can widgets help? International Journal of Technology Enhanced Learning, 3 (1), 93 – 109.</bibtext> </blist> <blist> <bibtext> Wang, M., Yuan, B., Kirschner, P. A., Kushniruk, A. W., & Peng, J. (2018). Reflective learning with complex problems in a visualization‐based learning environment with expert support. Computers in Human Behavior, 87, 406 – 415.</bibtext> </blist> </ref> <aug> <p>By Mei‐Rong Alice Chen; Gwo‐Jen Hwang and Yu‐Ying Chang</p> <p>Reported by Author; Author; Author</p> <p></p> <p>Mei‐Rong Alice Chen is a PhD candidate at the Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology. Her research interests include flipped learning and digital game‐based learning.</p> <p>Gwo‐Jen Hwang is a chair professor at the Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology. His research interests include mobile learning, digital game‐based learning, flipped classroom and AI in education.</p> <p>Yu‐Ying Chang is an assistant professor at the Department of English, Tamkang University, Taiwan. Her research interests include EFL learning, flipped learning, and digital learning.</p> </aug> <nolink nlid="nl1" bibid="bib41" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib65" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib11" firstref="ref5"></nolink> <nolink nlid="nl4" bibid="bib34" firstref="ref6"></nolink> <nolink nlid="nl5" bibid="bib47" firstref="ref7"></nolink> <nolink nlid="nl6" bibid="bib67" firstref="ref9"></nolink> <nolink nlid="nl7" bibid="bib14" firstref="ref10"></nolink> <nolink nlid="nl8" bibid="bib35" firstref="ref12"></nolink> <nolink nlid="nl9" bibid="bib55" firstref="ref13"></nolink> <nolink nlid="nl10" bibid="bib40" firstref="ref16"></nolink> <nolink nlid="nl11" bibid="bib36" firstref="ref17"></nolink> <nolink nlid="nl12" bibid="bib15" firstref="ref19"></nolink> <nolink nlid="nl13" bibid="bib32" firstref="ref20"></nolink> <nolink nlid="nl14" bibid="bib38" firstref="ref21"></nolink> <nolink nlid="nl15" bibid="bib37" firstref="ref22"></nolink> <nolink nlid="nl16" bibid="bib64" firstref="ref28"></nolink> <nolink nlid="nl17" bibid="bib60" firstref="ref35"></nolink> <nolink nlid="nl18" bibid="bib61" firstref="ref37"></nolink> <nolink nlid="nl19" bibid="bib24" firstref="ref38"></nolink> <nolink nlid="nl20" bibid="bib49" firstref="ref40"></nolink> <nolink nlid="nl21" bibid="bib66" firstref="ref41"></nolink> <nolink nlid="nl22" bibid="bib44" firstref="ref43"></nolink> <nolink nlid="nl23" bibid="bib19" firstref="ref45"></nolink> <nolink nlid="nl24" bibid="bib63" firstref="ref46"></nolink> <nolink nlid="nl25" bibid="bib16" firstref="ref48"></nolink> <nolink nlid="nl26" bibid="bib17" firstref="ref49"></nolink> <nolink nlid="nl27" bibid="bib62" firstref="ref52"></nolink> <nolink nlid="nl28" bibid="bib27" firstref="ref53"></nolink> <nolink nlid="nl29" bibid="bib68" firstref="ref54"></nolink> <nolink nlid="nl30" bibid="bib28" firstref="ref55"></nolink> <nolink nlid="nl31" bibid="bib46" firstref="ref56"></nolink> <nolink nlid="nl32" bibid="bib56" firstref="ref57"></nolink> <nolink nlid="nl33" bibid="bib52" firstref="ref58"></nolink> <nolink nlid="nl34" bibid="bib50" firstref="ref60"></nolink> <nolink nlid="nl35" bibid="bib26" firstref="ref61"></nolink> <nolink nlid="nl36" bibid="bib69" firstref="ref62"></nolink> <nolink nlid="nl37" bibid="bib22" firstref="ref63"></nolink> <nolink nlid="nl38" bibid="bib59" firstref="ref64"></nolink> <nolink nlid="nl39" bibid="bib45" firstref="ref70"></nolink> <nolink nlid="nl40" bibid="bib25" firstref="ref71"></nolink> <nolink nlid="nl41" bibid="bib58" firstref="ref72"></nolink> <nolink nlid="nl42" bibid="bib70" firstref="ref73"></nolink> <nolink nlid="nl43" bibid="bib39" firstref="ref78"></nolink> <nolink nlid="nl44" bibid="bib71" firstref="ref82"></nolink> <nolink nlid="nl45" bibid="bib43" firstref="ref90"></nolink> <nolink nlid="nl46" bibid="bib23" firstref="ref91"></nolink> <nolink nlid="nl47" bibid="bib57" firstref="ref92"></nolink> <nolink nlid="nl48" bibid="bib51" firstref="ref95"></nolink> <nolink nlid="nl49" bibid="bib29" firstref="ref97"></nolink> <nolink nlid="nl50" bibid="bib12" firstref="ref99"></nolink> <nolink nlid="nl51" bibid="bib13" firstref="ref100"></nolink> <nolink nlid="nl52" bibid="bib33" firstref="ref103"></nolink> <nolink nlid="nl53" bibid="bib54" firstref="ref104"></nolink> <nolink nlid="nl54" bibid="bib20" firstref="ref105"></nolink> <nolink nlid="nl55" bibid="bib21" firstref="ref112"></nolink> <nolink nlid="nl56" bibid="bib48" firstref="ref114"></nolink> <nolink nlid="nl57" bibid="bib10" firstref="ref115"></nolink> <nolink nlid="nl58" bibid="bib18" firstref="ref116"></nolink> <nolink nlid="nl59" bibid="bib53" firstref="ref118"></nolink> <nolink nlid="nl60" bibid="bib30" firstref="ref120"></nolink> <nolink nlid="nl61" bibid="bib42" firstref="ref121"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1225719
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: A Reflective Thinking-Promoting Approach to Enhancing Graduate Students' Flipped Learning Engagement, Participation Behaviors, Reflective Thinking and Project Learning Outcomes
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Chen%2C+Mei-Rong+Alice%22">Chen, Mei-Rong Alice</searchLink><br /><searchLink fieldCode="AR" term="%22Hwang%2C+Gwo-Jen%22">Hwang, Gwo-Jen</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-5155-276X">0000-0001-5155-276X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Chang%2C+Yu-Ying%22">Chang, Yu-Ying</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22British+Journal+of+Educational+Technology%22"><i>British Journal of Educational Technology</i></searchLink>. Sep 2019 50(5):2288-2307.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 20
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2019
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Graduate+Students%22">Graduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22Reflection%22">Reflection</searchLink><br /><searchLink fieldCode="DE" term="%22Blended+Learning%22">Blended Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Learner+Engagement%22">Learner Engagement</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Participation%22">Student Participation</searchLink><br /><searchLink fieldCode="DE" term="%22Active+Learning%22">Active Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Projects%22">Student Projects</searchLink><br /><searchLink fieldCode="DE" term="%22Outcomes+of+Education%22">Outcomes of Education</searchLink><br /><searchLink fieldCode="DE" term="%22Instructional+Effectiveness%22">Instructional Effectiveness</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/bjet.12823
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0007-1013
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Although flipped learning has been recognized as being a potential approach enabling students to learn at their own pace before the class and facilitating in-depth peer-to-peer and student-to-teacher interactions in the class, it remains a challenge to promote students' active learning in the before-class stage, which could significantly affect their in-class engagement and learning performance. In this study, a reflective thinking-promoting approach is proposed to facilitate students' learning design project performance, technology-enhanced active engagement, and their reflective thinking and participation in the before-class stage of flipped learning. A quasi-experiment was conducted on a flipped Digital Learning course of a Master's program in a university to evaluate the effects of the approach on students' learning design performance, engagement, reflective thinking and participation. A total of 19 students (7 male and 12 female) were in the experimental group learning with the reflective thinking-promoting approach, while 19 (4 male and 15 female) were in the control group learning with the conventional flipped learning approach. The results indicated that the proposed approach significantly enhanced not only the students' learning design project outcomes and reflective thinking, but also their engagement and participation in the before-class stage of flipped learning.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2019
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1225719
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1225719
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/bjet.12823
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 20
        StartPage: 2288
    Subjects:
      – SubjectFull: Graduate Students
        Type: general
      – SubjectFull: Reflection
        Type: general
      – SubjectFull: Blended Learning
        Type: general
      – SubjectFull: Learner Engagement
        Type: general
      – SubjectFull: Student Participation
        Type: general
      – SubjectFull: Active Learning
        Type: general
      – SubjectFull: Student Projects
        Type: general
      – SubjectFull: Outcomes of Education
        Type: general
      – SubjectFull: Instructional Effectiveness
        Type: general
    Titles:
      – TitleFull: A Reflective Thinking-Promoting Approach to Enhancing Graduate Students' Flipped Learning Engagement, Participation Behaviors, Reflective Thinking and Project Learning Outcomes
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Chen, Mei-Rong Alice
      – PersonEntity:
          Name:
            NameFull: Hwang, Gwo-Jen
      – PersonEntity:
          Name:
            NameFull: Chang, Yu-Ying
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 09
              Type: published
              Y: 2019
          Identifiers:
            – Type: issn-print
              Value: 0007-1013
          Numbering:
            – Type: volume
              Value: 50
            – Type: issue
              Value: 5
          Titles:
            – TitleFull: British Journal of Educational Technology
              Type: main
ResultId 1