View in EDS HTML Full Text PDF Full Text

Barriers Instructors Experience in Adopting Active Learning: Instrument Development

Saved in:
Bibliographic Details
Title: Barriers Instructors Experience in Adopting Active Learning: Instrument Development
Language: English
Authors: Carroll, Laura J. (ORCID 0000-0003-4426-9979), Reeping, David (ORCID 0000-0002-0803-7532), Finelli, Cynthia J. (ORCID 0000-0001-9148-1492), Prince, Michael J., Husman, Jenefer (ORCID 0000-0001-9886-5966), Graham, Matthew (ORCID 0000-0002-0235-5126), Borrego, Maura J. (ORCID 0000-0001-7131-4611)
Source: Journal of Engineering Education. 2023 112(4):1079-1108.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 30
Publication Date: 2023
Sponsoring Agency: National Science Foundation (NSF), Division of Undergraduate Education (DUE)
Contract Number: 1821092
1821036
1821488
1821277
Document Type: Journal Articles
Reports - Research
Descriptors: Active Learning, Engineering Education, Barriers, Teaching Methods, Test Construction, Teacher Attitudes, Time Factors (Learning), Readiness, Learner Engagement, Teacher Role, Educational Environment, Test Reliability, STEM Education
DOI: 10.1002/jee.20557
ISSN: 1069-4730
2168-9830
Abstract: Background: Despite well-documented benefits, instructor adoption of active learning has been limited in engineering education. Studies have identified barriers to instructors' adoption of active learning, but there is no well-tested instrument to measure instructors perceptions of these barriers. Purpose: We developed and tested an instrument to measure instructors' perceptions of barriers to adopting active learning and identify the constructs that coherently categorize those barriers. Method: We used a five-phase process to develop an instrument to measure instructors' perceived barriers to adopting active learning. In Phase 1, we built upon the Faculty Instructional Barriers and Identity Survey (FIBIS) to create a draft instrument. In Phases 2 and 3, we conducted exploratory factor analysis (EFA) on an initial 45-item instrument and a refined 21-item instrument, respectively. We conducted confirmatory factor analysis (CFA) in Phases 4 and 5 to test the factor structure identified in Phases 2 and 3. Results: Our final instrument consists of 17 items and four factors: (1) student preparation and engagement; (2) instructional support; (3) instructor comfort and confidence; and (4) institutional environment/rewards. Instructor responses indicated that time considerations do not emerge as a standalone factor. Conclusions: Our 17-item instrument exhibits a sound factor structure and is reliable, enabling the assessment of perceived barriers to adopting active learning in different contexts. The four factors align with an existing model of instructional change in science, technology, engineering, and mathematics (STEM). Although time is a substantial instructor concern that did not comprise a standalone factor, it is closely related to multiple constructs in our final model.
Abstractor: As Provided
Entry Date: 2023
Accession Number: EJ1396511
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHK-Plg2ge5jO9O-jU5B-BtAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDLF02iWwKLZh1hJs1wIBEICBmxK8HIy3BG-x_fBEHBAgVWUdQqWnqPvkt2ZsVFStk4zdlyFsm4hiZtB2vYUuGZQfeEofMmIoZoJgcTzDZMR3Ot1TtRwjTku0v3FCFPvJvT0V1hyba87jrsQeJOID_T15QOzDpLO0lWBSwi4KnruJx5-e_GoJmehfIpaW3nNiER2tHeSEZ1qytI6W52OHI-vb9xeAU6L0cShbaZAI
Text:
  Availability: 1
  Value: <anid>AN0173054174;6m401oct.23;2023Oct20.06:18;v2.2.500</anid> <title id="AN0173054174-1">Barriers instructors experience in adopting active learning: Instrument development </title> <p>Background: Despite well‐documented benefits, instructor adoption of active learning has been limited in engineering education. Studies have identified barriers to instructors' adoption of active learning, but there is no well‐tested instrument to measure instructors perceptions of these barriers. Purpose: We developed and tested an instrument to measure instructors' perceptions of barriers to adopting active learning and identify the constructs that coherently categorize those barriers. Method: We used a five‐phase process to develop an instrument to measure instructors' perceived barriers to adopting active learning. In Phase 1, we built upon the Faculty Instructional Barriers and Identity Survey (FIBIS) to create a draft instrument. In Phases 2 and 3, we conducted exploratory factor analysis (EFA) on an initial 45‐item instrument and a refined 21‐item instrument, respectively. We conducted confirmatory factor analysis (CFA) in Phases 4 and 5 to test the factor structure identified in Phases 2 and 3. Results: Our final instrument consists of 17 items and four factors: (<reflink idref="bib1" id="ref1">1</reflink>) student preparation and engagement; (<reflink idref="bib2" id="ref2">2</reflink>) instructional support; (<reflink idref="bib3" id="ref3">3</reflink>) instructor comfort and confidence; and (<reflink idref="bib4" id="ref4">4</reflink>) institutional environment/rewards. Instructor responses indicated that time considerations do not emerge as a standalone factor. Conclusions: Our 17‐item instrument exhibits a sound factor structure and is reliable, enabling the assessment of perceived barriers to adopting active learning in different contexts. The four factors align with an existing model of instructional change in science, technology, engineering, and mathematics (STEM). Although time is a substantial instructor concern that did not comprise a standalone factor, it is closely related to multiple constructs in our final model.</p> <p>Keywords: active learning; adoption; barriers; factor analysis; faculty development; instructional change</p> <hd id="AN0173054174-2">INTRODUCTION</hd> <p>Research repeatedly demonstrates the benefits of instructors adopting active learning, defined as classroom activities that engage students in their learning beyond listening to lectures and taking notes (Nguyen et al., [<reflink idref="bib60" id="ref5">60</reflink>]; Prince, [<reflink idref="bib67" id="ref6">67</reflink>]; Theobald et al., [<reflink idref="bib84" id="ref7">84</reflink>]). Active learning occurs when instructors engage students during class time in "meaningful learning activities and think about what they are doing," such as by having students solve problems in groups (Prince, [<reflink idref="bib67" id="ref8">67</reflink>], p. 223). We use this definition of active learning to communicate an alternative to lecture (Lombardi et al., [<reflink idref="bib47" id="ref9">47</reflink>], p. 8). Our definition of active learning encompasses effective teaching practices (e.g., Finelli et al., [<reflink idref="bib24" id="ref10">24</reflink>]), evidence‐based teaching strategies (e.g., Brownell & Tanner, [<reflink idref="bib9" id="ref11">9</reflink>]), and evidence‐based instructional practices (e.g., Lund & Stains, [<reflink idref="bib49" id="ref12">49</reflink>]). Within engineering education, the benefits of active learning include increased student engagement (Lucke et al., [<reflink idref="bib48" id="ref13">48</reflink>]; Prince, [<reflink idref="bib67" id="ref14">67</reflink>]; Rodríguez et al., [<reflink idref="bib70" id="ref15">70</reflink>]), enhanced student learning and academic performance (Freeman et al., [<reflink idref="bib27" id="ref16">27</reflink>]; Prince, [<reflink idref="bib67" id="ref17">67</reflink>]; Theobald et al., [<reflink idref="bib84" id="ref18">84</reflink>]), improved conceptual understanding and problem‐solving skills (National Research Council, [<reflink idref="bib58" id="ref19">58</reflink>]), and improved persistence (Freeman et al., [<reflink idref="bib27" id="ref20">27</reflink>]; Theobald et al., [<reflink idref="bib84" id="ref21">84</reflink>]).</p> <p>Despite the benefits of active learning espoused in the literature, instructors in engineering (Borrego et al., [<reflink idref="bib7" id="ref22">7</reflink>]; Cutler et al., [<reflink idref="bib16" id="ref23">16</reflink>]) and, more broadly, science, technology, engineering, and mathematics (STEM; Apkarian et al., [<reflink idref="bib2" id="ref24">2</reflink>]; Manduca et al., [<reflink idref="bib51" id="ref25">51</reflink>]; Stains et al., [<reflink idref="bib80" id="ref26">80</reflink>]; Viskupic et al., [<reflink idref="bib90" id="ref27">90</reflink>]) have not widely adopted active learning because of obstacles to adoption they have either experienced or have expected to experience in the future (e.g., Finelli et al., [<reflink idref="bib24" id="ref28">24</reflink>]; Shadle et al., [<reflink idref="bib77" id="ref29">77</reflink>]). Given the extensive research on the effectiveness of active learning, we argue that understanding instructors' perceptions of the barriers they experience in adopting active learning and how to promote its broad adoption are necessary to support student learning in engineering (Finelli et al., [<reflink idref="bib24" id="ref30">24</reflink>]). In this study, we develop and test an instrument to measure instructors' perceptions of the barriers to adopting active learning. First, we review overarching themes across exploratory studies on barriers and identify a model of the barriers instructors experience in adopting active learning. Our broad definition of active learning allows us to incorporate the full range of applicable studies on barriers to instructional change. We then adapt an existing instrument of barriers based on our model and iteratively test the reliability and validity of that instrument.</p> <hd id="AN0173054174-3">Barriers to adopting active learning</hd> <p>Researchers have engaged in exploratory research to identify the barriers that instructors believe prevent their adoption of active learning (e.g., Brownell & Tanner, [<reflink idref="bib9" id="ref31">9</reflink>]; Dancy & Henderson, [<reflink idref="bib17" id="ref32">17</reflink>]; Finelli et al., [<reflink idref="bib24" id="ref33">24</reflink>]; Henderson, [<reflink idref="bib30" id="ref34">30</reflink>]; Henderson et al., [<reflink idref="bib31" id="ref35">31</reflink>]; Henderson & Dancy, [<reflink idref="bib32" id="ref36">32</reflink>]; Michael, [<reflink idref="bib55" id="ref37">55</reflink>]; Morales & Prince, [<reflink idref="bib56" id="ref38">56</reflink>]; Shadle et al., [<reflink idref="bib77" id="ref39">77</reflink>]; Sturtevant & Wheeler, [<reflink idref="bib83" id="ref40">83</reflink>]; Tsang & Harris, [<reflink idref="bib87" id="ref41">87</reflink>]). Among the myriad of issues this body of literature has uncovered, examples of identified barriers include inadequate preparation time, concerns about content coverage, unsupportive colleagues, a lack of access to professional development opportunities, and misalignment with instructor reward structures (e.g., Brownell & Tanner, [<reflink idref="bib9" id="ref42">9</reflink>]; Finelli et al., [<reflink idref="bib24" id="ref43">24</reflink>]; Henderson, [<reflink idref="bib30" id="ref44">30</reflink>]; Shadle et al., [<reflink idref="bib77" id="ref45">77</reflink>]; Sturtevant & Wheeler, [<reflink idref="bib83" id="ref46">83</reflink>]). Instructors consistently identify a lack of time as a key barrier to instructional change, such as the initial time commitment to transitioning a class from a traditional lecture‐focused pedagogical approach to an active learning model (Brownell & Tanner, [<reflink idref="bib9" id="ref47">9</reflink>]; Dancy & Henderson, [<reflink idref="bib17" id="ref48">17</reflink>]; Finelli et al., [<reflink idref="bib24" id="ref49">24</reflink>]; Henderson et al., [<reflink idref="bib31" id="ref50">31</reflink>]; Henderson & Dancy, [<reflink idref="bib32" id="ref51">32</reflink>]; Morales & Prince, [<reflink idref="bib56" id="ref52">56</reflink>]; Shadle et al., [<reflink idref="bib77" id="ref53">77</reflink>]; Sturtevant & Wheeler, [<reflink idref="bib83" id="ref54">83</reflink>]).</p> <p>Although researchers have identified many barriers to instructors' adoption of active learning, researchers need more than a list of barriers to examine the effectiveness of interventions and policies to implement instructional change. Identifying underlying factors that create barriers to active learning adoption and an instrument that measures those factors is needed. Such an instrument could enable practitioners and researchers to quantify how strongly an instructor might believe that adopting active learning in their classrooms is infeasible, allowing for interventions to aid apprehensive individuals in exploring the range of possibilities offered by active learning pedagogies.</p> <hd id="AN0173054174-4">Modeling and measuring instructors' perceptions of barriers to adopting active learning</hd> <p>Exploratory research has led to lists of barriers to instructors' adoption of active learning, which have been explored using different perspectives to understand the underlying mechanisms for these beliefs and how misperceptions can be dispelled. For instance, researchers have applied theories of motivation and instructional change to identify overarching themes connecting these barriers (e.g., Finelli et al., [<reflink idref="bib24" id="ref55">24</reflink>]; Lund & Stains, [<reflink idref="bib49" id="ref56">49</reflink>]). Despite attempts at reconciling the diverse barriers instructors report into a more parsimonious framework, there is a lack of consistency among these themes across studies of instructors' perceptions of barriers. Accordingly, we review prior work on lists of barriers and identify connecting themes common across studies. This work is necessary for (<reflink idref="bib1" id="ref57">1</reflink>) instrument development (e.g., Watkins, [<reflink idref="bib91" id="ref58">91</reflink>]), (<reflink idref="bib2" id="ref59">2</reflink>) research studies (e.g., Babbie, [<reflink idref="bib3" id="ref60">3</reflink>]), and ultimately (<reflink idref="bib3" id="ref61">3</reflink>) strategies to effectively and efficiently reduce barriers within institutions.</p> <p>One recently developed instrument to measure barriers to instructors' adoption of active learning is called the Faculty Instructional Barriers and Identity Survey (FIBIS; Sturtevant & Wheeler, [<reflink idref="bib83" id="ref62">83</reflink>]). The FIBIS items were developed through a thorough literature review, survey item development, and consideration of face and content validity; however, the instrument is still in an early stage of development. At this time, published papers about the FIBIS have not demonstrated sufficient evidence of reliable subscales or construct validity using techniques like confirmatory factor analysis; the original study's results were described using pilot data (<emph>n</emph> = 69), so this limitation is understandable. In our study, we adapted the "Barriers to the Use of Evidence‐Based Instructional Practices" section of the FIBIS based on our comprehensive model of faculty perceptions of barriers to adopting active learning arising from our literature review. Next, we engaged in exploratory and confirmatory factor analysis, testing the instrument's reliability and construct validity.</p> <hd id="AN0173054174-5">RESEARCHER POSITIONALITY</hd> <p>The authors of this manuscript all conduct their research within the engineering education research community, with two researchers specializing in educational psychology research and a third in quantitative educational research methodology. Four of the seven authors have taught engineering courses using active learning, and five have led faculty development seminars. Despite having a shared commitment to improving student learning using active learning, all authors have experienced frustration using such pedagogies in large and small undergraduate courses, which motivates our interest in this topic.</p> <p>As quantitative, post‐positivist researchers, we ontologically assume there is a "true" set of latent constructs describing barriers to instructional change, which can be epistemologically uncovered by applying theory to analyze self‐reported barriers of a large sample of instructors. Author 2 diverges in his perspective, adopting a critical realism worldview—which establishes a distinction between the real and observable world. We agree that an important shift in individual instructors' approaches to teaching from information‐transmission/teacher‐focused to conceptual‐change/student‐focused (Trigwell et al., [<reflink idref="bib86" id="ref63">86</reflink>]) is necessary for full adoption of active learning. Yet, we note that barriers are most often self‐reported by faculty who may be unable or unwilling to discuss attitudinal barriers explicitly. Thus, we approach this research anticipating that faculty beliefs could be best changed once latent barrier constructs are identified. The goal of the manuscript is, therefore, to identify these latent constructs and expand upon the theory.</p> <hd id="AN0173054174-6">LITERATURE REVIEW</hd> <p>For researchers to understand and explore the barriers instructors experience in adopting active learning, an instrument to measure the perceptions of these barriers is needed (Marlor et al., [<reflink idref="bib52" id="ref64">52</reflink>]). Studies on instructors' perceptions of the barriers to adopting active learning have primarily identified lists of barriers instructors experience in adopting active learning (e.g., Finelli et al., [<reflink idref="bib24" id="ref65">24</reflink>]; Shadle et al., [<reflink idref="bib77" id="ref66">77</reflink>]). Despite the proliferation of work about barriers to active learning, no unified model focused on instructors' perceptions of these barriers has emerged. Instead, researchers have applied many different models (e.g., Shadle et al., [<reflink idref="bib77" id="ref67">77</reflink>]), as well as theories of motivation (e.g., Finelli et al., [<reflink idref="bib24" id="ref68">24</reflink>]) and change (e.g., Lund & Stains, [<reflink idref="bib49" id="ref69">49</reflink>]), in exploring barriers. A model provides a starting point, providing hypothesized constructs, for instrument development. Thus, the absence of a clear choice of a model to guide instrument development makes it difficult to develop a barriers instrument because instrument development is inherently reliant on theory (Watkins, [<reflink idref="bib91" id="ref70">91</reflink>]).</p> <hd id="AN0173054174-7">Models and theories applied to study barriers</hd> <p>Studies on barriers to instructors' adoption of active learning employ components of broader instructional change models or theories (Borrego & Henderson, [<reflink idref="bib8" id="ref71">8</reflink>]). Because researchers have yet to converge on a categorization of instructor barriers, we align their findings in this literature review. However, this lack of convergence underscores the need for theory‐based, quantitative development of an instrument for understanding instructor perceptions of these barriers. Although we considered many researchers' characterization of barriers (e.g., Andrews & Lemons, [<reflink idref="bib1" id="ref72">1</reflink>]; Dancy & Henderson, [<reflink idref="bib17" id="ref73">17</reflink>]; Lattuca & Pollard, [<reflink idref="bib46" id="ref74">46</reflink>]; Shadle et al., [<reflink idref="bib77" id="ref75">77</reflink>]), we selected three distinct ways of characterizing barriers introduced by Brownell and Tanner ([<reflink idref="bib9" id="ref76">9</reflink>]), Finelli et al. ([<reflink idref="bib24" id="ref77">24</reflink>]), and Lund and Stains ([<reflink idref="bib49" id="ref78">49</reflink>]). The studies we review here consider faculty beliefs about learning as a social versus cognitive process to varying degrees. Still, we acknowledge that different discipline‐based education research communities hold different ontological assumptions about the nature of learning and definitions of active learning, which influence the choice of research methods and questions (Lombardi et al., [<reflink idref="bib47" id="ref79">47</reflink>]). We will review the premise of each framework individually to elucidate each of their perspectives.</p> <p>First, Brownell and Tanner ([<reflink idref="bib9" id="ref80">9</reflink>]) situated instructors' perceived barriers to active learning in relation to scientists' professional identity and conceptually expanded upon three commonly identified categories of barriers: lack of training, lack of time, and lack of incentives (p. 340). Lack of time includes competition with time for research, teaching load, and class preparation time. Brownell and Tanner ([<reflink idref="bib9" id="ref81">9</reflink>]) identified barriers falling into a new, fourth category, scientists' professional identity, such as cultural norms in science prioritizing research over teaching. Furthermore, they noted a misalignment of scientists' professional identities with the premise of instructional change, such as adopting active learning. Brownell and Tanner ([<reflink idref="bib9" id="ref82">9</reflink>]) frame the practice of teaching in the same way one was taught (primarily in a lecture format) as a scientific norm and barrier, but this article does not engage critically with theories of learning or the nature of active learning.</p> <p>Second, instead of approaching the issue of barriers from the perspective of identity, Finelli et al. ([<reflink idref="bib24" id="ref83">24</reflink>]) applied expectancy‐value theory (EVT; Eccles & Wigfield, [<reflink idref="bib21" id="ref84">21</reflink>]) when they interviewed groups of engineering faculty about influences on their adoption of effective teaching practices. Applying EVT to instructional change, they found that an instructor's motivation to change is related to the value they ascribe to the change and the extent to which they expect to succeed (Finelli et al., [<reflink idref="bib24" id="ref85">24</reflink>]). To elicit these barriers, the protocol focused on three effective teaching practices consistent with a broad, engineering education definition of active learning as actively engaging students with each other or example problems during class time (Lombardi et al., [<reflink idref="bib47" id="ref86">47</reflink>]). This application of EVT indirectly addresses instructor ontological beliefs about learning, in that instructors would not see value unless they viewed learning as a social process. Through this application of EVT, Finelli et al. ([<reflink idref="bib24" id="ref87">24</reflink>]) inductively identified seven categories of barriers: infrastructure and culture; knowledge and skills of effective teaching practices; student experience; time; classroom and curriculum; personal disposition; and networking and community. They include a distinct time category, which incorporates the "time to restructure a course," "time to learn about effective teaching practices," and "preparation time for class sessions" (p. 340).</p> <p>Finally, chemistry education researchers Lund and Stains ([<reflink idref="bib49" id="ref88">49</reflink>]) developed the instructional innovation‐decision process by applying Rogers' ([<reflink idref="bib71" id="ref89">71</reflink>]) model of change, the innovation‐decision process, to the instructional context. Like Finelli et al. ([<reflink idref="bib24" id="ref90">24</reflink>]), Lund and Stains emphasized the social processes underlying instructional change and asked participants about specific evidence‐based instructional practices. They focused on identifying departmental and disciplinary differences in science instructors' awareness and adoption of evidence‐based instructional practices. Their instructional change model identified four overarching categories of influences, encompassing barriers. Contextual influences include (<reflink idref="bib1" id="ref91">1</reflink>) characteristics of the learning environment, such as classroom aspects such as size, layout, instructional support, student preparation, and learning outcomes, and (<reflink idref="bib2" id="ref92">2</reflink>) departmental factors, such as instructional flexibility, rewards/promotion, and time constraints due to research and service (p. 4–5). Individual influences include (<reflink idref="bib3" id="ref93">3</reflink>) attitudes and beliefs about teaching as well as (<reflink idref="bib4" id="ref94">4</reflink>) pedagogical experiences. By emphasizing attitudes and beliefs about teaching, this change model considers instructors' ontological beliefs about whether and how students learn by actively engaging with peers and concepts. Lund and Stains' ([<reflink idref="bib49" id="ref95">49</reflink>]) model does not include an influence category specific to time; instead, time permeates each category. Of the three approaches we presented, Lund and Stains ([<reflink idref="bib49" id="ref96">49</reflink>]) is the only example of researchers applying a theory or model to study instructor barriers that incorporates a quantitative perspective. Lund and Stains ([<reflink idref="bib49" id="ref97">49</reflink>]) developed and administered survey items for their four influence categories. However, their quantitative analysis did not include factor analysis, correlations among items, or Cronbach's alphas for reliability analyses to support their characterization scheme.</p> <p>In summary, models and theories used to study barriers instructors experience in adopting active learning offer multiple perspectives on characterizing barriers, but they lack quantitative evidence of their causality. A proxy for this causality is to understand the extent to which instructors perceive the myriad of barriers as applicable to them using an instrument. We emphasize the importance of instructors' perceptions in their adoption of active learning because they are choosing to make an instructional change. Therefore, if we understand instructors' perceptions of the barriers to adopting active learning, we can systematically focus faculty development efforts on changing these perceptions through appropriate interventions.</p> <hd id="AN0173054174-8">Time as a barrier</hd> <p>Given the literature reviewed thus far, we would like to highlight one barrier in particular, time. Time as a barrier can fit into several categories as conceptualized in the literature, and little work explores how and to what extent an explicit focus on time captures a reliable barrier. Of the 18 STEM instructor barrier categories identified by Shadle et al. ([<reflink idref="bib77" id="ref98">77</reflink>]), time is the barrier shared most frequently by instructors. However, more than merely providing instructors with sufficient time is necessary to ensure instructional change (Collinson & Fedoruk Cook, [<reflink idref="bib12" id="ref99">12</reflink>]; Shadle et al., [<reflink idref="bib77" id="ref100">77</reflink>]). Time‐related barriers are broad in scope; they encompass inadequate time for learning new instructional practices and developing course materials and in‐class activities (Finelli et al., [<reflink idref="bib24" id="ref101">24</reflink>]). Faculty do not describe time as a single consideration; instead, it is multifaceted—faculty report having only short, fragmented blocks of time that hampers change efforts, having other things compete for their time, and lacking the motivation to use available time in specific ways (Zibrowski et al., [<reflink idref="bib92" id="ref102">92</reflink>]). Furthermore, time permeates other barriers, such as personal and institutional priorities, tenure/promotion, and instructional support (Shadle et al., [<reflink idref="bib77" id="ref103">77</reflink>]). In our study, we attempt to isolate time as a factor in our instrument development process.</p> <hd id="AN0173054174-9">Prior instrument development for measuring barriers</hd> <p>In contrast to the availability of applicable models to describe the various barriers to instructor adoption of active learning, only one instrument measures these perceived barriers to instructors adopting active learning. The FIBIS is in the early stages of instrument development and includes a 48‐item section on instructional barriers included in the supplemental material (Sturtevant & Wheeler, [<reflink idref="bib83" id="ref104">83</reflink>]). Sturtevant and Wheeler's item development process involved an extensive literature review and compiling barriers shared by instructors from disciplines within and outside of STEM (e.g., Michael, [<reflink idref="bib55" id="ref105">55</reflink>]). They initially organized barriers into four categories: student characteristics, teacher characteristics, technical/pedagogical issues, and departmental/institutional. Item revision involved seeking recommendations from an expert panel review to achieve face and content validity (Sturtevant & Wheeler, [<reflink idref="bib83" id="ref106">83</reflink>]), but the instrument was not systematically tested using conventional techniques for instrument development.</p> <p>Our literature review indicates that the barriers section of the FIBIS needs further expansion and refinement before conducting quantitative assessments. Several barriers to instructors' adoption of active learning identified by Finelli et al. ([<reflink idref="bib24" id="ref107">24</reflink>]) are absent from the FIBIS. These barriers are related to course learning outcomes, teaching policies, students' pedagogy expectations, professional development time, instructors' comfort in the facilitator role, interactions with students, and student feedback. Additionally, the wording of a few FIBIS items lacks a clear, consistent meaning (e.g., "I am afraid that trying evidence‐based instructional practices in my course will not work—I am afraid to fail.") or addresses multiple barriers in a single double‐barreled item (e.g., "I have to cover too much content and do not have time to implement evidence‐based instructional practices;" Sturtevant & Wheeler, [<reflink idref="bib83" id="ref108">83</reflink>], supplemental material).</p> <p>Furthermore, continued instrument development, such as in the case of the FIBIS, requires theory‐guided factor analysis (Watkins, [<reflink idref="bib91" id="ref109">91</reflink>]). Initial administration of the FIBIS to STEM instructors resulted in a limited sample size (<emph>n</emph> = 69) and the inability to conduct an exploratory factor analysis (EFA) to identify the underlying factor structure (Sturtevant & Wheeler, [<reflink idref="bib83" id="ref110">83</reflink>]). Instead, items were grouped into eight categories based on correlations and Cronbach's alphas. Five reliable categories emerged: students as barriers (<emph>α</emph> = .76), departmental barriers (<emph>α</emph> = .78), faculty beliefs (<emph>α</emph> = .80), limited support for instruction (<emph>α</emph> = .68), and pedagogical experiences (<emph>α</emph> = .80). Two categories had Cronbach's alphas below accepted cutoffs and were deemed unreliable: technical/classroom issues and standalone statements (alphas not provided in reference). Lastly, an "other" category was added after their pilot survey administration (Sturtevant & Wheeler, [<reflink idref="bib83" id="ref111">83</reflink>]).</p> <hd id="AN0173054174-10">Instrument and model alignment</hd> <p>The FIBIS categories provide a starting point to compare other applicable models for instructors' perceptions of barriers to adopting active learning. In Table 1, we show how categories from selected models of our literature review (i.e., Brownell & Tanner, [<reflink idref="bib9" id="ref112">9</reflink>]; Finelli et al., [<reflink idref="bib24" id="ref113">24</reflink>]; Lund & Stains, [<reflink idref="bib49" id="ref114">49</reflink>]) map to the FIBIS categories and the associated instructor barrier items. It illustrates the differences between the FIBIS categories, FIBIS items, and other models used to characterize barriers. The FIBIS categories, items, and barriers are in the Supporting Information and align with Table 1.</p> <p>1 TABLE Barriers to adopting active learning (Column 2) mapped to the eight FIBIS categories (Column 1) and categories from other relevant models (Columns 3–5).</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">FIBIS category (Sturtevant & Wheeler, <xref ref-type="bibr" rid="bibr83">2019</xref>, p. 18)</th><th align="left">Instructor barrier derived from FIBIS items</th><th align="left">Finelli et al. (<xref ref-type="bibr" rid="bibr24">2014</xref>, p. 340) Category</th><th align="left">Brownell and Tanner (<xref ref-type="bibr" rid="bibr9">2012</xref>, p. 340, 341) Category</th><th align="left">Lund and Stains (<xref ref-type="bibr" rid="bibr49">2015</xref>, p. 4) Category</th></tr></thead><tbody valign="top"><tr><td align="left">Students as barriers</td><td align="left">Student engagement</td><td align="left">Student experience</td><td align="left" /><td align="left" /></tr><tr><td align="left">Student preparation</td><td align="left" /><td align="left" /><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Student response</td><td align="left">Student experience</td><td align="left" /><td align="left" /></tr><tr><td align="left">Student resistance</td><td align="left">Student experience</td><td align="left" /><td align="left" /></tr><tr><td align="left">Student outcomes<xref ref-type="fn" rid="tfn1" /></td><td align="left">Student experience</td><td align="left" /><td align="left" /></tr><tr><td align="left">Instructor–student rapport<xref ref-type="fn" rid="tfn1" /></td><td align="left">Student experience</td><td align="left" /><td align="left" /></tr><tr><td align="left">Student feedback<xref ref-type="fn" rid="tfn1" /></td><td align="left">Student experience</td><td align="left" /><td align="left" /></tr><tr><td align="left">Student expectations<xref ref-type="fn" rid="tfn1" /></td><td align="left">Infrastructure and culture</td><td align="left" /><td align="left">Departmental factors</td></tr><tr><td align="left">Departmental barriers</td><td align="left">Logistics of change</td><td align="left">Infrastructure and culture</td><td align="left">Professional identity</td><td align="left">Departmental factors</td></tr><tr><td align="left">Colleagues' perspectives</td><td align="left">Networking and community</td><td align="left" /><td align="left">Departmental factors</td></tr><tr><td align="left">Teaching norms</td><td align="left">Infrastructure and culture</td><td align="left">Professional identity</td><td align="left">Departmental factors</td></tr><tr><td align="left">Departmental/institutional priorities</td><td align="left">Infrastructure and culture</td><td align="left">Professional identity, Lack of time</td><td align="left">Departmental factors</td></tr><tr><td align="left">Teaching policies<xref ref-type="fn" rid="tfn1" /></td><td align="left">Infrastructure and culture</td><td align="left" /><td align="left">Departmental factors</td></tr><tr><td align="left">Faculty beliefs</td><td align="left">Course planning time</td><td align="left">Time</td><td align="left">Lack of time</td><td align="left">Attitudes and beliefs about teaching</td></tr><tr><td align="left">Professional development time<xref ref-type="fn" rid="tfn1" /></td><td align="left">Time</td><td align="left">Lack of time</td><td align="left">Attitudes and beliefs about teaching</td></tr><tr><td align="left">Teaching evaluations</td><td align="left">Infrastructure and culture</td><td align="left">Lack of incentives</td><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Classroom management</td><td align="left">Personal disposition</td><td align="left">Lack of training</td><td align="left">Attitudes and beliefs about teaching</td></tr><tr><td align="left">Pedagogical knowledge</td><td align="left" /><td align="left">Lack of training</td><td align="left">Pedagogical experiences</td></tr><tr><td align="left">Class preparation time</td><td align="left">Time</td><td align="left">Lack of time</td><td align="left">Attitudes and beliefs about teaching</td></tr><tr><td align="left">Course content</td><td align="left">Student experience</td><td align="left" /><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Course subject matter</td><td align="left" /><td align="left" /><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Student assessment</td><td align="left" /><td align="left" /><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Research evidence</td><td align="left">Knowledge and skills</td><td align="left">Lack of training</td><td align="left">Attitudes and beliefs about teaching</td></tr><tr><td align="left">Student response</td><td align="left">Student experience</td><td align="left" /><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Confidence in implementation</td><td align="left">Personal disposition</td><td align="left">Lack of training</td><td align="left">Attitudes and beliefs about teaching</td></tr><tr><td align="left">Personal priorities</td><td align="left">Personal disposition</td><td align="left">Professional identity</td><td align="left">Attitudes and beliefs about teaching</td></tr><tr><td align="left">Implementation logistics</td><td align="left" /><td align="left">Lack of training</td><td align="left" /></tr><tr><td align="left">Availability of information</td><td align="left">Knowledge and skills</td><td align="left">Lack of training</td><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Comfort with role change<xref ref-type="fn" rid="tfn1" /></td><td align="left">Personal disposition</td><td align="left">Professional identity</td><td align="left" /></tr><tr><td align="left">Limited support for instruction</td><td align="left">Financial resources</td><td align="left" /><td align="left" /><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Technology support</td><td align="left">Knowledge and skills</td><td align="left" /><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Teaching assistant (TA) support</td><td align="left">Knowledge and skills</td><td align="left" /><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Colleagues' perspectives</td><td align="left">Networking and community</td><td align="left" /><td align="left">Departmental factors</td></tr><tr><td align="left">Instructional support staff</td><td align="left">Knowledge and skills</td><td align="left" /><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Past experiences</td><td align="left">Past experiences</td><td align="left" /><td align="left" /><td align="left">Pedagogical experiences</td></tr><tr><td align="left">Technical/classroom issues</td><td align="left">Required technology</td><td align="left">Knowledge and skills</td><td align="left">Lack of training</td><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Physical classroom layout</td><td align="left">Classroom and curriculum</td><td align="left" /><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Standalone statements</td><td align="left">Tenure/promotion criteria</td><td align="left">Infrastructure and culture</td><td align="left">Lack of incentives</td><td align="left">Departmental factors</td></tr><tr><td align="left">Class size</td><td align="left">Classroom and curriculum</td><td align="left" /><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Physical classroom layout</td><td align="left">Classroom and curriculum</td><td align="left" /><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Time structure of course</td><td align="left">Time</td><td align="left" /><td align="left">Characteristics of the learning environment</td></tr><tr><td align="left">Other</td><td align="left">Incentives and rewards</td><td align="left">Infrastructure and culture</td><td align="left">Lack of incentives</td><td align="left">Departmental factors</td></tr></tbody></table> </ephtml> </p> <p>1 a These barriers are derived from Dancy and Henderson ([<reflink idref="bib17" id="ref115">17</reflink>], p. 9), Finelli et al. ([<reflink idref="bib24" id="ref116">24</reflink>], p. 340), and Shadle et al. ([<reflink idref="bib77" id="ref117">77</reflink>], p. 5–6), rather than from FIBIS items.</p> <p>We describe this mapping in detail for the instructional innovation‐decision process influences (Lund & Stains, [<reflink idref="bib49" id="ref118">49</reflink>]; Table 1, fifth column). We find that the barriers addressed by items residing in a single FIBIS category often map to multiple instructional innovation‐decision process influence categories. For each item's barrier listed in the second column, we compare the FIBIS category in which it resides and the associated innovation‐decision process category (i.e., influence). First, within the FIBIS students as barriers category, the two items addressing barriers of student preparation and expectations are characterized by Lund and Stains ([<reflink idref="bib49" id="ref119">49</reflink>]) as characteristics of the learning environment and departmental factors, respectively. Lund and Stains ([<reflink idref="bib49" id="ref120">49</reflink>]) do not specifically address the other six barriers within this FIBIS category. Second, within the departmental barriers FIBIS category, all five barriers align with Lund and Stains' ([<reflink idref="bib49" id="ref121">49</reflink>]) category of the same name. Third, within the FIBIS faculty beliefs category, the instructional innovation‐decision process model groups the 14 barriers into one of three different influence categories: characteristics of the learning environment, attitudes and beliefs about teaching, or pedagogical experiences. Some categories do not align with the Lund and Stains ([<reflink idref="bib49" id="ref122">49</reflink>]) categorization; for example, Lund and Stains' results do not cover two items (implementation logistics and comfort with role change) in the FIBIS faculty beliefs category. Fourth, within the limited support for instruction FIBIS category, they describe four barriers as characteristics of the learning environment and one barrier as departmental factors in the instructional innovation‐decision process. Fifth, the past experiences FIBIS category contains items addressing barriers within the pedagogical experiences influences in the instructional innovation‐decision process. Lastly, barriers comprising the FIBIS categories of technical/classroom issues, standalone statements, and other map to Lund and Stains' ([<reflink idref="bib49" id="ref123">49</reflink>]) characteristics of the learning environment or departmental factors influences.</p> <p>Based on our review of the literature, the themes of instructors' perceptions of barriers to adopting active learning (e.g., Finelli et al., [<reflink idref="bib24" id="ref124">24</reflink>]; Shadle et al., [<reflink idref="bib77" id="ref125">77</reflink>]) are not fully represented by the items or the categories within the one instrument (FIBIS) that we have found related to barriers to adopting active learning. However, the FIBIS provides us with a framework to explore engineering instructors' perceptions of barriers to adopting active learning. Using the FIBIS categories as a framework for looking across qualitative and quantitative research on instructors' perceptions of barriers (Table 1), we determined that the categories identified by Sturtevant and Wheeler ([<reflink idref="bib83" id="ref126">83</reflink>]) could better align with other research on instructors' perceptions of barriers. Accordingly, we build on the work of Sturtevant and Wheeler ([<reflink idref="bib83" id="ref127">83</reflink>]), using the FIBIS categories and items as an initial model, and we include items from other models used to characterize barriers instructors face in adopting active learning.</p> <hd id="AN0173054174-11">RESEARCH AIM</hd> <p>Broadly, our research aim is to improve engineering education through more effective teaching practices, such as active learning, and this improvement requires instructional change. In this article, we develop and test a concise instrument with reliable subscales to measure instructors' perceptions of barriers to adopting active learning by building on the barriers section of the FIBIS (Sturtevant & Wheeler, [<reflink idref="bib83" id="ref128">83</reflink>]). Following recognized instrument development practices (McCoach et al., [<reflink idref="bib54" id="ref129">54</reflink>]), we use a five‐phase design process leveraging factor analysis. Subsequently, we compare our subscales to the characterization of barriers (i.e., categories or influences) in previously proposed models and propose a model modification based on our findings. Finally, we discuss our findings and present conceptual and practical implications, the limitations of our work, and directions for future research.</p> <p>We pose two research questions:</p> <hd id="AN0173054174-12">RQ1</hd> <p>What is the factor structure of an instrument to measure instructors' perceptions of barriers to adopting active learning? How is time related to the factor structure?</p> <hd id="AN0173054174-13">RQ2</hd> <p>What model(s) best align with the factor structure, providing a theoretically based means to categorize instructors' perceptions of barriers to adopting active learning?</p> <hd id="AN0173054174-14">OVERVIEW OF STUDY PHASES</hd> <p>In the five‐phase study design, shown in Table 2, we build upon the 48‐item FIBIS to develop and test a concise instrument to measure instructors' perceived barriers to adopting active learning. In Phase 1, we refine a draft 45‐item instrument, and in Phases 2 and 3, we conduct EFAs and refine the instruments to comprise 21 items and 17 items, respectively. In Phases 4 and 5, we conduct confirmatory factor analyses (CFAs) using two different datasets to verify the identified factor structure.</p> <p>2 TABLE Phases of our instrument development process.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">Study phases</th><th align="left">Description of the phase</th></tr></thead><tbody valign="top"><tr><td align="left">Phase 1. Draft instrument refinement</td><td align="left"><list list-type="Bullet"><list-item><p>Refine the 48‐item FIBIS</p></list-item><list-item><p>Outcome: 45‐item draft instrument</p></list-item></list></td></tr><tr><td align="left">Phase 2. First exploratory factor analysis (EFA)</td><td align="left"><list list-type="Bullet"><list-item><p>Administer the 45‐item draft instrument (n  = 368)</p></list-item><list-item><p>Conduct EFA to explore the underlying factor structure</p></list-item><list-item><p>Revise the draft instrument</p></list-item><list-item><p>Outcome: 21‐item refined instrument</p></list-item></list></td></tr><tr><td align="left">Phase 3. Second exploratory factor analysis</td><td align="left"><list list-type="Bullet"><list-item><p>Administer 21‐item refined instrument (n  = 410)</p></list-item><list-item><p>Conduct EFA of the 21‐item refined instrument (n  = 205)</p></list-item><list-item><p>Revise the refined instrument</p></list-item><list-item><p>Outcome: 17‐item final instrument</p></list-item></list></td></tr><tr><td align="left">Phase 4. Confirmatory factor analysis (CFA)</td><td align="left"><list list-type="Bullet"><list-item><p>Conduct CFA of the 17‐item final instrument (n  = 205)</p></list-item><list-item><p>Outcome: 17‐item final instrument</p></list-item></list></td></tr><tr><td align="left">Phase 5. Confirmatory factor analysis</td><td align="left"><list list-type="Bullet"><list-item><p>Administer 17‐item refined instrument (n  = 175)</p></list-item><list-item><p>Conduct CFA to evaluate the factor structure</p></list-item><list-item><p>Outcome: Confirmation of 17‐item final instrument</p></list-item></list></td></tr></tbody></table> </ephtml> </p> <hd id="AN0173054174-15">PHASE 1: DRAFT INSTRUMENT DEVELOPMENT</hd> <p>In the first phase of our project, we drew items from the existing FIBIS survey (48 items; Sturtevant & Wheeler, [<reflink idref="bib83" id="ref130">83</reflink>]) to develop a 45‐item draft instrument. Our items and, if applicable, the corresponding FIBIS items are provided in the Supporting Information. We slightly reworded some FIBIS items. For example, we used "instructor" instead of "faculty" and "active learning" instead of "evidence‐based instructional practices" and added the personal pronoun "my." We modified survey items that lacked a clear, consistent meaning or addressed multiple barriers, such as "I have to cover too much content and do not have time to implement evidence‐based instructional practices." Additionally, we dropped 10 repetitive items, resulting in 38 potential survey items.</p> <p>We then added seven items to include barriers that, according to our literature review, were not included in the FIBIS. These barriers related to professional development time, course learning outcomes, instructor–student rapport, student feedback, teaching policies, student expectations, and comfort with role changes. We relied heavily on the qualitative analysis of faculty focus groups by Finelli et al. ([<reflink idref="bib24" id="ref131">24</reflink>]) and faculty interviews by DeMonbrun et al. ([<reflink idref="bib18" id="ref132">18</reflink>]) for the selection and wording of these items. These items are specifically noted with a footnote in Table 3. For face validity, four faculty members in engineering and education, selected for their subject matter expertise, reviewed item wording, discussed unclear items, recommended revised language, and then agreed on changes by majority consensus. Our resulting draft survey had 45 items.</p> <p>3 TABLE Phase 2 EFA results (for factor loadings greater than 0.25) for our initial 45‐item draft instrument, with the 21 survey items we retain for Phase 3 bolded.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">FIBIS category</th><th align="left">Survey items</th><th align="left">Factor</th></tr><tr><th align="left">1</th><th align="left">2</th><th align="left">3</th><th align="left">4</th></tr></thead><tbody valign="top"><tr><td align="left">Students as barriers</td></tr><tr><td align="left" /><td align="left">My students do not know how to engage in active learning.</td><td align="char" char=".">0.69</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">My students are often absent from class.</td><td align="char" char=".">0.53</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">My students do not come prepared for in‐class activities.</td><td align="char" char=".">0.77</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">My students have too wide of a disparity in preparation for my courses, making it difficult to use active learning.<xref ref-type="fn" rid="tfn5" /></td><td align="char" char=".">0.61</td><td align="char" char=".">0.28</td><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">My students play the system to circumvent learning objectives.</td><td align="char" char=".">0.71</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">My students are resistant to engage in active learning.</td><td align="char" char=".">0.82</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">My students are less likely to achieve learning outcomes using active learning.<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">0.55</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">I feel uncomfortable interacting with my students when it is not required.<xref ref-type="fn" rid="tfn3" /></td><td align="left" /><td align="left" /><td align="char" char=".">0.49</td><td align="left" /></tr><tr><td align="left" /><td align="left">My students will share negative feedback related to active learning in my class.<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">0.43</td><td align="char" char=".">−0.27</td><td align="char" char=".">0.26</td><td align="left" /></tr><tr><td align="left" /><td align="left">My students do not expect to engage in active learning during lecture.<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">0.57</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">Departmental barriers</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">Instructional changes are too difficult to implement as an individual instructor.</td><td align="left" /><td align="char" char=".">0.28</td><td align="char" char=".">0.44</td><td align="left" /></tr><tr><td align="left" /><td align="left">My colleagues are generally encouraging about using active learning.</td><td align="char" char=".">−0.27</td><td align="left" /><td align="left" /><td align="char" char=".">−0.51</td></tr><tr><td align="left" /><td align="left">I am uncomfortable teaching differently than the norm of lecturing.</td><td align="left" /><td align="left" /><td align="char" char=".">0.73</td><td align="left" /></tr><tr><td align="left" /><td align="left">It is expected that I focus on research, so I do not feel comfortable trying active learning in my classes.</td><td align="left" /><td align="left" /><td align="char" char=".">0.45</td><td align="char" char=".">0.48</td></tr><tr><td align="left" /><td align="left">The teaching policies at my institution do not support active learning.<xref ref-type="fn" rid="tfn4" /></td><td align="left" /><td align="left" /><td align="left" /><td align="char" char=".">0.64</td></tr><tr><td align="left">Faculty beliefs</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">I do not have time to restructure my class to include active learning.</td><td align="left" /><td align="char" char=".">0.45</td><td align="char" char=".">0.52</td><td align="left" /></tr><tr><td align="left" /><td align="left">I do not have time to learn about active learning.<xref ref-type="fn" rid="tfn3" /></td><td align="left" /><td align="char" char=".">0.34</td><td align="char" char=".">0.54</td><td align="left" /></tr><tr><td align="left" /><td align="left">Implementing active learning may lower my teaching evaluations.</td><td align="char" char=".">0.38</td><td align="left" /><td align="char" char=".">0.31</td><td align="char" char=".">0.36</td></tr><tr><td align="left" /><td align="left">I have less control over my class when I implement active learning.</td><td align="left" /><td align="left" /><td align="char" char=".">0.60</td><td align="left" /></tr><tr><td align="left" /><td align="left">I am not confident that I have adequate skills to use active learning in my classes.</td><td align="left" /><td align="left" /><td align="char" char=".">0.64</td><td align="left" /></tr><tr><td align="left">I get anxious when trying active learning in class.</td><td align="left" /><td align="left" /><td align="char" char=".">0.72</td><td align="left" /></tr><tr><td align="left" /><td align="left">Too much class preparation time is required to implement active learning.<xref ref-type="fn" rid="tfn5" /></td><td align="left" /><td align="char" char=".">0.46</td><td align="char" char=".">0.37</td><td align="left" /></tr><tr><td align="left" /><td align="left">Using active learning will prevent me from covering all necessary content.</td><td align="left" /><td align="char" char=".">0.36</td><td align="char" char=".">0.42</td><td align="left" /></tr><tr><td align="left" /><td align="left">My subject matter is not conducive to using active learning.</td><td align="left" /><td align="left" /><td align="char" char=".">0.43</td><td align="left" /></tr><tr><td align="left" /><td align="left">Assessment of student learning in an active learning classroom is difficult.</td><td align="left" /><td align="char" char=".">0.46</td><td align="char" char=".">0.29</td><td align="left" /></tr><tr><td align="left" /><td align="left">I have not seen strong enough evidence of active learning's efficacy to consider using it.</td><td align="char" char=".">0.34</td><td align="left" /><td align="char" char=".">0.41</td><td align="left" /></tr><tr><td align="left" /><td align="left">Students respect me when I lecture but challenge me when I use active learning.</td><td align="char" char=".">0.31</td><td align="left" /><td align="char" char=".">0.33</td><td align="left" /></tr><tr><td align="left" /><td align="left">I personally prioritize time for research over innovating my teaching.</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">It is difficult to balance delivering lecture content and using active learning.</td><td align="left" /><td align="char" char=".">0.37</td><td align="char" char=".">0.45</td><td align="left" /></tr><tr><td align="left" /><td align="left">There is not enough easily accessible material to use active learning in my course.</td><td align="left" /><td align="char" char=".">0.66</td><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">I am more comfortable in the role of a lecturer, rather than a facilitator.<xref ref-type="fn" rid="tfn4" /></td><td align="left" /><td align="left" /><td align="char" char=".">0.58</td><td align="left" /></tr><tr><td align="left">Limited support for instruction</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">There are insufficient funds available to support implementing active learning (e.g., grant opportunities, teaching release).</td><td align="left" /><td align="char" char=".">0.57</td><td align="left" /><td align="char" char=".">0.35</td></tr><tr><td align="left" /><td align="left">There is insufficient technological support to ensure active learning works in my classroom.</td><td align="left" /><td align="char" char=".">0.76</td><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">My classes do not have the teaching assistant support necessary for active learning.</td><td align="left" /><td align="char" char=".">0.57</td><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">My colleagues are not supportive of changing to a curriculum that uses active learning.</td><td align="char" char=".">0.27</td><td align="left" /><td align="left" /><td align="char" char=".">0.66</td></tr><tr><td align="left" /><td align="left">Educational support staff do not provide enough personalized support for me to make changes in my classes.</td><td align="left" /><td align="char" char=".">0.62</td><td align="left" /><td align="left" /></tr><tr><td align="left">Prior experiences</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">I had a bad experience with active learning as a student and do not want to try it.</td><td align="left" /><td align="left" /><td align="char" char=".">0.54</td><td align="left" /></tr><tr><td align="left" /><td align="left">I had a bad experience implementing active learning in the past.</td><td align="left" /><td align="left" /><td align="char" char=".">0.63</td><td align="left" /></tr><tr><td align="left">Technical/classroom issues</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">The technology needed to implement active learning in my course is too complex (e.g., clickers, screencasts).</td><td align="left" /><td align="char" char=".">0.47</td><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">Because I do not know the layout of my classroom far enough in advance, I cannot prepare my course accordingly.</td><td align="left" /><td align="char" char=".">0.44</td><td align="left" /><td align="left" /></tr><tr><td align="left">Other</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">There is no incentive to innovate in my teaching.</td><td align="left" /><td align="left" /><td align="left" /><td align="char" char=".">0.64</td></tr><tr><td align="left">Standalone statements</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">Instructor reward structures (e.g., tenure and promotion) favor effort spent on research over teaching.</td><td align="left" /><td align="left" /><td align="left" /><td align="char" char=".">0.56</td></tr><tr><td align="left" /><td align="left">The classrooms available for my course are inappropriately designed to implement active learning (e.g., fixed chairs, lecture hall).</td><td align="left" /><td align="char" char=".">0.52</td><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">My class is too large to use active learning.</td><td align="left">0.32</td><td align="left">0.33</td><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left">My classes are too short time‐wise to use active learning.</td><td align="left">0.26</td><td align="left">0.39</td><td align="left" /><td align="left" /></tr><tr><td align="left">Variance</td><td align="left" /><td align="char" char=".">8.25</td><td align="char" char=".">7.97</td><td align="char" char=".">9.25</td><td align="char" char=".">4.11</td></tr><tr><td align="left">Proportion of variance</td><td align="left" /><td align="char" char=".">0.30</td><td align="char" char=".">0.29</td><td align="char" char=".">0.33</td><td align="char" char=".">0.15</td></tr></tbody></table> </ephtml> </p> <ulist> <item>2 <emph>Note</emph>: Bolded factor loadings met our 0.5/0.3 loading criteria.</item> <item>3 a Survey item added based on DeMonbrun et al. ([<reflink idref="bib18" id="ref133">18</reflink>]).</item> <item>4 b Survey item added based on Finelli et al. ([<reflink idref="bib24" id="ref134">24</reflink>]).</item> <item>5 c Survey item modified after Phase 2.</item> </ulist> <p>We did not change the anchors from the FIBIS; thus, the items in our draft instrument are rated on a five‐point Likert scale (i.e., strongly disagree, somewhat disagree, neutral, somewhat agree, strongly agree). These anchors provide relatively equal distances between response categories (Dillman et al., [<reflink idref="bib19" id="ref135">19</reflink>]). We provided the anchors to participants by labeling each response category, thereby decreasing the susceptibility of our instrument to context effects and participant interpretation (Krosnick & Fabrigar, [<reflink idref="bib44" id="ref136">44</reflink>]).</p> <hd id="AN0173054174-16">PHASE 2: FIRST EXPLORATORY FACTOR ANALYSIS</hd> <p>In Phase 2 of our study, we conducted an EFA using the described 45‐item draft instrument (Table 3) to explore the underlying factor structure and revise our instrument.</p> <hd id="AN0173054174-17">Data collection</hd> <p>Our sampling frame consisted of a contact list of engineering instructors at all engineering programs across the United States and Canada collected from university web pages (<emph>N</emph> = 11,129). We collected data using Qualtrics software ([<reflink idref="bib68" id="ref137">68</reflink>]) between April and May, inviting a random sample of half our sampling frame. No incentive was offered for completing the instrument. After the initial invitation, we sent three reminders over a 16‐day timeframe, receiving 429 total responses. Of the 429 responses, 45 were marked by Qualtrics as incomplete (36 were completely blank, and nine were less than half complete). Sixteen of the remaining 384 responses had a least one missing response by observation (and are not included in our subsequent analyses)—this 4.2% listwise missingness has no discernable pattern and is small enough to be considered inconsequential (Bennett, [<reflink idref="bib6" id="ref138">6</reflink>]; Schafer, [<reflink idref="bib76" id="ref139">76</reflink>]). Overall, we had a completion rate of 6.7% for the first administration of our instrument (<emph>n</emph> = 368). We expected at least four factors and had a 45‐item scale; therefore, our sample (<emph>n</emph> = 368) is slightly smaller than the recommended 400 participants for conducting an EFA (Rouquette & Falissard, [<reflink idref="bib73" id="ref140">73</reflink>]). The 400‐participant recommendation is based on simulation data for an EFA with four‐point Likert responses, a 45‐item scale, and four factors.</p> <hd id="AN0173054174-18">Analysis</hd> <p>Using STATA 16 (StataCorp, [<reflink idref="bib81" id="ref141">81</reflink>]) and 17 (StataCorp, [<reflink idref="bib82" id="ref142">82</reflink>]), we conducted an EFA using principal axis factoring on a polychoric correlation matrix (Kolenikov, [<reflink idref="bib43" id="ref143">43</reflink>]) because of the five‐point ordinal nature of the item anchors (Baglin, [<reflink idref="bib4" id="ref144">4</reflink>]). Items were screened to check for violations of univariate and multivariate normality; severe violations can impact the estimation procedures (Curran et al., [<reflink idref="bib15" id="ref145">15</reflink>]). For univariate normality, our cutoffs for removal were 2.0 or higher for skewness and 7.0 or higher for kurtosis (Curran et al., [<reflink idref="bib15" id="ref146">15</reflink>]; Muthen & Kaplan, [<reflink idref="bib57" id="ref147">57</reflink>]). We assessed Mardia's kurtosis for multivariate normality, expecting a value of <emph>v</emph>(<emph>v</emph> + 2) or 2115 for 45 variables (<emph>v</emph>; Finney & DiStefano, [<reflink idref="bib25" id="ref148">25</reflink>]). To explore whether our data were suitable for EFA, we calculated the Kaiser–Meyer–Olkin (KMO) test for sampling adequacy, expecting a value of at least 0.8 in the "meritorious" range of the index (Kaiser & Rice, [<reflink idref="bib41" id="ref149">41</reflink>]), and we conducted Bartlett's test of sphericity, considering the data factorable when <emph>p</emph> < .05 (Bartlett, [<reflink idref="bib5" id="ref150">5</reflink>]).</p> <p>Then, to decide how many factors to retain, we considered the scree plot, parallel analysis (PA; Horn, [<reflink idref="bib36" id="ref151">36</reflink>]; using Stata user written command: <emph>fapara</emph>; Ender, [<reflink idref="bib22" id="ref152">22</reflink>]), and minimum average partials (MAP; Velicer, [<reflink idref="bib88" id="ref153">88</reflink>]; using Stata user written command: <emph>minap</emph>; Soldz, [<reflink idref="bib78" id="ref154">78</reflink>]). PA involves simulating datasets with no factors, averaging the eigenvalues, and comparing those averages with observed eigenvalues. Observed eigenvalues in the 95th percentile above the random averaged eigenvalues are retained (McCoach et al., [<reflink idref="bib54" id="ref155">54</reflink>]). MAP is another useful method to determine the number of factors to extract (Gorsuch, [<reflink idref="bib28" id="ref156">28</reflink>]). MAP "separates common and unique variance and retains only components consisting of common variance" (Watkins, [<reflink idref="bib91" id="ref157">91</reflink>], p. 80). We use all three methods (i.e., the scree plot, PA, and MAP), as Velicer et al. ([<reflink idref="bib89" id="ref158">89</reflink>]) recommend.</p> <p>We normalized the factor loadings for the EFA before applying an oblimin rotation with γ = 0 (i.e., direct quartimin) using a Kaiser normalization—denormalizing after rotation. To balance the orthogonality of the extracted components with the conceptual overlap between theorized factors, we set the γ‐value controlling obliquity to zero. We chose an oblique rotation because it is a more reasonable representation of social science phenomena than strict orthogonality (Fabrigar et al., [<reflink idref="bib23" id="ref159">23</reflink>]).</p> <p>To screen the items for sufficient loadings, we applied a 0.5/0.3‐factor loading spread (Matsunaga, [<reflink idref="bib53" id="ref160">53</reflink>]), requiring that (<reflink idref="bib1" id="ref161">1</reflink>) retained items have at least a 0.50 factor loading and (<reflink idref="bib2" id="ref162">2</reflink>) the second highest loading factor does not exceed 0.30. We considered the lack of consensus regarding appropriate factor loadings and addressing cross‐loadings to justify our decision. The most popular cutoff is 0.4 (Hinkin, [<reflink idref="bib33" id="ref163">33</reflink>], [<reflink idref="bib34" id="ref164">34</reflink>]), with other values like 0.30, 0.32, and 0.45 having been proposed (Howard, [<reflink idref="bib37" id="ref165">37</reflink>]). Costello and Osborne ([<reflink idref="bib13" id="ref166">13</reflink>]) recommended that the alternative factor loadings should not exceed 0.32 or 0.40. Hinkin ([<reflink idref="bib34" id="ref167">34</reflink>]) also suggested that the difference between loadings should be considered, for example, a difference no smaller than 0.20 with a.40 cutoff. Howard ([<reflink idref="bib37" id="ref168">37</reflink>]) proposed a combination rule of 0.4/0.3/0.2, a primary loading of 0.40, no other factor loading greater than 0.30, and a difference of at least 0.20 between alternative loadings. Our rule is more stringent, demanding a higher primary loading to trim weaker items because we are further developing an existing instrument, the FIBIS.</p> <p>After comparing our factor solution with the five‐factor solution suggested by the FIBIS, we then considered the alignment with the seven categories of barriers suggested by Finelli et al. ([<reflink idref="bib24" id="ref169">24</reflink>]; infrastructure and culture; knowledge and skills of effective teaching practices; student experience; time; classroom and curriculum; personal disposition; and networking and community) and the four categories of influences in the instructional innovation‐decision process model (Lund & Stains, [<reflink idref="bib49" id="ref170">49</reflink>]; characteristics of the learning environment, departmental factors, attitudes and beliefs about teaching, and pedagogical experiences). We then removed items that did not meet the 0.5/0.3‐factor loading criterion and reran the EFA to examine the stability of the factor structure without the deleted items. We considered the four to five items having the greatest loading magnitudes to identify tentative construct names and to decide upon items for the second administration of our instrument (Nunnally & Bernstein, [<reflink idref="bib63" id="ref171">63</reflink>]).</p> <hd id="AN0173054174-19">Results</hd> <p>We rejected the hypothesis of multivariate normality (<emph>p</emph> < .001), thereby supporting the appropriateness of our decision to analyze the factor structure of our initial 45 items using the principal axis factoring estimation method on the polychoric correlation matrix (Baglin, [<reflink idref="bib4" id="ref172">4</reflink>]). The matrix was suitable for EFA, as evidenced by the Kaiser–Meyer–Olkin measure of.87. Furthermore, we rejected the null hypothesis of an identity matrix using Bartlett's test of sphericity (<emph>χ</emph><sups>2</sups>(<reflink idref="bib990" id="ref173">990</reflink>) = 6968.44, <emph>p</emph> < 0.05). We observed a clear indication of four factors using the scree plot and MAPs for our initial extraction. However, PA suggested that a 10‐factor structure best fit our data. We explored both a 4‐ and a 10‐factor solution. For the 10‐factor solution, only one or two items loaded onto each of the additional six factors. Therefore, we continued with the four‐factor solution.</p> <p>In our first EFA, the survey items loaded onto four extracted factors, as shown in Table 3. After estimating the initial factor structure, we found that 25 items, with factor loadings presented in boldface font, met the 0.5/0.3‐factor loading criterion. Consistent with our goal of a concise instrument, we removed the 20 items that did not meet the criterion. Then, we reran the EFA to ensure that eliminating those items did not change our factor structure. The resulting extraction (loadings not presented) still suggested a four‐factor structure, and the remaining 25 items factored in the same manner as with all 45 items. We retained a reworded version of one item that failed to meet our 0.5/0.3 criterion ("Too much class preparation time is required to implement active learning") because time has been consistently identified as a prominent barrier to instructor adoption of active learning.</p> <p>Next, we dropped four items that met our loading criterion but were similar in some way to another survey item. For example, "My students are less likely to achieve learning outcomes" is similar to "My students play the system to circumvent learning objectives." "My students are often absent from class" is similar to "My students do not come prepared for in‐class activities." Then, we dropped a fifth item that no longer met the 0.50 threshold for loading on a single factor when we reran our EFA, "The classrooms available for my course are inappropriately designed to implement active learning (e.g., fixed chairs, lecture hall)." Finally, we revised the language of two survey items indicated by a footnote in Table 3. The resulting 21 survey items that we used for our refined instrument is bolded in Table 3.</p> <p>Factor 1, which we subsequently label "student preparation and engagement," has a student focus, similar to the FIBIS students as barriers. It comprises six items about instructors' perceptions of students' expectations, experiences, behavior, and learning. We label Factor 2 as "instructional support." It includes five items that originate from two different FIBIS categories: faculty beliefs and limited support for instruction. Factor 3, which we name "instructor comfort and confidence," comprises five items originating from multiple FIBIS categories: departmental barriers, faculty beliefs, and prior experiences. Lastly, the five items extracted from Factor 4 originate from multiple FIBIS categories: departmental barriers, limited support for instruction, other, and standalone statements. We label the fourth factor as "institutional environment/rewards."</p> <hd id="AN0173054174-20">PHASE 3: SECOND EXPLORATORY FACTOR ANALYSIS</hd> <p>In Phase 3, we administered our refined 21‐item instrument and conducted a second EFA. This allowed us to explore the factor structure further and refine our instrument.</p> <hd id="AN0173054174-21">Data collection</hd> <p>We used the same sampling frame as for our Phase 2 EFA, consisting of engineering instructors at all engineering programs across the United States and Canada (<emph>N</emph> = 11,129). In this second survey administration, we invited the remaining half of our sampling frame to complete our 21‐item instrument, again not offering any incentive. We collected data using Qualtrics software ([<reflink idref="bib68" id="ref174">68</reflink>]) between June and July, sending three reminders over a 17‐day timeframe, and received 458 responses. Of the 458 responses, 37 were incomplete (28 were completely blank, and nine were less than half complete). Eleven of the remaining 421 responses (2.6%) had at least one missing value by observation, again, a small enough value with no discernable pattern of missingness to be considered inconsequential (Bennett, [<reflink idref="bib6" id="ref175">6</reflink>]; Schafer, [<reflink idref="bib76" id="ref176">76</reflink>]). We had a completion rate of 7.4%.</p> <p>We formed a split sample with this second dataset (<emph>n</emph> = 410) for use in Phase 3 (second EFA) and Phase 4 (first CFA) of our instrument development. The first half of these data (<emph>n</emph> = 205) enabled us to refine our instrument and verify our factor structure with a second EFA with substantially fewer items than the 45 items administered for our Phase 2 EFA prior to conducting our Phase 4 CFA using the second half of these data (<emph>n</emph> = 205). The number of participants in our second EFA (<emph>n</emph> = 205) is slightly less than the recommended 240 participants for normal data with four variables per factor (Watkins, [<reflink idref="bib91" id="ref177">91</reflink>], p. 42), yet it exceeds the recommended 110 participants when allowing five variables per factor (Watkins, [<reflink idref="bib91" id="ref178">91</reflink>], p. 42).</p> <p>We again screened items to check for violations of univariate and multivariate normality and assessed Mardia's kurtosis for multivariate normality, consistent with our procedure in Phase 2. Most items' skewness and kurtosis values indicated near‐normality; however, several were just outside of the −1.0 to 1.0 range: the items' skew and zero‐normed kurtosis ranged from −1.20 to 1.03 and −1.15 to 0.75, respectively. For Mardia's kurtosis, the expected value is 483 for 21 variables (Finney & DiStefano, [<reflink idref="bib25" id="ref179">25</reflink>]); we found Mardia's kurtosis (514.66, <emph>p</emph> < .0001) is above this expected value for multivariate normality. Because these univariate and multivariate normality measures slightly exceed the threshold for normality, we conducted the Phase 3 EFA using both Pearson's and polychoric correlations.</p> <p>For Phase 3, we expanded the number of response categories from five to seven because a seven‐point scale enables more analytical options than a five‐point scale. We determined that seven levels would provide sufficient granularity in responses to construct the scale (Cummins & Gullone, [<reflink idref="bib14" id="ref180">14</reflink>]; Nunnally, [<reflink idref="bib62" id="ref181">62</reflink>]). We based this decision on the empirical work of Finstad ([<reflink idref="bib26" id="ref182">26</reflink>]), who found that seven‐point scales appear to better measure a person's evaluation based on a prompt than five‐point scales. Our new anchors were similar to the five‐point scale from before but with the two additional anchors of disagree and agree. Thus, our seven‐point scale comprises the following anchors: strongly disagree, disagree, somewhat disagree, neutral, somewhat agree, agree, and strongly agree.</p> <p>We also added a demographics section to our instrument based on feedback we received via email from participants after our first administration. This section again drew from the FIBIS (Sturtevant & Wheeler, [<reflink idref="bib83" id="ref183">83</reflink>]), containing items related to the respondent's location in the United States or Canada, institution type, faculty rank, tenure versus nontenure‐track status, whether they were a professor of practice or clinical professor, years of teaching experience, number of years they have used active learning, gender identity, and race/ethnicity. We present demographic data for our Phase 3 sample by race/ethnicity and gender in Appendix A in Table A1, faculty rank and institution type in Table A2, and years of experience teaching and using active learning in Figure A1.</p> <p>Although we do not have the sample size to test for invariance across subgroups, our goal in adding demographic items is to demonstrate representation. For our Phase 3 EFA, our sample is mainly represented by White men (93 respondents, 45.3%) and women (37 respondents, 18.0%) and from participants at public MS/PhD granting institutions (158 respondents, 77.1%), followed by private MS/PhD granting institutions (33 respondents, 16.1%). One hundred seven respondents (52.2%) were from Canada, and 98 (47.8%) were from the United States. Regarding teaching experience, our EFA sample exhibits a decreasing number of instructors as the number of years of experience increases. Seventeen (8.3%) of the respondents had not implemented active learning in their classroom, whereas 188 (91.7%) had tried active learning at least once, with a mean of 8.21 (<emph>SD</emph> = 7.52) years of experience.</p> <hd id="AN0173054174-22">Analysis</hd> <p>For Phase 3, we conducted an EFA on our refined, 21‐item instrument, enabling us to explore the stability of our initial factor structure. We repeated the procedure described for our first EFA, but we used the Pearson correlation matrix instead of the polychoric correlation matrix (i.e., treating the data as interval instead of ordinal). Norman ([<reflink idref="bib61" id="ref184">61</reflink>]) provides a detailed argument for this analytical decision, part of which is founded on the robustness of the Pearson correlation coefficient to non‐normality and skewness, citing Pearson ([<reflink idref="bib64" id="ref185">64</reflink>], [<reflink idref="bib65" id="ref186">65</reflink>], [<reflink idref="bib66" id="ref187">66</reflink>]) himself, Dunlap ([<reflink idref="bib20" id="ref188">20</reflink>]), and Havlicek and Peterson ([<reflink idref="bib29" id="ref189">29</reflink>]). For comparison purposes, we also conducted an EFA using a polychoric correlation matrix and found the factor solution unaffected; the factor loadings differed minimally.</p> <hd id="AN0173054174-23">Results</hd> <p>We assessed the factor structure of our refined set of items (21 bolded items in Table 3) using the principal axis factoring estimation method on the Pearson correlation matrix (Baglin, [<reflink idref="bib4" id="ref190">4</reflink>]). The correlation matrix was factorable; KMO = .84. The scree plot and PA indicated that the four‐factor structure best fit our data. We present items and factor loadings in Table 4 for the items that met our 0.5/0.3 loading criterion. Six items loaded on Factor 1, student preparation and engagement; four on Factor 2, instructional support; four on Factor 3, instructor comfort and confidence; and four on Factor 4, institutional environment/rewards.</p> <p>4 TABLE Phase 3 EFA results for the 18 items from our 21‐item refined instrument with factor loadings that met our 0.5/0.3 loading criteria. We retain 17 items for our final survey.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" /><th align="left">Factor loadings</th></tr><tr><th align="left">Items</th><th align="left">1</th><th align="left">2</th><th align="left">3</th><th align="left">4</th></tr></thead><tbody valign="top"><tr><td align="left">Factor 1: student preparation and engagement</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">My students do not come prepared for in‐class activities.</td><td align="char" char=".">0.80</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">My students do not know how to engage in active learning.</td><td align="char" char=".">0.76</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">My students play the system to circumvent learning objectives.</td><td align="char" char=".">0.74</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">My students are resistant to engage in active learning.</td><td align="char" char=".">0.67</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">My students do not expect to engage in active learning during lecture.</td><td align="char" char=".">0.62</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">My students have too wide of a disparity in preparation for my courses.<xref ref-type="fn" rid="tfn6" /></td><td align="char" char=".">0.60</td><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">Factor 2: instructional support</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">There is not enough easily accessible material to use active learning in my course.</td><td align="left" /><td align="char" char=".">0.73</td><td align="left" /><td align="left" /></tr><tr><td align="left">Educational support staff do not provide enough personalized support for me to make changes in my classes.</td><td align="left" /><td align="char" char=".">0.69</td><td align="left" /><td align="left" /></tr><tr><td align="left">There is insufficient technological support to ensure active learning works in my classroom.</td><td align="left" /><td align="char" char=".">0.57</td><td align="left" /><td align="left" /></tr><tr><td align="left">My classes do not have the teaching assistant support necessary for active learning.</td><td align="left" /><td align="char" char=".">0.52</td><td align="left" /><td align="left" /></tr><tr><td align="left">Factor 3: instructor comfort and confidence</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">I get anxious when trying active learning in class.</td><td align="left" /><td align="left" /><td align="char" char=".">0.76</td><td align="left" /></tr><tr><td align="left">I am more comfortable in the role of a lecturer, rather than a facilitator.</td><td align="left" /><td align="left" /><td align="char" char=".">0.63</td><td align="left" /></tr><tr><td align="left">I am not confident that I have adequate skills to use active learning in my classes.</td><td align="left" /><td align="left" /><td align="char" char=".">0.62</td><td align="left" /></tr><tr><td align="left">I am uncomfortable teaching differently than the norm of lecturing.</td><td align="left" /><td align="left" /><td align="char" char=".">0.54</td><td align="left" /></tr><tr><td align="left">Factor 4: institutional environment/rewards</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left">My colleagues are not supportive of changing to a curriculum that uses active learning.</td><td align="left" /><td align="left" /><td align="left" /><td align="char" char=".">0.80</td></tr><tr><td align="left">The teaching policies at my institution do not support active learning.</td><td align="left" /><td align="left" /><td align="left" /><td align="char" char=".">0.70</td></tr><tr><td align="left">My colleagues are generally encouraging about using active learning.</td><td align="left" /><td align="left" /><td align="left" /><td align="char" char=".">−0.59</td></tr><tr><td align="left">There is no incentive to innovate in my teaching.</td><td align="left" /><td align="left" /><td align="left" /><td align="char" char=".">0.53</td></tr><tr><td align="left">Variance</td><td align="char" char=".">4.30</td><td align="char" char=".">3.02</td><td align="char" char=".">2.54</td><td align="char" char=".">2.93</td></tr><tr><td align="left">Proportion of variance</td><td align="char" char=".">0.47</td><td align="char" char=".">0.33</td><td align="char" char=".">0.27</td><td align="char" char=".">0.32</td></tr><tr><td align="left">Alpha reliability</td><td align="char" char=".">0.87</td><td align="char" char=".">0.78</td><td align="char" char=".">0.74</td><td align="char" char=".">0.76</td></tr></tbody></table> </ephtml> </p> <p>6 a Survey item was dropped from our instrument because the wording lacked clarity.</p> <p>Four items we included in our 21‐item instrument for Phase 3 were not retained: three did not meet our 0.5/0.3 loading criterion following our second EFA (these three are not included in Table 4), and one was unclearly worded (this item is included in Table 4). First, the item, "Instructor reward structures (e.g., tenure and promotion) favor effort spent on research over teaching," loaded on Factor 4, <emph>institutional environment/rewards</emph>, but at a level (0.28) well below the 0.50 threshold. The second item, "I had a bad experience implementing active learning in the past," cross‐loaded on Factor 1 (0.25), student preparation and engagement, and Factor 3 (0.31), instructor comfort and confidence. Third, the modified survey item that we chose to retain for Phase 3, even though with its original wording it did not meet our inclusion criterion, "Too much preparation time is required to implement active learning," cross‐loaded on Factor 2 (0.36), instructional support, and Factor 3 (0.43), instructor comfort and confidence. And fourth, we determined that the wording of one item, "My students have too wide of a disparity in preparation for my courses," lacked clarity in the meaning of "preparation." "Preparation" may refer to students' preparedness for the course, an individual class session, or necessary study/organization skills. Therefore, we dropped these four items from our instrument. Following Phase 3 of our instrument development, our revised instrument comprised 17 items.</p> <hd id="AN0173054174-24">PHASE 4: CONFIRMATORY FACTOR ANALYSIS</hd> <p>In Phase 4, we conducted a CFA on the 17 items of our final instrument to evaluate the factor structure derived in Phase 3 using the laavan package in R (R Core Team, [<reflink idref="bib69" id="ref191">69</reflink>]; Rosseel, [<reflink idref="bib72" id="ref192">72</reflink>]).</p> <hd id="AN0173054174-25">Data collection</hd> <p>We retained the second half of the dataset (<emph>n</emph> = 205) from our second survey administration to perform this Phase 4 CFA. Determining a clear rule for an appropriate CFA sample size is difficult (Kline, [<reflink idref="bib42" id="ref193">42</reflink>]). There was apparent support for considering the ratio of estimated parameters to sample size (Jackson, [<reflink idref="bib39" id="ref194">39</reflink>]), with Kline ([<reflink idref="bib42" id="ref195">42</reflink>]) suggesting a 10:1 ratio as a minimum. The ratio of sample size to indicators in this CFA is approximately 12 (205:17). More contemporary guidance on sample size is derived from Monte Carlo simulations, whereas arbitrary cutoffs and ratios have fallen out of favor (Kyriazos, [<reflink idref="bib45" id="ref196">45</reflink>]). As is the case for this study, simpler models with continuous, normally distributed variables and linear effects require smaller sample sizes (Kline, [<reflink idref="bib42" id="ref197">42</reflink>]). A power analysis using a conventional SEM calculator dictates a minimum sample size of 137 to detect a medium effect with a power of 0.8. Moreover, using the semTools package in R (Jorgensen et al., [<reflink idref="bib40" id="ref198">40</reflink>]), we found that the minimum sample size to achieve the same power of 0.8 and <emph>df</emph> = 113 to reject a model with misspecifications (where RMSEA ≥.05) was 153. Thus, our sample size appears to be adequate.</p> <p>We present demographic data for our Phase 4 sample by gender and race/ethnicity in Appendix A in Table A3, faculty rank and institution type in Table A4, and years of experience with active learning in Figure A2. Like the sample from our Phase 3 EFA, our CFA sample is mostly represented by White men (<reflink idref="bib101" id="ref199">101</reflink>, 49.3%) and women (<reflink idref="bib32" id="ref200">32</reflink>, 15.6%) and by participants at public MS/PhD granting institutions (<reflink idref="bib156" id="ref201">156</reflink>, 76.1%), followed by private MS/PhD granting institutions (<reflink idref="bib32" id="ref202">32</reflink>, 15.6%). Eighty‐nine (43.4%) respondents were from Canada, and 116 (56.6%) were from the United States. Regarding teaching experience, our CFA sample exhibits a starker decreasing count of instructors as the number of years increases than our Phase 3 EFA histogram. There was a much higher concentration of instructors with 0–10 years of experience. Twenty (9.8%) of the respondents had not implemented active learning in their classroom, whereas 183 (89.3%) had tried active learning at least once with a mean of 7.36 (<emph>SD</emph> = 7.13) years of experience.</p> <hd id="AN0173054174-26">Analysis</hd> <p>To ensure our model was identified, we imposed the reference variable identification constraint (Kline, [<reflink idref="bib42" id="ref203">42</reflink>]). The covariance among factors was allowed to be estimated because we chose an oblique rotation during our EFA procedures. We chose the conventional maximum likelihood method (ML) for our estimator. We again screened items beforehand using the skewness and kurtosis cutoffs to assess violations of univariate normality. No items exceeded the larger cutoffs for univariate normality, and most exhibited near‐normality: the skew ranged from −0.506 to 1.10, with only one item exceeding the 1.0 threshold, and the zero‐normed kurtosis ranged from −1.24 to 0.49. Mardia's kurtosis (419.00, <emph>p</emph> < .0001) slightly exceed the expected 399 for multivariate normality.</p> <p>We assessed our model fit using Byrne's suggestions ([<reflink idref="bib10" id="ref204">10</reflink>]) for indices: model chi‐square (<emph>p ></emph> .05), Comparative Fit Index (CFI > .90,.95 preferred; Hu & Bentler, [<reflink idref="bib38" id="ref205">38</reflink>]), Tucker Lewis Index (TLI > .90 at minimum,.95 preferred; Hu & Bentler, [<reflink idref="bib38" id="ref206">38</reflink>]), root mean square error of approximation (RMSEA <.01 is excellent, <.05 is good, and.08 is moderate; MacCallum et al., [<reflink idref="bib50" id="ref207">50</reflink>]), and standardized root mean square residual (SRMR <.10 preferred; Kline, [<reflink idref="bib42" id="ref208">42</reflink>]). Finally, we assessed internal consistency within factors with Cronbach's alpha and set a benchmark of at least.70 for each factor—as this cutoff is generally considered acceptable for new scales (Thorndike et al., [<reflink idref="bib85" id="ref209">85</reflink>]).</p> <p>After examining the model, we used the process advocated by Saris et al. ([<reflink idref="bib74" id="ref210">74</reflink>]) to investigate model misspecifications, particularly correlated errors, which involved the combination of power, modification index, and expected parameter change (EPC). Saris et al. ([<reflink idref="bib74" id="ref211">74</reflink>]) recommended an absolute EPC of 0.1 as greater than acceptable; moreover, modification indices greater than 3.84 are statistically significant. If the power is greater than 75%, the modification index is significant, and EPC is greater than acceptable, then there is a potential misspecification. If the EPC or modification index is not significant, then there is no misspecification. Likewise, if the power is less than 75% and the modification index is significant, a potential misspecification could be present; otherwise, the result is inconclusive. Each potential misspecification was reviewed to ensure that estimating the new path was theoretically reasonable.</p> <hd id="AN0173054174-27">Results</hd> <p>Using the four‐factor model from our previous EFA, we found our model exhibited internal consistency and an adequate model fit for our 17‐item instrument. Cronbach's alpha for each of the four factors was above the acceptable cutoff of.70 (<emph>α</emph> = .76 to.84), providing evidence of internal consistency. Additionally, the model fit parameters indicate an adequate fit, CFI = .922, TLI = .906, RMSEA = .068, 90% CI [.054,.081], <emph>p</emph><subs>close</subs> = .017, and SRMR = .067. In contrast, the chi‐square test, <emph>χ</emph><sups><emph>2</emph></sups>(<reflink idref="bib113" id="ref212">113</reflink>) = 218.91, indicates a significant result (<emph>p</emph> < .0001) and a lack of model fit (Kline, [<reflink idref="bib42" id="ref213">42</reflink>]).</p> <p>Our exploration of modification indices revealed one potential suggested correlated error that could be justified based on Saris et al.'s ([<reflink idref="bib74" id="ref214">74</reflink>]) recommendation. The second and third items from "institutional environment/rewards" had a significant modification index (MI = 15.09, <emph>p</emph> < .01), a power of.135, and an EPC of −0.46. The item pair from this factor concerned instructor support in the department, which addresses a specific component of the institutional environment—unlike the other two distinct items. Therefore, we allowed the errors to correlate. Running the new model, we found the modifications moderately improved our global fit, <emph>χ</emph><sups><emph>2</emph></sups>(<reflink idref="bib112" id="ref215">112</reflink>) = 205.26 (<emph>p</emph> < .0001), CFI = .931, TLI = .916, RMSEA = .064, 90% CI [.050,.077], <emph>p</emph><subs>close</subs> = .052, and SRMR = .064. We present the tabular representation of this model with standardized parameter estimates and reliability parameters in Table 5. For the second and third items of Factor 4, the unstandardized covariance was −0.42 with a standard error of 0.12 (<emph>z</emph> = −3.37, <emph>p</emph> = 0.001), and the standardized covariance was −0.32 with a standard error of 0.08 (<emph>z</emph> = −4.26, <emph>p</emph> < 0.001).</p> <p>5 TABLE Model statistics for our Phase 4 CFA.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" /><th align="left">Unstandardized factor loadings</th><th align="left">Standardized factor loadings</th><th align="left">Standard error</th><th align="left">Item reliability (<italic>R</italic><sup>2</sup>)</th><th align="left">Construct reliability</th></tr></thead><tbody valign="top"><tr><td align="left">Factor 1: student preparation and engagement</td><td align="left" /><td align="char" char=".">.84</td></tr><tr><td align="left">My students do not know how to engage in active learning.</td><td align="char" char=".">1.00</td><td align="char" char=".">0.77</td><td align="char" char=".">0.04</td><td align="char" char=".">.60</td><td align="left" /></tr><tr><td align="left">My students do not come prepared for in‐class activities.</td><td align="char" char=".">0.95</td><td align="char" char=".">0.72</td><td align="char" char=".">0.04</td><td align="char" char=".">.51</td><td align="left" /></tr><tr><td align="left">My students play the system to circumvent learning objectives.</td><td align="char" char=".">0.88</td><td align="char" char=".">0.65</td><td align="char" char=".">0.04</td><td align="char" char=".">.42</td><td align="left" /></tr><tr><td align="left">My students are resistant to engage in active learning.</td><td align="char" char=".">1.05</td><td align="char" char=".">0.73</td><td align="char" char=".">0.04</td><td align="char" char=".">.53</td><td align="left" /></tr><tr><td align="left">My students do not expect to engage in active learning during lecture.</td><td align="char" char=".">1.02</td><td align="char" char=".">0.73</td><td align="char" char=".">0.04</td><td align="char" char=".">.53</td><td align="left" /></tr><tr><td align="left">Factor 2: instructional support</td><td align="left" /><td align="char" char=".">.81</td></tr><tr><td align="left">Educational support staff do not provide enough personalized support for me to make changes in my classes.</td><td align="char" char=".">1.00</td><td align="char" char=".">0.84</td><td align="char" char=".">0.03</td><td align="char" char=".">.71</td><td align="left" /></tr><tr><td align="left">There is not enough easily accessible material to use active learning in my course.</td><td align="char" char=".">0.80</td><td align="char" char=".">0.67</td><td align="char" char=".">0.05</td><td align="char" char=".">.45</td><td align="left" /></tr><tr><td align="left">There is insufficient technological support to ensure active learning works in my classroom.</td><td align="char" char=".">0.83</td><td align="char" char=".">0.71</td><td align="char" char=".">0.04</td><td align="char" char=".">.50</td><td align="left" /></tr><tr><td align="left">My classes do not have the teaching assistant support necessary for active learning.</td><td align="char" char=".">0.81</td><td align="char" char=".">0.66</td><td align="char" char=".">0.05</td><td align="char" char=".">.44</td><td align="left" /></tr><tr><td align="left">Factor 3: instructor comfort and confidence</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="char" char=".">.76</td></tr><tr><td align="left">I get anxious when trying active learning in class.</td><td align="char" char=".">1.00</td><td align="char" char=".">0.80</td><td align="char" char=".">0.04</td><td align="char" char=".">.63</td><td align="left" /></tr><tr><td align="left">I am uncomfortable teaching differently than the norm of lecturing.</td><td align="char" char=".">0.85</td><td align="char" char=".">0.53</td><td align="char" char=".">0.06</td><td align="char" char=".">.28</td><td align="left" /></tr><tr><td align="left">I am not confident that I have adequate skills to use active learning in my classes.</td><td align="char" char=".">0.90</td><td align="char" char=".">0.67</td><td align="char" char=".">0.05</td><td align="char" char=".">.45</td><td align="left" /></tr><tr><td align="left">I am more comfortable in the role of a lecturer rather than a facilitator.</td><td align="char" char=".">1.05</td><td align="char" char=".">0.71</td><td align="char" char=".">0.05</td><td align="char" char=".">.50</td><td align="left" /></tr><tr><td align="left">Factor 4: institutional environment/rewards</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="char" char=".">.77</td></tr><tr><td align="left">The teaching policies at my institution do not support active learning.</td><td align="char" char=".">1.00</td><td align="char" char=".">0.82</td><td align="char" char=".">0.04</td><td align="char" char=".">.67</td><td align="left" /></tr><tr><td align="left">My colleagues are generally encouraging about using active learning.</td><td align="char" char=".">−0.60</td><td align="char" char=".">−0.55</td><td align="char" char=".">0.06</td><td align="char" char=".">.30</td><td align="left" /></tr><tr><td align="left">My colleagues are not supportive of changing to a curriculum that uses active learning.</td><td align="char" char=".">1.14</td><td align="char" char=".">0.83</td><td align="char" char=".">0.04</td><td align="char" char=".">.70</td><td align="left" /></tr><tr><td align="left">There is no incentive to innovate in my teaching.</td><td align="char" char=".">0.84</td><td align="char" char=".">0.56</td><td align="char" char=".">0.06</td><td align="char" char=".">.30</td><td align="left" /></tr></tbody></table> </ephtml> </p> <hd id="AN0173054174-28">PHASE 5: CONFIRMATORY FACTOR ANALYSIS</hd> <p>In Phase 5, we conducted a second CFA with a new sample of faculty from diverse STEM fields on the 17 items of our revised instrument to confirm the factor structure established in Phase 4. Appendix B includes information on the data collection, analysis, and results of Phase 5. Our Phase 5 CFA supported the factor structure from previous phases. Consistent with the Phase 4 CFA, the relative fit indices indicated an adequate model fit, and the construct reliability ranged from 0.78 to 0.85.</p> <hd id="AN0173054174-29">DISCUSSION</hd> <p>To develop our instrument, we used a five‐phase process and recognized instrument development practices (McCoach et al., [<reflink idref="bib54" id="ref216">54</reflink>]) to build on the barriers section of the FIBIS (Sturtevant & Wheeler, [<reflink idref="bib83" id="ref217">83</reflink>]). We conducted an EFA using existing FIBIS items, plus the addition of seven items related to course learning outcomes, teaching policies, students' pedagogy expectations, professional development time, instructors' comfort in the facilitator role, interacting with students, and receiving student feedback. Using our review of previous studies and factor analysis results, we identified four underlying constructs: student preparation and engagement, instructional support, instructor comfort and confidence, and institutional environment/rewards. We chose items that best represented each of those constructs and completed another exploratory and two confirmatory factor analyses, all demonstrating reliability (Cronbach's alpha >.75) and validity of this new instrument. The final 17‐item instrument presented here is concise, provides a way to assess instructors' perceived barriers in different settings, and offers insight into how instructors categorize or think about barriers.</p> <p>The factor structure that emerged from our initial factor analysis differed from that suggested by the FIBIS. The departmental barriers category from the FIBIS narrowly incorporated the context in which instructors teach. As indicated by our analysis, this broader context should include the institutional environment, such as teaching policies; the culture surrounding incentives and rewards, such as tenure and promotion; and support for active learning instruction. The faculty beliefs category from the FIBIS comprised many loosely associated items. For example, the item "There is not enough material/problems easily accessible and available to use these evidence‐based instructional practices in my course" (Sturtevant & Wheeler, [<reflink idref="bib83" id="ref218">83</reflink>], supplemental material) represents limited support for instruction rather than faculty beliefs.</p> <hd id="AN0173054174-30">Model implications</hd> <p>A clear alignment between our four factors and the four influence categories of the instructional innovation‐design process (Lund & Stains, [<reflink idref="bib49" id="ref219">49</reflink>]) provides a coherent means of categorizing instructors' perceived barriers to employing active learning (see Table 6). The first and second columns in Table 6 illustrate the consistency of our factors with the influences in the instructional innovation‐decision process model. Three factors, student preparation and engagement (Factor 1), instructional support (Factor 2), and institutional environment/rewards (Factor 4), address "the importance of the context in which faculty are working" (Lund & Stains, [<reflink idref="bib49" id="ref220">49</reflink>], p. 12) on instructors' adoption of active learning. They are contextual influences. Both student preparation and engagement (Factor 1) and instructional support (Factor 2) align with the characteristics of the learning environment subcategory. Institutional environment/rewards most closely aligns with the departmental factors subcategory. The remaining factor from our analysis, instructor comfort and confidence (Factor 3), aligns with individual influences, which has two subcategories, attitudes and beliefs about teaching and pedagogical experiences, in the instructional innovation‐decision process.</p> <p>6 TABLE Alignment of the four factors in our final 17‐item barriers instrument with the four influences of the instructional innovation‐decision process model and our proposed modifications to the contextual influences (CIs) and individual influences (IIs) subcategories.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">Influences in the instructional innovation‐decision process (Lund & Stains, <xref ref-type="bibr" rid="bibr49">2015</xref>)</th><th align="left">Our proposed modifications to the instructional innovation‐decision process</th></tr></thead><tbody valign="top"><tr><td align="left">Characteristics of the learning environment (CI)</td><td align="left">Student preparation and engagement (CI)</td></tr><tr><td align="left">Instructional support (CI)</td></tr><tr><td align="left">Attitudes and beliefs about teaching (II)</td><td align="left">Instructor comfort and confidence (II)</td></tr><tr><td align="left">Pedagogical experiences (II)</td></tr><tr><td align="left">Departmental factors (CI)</td><td align="left">Institutional environment/rewards (CI)</td></tr></tbody></table> </ephtml> </p> <p>We propose minor modifications to the instructional innovation‐decision process influence categories based on our findings. Rather than having two subcategories for each of the two influences in the model, our findings support having three contextual influences subcategories and just one individual influences subcategory, as outlined in Table 6. For contextual influences, we subdivide characteristics of the learning environment into student preparation and engagement and instructional support because these emerged as two distinct factors in our research. We rename the other contextual influences category of the instructional innovation‐design process, departmental factors, to institutional environment/rewards because this subcategory includes institutional influences (e.g., teaching policies) and incentives and rewards that transcend the department. Regarding individual influences in the instructional innovation‐design process model, we merge the two subcategories into one—instructor comfort and confidence. Based on our factors, we find that instructor comfort and confidence are related to their pedagogical experiences with active learning and their attitudes and beliefs about active learning. We would also like to note that these modifications apply to Lattuca and Pollard's ([<reflink idref="bib46" id="ref221">46</reflink>]) faculty decision‐making process because the influential factors are similar.</p> <hd id="AN0173054174-31">Reflection on time as a barrier</hd> <p>Despite the emphasis on time in some models of barriers to adopting active learning in the literature, there is no time factor in our final instrument, nor are there any individual time items. Our initial EFA did not support the retention of any of the five items from our original 45‐item instrument that addressed time as a barrier to instructor adoption of active learning. This does not imply that instructors do not perceive time as a substantial barrier, just that these items do not reside in a single time construct. Instead, our results suggest that respondents' time‐related concerns are interrelated with other more encompassing factors.</p> <p>Although instructors consistently agree that time is a barrier to adopting active learning (e.g., Shadle et al., [<reflink idref="bib77" id="ref222">77</reflink>]), the time items did not cluster into a single coherent factor. Instead, items related to time cross‐loaded between two of our factors, instructional support and instructor comfort and confidence. This suggests that when instructors raise time‐related items as a barrier (e.g., Finelli et al., [<reflink idref="bib24" id="ref223">24</reflink>]; Shadle et al., [<reflink idref="bib77" id="ref224">77</reflink>]), they might not conceptualize the barrier primarily in the same way as others; instead, they use time as a proxy for other concerns (Zibrowski et al., [<reflink idref="bib92" id="ref225">92</reflink>]). For example, when instructors say that they do not have time to learn about and use active learning, they may mean that they do not prioritize time for active learning because it pushes them out of their comfort zone as teachers. A lack of time, an almost universal instructor concern (Shadle et al., [<reflink idref="bib77" id="ref226">77</reflink>]), might be a more acceptable rationale than discomfort for not adopting evidence‐based teaching methods. Similarly, the finding that barriers related to time load onto the instructional support factor suggests that the primary barrier might not necessarily be time; instead, it may be the perception of inadequate support to successfully adopt active learning in instructors' classes. This is consistent with Shadle et al.'s ([<reflink idref="bib77" id="ref227">77</reflink>]) finding that time permeates other barriers, such as instructional support.</p> <hd id="AN0173054174-32">Practical implications</hd> <p>The concise nature of our 17‐item barriers instrument and the small number of factors (i.e., four) that encapsulate perceived barriers to instructor adoption of active learning provides direction to faculty developers and administrators motivated to reduce the obstacles to instructional change. An instrument with validation evidence, like ours, provides a way to assess which of these common categories of barriers and individual items within those categories are most prevalent in a given environment, enabling effective and efficient efforts to promote change. It also enhances our ability to research barriers in other contexts.</p> <p>These findings of how instructors themselves conceptualize and categorize barriers are relevant for those seeking to promote instructional change (e.g., Finelli et al., [<reflink idref="bib24" id="ref228">24</reflink>]; Shadle et al., [<reflink idref="bib77" id="ref229">77</reflink>]) because they may more effectively focus instructional change efforts and institutional resources. We use the practical implications of our findings for the key barrier, time, to illustrate this. If a critical barrier to adopting active learning is preparation time or time to cover the syllabus, faculty developers would naturally focus on promoting the use of quick and simple techniques that require little preparation or course time, and administrators may provide instructors with additional course planning time. If, however, the real barrier is not preparation or course time but rather the instructor's discomfort or lack of confidence with using active learning, then faculty development efforts should focus on increasing instructors' self‐efficacy for using these techniques. Similarly, if instructors avoid adopting active learning because of a perception of inadequate institutional support, administrators might devote targeted resources toward addressing those concerns. Efforts from faculty developers or administrators who singly address time, such as providing instructors with additional time to adopt active learning in their courses, may be less effective for promoting instructional change than addressing the four underlying constructs identified in our factor analysis.</p> <hd id="AN0173054174-33">Limitations</hd> <p>Four distinct limitations bind our study. First, we had a low response rate to the Phase 2, 3, and 4 administrations of our survey (6.7%, 7.3%, and 7.3%), likely because we did not offer an incentive for participation. Despite implementing a strategy for incremental reminders that involved extending the participation deadline, we were unable to motivate significantly more respondents to offer their insights. Our sample was slightly undersized in our Phase 2 EFA (<emph>n</emph> = 368 compared with the recommended 400; Rouquette & Falissard, [<reflink idref="bib73" id="ref230">73</reflink>]). In Phase 3, univariate and multivariate normality measures indicated that our sample (<emph>n</emph> = 205) slightly exceeded common thresholds for near‐normality. Non‐normality increases recommended EFA sample sizes (Rouquette & Falissard, [<reflink idref="bib73" id="ref231">73</reflink>]; Watkins, [<reflink idref="bib91" id="ref232">91</reflink>]). For normally distributed data, the recommended minimum sample size is 240 (Watkins, [<reflink idref="bib91" id="ref233">91</reflink>]), and for ordinal data on a four‐point Likert scale, it is 500 (Rouquette & Falissard, [<reflink idref="bib73" id="ref234">73</reflink>]). However, we are confident that there is adequate validation evidence for our instrument and its factor structure because our specified model has a theoretical basis. We conducted multiple EFAs with a total number of 573 survey responses, and we used Pearson's and polychoric correlations in determining our factor structure—yielding nearly identical results. For future applications, we recommend a larger sample size for researchers using this instrument in the future in order to further assess model fit and test measurement invariance across participant subgroups (e.g., years of experience using active learning).</p> <p>Second, our respondents' demographics do not necessarily reflect engineering instructors broadly. Although we had a relatively balanced mix of faculty ranks (i.e., assistant, associate, and full) among the respondents, our sample for Phases 2, 3, and 4 is mostly representative of faculty at public MS/PhD granting institutions and few respondents reported their faculty rank as an instructor. Further, the percentages of women respondents were 28% and 26% for the Phase 3 EFA and the Phase 4 CFA, respectively—which, although not reflective of the 34.5% of women STEM faculty, is by no means an implication of the underrepresentation of women in our data (National Science Foundation National Center for Science and Engineering Statistics, [<reflink idref="bib59" id="ref235">59</reflink>]). However, we did not receive many responses from Black/African American engineering faculty or instructors.</p> <p>Third, changes made between our instrument's first and second administration required us to perform a split sample analysis to corroborate our findings from the Phase 2 EFA. These changes, beyond dropping items, included changing to a seven‐point scale and rewording a survey item. These complications did not inhibit us from corroborating our initial EFA with a second EFA or conducting our Phase 4 CFA; however, the decision point leading to the detailed changes in the instrument's structure cut our intended sample size for the first CFA in half, reducing statistical power. We did conduct a second CFA (Phase 5) that was not limited to engineering instructors to engage a large sample; we included science, technology, and mathematics instructors.</p> <p>Finally, our instrument is based on the work by Sturtevant and Wheeler ([<reflink idref="bib83" id="ref236">83</reflink>]) and a synthesis of the literature related to barriers to implementing active learning. Typically, researchers suggest making items all positive and avoiding the inclusion of negatively worded items (Sauro & Lewis, [<reflink idref="bib75" id="ref237">75</reflink>]; Sonderen et al., [<reflink idref="bib79" id="ref238">79</reflink>]). If reverse‐worded items are used, they should be used as sparingly as possible (Chyung et al., [<reflink idref="bib11" id="ref239">11</reflink>]). Because the original instrument focused on barriers, the items were almost all worded to align with the idea of critiques or complaints made by instructors skeptical of evidence‐based practices—active learning, in our case. Thus, the wording of the items can be perceived as negative. Because nearly all the items were worded this way, our final instrument did not go against the literature's recommendation of mixing negative and positive items in the same instrument. However, we did use one positively worded item that contrasted with the rest of the instrument, "My colleagues are generally encouraging about using active learning." We do not have substantive evidence that including a positively worded item that contradicts the "barriers" construct impacted our results because we did not distribute a version with this item negatively worded simultaneously. However, the impact of the positive wording in one of the items can be tested in future distributions.</p> <p>Considering the limitations we outlined, there is room for future work to improve our short‐form version of the FIBIS sub‐section on barriers to using evidence‐based instructional practices. Those interested in further exploring the reliability and validity of the constructs we proposed can most readily apply the instrument with a larger sample size. The sample sizes for our study were generally acceptable based on rules of thumb in the literature but did not reach the necessary level for other conventions. Thus, additional higher powered studies using this instrument can more confidently leverage the global and local fit statistics to provide additional, stronger evidence of construct validity. Moreover, effort can be placed in investigating the impact of time on instructors' hesitancy to adopt active learning in their classrooms. Additional items to address time as a barrier we considered in our study consistently cross‐loaded with two or more factors, suggesting that time might be a higher order factor that is directly or indirectly related to instructor decision‐making. Other researchers may consider adding new items to investigate whether time can emerge as a distinct factor, which would better align with the myriad findings in the literature about barriers to implementing active learning. Finally, applying this instrument in other instructional contexts, beyond engineering, is a worthy effort. With a large enough sample of STEM instructors, for example, one could conduct a multiple‐group CFA to examine whether measurement invariance holds among disciplines.</p> <hd id="AN0173054174-34">CONCLUSIONS</hd> <p>Engineering instructors identify many barriers to adopting active learning in their classrooms. Yet, there is no previous instrument with validation evidence to measure those barriers and no generally accepted model to categorize them. We developed such an instrument, finding that the barriers resolved into four coherent factors: student preparation and engagement, instructional support, instructor comfort and confidence, and institutional environment/rewards. This study, starting from a comprehensive list of 45 barrier items, surveying a large population of instructors (total <emph>n</emph> = 953), adopting a rigorous and thorough methodology, and arriving at a 17‐item instrument with high levels of reliability, advances the state of the literature.</p> <p>We compared the factors inherent in our instrument with existing models and found that they most closely align with Lund and Stains' ([<reflink idref="bib49" id="ref240">49</reflink>]) instructional innovation‐decision model. To reflect instructors' perceptions in categorizing barriers, we suggest modifying the influence subcategories in the instructional innovation‐decision model. We recommend including student preparation and engagement, instructional support, and institutional environment/rewards within contextual influences and a single subcategory, instructor comfort and confidence, within individual influences.</p> <p>The primary practical implication of our work is that our instrument enables researchers to assess perceived barriers more confidently, thereby improving research and ultimately offering insights into how to reduce barriers and promote positive change. Secondarily, our study suggests some possible interesting implications regarding time, one of the barriers most commonly identified in previous studies. We found that although instructors consistently identify time as a barrier to adopting active learning, time does not emerge as a standalone factor. Instead, it permeated the four coherent factors: student preparation and engagement, instructional support, instructor comfort and confidence, and institutional environment/rewards. This potentially suggests that instructional change efforts targeting time as a barrier may be less effective. Instead, the root causes of a perceived lack of time should be targeted.</p> <p>We anticipate that the instrument developed from this effort will be a useful anchor for research on barriers to active learning in engineering classrooms. Given the synthesized factor structure found as part of our analytical procedures, we anticipate that our instrument will be useful in advancing work related to instructor interventions to instill an active learning pedagogical approach by focusing on vital pain points experienced by many instructors across contexts.</p> <hd id="AN0173054174-35">ACKNOWLEDGMENTS</hd> <p>We would like to thank the instructors who responded to our survey and Hannah Sturtevant and Lindsay Wheeler for their work on the FIBIS, the basis for our barriers instrument development.</p> <hd id="AN0173054174-36">FUNDING INFORMATION</hd> <p>This work was supported by the National Science Foundation under grant numbers DUE‐1821092, DUE‐1821036, DUE‐1821488, and DUE‐1821277.</p> <hd id="AN0173054174-37">A APPENDIX</hd> <p></p> <hd id="AN0173054174-38">PHASE 3, 4, AND 5 DEMOGRAPHIC DATA</hd> <p>The distributions of gender, race/ethnicity, faculty rank, and institution type of Phase 3, 4, and 5 samples are included in Tables A1–A6. Figures A1 and A2 are histograms of years of experience teaching and using active learning of Phase 3 and 4 samples, respectively.</p> <p>A1 TABLE Distribution by race/ethnicity and gender for our Phase 3 EFA sample (n = 205).</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" /><th align="left">Race/ethnicity</th></tr><tr><th align="left">Gender</th><th align="left">Not listed</th><th align="left">A/PI</th><th align="left">B/AA</th><th align="left">H/L</th><th align="left">M/B</th><th align="left">ND</th><th align="left">W</th></tr></thead><tbody valign="top"><tr><td align="left">Man</td><td align="char" char=".">10</td><td align="char" char=".">19</td><td align="char" char=".">1</td><td align="char" char=".">7</td><td align="char" char=".">3</td><td align="char" char=".">8</td><td align="char" char=".">93</td></tr><tr><td align="left">Non‐binary</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">1</td></tr><tr><td align="left">Prefer not to disclose</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">4</td><td align="char" char=".">0</td></tr><tr><td align="left">Prefer to self‐describe</td><td align="char" char=".">2</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td></tr><tr><td align="left">Woman</td><td align="char" char=".">4</td><td align="char" char=".">5</td><td align="char" char=".">0</td><td align="char" char=".">6</td><td align="char" char=".">2</td><td align="char" char=".">2</td><td align="char" char=".">37</td></tr></tbody></table> </ephtml> </p> <p>7 Abbreviations: A/PI, Asian/Pacific Islander; B/AA, Black/African American; H/L, Hispanic/Latinx; M/B, multi/biracial; ND, not disclosed; W, White, 1 missing value.</p> <p>A2 TABLE Distribution by faculty rank and institution type for our Phase 3 EFA sample (n = 205).</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" /><th align="left">Faculty rank</th></tr><tr><th align="left">Institution type</th><th align="left">Adjunct</th><th align="left">Assistant</th><th align="left">Associate</th><th align="left">Full</th><th align="left">Instructor</th><th align="left">Not listed</th></tr></thead><tbody valign="top"><tr><td align="left">Private MS/PhD granting</td><td align="char" char=".">0</td><td align="char" char=".">8</td><td align="char" char=".">11</td><td align="char" char=".">9</td><td align="char" char=".">4</td><td align="char" char=".">1</td></tr><tr><td align="left">Private non‐MS/PhD granting</td><td align="char" char=".">0</td><td align="char" char=".">2</td><td align="char" char=".">1</td><td align="char" char=".">3</td><td align="char" char=".">0</td><td align="char" char=".">0</td></tr><tr><td align="left">Public MS/PhD granting</td><td align="char" char=".">1</td><td align="char" char=".">39</td><td align="char" char=".">41</td><td align="char" char=".">60</td><td align="char" char=".">5</td><td align="char" char=".">12</td></tr><tr><td align="left">Public non‐MS/PhD granting</td><td align="char" char=".">0</td><td align="char" char=".">1</td><td align="char" char=".">1</td><td align="char" char=".">2</td><td align="char" char=".">3</td><td align="char" char=".">0</td></tr><tr><td align="left">Two‐year college</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">1</td><td align="char" char=".">0</td></tr></tbody></table> </ephtml> </p> <p>A3 TABLE Distribution by race/ethnicity and gender for our Phase 4 CFA sample (n = 205).</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">Gender</th><th align="left">Not listed</th><th align="left">A/PI</th><th align="left">B/AA</th><th align="left">H/L</th><th align="left">M/B</th><th align="left">ND</th><th align="left">NA</th><th align="left">W</th></tr></thead><tbody valign="top"><tr><td align="left">Man</td><td align="char" char=".">9</td><td align="char" char=".">21</td><td align="char" char=".">1</td><td align="char" char=".">5</td><td align="char" char=".">3</td><td align="char" char=".">0</td><td align="char" char=".">5</td><td align="char" char=".">101</td></tr><tr><td align="left">Prefer not to disclose</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">6</td><td align="char" char=".">2</td></tr><tr><td align="left">Woman</td><td align="char" char=".">3</td><td align="char" char=".">10</td><td align="char" char=".">0</td><td align="char" char=".">1</td><td align="char" char=".">2</td><td align="char" char=".">1</td><td align="char" char=".">2</td><td align="char" char=".">32</td></tr></tbody></table> </ephtml> </p> <p>8 Abbreviations: A/PI, Asian/Pacific Islander; B/AA, Black/African American; H/L, Hispanic/Latinx; M/B, multi/biracial; ND, not disclosed; NA, Native American; W, White, 1 missing value.</p> <p>A4 TABLE Distribution by faculty rank and institution type for our Phase 4 CFA sample (n = 205).</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" /><th align="left">Adjunct</th><th align="left">Assistant</th><th align="left">Associate</th><th align="left">Full</th><th align="left">Instructor</th><th align="left">Not listed</th></tr></thead><tbody valign="top"><tr><td align="left">Private MS/PhD granting</td><td align="char" char=".">2</td><td align="char" char=".">6</td><td align="char" char=".">11</td><td align="char" char=".">6</td><td align="char" char=".">5</td><td align="char" char=".">2</td></tr><tr><td align="left">Private non‐MS/PhD granting</td><td align="char" char=".">0</td><td align="char" char=".">2</td><td align="char" char=".">2</td><td align="char" char=".">1</td><td align="char" char=".">1</td><td align="char" char=".">0</td></tr><tr><td align="left">Public MS/PhD granting</td><td align="char" char=".">3</td><td align="char" char=".">39</td><td align="char" char=".">41</td><td align="char" char=".">50</td><td align="char" char=".">9</td><td align="char" char=".">14</td></tr><tr><td align="left">Public non‐MS/PhD granting</td><td align="char" char=".">0</td><td align="char" char=".">3</td><td align="char" char=".">3</td><td align="char" char=".">1</td><td align="char" char=".">0</td><td align="char" char=".">1</td></tr><tr><td align="left">Two‐year college</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">3</td><td align="char" char=".">0</td></tr></tbody></table> </ephtml> </p> <p>A5 TABLE Distribution by race/ethnicity and sex for Phase 5 CFA sample (n = 175).</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" /><th align="left">A/PI</th><th align="left">B/AA</th><th align="left">WnH</th><th align="left">WHS</th><th align="left">M/B</th><th align="left">Other</th><th align="left">ND</th></tr></thead><tbody valign="top"><tr><td align="left">Female</td><td align="char" char=".">17</td><td align="char" char=".">3</td><td align="char" char=".">67</td><td align="char" char=".">0</td><td align="char" char=".">6</td><td align="char" char=".">3</td><td align="char" char=".">0</td></tr><tr><td align="left">Male</td><td align="char" char=".">18</td><td align="char" char=".">2</td><td align="char" char=".">39</td><td align="char" char=".">5</td><td align="char" char=".">3</td><td align="char" char=".">4</td><td align="char" char=".">0</td></tr><tr><td align="left">Something else</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">1</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td></tr><tr><td align="left">Not sure</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">2</td><td align="char" char=".">0</td></tr><tr><td align="left">ND</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">0</td><td align="char" char=".">5</td></tr></tbody></table> </ephtml> </p> <p>9 Abbreviations: A/PI, Asian/Pacific Islander, B/AA, Black/African American, WnH, White or Caucasian non‐Hispanic, WHS, White Hispanic/Latinx, M/B, multi/biracial, ND, not disclosed.</p> <p>A6 TABLE Distribution by faculty rank and institution type for our Phase 5 CFA sample (n = 175).</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" /><th align="left">Adjunct</th><th align="left">Assistant</th><th align="left">Associate</th><th align="left">Full</th><th align="left">Department head/director</th><th align="left">Instructor</th><th align="left">Not listed</th></tr></thead><tbody valign="top"><tr><td align="left">Associates college</td><td align="char" char=".">5</td><td align="char" char=".">1</td><td align="char" char=".">3</td><td align="char" char=".">5</td><td align="char" char=".">1</td><td align="char" char=".">11</td><td align="char" char=".">0</td></tr><tr><td align="left">Baccalaureate college</td><td align="char" char=".">5</td><td align="char" char=".">11</td><td align="char" char=".">10</td><td align="char" char=".">4</td><td align="char" char=".">4</td><td align="char" char=".">9</td><td align="char" char=".">0</td></tr><tr><td align="left">Master's college or university</td><td align="char" char=".">0</td><td align="char" char=".">14</td><td align="char" char=".">14</td><td align="char" char=".">3</td><td align="char" char=".">3</td><td align="char" char=".">8</td><td align="char" char=".">1</td></tr><tr><td align="left">Doctorate‐granting university</td><td align="char" char=".">4</td><td align="char" char=".">15</td><td align="char" char=".">13</td><td align="char" char=".">7</td><td align="char" char=".">0</td><td align="char" char=".">24</td><td align="char" char=".">0</td></tr></tbody></table> </ephtml> </p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/6M4/01oct23/jee20557-fig-0001.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="jee20557-fig-0001.jpg" title="A1 Histogram of instructors' years of experience teaching (left panel) and using active learning (right panel) for our Phase 3 EFA sample." /> </p> <p></p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/6M4/01oct23/jee20557-fig-0002.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="jee20557-fig-0002.jpg" title="A2 Histogram of instructor experience teaching and with active learning for our Phase 4 CFA sample." /> </p> <p></p> <hd id="AN0173054174-41">B APPENDIX</hd> <p></p> <hd id="AN0173054174-42">PHASE 5 METHODS AND RESULTS</hd> <p></p> <hd id="AN0173054174-43">Phase 5: Confirmatory factor analysis</hd> <p>In Phase 5, we conducted a second CFA with a new sample of faculty from diverse science, technology, engineering, and mathematics fields on the 17 items of our revised instrument to confirm the factor structure established in Phase 4.</p> <hd id="AN0173054174-44">Data collection</hd> <p>We recruited participants (<emph>n</emph> = 175) from 64 colleges and universities from a range of disciplines (82 from science, 27 from engineering, and 66 from math/computer science) as part of a larger study (Marlor et al., [<reflink idref="bib52" id="ref241">52</reflink>]) on instructional practices. The ratio of the sample size to measured variables in this CFA is approximately 10 (175:17) and is adequate for the relative simplicity of the model and the number of parameters estimated (Kline, [<reflink idref="bib42" id="ref242">42</reflink>]). After obtaining IRB and departmental approval at each institution from which we recruited participants, we administered a survey, which included the 17‐item instrument from Phase 4 and items asking about demographic data, to these instructors. We present these demographic data by race/ethnicity and sex (Holzberg et al., [<reflink idref="bib35" id="ref243">35</reflink>]) in Table A5 and by faculty rank and institution type in Table A6.</p> <hd id="AN0173054174-45">Analysis</hd> <p>We conducted a CFA using ML estimation. Based on our findings in Phase 4, we allowed the residual variance for the second and third items in the institutional environment/rewards factor ("My colleagues are not supportive of changing to a curriculum that uses active learning." and "My colleagues are generally encouraging about using active learning.") to correlate. For consistency, we reverse‐coded the responses for the one positively worded item, "My colleagues are generally encouraging about using active learning."</p> <p>We assessed model fit using Byrne's ([<reflink idref="bib10" id="ref244">10</reflink>]) suggestions for indices: model chi‐square (<emph>p</emph> > .05), CFI (>.9,.95 preferred; Hu & Bentler, [<reflink idref="bib38" id="ref245">38</reflink>]), TLI (>.90 at minimum,.95 preferred; Hu & Bentler, [<reflink idref="bib38" id="ref246">38</reflink>]), RMSEA (<.01 is excellent, <.05 is good, and.08 is moderate; MacCallum et al., [<reflink idref="bib50" id="ref247">50</reflink>]), and SRMR (<.10 preferred; Kline, [<reflink idref="bib42" id="ref248">42</reflink>]). We again set a benchmark Cronbach's alpha for each factor of at least.70 (Thorndike et al., [<reflink idref="bib85" id="ref249">85</reflink>]).</p> <hd id="AN0173054174-46">Results</hd> <p>Our CFA of the four‐factor model with residual covariance between the second and fourth items of the institutional environment/rewards factor suggests high internal consistency and adequate model fit. Internal consistency for each item was excellent (<emph>α</emph> = .78 to.85). Our CFA indicated adequate fit (<emph>χ</emph><sups><emph>2</emph></sups>(<reflink idref="bib112" id="ref250">112</reflink>) = 183.727, <emph>p</emph> < .05, CFI = .935, TLI = .921, RMSEA = .064, 90% CI [.047,.080], <emph>p</emph><subs>close</subs> = .111, and SRMR = .063). We present parameter estimates in Table B1.</p> <p>B1 TABLE Model statistics for our Phase 5 CFA.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" /><th align="left">Unstandardized factor loadings</th><th align="left">Standardized factor loadings</th><th align="left">Standard error</th><th align="left">Item reliability (<italic>R</italic><sup>2</sup>)</th><th align="left">Construct reliability</th></tr></thead><tbody valign="top"><tr><td align="left">Factor 1: student preparation and engagement</td><td align="left" /><td align="char" char=".">.85</td></tr><tr><td align="left">My students do not know how to engage in active learning.</td><td align="char" char=".">1.00</td><td align="char" char=".">0.80</td><td align="char" char=".">0.04</td><td align="char" char=".">.64</td><td align="left" /></tr><tr><td align="left">My students are resistant to engage in active learning.</td><td align="char" char=".">0.95</td><td align="char" char=".">0.74</td><td align="char" char=".">0.05</td><td align="char" char=".">.54</td><td align="left" /></tr><tr><td align="left">My students do not come prepared for in‐class activities.</td><td align="char" char=".">0.61</td><td align="char" char=".">0.53</td><td align="char" char=".">0.07</td><td align="char" char=".">.28</td><td align="left" /></tr><tr><td align="left">My students do not expect to engage in active learning during lecture.</td><td align="char" char=".">1.10</td><td align="char" char=".">0.85</td><td align="char" char=".">0.03</td><td align="char" char=".">.72</td><td align="left" /></tr><tr><td align="left">My students play the system to circumvent learning objectives.</td><td align="char" char=".">1.07</td><td align="char" char=".">0.75</td><td align="char" char=".">0.05</td><td align="char" char=".">.56</td><td align="left" /></tr><tr><td align="left">Factor 2: instructional support</td><td align="left" /><td align="char" char=".">.80</td></tr><tr><td align="left">Educational support staff do not provide enough personalized support for me to make changes in my classes.</td><td align="char" char=".">1.00</td><td align="char" char=".">0.73</td><td align="char" char=".">0.06</td><td align="char" char=".">.54</td><td align="left" /></tr><tr><td align="left">There is not enough easily accessible material to use active learning in my course.</td><td align="char" char=".">1.09</td><td align="char" char=".">0.68</td><td align="char" char=".">0.07</td><td align="char" char=".">.46</td><td align="left" /></tr><tr><td align="left">There is insufficient technological support to ensure active learning works in my classroom.</td><td align="char" char=".">1.21</td><td align="char" char=".">0.79</td><td align="char" char=".">0.05</td><td align="char" char=".">.63</td><td align="left" /></tr><tr><td align="left">My classes do not have the teaching assistant support necessary for active learning.</td><td align="char" char=".">1.00</td><td align="char" char=".">0.66</td><td align="char" char=".">0.06</td><td align="char" char=".">.44</td><td align="left" /></tr><tr><td align="left">Factor 3: instructor comfort and confidence</td><td align="char" char=".">.81</td></tr><tr><td align="left">I get anxious when trying active learning in class.</td><td align="char" char=".">1.00</td><td align="char" char=".">0.74</td><td align="char" char=".">0.07</td><td align="char" char=".">.54</td><td align="left" /></tr><tr><td align="left">I am uncomfortable teaching differently than the norm of lecturing.</td><td align="char" char=".">1.02</td><td align="char" char=".">0.67</td><td align="char" char=".">0.06</td><td align="char" char=".">.46</td><td align="left" /></tr><tr><td align="left">I am not confident that I have adequate skills to use active learning in my classes.</td><td align="char" char=".">0.99</td><td align="char" char=".">0.78</td><td align="char" char=".">0.05</td><td align="char" char=".">.61</td><td align="left" /></tr><tr><td align="left">I am more comfortable in the role of a lecturer rather than a facilitator.</td><td align="char" char=".">0.85</td><td align="char" char=".">0.69</td><td align="char" char=".">0.06</td><td align="char" char=".">.47</td><td align="left" /></tr><tr><td align="left">Factor 4: institutional environment/rewards</td><td align="char" char=".">.78</td></tr><tr><td align="left">The teaching policies at my institution do not support active learning.</td><td align="char" char=".">1.00</td><td align="char" char=".">0.74</td><td align="char" char=".">0.05</td><td align="char" char=".">.55</td><td align="left" /></tr><tr><td align="left">My colleagues are generally encouraging about using active learning.</td><td align="char" char=".">0.72</td><td align="char" char=".">0.60</td><td align="char" char=".">0.09</td><td align="char" char=".">.36</td><td align="left" /></tr><tr><td align="left">My colleagues are not supportive of changing to a curriculum that uses active learning.</td><td align="char" char=".">0.82</td><td align="char" char=".">0.73</td><td align="char" char=".">0.06</td><td align="char" char=".">.53</td><td align="left" /></tr><tr><td align="left">There is no incentive to innovate in my teaching.</td><td align="char" char=".">0.94</td><td align="char" char=".">0.61</td><td align="char" char=".">0.06</td><td align="char" char=".">.38</td><td align="left" /></tr></tbody></table> </ephtml> </p> <p>GRAPH: Data S1. Supporting Information.</p> <ref id="AN0173054174-47"> <title> Footnotes </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> Any opinions, findings, and conclusions or recommendations expressed in this study are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.</bibtext> </blist> </ref> <ref id="AN0173054174-48"> <title> REFERENCES </title> <blist> <bibtext> Andrews, T. C., & Lemons, P. P. (2015). It's personal: Biology instructors prioritize personal evidence over empirical evidence in teaching decisions. CBE Life Sciences Education, 14 (1), ar7. https://doi.org/10.1187/cbe.14-05-0084</bibtext> </blist> <blist> <bibl id="bib2" idref="ref2" type="bt">2</bibl> <bibtext> Apkarian, N., Henderson, C., Stains, M., Raker, J., Johnson, E., & Dancy, M. (2021). What really impacts the use of active learning in undergraduate STEM education? Results from a national survey of chemistry, mathematics, and physics instructors. PLoS One, 16 (2), e0247544. https://doi.org/10.1371/journal.pone.0247544</bibtext> </blist> <blist> <bibl id="bib3" idref="ref3" type="bt">3</bibl> <bibtext> Babbie, E. (2011). Paradigms, theory, and research. In E. Mitchell, D. Moneypenny, E. Parkins & M. Cregger (Eds.), The basics of social research (5th ed., pp. 31 – 63). Wadsworth.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref4" type="bt">4</bibl> <bibtext> Baglin, J. (2014). Improving your exploratory factor analysis for ordinal data: A demonstration using FACTOR. Practical Assessment, Research, and Evaluation, 19 (1), 1 – 15. https://doi.org/10.7275/dsep-4220</bibtext> </blist> <blist> <bibl id="bib5" idref="ref150" type="bt">5</bibl> <bibtext> Bartlett, M. S. (1950). Test of significance in factor analysis. British Journal of Psychology, 3 (2), 77 – 85. https://doi.org/10.1111/j.2044-8317.1950.tb00285.x</bibtext> </blist> <blist> <bibl id="bib6" idref="ref138" type="bt">6</bibl> <bibtext> Bennett, D. A. (2001). How can I deal with missing data in my study? Australian and New Zealand Journal of Public Health, 25 (5), 464 – 469. https://doi.org/10.1111/j.1467-842X.2001.tb00294.x</bibtext> </blist> <blist> <bibl id="bib7" idref="ref22" type="bt">7</bibl> <bibtext> Borrego, M., Froyd, J. E., & Hall, T. S. (2010). Diffusion of engineering education innovations: A survey of awareness and adoption rates in U.S. engineering departments. Journal of Engineering Education, 99 (3), 185 – 207. https://doi.org/10.1002/j.2168-9830.2010.tb01056.x</bibtext> </blist> <blist> <bibl id="bib8" idref="ref71" type="bt">8</bibl> <bibtext> Borrego, M., & Henderson, C. (2014). Increasing the use of evidence‐based teaching in STEM higher education: A comparison of eight change strategies. Journal of Engineering Education, 103 (2), 220 – 252. https://doi.org/10.1002/jee.20040</bibtext> </blist> <blist> <bibl id="bib9" idref="ref11" type="bt">9</bibl> <bibtext> Brownell, S. E., & Tanner, K. D. (2012). Barriers to faculty pedagogical change: Lack of training, time, incentives, and ... tensions with professional identity? CBE Life Sciences Education, 11 (4), 339 – 346. https://doi.org/10.1187/cbe.12-09-0163</bibtext> </blist> <blist> <bibtext> Byrne, B. M. (1994). Structural equation modeling with EQS and EQS/Windows: Basic concepts, applications, and programming. Sage Publications.</bibtext> </blist> <blist> <bibtext> Chyung, S. Y., Barkin, J. R., & Shamsy, J. A. (2018). Evidence‐based survey design: The use of negatively worded items in surveys. Performance Improvement, 57 (3), 16 – 25. https://doi.org/10.1002/pfi.21749</bibtext> </blist> <blist> <bibtext> Collinson, V., & Fedoruk Cook, T. (2001). "I don't have enough time": Teachers' interpretations of time as a key to learning and school change. Journal of Educational Administration, 39 (3), 266 – 281. https://doi.org/10.1108/09578230110392884</bibtext> </blist> <blist> <bibtext> Costello, A. B., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10, 1–9. https://doi.org/10.7275/jyj1-4868</bibtext> </blist> <blist> <bibtext> Cummins, R. A., & Gullone, E. (2000). Why we should not use 5‐point Likert scales: The case for subjective quality of life measurement. Second International Conference on Quality of Life in Cities, 74 (2), 74 – 93.</bibtext> </blist> <blist> <bibtext> Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1 (1), 16 – 29. https://doi.org/10.1037/1082-989X.1.1.16</bibtext> </blist> <blist> <bibtext> Cutler, S., Borrego, M., Henderson, C., Prince, M., & Froyd, J. (2012). A comparison of electrical, computer, and chemical engineering faculty's progression through the innovation‐decision process. Paper presented at the Meeting of Frontiers in Education Conference Proceedings, Seattle, WA. https://doi.org/10.1109/FIE.2012.6462405</bibtext> </blist> <blist> <bibtext> Dancy, M., & Henderson, C. (2008). Barriers and promises in STEM reform. The National Academies of Sciences Engineering Medicine. https://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_072636.pdf</bibtext> </blist> <blist> <bibtext> DeMonbrun, M., Cañas, J., & Finelli, C. J. (2019). Influencing changes in teaching: Addressing motivators and barriers to adopting evidence‐based teaching practices in engineering. Journal on Excellence in College Teaching, 30 (1), 101 – 125.</bibtext> </blist> <blist> <bibtext> Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail and mixed‐mode surveys: The tailored design method. John Whiley & Sons.</bibtext> </blist> <blist> <bibtext> Dunlap, H. F. (1931). An empirical determination of means, standard deviations and correlation coefficients drawn from rectangular distributions. Annals of Mathematical Statistics, 2, 66 – 81. https://doi.org/10.1214/aoms/1177733043</bibtext> </blist> <blist> <bibtext> Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53 (1), 109 – 132. https://doi.org/10.1146/annurev.psych.53.100901.135153</bibtext> </blist> <blist> <bibtext> Ender, P. B. (2010). fapara. Factor analysis parallel analysis. Stata User Written Command. https://stats.idre.ucla.edu/stat/stata/ado/analysis</bibtext> </blist> <blist> <bibtext> Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4 (3), 272 – 299. https://doi.org/10.1037/1082-989X.4.3.272</bibtext> </blist> <blist> <bibtext> Finelli, C. J., Daly, S. R., & Richardson, K. M. (2014). Bridging the research‐to‐practice gap: Designing an institutional change plan using local evidence. Journal of Engineering Education, 103 (2), 331 – 361. https://doi.org/10.1002/jee.20042</bibtext> </blist> <blist> <bibtext> Finney, S. J., & DiStefano, C. (2013). Nonnormal and categorical data in structural equation modeling. In G. R. Hancock & R. O. Mueller (Eds.), Structural equation modeling: A second course (2nd ed., pp. 439 – 492). Information Age Publishing.</bibtext> </blist> <blist> <bibtext> Finstad, K. (2010). Response interpolation and scale sensitivity: Evidence against 5‐point scales. Journal of Usability Studies, 5 (3), 104 – 110.</bibtext> </blist> <blist> <bibtext> Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111 (23), 8410 – 8415. https://doi.org/10.1073/pnas.1319030111</bibtext> </blist> <blist> <bibtext> Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Erlbaum Associates.</bibtext> </blist> <blist> <bibtext> Havlicek, L. L., & Peterson, N. L. (1976). Robustness of the Pearson correlation against violation of assumption. Perceptual and Motor Skills, 43 (3), 1319 – 1334. https://doi.org/10.2466/pms.1976.43.3f.1319</bibtext> </blist> <blist> <bibtext> Henderson, C. (2008). Promoting instructional change in new faculty: An evaluation of the physics and astronomy new faculty workshop. American Journal of Physics, 76 (2), 179 – 187. https://doi.org/10.1119/1.2820393</bibtext> </blist> <blist> <bibtext> Henderson, C., Beach, A., & Finkelstein, N. (2011). Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature. Journal of Research in Science Teaching, 48 (8), 952 – 984. https://doi.org/10.1002/tea.20439</bibtext> </blist> <blist> <bibtext> Henderson, C., & Dancy, M. (2007). Barriers to the use of research‐based instructional strategies: The influence of both individual and situational characteristics. Physical Review Special Topics—Physics Education Research, 3 (2), 020102. https://doi.org/10.1103/PhysRevSTPER.3.020102</bibtext> </blist> <blist> <bibtext> Hinkin, T. R. (1995). A review of scale development practices in the study of organizations. Journal of Management, 21 (5), 967 – 988. https://doi.org/10.1177/014920639502100509</bibtext> </blist> <blist> <bibtext> Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1 (1), 104 – 121. https://doi.org/10.1177/109442819800100106</bibtext> </blist> <blist> <bibtext> Holzberg, J., Ellis, R., Virgile, M., Nelson, D., Edgar, J., Phipps, P., & Kaplan, R. (2017). Assessing the feasibility of asking about gender identity in the current population survey: results from focus groups with members of the transgender population (Survey Methodology #2018‐05). U.S. Census Bureau. <ulink href="http://www.census.gov/content/dam/Census/library/working-papers/2018/adrm/rsm2018-05.pdf">http://www.census.gov/content/dam/Census/library/working-papers/2018/adrm/rsm2018-05.pdf</ulink></bibtext> </blist> <blist> <bibtext> Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30 (2), 179 – 185. https://doi.org/10.1007/bf02289447</bibtext> </blist> <blist> <bibtext> Howard, M. C. (2016). A review of exploratory factor analysis decisions and overview of current practices: What we are doing and how can we improve? International Journal of Human‐Computer Interaction, 32 (1), 51 – 62. https://doi.org/10.1080/10447318.2015.1087664</bibtext> </blist> <blist> <bibtext> Hu, L., & Bentler, P. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 76 – 99). Sage Publications.</bibtext> </blist> <blist> <bibtext> Jackson, D. L. (2003). Revisiting sample size and number of parameter estimates: Some support for the N:Q hypothesis. Structural Equation Modeling, 10 (1), 128 – 141. https://doi.org/10.1207/S15328007SEM1001_6</bibtext> </blist> <blist> <bibtext> Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y. (2022). semTools: Useful tools for structural equation modeling. R package version 0.5‐6. https://CRAN.R-project.org/package=semTools</bibtext> </blist> <blist> <bibtext> Kaiser, H. F., & Rice, J. (1974). Little jiffy, mark IV. Educational and Psychological Measurement, 34 (1), 111 – 117. https://doi.org/10.1177/001316447403400115</bibtext> </blist> <blist> <bibtext> Kline, R. B. (2016). Principles and practice of structural equation modeling (4th ed.). The Guilford Press.</bibtext> </blist> <blist> <bibtext> Kolenikov, S. (n.d.). polychoric—The polychoric correlation package (Version No. 1.4). Stata User Written Command. <ulink href="http://staskolenikov.net/stata">http://staskolenikov.net/stata</ulink></bibtext> </blist> <blist> <bibtext> Krosnick, J. A., & Fabrigar, L. R. (1997). Designing rating scales for effective measurement in surveys. In L. Lyberg, P. Biemer, M. Collins, E. de Leeuw, C. Dippo, N. Schwarz, & D. Trewin (Eds.), Survey measurement and process quality (pp. 141 – 164). John Wiley & Sons. https://doi.org/10.1002/9781118490013.ch6</bibtext> </blist> <blist> <bibtext> Kyriazos, T. A. (2018). Applied psychometrics: Sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general. Psychology, 9 (8), 2207 – 2230. https://doi.org/10.4236/psych.2018.98126</bibtext> </blist> <blist> <bibtext> Lattuca, L. R., & Pollard, J. R. (2016). Toward a conceptualization of faculty decision‐making about curricular and instructional change. In L. Leisyte & U. Wilkesmann (Eds.), Organizing academic work in higher education: Teaching, learning and identities (pp. 89 – 108). Taylor & Francis Group.</bibtext> </blist> <blist> <bibtext> Lombardi, D., Shipley, T. F., Bailey, J. M., Bretones, P. S., Prather, E. E., Ballen, C. J., Knight, J. K., Smith, M. K., Stowe, R. L., Cooper, M. M., Prince, M., Atit, K., Uttal, D. H., LaDue, N. D., McNeal, P. M., Ryker, K., St. John, K., van der Hoeven Kraft, K. J., & Docktor, J. L. (2021). The curious construct of active learning. Psychological Science in the Public Interest, 22 (1), 8 – 43. https://doi.org/10.1177/1529100620973974</bibtext> </blist> <blist> <bibtext> Lucke, T., Dunn, P. K., & Christie, M. (2017). Activating learning in engineering education using ICT and the concept of flipping the classroom. European Journal of Engineering Education, 42 (1), 45 – 57. https://doi.org/10.1080/03043797.2016.1201460</bibtext> </blist> <blist> <bibtext> Lund, T. J., & Stains, M. (2015). The importance of context: An exploration of factors influencing the adoption of student‐centered teaching among chemistry, biology, and physics faculty. International Journal of STEM Education, 2 (1), 1 – 21. https://doi.org/10.1186/s40594-015-0026-8</bibtext> </blist> <blist> <bibtext> MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1 (2), 130 – 149. https://doi.org/10.1037/1082-989X.1.2.130</bibtext> </blist> <blist> <bibtext> Manduca, C. A., Iverson, E. R., Luxenberg, M., Macdonald, R. H., McConnell, D. A., Mogk, D. W., & Tewksbury, B. J. (2017). Improving undergraduate STEM education: The efficacy of discipline‐based professional development. Science Advances, 3 (2), e1600193. https://doi.org/10.1126/sciadv.1600193</bibtext> </blist> <blist> <bibtext> Marlor, L., Finelli, C. J., Husman, J., Borrego, M., Graham, M., Smith, M., & Prince, M. (2022). Reducing student resistance to active learning through faculty development: Project update. Paper presented at the ASEE Annual Conference and Exposition, Minneapolis, MN.</bibtext> </blist> <blist> <bibtext> Matsunaga, M. (2010). How to factor‐analyze your data right: Do's, don'ts, and how‐to's. International Journal of Psychological Research, 3 (1), 97 – 110. https://doi.org/10.21500/20112084.854</bibtext> </blist> <blist> <bibtext> McCoach, D. B., Gable, R. K., & Madura, J. P. (2013). Instrument development in the affective domain: School and corporate applications (3rd ed.). Springer.</bibtext> </blist> <blist> <bibtext> Michael, J. (2007). Faculty perceptions about barriers to active learning. College Teaching, 55 (2), 42 – 47. https://doi.org/10.3200/CTCH.55.2.42-47</bibtext> </blist> <blist> <bibtext> Morales, J. C., & Prince, M. J. (2019). Promoting lasting change in teaching practices through a summer immersion faculty development program. International Journal of Engineering Education, 35 (3), 968 – 985.</bibtext> </blist> <blist> <bibtext> Muthen, B., & Kaplan, D. (1992). A comparison of some methodologies for the factor analysis of non‐normal Likert variables: A note on the size of the model. British Journal of Mathematical and Statistical Psychology, 45 (1), 19 – 30. https://doi.org/10.1111/j.2044-8317.1992.tb00975.x</bibtext> </blist> <blist> <bibtext> National Research Council. (2012). Instructional Strategies. In S. R. Singer, N. R. Nielsen, & H. A. Schweingruber (Eds.), Discipline‐based education research: Understanding and improving learning in undergraduate science and engineering. The National Academies Press; 119–139.</bibtext> </blist> <blist> <bibtext> National Science Foundation National Center for Science and Engineering Statistics. (2019). Women, minorities, and persons with disabilities in science and engineering: 2019. (Special report NSF 19–304). https://ncses.nsf.gov/pubs/nsf19304/data</bibtext> </blist> <blist> <bibtext> Nguyen, K. A., Borrego, M., Finelli, C. J., DeMonbrun, M., Crockett, C., Tharayil, S., Shekhar, P., Waters, C., & Rosenberg, R. (2021). Instructor strategies to aid implementation of active learning: A systematic literature review. International Journal of STEM Education, 8 (1), 1 – 18. https://doi.org/10.1186/s40594-021-00270-7</bibtext> </blist> <blist> <bibtext> Norman, G. (2010). Likert scales, levels of measurement and the "laws" of statistics. Advances in Health Sciences Education, 15 (5), 625 – 632. https://doi.org/10.1007/s10459-010-9222-y</bibtext> </blist> <blist> <bibtext> Nunnally, J. (1978). Psychometric theory (2nd ed.). McGraw‐Hill.</bibtext> </blist> <blist> <bibtext> Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw‐Hill.</bibtext> </blist> <blist> <bibtext> Pearson, E. S. (1931a). The analysis of variance in the case of non‐normal variation. Biometrika, 23 (1–2), 114 – 133. https://doi.org/10.1093/biomet/23.1-2.114</bibtext> </blist> <blist> <bibtext> Pearson, E. S. (1931b). The test of significance for the correlation coefficient. Journal of the American Statistical Association, 26 (174), 128 – 134. https://doi.org/10.1080/01621459.1931.10503208</bibtext> </blist> <blist> <bibtext> Pearson, E. S. (1932). The test of significance for the correlation coefficient: Some further results. Journal of the American Statistical Association, 27 (180), 424 – 426. https://doi.org/10.1080/01621459.1932.10502244</bibtext> </blist> <blist> <bibtext> Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93 (3), 223 – 232. https://doi.org/10.1002/j.2168-9830.2004.tb00809.x</bibtext> </blist> <blist> <bibtext> Qualtrics. (2021). Provo, UT. https://<ulink href="http://www.qualtrics.com">www.qualtrics.com</ulink></bibtext> </blist> <blist> <bibtext> R Core Team. (2023). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://<ulink href="http://www.R-project.org/">www.R-project.org/</ulink></bibtext> </blist> <blist> <bibtext> Rodríguez, M., Díaz, I., Gonzalez, E. J., & González‐Miquel, M. (2018). Motivational active learning: An integrated approach to teaching and learning process control. Education for Chemical Engineers, 24, 7 – 12. https://doi.org/10.1016/j.ece.2018.06.003</bibtext> </blist> <blist> <bibtext> Rogers, E. M. (2003). Diffusion of innovations (5th ed., pp. 166 – 206). Free Press.</bibtext> </blist> <blist> <bibtext> Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48 (2), 1 – 36. <ulink href="http://www.jstatsoft.org/v48/i02/">http://www.jstatsoft.org/v48/i02/</ulink></bibtext> </blist> <blist> <bibtext> Rouquette, A., & Falissard, B. (2011). Sample size requirements for the internal validation of psychiatric scales. International Journal of Methods in Psychiatric Research, 20 (4), 235 – 249. https://doi.org/10.1002/mpr.352</bibtext> </blist> <blist> <bibtext> Saris, W. E., Satorra, A., & Van der Veld, W. M. (2009). Testing structural equation models or detection of misspecifications? Structural Equation Modeling, 16 (4), 561 – 582. https://doi.org/10.1080/10705510903203433</bibtext> </blist> <blist> <bibtext> Sauro, J., & Lewis, J. R. (2011). When designing usability questionnaires, does it hurt to be positive? Paper presented at the SIGCHI Conference on Human Factors in Computing Systems, BC, Vancouver, Canada. https://doi.org/10.1145/1978942.1979266</bibtext> </blist> <blist> <bibtext> Schafer, J. L. (1999). Multiple imputation: A primer. Statistical Methods in Medical Research, 8 (1), 3 – 15. https://doi.org/10.1177/096228029900800102</bibtext> </blist> <blist> <bibtext> Shadle, S. E., Marker, A., & Earl, B. (2017). Faculty drivers and barriers: Laying the groundwork for undergraduate STEM education reform in academic departments. International Journal of STEM Education, 4 (8), 1 – 13. https://doi.org/10.1186/s40594-017-0062-7</bibtext> </blist> <blist> <bibtext> Soldz, S. (2002). 'MINAP': Module to calculate minimum average partial correlation for principal components. Stata User Written Command. <ulink href="http://fmwww.bc.edu/RePEc/bocode/m">http://fmwww.bc.edu/RePEc/bocode/m</ulink></bibtext> </blist> <blist> <bibtext> Sonderen, E. V., Sanderman, R., & Coyne, J. C. (2013). Ineffectiveness of reverse wording of questionnaire items: Let's learn from cows in the rain. PLoS One, 8 (7). https://doi.org/10.1371/journal.pone.0068967</bibtext> </blist> <blist> <bibtext> Stains, M., Harshman, J., Barker, M. K., Chasteen, S. V., Cole, R., DeChenne‐Peters, S. E., Eagan, M. K., Esson, J. M., Knight, J. K., Laski, F. A., Levis‐Fitzgerald, M., Lee, C. J., Lo, S. M., McDonnell, L. M., McKay, T. A., Michelotti, N., Musgrove, A., Palmer, M. S., Plank, K. M., ... Young, A. M. (2018). Anatomy of STEM teaching in North American universities. Science, 359 (6383), 1468 – 1470. https://doi.org/10.1126/science.aap8892</bibtext> </blist> <blist> <bibtext> StataCorp. (2019). Stata statistical software: Release 16. StataCorp LLC. https://<ulink href="http://www.stata.com/">www.stata.com/</ulink></bibtext> </blist> <blist> <bibtext> StataCorp. (2021). Stata statistical software: Release 17. StataCorp LLC. https://<ulink href="http://www.stata.com/">www.stata.com/</ulink></bibtext> </blist> <blist> <bibtext> Sturtevant, H., & Wheeler, L. (2019). The STEM faculty instructional barriers and identity survey (FIBIS): Development and exploratory results. International Journal of STEM Education, 6 (35), 1 – 22. https://doi.org/10.1186/s40594-019-0185-0</bibtext> </blist> <blist> <bibtext> Theobald, E. J., Hill, M. J., Tran, E., Agrawal, S., Arroyo, E. N., Behling, S., Chambwe, N., Cintrón, D. L., Cooper, J. D., Dunster, G., Grummer, J. A., Hennessey, K., Hsiao, J., Iranon, N., Jones, L., Jordt, H., Keller, M., Lacey, M. E., Littlefield, C. E., & Freeman, S. (2020). Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proceedings of the National Academy of Sciences of the United States of America, 117 (12), 6476 – 6483. https://doi.org/10.1073/pnas.1916903117</bibtext> </blist> <blist> <bibtext> Thorndike, R. M., Cunningham, G. K., Thorndike, R. L., & Hagen, E. P. (1991). Measurement and evaluation in psychology and education (5th ed.). Macmillan Publishing.</bibtext> </blist> <blist> <bibtext> Trigwell, K., Prosser, M., & Ginns, P. (2005). Phenomenographic pedagogy and a revised approaches to teaching inventory. Higher Education Research & Development, 24 (4), 349 – 360. https://doi.org/10.1080/07294360500284730</bibtext> </blist> <blist> <bibtext> Tsang, A., & Harris, D. M. (2016). Faculty and second‐year medical student perceptions of active learning in an integrated curriculum. Advances in Physiological Education, 40 (4), 446 – 453. https://doi.org/10.1152/advan.00079.2016</bibtext> </blist> <blist> <bibtext> Velicer, W. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41 (3), 321 – 327. https://doi.org/10.1007/BF02293557</bibtext> </blist> <blist> <bibtext> Velicer, W. F., Eaton, C. A., & Fava, J. L. (2000). Construct explication through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factor or components. In R. D. Goffin & E. Helmes (Eds.), Problems and solutions in human assessment: Honoring Douglas N. Jackson at seventy (pp. 41 – 71). Kluwer Academic Publishers.</bibtext> </blist> <blist> <bibtext> Viskupic, K., Ryker, K., Teasdale, R., Manduca, C., Iverson, E., Farthing, D., Bruckner, M. Z., & McFadden, R. (2019). Classroom observations indicate the positive impacts of discipline‐based professional development. Journal for STEM Education Research, 2 (2), 201 – 228. https://doi.org/10.1007/s41979-019-00015-w</bibtext> </blist> <blist> <bibtext> Watkins, M. W. (2022). A step‐by‐step guide to exploratory factor analysis with Stata. Routledge.</bibtext> </blist> <blist> <bibtext> Zibrowski, E. M., Weston, W. W., & Goldszmidt, M. A. (2008). 'I don't have time': Issues of fragmentation, prioritisation and motivation for education scholarship among medical faculty. Medical Education, 42 (9), 872 – 878. https://doi.org/10.1111/j.1365-2923.2008.03145.x</bibtext> </blist> </ref> <aug> <p>By Laura J. Carroll; David Reeping; Cynthia J. Finelli; Michael J. Prince; Jenefer Husman; Matthew Graham and Maura J. Borrego</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author</p> <p></p> <p>Laura J. Carroll is Research Fellow in Engineering Education Research (EER) at the University of Michigan, Ann Arbor, MI 48109, USA;.</p> <p>David Reeping is Assistant Professor in Engineering and Computing Education at the University of Cincinnati, Cincinnati, OH 45221, USA;.</p> <p>Cynthia J. Finelli is Professor of Electrical Engineering & Computer Science, Professor of Higher Education, and Director of Engineering Education Research at the University of Michigan, Ann Arbor, MI 48109, USA;.</p> <p>Michael J. Prince is Professor of Chemical Engineering at Bucknell University, Lewisburg, PA 17837, USA;.</p> <p>Jenefer Husman is Professor in Education Studies at the University of Oregon, University of Oregon, Eugene, OR 97403, USA;.</p> <p>Matthew Graham is Research Associate in the Oregon Education Science Laboratory at the University of Oregon, Eugene, OR 96403, USA;.</p> <p>Maura J. Borrego is Director of the Center for Engineering Education and Professor of Mechanical Engineering and STEM Education at the University of Texas at Austin, Austin, TX 78712, USA;.</p> </aug> <nolink nlid="nl1" bibid="bib60" firstref="ref5"></nolink> <nolink nlid="nl2" bibid="bib67" firstref="ref6"></nolink> <nolink nlid="nl3" bibid="bib84" firstref="ref7"></nolink> <nolink nlid="nl4" bibid="bib47" firstref="ref9"></nolink> <nolink nlid="nl5" bibid="bib24" firstref="ref10"></nolink> <nolink nlid="nl6" bibid="bib49" firstref="ref12"></nolink> <nolink nlid="nl7" bibid="bib48" firstref="ref13"></nolink> <nolink nlid="nl8" bibid="bib70" firstref="ref15"></nolink> <nolink nlid="nl9" bibid="bib27" firstref="ref16"></nolink> <nolink nlid="nl10" bibid="bib58" firstref="ref19"></nolink> <nolink nlid="nl11" bibid="bib16" firstref="ref23"></nolink> <nolink nlid="nl12" bibid="bib51" firstref="ref25"></nolink> <nolink nlid="nl13" bibid="bib80" firstref="ref26"></nolink> <nolink nlid="nl14" bibid="bib90" firstref="ref27"></nolink> <nolink nlid="nl15" bibid="bib77" firstref="ref29"></nolink> <nolink nlid="nl16" bibid="bib17" firstref="ref32"></nolink> <nolink nlid="nl17" bibid="bib30" firstref="ref34"></nolink> <nolink nlid="nl18" bibid="bib31" firstref="ref35"></nolink> <nolink nlid="nl19" bibid="bib32" firstref="ref36"></nolink> <nolink nlid="nl20" bibid="bib55" firstref="ref37"></nolink> <nolink nlid="nl21" bibid="bib56" firstref="ref38"></nolink> <nolink nlid="nl22" bibid="bib83" firstref="ref40"></nolink> <nolink nlid="nl23" bibid="bib87" firstref="ref41"></nolink> <nolink nlid="nl24" bibid="bib91" firstref="ref58"></nolink> <nolink nlid="nl25" bibid="bib86" firstref="ref63"></nolink> <nolink nlid="nl26" bibid="bib52" firstref="ref64"></nolink> <nolink nlid="nl27" bibid="bib46" firstref="ref74"></nolink> <nolink nlid="nl28" bibid="bib21" firstref="ref84"></nolink> <nolink nlid="nl29" bibid="bib71" firstref="ref89"></nolink> <nolink nlid="nl30" bibid="bib12" firstref="ref99"></nolink> <nolink nlid="nl31" bibid="bib92" firstref="ref102"></nolink> <nolink nlid="nl32" bibid="bib54" firstref="ref129"></nolink> <nolink nlid="nl33" bibid="bib18" firstref="ref132"></nolink> <nolink nlid="nl34" bibid="bib19" firstref="ref135"></nolink> <nolink nlid="nl35" bibid="bib44" firstref="ref136"></nolink> <nolink nlid="nl36" bibid="bib68" firstref="ref137"></nolink> <nolink nlid="nl37" bibid="bib76" firstref="ref139"></nolink> <nolink nlid="nl38" bibid="bib73" firstref="ref140"></nolink> <nolink nlid="nl39" bibid="bib81" firstref="ref141"></nolink> <nolink nlid="nl40" bibid="bib82" firstref="ref142"></nolink> <nolink nlid="nl41" bibid="bib43" firstref="ref143"></nolink> <nolink nlid="nl42" bibid="bib15" firstref="ref145"></nolink> <nolink nlid="nl43" bibid="bib57" firstref="ref147"></nolink> <nolink nlid="nl44" bibid="bib25" firstref="ref148"></nolink> <nolink nlid="nl45" bibid="bib41" firstref="ref149"></nolink> <nolink nlid="nl46" bibid="bib36" firstref="ref151"></nolink> <nolink nlid="nl47" bibid="bib22" firstref="ref152"></nolink> <nolink nlid="nl48" bibid="bib88" firstref="ref153"></nolink> <nolink nlid="nl49" bibid="bib78" firstref="ref154"></nolink> <nolink nlid="nl50" bibid="bib28" firstref="ref156"></nolink> <nolink nlid="nl51" bibid="bib89" firstref="ref158"></nolink> <nolink nlid="nl52" bibid="bib23" firstref="ref159"></nolink> <nolink nlid="nl53" bibid="bib53" firstref="ref160"></nolink> <nolink nlid="nl54" bibid="bib33" firstref="ref163"></nolink> <nolink nlid="nl55" bibid="bib34" firstref="ref164"></nolink> <nolink nlid="nl56" bibid="bib37" firstref="ref165"></nolink> <nolink nlid="nl57" bibid="bib13" firstref="ref166"></nolink> <nolink nlid="nl58" bibid="bib63" firstref="ref171"></nolink> <nolink nlid="nl59" bibid="bib990" firstref="ref173"></nolink> <nolink nlid="nl60" bibid="bib14" firstref="ref180"></nolink> <nolink nlid="nl61" bibid="bib62" firstref="ref181"></nolink> <nolink nlid="nl62" bibid="bib26" firstref="ref182"></nolink> <nolink nlid="nl63" bibid="bib61" firstref="ref184"></nolink> <nolink nlid="nl64" bibid="bib64" firstref="ref185"></nolink> <nolink nlid="nl65" bibid="bib65" firstref="ref186"></nolink> <nolink nlid="nl66" bibid="bib66" firstref="ref187"></nolink> <nolink nlid="nl67" bibid="bib20" firstref="ref188"></nolink> <nolink nlid="nl68" bibid="bib29" firstref="ref189"></nolink> <nolink nlid="nl69" bibid="bib69" firstref="ref191"></nolink> <nolink nlid="nl70" bibid="bib72" firstref="ref192"></nolink> <nolink nlid="nl71" bibid="bib42" firstref="ref193"></nolink> <nolink nlid="nl72" bibid="bib39" firstref="ref194"></nolink> <nolink nlid="nl73" bibid="bib45" firstref="ref196"></nolink> <nolink nlid="nl74" bibid="bib40" firstref="ref198"></nolink> <nolink nlid="nl75" bibid="bib101" firstref="ref199"></nolink> <nolink nlid="nl76" bibid="bib156" firstref="ref201"></nolink> <nolink nlid="nl77" bibid="bib10" firstref="ref204"></nolink> <nolink nlid="nl78" bibid="bib38" firstref="ref205"></nolink> <nolink nlid="nl79" bibid="bib50" firstref="ref207"></nolink> <nolink nlid="nl80" bibid="bib85" firstref="ref209"></nolink> <nolink nlid="nl81" bibid="bib74" firstref="ref210"></nolink> <nolink nlid="nl82" bibid="bib113" firstref="ref212"></nolink> <nolink nlid="nl83" bibid="bib112" firstref="ref215"></nolink> <nolink nlid="nl84" bibid="bib59" firstref="ref235"></nolink> <nolink nlid="nl85" bibid="bib75" firstref="ref237"></nolink> <nolink nlid="nl86" bibid="bib79" firstref="ref238"></nolink> <nolink nlid="nl87" bibid="bib11" firstref="ref239"></nolink> <nolink nlid="nl88" bibid="bib35" firstref="ref243"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1396511
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Barriers Instructors Experience in Adopting Active Learning: Instrument Development
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Carroll%2C+Laura+J%2E%22">Carroll, Laura J.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-4426-9979">0000-0003-4426-9979</externalLink>)<br /><searchLink fieldCode="AR" term="%22Reeping%2C+David%22">Reeping, David</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-0803-7532">0000-0002-0803-7532</externalLink>)<br /><searchLink fieldCode="AR" term="%22Finelli%2C+Cynthia+J%2E%22">Finelli, Cynthia J.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-9148-1492">0000-0001-9148-1492</externalLink>)<br /><searchLink fieldCode="AR" term="%22Prince%2C+Michael+J%2E%22">Prince, Michael J.</searchLink><br /><searchLink fieldCode="AR" term="%22Husman%2C+Jenefer%22">Husman, Jenefer</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-9886-5966">0000-0001-9886-5966</externalLink>)<br /><searchLink fieldCode="AR" term="%22Graham%2C+Matthew%22">Graham, Matthew</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-0235-5126">0000-0002-0235-5126</externalLink>)<br /><searchLink fieldCode="AR" term="%22Borrego%2C+Maura+J%2E%22">Borrego, Maura J.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-7131-4611">0000-0001-7131-4611</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Engineering+Education%22"><i>Journal of Engineering Education</i></searchLink>. 2023 112(4):1079-1108.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 30
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: National Science Foundation (NSF), Division of Undergraduate Education (DUE)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: 1821092<br />1821036<br />1821488<br />1821277
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Active+Learning%22">Active Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Engineering+Education%22">Engineering Education</searchLink><br /><searchLink fieldCode="DE" term="%22Barriers%22">Barriers</searchLink><br /><searchLink fieldCode="DE" term="%22Teaching+Methods%22">Teaching Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Construction%22">Test Construction</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Attitudes%22">Teacher Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Time+Factors+%28Learning%29%22">Time Factors (Learning)</searchLink><br /><searchLink fieldCode="DE" term="%22Readiness%22">Readiness</searchLink><br /><searchLink fieldCode="DE" term="%22Learner+Engagement%22">Learner Engagement</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Role%22">Teacher Role</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Environment%22">Educational Environment</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22STEM+Education%22">STEM Education</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1002/jee.20557
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1069-4730<br />2168-9830
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: Despite well-documented benefits, instructor adoption of active learning has been limited in engineering education. Studies have identified barriers to instructors' adoption of active learning, but there is no well-tested instrument to measure instructors perceptions of these barriers. Purpose: We developed and tested an instrument to measure instructors' perceptions of barriers to adopting active learning and identify the constructs that coherently categorize those barriers. Method: We used a five-phase process to develop an instrument to measure instructors' perceived barriers to adopting active learning. In Phase 1, we built upon the Faculty Instructional Barriers and Identity Survey (FIBIS) to create a draft instrument. In Phases 2 and 3, we conducted exploratory factor analysis (EFA) on an initial 45-item instrument and a refined 21-item instrument, respectively. We conducted confirmatory factor analysis (CFA) in Phases 4 and 5 to test the factor structure identified in Phases 2 and 3. Results: Our final instrument consists of 17 items and four factors: (1) student preparation and engagement; (2) instructional support; (3) instructor comfort and confidence; and (4) institutional environment/rewards. Instructor responses indicated that time considerations do not emerge as a standalone factor. Conclusions: Our 17-item instrument exhibits a sound factor structure and is reliable, enabling the assessment of perceived barriers to adopting active learning in different contexts. The four factors align with an existing model of instructional change in science, technology, engineering, and mathematics (STEM). Although time is a substantial instructor concern that did not comprise a standalone factor, it is closely related to multiple constructs in our final model.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1396511
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1396511
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1002/jee.20557
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 30
        StartPage: 1079
    Subjects:
      – SubjectFull: Active Learning
        Type: general
      – SubjectFull: Engineering Education
        Type: general
      – SubjectFull: Barriers
        Type: general
      – SubjectFull: Teaching Methods
        Type: general
      – SubjectFull: Test Construction
        Type: general
      – SubjectFull: Teacher Attitudes
        Type: general
      – SubjectFull: Time Factors (Learning)
        Type: general
      – SubjectFull: Readiness
        Type: general
      – SubjectFull: Learner Engagement
        Type: general
      – SubjectFull: Teacher Role
        Type: general
      – SubjectFull: Educational Environment
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: STEM Education
        Type: general
    Titles:
      – TitleFull: Barriers Instructors Experience in Adopting Active Learning: Instrument Development
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Carroll, Laura J.
      – PersonEntity:
          Name:
            NameFull: Reeping, David
      – PersonEntity:
          Name:
            NameFull: Finelli, Cynthia J.
      – PersonEntity:
          Name:
            NameFull: Prince, Michael J.
      – PersonEntity:
          Name:
            NameFull: Husman, Jenefer
      – PersonEntity:
          Name:
            NameFull: Graham, Matthew
      – PersonEntity:
          Name:
            NameFull: Borrego, Maura J.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2023
          Identifiers:
            – Type: issn-print
              Value: 1069-4730
            – Type: issn-electronic
              Value: 2168-9830
          Numbering:
            – Type: volume
              Value: 112
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Journal of Engineering Education
              Type: main
ResultId 1