View in EDS HTML Full Text PDF Full Text

Engineering Design Pedagogy: A Performance Analysis

Saved in:
Bibliographic Details
Title: Engineering Design Pedagogy: A Performance Analysis
Language: English
Authors: Emami, M. Reza (ORCID 0000-0003-4977-6339), Bazzocchi, Michael C. F., Hakima, Houman
Source: International Journal of Technology and Design Education. Jul 2020 30(3):553-585.
Availability: Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 33
Publication Date: 2020
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Undergraduate Students, Engineering Education, Design, Personality Measures, Personality Traits, Cognitive Style, Gender Differences, Academic Achievement, Predictor Variables, Instructional Design
Assessment and Survey Identifiers: Myers Briggs Type Indicator
DOI: 10.1007/s10798-019-09515-7
ISSN: 0957-7572
Abstract: Cornerstone design courses have become a major part of engineering curricula, where students with different personality types and learning styles work together to design, develop, build, and demonstrate the functionality of a prototype within the duration of a term. This study analyzes student and team performance against gender, personality types, and learning styles in a second-year engineering design course. Further, the correlations between several assessment mechanisms are studied, and the effects of three different instructional design approaches on students' performance are explored. Data have been collected on student performance and psychometrics, including marks, gender, personality type, and learning style from 2001 to 2018. To identify students' personality types and learning styles, Myers-Briggs Type Indicators (MBTI) and Neil Fleming's Learning VARK tests were administered. To evaluate students' performance in the course, a number of assessment mechanisms have been defined. Several statistical methods are used to analyze data, and to determine correlation between datasets. Over nearly two decades of marks, gender, MBTI, and VARK data for 2637 students are presented for an engineering design course. The results demonstrated that there was no significant difference in performance across most assessments based on gender or gender distribution on a team. A better performance was observed from VK bimodal and quadmodal learning styles in most assessment mechanisms. Further, certain MBTI groups, namely, judging types outperformed their peers in engineering design assessments, with interesting interplay between MBTI dimensions for specific assessments and team dynamics. Traditional assessment mechanisms, such as engineering notebook and design proposals, are shown to be good predictors of student success. Lastly, scaffolded design activities and front-loading of lecture content were shown to be beneficial for student learning. There is negligible performance difference between female and male students in the engineering design course. Students whose preferred learning styles align with the assessment themes showed better performance in the course. The outcomes of this paper can be readily applied by instructors for design of assessment mechanisms, course materials, team formation, and instructional design.
Abstractor: As Provided
Entry Date: 2020
Accession Number: EJ1261104
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHESp_rkbcoV3F5-ScGW7SEAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDN4fHNE9r7kP7w95JQIBEICBm8t8VfBB01mAEfpl4vsZYwZydVcgJUi66RqdHfCWQasw_JsZuqN5yg6ExfNCjBlLpDCk0CrdAvmeqFLknMBxn8uCVTmfSuzpFK7aOW7J0dirKR1q2ty0TsxmOdZCbTxIAwfWJd6-N-O-_pTwbs1jcNdMJVJipk8HW-sPjcRo44mvHTgu41kIbgwFHBxcXEHkNzzD5kcgizVu5t1V
Text:
  Availability: 1
  Value: <anid>AN0144641886;ogv01jul.20;2020Jul20.07:25;v2.2.500</anid> <title id="AN0144641886-1">Engineering design pedagogy: a performance analysis </title> <p>Cornerstone design courses have become a major part of engineering curricula, where students with different personality types and learning styles work together to design, develop, build, and demonstrate the functionality of a prototype within the duration of a term. This study analyzes student and team performance against gender, personality types, and learning styles in a second-year engineering design course. Further, the correlations between several assessment mechanisms are studied, and the effects of three different instructional design approaches on students' performance are explored. Data have been collected on student performance and psychometrics, including marks, gender, personality type, and learning style from 2001 to 2018. To identify students' personality types and learning styles, Myers–Briggs Type Indicators (MBTI) and Neil Fleming's Learning VARK tests were administered. To evaluate students' performance in the course, a number of assessment mechanisms have been defined. Several statistical methods are used to analyze data, and to determine correlation between datasets. Over nearly two decades of marks, gender, MBTI, and VARK data for 2637 students are presented for an engineering design course. The results demonstrated that there was no significant difference in performance across most assessments based on gender or gender distribution on a team. A better performance was observed from VK bimodal and quadmodal learning styles in most assessment mechanisms. Further, certain MBTI groups, namely, judging types outperformed their peers in engineering design assessments, with interesting interplay between MBTI dimensions for specific assessments and team dynamics. Traditional assessment mechanisms, such as engineering notebook and design proposals, are shown to be good predictors of student success. Lastly, scaffolded design activities and front-loading of lecture content were shown to be beneficial for student learning. There is negligible performance difference between female and male students in the engineering design course. Students whose preferred learning styles align with the assessment themes showed better performance in the course. The outcomes of this paper can be readily applied by instructors for design of assessment mechanisms, course materials, team formation, and instructional design.</p> <p>Keywords: Cornerstone design course; Assessment mechanisms; Instructional design; Personality types; Learning styles; Engineering design</p> <hd id="AN0144641886-2">Introduction</hd> <p>A notable transformation in engineering design education has been underway over the past two decades, transitioning from the traditional behaviourist approach to more constructivist, hands-on, and project-based courses. The behaviourist approach views design as a by-product of engineering education, instructors conveying their knowledge of design to students through traditional means (Dym [<reflink idref="bib15" id="ref1">15</reflink>]), whereas the new trend argues that such knowledge transmission, if possible at all, does not adequately prepare students for tackling real-life engineering problems. Further, design can be viewed as a means of learning engineering in the constructivist approach, not just a result of it. Following these arguments, hands-on, project-based courses have proven to be useful in addressing the critical feedback from industry, that they perceive graduating engineers as unable to manage real-life professional design projects, due to the focus on the theoretical aspects of engineering in university curricula (Dutson et al. [<reflink idref="bib14" id="ref2">14</reflink>]; Farr et al. [<reflink idref="bib19" id="ref3">19</reflink>]). Introduction of cornerstone design courses in the first and second years has indicated that such courses enhance students' motivation, their retention in engineering programs, and their performance in senior engineering science and capstone design courses (Adams et al. [<reflink idref="bib1" id="ref4">1</reflink>]; Bright and Dym [<reflink idref="bib6" id="ref5">6</reflink>]).</p> <p>A previous work (Emami [<reflink idref="bib16" id="ref6">16</reflink>]) details the creation of a hybrid instructional design framework, based on Reigeluth's Elaboration Theory (Reigeluth [<reflink idref="bib53" id="ref7">53</reflink>]), merging behaviourist and constructivist learning models for teaching a second-year cornerstone engineering design course. The course was implemented at the University of Toronto as part of the Engineering Science program, and students' performance was measured from 2001 to 2018. In particular, various data were accumulated regarding the learning styles, personality types, gender, and academic performance for 2637 students who partook in the course. Additionally, the course structure, while largely the same throughout the past 18 years, was adjusted with respect to two major factors: (<reflink idref="bib1" id="ref8">1</reflink>) timeframe, i.e., full-year to half-year, and (<reflink idref="bib2" id="ref9">2</reflink>) instructional design, i.e., distributing lecture content over the course of the term to front-loading of the lectures. Moreover, there were several assessment mechanisms that remained nearly identical over the time period the course has been offered, providing a solid baseline for comparison between student characteristics, performance predictors, and instructional design models. It is the intent of this paper to analyze student characteristics, such as gender, learning style, and personality type, with regards to different types of assessment mechanisms used in engineering design, e.g., written versus presentation, individual versus team, process versus outcome. The paper further investigates the use of specific assessment mechanisms, such as engineering notebook and design proposals, as performance indicators in engineering design. Lastly, the instructional design models used in the course are compared and contrasted in light of behaviourist and constructivist pedagogies. The subsequent sections provide a background of the developed engineering design course, the relevant theory in psychometrics and educational models, as well as the details of the statistical tools employed. The research questions are then outlined more precisely, followed by a discussion of the results. Lastly, some concluding remarks and overarching observations are presented.</p> <hd id="AN0144641886-3">Engineering design course structure</hd> <p>The Division of Engineering Science at the University of Toronto offers a second-year, hands-on and project-based design course called <emph>Aer201</emph>—<emph>Engineering Design</emph>. The course is part of the mandatory curriculum for Engineering Science students, which addresses open-ended and multidisciplinary design problems, and requires students to conceptualize, design, analyze, and fabricate a proof-of-concept prototype of a machine that performs certain tasks autonomously. The core activity of the course is the design project. Each year there are three different projects that have the same theme and are largely at the same level of complexity. Students are divided into three sections to work on one of the projects. Despite the fact that the projects have a common theme, they tackle different engineering problems. Some examples of the past projects include: (<reflink idref="bib1" id="ref10">1</reflink>) sorting machines that group objects based on shape, colour, size, etc.; (<reflink idref="bib2" id="ref11">2</reflink>) machines that perform quality control testing of products; and (<reflink idref="bib3" id="ref12">3</reflink>) mobile robots that manipulate objects while moving among other objects. (See Emami [<reflink idref="bib16" id="ref13">16</reflink>] for more details.) The rationale for three projects with a common theme is to increase the design space and encourage creativity. From year to year the projects have a similar level of difficulty and set of expectations, which allows for a unified set of educational materials as well as evaluation schemes. To develop new course projects each year, the consumer market is consulted, and industry experts are sought for advice. The projects are such that multidisciplinary student teams are required. Students form teams of three, consisting of an electromechanical member (structure, mechanisms, and actuators), an electrical member (circuits and instrumentation), and microcontroller member (programming and control). Each project is provided to students through a formal request for proposal (RFP) to give them an authentic sense of professionalism, responsibility, and importance. Projects include major design constraints, such as weight, dimension, and cost of prototype, as well as other project-specific constraints. During the last week of the course, a public demonstration is held where teams compete with their peers in their section. The developed machines are required to perform specific tasks, and are assessed for additional functionalities according to the scoring metrics outlined in the RFP. Aside from the physical prototype, there are several other outcomes of the design projects that will be subjected to evaluation, which are outlined in the "Assessment Mechanisms" section.</p> <hd id="AN0144641886-4">Lecture content</hd> <p>To begin, the lectures are divided into two main types, namely, design lectures and technical lectures. A series of design lectures are presented to students, which include some practical and essential notions of engineering design. They are intended to provide guidance to students as they start their design projects, and to familiarize them with topics such as learning strategies (why, what, and how to learn), project management (group dynamics, teamwork, project planning, etc.), engineering fundamentals and conceptual design, communication and information (writing proposals, technical reports, and engineering notebooks), decision making and risk assessment, and project definition (the three design projects of the year). Overall, there are about 18 h of design lectures that students must attend.</p> <p>In addition to the design lectures, there are a series of lectures that aim to improve students' practical knowledge about different disciplines. As stated earlier, projects are such that they require multidisciplinary teams of students. Hence, to design and build a functional prototype, students need to possess certain practical knowledge. Accordingly, a number of topics are discussed in the technical lectures, including digital and analogue electronics, sensors, and circuits, electromechanical systems, and microcontrollers. There are in total 30 h of technical lectures, with 10 h of content for each subsystem. These lectures connect students' theoretical <emph>know</emph>-<emph>what</emph> (obtained in classroom) to the practical <emph>know</emph>-<emph>how</emph> that they need to experience and construct throughout the design process. As part of the technical lectures, students build basic mechanical systems and electrical circuits, using the components that are provided to them in their project kits.</p> <hd id="AN0144641886-5">Assessment mechanisms</hd> <p>Students are assessed at different stages of their prototype development in the course. The classification of assessment mechanisms is twofold (see Table 1). On the one hand, assessments can be either individual-based or team-based. On the other, they can be outcome-based or process-based. Individual-based assessments consist of two performance evaluations in weeks 5 and 8 as well as notebook evaluations in the middle and at the end of the semester. Each performance evaluation is divided into two equally weighed parts to assess the progress of developing the assigned subsystem (outcome-based) and to examine the student's conduct and learning pace (process-based). Team-based assessments also include three performance evaluations in weeks 10, 12 and 14, equally making outcome-and process-based evaluations, in addition to the proposal and final report, both of which are outcome-based assessments. The assessments are performed by experienced teaching assistants, who constantly monitor students' performance based on specific metrics and provide them with feedback at each stage. The instructor supervises the process and makes sure that the assessments are performed in a unified and unbiased manner, detailing the criteria for evaluations sufficiently to ensure objective assessments.</p> <p>AER201 course assessment matrix with percentage weight of each assessment in brackets</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left"><p>Individual-based</p></th><th align="left"><p>Team-based</p></th></tr></thead><tbody><tr><td align="left" rowspan="5"><p>Outcome-based</p></td><td align="left" rowspan="3"><p>Week-5 sub-system integration—part I (5%)</p></td><td align="left"><p>Design proposal (10%)</p></td></tr><tr><td align="left"><p>Week-10 system integration—part I (3.75%)</p></td></tr><tr><td align="left"><p>Week-12 system functionality—part I (6.25%)</p></td></tr><tr><td align="left" rowspan="2"><p>Week-8 sub-system Functionality—part I</p><p>(6.25%)</p></td><td align="left"><p>Week 14 project demonstration—part I (6.25%)</p></td></tr><tr><td align="left"><p>Final report (20%)</p></td></tr><tr><td align="left" rowspan="5"><p>Process-based</p></td><td align="left" rowspan="2"><p>Week-5 sub-system integration—part II (5%)</p></td><td align="left"><p>Week-10 system integration—part II (3.75%)</p></td></tr><tr><td align="left" rowspan="2"><p>Week-12 system functionality—part II (6.25%)</p></td></tr><tr><td align="left"><p>Interim notebook (7.5%)</p></td></tr><tr><td align="left"><p>Week-8 sub-system functionality—part II (6.25%)</p></td><td align="left" rowspan="2"><p>Week-14 project demonstration—part II (6.25%)</p></td></tr><tr><td align="left"><p>Final notebook (7.5%)</p></td></tr></tbody></table> </ephtml> </p> <p>Certain milestones are expected to be accomplished by each performance evaluation. In week 5, each student is required to have completed the integration of their subsystem, whereas in week 8 complete functionality of the subsystem is expected. In week 10 and week 12 evaluations, teams are assessed for their progress on the integration and debugging of their prototype, respectively. In addition, these evaluations are equally attributed to the learning progress of each student and/or team, in terms of metrics such as communication, innovation, acquisition of new skills, problem-solving abilities, contribution to the team, safe conduct, etc. The interim and final notebook evaluations are assessments of the students' documentation of the design process in their engineering notebook. The notebooks are marked based on specific criteria, such as organization and technical content. During project demonstration in week 14, teams are assessed based on their presentation of their prototype and their ability to answer questions regarding their machine and their design process. The proposal is a comprehensive written response to the RFP outlining the team's machine, design process, project management, and how their solution satisfies the requirements, constraints, and criteria of the project. The final report is a technical document detailing the design, fabrication, integration, and testing of the team's solution to the RFP, as well as their design process throughout the course.</p> <hd id="AN0144641886-6">Course structure</hd> <p>The structure of the course has been detailed in the previous work (Emami [<reflink idref="bib16" id="ref14">16</reflink>]). However, notable modifications were made to the course structure in 2007 and in 2016. These changes, while remaining true to the original course design, provide interesting variations to the instructional design that can be contrasted in a meaningful way through considering the outcome of student performance. The course structure can be divided into three approaches, namely, full-year (prior to 2007), half-year (between 2007 and 2014), and front-loaded half-year (since 2016).</p> <p>The full-year course is the original course design that was described in Emami ([<reflink idref="bib16" id="ref15">16</reflink>]), and was in practice from 2001 to 2006. This approach incorporated all of the elements discussed in the previous sections, with the design project concentrated in the second term. The course lectures were primarily in the first term. Further, the course integrated supplemental written assignments, quizzes, and laboratory experiments to solidify student learning. There was a transition of the course from full-year format to half-year format in 2007. As a result, the supplemental written assignments, quizzes, and laboratory experiments were removed. The same course materials from the full-year format were still presented throughout the length of the half-year course, but with a higher concentration of lectures in the first weeks. It is worth mentioning that between 2001 and 2014, students attended all technical lectures. Hence, they had to attend lectures for the most part of the course while working on their projects. However, since 2016 all lectures are delivered in the first 3 weeks of the course, i.e., front-loading the lectures. While all students receive the design lectures, each student is required to attend only the technical lectures that are relevant to their subsystem. That is, the electrical member attends the electronics lectures, the electromechanical member attends the electromechanical lectures, and the microcontroller member attends the microcontroller lectures. The technical lectures occur in parallel, and students are able to watch recordings of all lectures online. As a result, all lectures (technical and design) are concluded by the end of third week of the course, allowing students to focus solely on their projects for the remainder of the course (i.e., 11 weeks). This structure compressed the timeline for lecture delivery and front-loaded lecture content while maintaining the same content from previous years.</p> <hd id="AN0144641886-7">Engineering design and learning models</hd> <p>There has been a movement in engineering education towards project-based learning. This particular approach is found most extensively in engineering design courses, and represents a shift from the traditional behaviourist learning model to a more constructivist approach. This is not to say that behaviourism has no place in engineering courses, but that this shift is a natural transition for design education that often emphasizes the concrete application of abstract notions. In hands-on learning approaches the student is tasked with applying abstract notions to a real design problem, both captivating their interest and solidifying their learning. This approach can be best described as constructivist, whereby students create or generate knowledge through their own experience, rather than having the knowledge conveyed or transmitted to them (Bruner [<reflink idref="bib7" id="ref16">7</reflink>]; Piaget [<reflink idref="bib50" id="ref17">50</reflink>]; Vygotsky [<reflink idref="bib65" id="ref18">65</reflink>]). Constructivism views knowledge not just as an awareness of an object's or process' existence, but also as that knowledge is a product of a learner's constructed sensory and social experience of the world, both subjectively and dynamically (von Glasersfeld [<reflink idref="bib64" id="ref19">64</reflink>]). Thus, students learn through the experiences they create, while solving engineering design problems. Meanwhile, the role of the educator is to facilitate and mediate the learning process, motivating students and providing them with opportunities to construct knowledge. This is contrasted against the more traditional model of learning, i.e., behaviourism, whereby knowledge is obtained by the learner through response to external stimuli (Skinner [<reflink idref="bib61" id="ref20">61</reflink>]; Jonassen [<reflink idref="bib31" id="ref21">31</reflink>]; von Glasersfeld [<reflink idref="bib63" id="ref22">63</reflink>]). Behaviourism perceives knowledge as something that is conveyed to the learner through abstract representation, and learning as the process of conditioning behaviour and reinforcing specific responses to observable stimuli or events. In the same vein, teaching is viewed as the task of an expert transferring knowledge to the learner (Jonassen [<reflink idref="bib32" id="ref23">32</reflink>]). Behaviourism is more dominant in engineering education and, in many ways, is considered the traditional form of teaching. In fact, some may even argue that it is the preferred and more practical approach for teaching abstract theories or procedural concepts in engineering. Through the dissemination of knowledge in a linear and gradual form, instructors are able to present topics as more generalized representations of reality, both in classroom and in laboratory. Learning is evaluated by assessing a student's behaviour and its correspondence with the expected response or outcome.</p> <p>Through careful attention to proper instructional design, these two learning theories can be uniquely expressed and realized. In the case of constructivism, the primary focus of the instructional design is on the design of the environment that best facilitates learning and not on the content or order of instruction. This is a direct result of the notion that knowledge is constructed by the learner; hence, the sequence and content of learning is determined by the student, and their environment is designed to support such knowledge construction. Along the same line, assessment must be more subjective, since it depends heavily on the learner's process, reflection, and self-evaluation. In contrast, in a behaviourist approach, the instructional design focuses on the strategic dissemination of knowledge through the proper ordering of subjects to achieve specific and measurable learning objectives. The determination of the most suitable model is heavily dependent on the learning context and subject matter. It is important to note that both the constructivist and behaviourist learning models, as well as their resultant instructional design, are suitable depending on the circumstances of the learning situation, most importantly in terms of the content and level of learning (Jonassen [<reflink idref="bib31" id="ref24">31</reflink>]; Ertmer [<reflink idref="bib44" id="ref25">44</reflink>]; Schwier [<reflink idref="bib57" id="ref26">57</reflink>]). Specifically, the behaviourist model appears to be most effective for basic, prescriptive knowledge at an introductory learning level when the learner has little or no prior knowledge of the content area, since it is predetermined, sequential and structured, so that the learner can develop some anchors for their future knowledge construction. Such learning circumstance can be characterized as <emph>know</emph>-<emph>what</emph> knowledge acquisition. On the other hand, the constructivist model can work more effectively for the <emph>know</emph>-<emph>how</emph> knowledge construction in the advanced learning stages, where the learner is able to take ownership of their own learning and gain metacognition of the content area.</p> <p>The knowledge that should be acquired through an engineering design learning experience depends on the qualities expected in a design engineer. Such qualities have been originally discussed in Sheppard and Jenison ([<reflink idref="bib59" id="ref27">59</reflink>]), and later expanded upon in Emami ([<reflink idref="bib16" id="ref28">16</reflink>]) where it is also argued that such expected knowledge is a combination of <emph>know</emph>-<emph>what</emph> and <emph>know</emph>-<emph>how</emph>, and hence the instruction method should utilize both behaviourist and constructivist models coherently. A proper approach to this coalescence is through an instructional design theory called Elaboration Theory (Reigeluth [<reflink idref="bib53" id="ref29">53</reflink>]), which systematically organizes the instruction in increasing order of complexity, moving from the simplest representations of the basic learning content (<emph>know</emph>-<emph>what</emph>) through prescriptive techniques, e.g., direct instruction, toward gradually enabling the learner to take the ownership of the content (and learning) and succeed levels of elaboration by relaxing the simplifying conditions, i.e., more realistic circumstances, so that the tasks relate to more and more complex knowledge (<emph>know</emph>-<emph>how</emph>). Therefore, Elaboration Theory appears to provide a suitable mechanism for teaching subjects that require both behaviourist and constructivist models, such as engineering design. More details on the utilization of Elaboration Theory for the engineering design course are discussed in Emami ([<reflink idref="bib16" id="ref30">16</reflink>]). Further synthesis can be achieved to best enhance the engineering design learning experience by applying certain techniques from each model, which are most apt for the materials to be learned.</p> <hd id="AN0144641886-8">Learning styles and personality types</hd> <p>Educational researchers postulate that every learner has a certain learning style and personality type, and if instruction is adapted to the learner's individual characteristics, enhanced learning outcomes should be expected. Numerous psychometric tests and models have been developed to gain insights into individual's learning style, personality, and behaviour, with the ultimate goal of determining the most effective learning process pertaining to the individual. The outcome of such examinations have been used in the field of engineering pedagogy to improve creativity (Kassim [<reflink idref="bib35" id="ref31">35</reflink>]), create more effective design teams (Brickell et al. [<reflink idref="bib5" id="ref32">5</reflink>]; Jensen et al. [<reflink idref="bib29" id="ref33">29</reflink>], [<reflink idref="bib30" id="ref34">30</reflink>]), to aid students in their use of web-based classes and electronic learning (e-learning) (Manochehr [<reflink idref="bib39" id="ref35">39</reflink>]; Aragon et al. [<reflink idref="bib2" id="ref36">2</reflink>]), and to tailor the learning environment to meet students' differing preferences (Peyman et al. [<reflink idref="bib49" id="ref37">49</reflink>]). Further, such indicators can provide employers with proper information about the skills their prospective employees may possess (Jenkins [<reflink idref="bib28" id="ref38">28</reflink>]).</p> <p>An individual's learning style can be defined as "natural or habitual pattern of acquiring and processing information in learning situations" (James and Gardner [<reflink idref="bib27" id="ref39">27</reflink>]). Some educators argue that individuals will benefit greatly if they can identify their learning preferences. Knowing student's learning styles can help in many ways to enhance teaching and learning. Teachers can benefit by knowing about how their students learn, which can support the preparation and explanation of the course materials. Making students aware of their learning styles and showing them their individual strengths and weaknesses can also help students understand why learning is difficult at times, and hence can be a basis for improving their weaknesses. In addition, students can be further supported by adapting the teaching method to their learning style. Providing students with learning materials and activities that resonate with their preferred ways of learning can make their learning experience more pleasant. This adaptation hypothesis is supported by a number of educational theories, as stated and described in Coffield et al. ([<reflink idref="bib9" id="ref40">9</reflink>]), Bajraktarevic et al. ([<reflink idref="bib3" id="ref41">3</reflink>]) and Graf and Kinshuk ([<reflink idref="bib36" id="ref42">36</reflink>]). One method of discovering an individual's learning style is to uncover their dominance in one or more of the four sensory modalities of Visual, Aural, Read, and Kinesthetic (VARK), which will be discussed later in this section.</p> <p>The learner's personality type is another factor that has been explored by educators seeking to find effective ways of teaching. The personality type can be defined as "a collection of personality traits that are thought to occur together consistently, especially as determined by a certain pattern of responses to a personality inventory" (Oxford Dictionaries [<reflink idref="bib47" id="ref43">47</reflink>]). There are several models and theories to describe personality types, including Myers–Briggs Type Indicator (MBTI<sups>®</sups>). The MBTI is a psychometric test that is used to determine how people perceive the world and make decisions. It has been employed to perform investigations about the relationship between individual characteristics and academic performance of engineering students (Rosati [<reflink idref="bib55" id="ref44">55</reflink>]; Felder and Brent [<reflink idref="bib20" id="ref45">20</reflink>]), and more specifically studies concerning the effects of personality type on engineering education (McCaulley et al. [<reflink idref="bib41" id="ref46">41</reflink>]; O'Brien et al. [<reflink idref="bib45" id="ref47">45</reflink>]). According to Orifici ([<reflink idref="bib46" id="ref48">46</reflink>]), the MBTI "offers a systematic means of identifying differences among students with respect to their preferences for perceiving information and making decisions—processes which are clearly related to learning." This paper focuses on the pedagogical applications of these psychometric tests, namely, MBTI and VARK, in an engineering design course. Further details and theory on each test are provided in the subsequent sections.</p> <hd id="AN0144641886-9">Myers–Briggs type indicator</hd> <p>Carl Gustav Jung, a Swiss psychiatrist, categorized people into primary types of psychological function (Jung [<reflink idref="bib33" id="ref49">33</reflink>]). In summary, psychological classification based on Jung's personality theory is based on three elements: two functional (sensation–intuition and thinking–feeling) and one attitude (extraversion–introversion) dimensions. Jung theorized that in each person, the functions are modified by the type of attitude that person possesses. He further claimed that these functions exist in all people but in different proportions. Therefore, one of these functions is superior, while the other functions become secondary, tertiary and inferior functions. In Jung's writings, extroverts are depicted as those who subordinate inner life to external necessity demands rather than inner promptings. Introverts, on the other hand, are depicted as being concerned primarily with their own subjective reality, rather than with objective reality (Miller [<reflink idref="bib42" id="ref50">42</reflink>]). Many have misinterpreted the concepts of extroversion–introversion that Jung had introduced in his writings, by relating these two concepts to individual differences in surgency and impulsiveness. Jung's psychological classification later became the foundation of one of the most commonly used type indicators, MBTI.</p> <p>Based on Jung's writings, Katharine Briggs and her daughter Isabel Briggs Myers developed a paper-and-pencil test, which came to be called Myers–Briggs Type Indicator. The MBTI questionnaire is designed to measure psychological preferences in how people perceive the world and make decisions. The key attribute of MBTI (which occasionally is overlooked by its users) is that it allows the clarity of a preference to be ascertained and not the strength of a preference or degree of aptitude (The Myers & Briggs Foundation [<reflink idref="bib62" id="ref51">62</reflink>]). MBTI assesses four underlying bipolar constructs: (<reflink idref="bib1" id="ref52">1</reflink>) extroversion (E) versus introversion (I), (<reflink idref="bib2" id="ref53">2</reflink>) sensing (S) versus intuition (N), (<reflink idref="bib3" id="ref54">3</reflink>) thinking (T) versus feeling (F), and (<reflink idref="bib4" id="ref55">4</reflink>) judging (J) versus perceiving (P). Therefore, there are sixteen distinct personality types that can be defined (i.e., ESTJ, ESTP,..., INFJ, and INFP). There are many texts in MBTI literature which address the certain characteristics associated with each of the 16 types. Myers and Briggs asserted that for each of the 16 types, one function is the most dominant and most likely to be evident earliest in life, while auxiliary functions (i.e., the second, third and fourth functions) typically become more evident later in life (Myers et al. [<reflink idref="bib43" id="ref56">43</reflink>]). Two common variations of the four-factor MBTI are the five-factor (Comrey [<reflink idref="bib11" id="ref57">11</reflink>]) and the six-factor (Sipps et al. [<reflink idref="bib60" id="ref58">60</reflink>]) solutions. The former splits T and F constructs, while the latter claims that J/P and S/N scales correlate with one another. For the purpose of this study, the original four-factor type indicator is utilized, which is summarized in Table 2.</p> <p>Learning style characteristics for each personality type (Kam et al. [<reflink idref="bib34" id="ref59">34</reflink>])</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Personality preference</p></th><th align="left"><p>Function</p></th><th align="left"><p>Learning style characteristic</p></th></tr></thead><tbody><tr><td align="left"><p>Extraversion (E)</p></td><td align="left"><p>Energy is directed outwards, focus on events and people in the outer world</p></td><td align="left"><p>Learn best through interacting with people, action and things</p></td></tr><tr><td align="left"><p>Introversion (I)</p></td><td align="left"><p>Energy is directed inwards, focus on internal thoughts and ideas</p></td><td align="left"><p>Learn best through quiet reflection and individual ways</p></td></tr><tr><td align="left"><p>Sensing (S)</p></td><td align="left"><p>Takes in information through the five senses, focus on concrete facts and experiences</p></td><td align="left"><p>Learn best through concrete experience, moving step by step with known things to the abstract</p></td></tr><tr><td align="left"><p>Intuition (N)</p></td><td align="left"><p>Takes in information through patterns and associations, focus on imagination and possibilities</p></td><td align="left"><p>Learn best through inspiration, starting with concepts before practical de-tails</p></td></tr><tr><td align="left"><p>Thinking (T)</p></td><td align="left"><p>Make decisions using logical reasoning, focus on objectivity and people's thoughts</p></td><td align="left"><p>Learn best through clear logical material, analyzing experiences to find objective truth</p></td></tr><tr><td align="left"><p>Feeling (F)</p></td><td align="left"><p>Make decisions using personal values, focus on harmony and people's feelings</p></td><td align="left"><p>Learn best through personal relationships, personalizing issues and causes that are important</p></td></tr><tr><td align="left"><p>Judging (J)</p></td><td align="left"><p>Oriented to the outer world in a planned and controlled manner, focus on making decisions and setting limits</p></td><td align="left"><p>Learn best through instruction that is organized and which moves in predictable ways, toward closure</p></td></tr><tr><td align="left"><p>Perceiving (P)</p></td><td align="left"><p>Oriented to the outer world in a flexible and spontaneous manner, focus on exploring options and being resourceful</p></td><td align="left"><p>Learn best through stimulation of something new and different, opportunity for open exploration</p></td></tr></tbody></table> </ephtml> </p> <p>Despite its popularity, especially in the business sector recently, MBTI has not remained without controversy. Several studies have challenged the stereotypic nature of the indicator (e.g., Emre [<reflink idref="bib17" id="ref60">17</reflink>]), whereas several others have suggested alternative approaches to categorizing personality types (e.g., Gerach et al. [<reflink idref="bib25" id="ref61">25</reflink>]). Instead of arguing for or against the effectiveness of MBTI (or any other personality type indicator), this paper attempts to explore any potential correlation between what MBTI assigns and how students from several years perform in different aspects of a hands-on, project-based engineering design course.</p> <hd id="AN0144641886-10">VARK learning styles</hd> <p>The acronym VARK stands for Visual, Aural, Read/write, and Kinesthetic. VARK questionnaires seek to extrapolate individuals' learning preferences by uncovering their dominance in one of the four sensory modalities. Recently, multimodality (the state of having more than one learning preference) has also been recognized as one of the outcomes of the VARK test. The VARK algorithm classifies individuals into having mild, strong, or very strong single learning style preferences or any combination of two, three, or four learning style preferences. Since the individuals taking the VARK questionnaire are allowed to select more than one option within a question, the prevalence of individuals classified as having multiple learning styles is high (Leite et al. [<reflink idref="bib38" id="ref62">38</reflink>]).</p> <p>Visual learners are most efficient when presented with visual information as a picture or illustration. Depiction of information in the form of charts, graphs, diagrams, etc. can assist them in learning. They more aware of their immediate environment, hence, study groups may not be useful to them (Fleming [<reflink idref="bib23" id="ref63">23</reflink>]; Bonva and Mihova [<reflink idref="bib4" id="ref64">4</reflink>]). Aural learners benefit from information that is presented in spoken language. They gain from lectures and enjoy group discussion (Fleming [<reflink idref="bib23" id="ref65">23</reflink>]) or listening to audio tapes. Read/write learners are more efficient when presented with information in the written form. They do well with the lectures that have slides or a suitable written outline. They tend to learn better by going to lectures and reading textbooks and notes (Fleming [<reflink idref="bib23" id="ref66">23</reflink>]). Kinesthetic learners are most efficient with hands-on style courses. They prefer to engage in activities and practice what they have learned, and benefit from practical demonstrations.</p> <p>The degree of strength of each of the modes defined above can vary from mild to strong or to very strong. The strength of a mode is determined by the score a person receives for the mode on the VARK test. Many people, however, are mixtures of the four aforementioned categories. They tend to learn in more complex ways, and are known as multimodals. Multimodality includes any combination of the four modes, i.e., VA, VR, VK, AR, AK, RK, ARK, VRK, VAR, VAK, and VARK. People with two preferred modes are known bimodals, those with three preferred modes are called trimodals, and those with all four modes are called quadmodals. Quadmodals are then grouped into VARK Type 1, VARK Type 2, and VARK Transition.</p> <p>People with VARK Type 1 learning modality tend to be more flexible with their modes to suit the occasion. For each situation, they utilize the one or two modes that best fit the context. For instance, a VARK Type 1 student can improve the learning process by utilizing the aural and read modes during a lecture. People in VARK Type 2 group use several modes in most situations, and are reluctant to make major decisions using only one or two modes, regardless of the context. Lastly, VARK Transition type can shift between Type 1 and Type 2. It is worth noting that to interpret VARK scores and determine preferences, the raw VARK values must be sent to the VARK Learn Ltd. company (for example, by email). The company then uses an algorithm known as the Research Algorithm (proprietary information) to output VARK types for each student. Alternatively, the online questionnaire can be used which outputs the VARK type upon finishing the test by an individual.</p> <p>It should also be mentioned that several recent studies question the dominance of VARK metric as a definite indicator of one's learning style (e.g., see Husmann and O'Loughlin [<reflink idref="bib26" id="ref67">26</reflink>]; Knoll et al. [<reflink idref="bib37" id="ref68">37</reflink>]; Rogowsky et al. [<reflink idref="bib54" id="ref69">54</reflink>]; Scott [<reflink idref="bib58" id="ref70">58</reflink>]). Such studies argue that learners may not necessarily have a certain type of learning style, nor can it be verified empirically that one's learning style does not evolve over time. Hence, VARK (or any other similar measure) should be used with care in classifying students in different learning styles. In this work, we closely investigate the lack or existence of any correlation between VARK and students' performance in a hands-on engineering design course over a number of years.</p> <hd id="AN0144641886-11">Research questions</hd> <p>This paper investigates several research questions pertaining to engineering design education, student characteristics, and their performance. Here, performance refers to the marks or grades that students obtain after each individual or team evaluation, including process-or outcome-based assessments. The following list represents five categories of questions that are addressed in the study:</p> <p></p> <ulist> <item> Is there a correlation between performance and gender or gender distribution on a team?</item> <p></p> <item> Is there a correlation between performance and learning style (as indicated by VARK)?</item> <p></p> <item> Is there a correlation between performance and personality type (as indicated by MBTI)?</item> <p></p> <item> Is there a correlation between traditional assessment mechanisms, in particular, engineering notebooks or design proposals, and overall performance in the course?</item> <p></p> <item> Is there a significant difference in performance between the three instructional design approaches used from 2001 to 2018?</item> </ulist> <hd id="AN0144641886-12">Methodology</hd> <p>Since the year 2011, students have been instructed to complete the MBTI and VARK questionnaires at the beginning of the term. The standardized MBTI questionnaire used in the course has four questions. Students' response to these four questions determines their personality types. The VARK questionnaire, on the other hand, has sixteen four-choice questions and is identical to the general VARK Questionnaire (version 7.0) provided by VARK Learn Ltd. online. Each choice that is provided in a question corresponds to one of the four VARK categories. Students find out about their VARK preference only after 'mapping' their choices to VARK categories on the scoring chart that is provided with the VARK questionnaire. In the final step, students calculate their scores in each of the V, A, R, and K categories by counting the number of times they have selected that category (note that multiple answers are allowed for each question). The generated VARK results are then sent to VARK Learn Ltd., and are converted from numerical VARK scores into the VARK learning styles using their research and standard algorithms. For the purpose of this work the results obtained from the research algorithm are used, since this algorithm is the most applicable for research (Fleming [<reflink idref="bib21" id="ref71">21</reflink>], [<reflink idref="bib22" id="ref72">22</reflink>], [<reflink idref="bib24" id="ref73">24</reflink>]).</p> <hd id="AN0144641886-13">Statistical tools</hd> <p>In this analysis, several statistical methods are used for the comparison of data. In what follows, the statistical tools that are used throughout the remainder of this paper are presented in their general form. The two main categories of statistical methods utilized are: (<reflink idref="bib1" id="ref74">1</reflink>) techniques for comparison of means and standard deviations, i.e., Welch's <emph>t</emph> test and Cohen's <emph>d</emph>, and (<reflink idref="bib2" id="ref75">2</reflink>) techniques for determining goodness of fit for a correlation, i.e., Coefficient of Correlation and Coefficient of Determination.</p> <hd id="AN0144641886-14">Welch's t test</hd> <p>Welch's <emph>t</emph> test is used in this work to determine whether the distributions of two samples are significantly different. The test is a modification of the standard two-variable <emph>t</emph> test, which is more robust to unequal variances in the samples (Derrick et al. [<reflink idref="bib12" id="ref76">12</reflink>]). The null hypothesis of Welch's test posits that the two sample groups are independent random samples that may have unequal variances and unequal sample sizes but equal means. If the null hypothesis is rejected, given a 5% significance level, then the samples have unequal means. Thus, when comparing student marks for different samples, a rejected null hypothesis indicates a significant difference in means. The value of Welch's <emph>t</emph> test can be calculated as follows (Welch [<reflink idref="bib67" id="ref77">67</reflink>]; MATLAB [<reflink idref="bib40" id="ref78">40</reflink>]):</p> <olist> <item> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>t</mi><mo>=</mo><mfrac><mrow><msub><mover accent="true"><mrow><mi>x</mi></mrow><mrow><mo stretchy="false">¯</mo></mrow></mover><mn>1</mn></msub><mo>-</mo><msub><mover accent="true"><mrow><mi>x</mi></mrow><mrow><mo stretchy="false">¯</mo></mrow></mover><mn>2</mn></msub></mrow><msqrt><mrow><mfrac><msubsup><mi>s</mi><mrow><mn>1</mn></mrow><mn>2</mn></msubsup><msub><mi>n</mi><mn>1</mn></msub></mfrac><mo>+</mo><mfrac><msubsup><mi>s</mi><mrow><mn>2</mn></mrow><mn>2</mn></msubsup><msub><mi>n</mi><mn>2</mn></msub></mfrac></mrow></msqrt></mfrac></mrow></math> </ephtml> </item> </olist> <p>Graph</p> <p>where <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mover accent="true"><mrow><mi>x</mi></mrow><mrow><mo stretchy="false">¯</mo></mrow></mover><mn>1</mn></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mover accent="true"><mrow><mi>x</mi></mrow><mrow><mo stretchy="false">¯</mo></mrow></mover><mn>2</mn></msub></math> </ephtml> are the means of sample groups <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>x</mi><mn>1</mn></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>x</mi><mn>2</mn></msub></math> </ephtml> , <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>n</mi><mn>1</mn></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>n</mi><mn>2</mn></msub></math> </ephtml> are the sample sizes of <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>x</mi><mn>1</mn></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>x</mi><mn>2</mn></msub></math> </ephtml> , and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>s</mi><mn>1</mn></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>s</mi><mn>2</mn></msub></math> </ephtml> are the sample standard deviations of <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>x</mi><mn>1</mn></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>x</mi><mn>2</mn></msub></math> </ephtml> , respectively. Further the degrees of freedom, <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>ν</mi></math> </ephtml> , of the <emph>t</emph> distribution are calculated by the Satterthwaite approximation (Fagerland and Sandvik [<reflink idref="bib18" id="ref79">18</reflink>]),</p> <p>2 <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>ν</mi><mo>=</mo><mfrac><msup><mfenced close=")" open="("><mrow><mfrac><msubsup><mi>s</mi><mrow><mn>1</mn></mrow><mn>2</mn></msubsup><msub><mi>n</mi><mn>1</mn></msub></mfrac><mo>+</mo><mfrac><msubsup><mi>s</mi><mrow><mn>2</mn></mrow><mn>2</mn></msubsup><msub><mi>n</mi><mn>2</mn></msub></mfrac></mrow></mfenced><mn>2</mn></msup><mrow><mfrac><msubsup><mi>s</mi><mrow><mn>1</mn></mrow><mn>4</mn></msubsup><mrow><msubsup><mi>n</mi><mrow><mn>1</mn></mrow><mn>2</mn></msubsup><msub><mi>ν</mi><mn>1</mn></msub></mrow></mfrac><mo>+</mo><mfrac><msubsup><mi>s</mi><mrow><mn>2</mn></mrow><mn>2</mn></msubsup><mrow><msubsup><mi>n</mi><mrow><mn>2</mn></mrow><mn>2</mn></msubsup><msub><mi>ν</mi><mn>2</mn></msub></mrow></mfrac></mrow></mfrac></mrow></math> </ephtml></p> <p>Graph</p> <p>3 <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>ν</mi><mn>1</mn></msub><mo>=</mo><msub><mi>n</mi><mn>1</mn></msub><mo>-</mo><mn>1</mn></mrow></math> </ephtml></p> <p>Graph</p> <p>4 <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>ν</mi><mn>2</mn></msub><mo>=</mo><msub><mi>n</mi><mn>2</mn></msub><mo>-</mo><mn>1</mn></mrow></math> </ephtml></p> <p>Graph</p> <p>where <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>ν</mi><mn>1</mn></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>ν</mi><mn>2</mn></msub></math> </ephtml> are the degrees of freedom for sample groups <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>x</mi><mn>1</mn></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>x</mi><mn>2</mn></msub></math> </ephtml> , respectively. Once the <emph>t</emph> value and degrees of freedom are determined, a <emph>p</emph> value can be found. The <emph>p</emph> value is then used to determine the test decision, i.e., acceptance or rejection of the null hypothesis. In this paper, only the test decision is presented since it is the main outcome of Welch's <emph>t</emph> test.</p> <hd id="AN0144641886-15">Cohen's d</hd> <p>While Welch's <emph>t</emph> test indicates whether two sample groups are significantly distinguished, it does not provide information on the strength of the difference in the two sets. Cohen's <emph>d</emph> is used in this work to determine the strength of the difference between the means of two sample groups (Cohen [<reflink idref="bib10" id="ref80">10</reflink>]). It provides an insight into the magnitude of the difference between two sample groups, determined by the parameter <emph>d</emph>, obtained as follows:</p> <p>5 <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>d</mi><mo>=</mo><mfrac><mrow><msub><mover accent="true"><mrow><mi>x</mi></mrow><mrow><mo stretchy="false">¯</mo></mrow></mover><mn>1</mn></msub><mo>-</mo><msub><mover accent="true"><mrow><mi>x</mi></mrow><mrow><mo stretchy="false">¯</mo></mrow></mover><mn>2</mn></msub></mrow><mi>s</mi></mfrac></mrow></math> </ephtml></p> <p>Graph</p> <p>6 <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>s</mi><mo>=</mo><msqrt><mfrac><mrow><mfenced close=")" open="("><mrow><msub><mi>n</mi><mn>1</mn></msub><mo>-</mo><mn>1</mn></mrow></mfenced><msubsup><mi>s</mi><mrow><mn>1</mn></mrow><mn>2</mn></msubsup><mo>+</mo><mfenced close=")" open="("><mrow><msub><mi>n</mi><mn>2</mn></msub><mo>-</mo><mn>1</mn></mrow></mfenced><msubsup><mi>s</mi><mrow><mn>2</mn></mrow><mn>2</mn></msubsup></mrow><mrow><msub><mi>n</mi><mn>1</mn></msub><mo>+</mo><msub><mi>n</mi><mn>2</mn></msub><mo>-</mo><mn>2</mn></mrow></mfrac></msqrt></mrow></math> </ephtml></p> <p>Graph</p> <p>where <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>s</mi></math> </ephtml> is the pooled variance, and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>s</mi><mn>1</mn></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>s</mi><mn>2</mn></msub></math> </ephtml> are the sample variances of the sample groups <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>x</mi><mn>1</mn></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>x</mi><mn>2</mn></msub></math> </ephtml> , respectively, calculated by:</p> <p>7 <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msubsup><mi>s</mi><mrow><mn>1</mn></mrow><mn>2</mn></msubsup><mo>=</mo><mfrac><mn>1</mn><mrow><msub><mi>n</mi><mn>1</mn></msub><mo>-</mo><mn>1</mn></mrow></mfrac><munderover><mo movablelimits="false">∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>n</mi><mn>1</mn></msub></munderover><msup><mfenced close=")" open="("><mrow><msub><mi>x</mi><mrow><mn>1</mn><mo>,</mo><mi>i</mi></mrow></msub><mo>-</mo><msub><mover accent="true"><mrow><mi>x</mi></mrow><mrow><mo stretchy="false">¯</mo></mrow></mover><mn>1</mn></msub></mrow></mfenced><mn>2</mn></msup></mrow></math> </ephtml></p> <p>Graph</p> <p>In Eq. 7, <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>x</mi><mrow><mn>1</mn><mo>,</mo><mi>i</mi></mrow></msub></math> </ephtml> denotes the <emph>i</emph>th element of the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>x</mi><mn>1</mn></msub></math> </ephtml> sample group. In general, the strength of the difference (i.e., the effect size) must be assessed within the context of the two samples being compared. However, Table 3 presents a general guideline for interpreting the effect size based on the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>d</mi></math> </ephtml> -parameter results.</p> <p>Effect size and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>d</mi></math> </ephtml> -parameter rules of thumb (Sawilowsky [<reflink idref="bib56" id="ref81">56</reflink>])</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mi xmlns="">d</mi></math><inline-graphic href="10798_2019_9515_Article_IEq35.gif" />-parameter</p></th><th align="left"><p>0.01</p></th><th align="left"><p>0.20</p></th><th align="left"><p>0.50</p></th><th align="left"><p>0.80</p></th><th align="left"><p>1.20</p></th><th align="left"><p>2.00</p></th></tr></thead><tbody><tr><td align="left"><p>Effect size</p></td><td align="left"><p>Very small</p></td><td align="left"><p>Small</p></td><td align="left"><p>Medium</p></td><td align="left"><p>Large</p></td><td align="left"><p>Very large</p></td><td align="left"><p>Huge</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0144641886-16">Coefficient of correlation</hd> <p>The sum of squared errors (SSE) is calculated in this work to determine the coefficient of correlation between the sample groups, which is a measure of the fit of a sample to a modelled regression. By summing the square of the residuals, a value indicating the goodness of the fit is determined. An <emph>SSE</emph> value closer to zero indicates that the modelled regression better predicts the sample distribution. It is calculated as follows:</p> <p>8 <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>S</mi><mi>S</mi><mi>E</mi><mo>=</mo><munderover><mo movablelimits="false">∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>n</mi><mn>1</mn></msub></munderover><msup><mfenced close=")" open="("><mrow><msub><mi>x</mi><mrow><mn>1</mn><mo>,</mo><mi>i</mi></mrow></msub><mo>-</mo><msub><mover accent="true"><mi>x</mi><mo stretchy="false">^</mo></mover><mrow><mn>1</mn><mo>,</mo><mi>i</mi></mrow></msub></mrow></mfenced><mn>2</mn></msup></mrow></math> </ephtml></p> <p>Graph</p> <p>where <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mover accent="true"><mi>x</mi><mo stretchy="false">^</mo></mover><mrow><mn>1</mn><mo>,</mo><mi>i</mi></mrow></msub></math> </ephtml> is the predicted value for <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>x</mi><mrow><mn>1</mn><mo>,</mo><mi>i</mi></mrow></msub></math> </ephtml> based on the modelled regression.</p> <hd id="AN0144641886-17">Coefficient of determination (R-squared)</hd> <p>In this work, the coefficient of determination (also known as <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mi>R</mi><mn>2</mn></msup></math> </ephtml> or <emph>R</emph>-squared) is used in order to determine the amount of variation that can be accounted for between a variable and a linear regression model. The coefficient of determination ranges from 0 to 1, where a higher value indicates that the model explains the variability of the sample variable to a high-degree, whereas a lower value indicates that the model poorly predicts the variation of the sample variable. The coefficient of determination can be described by the following equation:</p> <p>9 <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msup><mi>R</mi><mn>2</mn></msup><mo>=</mo><mn>1</mn><mo>-</mo><mfrac><mrow><mi mathvariant="italic">SSE</mi></mrow><mrow><msubsup><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>n</mi><mn>1</mn></msub></msubsup><msup><mfenced close=")" open="("><mrow><msub><mi>x</mi><mrow><mn>1</mn><mo>,</mo><mi>i</mi></mrow></msub><mo>-</mo><msub><mover accent="true"><mrow><mi>x</mi></mrow><mrow><mo stretchy="false">¯</mo></mrow></mover><mn>1</mn></msub></mrow></mfenced><mn>2</mn></msup></mrow></mfrac></mrow></math> </ephtml></p> <p>Graph</p> <p>where the denominator represents the sum of squares about the mean. Note that the coefficient of determination is also equal to the squared Pearson correlation coefficient.</p> <hd id="AN0144641886-18">Results</hd> <p>The results are divided into sections pertaining to the aforementioned research questions. Overall, 2637 students were enrolled in the course between the years of 2001 and 2018, with 1367 students participating between 2011 and 2018 for whom gender, VARK, and MBTI data are available. To answer each research question, a subset of these data is employed since some students may have incomplete data in certain areas (e.g., a missed evaluation), or that the inclusion of certain data might artificially skew results (e.g., comparing the results of two-or four-person teams to the majority three-person). Thus, in each section the specifics of dataset are defined in greater detail. Moreover, the results and discussion pertinent to each research question are presented in their respective sections, namely, Gender, Learning Style, Personality Type, Assessment Mechanisms, and Instructional Design.</p> <hd id="AN0144641886-19">Gender</hd> <p>The first parameter considered with respect to performance is gender. In this study, the students self-identified their gender between the years of 2011 and 2018, and their distribution is presented in Table 4. It is worth noting that most students identified themselves as either male (73.7%) or female (24.6%), with undefined or non-binary students comprising a small portion (1.7%). Since the year-to-year and total population of students who are undefined or non-binary is very small, the remainder of this section focuses on the self-identified, female or male students. The two main areas of inquiry regarding gender and student performance are: (<reflink idref="bib1" id="ref82">1</reflink>) performance and gender correlation, and (<reflink idref="bib2" id="ref83">2</reflink>) performance and gender distribution on a team.</p> <p>Gender distribution from years 2011 to 2018</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left"><p>2011</p></th><th align="left"><p>2012</p></th><th align="left"><p>2013</p></th><th align="left"><p>2014</p></th><th align="left"><p>2016</p></th><th align="left"><p>2017</p></th><th align="left"><p>2018</p></th><th align="left"><p>Total</p></th></tr></thead><tbody><tr><td align="left"><p>Female</p></td><td char="." align="char"><p>48</p></td><td char="." align="char"><p>39</p></td><td char="." align="char"><p>49</p></td><td char="." align="char"><p>48</p></td><td char="." align="char"><p>49</p></td><td char="." align="char"><p>57</p></td><td char="." align="char"><p>47</p></td><td char="." align="char"><p>337</p></td></tr><tr><td align="left"><p>Male</p></td><td char="." align="char"><p>143</p></td><td char="." align="char"><p>134</p></td><td char="." align="char"><p>158</p></td><td char="." align="char"><p>156</p></td><td char="." align="char"><p>139</p></td><td char="." align="char"><p>169</p></td><td char="." align="char"><p>108</p></td><td char="." align="char"><p>1007</p></td></tr><tr><td align="left"><p>Non-binary or undefined</p></td><td char="." align="char"><p>12</p></td><td char="." align="char"><p>4</p></td><td char="." align="char"><p>0</p></td><td char="." align="char"><p>1</p></td><td char="." align="char"><p>6</p></td><td char="." align="char"><p>0</p></td><td char="." align="char"><p>0</p></td><td char="." align="char"><p>23</p></td></tr></tbody></table> </ephtml> </p> <p>To begin, consider the performance of female and male students, from 2011 to 2018, in individual-based assignments and final marks. Table 5 presents the mean marks for the combined interim and final notebook evaluations (Notebook), the combined marks for the week 5 and week 8 individual evaluations (Individual), and the final marks. The results for Welch's <emph>t</emph> test and Cohen's <emph>d</emph> (with effect size) are also presented to compare each set of marks for female and male students. It is important to note that the statistics indicate an insignificant difference in individual and overall performance between female and male students (Welch's <emph>t</emph> test outputs 0 and the effect size is small). There is, however, a medium effect size between the means for the notebook evaluations with regard to gender. In this case, females outperform males by a small margin. Even though the difference is quite nominal, since comprehensive, ongoing note-taking practices and organization are the key performance measures for notebook evaluations, this result is consistent with some of the previous studies that show a better performance by female college students than their male counterparts in both lecture (Reddington et al. [<reflink idref="bib52" id="ref84">52</reflink>]) and field note-taking (Dohaney et al. [<reflink idref="bib13" id="ref85">13</reflink>]). Such difference has been attributed to the variation in fine motor skills between female and male students, primarily dependent on finger size (Peters et al. [<reflink idref="bib48" id="ref86">48</reflink>]), as well as cognitive variables such as handwriting speed and working memory (Reddington [<reflink idref="bib51" id="ref87">51</reflink>]). Nevertheless, given that the remaining areas of inquiry show insignificant difference in performance based on gender, the authors would encourage further investigation into the effects of gender on design notebook evaluations.</p> <p>Statistical results for individual performance and gender</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left"><p>Notebook</p></th><th align="left"><p>Individual</p></th><th align="left"><p>Final mark</p></th></tr></thead><tbody><tr><td align="left"><p>Mean (female)</p></td><td align="left"><p>77.81%</p></td><td align="left"><p>78.83%</p></td><td align="left"><p>78.38%</p></td></tr><tr><td align="left"><p>Mean (male)</p></td><td align="left"><p>74.11%</p></td><td align="left"><p>78.32%</p></td><td align="left"><p>77.54%</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test</p></td><td align="left"><p>1</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td></tr><tr><td align="left"><p>Cohen's <italic>d</italic></p></td><td align="left"><p>0.30</p></td><td align="left"><p>0.06</p></td><td align="left"><p>0.12</p></td></tr><tr><td align="left"><p>Effect size</p></td><td align="left"><p>Medium</p></td><td align="left"><p>Small</p></td><td align="left"><p>Small</p></td></tr></tbody></table> </ephtml> </p> <p>In the current study, teams' gender distributions were determined only for three-member teams. Those that had two or four members were omitted since they were few. The gender distribution on each team is represented by a sequence of letters, where female is denoted by 'F' and male by 'M'. Further, the focus was on the combination and not the sequence of members in a team. Hence, FMM, MFM, and MMF represent the same distribution (one female and two male students). The total number of teams for each gender distribution are: FFF = 27, FFM = 56, FMM = 126, and MMM = 221. For each of the team gender distributions three parameters were considered, namely, team evaluations (weeks 10, 12 and 14), written assessments (proposal and report), and team final mark (average of the three team members final mark).</p> <p>The statistical results for each of these performance evaluations for the team gender distributions are presented in Table 6. The data demonstrate that there is no significant difference in performance across all three parameters when considering gender distribution. In all 18 cases, Welch's <emph>t</emph> test did not reject the null hypothesis (significance level less than 5%) and Cohen's <emph>d</emph> was either very small, small or on the low end of the medium effect size. Thus, it is clear from the given data that gender distribution is not a key factor or indicator with regard to team performance success.</p> <p>Statistical results for team performance and gender distribution</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left"><p>Team</p></th><th align="left"><p>Written</p></th><th align="left"><p>Team final mark</p></th></tr></thead><tbody><tr><td align="left"><p>Mean (FFF)</p></td><td align="left"><p>75.99%</p></td><td align="left"><p>79.79%</p></td><td align="left"><p>78.33%</p></td></tr><tr><td align="left"><p>Mean (FFM)</p></td><td align="left"><p>78.05%</p></td><td align="left"><p>80.39%</p></td><td align="left"><p>78.80%</p></td></tr><tr><td align="left"><p>Mean (FMM)</p></td><td align="left"><p>76.41%</p></td><td align="left"><p>79.15%</p></td><td align="left"><p>77.44%</p></td></tr><tr><td align="left"><p>Mean (MMM)</p></td><td align="left"><p>77.35%</p></td><td align="left"><p>78.82%</p></td><td align="left"><p>77.77%</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test (FFF, FFM)</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test (FFF, FMM)</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test (FFF, MMM)</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test (FFM, MMM)</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test (FFM, FMM)</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test (FMM, MMM)</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td></tr><tr><td align="left"><p>Cohen's <italic>d,</italic> effect size (FFF, FFM)</p></td><td align="left"><p>0.04 (small)</p></td><td align="left"><p>0.09 (small)</p></td><td align="left"><p>0.13 (small)</p></td></tr><tr><td align="left"><p>Cohen's <italic>d,</italic> effect size (FFF, FMM)</p></td><td align="left"><p>0.22 (medium)</p></td><td align="left"><p>0.09 (small)</p></td><td align="left"><p>0.08 (small)</p></td></tr><tr><td align="left"><p>Cohen's <italic>d,</italic> effect size (FFF, MMM)</p></td><td align="left"><p>0.16 (small)</p></td><td align="left"><p>0.15 (small)</p></td><td align="left"><p>0.09 (small)</p></td></tr><tr><td align="left"><p>Cohen's <italic>d,</italic> effect size (FFM, MMM)</p></td><td align="left"><p>0.08 (small)</p></td><td align="left"><p>0.24 (medium)</p></td><td align="left"><p>0.16 (small)</p></td></tr><tr><td align="left"><p>Cohen's <italic>d,</italic> effect size (FFM, FMM)</p></td><td align="left"><p>0.16 (small)</p></td><td align="left"><p>0.17 (small)</p></td><td align="left"><p>0.20 (small)</p></td></tr><tr><td align="left"><p>Cohen's <italic>d,</italic> effect size (FMM, MMM)</p></td><td align="left"><p>0.10 (small)</p></td><td align="left"><p>0.05 (small)</p></td><td align="left"><p>0.05 (very small)</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0144641886-20">Learning style</hd> <p>The VARK questionnaires completed by students were analyzed using the Research Algorithm to determine their learning preferences. Table 7 presents the distributions of the students' learning preferences, group's average performance in the notebook (interim and final) and in the individual (weeks 5 and 8) evaluations, as well as the average final marks and standard deviations of final marks for each group.</p> <p>Research algorithm results and mean performance metrics for VARK learning styles</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Learning style</p></th><th align="left"><p>Population</p></th><th align="left"><p>Notebook mark (%)</p></th><th align="left"><p>Individual mark (%)</p></th><th align="left"><p>Final mark (%)</p></th><th align="left"><p>Final mark std. dev.</p></th></tr></thead><tbody><tr><td align="left"><p>V, mild</p></td><td char="." align="char"><p>243</p></td><td align="left"><p>76.08</p></td><td align="left"><p>78.93</p></td><td align="left"><p>78.15</p></td><td align="left"><p>6.83</p></td></tr><tr><td align="left"><p>V, strong</p></td><td char="." align="char"><p>58</p></td><td align="left"><p>75.67</p></td><td align="left"><p>79.16</p></td><td align="left"><p>78.28</p></td><td align="left"><p>7.01</p></td></tr><tr><td align="left"><p>V, very strong</p></td><td char="." align="char"><p>13</p></td><td align="left"><p>75.67</p></td><td align="left"><p>71.55</p></td><td align="left"><p>77.46</p></td><td align="left"><p>8.88</p></td></tr><tr><td align="left"><p><bold>Subtotal V</bold></p></td><td char="." align="char"><p><bold>314</bold></p></td><td align="left"><p><bold>75.99</bold></p></td><td align="left"><p><bold>78.67</bold></p></td><td align="left"><p><bold>78.15</bold></p></td><td align="left"><p><bold>6.93</bold></p></td></tr><tr><td align="left"><p>A, mild</p></td><td char="." align="char"><p>108</p></td><td align="left"><p>73.95</p></td><td align="left"><p>78.22</p></td><td align="left"><p>77.08</p></td><td align="left"><p>6.83</p></td></tr><tr><td align="left"><p>A, strong</p></td><td char="." align="char"><p>16</p></td><td align="left"><p>64.01</p></td><td align="left"><p>70.93</p></td><td align="left"><p>74.19</p></td><td align="left"><p>8.80</p></td></tr><tr><td align="left"><p>A, very strong</p></td><td char="." align="char"><p>4</p></td><td align="left"><p>78.78</p></td><td align="left"><p>78.51</p></td><td align="left"><p>79.75</p></td><td align="left"><p>5.56</p></td></tr><tr><td align="left"><p><bold>Subtotal A</bold></p></td><td char="." align="char"><p><bold>128</bold></p></td><td align="left"><p><bold>72.86</bold></p></td><td align="left"><p><bold>77.32</bold></p></td><td align="left"><p><bold>76.80</bold></p></td><td align="left"><p><bold>7.14</bold></p></td></tr><tr><td align="left"><p>R, mild</p></td><td char="." align="char"><p>106</p></td><td align="left"><p>74.24</p></td><td align="left"><p>78.15</p></td><td align="left"><p>76.95</p></td><td align="left"><p>6.13</p></td></tr><tr><td align="left"><p>R, strong</p></td><td char="." align="char"><p>15</p></td><td align="left"><p>75.05</p></td><td align="left"><p>79.21</p></td><td align="left"><p>76.27</p></td><td align="left"><p>8.54</p></td></tr><tr><td align="left"><p>R, very strong</p></td><td char="." align="char"><p>3</p></td><td align="left"><p>70.27</p></td><td align="left"><p>70.18</p></td><td align="left"><p>70.33</p></td><td align="left"><p>14.98</p></td></tr><tr><td align="left"><p><bold>Subtotal R</bold></p></td><td char="." align="char"><p><bold>124</bold></p></td><td align="left"><p><bold>74.24</bold></p></td><td align="left"><p><bold>78.09</bold></p></td><td align="left"><p><bold>76.71</bold></p></td><td align="left"><p><bold>6.72</bold></p></td></tr><tr><td align="left"><p>K, mild</p></td><td char="." align="char"><p>135</p></td><td align="left"><p>72.41</p></td><td align="left"><p>76.86</p></td><td align="left"><p>76.76</p></td><td align="left"><p>10.16</p></td></tr><tr><td align="left"><p>K, strong</p></td><td char="." align="char"><p>31</p></td><td align="left"><p>75.90</p></td><td align="left"><p>78.28</p></td><td align="left"><p>77.87</p></td><td align="left"><p>5.54</p></td></tr><tr><td align="left"><p>K, very strong</p></td><td char="." align="char"><p>3</p></td><td align="left"><p>79.2</p></td><td align="left"><p>85.43</p></td><td align="left"><p>78.33</p></td><td align="left"><p>3.06</p></td></tr><tr><td align="left"><p><bold>Subtotal K</bold></p></td><td char="." align="char"><p><bold>169</bold></p></td><td align="left"><p><bold>73.17</bold></p></td><td align="left"><p><bold>77.27</bold></p></td><td align="left"><p><bold>76.99</bold></p></td><td align="left"><p><bold>9.39</bold></p></td></tr><tr><td align="left"><p><bold>Subtotal single modal</bold></p></td><td char="." align="char"><p><bold>735</bold></p></td><td align="left"><p><bold>74.50</bold></p></td><td align="left"><p><bold>78.02</bold></p></td><td align="left"><p><bold>77.41</bold></p></td><td align="left"><p><bold>7.45</bold></p></td></tr><tr><td align="left"><p>VA</p></td><td char="." align="char"><p>9</p></td><td align="left"><p>71.24</p></td><td align="left"><p>74.60</p></td><td align="left"><p>75.00</p></td><td align="left"><p>4.77</p></td></tr><tr><td align="left"><p>VR</p></td><td char="." align="char"><p>17</p></td><td align="left"><p>73.45</p></td><td align="left"><p>74.54</p></td><td align="left"><p>74.47</p></td><td align="left"><p>7.83</p></td></tr><tr><td align="left"><p>VK</p></td><td char="." align="char"><p>30</p></td><td align="left"><p>75.58</p></td><td align="left"><p>81.01</p></td><td align="left"><p>79.23</p></td><td align="left"><p>6.12</p></td></tr><tr><td align="left"><p>AR</p></td><td char="." align="char"><p>2</p></td><td align="left"><p>79.03</p></td><td align="left"><p>84.25</p></td><td align="left"><p>83.5</p></td><td align="left"><p>2.12</p></td></tr><tr><td align="left"><p>AK</p></td><td char="." align="char"><p>8</p></td><td align="left"><p>77.94</p></td><td align="left"><p>75.13</p></td><td align="left"><p>76.75</p></td><td align="left"><p>5.60</p></td></tr><tr><td align="left"><p>RK</p></td><td char="." align="char"><p>0</p></td><td align="left"><p>–</p></td><td align="left"><p>–</p></td><td align="left"><p>–</p></td><td align="left"><p>–</p></td></tr><tr><td align="left"><p><bold>Subtotal bimodal</bold></p></td><td char="." align="char"><p>66</p></td><td align="left"><p>74.83</p></td><td align="left"><p>77.85</p></td><td align="left"><p>77.26</p></td><td align="left"><p>6.19</p></td></tr><tr><td align="left"><p>VARK type 1</p></td><td char="." align="char"><p>304</p></td><td align="left"><p>75.22</p></td><td align="left"><p>78.96</p></td><td align="left"><p>78.15</p></td><td align="left"><p>7.52</p></td></tr><tr><td align="left"><p>VARK type 2</p></td><td char="." align="char"><p>141</p></td><td align="left"><p>76.95</p></td><td align="left"><p>79.39</p></td><td align="left"><p>78.40</p></td><td align="left"><p>6.91</p></td></tr><tr><td align="left"><p>VARK transition</p></td><td char="." align="char"><p>96</p></td><td align="left"><p>75.88</p></td><td align="left"><p>78.90</p></td><td align="left"><p>78.70</p></td><td align="left"><p>5.42</p></td></tr><tr><td align="left"><p><bold>Subtotal quadmodal</bold></p></td><td char="." align="char"><p><bold>541</bold></p></td><td align="left"><p><bold>75.88</bold></p></td><td align="left"><p><bold>79.06</bold></p></td><td align="left"><p><bold>78.31</bold></p></td><td align="left"><p><bold>6.99</bold></p></td></tr><tr><td align="left"><p><bold>Total</bold></p></td><td char="." align="char"><p><bold>1342</bold></p></td><td align="left"><p><bold>75.07</bold></p></td><td align="left"><p><bold>78.43</bold></p></td><td align="left"><p><bold>77.77</bold></p></td><td align="left"><p><bold>7.20</bold></p></td></tr></tbody></table> </ephtml> </p> <p>Statistical methods were then applied to the results that are presented in Table 7. However, before analyzing the results, a few comments on the dataset itself should be noted. First, and foremost, it is clear that the population across different learning preferences are non-uniform (as expected), and that certain modes are much more highly represented. For example, the population of students with bimodal learning styles is quite small relative to single modal or quadmodal preferences. Also, there are no students with the RK learning style. It should also be noted that the bimodal learning styles are comprised of students that might be a combination of mild and strong preferences in the two different modes. For instance, for the bimodal visual–aural (VA), learning preferences can be V-mild and A-mild, V-mild and A-strong, V-strong and A-mild, or V-strong and A-strong. In our dataset, there were no instances of 'very strong' learning preference in the bimodal students. In fact, most of the bimodal students had two mild learning preferences (e.g., V-mild and A-mild). As such, their strengths in learning preferences are ignored. In a similar fashion, the results for single modal students are combined, i.e., V includes students with V-mild, V-strong, and V-very-strong as their learning preferences. Lastly, there are several learning styles with populations that are not statistically significant and have been omitted from the remainder of the analysis accordingly. These learning styles are the bimodal students with VA, VR, AR, AK, and RK learning preferences. The Cohen's <emph>d</emph> results comparing the learning styles for the notebook assessments, individual evaluations, and final marks will now be discussed.</p> <p>Table 8 presents the Cohen's <emph>d</emph> results for the notebook marks between students in each learning style category. In this table, and subsequent Cohen's <emph>d</emph> tables, a positive value indicates that the row sample group has performed better than the column sample group. For example, Cohen's <emph>d</emph> for V versus A results in a difference value of 0.26, which means V has a medium-strength better performance than A. Similarly, a negative value indicates that the row sample group has performed worse than the column sample group. For instance, Cohen's <emph>d</emph> for A versus V is − 0.26, denoting that A has a medium-strength worse performance relative to V. Through using a colour gradient from the most negative value (blue) to the most positive value (red), trends in the data can be more easily visualized.</p> <p>Graph</p> <p>From Table 8 it is clear that VARK Type 2 outperforms the rest of the learning styles in the notebook evaluation. In particular, VARK Type 2 performs much better (with a medium effect size) than A, R, and K single modal learners. It appears, with the exception of visual learners, that single modal learners do worse than their bimodal or quadmodal counterparts (a trend that will also be evident in the final marks.) Technical documentation in a notebook tends to be primarily a visual task (e.g., sketching, drawing flow charts and schematics,) but also relies on multiple other modes of expression. As such, it is likely that single modal learners are unable to leverage other modes of learning, resulting in an inferior performance. Further, amongst the single modal learners, the visual learners have the strongest performance, as expected, followed by read/write, kinesthetic, and aural types in order. The success of the VARK Type 2 can likely be attributed to their desire to experience multiple learning modes. As such, a VARK Type 2 learner might document in their notebook technical work and design experience in multiple forms (e.g., use of diagrams, outline of an experiment, writing of formulas,) which provide a much more comprehensive outline of the technical content and design process. There is not a significant difference among the quadmodal types (VARK Type 1, Type 2, and Transition), unlike what was previously observed between the VARK Type 2 and the single modals. But, there is a small difference between VARK Type 2 and VARK Type 1, which further supports the notion that the desire to learn in multiple modes is an asset to the VARK Type 2 learners, since VARK Type 1 would not necessarily express the same content in multiple forms. Instead, they just use the mode which they find the most effective. Lastly, VARK Transition lies between VARK Type 1 and VARK Type 2 when performance is considered.</p> <p>Table 9 presents the Cohen's <emph>d</emph> results for individual performance evaluations of weeks 5 and 8 evaluations. Evidently, there is a clear shift in the outcome from the previously-discussed notebook assessment. In weeks 5 and 8 evaluations, which are individual assessments, VK bimodal learners outperform all other learning styles. The performance in the individual evaluations rely heavily on the effort and outcome of a student's design activities. Hence, the physical manifestations of students learning (e.g., prototype fabrication, circuit development, code implementation), are highly valued in the individual evaluations. For this type of assessment, there is a clear preference to VK bimodal learners, since the evaluation hinges on the results (and efforts) of the design process. Moreover, in many cases, articulating a design visually (e.g., through schematics, functional diagrams, flow charts of computer codes), can greatly benefit the student's ability to construct and implement a design. Thus, based on this observation, a visual learner would also be well prepared to perform in this type of evaluation, whereas a kinesthetic learner might implement their design prematurely resulting in a poorer performance. This is also reflected in the results, where V-type learners outperform other single modal learners. Also, there is a very small effect size between the visual and quadmodal learners, despite the fact that the latter generally outperforms the single modal learners. Once again, the VARK Type 2 is dominant amongst the quadmodal learners. This may be attributed to the same effect seen in the bimodal VK versus single modal V or K learners. Employing multiple modes for the task is more effective than using a single mode (despite being the most apt for the task at hand).</p> <p>Graph</p> <p>The last element of the VARK analysis to be investigated is the overall performance (i.e., final marks) of the students with different learning styles. In Table 10, the Cohen's <emph>d</emph> results for the final mark distributions of various learning styles are presented. The final course marks demonstrate more clearly the underlying theme that multiple modes are preferential in this style of hands-on engineering design course. The bimodal VK outperforms all other learning styles, followed closely by the quadmodal students. While it is anticipated that VK learners are more successful in this type of course, it is less intuitive that a strictly kinesthetic or visual preference would not result in a high performance. As was the case with the individual evaluation, the visual learners are the strongest among the single modals. Further, all single modal learners show inferior performances relative to multimodal learners, although V-type has only a very small effect size difference. Considering the overall performance of the students, VARK Transition outperforms VARK Type 1 and VARK Type 2. This is in contrast to our previously-discussed notebook and individual evaluations marks, which may be explained as follows: the ability to transition between VARK Type 1 and Type 2 can be an asset when there are multiple assessment modes (e.g., written, presentation, performance). Hence, VARK Transition learners are more capable of shifting between single and multimodal learning styles based on the particular context. Thus, they benefit from the application of multiple modes in assessments that require it, but also from the efficiency of single modal learning in contexts that require only one particular mode to be utilized.</p> <p>Graph</p> <hd id="AN0144641886-21">Personality type</hd> <p>In the following discussion, the effects of personality types on the individual performance and the team performance are considered. Table 11 presents the population size for all MBTI groups, as well as their mean notebook marks (i.e., combined interim and final notebook marks), mean individual marks (i.e., combined weeks 5 and 8 marks), and final marks. Further, in Tables 12, 13 and 14 the Cohen's <emph>d</emph> results for notebook, individual, and final marks are presented respectively. Unlike the case of VARK learning styles, the population size for each MBTI group is statistically significant. But the distributions are still non-uniform and the lowest population belongs to ENFJ with 14 students.</p> <p>MBTI population size and mean marks for their notebook, individual and final marks</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>MBTI</p></th><th align="left"><p>Population</p></th><th align="left"><p>Notebook (%)</p></th><th align="left"><p>Individual (%)</p></th><th align="left"><p>Final mark (%)</p></th></tr></thead><tbody><tr><td align="left"><p>ESTJ</p></td><td char="." align="char"><p>62</p></td><td char="." align="char"><p>78.55</p></td><td char="." align="char"><p>79.41</p></td><td char="." align="char"><p>79.52</p></td></tr><tr><td align="left"><p>ESTP</p></td><td char="." align="char"><p>68</p></td><td char="." align="char"><p>73.49</p></td><td char="." align="char"><p>76.86</p></td><td char="." align="char"><p>77.50</p></td></tr><tr><td align="left"><p>ESFJ</p></td><td char="." align="char"><p>27</p></td><td char="." align="char"><p>77.87</p></td><td char="." align="char"><p>78.49</p></td><td char="." align="char"><p>78.19</p></td></tr><tr><td align="left"><p>ESFP</p></td><td char="." align="char"><p>42</p></td><td char="." align="char"><p>70.76</p></td><td char="." align="char"><p>73.39</p></td><td char="." align="char"><p>75.93</p></td></tr><tr><td align="left"><p>ENTJ</p></td><td char="." align="char"><p>54</p></td><td char="." align="char"><p>74.01</p></td><td char="." align="char"><p>76.05</p></td><td char="." align="char"><p>78.43</p></td></tr><tr><td align="left"><p>ENTP</p></td><td char="." align="char"><p>55</p></td><td char="." align="char"><p>70.73</p></td><td char="." align="char"><p>74.52</p></td><td char="." align="char"><p>76.36</p></td></tr><tr><td align="left"><p>ENFJ</p></td><td char="." align="char"><p>14</p></td><td char="." align="char"><p>77.12</p></td><td char="." align="char"><p>78.52</p></td><td char="." align="char"><p>80.29</p></td></tr><tr><td align="left"><p>ENFP</p></td><td char="." align="char"><p>41</p></td><td char="." align="char"><p>72.88</p></td><td char="." align="char"><p>75.97</p></td><td char="." align="char"><p>76.95</p></td></tr><tr><td align="left"><p>ISTJ</p></td><td char="." align="char"><p>190</p></td><td char="." align="char"><p>77.61</p></td><td char="." align="char"><p>78.35</p></td><td char="." align="char"><p>78.65</p></td></tr><tr><td align="left"><p>ISTP</p></td><td char="." align="char"><p>121</p></td><td char="." align="char"><p>74.01</p></td><td char="." align="char"><p>76.80</p></td><td char="." align="char"><p>77.45</p></td></tr><tr><td align="left"><p>ISFJ</p></td><td char="." align="char"><p>99</p></td><td char="." align="char"><p>77.92</p></td><td char="." align="char"><p>78.52</p></td><td char="." align="char"><p>78.04</p></td></tr><tr><td align="left"><p>ISFP</p></td><td char="." align="char"><p>57</p></td><td char="." align="char"><p>74.70</p></td><td char="." align="char"><p>75.64</p></td><td char="." align="char"><p>76.96</p></td></tr><tr><td align="left"><p>INTJ</p></td><td char="." align="char"><p>218</p></td><td char="." align="char"><p>75.37</p></td><td char="." align="char"><p>77.98</p></td><td char="." align="char"><p>78.04</p></td></tr><tr><td align="left"><p>INTP</p></td><td char="." align="char"><p>153</p></td><td char="." align="char"><p>72.05</p></td><td char="." align="char"><p>75.37</p></td><td char="." align="char"><p>76.78</p></td></tr><tr><td align="left"><p>INFJ</p></td><td char="." align="char"><p>58</p></td><td char="." align="char"><p>77.29</p></td><td char="." align="char"><p>78.12</p></td><td char="." align="char"><p>78.17</p></td></tr><tr><td align="left"><p>INFP</p></td><td char="." align="char"><p>69</p></td><td char="." align="char"><p>73.57</p></td><td char="." align="char"><p>75.79</p></td><td char="." align="char"><p>76.71</p></td></tr></tbody></table> </ephtml> </p> <p>Graph</p> <p>Graph</p> <p>Graph</p> <p>Table 12 shows the Cohen's <emph>d</emph> results for different MBTI sample groups with respect to the cumulative notebook evaluation marks. In the last row of the table, the sum of the differences for each column has been included and coloured accordingly. These sums represent the sum of differences for each MBTI group relative to the other groups, where a positive number indicates poor performance (red) and a negative number indicates a strong performance (blue) relative to the average. As expected, the sums of differences have a total (and mean) of zero. The same statements apply to Table 13, which shows the Cohen's <emph>d</emph> results for different MBTI sample groups with respect to the mean individual evaluations marks.</p> <p>The first observation (in both tables) is that the columns alternate between positive and negative values. The pattern emerges in both tables and while the columns (or rows) are not entirely dominated by values of a particular sign, there is clearly a trend of certain MBTI groups outperforming others. It should be noted that individuals with the judging trait performed well relative to their peers with the perceiving trait (with the exception of ENTJ group). Moreover, the four strongest performances belong to the four groups that had the combination of sensing and judging traits, namely, ESTJ, ISFJ, ESFJ, and ISTJ. It is possible that this combination of traits is synergistic and advantageous with regard to individual engineering design assessments. As sensing types, students are more inclined to focus on concrete experience and practical applications as they work on their notebooks or individual design work. As judging types, students focus on articulating clear plans, making decisions, and setting and achieving deadlines. Thus, S–J type students have the natural inclination to develop and articulate plans and deadlines and focus their design work on practical, concrete and achievable tasks. In addition, the judging trait is likely a strong performer across all categories due to the highly demanding and short timeline maintained in the course. In a nutshell, a natural inclination for task planning, quick and definitive decisions, and maintaining deadlines is an asset.</p> <p>In Table 14 the Cohen's <emph>d</emph> results for final marks of the MBTI sample groups are presented. These results show some similarities and some differences to Cohen's <emph>d</emph> results for the notebook and individual evaluations (i.e., Tables 12 and 13). First, the trend that judging types outperform their perceiving type peers, noted previously, is even more evident. In the final marks, all eight judging type indicators outperform their perceiving type counterparts, reinforcing further that the judging type indicator is an asset in this type of engineering design course. However, unlike in the previous discussion, the sensing–judging combination is no longer the best performing. In fact, the strongest is seen to be the ENFJ type, with S–J types (e.g., ESTJ) ranked among the top eight. It should be noted that the high performance of ENFJ type may be artificially inflated by unique strong performers, since it has the smallest sample size (population of 14) among all MBTI types. However, the ENFJ type possesses several qualities that may be advantageous in a team-based engineering design course. For instance, ENFJ's are known to be team players, excellent communicators, and strong leaders (Watson [<reflink idref="bib66" id="ref88">66</reflink>]). Arguably, they are more sensitive to the opinions of their teammates and mindful of the big picture. At the same time, they are able to make effective decisions, determine and assign tasks to team members, and establish schedules. As such, they may be able to outperform other MBTI types once the team evaluations are factored into the performance (particularly since they were fairly strong performers in the individual evaluations as well).</p> <p>This concept of particular types or traits improving the performance of a team is expanded upon in Table 15, which outlines the statistical results for the average final marks of teams based on their member distribution across the four MBTI dimensions. While the comparison of specific combinations of all four types is an interesting topic, there are 560 different combinations of three-person teams ("16 choose 3" or <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mfenced close=")" open="("><mrow><mn>16</mn><mo>,</mo><mn>3</mn></mrow></mfenced></mrow></math> </ephtml> , a combinatorial problem).</p> <p>Statistical results for team MBTI distributions</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left"><p>E/I</p></th><th align="left"><p>S/N</p></th><th align="left"><p>T/F</p></th><th align="left"><p>J/P</p></th></tr></thead><tbody><tr><td align="left"><p>Mean (aaa)</p></td><td align="left"><p>76.57% (10)</p></td><td align="left"><p>77.49% (58)</p></td><td align="left"><p>78.32% (151)</p></td><td align="left"><p>79.57% (74)</p></td></tr><tr><td align="left"><p>Mean (aab)</p></td><td align="left"><p>78.65% (73)</p></td><td align="left"><p>78.10% (145)</p></td><td align="left"><p>77.50% (162)</p></td><td align="left"><p>78.43% (161)</p></td></tr><tr><td align="left"><p>Mean (abb)</p></td><td align="left"><p>77.45% (162)</p></td><td align="left"><p>78.15% (153)</p></td><td align="left"><p>78.22% (79)</p></td><td align="left"><p>76.37% (124)</p></td></tr><tr><td align="left"><p>Mean (bbb)</p></td><td align="left"><p>78.14% (165)</p></td><td align="left"><p>77.14% (54)</p></td><td align="left"><p>76.85% (18)</p></td><td align="left"><p>77.20% (51)</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test (aaa, aab)</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test (aaa, abb)</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test (aaa, bbb)</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test (aab, abb)</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test (aab, bbb)</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test (abb, bbb)</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td></tr><tr><td align="left"><p>Cohen's <italic>d,</italic> effect size (aaa, aab)</p></td><td align="left"><p>0.36 (medium)</p></td><td align="left"><p>0.11 (small)</p></td><td align="left"><p>0.12 (small)</p></td><td align="left"><p>0.18 (small)</p></td></tr><tr><td align="left"><p>Cohen's <italic>d,</italic> effect size (aaa, abb)</p></td><td align="left"><p>0.14 (small)</p></td><td align="left"><p>0.10 (small)</p></td><td align="left"><p>0.02 (small)</p></td><td align="left"><p>0.48 (medium)</p></td></tr><tr><td align="left"><p>Cohen's <italic>d,</italic> effect size (aaa, bbb)</p></td><td align="left"><p>0.24 (medium)</p></td><td align="left"><p>0.05 (small)</p></td><td align="left"><p>0.22 (medium)</p></td><td align="left"><p>0.38 (medium)</p></td></tr><tr><td align="left"><p>Cohen's <italic>d,</italic> effect size (aab, abb)</p></td><td align="left"><p>0.19 (small)</p></td><td align="left"><p>0.01 (v. small)</p></td><td align="left"><p>0.12 (small)</p></td><td align="left"><p>0.33 (medium)</p></td></tr><tr><td align="left"><p>Cohen's <italic>d,</italic> effect size (aab, bbb)</p></td><td align="left"><p>0.11 (small)</p></td><td align="left"><p>0.14 (small)</p></td><td align="left"><p>0.25 (medium)</p></td><td align="left"><p>0.13 (small)</p></td></tr><tr><td align="left"><p>Cohen's <italic>d,</italic> effect size (abb, bbb)</p></td><td align="left"><p>0.08 (small)</p></td><td align="left"><p>0.15 (small)</p></td><td align="left"><p>0.10 (small)</p></td><td align="left"><p>0.21 (medium)</p></td></tr></tbody></table> </ephtml> </p> <p>The first column uses 'a' and 'b' to denote the first and second letter in each MBTI dimension. Numbers in parentheses show the population size</p> <p>After processing the data, it turned out that the populations of almost all combinations were too low and hence were statistically insignificant. As such, the analysis of distribution of a particular dimension of MBTI on a team (i.e., E/I, S/N, T/F, and J/P) was considered in Table 15. It should be noted that Welch's <emph>t</emph> test only indicated a significant difference between results in the J/P dimension. The effect size in other dimensions is relatively small or very small for the most part. In contrast, four out of the six J/P combinations have medium effect sizes, as noted by their Cohen's <emph>d</emph> values, and three out of six had significant difference considering the Welch's <emph>t</emph> test. The population size was also fairly large in each case, with the exception of teams with three extroversion traits or three feeling traits.</p> <p>Table 16 shows the results of the Welch's <emph>t</emph> test and Cohen's <emph>d</emph> for the J/P dimension in a table format. Teams with three judging members perform the best, which supports our observations in the individual evaluations. Interestingly, teams with two or more perceiving members demonstrate significantly worse performance. This further bolsters the hypothesis that certain qualities of judging-type students, such as the tendency towards task planning, achieving deadlines, and making quick yet effective decisions, can be advantageous. Further, teams with three perceiving members do not perform the worst overall. Instead, teams with two perceiving members and one judging member are shown to have the weakest performance. This may appear to contradict the previous hypothesis—that judgers have qualities that are assets in engineering design courses. To further understand and explain this observation, i.e., JPP performs worse than PPP, the dynamics between judging and perceiving team members should be taken into account as a potential factor. In the case where there is one judger and two perceivers, the team's dynamics can become so complex (and perhaps detrimental) that it can hinder team's performance success. One possible dynamics could be that since the judging team member sees the perceiving team member(s) as procrastinators and slow decision makers (Coe [<reflink idref="bib8" id="ref89">8</reflink>]), the judging team member takes control of the team's decision making and planning process without adequate input from the perceiving members, resulting in poorer decision making by an individual on the team. Another possible dynamic could be that the perceiving team members dominate the team, resulting in a performance that might be similar to a team with all three perceiving members. Although these are only two possible dynamics of many, a more detailed analysis of the rationale for poor performance amongst JPP teams is worthy of further investigation in future studies. As a result of these analyses, an instructor or learner may want to consider focusing on those characteristics attributed to judging types, such as task planning and quick decision making, in order to improve their performance in engineering design courses.</p> <p>Graph</p> <hd id="AN0144641886-22">Assessment mechanisms</hd> <p>Another aspect of this study is to identify whether students' performance in two specific assessment mechanisms, namely the engineering notebook and the project proposal, are correlated to their overall performance in the course. To this end, students' normalized notebook marks are plotted against their final marks using a scatter plot, along with the obtained coefficient of determination (i.e., <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mi>R</mi><mn>2</mn></msup></math> </ephtml> ) and coefficient of correlation (i.e., <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi mathvariant="italic">SSE</mi></mrow></math> </ephtml> ). The correlation between the (normalized) proposal marks and final marks are investigated in a similar fashion.</p> <p>Figure 1 shows students' notebook marks against their final marks, and the line of best fit in red. It can be seen that generally those with higher notebook marks achieve higher overall marks. This can be attributed to the fact that students with high notebook marks can log and track their calculations, meeting minutes, tasks, design iterations, etc., in greater detail and more successfully than those with lower notebook marks. Therefore, they can make use of their journaled information when preparing for the written course evaluations, such as proposal and final report. It is also worth mentioning that after individual and team evaluations, each teaching assistant communicates detailed feedback to the students that they have evaluated. It is then expected that the students will make modifications to their design or adjust their pace according to the feedback, so as to demonstrate improvement by the next evaluation. Clearly, students who do not take note of the comments provided by the teaching assistant and instructor are more likely to fail to recall the expected improvements to their design or subsystem when preparing for the upcoming evaluations.</p> <p>Graph: Fig. 1 Students' notebook marks versus final marks. SSE=28950,R2=0.52</p> <p>Figure 2 shows students' proposal marks versus their final marks. Since proposal is a team assessment (i.e., all members of a team receive the same mark for their proposal), lower dispersion of data points along the x-axis but higher dispersion along the y-axis is expected compared to an individual evaluation such as notebook (compare Fig. 2 with Fig. 1). The former is due to the fact that, as a team evaluation, it is less likely that a team will fail the proposal evaluation, since contributions of stronger members will improve the mark of weaker members. Similarly, the higher dispersion in the y-axis represents that the result of the team evaluation (e.g., proposal) does not necessarily correlate to a specific team member. The graph of Fig. 2 shows that teams with higher proposal marks also have higher final marks (with 28% of the variation accounted for, as per <emph>R</emph><sups>2</sups>). This can be attributed to the fact that a team receives a high proposal mark when the members of the team are able to clearly articulate their discussion and brainstorming on their design, project timelines, and their roles within the team. In fact, students who front-load, i.e., spend a considerable amount of time on their project at the beginning of the process, benefit greatly in their proposal, which then lead to a better overall performance in the course. They are able to better overcome course challenges, and can handle more readily unanticipated situations or design problems throughout the course.</p> <p>Graph: Fig. 2 Students' proposal marks versus final marks. SSE=39980,R2=0.28</p> <hd id="AN0144641886-23">Instructional design</hd> <p>The course structure, as described previously, has been designed with three different variations over the last two decades, namely, full-year, half-year, and front-loaded half-year courses. To begin, this section considers the shift in lecture delivery in the two half-year course formats. In the years 2007 to 2014, the lecture content was delivered progressively over the length of the course with more content delivered in the beginning of the course. Contrastingly, from 2016 to 2018, the lecture content was delivered in the first 3 weeks of the course; essentially front-loading the instructional material in the course. This instructional design change occurred without modification to the assessment mechanisms or expectations, and all practical sessions and requirements remained the same. As such, the performance of students in the two different instructional design approaches can be compared and contrasted.</p> <p>In Table 17 the mean results for each of the assessments in the two approaches are presented alongside the results of Welch's <emph>t</emph> test and Cohen's <emph>d</emph> comparing the approaches for each assessment. The results for the 3 years prior to the instructional design change in 2016 are contrasted with the subsequent 3 years, i.e., 2012–2014 versus 2016–2018. Additionally, Fig. 3 provides a bar graph of the two sets of mean results. The results clearly indicate that there was a significant improvement (as per Welch's <emph>t</emph> test) across all assessment mechanisms (with the exception of the proposal mark, which is only a slight improvement) after transitioning to a front-loaded instruction approach.</p> <p>The statistical results for various evaluations between 2011–2014 and 2016–2018</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left"><p>Mean (%)</p></th><th align="left"><p>Standard deviation (%)</p></th><th align="left"><p>Welch's <italic>t</italic> test</p></th><th align="left"><p>Cohen's <italic>d,</italic> effect size</p></th></tr></thead><tbody><tr><td align="left"><p>Interim notebook (2012–2014)</p></td><td char="." align="char"><p>70.63</p></td><td char="." align="char"><p>14.38</p></td><td char="." align="char" rowspan="2"><p>1</p></td><td align="left" rowspan="2"><p>0.34 (medium)</p></td></tr><tr><td align="left"><p>Interim notebook (2016–2018)</p></td><td char="." align="char"><p>75.46</p></td><td char="." align="char"><p>14.19</p></td></tr><tr><td align="left"><p>Final notebook (2012–2014)</p></td><td char="." align="char"><p>73.62</p></td><td char="." align="char"><p>14.13</p></td><td char="." align="char" rowspan="2"><p>1</p></td><td align="left" rowspan="2"><p>0.46 (medium)</p></td></tr><tr><td align="left"><p>Final notebook (2016–2018)</p></td><td char="." align="char"><p>79.43</p></td><td char="." align="char"><p>10.60</p></td></tr><tr><td align="left"><p>Proposal (2012–2014)</p></td><td char="." align="char"><p>76.78</p></td><td char="." align="char"><p>8.54</p></td><td char="." align="char" rowspan="2"><p>0</p></td><td align="left" rowspan="2"><p>0.09 (small)</p></td></tr><tr><td align="left"><p>Proposal (2016–2018)</p></td><td char="." align="char"><p>77.83</p></td><td char="." align="char"><p>13.82</p></td></tr><tr><td align="left"><p>Final report (2012–2014)</p></td><td char="." align="char"><p>77.96</p></td><td char="." align="char"><p>8.19</p></td><td char="." align="char" rowspan="2"><p>1</p></td><td align="left" rowspan="2"><p>0.46 (medium)</p></td></tr><tr><td align="left"><p>Final report (2016–2018)</p></td><td char="." align="char"><p>81.47</p></td><td char="." align="char"><p>7.10</p></td></tr><tr><td align="left"><p>Week 5 evaluation (2012–2014)</p></td><td char="." align="char"><p>75.91</p></td><td char="." align="char"><p>10.50</p></td><td char="." align="char" rowspan="2"><p>1</p></td><td align="left" rowspan="2"><p>0.48 (medium)</p></td></tr><tr><td align="left"><p>Week 5 evaluation (2016–2018)</p></td><td char="." align="char"><p>80.59</p></td><td char="." align="char"><p>8.88</p></td></tr><tr><td align="left"><p>Week 8 evaluation (2012–2014)</p></td><td char="." align="char"><p>76.17</p></td><td char="." align="char"><p>11.08</p></td><td char="." align="char" rowspan="2"><p>1</p></td><td align="left" rowspan="2"><p>0.39 (medium)</p></td></tr><tr><td align="left"><p>Week 8 evaluation (2016–2018)</p></td><td char="." align="char"><p>80.24</p></td><td char="." align="char"><p>9.71</p></td></tr><tr><td align="left"><p>Week 10 evaluation (2012–2014)</p></td><td char="." align="char"><p>75.40</p></td><td char="." align="char"><p>11.01</p></td><td char="." align="char" rowspan="2"><p>1</p></td><td align="left" rowspan="2"><p>0.34 (medium)</p></td></tr><tr><td align="left"><p>Week 10 evaluation (2016–2018)</p></td><td char="." align="char"><p>78.86</p></td><td char="." align="char"><p>9.17</p></td></tr><tr><td align="left"><p>Week 12 evaluation (2012–2014)</p></td><td char="." align="char"><p>75.50</p></td><td char="." align="char"><p>11.07</p></td><td char="." align="char" rowspan="2"><p>1</p></td><td align="left" rowspan="2"><p>0.28 (medium)</p></td></tr><tr><td align="left"><p>Week 12 evaluation (2016–2018)</p></td><td char="." align="char"><p>78.43</p></td><td char="." align="char"><p>10.60</p></td></tr><tr><td align="left"><p>Week 14 evaluation (2012–2014)</p></td><td char="." align="char"><p>75.44</p></td><td char="." align="char"><p>13.14</p></td><td char="." align="char" rowspan="2"><p>1</p></td><td align="left" rowspan="2"><p>0.41 (medium)</p></td></tr><tr><td align="left"><p>Week 14 evaluation (2016–2018)</p></td><td char="." align="char"><p>80.11</p></td><td char="." align="char"><p>9.17</p></td></tr><tr><td align="left"><p>Final mark (2012–2014)</p></td><td char="." align="char"><p>75.81</p></td><td char="." align="char"><p>7.53</p></td><td char="." align="char" rowspan="2"><p>1</p></td><td align="left" rowspan="2"><p>0.55 (medium)</p></td></tr><tr><td align="left"><p>Final mark (2016–2018)</p></td><td char="." align="char"><p>79.63</p></td><td char="." align="char"><p>6.13</p></td></tr></tbody></table> </ephtml> </p> <p>Graph: Fig. 3 Average marks across students for various evaluations between 2011–2014 (blue) and 2016–2018 (orange). (Color figure online)</p> <p>This was notable in all the evaluations (except for the proposal), where <emph>d</emph> values ranged from 0.28 to 0.55. The incremental improvement in the proposal marks (<emph>d</emph> value of 0.09) is likely a result of the timeframe for its submission. The proposal is the first assessment, which the students commence preparing right at the beginning of the course, submitting by the fourth week. As such, the students have had little time to construct knowledge in the laboratory environment before completing the proposal. As such, in both cases, the students have not had time to construct knowledge through contextual or relevant exercises or design problems. This result strengthens the case of constructivism as a learning model for engineering design, as it essentially demonstrates that there is not a significant difference in learning until additional time is permitted for constructing knowledge. As such, in each of the subsequent evaluations, by providing the students with the necessary scaffolding and information at the beginning of the course, it permitted more time for students to select, contextualize, apply, and integrate that information into their active design and experimentation process. This, in turn, resulted in stronger performance in all assessments, whether outcome-or process-based, individual or team. Additionally, this particular approach of front-loading the information and then providing facilitated learning opportunities via the instructors and teaching assistants, allowed students to take greater ownership of their learning, drawing from the information provided and applying it to the design process as they deemed fit, rather than being provided some of that information later on in the course (as in the 2007–2014 approach).</p> <p>In addition to the instructional design shift that occurred in 2016, there was an even more significant change in the course that occurred in 2007. As discussed previously in the article, prior to 2007 it was offered as a full-year (8 month) design course, with the first half-year (4 months) focusing on learning engineering design with hands-on experiments and assignments, and the second half-year (4 months) focusing on a design project. In 2007, however, it was compressed to a half-year (4 month) design course. The assignments intended to build hands-on skills were removed, while the technical content through lectures was maintained along with the design project. As such, several assessments between the half-and full-year courses were common, namely, the proposal, notebook evaluations, and final report. Moreover, the time allocated for students to complete the assessments and the metrics for evaluating the assessments were the same in both formats. The only exception is the proposal assessment, where students in the full-year course were given 6 weeks instead of 4 weeks (in the half-year course). However, at the same time students were expected to also complete other assignments, so the available time for completing the proposal remained fairly consistent. Thus, comparing students' performance in notebook, proposal, final report, and final mark is fairly valid, and the values are presented in Table 18.</p> <p>Average marks across students for common assignments from 2001 to 2018</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Year(s)</p></th><th align="left"><p>Population size</p></th><th align="left"><p>Proposal</p></th><th align="left"><p>Notebook</p></th><th align="left"><p>Final report</p></th><th align="left"><p>Final mark</p></th></tr></thead><tbody><tr><td align="left"><p>2001–2002</p></td><td align="left"><p>180</p></td><td align="left"><p>83.08%</p></td><td align="left"><p>77.35%</p></td><td align="left"><p>88.62%</p></td><td align="left"><p>81.18%</p></td></tr><tr><td align="left"><p>2002–2003</p></td><td align="left"><p>159</p></td><td align="left"><p>82.40%</p></td><td align="left"><p>78.40%</p></td><td align="left"><p>85.05%</p></td><td align="left"><p>80.20%</p></td></tr><tr><td align="left"><p>2003–2004</p></td><td align="left"><p>171</p></td><td align="left"><p>83.60%</p></td><td align="left"><p>77.60%</p></td><td align="left"><p>85.55%</p></td><td align="left"><p>81.31%</p></td></tr><tr><td align="left"><p>2004–2005</p></td><td align="left"><p>192</p></td><td align="left"><p>79.40%</p></td><td align="left"><p>72.00%</p></td><td align="left"><p>81.80%</p></td><td align="left"><p>76.31%</p></td></tr><tr><td align="left"><p>2005–2006</p></td><td align="left"><p>213</p></td><td align="left"><p>81.40%</p></td><td align="left"><p>74.80%</p></td><td align="left"><p>81.30%</p></td><td align="left"><p>79.24%</p></td></tr><tr><td align="left"><p>Average for FY</p></td><td align="left"><p>735</p></td><td align="left"><p>81.70%</p></td><td align="left"><p>75.70%</p></td><td align="left"><p>83.43%</p></td><td align="left"><p>79.27%</p></td></tr><tr><td align="left"><p>2007</p></td><td align="left"><p>196</p></td><td align="left"><p>83.10%</p></td><td align="left"><p>80.67%</p></td><td align="left"><p>83.15%</p></td><td align="left"><p>79.98%</p></td></tr><tr><td align="left"><p>2008</p></td><td align="left"><p>180</p></td><td align="left"><p>79.60%</p></td><td align="left"><p>78.13%</p></td><td align="left"><p>82.70%</p></td><td align="left"><p>78.01%</p></td></tr><tr><td align="left"><p>2009</p></td><td align="left"><p>159</p></td><td align="left"><p>82.00%</p></td><td align="left"><p>77.87%</p></td><td align="left"><p>79.85%</p></td><td align="left"><p>79.22%</p></td></tr><tr><td align="left"><p>2011</p></td><td align="left"><p>203</p></td><td align="left"><p>80.20%</p></td><td align="left"><p>77.87%</p></td><td align="left"><p>80.00%</p></td><td align="left"><p>77.67%</p></td></tr><tr><td align="left"><p>2012</p></td><td align="left"><p>177</p></td><td align="left"><p>77.40%</p></td><td align="left"><p>73.20%</p></td><td align="left"><p>78.75%</p></td><td align="left"><p>75.88%</p></td></tr><tr><td align="left"><p>2013</p></td><td align="left"><p>207</p></td><td align="left"><p>77.10%</p></td><td align="left"><p>71.60%</p></td><td align="left"><p>76.75%</p></td><td align="left"><p>75.10%</p></td></tr><tr><td align="left"><p>2014</p></td><td align="left"><p>205</p></td><td align="left"><p>76.00%</p></td><td align="left"><p>76.00%</p></td><td align="left"><p>78.50%</p></td><td align="left"><p>76.55%</p></td></tr><tr><td align="left"><p>2016</p></td><td align="left"><p>194</p></td><td align="left"><p>75.00%</p></td><td align="left"><p>79.20%</p></td><td align="left"><p>80.75%</p></td><td align="left"><p>78.08%</p></td></tr><tr><td align="left"><p>2017</p></td><td align="left"><p>226</p></td><td align="left"><p>80.20%</p></td><td align="left"><p>80.53%</p></td><td align="left"><p>82.25%</p></td><td align="left"><p>80.71%</p></td></tr><tr><td align="left"><p>2018</p></td><td align="left"><p>155</p></td><td align="left"><p>77.50%</p></td><td align="left"><p>70.67%</p></td><td align="left"><p>80.80%</p></td><td align="left"><p>79.15%</p></td></tr><tr><td align="left"><p>Average for HY</p></td><td align="left"><p>1902</p></td><td align="left"><p>78.81%</p></td><td align="left"><p>76.57%</p></td><td align="left"><p>80.35%</p></td><td align="left"><p>78.04%</p></td></tr><tr><td align="left"><p>Welch's <italic>t</italic> test</p></td><td align="left"><p>FY versus HY</p></td><td align="left"><p>1</p></td><td align="left"><p>0</p></td><td align="left"><p>1</p></td><td align="left"><p>1</p></td></tr><tr><td align="left"><p>Cohen's <italic>d</italic></p></td><td align="left"><p>FY versus HY</p></td><td align="left"><p>0.40</p></td><td align="left"><p>− 0.01</p></td><td align="left"><p>0.44</p></td><td align="left"><p>0.22</p></td></tr></tbody></table> </ephtml> </p> <p>It is interesting to note that in the case of the proposal, final report, and final mark, there was a statistically significant better performance in the full-year course (as per Welch's <emph>t</emph> test). This performance difference can likely be attributed to the additional time students had in the full-year course to exercise and construct knowledge through structured design assignments. In particular, the inclusion of design projects and assignment reports in the first half of the term, well-prepared students for effectively tackling the proposal and final report assessments in the second term (for more information on the design assignments and structure of the full year course, please refer to Emami [<reflink idref="bib16" id="ref90">16</reflink>]). Thus, as might be expected, the students in the half-year course, who did not have the same learning and knowledge construction opportunities, performed notably worse in the proposal and final report (as per the Cohen's <emph>d</emph> results). Conversely, there were no notebook-style process evaluations in the full-year course prior to the second term. As a result, the notebook performance in both terms is quite similar, and in fact they are not significantly different (as per Welch's <emph>t</emph> test). Although the lack of improvement in the notebook evaluations might at first seem discordant with the results for proposal and final report, it actually provides a unique insight into the relationship between knowledge construction in engineering design and assessment mechanisms. Aside from the implementation and result of engineering design experience, it can be posited that experience in specific methods and modes of expressing engineering design is significantly important. Thus, for students to effectively construct engineering design knowledge, instructors have to pay particular attention to the means of expressing the process and outcome of engineering design, and that those modes are aligned with the requirements of the engineering profession, since the constructed knowledge may not always be transferable between different applications (or in this case assessments).</p> <hd id="AN0144641886-24">Student feedback</hd> <p>In order to provide a qualitative measure of students' satisfaction with the course processes and outcomes, a short list of survey feedbacks is provided in this section. Some typical responses to the course evaluation survey include:</p> <hd id="AN0144641886-25">Team dynamics (learning styles and personality types)</hd> <p></p> <ulist> <item> "In spite of the stress our group had, we really learned a lot not just about the technical aspects of the project, but also about each other as people and how we function together as a group. We found that certain character traits we had clashed while other helped overcome seemingly impossible problems. It's really interesting how a person's true personality (good and bad) is accentuated under group work conditions!"</item> <p></p> <item> "Design definitely increased my confidence and make me really realize how important teamwork is."</item> <p></p> <item> "The most unforgettable life lesson was: you may think you are already working as hard as you can, but the results are still unsatisfactory; when this happens, look at the big picture instead of concentrating on the work at hand, maybe there is something wrong with the method, with task division, or with team member communication."</item> <p></p> <item> "I learnt a lot [of] stuff that I could not have from class, like teamwork, staying on schedule, debugging problems (circuits, pic and electromech), and dealing with stress and anxiety that I have never experienced."</item> </ulist> <hd id="AN0144641886-26">Learning process (instructional design) and outcomes</hd> <p></p> <ulist> <item> "You gave us a very high-level background of the engineering design process (with a lot of resources for each subsystem), but ultimately let us discover and make mistakes by ourselves."</item> <p></p> <item> "I can't express the amount of knowledge I gained from taking this course. Regardless of the outcome of the competition, the entire process was truly amazing for me."</item> <p></p> <item> "I will carry the many lessons I learned from this course into my professional career and personal life."</item> <p></p> <item> "I have learned more in this course than any other course I have taken before because my teammates and I were forced to figure everything out by ourselves and build a complex prototype with limited prior experience."</item> <p></p> <item> "One of my main concerns was that design did not offer the security of lecture based courses. You go to class, study, and you do well. Design is not the same. I learned how to voice my opinion when conflicting desires in the group arises. I learned how to deal with frustration in building circuits (especially making my own metal detectors). I always used to view that if I didn't get a problem right, it was my fault and I wasn't spending enough time on it. Design changed my view. I learned to grab at the essence of the problem, and did less complaining when something didn't work. I think I learned most from all the arguing I never thought I'd do with my friends. Overall, design was a profound lifetime experience."</item> <p></p> <item> "It forces us into situations that we are not yet prepared to deal with and through this we learn so much more than any theoretical lecture can provide."</item> <p></p> <item> "I think this course gave me an experience of intellectual and innovative thinking in engineering."</item> </ulist> <hd id="AN0144641886-27">Conclusion</hd> <p>This paper investigates the application of psychometric evaluations, assessment mechanisms, and instructional design approaches to the education and performance of students in a second-year engineering design course. The results indicated that there were negligible performance differences between female and male students, or with respect to gender distribution on teams, across various assessment mechanisms. The students with certain learning styles were notably successful in specific assessments that were more readily aligned with their preferred mode of learning and expression. Moreover, students with bimodal visual-kinesthetic or quadmodal learning styles were shown to be particularly apt for success in the course, with strong performance across several assessment mechanisms. The investigation of personality types revealed that certain MBTI are reasonably good indicators for performance in the course due to the natural inclinations that students with those indicators possess. Moreover, the interplay between specific indicators, such as the sensing and judging characteristics, were shown to be beneficial for certain types of assessments, namely, notebook and individual progress evaluations. Further, this interplay between characteristics seemed to further extend to the team dynamics, with certain combinations of personality types perceived as more influential towards team performance success than others. The psychometric analysis performed may help educators create assessment schemes that are less biased to specific learning styles and personality types. In addition, the results of the various personality dynamics on teams may inform team formation strategies employed in engineering design courses. The correlation and utility of certain traditional assessment mechanisms in engineering design, such as engineering notebooks and design proposals, were analyzed with respect to students' overall performance in the course. Lastly, three different instructional design methods were contrasted and compared. The front-loading of lecture content in the course proved to be particularly beneficial for the engineering design process and outcome in the course. Then, when contrasting the full-year and half-year versions of the course, the use of specific assessment mechanisms and scaffolded design activities in the first half of the full-year course demonstrated that their inclusion significantly improved student performance in similar assessments in the following semester. The instructional design approaches were also discussed in light of their implications for learning theory in engineering. For engineering design educators, the findings of this paper can inform teaching and be readily applied to the classroom and curriculum. In particular, the findings regarding front-loading of materials and scaffolded design activities are useful for instructional design and improving student performance in engineering design projects. Future work could consider the effect of specific metrics in assessments mechanisms as well as contrast performance outcomes for teams with their perceived team dynamics over the length of an engineering design course.</p> <hd id="AN0144641886-28">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0144641886-29"> <title> References </title> <blist> <bibl id="bib1" idref="ref4" type="bt">1</bibl> <bibtext> Adams R, Turns J, Atman CJ. Educating effective engineering designers: The role of reflective practice. Design Studies. 2004; 24; 3: 275-294</bibtext> </blist> <blist> <bibl id="bib2" idref="ref9" type="bt">2</bibl> <bibtext> Aragon SR, Johnson SD, Shaik N. The influence of learning style preferences on student access in online versus face-to-face environments. American Journal of Distance Education. 2002; 16; 4: 227-243</bibtext> </blist> <blist> <bibl id="bib3" idref="ref12" type="bt">3</bibl> <bibtext> Bajraktarevic, N, Hall, W, & Fullick, P. (2003). Incorporating learning styles in hypermedia environment: Empirical evaluation. In Proceedings of workshop on adaptive hypermedia and adaptive web-based systems, Budapest.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref55" type="bt">4</bibl> <bibtext> Bonva, D, & Mihova, E. (2011). Learning styles and learning preferences. In Dyslexia and additional academic language learning, Bulgaria. Delhi: Dyslexia Association.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref32" type="bt">5</bibl> <bibtext> Brickell JL, Porter DB, Reynold MF, Gosgrove RD. Assigning students to groups for engineering design projects: A comparison of five methods. Journal of Engineering Education. 1994; 83; 3: 259-262</bibtext> </blist> <blist> <bibl id="bib6" idref="ref5" type="bt">6</bibl> <bibtext> Bright, A, & Dym, C. L. (2004). General engineering at Harvey Mudd: 1957–2003. In Proceedings of ASEE conference and exhibition, Salt Lake, Utah.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref16" type="bt">7</bibl> <bibtext> Bruner J. Toward a theory of instruction. 1966: Cambridge; Harvard University Press</bibtext> </blist> <blist> <bibl id="bib8" idref="ref89" type="bt">8</bibl> <bibtext> Coe CK. The MBTI: Potential uses and misuses in personnel administration. Public Personnel Management. 1992; 21; 4: 511-522</bibtext> </blist> <blist> <bibl id="bib9" idref="ref40" type="bt">9</bibl> <bibtext> Coffield F, Moseley D, Hall E, Ecclestone K. Learning styles and pedagogy in post-16 learning: A systematic and critical review. 2004: London; Learning and Skills Research Centre</bibtext> </blist> <blist> <bibtext> Cohen J. Statistical power analysis for the behavioral sciences. 19882: New York; Lawrence Erlbaum Associates</bibtext> </blist> <blist> <bibtext> Comrey AL. An evolution of the Myers–Briggs type indicator. Academic Psychology Bulletin. 1983; 5; 1: 115-129</bibtext> </blist> <blist> <bibtext> Derrick B, Toher D, White P. Why Welch's test is type 1 error robust. The Quantitative Methods for Psychology. 2016; 12; 1: 30-38</bibtext> </blist> <blist> <bibtext> Dohaney J, Brogt E, Kennedy B. Strategies and perceptions of students' field note-taking skills: Insights from a geothermal field lesson. Journal of Geoscience Education. 2015; 63: 233-249</bibtext> </blist> <blist> <bibtext> Dutson AJ, Todd RH, Magleby SP, Sorensen CD. A review of literature on teaching design through project-oriented capstone courses. Journal of Engineering Education. 1997; 76; 1: 17-28</bibtext> </blist> <blist> <bibtext> Dym CL. Design systems and engineering education. International Journal of Engineering Education. 2014; 20; 3: 305-312</bibtext> </blist> <blist> <bibtext> Emami, M. R. (2009). Application of learning models to the engineering design pedagogy. In Proceedings of ASEE annual conference and exhibition, Austin.</bibtext> </blist> <blist> <bibtext> Emre M. The personality brokers: The strange history of Myers–Briggs and the birth of personality testing. 2018: Toronto; Random House Canada</bibtext> </blist> <blist> <bibtext> Fagerland MW, Sandvik L. Performance of five two-sample location tests for skewed distributions with unequal variances. Contemporary Clinical Trials. 2009; 30; 5: 490-496</bibtext> </blist> <blist> <bibtext> Farr JV, Lee MA, Sutton JP. Using a systematic engineering design process to conduct undergraduate engineering management capstone projects. Journal of Engineering Education. 2001; 90; 2: 193-197</bibtext> </blist> <blist> <bibtext> Felder RM, Brent R. Understanding student differences. Journal of Engineering Education. 2005; 94; 1: 57-72</bibtext> </blist> <blist> <bibtext> Fleming ND. How not to research using VARK!. 2016: Christchurch; VARK Learn Ltd.</bibtext> </blist> <blist> <bibtext> Fleming ND. Using VARK for pre-test/post-test research. 2017: Christchurch; VARK Learn Ltd.</bibtext> </blist> <blist> <bibtext> Fleming ND. VARK definitions. 2018: Christchurch; VARK Learn Ltd.</bibtext> </blist> <blist> <bibtext> Fleming ND. Using VARK. 2018: Christchurch; VARK Learn Ltd.</bibtext> </blist> <blist> <bibtext> Gerach M, Farb B, Revelle W, Nunes-Amaral LA. A robust data-driven approach identifies four personality types across four large data sets. Nature Human Behaviour. 2018; 2: 735-742</bibtext> </blist> <blist> <bibtext> Husmann PR, O'Loughlin VD. Another nail in the coffin for learning styles? Disparities among undergraduate anatomy students' study strategies, class performance, and reported VARK learning styles. Anatomical Science Education. 2019; 12; 1: 6-9</bibtext> </blist> <blist> <bibtext> James WB, Gardner DL. Learning style: Implications for distance learning. New Directions for Adult and Continuing Education. 1995; 67: 19-31</bibtext> </blist> <blist> <bibtext> Jenkins A. Company's use of psychometric testing and the changing demand for skills: A review literature. 2001: London; Centre for Economics of Education, London School of Economics and Political Science</bibtext> </blist> <blist> <bibtext> Jensen, D, Feland, J, Bowe, M, & Self, B. (2000) A 6-hats based team formation strategy: Development and comparison with an MBTI based approach. The annual meeting of the American Society for Engineering Education, ASEE. St. Louis, MO.</bibtext> </blist> <blist> <bibtext> Jensen D, Wood J, Wood K. Hands-on activities, interactive multimedia and improved team dynamics for enhancing mechanical engineering curricula. International Journal of Engineering Education. 2003; 19; 6: 874-884</bibtext> </blist> <blist> <bibtext> Jonassen DH. Objectivism versus constructivism: Do we need a new philosophical paradigm?. Educational Technology Research and Development. 1991; 66; 296: 5-14</bibtext> </blist> <blist> <bibtext> Jonassen DH. Toward a constructivist design model. Educational Technology. 1994; 34; 4: 34-37</bibtext> </blist> <blist> <bibtext> Jung C. Psychologische typen. 1921: Zürich; Rascher</bibtext> </blist> <blist> <bibtext> Kam, A. H, Chonh, H. P, Ram, R, & Goh, C. H. (2007). Personality type, learning modalities and academic performance in undergraduate engineering. In Proceedings of the 1st international conference on teaching and learning, PutraJaya.</bibtext> </blist> <blist> <bibtext> Kassim H. The relationship between learning styles, creative thinking performance and multimedia learning materials. Procedia—Social and Behavioral Sciences. 2013; 97: 229-237</bibtext> </blist> <blist> <bibtext> Kinshuk, G. S. (2007). Providing adaptive courses in learning management systems with respect to learning styles. In Proceedings of world conference on e-learning in corporate, government, healthcare, and higher education, Chesapeake, VA.</bibtext> </blist> <blist> <bibtext> Knoll AR, Otani H, Skeel RL, Horn RV. Learning style, judgements of learning, and learning of verbal and visual information. British Journal of Psychology. 2017; 108: 544-563</bibtext> </blist> <blist> <bibtext> Leite WL, Svinicki M, Shi Y. Attempted validation of the scores of the VARK: Learning styles inventory with multitrait-multimethod confirmatory factor analysis models. Educational and Psychological Measurement. 2010; 70; 2: 323-339</bibtext> </blist> <blist> <bibtext> Manochehr N-N. The influence of learning styles on learners in e-learning environments: An empirical study. Computers in Higher Education Economics Review. 2006; 18; 1: 10-14</bibtext> </blist> <blist> <bibtext> MATLAB. (2018). ttest2: Two-sample t test. MathWorks, [Online]. Available: https://<ulink href="http://www.mathworks.com/help/stats/ttest2.html">www.mathworks.com/help/stats/ttest2.html</ulink>. Accessed 01 Sept 2018.</bibtext> </blist> <blist> <bibtext> McCaulley MH, Godleski ES, Yokomoto CF, Sloan LHED. Application of psychological type in engineering education. Engineering Education. 1983; 73: 394</bibtext> </blist> <blist> <bibtext> Miller A. Personality types, learning styles, and educational goals. Educational Psychology. 1991; 11; 3: 217-238</bibtext> </blist> <blist> <bibtext> Myers IB, McCaulley MH, Quenk NL, Hammer AL. MBTI manual: A guide to the development and use of the Myers–Briggs type indicator. 19983: Palo Alto; Consulting Psychologists Press</bibtext> </blist> <blist> <bibtext> Newby T, Ertmer PA. Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly. 1993; 6; 4: 50-70</bibtext> </blist> <blist> <bibtext> O'Brien TP, Bernold LE, Akroyd D. Myers–Briggs type indicator and academic acheivement in engineering education. International Journal of Engineering Education. 1998; 14; 5: 191-199</bibtext> </blist> <blist> <bibtext> Orifici, K. A. (1990). The relationship between psychological type and the learning style preferences of graduate psychology students: Implications for training. Doctoral dissertation, Massachusetts School of Professional Psychology, Massachusetts.</bibtext> </blist> <blist> <bibtext> Oxford Dictionary of English. 2018: Oxford; Oxford University Press</bibtext> </blist> <blist> <bibtext> Peters M, Servos P, Day R. Marked sex differences on a fine motor skill task disappear when finger size is used as covariate. Journal of Applied Psychology. 1990; 75; 1: 87-90</bibtext> </blist> <blist> <bibtext> Peyman H, Sadeghifar J, Khajavikhan J, Yasemi M, Rasool M, Yaghoubi YM, Nahal MMH, Karim H. Using VARK approach for assessing preferred learning styles of first year medical sciences students: A survey from Iran. Journal of Clinical and Diagnostic Research. 2014; 8; 8: 1-4</bibtext> </blist> <blist> <bibtext> Piaget J. To understand is to invent: The future of education. 1972: New York; Viking Press Inc.</bibtext> </blist> <blist> <bibtext> Reddington, L. (2011). Gender difference variables predicting experience in lecture note-taking. Ph.D. Dissertation, Columbia University.</bibtext> </blist> <blist> <bibtext> Reddington L, Peverly S, Block C. An examination of some of the cognitive and motivation variables related to gender differences in lecture note-taking. Reading and Writing, An Interdisciplinary Journal. 2015; 29; 8: 1155-1185</bibtext> </blist> <blist> <bibtext> Reigeluth C. Elaborating the elaboration theory. Educational Technology Research and Development. 1992; 40; 3: 80-86</bibtext> </blist> <blist> <bibtext> Rogowsky BA, Calhoun BM, Tallal P. Matching learning style to instructional method: Effects on comprehension. Journal of Educational Psychology. 2015; 107; 1: 64-78</bibtext> </blist> <blist> <bibtext> Rosati P. Academic progress of Canadian engineering students in terms of MBTI personality type. International Journal of Engineering Education. 1998; 14; 5: 322-327</bibtext> </blist> <blist> <bibtext> Sawilowsky SS. New effect size rules of thumb. Journal of Modern Applied Statistical Methods. 2009; 8; 2: 597-599</bibtext> </blist> <blist> <bibtext> Schwier RAAnglin GJ. Issues in emerging interactive technologies. Instructional technology: Past, present, and future. 1995: Englewood; Libraries Unlimited Inc.: 119-127</bibtext> </blist> <blist> <bibtext> Scott C. An enduring appeal of "learning styles". Australian Journal of Education. 2010; 54; 1: 5-17</bibtext> </blist> <blist> <bibtext> Sheppard S, Jenison R. Examples of freshman design education. International Journal of Engineering Education. 1997; 13; 4: 248-261</bibtext> </blist> <blist> <bibtext> Sipps GJ, Alexander RA, Friedt L. Item analysis of the Myers–Briggs type indicator. Educational and Psychological Measurement. 1985; 45; 4: 789-796</bibtext> </blist> <blist> <bibtext> Skinner BF. The behaviour of organisms: An experimental analysis. 1938: New York; Appleton-Century Inc.</bibtext> </blist> <blist> <bibtext> The Myers & Briggs Foundation. MBTI basics. MBTI, [Online]. Available: https://<ulink href="http://www.myersbriggs.org/my-mbti-personality-type/mbti-basics/home.htm?bhcp=1">www.myersbriggs.org/my-mbti-personality-type/mbti-basics/home.htm?bhcp=1</ulink>. Accessed 01 Aug 2018.</bibtext> </blist> <blist> <bibtext> von Glasersfeld EWatzlawick P. An introduction to radical constructivism. The invented reality. 1984: New York; W. W. Norton & Company: 17-40</bibtext> </blist> <blist> <bibtext> von Glasersfeld ESteffe L, Gale J. A constructivist approach to teaching. Constructivism in education. 1995: Hilsdale; Lawrence Erlbaum Associates Inc, Erlbaum: 3-16</bibtext> </blist> <blist> <bibtext> Vygotsky L. Mind in society: The development of higher psychological processes. 1978: Harward; Harward University Press</bibtext> </blist> <blist> <bibtext> Watson L. Self-leadership: Becoming an exceptional leader. Radiologic Technology. 2004; 75; 6: 457-470</bibtext> </blist> <blist> <bibtext> Welch BL. The significance of the difference between two means when the population variances are unequal. Biometrika. 1938; 29; 3/4: 350-362</bibtext> </blist> </ref> <aug> <p>By M. Reza Emami; Michael C. F. Bazzocchi and Houman Hakima</p> <p>Reported by Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib15" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib14" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib19" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib16" firstref="ref6"></nolink> <nolink nlid="nl5" bibid="bib53" firstref="ref7"></nolink> <nolink nlid="nl6" bibid="bib50" firstref="ref17"></nolink> <nolink nlid="nl7" bibid="bib65" firstref="ref18"></nolink> <nolink nlid="nl8" bibid="bib64" firstref="ref19"></nolink> <nolink nlid="nl9" bibid="bib61" firstref="ref20"></nolink> <nolink nlid="nl10" bibid="bib31" firstref="ref21"></nolink> <nolink nlid="nl11" bibid="bib63" firstref="ref22"></nolink> <nolink nlid="nl12" bibid="bib32" firstref="ref23"></nolink> <nolink nlid="nl13" bibid="bib44" firstref="ref25"></nolink> <nolink nlid="nl14" bibid="bib57" firstref="ref26"></nolink> <nolink nlid="nl15" bibid="bib59" firstref="ref27"></nolink> <nolink nlid="nl16" bibid="bib35" firstref="ref31"></nolink> <nolink nlid="nl17" bibid="bib29" firstref="ref33"></nolink> <nolink nlid="nl18" bibid="bib30" firstref="ref34"></nolink> <nolink nlid="nl19" bibid="bib39" firstref="ref35"></nolink> <nolink nlid="nl20" bibid="bib49" firstref="ref37"></nolink> <nolink nlid="nl21" bibid="bib28" firstref="ref38"></nolink> <nolink nlid="nl22" bibid="bib27" firstref="ref39"></nolink> <nolink nlid="nl23" bibid="bib36" firstref="ref42"></nolink> <nolink nlid="nl24" bibid="bib47" firstref="ref43"></nolink> <nolink nlid="nl25" bibid="bib55" firstref="ref44"></nolink> <nolink nlid="nl26" bibid="bib20" firstref="ref45"></nolink> <nolink nlid="nl27" bibid="bib41" firstref="ref46"></nolink> <nolink nlid="nl28" bibid="bib45" firstref="ref47"></nolink> <nolink nlid="nl29" bibid="bib46" firstref="ref48"></nolink> <nolink nlid="nl30" bibid="bib33" firstref="ref49"></nolink> <nolink nlid="nl31" bibid="bib42" firstref="ref50"></nolink> <nolink nlid="nl32" bibid="bib62" firstref="ref51"></nolink> <nolink nlid="nl33" bibid="bib43" firstref="ref56"></nolink> <nolink nlid="nl34" bibid="bib11" firstref="ref57"></nolink> <nolink nlid="nl35" bibid="bib60" firstref="ref58"></nolink> <nolink nlid="nl36" bibid="bib34" firstref="ref59"></nolink> <nolink nlid="nl37" bibid="bib17" firstref="ref60"></nolink> <nolink nlid="nl38" bibid="bib25" firstref="ref61"></nolink> <nolink nlid="nl39" bibid="bib38" firstref="ref62"></nolink> <nolink nlid="nl40" bibid="bib23" firstref="ref63"></nolink> <nolink nlid="nl41" bibid="bib26" firstref="ref67"></nolink> <nolink nlid="nl42" bibid="bib37" firstref="ref68"></nolink> <nolink nlid="nl43" bibid="bib54" firstref="ref69"></nolink> <nolink nlid="nl44" bibid="bib58" firstref="ref70"></nolink> <nolink nlid="nl45" bibid="bib21" firstref="ref71"></nolink> <nolink nlid="nl46" bibid="bib22" firstref="ref72"></nolink> <nolink nlid="nl47" bibid="bib24" firstref="ref73"></nolink> <nolink nlid="nl48" bibid="bib12" firstref="ref76"></nolink> <nolink nlid="nl49" bibid="bib67" firstref="ref77"></nolink> <nolink nlid="nl50" bibid="bib40" firstref="ref78"></nolink> <nolink nlid="nl51" bibid="bib18" firstref="ref79"></nolink> <nolink nlid="nl52" bibid="bib10" firstref="ref80"></nolink> <nolink nlid="nl53" bibid="bib56" firstref="ref81"></nolink> <nolink nlid="nl54" bibid="bib52" firstref="ref84"></nolink> <nolink nlid="nl55" bibid="bib13" firstref="ref85"></nolink> <nolink nlid="nl56" bibid="bib48" firstref="ref86"></nolink> <nolink nlid="nl57" bibid="bib51" firstref="ref87"></nolink> <nolink nlid="nl58" bibid="bib66" firstref="ref88"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1261104
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Engineering Design Pedagogy: A Performance Analysis
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Emami%2C+M%2E+Reza%22">Emami, M. Reza</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0003-4977-6339">0000-0003-4977-6339</externalLink>)<br /><searchLink fieldCode="AR" term="%22Bazzocchi%2C+Michael+C%2E+F%2E%22">Bazzocchi, Michael C. F.</searchLink><br /><searchLink fieldCode="AR" term="%22Hakima%2C+Houman%22">Hakima, Houman</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Journal+of+Technology+and+Design+Education%22"><i>International Journal of Technology and Design Education</i></searchLink>. Jul 2020 30(3):553-585.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 33
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2020
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22Engineering+Education%22">Engineering Education</searchLink><br /><searchLink fieldCode="DE" term="%22Design%22">Design</searchLink><br /><searchLink fieldCode="DE" term="%22Personality+Measures%22">Personality Measures</searchLink><br /><searchLink fieldCode="DE" term="%22Personality+Traits%22">Personality Traits</searchLink><br /><searchLink fieldCode="DE" term="%22Cognitive+Style%22">Cognitive Style</searchLink><br /><searchLink fieldCode="DE" term="%22Gender+Differences%22">Gender Differences</searchLink><br /><searchLink fieldCode="DE" term="%22Academic+Achievement%22">Academic Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Predictor+Variables%22">Predictor Variables</searchLink><br /><searchLink fieldCode="DE" term="%22Instructional+Design%22">Instructional Design</searchLink>
– Name: SubjectThesaurus
  Label: Assessment and Survey Identifiers
  Group: Su
  Data: <searchLink fieldCode="SU" term="%22Myers+Briggs+Type+Indicator%22">Myers Briggs Type Indicator</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s10798-019-09515-7
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0957-7572
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Cornerstone design courses have become a major part of engineering curricula, where students with different personality types and learning styles work together to design, develop, build, and demonstrate the functionality of a prototype within the duration of a term. This study analyzes student and team performance against gender, personality types, and learning styles in a second-year engineering design course. Further, the correlations between several assessment mechanisms are studied, and the effects of three different instructional design approaches on students' performance are explored. Data have been collected on student performance and psychometrics, including marks, gender, personality type, and learning style from 2001 to 2018. To identify students' personality types and learning styles, Myers-Briggs Type Indicators (MBTI) and Neil Fleming's Learning VARK tests were administered. To evaluate students' performance in the course, a number of assessment mechanisms have been defined. Several statistical methods are used to analyze data, and to determine correlation between datasets. Over nearly two decades of marks, gender, MBTI, and VARK data for 2637 students are presented for an engineering design course. The results demonstrated that there was no significant difference in performance across most assessments based on gender or gender distribution on a team. A better performance was observed from VK bimodal and quadmodal learning styles in most assessment mechanisms. Further, certain MBTI groups, namely, judging types outperformed their peers in engineering design assessments, with interesting interplay between MBTI dimensions for specific assessments and team dynamics. Traditional assessment mechanisms, such as engineering notebook and design proposals, are shown to be good predictors of student success. Lastly, scaffolded design activities and front-loading of lecture content were shown to be beneficial for student learning. There is negligible performance difference between female and male students in the engineering design course. Students whose preferred learning styles align with the assessment themes showed better performance in the course. The outcomes of this paper can be readily applied by instructors for design of assessment mechanisms, course materials, team formation, and instructional design.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2020
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1261104
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1261104
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10798-019-09515-7
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 33
        StartPage: 553
    Subjects:
      – SubjectFull: Undergraduate Students
        Type: general
      – SubjectFull: Engineering Education
        Type: general
      – SubjectFull: Design
        Type: general
      – SubjectFull: Personality Measures
        Type: general
      – SubjectFull: Personality Traits
        Type: general
      – SubjectFull: Cognitive Style
        Type: general
      – SubjectFull: Gender Differences
        Type: general
      – SubjectFull: Academic Achievement
        Type: general
      – SubjectFull: Predictor Variables
        Type: general
      – SubjectFull: Instructional Design
        Type: general
      – SubjectFull: Myers Briggs Type Indicator
        Type: general
    Titles:
      – TitleFull: Engineering Design Pedagogy: A Performance Analysis
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Emami, M. Reza
      – PersonEntity:
          Name:
            NameFull: Bazzocchi, Michael C. F.
      – PersonEntity:
          Name:
            NameFull: Hakima, Houman
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 07
              Type: published
              Y: 2020
          Identifiers:
            – Type: issn-print
              Value: 0957-7572
          Numbering:
            – Type: volume
              Value: 30
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: International Journal of Technology and Design Education
              Type: main
ResultId 1