Measuring the Amorphous: Substantive and Methodological Outcomes from Concept Maps
Saved in:
| Title: | Measuring the Amorphous: Substantive and Methodological Outcomes from Concept Maps |
|---|---|
| Language: | English |
| Authors: | Foley, Rider W. (ORCID |
| Source: | Journal of Engineering Education. Jan 2021 110(1):161-183. |
| Availability: | Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us |
| Peer Reviewed: | Y |
| Page Count: | 23 |
| Publication Date: | 2021 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | Undergraduate Students, Engineering Education, Student Evaluation, Concept Mapping, Concept Formation, Problem Solving, Thinking Skills, Evaluation Methods, Accreditation (Institutions) |
| DOI: | 10.1002/jee.20373 |
| ISSN: | 1069-4730 |
| Abstract: | Background: Accreditation organizations broadened program assessment criteria in ways that present challenges for the evaluation of learning outcomes. This is especially the case in courses where there are not narrowly defined questions or definitive solutions, such as engineering ethics. While protocols for learning assessment exist, there is limited research exploring sociotechnical learning outcomes in a manner that combines theoretical, empirical, and procedural aspects of assessment. Purpose/Hypothesis: This paper shares 3 years of research into the effectiveness of concept maps as an assessment tool for engineering students in courses that prioritize professional skills development and sociotechnical thinking. We show that concept maps can offer evidence of knowledge formation and learning outcomes associated with courses that introduce complex problems with multiple possible interpretations or viable solutions. Design/Method: A concept mapping activity was completed by 614 undergraduate engineering students at the start and end of three different courses to evaluate sociotechnical thinking. Student-level longitudinal changes were evaluated using paired t tests and simple linear regression. The concept mapping activity was improved iteratively in response to pilot tests and focus groups. Results: Undergraduate engineers demonstrated greater complexity in the representation of sociotechnical relationships as observed in the structure and content of the concept maps. The methodological results offer lessons about the administration and analysis of concept maps often excluded from conversations on course and curriculum level assessment. Conclusions: Concept maps offer insights into student learning outcomes, and they can be deployed and analyzed with minimal resources. However, assessments must be carefully designed to account for administrative and analytical challenges. |
| Abstractor: | As Provided |
| Entry Date: | 2021 |
| Accession Number: | EJ1286729 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Background: Accreditation organizations broadened program assessment criteria in ways that present challenges for the evaluation of learning outcomes. This is especially the case in courses where there are not narrowly defined questions or definitive solutions, such as engineering ethics. While protocols for learning assessment exist, there is limited research exploring sociotechnical learning outcomes in a manner that combines theoretical, empirical, and procedural aspects of assessment. Purpose/Hypothesis: This paper shares 3 years of research into the effectiveness of concept maps as an assessment tool for engineering students in courses that prioritize professional skills development and sociotechnical thinking. We show that concept maps can offer evidence of knowledge formation and learning outcomes associated with courses that introduce complex problems with multiple possible interpretations or viable solutions. Design/Method: A concept mapping activity was completed by 614 undergraduate engineering students at the start and end of three different courses to evaluate sociotechnical thinking. Student-level longitudinal changes were evaluated using paired t tests and simple linear regression. The concept mapping activity was improved iteratively in response to pilot tests and focus groups. Results: Undergraduate engineers demonstrated greater complexity in the representation of sociotechnical relationships as observed in the structure and content of the concept maps. The methodological results offer lessons about the administration and analysis of concept maps often excluded from conversations on course and curriculum level assessment. Conclusions: Concept maps offer insights into student learning outcomes, and they can be deployed and analyzed with minimal resources. However, assessments must be carefully designed to account for administrative and analytical challenges. |
|---|---|
| ISSN: | 1069-4730 |
| DOI: | 10.1002/jee.20373 |
