Observing Response Processes with Eye Tracking in International Large-Scale Assessments: Evidence from the OECD PIAAC Assessment
Saved in:
| Title: | Observing Response Processes with Eye Tracking in International Large-Scale Assessments: Evidence from the OECD PIAAC Assessment |
|---|---|
| Language: | English |
| Authors: | Maddox, Bryan (ORCID |
| Source: | European Journal of Psychology of Education. Jul 2018 33(3):543-558. |
| Availability: | Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/ |
| Peer Reviewed: | Y |
| Page Count: | 16 |
| Publication Date: | 2018 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Secondary Education |
| Descriptors: | Achievement Tests, Foreign Countries, International Assessment, Adults, Competence, Adult Literacy, Pilot Projects, Eye Movements, Test Items, Item Response Theory, Physiology, Responses, Measurement Techniques, Computer Assisted Testing, Problem Solving, Ability |
| DOI: | 10.1007/s10212-018-0380-2 |
| ISSN: | 0256-2928 |
| Abstract: | This paper reports on a pilot study that used eye tracking techniques to make detailed observations of item response processes in the OECD Programme for the International Assessment of Adult Competencies (PIAAC). The lab-based study also recorded physiological responses using measures of pupil diameter and electrodermal activity. The study tested 14 adult respondents as they individually completed the PIAAC computer-based assessment. The eye tracking observations help to fill an 'explanatory gap' by providing data on variation in item response processes that are not captured by other sources of process data such as think aloud protocols or computer-generated log files. The data on fixations and saccades provided detailed information on test item response strategies, enabling profiling of respondent engagement and response processes associated with successful performance. Much of that activity does not include the use of the keyboard and mouse, and involves 'off-screen' use of pen and paper (and calculator) that are not captured by assessment log-files. In conclusion, this paper points toward an important application of eye tracking in large-scale assessments. This includes insights into response processes in new domains such as adaptive problem-solving that aim to identify individuals' ability to select and combine resources from the digital and physical environment. |
| Abstractor: | As Provided |
| Number of References: | 39 |
| Entry Date: | 2018 |
| Accession Number: | EJ1181841 |
| Database: | ERIC |
| Abstract: | This paper reports on a pilot study that used eye tracking techniques to make detailed observations of item response processes in the OECD Programme for the International Assessment of Adult Competencies (PIAAC). The lab-based study also recorded physiological responses using measures of pupil diameter and electrodermal activity. The study tested 14 adult respondents as they individually completed the PIAAC computer-based assessment. The eye tracking observations help to fill an 'explanatory gap' by providing data on variation in item response processes that are not captured by other sources of process data such as think aloud protocols or computer-generated log files. The data on fixations and saccades provided detailed information on test item response strategies, enabling profiling of respondent engagement and response processes associated with successful performance. Much of that activity does not include the use of the keyboard and mouse, and involves 'off-screen' use of pen and paper (and calculator) that are not captured by assessment log-files. In conclusion, this paper points toward an important application of eye tracking in large-scale assessments. This includes insights into response processes in new domains such as adaptive problem-solving that aim to identify individuals' ability to select and combine resources from the digital and physical environment. |
|---|---|
| ISSN: | 0256-2928 |
| DOI: | 10.1007/s10212-018-0380-2 |
