View in EDS

Observing Response Processes with Eye Tracking in International Large-Scale Assessments: Evidence from the OECD PIAAC Assessment

Saved in:
Bibliographic Details
Title: Observing Response Processes with Eye Tracking in International Large-Scale Assessments: Evidence from the OECD PIAAC Assessment
Language: English
Authors: Maddox, Bryan (ORCID 0000-0002-3886-4504), Bayliss, Andrew P., Fleming, Piers, Engelhardt, Paul E., Edwards, S. Gareth, Borgonovi, Francesca
Source: European Journal of Psychology of Education. Jul 2018 33(3):543-558.
Availability: Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 16
Publication Date: 2018
Document Type: Journal Articles
Reports - Research
Education Level: Secondary Education
Descriptors: Achievement Tests, Foreign Countries, International Assessment, Adults, Competence, Adult Literacy, Pilot Projects, Eye Movements, Test Items, Item Response Theory, Physiology, Responses, Measurement Techniques, Computer Assisted Testing, Problem Solving, Ability
DOI: 10.1007/s10212-018-0380-2
ISSN: 0256-2928
Abstract: This paper reports on a pilot study that used eye tracking techniques to make detailed observations of item response processes in the OECD Programme for the International Assessment of Adult Competencies (PIAAC). The lab-based study also recorded physiological responses using measures of pupil diameter and electrodermal activity. The study tested 14 adult respondents as they individually completed the PIAAC computer-based assessment. The eye tracking observations help to fill an 'explanatory gap' by providing data on variation in item response processes that are not captured by other sources of process data such as think aloud protocols or computer-generated log files. The data on fixations and saccades provided detailed information on test item response strategies, enabling profiling of respondent engagement and response processes associated with successful performance. Much of that activity does not include the use of the keyboard and mouse, and involves 'off-screen' use of pen and paper (and calculator) that are not captured by assessment log-files. In conclusion, this paper points toward an important application of eye tracking in large-scale assessments. This includes insights into response processes in new domains such as adaptive problem-solving that aim to identify individuals' ability to select and combine resources from the digital and physical environment.
Abstractor: As Provided
Number of References: 39
Entry Date: 2018
Accession Number: EJ1181841
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1181841
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Observing Response Processes with Eye Tracking in International Large-Scale Assessments: Evidence from the OECD PIAAC Assessment
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Maddox%2C+Bryan%22">Maddox, Bryan</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-3886-4504">0000-0002-3886-4504</externalLink>)<br /><searchLink fieldCode="AR" term="%22Bayliss%2C+Andrew+P%2E%22">Bayliss, Andrew P.</searchLink><br /><searchLink fieldCode="AR" term="%22Fleming%2C+Piers%22">Fleming, Piers</searchLink><br /><searchLink fieldCode="AR" term="%22Engelhardt%2C+Paul+E%2E%22">Engelhardt, Paul E.</searchLink><br /><searchLink fieldCode="AR" term="%22Edwards%2C+S%2E+Gareth%22">Edwards, S. Gareth</searchLink><br /><searchLink fieldCode="AR" term="%22Borgonovi%2C+Francesca%22">Borgonovi, Francesca</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22European+Journal+of+Psychology+of+Education%22"><i>European Journal of Psychology of Education</i></searchLink>. Jul 2018 33(3):543-558.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 16
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2018
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Achievement+Tests%22">Achievement Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22International+Assessment%22">International Assessment</searchLink><br /><searchLink fieldCode="DE" term="%22Adults%22">Adults</searchLink><br /><searchLink fieldCode="DE" term="%22Competence%22">Competence</searchLink><br /><searchLink fieldCode="DE" term="%22Adult+Literacy%22">Adult Literacy</searchLink><br /><searchLink fieldCode="DE" term="%22Pilot+Projects%22">Pilot Projects</searchLink><br /><searchLink fieldCode="DE" term="%22Eye+Movements%22">Eye Movements</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Response+Theory%22">Item Response Theory</searchLink><br /><searchLink fieldCode="DE" term="%22Physiology%22">Physiology</searchLink><br /><searchLink fieldCode="DE" term="%22Responses%22">Responses</searchLink><br /><searchLink fieldCode="DE" term="%22Measurement+Techniques%22">Measurement Techniques</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Problem+Solving%22">Problem Solving</searchLink><br /><searchLink fieldCode="DE" term="%22Ability%22">Ability</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s10212-018-0380-2
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0256-2928
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This paper reports on a pilot study that used eye tracking techniques to make detailed observations of item response processes in the OECD Programme for the International Assessment of Adult Competencies (PIAAC). The lab-based study also recorded physiological responses using measures of pupil diameter and electrodermal activity. The study tested 14 adult respondents as they individually completed the PIAAC computer-based assessment. The eye tracking observations help to fill an 'explanatory gap' by providing data on variation in item response processes that are not captured by other sources of process data such as think aloud protocols or computer-generated log files. The data on fixations and saccades provided detailed information on test item response strategies, enabling profiling of respondent engagement and response processes associated with successful performance. Much of that activity does not include the use of the keyboard and mouse, and involves 'off-screen' use of pen and paper (and calculator) that are not captured by assessment log-files. In conclusion, this paper points toward an important application of eye tracking in large-scale assessments. This includes insights into response processes in new domains such as adaptive problem-solving that aim to identify individuals' ability to select and combine resources from the digital and physical environment.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 39
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2018
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1181841
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1181841
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10212-018-0380-2
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 16
        StartPage: 543
    Subjects:
      – SubjectFull: Achievement Tests
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: International Assessment
        Type: general
      – SubjectFull: Adults
        Type: general
      – SubjectFull: Competence
        Type: general
      – SubjectFull: Adult Literacy
        Type: general
      – SubjectFull: Pilot Projects
        Type: general
      – SubjectFull: Eye Movements
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Item Response Theory
        Type: general
      – SubjectFull: Physiology
        Type: general
      – SubjectFull: Responses
        Type: general
      – SubjectFull: Measurement Techniques
        Type: general
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Problem Solving
        Type: general
      – SubjectFull: Ability
        Type: general
    Titles:
      – TitleFull: Observing Response Processes with Eye Tracking in International Large-Scale Assessments: Evidence from the OECD PIAAC Assessment
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Maddox, Bryan
      – PersonEntity:
          Name:
            NameFull: Bayliss, Andrew P.
      – PersonEntity:
          Name:
            NameFull: Fleming, Piers
      – PersonEntity:
          Name:
            NameFull: Engelhardt, Paul E.
      – PersonEntity:
          Name:
            NameFull: Edwards, S. Gareth
      – PersonEntity:
          Name:
            NameFull: Borgonovi, Francesca
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 07
              Type: published
              Y: 2018
          Identifiers:
            – Type: issn-print
              Value: 0256-2928
          Numbering:
            – Type: volume
              Value: 33
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: European Journal of Psychology of Education
              Type: main
ResultId 1