View in EDS

Semisupervised Learning Method to Adjust Biased Item Difficulty Estimates Caused by Nonignorable Missingness in a Virtual Learning Environment

Saved in:
Bibliographic Details
Title: Semisupervised Learning Method to Adjust Biased Item Difficulty Estimates Caused by Nonignorable Missingness in a Virtual Learning Environment
Language: English
Authors: Xue, Kang (ORCID 0000-0003-2161-6931), Huggins-Manley, Anne Corinne, Leite, Walter (ORCID 0000-0001-7655-5668)
Source: Educational and Psychological Measurement. Jun 2022 82(3):539-567.
Availability: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Peer Reviewed: Y
Page Count: 29
Publication Date: 2022
Sponsoring Agency: Institute of Education Sciences (ED)
Contract Number: R305C160004
Document Type: Journal Articles
Reports - Research
Descriptors: Virtual Classrooms, Artificial Intelligence, Item Response Theory, Item Analysis, Testing Programs, Man Machine Systems, Data Analysis, Academic Ability, Response Style (Tests), Test Items, Difficulty Level, Student Behavior, Testing
Geographic Terms: Florida
DOI: 10.1177/00131644211020494
ISSN: 0013-1644
1552-3888
Abstract: In data collected from virtual learning environments (VLEs), item response theory (IRT) models can be used to guide the ongoing measurement of student ability. However, such applications of IRT rely on unbiased item parameter estimates associated with test items in the VLE. Without formal piloting of the items, one can expect a large amount of nonignorable missing data in the VLE log file data, and this is expected to negatively affect IRT item parameter estimation accuracy, which then negatively affects any future ability estimates utilized in the VLE. In the psychometric literature, methods for handling missing data have been studied mostly around conditions in which the data and the amount of missing data are not as large as those that come from VLEs. In this article, we introduce a semisupervised learning method to deal with a large proportion of missingness contained in VLE data from which one needs to obtain unbiased item parameter estimates. First, we explored the factors relating to the missing data. Then we implemented a semisupervised learning method under the two-parameter logistic IRT model to estimate the latent abilities of students. Last, we applied two adjustment methods designed to reduce bias in item parameter estimates. The proposed framework showed its potential for obtaining unbiased item parameter estimates that can then be fixed in the VLE in order to obtain ongoing ability estimates for operational purposes.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2022
Accession Number: EJ1336691
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1336691
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1336691
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Semisupervised Learning Method to Adjust Biased Item Difficulty Estimates Caused by Nonignorable Missingness in a Virtual Learning Environment
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Xue%2C+Kang%22">Xue, Kang</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-2161-6931">0000-0003-2161-6931</externalLink>)<br /><searchLink fieldCode="AR" term="%22Huggins-Manley%2C+Anne+Corinne%22">Huggins-Manley, Anne Corinne</searchLink><br /><searchLink fieldCode="AR" term="%22Leite%2C+Walter%22">Leite, Walter</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-7655-5668">0000-0001-7655-5668</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Educational+and+Psychological+Measurement%22"><i>Educational and Psychological Measurement</i></searchLink>. Jun 2022 82(3):539-567.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 29
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2022
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305C160004
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Virtual+Classrooms%22">Virtual Classrooms</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Response+Theory%22">Item Response Theory</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Analysis%22">Item Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Testing+Programs%22">Testing Programs</searchLink><br /><searchLink fieldCode="DE" term="%22Man+Machine+Systems%22">Man Machine Systems</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Analysis%22">Data Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Academic+Ability%22">Academic Ability</searchLink><br /><searchLink fieldCode="DE" term="%22Response+Style+%28Tests%29%22">Response Style (Tests)</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Difficulty+Level%22">Difficulty Level</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Behavior%22">Student Behavior</searchLink><br /><searchLink fieldCode="DE" term="%22Testing%22">Testing</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Florida%22">Florida</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1177/00131644211020494
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0013-1644<br />1552-3888
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In data collected from virtual learning environments (VLEs), item response theory (IRT) models can be used to guide the ongoing measurement of student ability. However, such applications of IRT rely on unbiased item parameter estimates associated with test items in the VLE. Without formal piloting of the items, one can expect a large amount of nonignorable missing data in the VLE log file data, and this is expected to negatively affect IRT item parameter estimation accuracy, which then negatively affects any future ability estimates utilized in the VLE. In the psychometric literature, methods for handling missing data have been studied mostly around conditions in which the data and the amount of missing data are not as large as those that come from VLEs. In this article, we introduce a semisupervised learning method to deal with a large proportion of missingness contained in VLE data from which one needs to obtain unbiased item parameter estimates. First, we explored the factors relating to the missing data. Then we implemented a semisupervised learning method under the two-parameter logistic IRT model to estimate the latent abilities of students. Last, we applied two adjustment methods designed to reduce bias in item parameter estimates. The proposed framework showed its potential for obtaining unbiased item parameter estimates that can then be fixed in the VLE in order to obtain ongoing ability estimates for operational purposes.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2022
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1336691
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1336691
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/00131644211020494
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 29
        StartPage: 539
    Subjects:
      – SubjectFull: Virtual Classrooms
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Item Response Theory
        Type: general
      – SubjectFull: Item Analysis
        Type: general
      – SubjectFull: Testing Programs
        Type: general
      – SubjectFull: Man Machine Systems
        Type: general
      – SubjectFull: Data Analysis
        Type: general
      – SubjectFull: Academic Ability
        Type: general
      – SubjectFull: Response Style (Tests)
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Difficulty Level
        Type: general
      – SubjectFull: Student Behavior
        Type: general
      – SubjectFull: Testing
        Type: general
      – SubjectFull: Florida
        Type: general
    Titles:
      – TitleFull: Semisupervised Learning Method to Adjust Biased Item Difficulty Estimates Caused by Nonignorable Missingness in a Virtual Learning Environment
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Xue, Kang
      – PersonEntity:
          Name:
            NameFull: Huggins-Manley, Anne Corinne
      – PersonEntity:
          Name:
            NameFull: Leite, Walter
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 06
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-print
              Value: 0013-1644
            – Type: issn-electronic
              Value: 1552-3888
          Numbering:
            – Type: volume
              Value: 82
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Educational and Psychological Measurement
              Type: main
ResultId 1