View in EDS

Learning as It Happens: A Decade of Analyzing and Shaping a Large-Scale Online Learning System

Saved in:
Bibliographic Details
Title: Learning as It Happens: A Decade of Analyzing and Shaping a Large-Scale Online Learning System
Language: English
Authors: Brinkhuis, Matthieu J. S. (ORCID 0000-0003-1054-6683), Savi, Alexander O. (ORCID 0000-0002-9271-7476), Hofman, Abe D. (ORCID 0000-0003-4269-5296), Coomans, Frederik, van der Maas, Han L. J., Maris, Gunter
Source: Journal of Learning Analytics. 2018 5(2):29-46.
Availability: Society for Learning Analytics Research. 121 Pointe Marsan, Beaumont, AB T4X 0A2, Canada. Tel: +61-429-920-838; e-mail: info@solaresearch.org; Web site: http://learning-analytics.info/journals/index.php/JLA/
Peer Reviewed: Y
Page Count: 18
Publication Date: 2018
Document Type: Journal Articles
Reports - Research
Education Level: Elementary Education
Descriptors: Data Analysis, Mathematics Instruction, Accuracy, Reaction Time, Scoring, Difficulty Level, Prediction, Item Analysis, Goodness of Fit, Achievement Gains, Mathematics Achievement, Elementary School Students, Foreign Countries, Computer Assisted Testing, Mathematics Tests, Computer Assisted Instruction, Arithmetic
Geographic Terms: Netherlands
ISSN: 1929-7750
Abstract: With the advent of computers in education, and the ample availability of online learning and practice environments, enormous amounts of data on learning become available. The purpose of this paper is to present a decade of experience with analyzing and improving an online practice environment for math, which has thus far recorded over a billion responses. We present the methods we use to both steer and analyze this system in real-time, using scoring rules on accuracy and response times, a tailored rating system to provide both learners and items with current ability and difficulty ratings, and an adaptive engine that matches learners to items. Moreover, we explore the quality of fit by means of prediction accuracy and parallel item reliability. Limitations and pitfalls are discussed by diagnosing sources of misfit, like violations of unidimensionality and unforeseen dynamics. Finally, directions for development are discussed, including embedded learning analytics and a focus on online experimentation to evaluate both the system itself and the users' learning gains. Though many challenges remain open, we believe that large steps have been made in providing methods to efficiently manage and research educational big data from a massive online learning system.
Abstractor: As Provided
Number of References: 46
Entry Date: 2018
Accession Number: EJ1187391
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1187391
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1187391
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Learning as It Happens: A Decade of Analyzing and Shaping a Large-Scale Online Learning System
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Brinkhuis%2C+Matthieu+J%2E+S%2E%22">Brinkhuis, Matthieu J. S.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-1054-6683">0000-0003-1054-6683</externalLink>)<br /><searchLink fieldCode="AR" term="%22Savi%2C+Alexander+O%2E%22">Savi, Alexander O.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-9271-7476">0000-0002-9271-7476</externalLink>)<br /><searchLink fieldCode="AR" term="%22Hofman%2C+Abe+D%2E%22">Hofman, Abe D.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-4269-5296">0000-0003-4269-5296</externalLink>)<br /><searchLink fieldCode="AR" term="%22Coomans%2C+Frederik%22">Coomans, Frederik</searchLink><br /><searchLink fieldCode="AR" term="%22van+der+Maas%2C+Han+L%2E+J%2E%22">van der Maas, Han L. J.</searchLink><br /><searchLink fieldCode="AR" term="%22Maris%2C+Gunter%22">Maris, Gunter</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Learning+Analytics%22"><i>Journal of Learning Analytics</i></searchLink>. 2018 5(2):29-46.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Society for Learning Analytics Research. 121 Pointe Marsan, Beaumont, AB T4X 0A2, Canada. Tel: +61-429-920-838; e-mail: info@solaresearch.org; Web site: http://learning-analytics.info/journals/index.php/JLA/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 18
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2018
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Data+Analysis%22">Data Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics+Instruction%22">Mathematics Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Accuracy%22">Accuracy</searchLink><br /><searchLink fieldCode="DE" term="%22Reaction+Time%22">Reaction Time</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring%22">Scoring</searchLink><br /><searchLink fieldCode="DE" term="%22Difficulty+Level%22">Difficulty Level</searchLink><br /><searchLink fieldCode="DE" term="%22Prediction%22">Prediction</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Analysis%22">Item Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Goodness+of+Fit%22">Goodness of Fit</searchLink><br /><searchLink fieldCode="DE" term="%22Achievement+Gains%22">Achievement Gains</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics+Achievement%22">Mathematics Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+School+Students%22">Elementary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics+Tests%22">Mathematics Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Instruction%22">Computer Assisted Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Arithmetic%22">Arithmetic</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Netherlands%22">Netherlands</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1929-7750
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: With the advent of computers in education, and the ample availability of online learning and practice environments, enormous amounts of data on learning become available. The purpose of this paper is to present a decade of experience with analyzing and improving an online practice environment for math, which has thus far recorded over a billion responses. We present the methods we use to both steer and analyze this system in real-time, using scoring rules on accuracy and response times, a tailored rating system to provide both learners and items with current ability and difficulty ratings, and an adaptive engine that matches learners to items. Moreover, we explore the quality of fit by means of prediction accuracy and parallel item reliability. Limitations and pitfalls are discussed by diagnosing sources of misfit, like violations of unidimensionality and unforeseen dynamics. Finally, directions for development are discussed, including embedded learning analytics and a focus on online experimentation to evaluate both the system itself and the users' learning gains. Though many challenges remain open, we believe that large steps have been made in providing methods to efficiently manage and research educational big data from a massive online learning system.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 46
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2018
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1187391
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1187391
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 18
        StartPage: 29
    Subjects:
      – SubjectFull: Data Analysis
        Type: general
      – SubjectFull: Mathematics Instruction
        Type: general
      – SubjectFull: Accuracy
        Type: general
      – SubjectFull: Reaction Time
        Type: general
      – SubjectFull: Scoring
        Type: general
      – SubjectFull: Difficulty Level
        Type: general
      – SubjectFull: Prediction
        Type: general
      – SubjectFull: Item Analysis
        Type: general
      – SubjectFull: Goodness of Fit
        Type: general
      – SubjectFull: Achievement Gains
        Type: general
      – SubjectFull: Mathematics Achievement
        Type: general
      – SubjectFull: Elementary School Students
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Mathematics Tests
        Type: general
      – SubjectFull: Computer Assisted Instruction
        Type: general
      – SubjectFull: Arithmetic
        Type: general
      – SubjectFull: Netherlands
        Type: general
    Titles:
      – TitleFull: Learning as It Happens: A Decade of Analyzing and Shaping a Large-Scale Online Learning System
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Brinkhuis, Matthieu J. S.
      – PersonEntity:
          Name:
            NameFull: Savi, Alexander O.
      – PersonEntity:
          Name:
            NameFull: Hofman, Abe D.
      – PersonEntity:
          Name:
            NameFull: Coomans, Frederik
      – PersonEntity:
          Name:
            NameFull: van der Maas, Han L. J.
      – PersonEntity:
          Name:
            NameFull: Maris, Gunter
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2018
          Identifiers:
            – Type: issn-electronic
              Value: 1929-7750
          Numbering:
            – Type: volume
              Value: 5
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Journal of Learning Analytics
              Type: main
ResultId 1