View in EDS

Systematically Reviewing the Rigour of Immersive Virtual Reality Research in STEM Education: A Deep Dive Into Threats to Internal Validity.

Saved in:
Bibliographic Details
Title: Systematically Reviewing the Rigour of Immersive Virtual Reality Research in STEM Education: A Deep Dive Into Threats to Internal Validity.
Authors: Martella, Amédee Marchand (AUTHOR), Lawson, Alyssa P. (AUTHOR), Martella, Ronald C. (AUTHOR), LaBonte, Kristen (AUTHOR), Zhao, Fangzheng (AUTHOR), Delgado, Cynthia Y. (AUTHOR), Munns, Mitchell E. (AUTHOR), LeRoy, Ashleigh Wells (AUTHOR), Gluck, Justin A. (AUTHOR), Mayer, Richard E. (AUTHOR)
Source: Journal of Computer Assisted Learning. Feb2026, Vol. 42 Issue 1, p1-31. 31p.
Subjects: Statistical power analysis, Computer simulation, Research funding, Research evaluation, Sample size (Statistics), Educational tests & measurements, Descriptive statistics, Analysis of covariance, Learning, Virtual reality, Systematic reviews, MEDLINE, Experimental design, Research bias, One-way analysis of variance, Research methodology, Online information services, ERIC (Information retrieval system), Psychology information storage & retrieval systems
Abstract: Background: There have been many initiatives focused on implementing IVR in the classroom to either replace or supplement conventional instruction. The efficacy of these initiatives is often informed by IVR media comparison studies that examine the learning outcomes of students who learn academic content in IVR versus conventional media. However, the current methodological rigour of research on the use of IVR for learning in STEM education has not yet been established to ensure recommendations concerning its use in education are rooted in rigorous research studies. Objective: The present review aimed to fill this gap in the literature by evaluating published journal articles, conference proceedings, and dissertations (between the years 2013 and 2023) related to IVR comparison studies in STEM education. Method: The 44 articles were evaluated with respect to 13 internal validity controls (i.e., research design, justified sample size, group equivalence, attrition equivalence, operational definitions provided, matched dosage, matched timeframe, matched content, implementation fidelity, practice equivalence with dependent measures [DMs], equivalent DMs, interrater agreement for DM scoring and technical adequacy of DMs). Results: Results indicated there were no articles that explicitly met all 13 internal validity controls; the average number of control issues was 5.36. When articles were not penalised for missing information, 21 articles adhered to all internal validity controls. As can be seen when comparing these findings, the lack of methodological information was a glaring problem. Conclusion: These findings indicate that there are both methodological problems and reporting problems in IVR media comparison research that need to be addressed to advance future IVR research. Recommendations for productively moving forward are discussed. Lay Summary: Internal validity is necessary to establish for methodically rigorous research studies but has not been assessed in IVR literature.Controlling for and/or acknowledging threats to internal validity can aid in the interpretation of results.Most articles evaluated were missing critical information about internal validity.IVR research in STEM education has many areas of strengths but also many areas in need of improvement to advance knowledge on the effective use of IVR in education. Practitioner Notes: What is already known about this topic ○IVR is being investigated as a learning tool in STEM courses.○The literature on the use of IVR for learning outcomes in STEM courses is mixed.○The methodological rigour of this research has not been yet established.What this paper adds ○Studies on the learning impacts of IVR in STEM courses were evaluated with respect to 13 internal validity controls.○This research identified that a high percentage of articles did not meet the internal validity controls.○Many studies demonstrated a lack of methodological information contained within the manuscript.Implications for practice and/or policy ○Studies on the impact of IVR on STEM learning need to take into account internal validity controls to ensure the literature base is strong.○More transparency is necessary in publishing research to ensure that researchers are not violating internal validity controls. [ABSTRACT FROM AUTHOR]
Copyright of Journal of Computer Assisted Learning is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Psychology and Behavioral Sciences Collection
FullText Text:
  Availability: 0
Header DbId: pbh
DbLabel: Psychology and Behavioral Sciences Collection
An: 191181617
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Systematically Reviewing the Rigour of Immersive Virtual Reality Research in STEM Education: A Deep Dive Into Threats to Internal Validity.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Martella%2C+Amédee+Marchand%22">Martella, Amédee Marchand</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Lawson%2C+Alyssa+P%2E%22">Lawson, Alyssa P.</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Martella%2C+Ronald+C%2E%22">Martella, Ronald C.</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22LaBonte%2C+Kristen%22">LaBonte, Kristen</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Zhao%2C+Fangzheng%22">Zhao, Fangzheng</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Delgado%2C+Cynthia+Y%2E%22">Delgado, Cynthia Y.</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Munns%2C+Mitchell+E%2E%22">Munns, Mitchell E.</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22LeRoy%2C+Ashleigh+Wells%22">LeRoy, Ashleigh Wells</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Gluck%2C+Justin+A%2E%22">Gluck, Justin A.</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Mayer%2C+Richard+E%2E%22">Mayer, Richard E.</searchLink> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Journal+of+Computer+Assisted+Learning%22">Journal of Computer Assisted Learning</searchLink>. Feb2026, Vol. 42 Issue 1, p1-31. 31p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Statistical+power+analysis%22">Statistical power analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+simulation%22">Computer simulation</searchLink><br /><searchLink fieldCode="DE" term="%22Research+funding%22">Research funding</searchLink><br /><searchLink fieldCode="DE" term="%22Research+evaluation%22">Research evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Sample+size+%28Statistics%29%22">Sample size (Statistics)</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+tests+%26+measurements%22">Educational tests & measurements</searchLink><br /><searchLink fieldCode="DE" term="%22Descriptive+statistics%22">Descriptive statistics</searchLink><br /><searchLink fieldCode="DE" term="%22Analysis+of+covariance%22">Analysis of covariance</searchLink><br /><searchLink fieldCode="DE" term="%22Learning%22">Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Virtual+reality%22">Virtual reality</searchLink><br /><searchLink fieldCode="DE" term="%22Systematic+reviews%22">Systematic reviews</searchLink><br /><searchLink fieldCode="DE" term="%22MEDLINE%22">MEDLINE</searchLink><br /><searchLink fieldCode="DE" term="%22Experimental+design%22">Experimental design</searchLink><br /><searchLink fieldCode="DE" term="%22Research+bias%22">Research bias</searchLink><br /><searchLink fieldCode="DE" term="%22One-way+analysis+of+variance%22">One-way analysis of variance</searchLink><br /><searchLink fieldCode="DE" term="%22Research+methodology%22">Research methodology</searchLink><br /><searchLink fieldCode="DE" term="%22Online+information+services%22">Online information services</searchLink><br /><searchLink fieldCode="DE" term="%22ERIC+%28Information+retrieval+system%29%22">ERIC (Information retrieval system)</searchLink><br /><searchLink fieldCode="DE" term="%22Psychology+information+storage+%26+retrieval+systems%22">Psychology information storage & retrieval systems</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: There have been many initiatives focused on implementing IVR in the classroom to either replace or supplement conventional instruction. The efficacy of these initiatives is often informed by IVR media comparison studies that examine the learning outcomes of students who learn academic content in IVR versus conventional media. However, the current methodological rigour of research on the use of IVR for learning in STEM education has not yet been established to ensure recommendations concerning its use in education are rooted in rigorous research studies. Objective: The present review aimed to fill this gap in the literature by evaluating published journal articles, conference proceedings, and dissertations (between the years 2013 and 2023) related to IVR comparison studies in STEM education. Method: The 44 articles were evaluated with respect to 13 internal validity controls (i.e., research design, justified sample size, group equivalence, attrition equivalence, operational definitions provided, matched dosage, matched timeframe, matched content, implementation fidelity, practice equivalence with dependent measures [DMs], equivalent DMs, interrater agreement for DM scoring and technical adequacy of DMs). Results: Results indicated there were no articles that explicitly met all 13 internal validity controls; the average number of control issues was 5.36. When articles were not penalised for missing information, 21 articles adhered to all internal validity controls. As can be seen when comparing these findings, the lack of methodological information was a glaring problem. Conclusion: These findings indicate that there are both methodological problems and reporting problems in IVR media comparison research that need to be addressed to advance future IVR research. Recommendations for productively moving forward are discussed. Lay Summary: Internal validity is necessary to establish for methodically rigorous research studies but has not been assessed in IVR literature.Controlling for and/or acknowledging threats to internal validity can aid in the interpretation of results.Most articles evaluated were missing critical information about internal validity.IVR research in STEM education has many areas of strengths but also many areas in need of improvement to advance knowledge on the effective use of IVR in education. Practitioner Notes: What is already known about this topic ○IVR is being investigated as a learning tool in STEM courses.○The literature on the use of IVR for learning outcomes in STEM courses is mixed.○The methodological rigour of this research has not been yet established.What this paper adds ○Studies on the learning impacts of IVR in STEM courses were evaluated with respect to 13 internal validity controls.○This research identified that a high percentage of articles did not meet the internal validity controls.○Many studies demonstrated a lack of methodological information contained within the manuscript.Implications for practice and/or policy ○Studies on the impact of IVR on STEM learning need to take into account internal validity controls to ensure the literature base is strong.○More transparency is necessary in publishing research to ensure that researchers are not violating internal validity controls. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Journal of Computer Assisted Learning is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=pbh&AN=191181617
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1002/jcal.70165
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 31
        StartPage: 1
    Subjects:
      – SubjectFull: Statistical power analysis
        Type: general
      – SubjectFull: Computer simulation
        Type: general
      – SubjectFull: Research funding
        Type: general
      – SubjectFull: Research evaluation
        Type: general
      – SubjectFull: Sample size (Statistics)
        Type: general
      – SubjectFull: Educational tests & measurements
        Type: general
      – SubjectFull: Descriptive statistics
        Type: general
      – SubjectFull: Analysis of covariance
        Type: general
      – SubjectFull: Learning
        Type: general
      – SubjectFull: Virtual reality
        Type: general
      – SubjectFull: Systematic reviews
        Type: general
      – SubjectFull: MEDLINE
        Type: general
      – SubjectFull: Experimental design
        Type: general
      – SubjectFull: Research bias
        Type: general
      – SubjectFull: One-way analysis of variance
        Type: general
      – SubjectFull: Research methodology
        Type: general
      – SubjectFull: Online information services
        Type: general
      – SubjectFull: ERIC (Information retrieval system)
        Type: general
      – SubjectFull: Psychology information storage & retrieval systems
        Type: general
    Titles:
      – TitleFull: Systematically Reviewing the Rigour of Immersive Virtual Reality Research in STEM Education: A Deep Dive Into Threats to Internal Validity.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Martella, Amédee Marchand
      – PersonEntity:
          Name:
            NameFull: Lawson, Alyssa P.
      – PersonEntity:
          Name:
            NameFull: Martella, Ronald C.
      – PersonEntity:
          Name:
            NameFull: LaBonte, Kristen
      – PersonEntity:
          Name:
            NameFull: Zhao, Fangzheng
      – PersonEntity:
          Name:
            NameFull: Delgado, Cynthia Y.
      – PersonEntity:
          Name:
            NameFull: Munns, Mitchell E.
      – PersonEntity:
          Name:
            NameFull: LeRoy, Ashleigh Wells
      – PersonEntity:
          Name:
            NameFull: Gluck, Justin A.
      – PersonEntity:
          Name:
            NameFull: Mayer, Richard E.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 02
              Text: Feb2026
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-print
              Value: 02664909
          Numbering:
            – Type: volume
              Value: 42
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Journal of Computer Assisted Learning
              Type: main
ResultId 1