View in EDS

Investigating Evidence in Support of Validity and Reliability for Data Collected with the Meaningful Learning in the Laboratory Instrument (MLLI)

Saved in:
Bibliographic Details
Title: Investigating Evidence in Support of Validity and Reliability for Data Collected with the Meaningful Learning in the Laboratory Instrument (MLLI)
Language: English
Authors: Elizabeth B. Vaughan (ORCID 0009-0005-6688-0287), A. Montoya-Cowan, Jack Barbera (ORCID 0000-0003-3887-3301)
Source: Chemistry Education Research and Practice. 2024 25(1):313-326.
Availability: Royal Society of Chemistry. Thomas Graham House, Science Park, Milton Road, Cambridge, CB4 0WF, UK. Tel: +44-1223 420066; Fax: +44-1223 423623; e-mail: cerp@rsc.org; Web site: http://www.rsc.org/cerp
Peer Reviewed: Y
Page Count: 14
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Test Validity, Test Reliability, Expectation, Measures (Individuals), Science Laboratories, Factor Structure, Goodness of Fit, Undergraduate Students, Data Interpretation, Chemistry, College Science, Learning
Geographic Terms: Oregon (Portland)
DOI: 10.1039/d3rp00121k
ISSN: 1756-1108
Abstract: The Meaningful Learning in the Laboratory Instrument (MLLI) was designed to measure students' expectations before and after their laboratory courses and experiences. Although the MLLI has been used in various studies and laboratory environments to investigate students' cognitive and affective laboratory expectations, the authors of the instrument reported a discrepancy between the intended factor structure of the MLLI and the factor structure suggested by the data collected in preliminary studies. Therefore, the aim of this study was to investigate the validity and reliability evidence related to data collected with the MLLI, especially that related to structural validity. Evidence to support structural validity would provide greater meaning for the reporting and interpretation of MLLI scores. In this study, two possible "a priori" models for the factor structure of data collected from multiple institutions with the MLLI were investigated using confirmatory factory analysis (CFA). This initial investigation found poor data-model fit for each of the two tested models. Cognitive interviews and free response items were then used to inform modifications to the two "a priori structures," and a third alternative structure, which included a negative method factor, was also investigated. Once a best fitting model was identified, further model revisions were informed by a combination of modification indices and qualitative data. Evidence of adequate-to-good data model fit was found for the final revised version of the MLLI, deemed the MLLIv2. Additionally, evidence of both internal structure validity and single administration reliability were found for each of the MLLIv2 factors. The structure of the data from these items leads to scale scores that likely represent student expectations that contribute to meaningful learning and student expectations that detract from meaningful learning. As the results of this study provide the first psychometrically supported scales for MLLI data, they have implications on the future reporting and analyses of MLLI scores.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1417834
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1417834
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Investigating Evidence in Support of Validity and Reliability for Data Collected with the Meaningful Learning in the Laboratory Instrument (MLLI)
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Elizabeth+B%2E+Vaughan%22">Elizabeth B. Vaughan</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0005-6688-0287">0009-0005-6688-0287</externalLink>)<br /><searchLink fieldCode="AR" term="%22A%2E+Montoya-Cowan%22">A. Montoya-Cowan</searchLink><br /><searchLink fieldCode="AR" term="%22Jack+Barbera%22">Jack Barbera</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3887-3301">0000-0003-3887-3301</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Chemistry+Education+Research+and+Practice%22"><i>Chemistry Education Research and Practice</i></searchLink>. 2024 25(1):313-326.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Royal Society of Chemistry. Thomas Graham House, Science Park, Milton Road, Cambridge, CB4 0WF, UK. Tel: +44-1223 420066; Fax: +44-1223 423623; e-mail: cerp@rsc.org; Web site: http://www.rsc.org/cerp
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 14
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Expectation%22">Expectation</searchLink><br /><searchLink fieldCode="DE" term="%22Measures+%28Individuals%29%22">Measures (Individuals)</searchLink><br /><searchLink fieldCode="DE" term="%22Science+Laboratories%22">Science Laboratories</searchLink><br /><searchLink fieldCode="DE" term="%22Factor+Structure%22">Factor Structure</searchLink><br /><searchLink fieldCode="DE" term="%22Goodness+of+Fit%22">Goodness of Fit</searchLink><br /><searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Interpretation%22">Data Interpretation</searchLink><br /><searchLink fieldCode="DE" term="%22Chemistry%22">Chemistry</searchLink><br /><searchLink fieldCode="DE" term="%22College+Science%22">College Science</searchLink><br /><searchLink fieldCode="DE" term="%22Learning%22">Learning</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Oregon+%28Portland%29%22">Oregon (Portland)</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1039/d3rp00121k
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1756-1108
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The Meaningful Learning in the Laboratory Instrument (MLLI) was designed to measure students' expectations before and after their laboratory courses and experiences. Although the MLLI has been used in various studies and laboratory environments to investigate students' cognitive and affective laboratory expectations, the authors of the instrument reported a discrepancy between the intended factor structure of the MLLI and the factor structure suggested by the data collected in preliminary studies. Therefore, the aim of this study was to investigate the validity and reliability evidence related to data collected with the MLLI, especially that related to structural validity. Evidence to support structural validity would provide greater meaning for the reporting and interpretation of MLLI scores. In this study, two possible "a priori" models for the factor structure of data collected from multiple institutions with the MLLI were investigated using confirmatory factory analysis (CFA). This initial investigation found poor data-model fit for each of the two tested models. Cognitive interviews and free response items were then used to inform modifications to the two "a priori structures," and a third alternative structure, which included a negative method factor, was also investigated. Once a best fitting model was identified, further model revisions were informed by a combination of modification indices and qualitative data. Evidence of adequate-to-good data model fit was found for the final revised version of the MLLI, deemed the MLLIv2. Additionally, evidence of both internal structure validity and single administration reliability were found for each of the MLLIv2 factors. The structure of the data from these items leads to scale scores that likely represent student expectations that contribute to meaningful learning and student expectations that detract from meaningful learning. As the results of this study provide the first psychometrically supported scales for MLLI data, they have implications on the future reporting and analyses of MLLI scores.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1417834
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1417834
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1039/d3rp00121k
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 14
        StartPage: 313
    Subjects:
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: Expectation
        Type: general
      – SubjectFull: Measures (Individuals)
        Type: general
      – SubjectFull: Science Laboratories
        Type: general
      – SubjectFull: Factor Structure
        Type: general
      – SubjectFull: Goodness of Fit
        Type: general
      – SubjectFull: Undergraduate Students
        Type: general
      – SubjectFull: Data Interpretation
        Type: general
      – SubjectFull: Chemistry
        Type: general
      – SubjectFull: College Science
        Type: general
      – SubjectFull: Learning
        Type: general
      – SubjectFull: Oregon (Portland)
        Type: general
    Titles:
      – TitleFull: Investigating Evidence in Support of Validity and Reliability for Data Collected with the Meaningful Learning in the Laboratory Instrument (MLLI)
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Elizabeth B. Vaughan
      – PersonEntity:
          Name:
            NameFull: A. Montoya-Cowan
      – PersonEntity:
          Name:
            NameFull: Jack Barbera
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-electronic
              Value: 1756-1108
          Numbering:
            – Type: volume
              Value: 25
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Chemistry Education Research and Practice
              Type: main
ResultId 1