View in EDS

Integrating Topic Modeling and LLM Prompt Engineering into a Human-Driven Approach to Analyze Interview Transcripts

Saved in:
Bibliographic Details
Title: Integrating Topic Modeling and LLM Prompt Engineering into a Human-Driven Approach to Analyze Interview Transcripts
Language: English
Authors: Teresa M. Ober, Karyssa A. Courey, Michael Flor
Source: Journal of Educational Data Mining. 2026 18(1):156-179.
Availability: International Educational Data Mining. e-mail: jedm.editor@gmail.com; Web site: https://jedm.educationaldatamining.org/index.php/JEDM
Peer Reviewed: Y
Page Count: 24
Publication Date: 2026
Document Type: Journal Articles
Reports - Research
Descriptors: Models, Natural Language Processing, Prompting, Interviews, Transcripts (Written Records), Artificial Intelligence, Man Machine Systems, Digital Literacy, Best Practices
ISSN: 2157-2100
Abstract: Topic modeling has become a widely used unsupervised machine learning method for extracting latent themes from large textual datasets. However, the interpretability of these themes often relies heavily on human judgment, which can limit transparency and reproducibility. Recent advances in large language models (LLMs) and prompt engineering offer new opportunities to enhance the interpretability and scalability of topic modeling outputs. This study presents a hybrid, human-in-the-loop methodological framework that integrates topic modeling, LLM prompting, and human-derived codes to support rigorous qualitative analysis. We apply this framework to focus group interviews with 13 U.S. teachers discussing the conceptualization and assessment of communication and digital literacy skills within competency-based education (CBE) contexts. The multi-stage process includes semantic clustering, LLM-assisted topic labeling, and iterative codebook refinement, enabling both scale and interpretive depth. Our findings demonstrate that this approach supports construct alignment, thematic stability, and methodological transparency, while preserving the contextual richness of qualitative data. We also highlight the importance of human oversight in guiding LLM outputs and ensuring theoretical coherence. This work contributes to emerging best practices for integrating AI tools into qualitative educational research by offering a replicable approach for analyzing complex, open-ended data that maintains both scalability and interpretability. The framework demonstrates how computational tools can augment human interpretive expertise while maintaining the epistemological integrity essential to qualitative inquiry. Supplemental materials are available at: https://doi.org/10.17605/osf.io/4q6w8.
Abstractor: As Provided
Entry Date: 2026
Accession Number: EJ1506379
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1506379
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1506379
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Integrating Topic Modeling and LLM Prompt Engineering into a Human-Driven Approach to Analyze Interview Transcripts
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Teresa+M%2E+Ober%22">Teresa M. Ober</searchLink><br /><searchLink fieldCode="AR" term="%22Karyssa+A%2E+Courey%22">Karyssa A. Courey</searchLink><br /><searchLink fieldCode="AR" term="%22Michael+Flor%22">Michael Flor</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Educational+Data+Mining%22"><i>Journal of Educational Data Mining</i></searchLink>. 2026 18(1):156-179.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: International Educational Data Mining. e-mail: jedm.editor@gmail.com; Web site: https://jedm.educationaldatamining.org/index.php/JEDM
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 24
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2026
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Models%22">Models</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Prompting%22">Prompting</searchLink><br /><searchLink fieldCode="DE" term="%22Interviews%22">Interviews</searchLink><br /><searchLink fieldCode="DE" term="%22Transcripts+%28Written+Records%29%22">Transcripts (Written Records)</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Man+Machine+Systems%22">Man Machine Systems</searchLink><br /><searchLink fieldCode="DE" term="%22Digital+Literacy%22">Digital Literacy</searchLink><br /><searchLink fieldCode="DE" term="%22Best+Practices%22">Best Practices</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2157-2100
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Topic modeling has become a widely used unsupervised machine learning method for extracting latent themes from large textual datasets. However, the interpretability of these themes often relies heavily on human judgment, which can limit transparency and reproducibility. Recent advances in large language models (LLMs) and prompt engineering offer new opportunities to enhance the interpretability and scalability of topic modeling outputs. This study presents a hybrid, human-in-the-loop methodological framework that integrates topic modeling, LLM prompting, and human-derived codes to support rigorous qualitative analysis. We apply this framework to focus group interviews with 13 U.S. teachers discussing the conceptualization and assessment of communication and digital literacy skills within competency-based education (CBE) contexts. The multi-stage process includes semantic clustering, LLM-assisted topic labeling, and iterative codebook refinement, enabling both scale and interpretive depth. Our findings demonstrate that this approach supports construct alignment, thematic stability, and methodological transparency, while preserving the contextual richness of qualitative data. We also highlight the importance of human oversight in guiding LLM outputs and ensuring theoretical coherence. This work contributes to emerging best practices for integrating AI tools into qualitative educational research by offering a replicable approach for analyzing complex, open-ended data that maintains both scalability and interpretability. The framework demonstrates how computational tools can augment human interpretive expertise while maintaining the epistemological integrity essential to qualitative inquiry. Supplemental materials are available at: https://doi.org/10.17605/osf.io/4q6w8.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1506379
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1506379
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 24
        StartPage: 156
    Subjects:
      – SubjectFull: Models
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Prompting
        Type: general
      – SubjectFull: Interviews
        Type: general
      – SubjectFull: Transcripts (Written Records)
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Man Machine Systems
        Type: general
      – SubjectFull: Digital Literacy
        Type: general
      – SubjectFull: Best Practices
        Type: general
    Titles:
      – TitleFull: Integrating Topic Modeling and LLM Prompt Engineering into a Human-Driven Approach to Analyze Interview Transcripts
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Teresa M. Ober
      – PersonEntity:
          Name:
            NameFull: Karyssa A. Courey
      – PersonEntity:
          Name:
            NameFull: Michael Flor
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-electronic
              Value: 2157-2100
          Numbering:
            – Type: volume
              Value: 18
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Journal of Educational Data Mining
              Type: main
ResultId 1