View in EDS HTML Full Text PDF Full Text

Revealing Teaching Quality through Lesson Semantics: A GPT-Assisted Analysis of Transcripts

Saved in:
Bibliographic Details
Title: Revealing Teaching Quality through Lesson Semantics: A GPT-Assisted Analysis of Transcripts
Language: English
Authors: Richard Göllner (ORCID 0000-0002-9442-7616), Rebecca Lazarides (ORCID 0000-0003-0392-4981), Philipp Stark
Source: British Journal of Educational Psychology. 2025 95(1):300-315.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 16
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Descriptors: Teacher Effectiveness, Educational Quality, Semantics, Artificial Intelligence, Technology Uses in Education, Transcripts (Written Records), Lesson Plans, Educational Indicators
DOI: 10.1111/bjep.70001
ISSN: 0007-0998
2044-8279
Abstract: Background: Existing conceptions of teaching quality assume that classroom interactions serve as the foundation for effective teaching. The resulting data necessitates analytical approaches capable of extracting the semantics of these interactions. Aim: This study investigates whether and to what extent lesson semantics provide insights into teaching quality (i.e., cognitive engagement, encouragement and warmth, multiple approaches, and the nature of discourse). To achieve this, GPT-4 was applied as a tool for analysing lesson transcripts. Sample: The study is based on data from the TALIS Video study, which included N = 50 teachers delivering two consecutive mathematics lessons in 9th grade. Teaching quality was annotated by trained observers across multiple dimensions. Method: The analysis involved embedding segmented lesson transcripts to examine their semantic characteristics and associations with human annotations of teaching quality. Additionally, we applied content-informed prompting to evaluate the interpretability of semantic characteristics for the considered dimensions. Results: GPT-4 identified five distinct semantic representations of transcripts, varying at both the teacher and lesson levels. These representations were related to teaching quality, accounting for up to 20% of variance in teaching quality annotations. Content-informed prompting aligned lesson segments more closely with semantic representations, supporting their interpretability. Conclusion: The findings suggest that lesson semantics serve as indicators of teaching quality, offering a promising approach to understanding effective classroom learning.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1483482
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFxTKBIax2iixoZLPy7Zl75AAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDHT8z4klbbkXv_nNHAIBEICBm8HoN3_SETzMEKcrmgVy0_Ru7Rfmofqn-M07T1Nph8KKqsyCcb6H_I_XfFrSKBkxJ-Zed88vEn53siNKu2nzY10W6C0rtyFycwJ7mk-x5p5lSrp-7Z2SioD-socla4yoOFAyFnz1Hrj4oUxIwLqxyo5Omu3cHQaB5P-nlHMNi8tPLLW0DBpOGIiQyv4O0eE-NV5OnahToqU5bVBj
Text:
  Availability: 1
  Value: <anid>AN0187948735;6kx02sep.25;2025Sep16.03:13;v2.2.500</anid> <title id="AN0187948735-1">Revealing teaching quality through lesson semantics: A GPT‐assisted analysis of transcripts </title> <p>Background: Existing conceptions of teaching quality assume that classroom interactions serve as the foundation for effective teaching. The resulting data necessitates analytical approaches capable of extracting the semantics of these interactions. Aim: This study investigates whether and to what extent lesson semantics provide insights into teaching quality (i.e., cognitive engagement, encouragement and warmth, multiple approaches, and the nature of discourse). To achieve this, GPT‐4 was applied as a tool for analysing lesson transcripts. Sample: The study is based on data from the TALIS Video study, which included N = 50 teachers delivering two consecutive mathematics lessons in 9th grade. Teaching quality was annotated by trained observers across multiple dimensions. Method: The analysis involved embedding segmented lesson transcripts to examine their semantic characteristics and associations with human annotations of teaching quality. Additionally, we applied content‐informed prompting to evaluate the interpretability of semantic characteristics for the considered dimensions. Results: GPT‐4 identified five distinct semantic representations of transcripts, varying at both the teacher and lesson levels. These representations were related to teaching quality, accounting for up to 20% of variance in teaching quality annotations. Content‐informed prompting aligned lesson segments more closely with semantic representations, supporting their interpretability. Conclusion: The findings suggest that lesson semantics serve as indicators of teaching quality, offering a promising approach to understanding effective classroom learning.</p> <p>Keywords: GPT‐4; lesson transcripts; mathematics lessons; teaching quality</p> <p>Teaching quality is a crucial factor in explaining students' motivation and learning achievements. This holds true even when considering students' learning backgrounds and other relevant characteristics related to the school environment and individual students (Bell et al., [<reflink idref="bib5" id="ref1">5</reflink>]; Kane & Staiger, [<reflink idref="bib25" id="ref2">25</reflink>]; Klette & Blikstad‐Balas, [<reflink idref="bib27" id="ref3">27</reflink>]).</p> <p>Scholars agree that the essence of teaching quality revolves around classroom interactions between teachers and students, which should be directly related to the learning content, foster a supportive learning environment, and encourage cognitive engagement (CE) (e.g., Pianta et al., [<reflink idref="bib39" id="ref4">39</reflink>]). How teachers structure and shape discourse, provide feedback, and frame instructional content is assumed to directly influence students' comprehension and engagement during classroom learning (Alqassab & León, [<reflink idref="bib1" id="ref5">1</reflink>]; Falcon et al., [<reflink idref="bib15" id="ref6">15</reflink>]; Putwain et al., [<reflink idref="bib40" id="ref7">40</reflink>]). However, these complex interactions result in challenging data that demand new analytical methods beyond those currently used in teaching quality research. Newly available technologies offer promising opportunities for analysing lesson semantics and their relevance to key dimensions of teaching quality.</p> <p>In this study, we used GPT‐4 to investigate whether and to what extent lesson semantics represent teaching quality in terms of well‐known quality dimensions. Drawing upon the German TALIS Video study (Grünkorn et al., [<reflink idref="bib21" id="ref8">21</reflink>]), we examined whether semantic differences exist between teachers and their lessons and explored the extent to which lesson semantics correlate with human annotations of teaching quality. As with other exploratory approaches, interpreting the results is a crucial step. Consequently, we further applied a procedure to more directly assess whether the identified semantic representations of lessons reflect teaching quality.</p> <hd id="AN0187948735-2">TEACHING QUALITY</hd> <p>Teaching quality is commonly understood as the professional behaviour of teachers and their ability to facilitate meaningful learning interactions within a classroom (Doyle, [<reflink idref="bib14" id="ref9">14</reflink>]; Fauth et al., [<reflink idref="bib16" id="ref10">16</reflink>]; Göllner et al., [<reflink idref="bib19" id="ref11">19</reflink>]; Hamre & Pianta, [<reflink idref="bib22" id="ref12">22</reflink>]; Kunter et al., [<reflink idref="bib28" id="ref13">28</reflink>]).</p> <p>These interactions are characterized by key qualities including a structured and non‐disturbing lessons course (classroom management), respectful, sensitive, and motivating teaching (learning support), and the use of cognitively challenging tasks (cognitive activation, also labelled as instructional support). Our question is, how these dimensions and the related social interactions manifest in lesson semantics, which refers to how meaning is constructed through language in classroom interactions (e.g., Goodwin et al., [<reflink idref="bib20" id="ref14">20</reflink>]; Schleppegrell, [<reflink idref="bib44" id="ref15">44</reflink>]). This encompasses not only the explicit content of what is said (such as vocabulary and sentence structures) but also the dynamic processes through which teachers and students collaboratively build understanding. It involves the coherence of discourse, the relational aspects of communication, and the ways in which dialogue shapes learning (Marzano, [<reflink idref="bib33" id="ref16">33</reflink>]).</p> <p>All these elements align with established conceptions of teaching quality (e.g., Pianta & Hamre, [<reflink idref="bib38" id="ref17">38</reflink>]), which conceptualize teaching effectiveness as reflected in the nature of classroom interactions. In this vein, it is also reasonable to assume that semantic information is inherently part of any assessment of teaching quality. In addition to other nonverbal cues (e.g., teachers' body language and the arrangement of pedagogical elements; Babad et al., [<reflink idref="bib3" id="ref18">3</reflink>]; Lazarides et al., [<reflink idref="bib29" id="ref19">29</reflink>]), classroom observations are likely to implicitly capture aspects of lesson semantics. Due to its verbal and interactional nature, the analysis of lesson semantics may be particularly well‐suited to the dimensions of learning support and cognitive activation, as these are closely linked to teachers' explanations, discourse facilitation, and emotional responsiveness (Hamre & Pianta, [<reflink idref="bib22" id="ref20">22</reflink>]). In contrast, classroom management may rely more heavily on nonverbal behaviours (e.g., gaze direction, movement), which cannot be fully captured through transcript data alone (Martikainen, [<reflink idref="bib32" id="ref21">32</reflink>]).</p> <p>In addition, teaching quality is assumed to manifest at various moments throughout classroom learning (Nolen, [<reflink idref="bib36" id="ref22">36</reflink>]). From a professionalization perspective, teaching quality is regarded as a competence that can be acquired and applied consistently across different instructional situations. In contrast, an interactional perspective emphasizes the situational variability of teaching quality, highlighting how it emerges in response to the dynamics between teachers and students. Accordingly, teaching quality is not only expected to vary across lessons (Daumiller et al., [<reflink idref="bib11" id="ref23">11</reflink>]) and between classrooms (e.g., Marder et al., [<reflink idref="bib31" id="ref24">31</reflink>]; Wagner et al., [<reflink idref="bib48" id="ref25">48</reflink>]), but also to fluctuate within a single lesson, shaped by moment‐to‐moment instructional and relational dynamics (Daumiller et al., [<reflink idref="bib11" id="ref26">11</reflink>]).</p> <p>That is, semantic elements are likely fundamental components of teaching quality and warrant further empirical investigation (Hamre & Pianta, [<reflink idref="bib22" id="ref27">22</reflink>]; Hou et al., [<reflink idref="bib23" id="ref28">23</reflink>]). However, analysing such semantic aspects is a complex task that requires consideration of various interrelated aspects of language and communication during a classroom lesson. Moreover, assessing the semantics of teaching quality cannot be reduced to a single feature; rather, teaching must be viewed as a dynamic system in which multiple elements interact to create an effective learning environment. Given this complexity, we employed a large language model (LLM) to investigate whether lesson semantics could be meaningfully differentiated and to what extent this differentiation aligned with human‐annotated assessments of teaching quality.</p> <hd id="AN0187948735-3">TECHNOLOGY‐ASSISTED ANALYSIS OF TEACHING QUALITY</hd> <p>The use of machine learning, such as LLMs, has become increasingly popular due to their growing accessibility to a wider range of users. These advancements enable more efficient and sophisticated data processing across various research fields. In education, technology‐assisted assessment has been employed to evaluate classroom activities and teaching quality. For instance, a study by Schlotterbeck et al. ([<reflink idref="bib45" id="ref29">45</reflink>]) used classroom audio data to predict different classroom activities, such as teacher‐led lectures, individual student work, and group work. The audio data captured only paralinguistic aspects of speech (i.e., prosody, number of speakers, and overlapping speech) without converting this information into linguistic content in terms of word meaning. In another study, Gencoglu et al. ([<reflink idref="bib18" id="ref30">18</reflink>]) applied machine learning to analyse large‐scale open‐ended student responses. Using a topic modelling approach, the authors identified distinct clusters of teaching behaviours based on students' evaluations of teaching quality and developed methods to categorize these responses. The findings demonstrated that technology could enhance assessment processes, achieving performance levels comparable to those of traditionally used assessment methods and procedures (see also Falcon et al., [<reflink idref="bib15" id="ref31">15</reflink>]).</p> <p>The present study investigated lesson semantics related to teaching quality using OpenAI's GPT‐4 model and its text embedding. Technically, GPT‐4 is built using unsupervised learning on language data from extensive sources, processed within an artificial neural network (Brown et al., [<reflink idref="bib6" id="ref32">6</reflink>]; Radford et al., [<reflink idref="bib41" id="ref33">41</reflink>]). Numerous studies highlight GPT‐4's capability to generate cohesive responses and engage in human‐like conversations (Floridi & Chiriatti, [<reflink idref="bib17" id="ref34">17</reflink>]). Additionally, GPT‐4 can analyse texts based on both linguistic and conversational features (see Demszky et al., [<reflink idref="bib12" id="ref35">12</reflink>]; Kasneci et al., [<reflink idref="bib26" id="ref36">26</reflink>] for an overview), going beyond simple word occurrences and superficial text features to offer deeper insights into text semantics (Buduma et al., [<reflink idref="bib7" id="ref37">7</reflink>]). Most importantly, for the purposes of this study, these methodologies enable the extraction of data that can be further analysed using well‐established social science research methods. This allows us to integrate these methodologies as new powerful attempts to understand teaching quality from a text‐driven perspective (Devlin et al., [<reflink idref="bib13" id="ref38">13</reflink>]).</p> <p>However, as with other exploratory analyses, interpreting the results of machine learning models requires a process of meaning‐making (e.g., Behrens & Yu, [<reflink idref="bib4" id="ref39">4</reflink>]). That is, demonstrating that a model can accurately predict an outcome is not sufficient; it is also necessary to understand whether the model relies on information that is theoretically expected to be relevant. As with traditional social science methods, findings derived from machine learning must align with theoretical frameworks to ensure their validity. In the context of language models, meaning‐making is particularly challenging, as one must determine the appropriate level of language information on which to ground interpretation. Considering model outputs at the lowest level (e.g., letters, words, or simpler linguistic patterns) is often too fine‐grained to establish meaningful theoretical connections. Conversely, focusing on higher level features may lead to overly broad interpretations that obscure the most relevant aspects.</p> <p>In the present study, we approached the meaning‐making process in two ways. First, in analysing the semantics of teaching quality, we required that the results not only distinguish between different teachers but also be sensitive enough to detect variation within and across individual lessons. Second, we applied an approach that allows for novel ways of assessing the meaningfulness of results. Specifically, we used LLMs to generate lesson transcripts that emphasize particular dimensions of teaching quality. These generated texts serve as reference points for evaluating whether the machine learning models capture semantically meaningful aspects of the specific teaching quality dimension under investigation (Cao et al., [<reflink idref="bib9" id="ref40">9</reflink>]; Hu et al., [<reflink idref="bib24" id="ref41">24</reflink>]; Shin et al., [<reflink idref="bib46" id="ref42">46</reflink>]).</p> <hd id="AN0187948735-4">THE PRESENT STUDY</hd> <p>In the present study, we analyse lesson semantics by applying OpenAI's zero‐shot GPT‐4 model to identify similarities among lesson transcripts. Specifically, we will test whether semantic representations derived from the model's text embeddings differ both between lesson segments, lessons as well as teachers. In addition, we will examine the relationship between semantic representations and human‐annotated teaching quality. Finally, we will explore the extent to which semantic information provided by GPT‐4 reflects the specific teaching quality dimensions. The following four hypotheses have been proposed:</p> <p>First, we expected GPT‐4 and its underlying LLM to provide semantic representations of segmented transcripts which can then be used for further analysis (Hypothesis 1). We apply its text embeddings to convert text input into a numerical format, which is then used to exert components to represent the semantic information of lessons transcripts.</p> <p>Following the situative turn in educational psychology (Nolen, [<reflink idref="bib36" id="ref43">36</reflink>]), we further expect that semantic information extracted from lessons will vary across situations (i.e., 16‐min segments), lessons, and teachers (Hypothesis 2a). Additionally, we hypothesize systematic relationships between extracted semantic information and human‐annotated teaching quality (Hypothesis 2b).</p> <p>Finally, we examined the extent to which lesson semantics directly reflect teaching quality dimensions. Specifically, we applied a prompting procedure to synthesize transcripts based on different teaching quality dimensions. We expected that the content‐prompted lesson transcripts would align more closely with the extracted semantic representations (Hypothesis 3), thereby providing insight into the validity of these representations in capturing aspects of teaching quality.</p> <hd id="AN0187948735-5">METHODS</hd> <p></p> <hd id="AN0187948735-6">Sample</hd> <p>The present study utilized data from the German TALIS Video Study (Grünkorn et al., [<reflink idref="bib21" id="ref44">21</reflink>]) which was funded by the Leibniz Association (2017–2020) in the context of the international TALIS study (Teaching and Learning International Survey; OECD, [<reflink idref="bib37" id="ref45">37</reflink>]). The study's primary objective is to investigate effective teaching practices in math for 8th and 9th‐grade students regarding quadratic equations. For this, instructional materials, classroom recordings, as well as student and teacher reports were used to assess teaching quality and its relationship with students' learning in mathematics. The German sample, which consists of <emph>N</emph> = 50 teachers from 38 secondary schools and 1,140 students from seven federal states, was used for the current study. The participating schools were from academic‐track (Gymnasiums) or vocational‐track schools (Realschule). All participating teachers were asked to conduct two successive lessons to teach quadratic equations to the same class, resulting in two recorded lessons for each teacher and classroom.</p> <hd id="AN0187948735-7">Measures</hd> <p>To address the research questions, we will make use of two different sets of information: linguistic and conversational information from the lesson transcripts and observational ratings of teaching quality based on human annotations of the videotaped lessons. Whereas the observational ratings stem from the national TALIS dataset, the linguistic and conversational information reflect the building block on which the analysis with GPT‐4 had been conducted. Below, we present the set of measures we used in detail:</p> <hd id="AN0187948735-8">Transcripts</hd> <p>Every spoken word during the lesson was systematically transcribed by watching the video recordings of the lessons. The transcription was already provided by the Research Data Centre Education (FDZ Bildung). Transcriptions for every lesson have been created to support the human annotations of videotaped lessons. The provided transcripts contained anonymous information about the speaker (S#: student(s) number #, T: teacher), the starting time of the utterance, and the utterance itself. One utterance consisted of one or more consecutive sentences spoken by the same speaker (T or S#).</p> <p>For the analysis, the transcripts were divided into 16‐minute segments corresponding to the rating segments used for annotating teaching quality. We ensured that utterances from the same speaker remained completely intact. However, since we only had access to the start times of individual utterances in the transcripts, long utterances by a single speaker could occasionally exceed the 16‐minute limit by a few seconds. Each 16‐minute segment was then further split into parts consisting of ~500 tokens, evaluated using the tiktoken library (https://github.com/openai/tiktoken). Every text was only ever split at the end of a sentence or at the end of a speaker's statement to preserve linguistic coherence. This additional segmentation was performed for two reasons: first, due to the token limitations of the GPT API; and second, because the specificity of embeddings for longer texts remains uncertain (Cao, [<reflink idref="bib10" id="ref46">10</reflink>]).</p> <hd id="AN0187948735-9">Measures of teaching quality</hd> <p>In line with existing conceptions of teaching quality, the TALIS video study (Grünkorn et al., [<reflink idref="bib21" id="ref47">21</reflink>]) operationalized teaching quality along multiple quality dimensions. In the present study, we focused on dimensions of learning support and cognitive activation. Specifically, for the domain of cognitive activation, we addressed multiple approaches (i.e., focusing on the various approaches students use to solve problems, rather than multiple solutions they generate) and students' cognitive engagement (i.e., teachers provide students with opportunities to engage in subject‐matter practices). For the domain of learning support, we focused on the nature of discourse (i.e., students are given opportunities to actively participate in classroom discussions) and encouragement and warmth (i.e., the teacher and/or peers provide encouragement to students throughout their learning processes). These quality dimensions were expected to be related to linguistic and conversational information provided by lesson transcripts. According to the rating protocol, two raters independently annotated these teaching quality dimensions. Raters provided annotations for each of the 16‐minute intervals using four‐point scales per indicator, resulting in ratings of three segments per lesson. The reliability of the averaged ratings per segment (ICC[<reflink idref="bib2" id="ref48">2</reflink>]) was.63,.65,.70, and.69, respectively, suggesting that measures of teaching quality at the segment level showed at least acceptable precision.</p> <hd id="AN0187948735-10">Analysis</hd> <p></p> <hd id="AN0187948735-11">Text embedding</hd> <p>All text‐related analyses were performed using Python 3.13. The 500‐token lesson segments were processed using OpenAI's text‐embedding model 'text‐embedding‐3‐large' (https://openai.com/index/new‐embedding‐models‐and‐api‐updates/) via the API. An embedding model transforms text into a numerical vector, encoding meaningful information from a text as numerical values within the vector. OpenAI's embedding models have been shown to effectively capture contextual and semantic information across diverse applications, including large‐scale text retrieval and representation tasks (Neelakantan et al., [<reflink idref="bib35" id="ref49">35</reflink>]). Studies have demonstrated their ability to perform robust semantic similarity assessments and document separation tasks, aligning with our approach to analysing teaching quality through lesson transcripts (Reimers & Gurevych, [<reflink idref="bib42" id="ref50">42</reflink>]). The text‐embedding‐3‐large model generated a 3072‐dimensional vector for each text part, capturing contextual and semantic information from the given text.</p> <p>After computing the embedding vectors for all 500‐token segmented lesson transcripts, we used Scikit‐learn's StandardScaler function, which normalizes features by subtracting the mean and scaling to unit variance. To reduce dimensionality, we applied principal component analysis (PCA; Andrews, [<reflink idref="bib2" id="ref51">2</reflink>]; Buitinck et al., [<reflink idref="bib8" id="ref52">8</reflink>]) to the text‐segment embedding vectors. As an unsupervised dimensionality‐reduction technique, PCA projects the high‐dimensional space onto a set of orthogonal, lower dimensional components. In the context of text embedding, this method has been shown to condense the data into compact representations while still preserving important semantic information (Zhang et al., [<reflink idref="bib50" id="ref53">50</reflink>]). The number of principal components was determined using the elbow method based on the error sum of squares (Nainggolan et al., [<reflink idref="bib34" id="ref54">34</reflink>]). The resulting component scores were calculated for each segment and subsequently used as predictors in the multilevel regression models.</p> <hd id="AN0187948735-12">Establishing the relationship between semantic representations and teaching quality</hd> <p>To further investigate the association between semantics and teaching quality, we applied a multilevel analytical approach using SAS software (SAS Institute Inc, [<reflink idref="bib43" id="ref55">43</reflink>]). A two‐level model was specified, with differences between teachers modelled at the higher level and differences between transcript segments at the lower level. As only two lessons were recorded per teacher, we accounted for the nesting of lessons within teachers by including a dummy variable to control for lesson‐level effects (Snijders & Bosker, [<reflink idref="bib47" id="ref56">47</reflink>]). The multilevel prediction model was implemented in two steps. First, we estimated variance components attributable to the different levels using a null model. Second, we included the semantic representations as predictors in a regression analysis to estimate their association with human‐annotated teaching quality. Semantic representations at the lesson and teacher levels were obtained by aggregating the corresponding segment‐level scores in their manifest form. All variables were group‐mean centred at their respective analytical level. This approach allowed us to assess the extent to which the semantic representations explained variation in teaching quality across segments, lessons, and teachers. Due to the number of variables, we tested the associations separately for each of the teaching quality dimensions. Given the relatively small number of teachers and lessons, we also opted not to include random slope parameters. This decision was further supported by our assumption that the relationships between semantics and human ratings would not systematically vary across lessons or segments. Regression results are reported in standardized form, and explained variance in teaching quality was calculated relative to the total variance in each respective teaching quality dimension.</p> <hd id="AN0187948735-13">Prompting teaching quality</hd> <p>To support the meaning‐making process of the extracted semantic representation, we generated synthetic lesson segments, modifying them to align with the investigated teaching quality dimensions. Specifically, we make use of GPT‐4's generative capabilities, which, however, were applied independently of the text embedding analysis used for PCA. We instructed GPT‐4 to adjust transcript segments based on definitions derived from the TALIS study report, the same definitions used by human coders when assessing teaching quality in fully videotaped lessons (OECD, [<reflink idref="bib37" id="ref57">37</reflink>]). For each of the four teaching quality dimensions, we formulated specific prompts to guide the adaptation of the original transcripts in the intended direction. The model was instructed to analyse each lesson transcript sentence by sentence and rewrite the segment according to the respective teaching quality prompt from the codebook. The prompts used for each teaching quality dimension are presented in Table 1. For automated processing, we employed the GPT API (https://platform.openai.com/docs/models), using GPT‐4 with the temperature parameter set to zero to ensure consistency. GPT‐4 has been used and the temperature parameter has been set to zero. We applied this procedure to 92 randomly selected segments, representing 25% of the available data. Finally, we examined whether the content‐prompting of a specific teaching quality dimension resulted in more pronounced differences of the adjusted segments compared with non‐selected segments across the revealed PCA components. To prevent circularity between the PCA and the generative approach, the adjusted texts were not included in the PCA before.</p> <p>1 TABLE Text prompts for adjusting lesson transcripts according to the respective teaching quality dimension.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">Dimension</th><th align="left">Prompt</th></tr></thead><tbody valign="top"><tr><td align="left">Multiple approaches (MA)</td><td align="left">'Gehe das folgende Unterrichtstranskript Satz für Satz durch. Schreibe das Unterrichtsgespräch so um, dass die Lehrkraft (L) unterschiedliche Lösungswege vorstellt, Aufgaben auf unterschiedliche Weise bearbeitet werden und Lösungsansätze miteinander verglichen werden. Hier ist das Transkript: ...''Go through the following lesson transcript sentence by sentence. Rewrite the classroom dialogue so that the teacher (T) presents different solution methods, approaches tasks in various ways, and compares different problem‐solving strategies. Here is the transcript: ...'</td></tr><tr><td align="left">Nature of discourse (ND)</td><td align="left">'Gehe das folgende Unterrichtstranskript Satz für Satz durch. Schreibe das Unterrichtsgespräch so um, dass die Lehrkraft (L) die Schüler (S) am Unterricht beteiligt, eine Interaktion zwischen der Lehrkaft und den Schülern stattfindet und gemeinsam diskutiert wird. Hier ist das Transkript: ...''Go through the following lesson transcript sentence by sentence. Rewrite the classroom dialogue so that the teacher (T) engages the students (S), fosters interaction between the teacher and students, and facilitates a collaborative discussion. Here is the transcript: ...'</td></tr><tr><td align="left">Encouragement and warmth (EW)</td><td align="left">'Gehe das folgende Unterrichtstranskript Satz für Satz durch. Schreibe das Unterrichtsgespräch so um, dass die Lehrkraft (L) lobt, höflich und geduldig ist und insgesamt ein respektvoller Um‐ gang herrscht (auch unter den Schülern (S)). Hier ist das Transkript: ...''Go through the following lesson transcript sentence by sentence. Rewrite the classroom dialogue to ensure that the teacher (T) offers praise, is polite and patient, and fosters a respectful environment overall (including among the students (S)). Here is the transcript: ...'</td></tr><tr><td align="left">Students' cognitive engagement (CE)</td><td align="left">'Gehe das folgende Unterrichtstranskript Satz für Satz durch. Schreibe das Unterrichtsgespräch so um, dass die Lehrkraft (L) zum Nachdenken anregt und die Schüler aktiviert. Die Schüler (S) sollen Fragen stellen, kritisch sein und sich für den Stoff begeistern. Hier ist das Transkript: ...''Go through the following lesson transcript sentence by sentence. Rewrite the classroom dialogue so that the teacher (T) encourages critical thinking and actively engages the students. The students (S) should be asking questions, thinking critically, and showing enthusiasm for the material. Here is the transcript: ...'</td></tr></tbody></table> </ephtml> </p> <hd id="AN0187948735-14">Evaluating the prompted texts</hd> <p>In the final step, we applied the embedding model to the prompted lesson segments, used the same scaling, and applied the previously implemented PCA model to the new embedding vectors of the adjusted segments. This resulted in new component scores for each selected content‐prompted segment. To test whether prompting led to changes in the scores, we conducted both univariate (ANOVA) and multivariate (MANOVA) comparisons, accounting for the nesting of segments within lessons and teachers. In general, univariate tests assessed each individual semantic component, while multivariate tests examined the semantic components in conjunction. Two different sets of comparisons were applied: First, we compared selected and non‐selected segments before prompting to determine whether the random selection process was successful. Second, we compared the original and synthesized segments to assess whether differences in component scores were attributable to the prompting procedure. As the selection of texts was applied at the segment level to control for potential teacher differences, the comparisons were conducted solely at the segment level and could not be performed at the lesson or teacher level.</p> <p>All statistical tests were conducted two‐tailed and with an α‐level of.05.</p> <hd id="AN0187948735-15">RESULTS</hd> <p></p> <hd id="AN0187948735-16">Semantic representation of lesson transcripts</hd> <p>Addressing the first hypothesis, we analysed the embeddings of transcripts to examine whether they can be characterized by their semantic properties. PCA revealed a solution consisting of five orthogonal components (see elbow plot, Figure 1), suggesting that five components adequately capture the semantic information of transcript segments comprising 500 tokens.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/6KX/02sep25/bjep70001-fig-0001.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="bjep70001-fig-0001.jpg" title="1 Elbow plot showing the percentage of explained variance in GPT‐4 embeddings for the first 20 components." /> </p> <p></p> <hd id="AN0187948735-18">Associations of semantic representations with human annotations of teaching quality</hd> <p>Regarding the hypothesis that semantic representations of lesson segments were associated with human annotations of teaching quality (Hypothesis 2a), multilevel regression models were applied. We began with a model that included no predictors to determine the extent to which semantic representations varied between segments, between lessons, and between teachers. The results revealed the following findings: First, component scores of semantic representations revealed consistency within lessons as well as some consistency within teachers. On average, 2% to 13% of the variance (median = 12%) in semantic information could be attributed to differences between teachers and 3% to 52% of the variances (median = 37%) could be attributed to lessons showing a semantic consistency within lessons and across the analysed transcript segments. The corresponding results for human annotations showed that 9% to 32% of the variance (median = 21%) in human annotations could be attributed to teacher differences and 11%–33% of the variance (median = 23%) in human annotations could be attributed to the consistency of ratings within lessons.</p> <p>We then used PCA scores to predict human‐annotated teaching quality at the segment, lesson, and teacher level (Hypothesis 2b). Table 2 presents the results for each of the quality dimensions. The results showed that semantic representations were associated with teaching quality at all levels, whereby the associations were particularly close for cognitive engagement (.08 ≤ <emph>R</emph><sups>2</sups> ≤.20) and nature of discourse (.14 ≤ <emph>R</emph><sups>2</sups> ≤.16). In contrast, the predictive power was lower for multiple approaches (.01 ≤ <emph>R</emph><sups>2</sups> ≤.06) as well as encouragement and warmth (.02 ≤ <emph>R</emph><sups>2</sups> ≤.17). Inspecting the single components, the results showed that teaching quality dimensions were each predicted by separate components at the different analytical levels. Additionally important, except for multiple approaches, the semantic components explained variations of teaching quality at the teachers' level. That is, semantic information derived from lesson transcripts provided information to differentiate between teachers in terms of their annotated teaching quality.</p> <p>2 TABLE Multilevel models predicting human annotations of teaching quality with semantic components.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" /><th align="left">Cognitive engagement</th><th align="left">Encouragement and warmth</th><th align="left">Multiple approaches</th><th align="left">Nature of discourse</th></tr><tr><th align="left"><italic>β</italic><sub>S</sub></th><th align="left"><italic>β</italic><sub>L</sub></th><th align="left"><italic>β</italic><sub>T</sub></th><th align="left"><italic>β</italic><sub>S</sub></th><th align="left"><italic>β</italic><sub>L</sub></th><th align="left"><italic>β</italic><sub>T</sub></th><th align="left"><italic>β</italic><sub>S</sub></th><th align="left"><italic>β</italic><sub>L</sub></th><th align="left"><italic>β</italic><sub>T</sub></th><th align="left"><italic>β</italic><sub>S</sub></th><th align="left"><italic>β</italic><sub>L</sub></th><th align="left"><italic>β</italic><sub>T</sub></th></tr></thead><tbody valign="top"><tr><td align="left">SR1</td><td align="char" char=".">−.24<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">−.14<xref ref-type="fn" rid="tfn4" /></td><td align="char" char=".">−.34<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">.03</td><td align="char" char=".">−.01</td><td align="char" char=".">−.18<xref ref-type="fn" rid="tfn5" /></td><td align="char" char=".">−.01</td><td align="char" char=".">.00</td><td align="char" char=".">−.05</td><td align="char" char=".">−.16<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">−.09<xref ref-type="fn" rid="tfn5" /></td><td align="char" char=".">−.25<xref ref-type="fn" rid="tfn5" /></td></tr><tr><td align="left">SR2</td><td align="char" char=".">.03</td><td align="char" char=".">.06</td><td align="char" char=".">.08</td><td align="char" char=".">.03</td><td align="char" char=".">.01</td><td align="char" char=".">−.32<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">.04</td><td align="char" char=".">.06</td><td align="char" char=".">.11</td><td align="char" char=".">−.19<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">−.19<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">−.14</td></tr><tr><td align="left">SR3</td><td align="char" char=".">.20<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">.17<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">.18<xref ref-type="fn" rid="tfn4" /></td><td align="char" char=".">.10<xref ref-type="fn" rid="tfn5" /></td><td align="char" char=".">.08<xref ref-type="fn" rid="tfn5" /></td><td align="char" char=".">.07</td><td align="char" char=".">.00</td><td align="char" char=".">.00</td><td align="char" char=".">.19<xref ref-type="fn" rid="tfn5" /></td><td align="char" char=".">.26<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">.24<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">.15</td></tr><tr><td align="left">SR4</td><td align="char" char=".">.11<xref ref-type="fn" rid="tfn4" /></td><td align="char" char=".">.06</td><td align="char" char=".">−.04</td><td align="char" char=".">.14<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">.09<xref ref-type="fn" rid="tfn5" /></td><td align="char" char=".">−.03</td><td align="char" char=".">−.11<xref ref-type="fn" rid="tfn5" /></td><td align="char" char=".">−.07</td><td align="char" char=".">−.13</td><td align="char" char=".">.11<xref ref-type="fn" rid="tfn5" /></td><td align="char" char=".">.07</td><td align="char" char=".">−.15</td></tr><tr><td align="left">SR5</td><td align="char" char=".">.15<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">.12<xref ref-type="fn" rid="tfn4" /></td><td align="char" char=".">.13<xref ref-type="fn" rid="tfn5" /></td><td align="char" char=".">−.02</td><td align="char" char=".">−.06</td><td align="char" char=".">−.18<xref ref-type="fn" rid="tfn5" /></td><td align="char" char=".">0.04</td><td align="char" char=".">.03</td><td align="char" char=".">−.07</td><td align="char" char=".">−.16<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">−.14<xref ref-type="fn" rid="tfn3" /></td><td align="char" char=".">−.19<xref ref-type="fn" rid="tfn5" /></td></tr><tr><td align="left">Lesson</td><td align="left" /><td align="char" char=".">.14</td><td align="left" /><td align="left" /><td align="char" char=".">.03</td><td align="left" /><td align="left" /><td align="char" char=".">−.08</td><td align="left" /><td align="left" /><td align="char" char=".">.20<xref ref-type="fn" rid="tfn5" /></td><td align="left" /></tr><tr><td align="left">R<sup>2</sup></td><td align="char" char=".">.17</td><td align="char" char=".">.08</td><td align="char" char=".">.20</td><td align="char" char=".">.03</td><td align="char" char=".">.02</td><td align="char" char=".">.17</td><td align="char" char=".">.02</td><td align="char" char=".">.01</td><td align="char" char=".">.06</td><td align="char" char=".">.16</td><td align="char" char=".">.14</td><td align="char" char=".">.15</td></tr></tbody></table> </ephtml> </p> <p>1 <emph>Note</emph>: Lessons were dummy coded, with the first lesson serving as the reference category. Due to the number of independent variables and the resulting model complexity, associations were tested separately for each teaching quality dimension. For presentation purposes only, the associations between the semantic representations and the teaching quality annotations at the different levels are shown side by side. <emph>R</emph><sups>2</sups> was computed as a pseudo‐<emph>R</emph><sups>2</sups>, referring to the proportion of total variance explained in the respective dependent variable.</p> <ulist> <item>2 Abbreviations: L, lesson‐level model; S, segment‐level model; SR, semantic representations; T, teacher‐level model.</item> <item>3 *** <emph>p</emph> < .001;</item> <item>4 ** <emph>p</emph> < .01;</item> <item>5 * <emph>p</emph> < .05.</item> </ulist> <hd id="AN0187948735-19">Explaining semantic representations using prompted transcript</hd> <p>Turning to our last hypothesis (Hypothesis 3), 25% of randomly selected segments were prompted to examine whether and to what extent semantic representations represent the semantic information of lessons in relation to the teaching quality dimensions in question. To achieve this, we compared the revealed representations before and after prompting the transcript segments. The resulting lesson segments demonstrated face validity (see https://osf.io/z2tcq/?view%5fonly=d2e49a98248e46a98e2a8837f391100c) and the number of tokens did not statistically significantly differ (<emph>p</emph> = .21) between the original (<emph>M</emph> = 587.82, <emph>SD</emph> = 141.59) and synthetic segments (<emph>M</emph> = 577.57, <emph>SD</emph> = 137.98). Furthermore, means of components did not reveal a difference between selected and non‐selected segments before prompting (original; Table 3). Neither univariate comparisons of the semantic components (<emph>p</emph>s > .05) nor a multivariate comparison of all components together showed a statistically significant difference, <emph>F</emph>(<reflink idref="bib5" id="ref58">5</reflink>, 373) = 1.53, <emph>p</emph> = .179, <emph>η</emph><sups>2</sups> = .02. In line with the hypothesis, the comparison of synthetic segments (Table 3) showed more accentuated mean differences in comparison to the original segments. All multivariate tests revealed statistically significant differences between original and synthetic transcripts, indicating that semantic information represented by the revealed semantic representations can be influenced through prompting based on a theoretically defined quality dimension. Moreover, the results of univariate comparisons showed that prompting did not lead to differences across all semantic representations but was limited to specific ones. Specifically, we found less pronounced differences for the first and second components, along with some homogeneity of effects for the remaining three components (see also Figure 2). This suggests that the prompting procedure did not influence all semantic representations equally. Instead, the semantic shaping of lesson segments was more nuanced and less uniformly distributed across components than initially expected.</p> <p>3 TABLE Summary of mean differences between original and synthetic transcript segments.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left" /><th align="left">Original</th><th align="left">Synthetic</th></tr><tr><th align="left">Selected</th><th align="left">Non‐selected</th><th align="left">Cognitive engagement</th><th align="left">Encouragement and warmth</th><th align="left">Multiple approaches</th><th align="left">Nature of discourse</th></tr></thead><tbody valign="top"><tr><td align="left">SR1</td><td align="left">−.01</td><td align="left">.01</td><td align="left">.01</td><td align="left">.00</td><td align="left">.02</td><td align="left">.00</td></tr><tr><td align="left">SR2</td><td align="left">−.01</td><td align="left">.01</td><td align="left">.01</td><td align="left">−.02*</td><td align="left">.01</td><td align="left">−.01</td></tr><tr><td align="left">SR3</td><td align="left">−.01</td><td align="left">−.01</td><td align="left">−.05***</td><td align="left">−.03**</td><td align="left">−.04***</td><td align="left">−.04***</td></tr><tr><td align="left">SR4</td><td align="left">.00</td><td align="left">.00</td><td align="left">−.05***</td><td align="left">−.03**</td><td align="left">−.06***</td><td align="left">−.04***</td></tr><tr><td align="left">SR5</td><td align="left">−.01</td><td align="left">.01</td><td align="left">.04***</td><td align="left">.01</td><td align="left">.05***</td><td align="left">.02</td></tr><tr><td align="left">Wilk's Lambda</td><td align="left" /><td align="left">F(5, 373) = 1.53,p = .179, η<sup>2</sup> = .02</td><td align="left">F(5, 373) = 14.32,p < .001, η<sup>2</sup> = .16</td><td align="left">F(5, 373) = 4.96,p < .001, η<sup>2</sup> = .06</td><td align="left">F(5, 373) = 17.72,p < .001, η<sup>2</sup> = .19</td><td align="left">F(5, 373) = 7.63,p < .001, η<sup>2</sup> = .09</td></tr></tbody></table> </ephtml> </p> <p>6 <emph>Note</emph>: Results presented factor means. All comparisons refer to the means of the selected transcript segments before prompting. Univariate tests were complemented with a multivariate test (Wilk's Lambda) to compare the configuration of semantic representations before and after prompting. ***<emph>p</emph> < .001; **<emph>p</emph> < .01; *<emph>p</emph> < .05.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/6KX/02sep25/bjep70001-fig-0002.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="bjep70001-fig-0002.jpg" title="2 Depiction of configurations of semantic representations before and after the prompting of cognitive engagement (CE), encouragement and warmth (EW), multiple approaches (MA), and nature of discourse (ND)." /> </p> <p></p> <hd id="AN0187948735-21">DISCUSSION</hd> <p>The present study examined whether and to what extent the semantics of lesson transcripts in math lessons reflect multiple dimensions of teaching quality. Using GPT‐4, transcripts were analysed based on a range of semantic information, from syntactical to pragmatic and conversational features, that shape teaching and learning interactions. The study findings demonstrated that the semantic representations of lessons align with teaching quality.</p> <p>First, the revealed semantic representations were shown to be sensitive enough to reflect variations across segments, lessons, and teachers, indicating that lesson semantics sensitively capture differences in ongoing teaching. These findings align closely with existing conceptions of teaching quality. In contemporary frameworks (e.g., Daumiller et al., [<reflink idref="bib11" id="ref59">11</reflink>]; Nolen, [<reflink idref="bib36" id="ref60">36</reflink>]), teaching quality and learning responses are no longer viewed as a static construct defined solely by a teacher's ability to provide effective instruction. Instead, it is understood as a dynamic process that manifests in teacher–student interactions and evolves throughout a lesson, which consists of various learning situations. The finding that semantic information varies across teachers, lessons, and segments highlights the potential of modern technologies to capture this complexity. It demonstrates how advanced analytical approaches can assist the assessment of teaching quality on the basis of semantic information (Demszky et al., [<reflink idref="bib12" id="ref61">12</reflink>]; Hou et al., [<reflink idref="bib23" id="ref62">23</reflink>]).</p> <p>Second, the present findings showed that the revealed semantic representations of lesson transcripts were associated with human‐annotated teaching quality dimensions. Prediction models indicated a significant amount of explained variance at different levels. Specifically, semantics accounted for up to 20% of the variation in human annotations across different quality dimensions. Most importantly, variance was explained at all levels, with the strongest effects observed at the teacher level. This finding supports the notion that semantic representations can effectively detect differences in teaching quality between teachers (Goodwin et al., [<reflink idref="bib20" id="ref63">20</reflink>]). When interpreting these findings, it is important to note that human raters were not instructed to focus solely on the linguistic or semantic aspects of the lessons. Instead, they were asked to assess teaching quality based on all available information in the videotaped lessons, including nonverbal behaviour and the arrangement of pedagogical elements. Within this context, the observed associations underscore the relevance of lesson semantics as a relevant component of effective classroom learning. The study's focused examination of semantic information, therefore, enables a more nuanced investigation of an important yet underexplored aspect of teaching quality from both scientific and practical perspectives (e.g., Leiss et al., [<reflink idref="bib30" id="ref64">30</reflink>]). Notably, the strongest associations between lesson semantics and teaching quality were found for students' cognitve engagement and the nature of discourse. While these two quality dimensions are understood as aspects of classroom interaction within the TALIS framework, the dimension of multiple approaches may be more closely related to task design and the use of learning materials. In the case of encouragement and warmth, one could argue that paraverbal and nonverbal information which was not captured in the analysis of lesson transcripts may also play a significant role (Lazarides et al., [<reflink idref="bib29" id="ref65">29</reflink>]). In this vein, the study findings demonstrate that GPT‐4 can effectively analyse conversational data and extract key indicators of teaching quality.</p> <p>Finally, technology‐assisted research, as conducted in the present study, still faces significant challenges. A particularly important issue is the need for additional steps to interpret empirical findings within a theoretical framework. To address this, we applied a structured procedure to enhance our understanding of the revealed associations between semantic information derived from lesson transcripts and teaching quality. Specifically, we used GPT‐4's capabilities to generate synthesized data, allowing us to explore the significance of lesson semantic representations. By creating synthetic transcripts that reflected theoretically derived variations in lesson quality dimensions, we examined whether the derived semantic representations of lessons accurately captured specific teaching quality attributes. Our study demonstrated that GPT‐4‐generated transcripts were face‐valid representations of typical secondary school mathematics lessons and that synthesized transcripts did not systematically differ from the originals in terms of average text length. Furthermore, content prompts effectively modified the lesson transcripts in alignment with the revealed semantic representations. Component scores became more pronounced for specific components, suggesting that the extracted semantic information reflects aspects of the targeted teaching quality dimension. In this regard, the findings indicate that the identified semantic components contained relevant information about teaching quality, rather than being merely influenced by construct‐irrelevant features of the analysed transcripts.</p> <p>On the other hand, our findings suggest that the transcript adaptations were less distinctive than initially expected. While the multivariate analyses indicated that segments differed collectively across all semantic components, the univariate results showed that the adaptations produced similar shifts, limited to only some of the extracted components. Ideally, the prompting was expected to elicit more specific combinations of components that would reflect the uniqueness of each targeted teaching quality dimension. However, this was not the case, raising the question of what specific types of semantic information were actually represented by the extracted semantic components. Answering this question requires further research that might apply a similar methodological approach as used in the present study, but vary the granularity of the prompts. By systematically manipulating the specificity and scope of the prompt instructions, future studies may be able to determine which types of semantic content related to teaching quality are being encoded in lesson transcripts.</p> <hd id="AN0187948735-22">Limitations and further research</hd> <p>In general, the present study provides insights into a semantic analysis of lesson transcripts and their associations with teaching quality dimensions. The text embedding and clustering of transcripts revealed semantic representations that varied both within and between lessons and were associated with human annotations of multiple teaching quality dimensions. While the approach successfully identified semantic representations related to human annotations of teaching quality, we acknowledge the potential limitations of this study. These limitations are closely tied to the use of machine learning in extracting meaningful characteristics that inform specific semantic features of effective classroom learning. The inherent 'black‐box' nature of these models presents a challenge, underscoring the need for robust theoretical frameworks to guide interpretation and meaning‐making. Furthermore, OpenAI's GPT models are not open source, which prevents us from following the processing procedures in the model. Additionally, it is essential to consider that we employed a zero‐shot GPT model rather than a model specifically trained on human‐labelled teaching quality data. The latter could potentially yield higher accuracy in predicting teaching quality. Furthermore, developing new models could incorporate relevant additional features. One particularly important aspect is the variability in human ratings of lessons, which often exhibit a degree of disagreement (White & Ronfeldt, [<reflink idref="bib49" id="ref66">49</reflink>]). Extending models to account for such discrepancies in human annotation could improve predictive accuracy by integrating measures of uncertainty and variability, thereby enhancing the robustness of current scaling procedures used in the field.</p> <p>Furthermore, one might assume that semantic representations may overlap with pedagogical elements (e.g., varying classroom activities) that are typically considered aspects of classroom organization rather than indicators of teaching quality. In this vein, the extent to which the revealed semantic representations are related to specific pedagogical elements remains an open question. Addressing this issue in future research may offer valuable insights into the interactional patterns that emerge through the orchestration of pedagogical elements, thereby contributing to a more nuanced understanding of how teaching quality is enacted in real classroom settings.</p> <p>Another challenge concerns the approach used to critically assess the meaning of semantic representations. While we believe that using synthetic data based on theoretical definitions is a promising strategy for deriving insights from model results, the present study did not aim to systematically investigate this procedure. Indeed, the findings cannot confirm that the synthetic data ultimately reflect the specific teaching quality dimension in question. Using human‐generated transcripts would provide greater assurance that the content offers a valid representation of teaching quality. Additionally, one might consider a study comparing human ratings of teaching quality across original and synthetic transcripts to explore systematic differences introduced through the synthetic generation process. Even when accepting the procedure of the present study, its practical implementation involves several considerations. In particular, the formulation of prompts guiding the model to generate lesson transcripts according to a specific definition is critical. Even for well‐established definitions of teaching quality, numerous model parameters must be carefully specified, including the required length of synthetic transcripts, the level of detail regarding classroom behaviours or events, the degree to which the adapted transcript may diverge from the original, and the optimal amount of text needed to generate sufficiently specific content without compromising the lesson's narrative structure. Finally, it has to be kept in mind that other approaches exist helping to bridge the gap between model outputs and theory‐based interpretation. These approaches range from non‐technically assisted strategies such as expert ratings of model outputs to post‐hoc feature attribution methods helping to identify which features most influenced model predictions. Future research should systematically explore the potentials of the approach used in the present study. Such efforts will be essential to determine whether the proposed strategy can evolve into a more robust approach for validating the results of LLMs in the context of teaching quality research.</p> <hd id="AN0187948735-23">CONCLUSION</hd> <p>The present study investigated the semantics of lesson transcripts in mathematics classrooms and examined whether the extracted semantic information was associated with human‐annotated teaching quality using a zero‐shot LLM. The results demonstrated that lesson semantics are sensitive to differences in teaching quality both within and between lessons as well as across teachers. Additionally, our findings revealed significant associations between lesson semantics and human‐annotated teaching quality, highlighting semantic information as a fundamental and integral component of teaching quality. Finally, the extracted semantic representations were found to reflect construct‐relevant aspects of teaching quality rather than construct‐irrelevant features, thereby underscoring their substantive meaning. At the same time, this raises the question of which specific semantic characteristics actually contribute to fostering teaching quality during regular classroom lessons.</p> <hd id="AN0187948735-24">AUTHOR CONTRIBUTIONS</hd> <p> <bold>Richard Göllner:</bold> Conceptualization; writing – original draft; funding acquisition; methodology; validation; resources; project administration. <bold>Rebecca Lazarides:</bold> Writing – review and editing; investigation; validation. <bold>Philipp Stark:</bold> Formal analysis; data curation; writing – review and editing; visualization; methodology; validation; software.</p> <hd id="AN0187948735-25">ACKNOWLEDGEMENT</hd> <p>Open Access funding enabled and organized by Projekt DEAL.</p> <hd id="AN0187948735-26">CONFLICT OF INTEREST STATEMENT</hd> <p>The authors have no conflict of interest to declare.</p> <hd id="AN0187948735-27">DATA AVAILABILITY STATEMENT</hd> <p>The data that support the findings of this study are available in FDZ at https://<ulink href="http://www.dipf.de/de/infrastrukturen/forschungsdaten/forschungsdatenzentrum‐bildung">www.dipf.de/de/infrastrukturen/forschungsdaten/forschungsdatenzentrum‐bildung</ulink>. These data were derived from the following resources available in the public domain: FDZ, https://<ulink href="http://www.dipf.de/de/infrastrukturen/forschungsdaten/forschungsdatenzentrum‐bildung">www.dipf.de/de/infrastrukturen/forschungsdaten/forschungsdatenzentrum‐bildung</ulink>.</p> <ref id="AN0187948735-28"> <title> Footnotes </title> <blist> <bibl id="bib1" idref="ref5" type="bt">1</bibl> <bibtext> [Corrections added on 2 July 2025, after first online publication: the affiliations have been updated in this version.]</bibtext> </blist> <blist> <bibl id="bib2" idref="ref48" type="bt">2</bibl> <bibtext> The temperature parameter influences the probability distribution from which the model selects the next token (word or piece of a word). A higher temperature (e.g., 1.0) makes the output more diverse and random, whereas a lower temperature result in more deterministic and reproduceable output.</bibtext> </blist> </ref> <ref id="AN0187948735-29"> <title> REFERENCES </title> <blist> <bibtext> Alqassab, M., & León, J. (2024). Motivational messages from teachers before exams: Links to intrinsic motivation, engagement, and academic performance. Teaching and Teacher Education, 151, 104750. https://doi.org/10.1016/j.tate.2024.104750</bibtext> </blist> <blist> <bibtext> Andrews, M. (2016). Compressing word embeddings. In Neural information processing: 23rd international conference, ICONIP 2016, Kyoto, Japan, October 16–21, 2016, proceedings, part IV 23 (pp. 413 – 422). Springer International Publishing.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref18" type="bt">3</bibl> <bibtext> Babad, E., Sahar‐Inbar, L., Hammer, R., Turgeman‐Lupo, K., & Nessis, S. (2021). Student evaluations fast and slow: It's time to integrate teachers' nonverbal behavior in evaluations of teaching effectiveness. Journal of Nonverbal Behavior, 45 (3), 321 – 338. https://doi.org/10.1007/s10919‐021‐00364‐4</bibtext> </blist> <blist> <bibl id="bib4" idref="ref39" type="bt">4</bibl> <bibtext> Behrens, J. T., & Yu, C.‐H. (2003). Exploratory data analysis. In J. A. Schinka, W. F. Velicer, J. A. Schinka, & W. F. Velicer (Eds.), Handbook of psychology: Research methods in psychology (pp. 33 – 64). John Wiley & Sons, Inc.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref1" type="bt">5</bibl> <bibtext> Bell, C. A., Dobbelaer, M. J., Klette, K., & Visscher, A. (2019). Qualities of classroom observation systems. School Effectiveness and School Improvement, 30 (1), 3 – 29. https://doi.org/10.1080/09243453.2018.1539014</bibtext> </blist> <blist> <bibl id="bib6" idref="ref32" type="bt">6</bibl> <bibtext> Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert‐Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., ... Amodei, D. (2020). Language models are few‐shot learners. Advances in Neural Information Processing Systems, 33, 1877 – 1901. https://arxiv.org/abs/2005.14165</bibtext> </blist> <blist> <bibl id="bib7" idref="ref37" type="bt">7</bibl> <bibtext> Buduma, N., Buduma, N., Papa, J., & Locascio, N. (2022). Fundamentals of deep learning: Designing next‐generation machine intelligence algorithms. O'Reilly Media, Inc.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref52" type="bt">8</bibl> <bibtext> Buitinck, L., Louppe, G., Blondel, M., Pedregosa, F., Mueller, A., Grisel, O., & Varoquaux, G. (2013). API design for machine learning software: Experiences from the scikit‐learn project. arXiv preprint arXiv:1309.0238. https://arxiv.org/abs/1309.0238</bibtext> </blist> <blist> <bibl id="bib9" idref="ref40" type="bt">9</bibl> <bibtext> Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P. S., & Sun, L. A. (2023). A comprehensive survey of AI‐generated content (AIGC): A history of generative AI from GAN to ChatGPT. arXiv [preprint]. https://doi.org/10.48550/arXiv.2303.04226</bibtext> </blist> <blist> <bibtext> Cao, H. (2024). Recent advances in text embedding: A comprehensive review of top‐performing methods on the MTEB Benchmark. arXiv. https://doi.org/10.48550/arXiv.2406.01607.</bibtext> </blist> <blist> <bibtext> Daumiller, M., Janke, S., Rinas, R., Hein, J., Dickhäuser, O., & Dresel, M. (2023). Different time and context = different goals and emotions? Temporal variability and context specificity of achievement goals for teaching and associations with discrete emotions. Contemporary Educational Psychology, 72, 102139. https://doi.org/10.1016/j.cedpsych.2022.102139</bibtext> </blist> <blist> <bibtext> Demszky, D., Yang, D., Yeager, D. S., Bryan, C. J., Clapper, M., Chandhok, S., Eichstaedt, J. C., Hecht, C. A., Jamieson, J. P., Johnson, M., Jones, M., Krettek‐Cobb, D., Lai, L., JonesMitchell, N., Ong, D. C., Dweck, C. S., Gross, J. J., & Pennebaker, J. (2023). Using large language models in psychology. Nature Reviews Psychology, 2, 688 – 701. https://doi.org/10.1038/s44159‐023‐00241‐5</bibtext> </blist> <blist> <bibtext> Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre‐training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. https://doi.org/10.48550/arXiv.1810.04805</bibtext> </blist> <blist> <bibtext> Doyle, W. (2006). Ecological approaches to classroom management. In C. M. Evertson & C. S. Weinstein (Eds.), Handbook of classroom management: Research, practice, and contemporary issues (pp. 97 – 125). Lawrence Erlbaum Associates Publishers.</bibtext> </blist> <blist> <bibtext> Falcon, S., Admiraal, W., & Leon, J. (2023). Teachers' engaging messages and the relationship with students' performance and teachers' enthusiasm. Learning & Instruction, 86, 101750. https://doi.org/10.1016/j.learninstruc.2023.101750</bibtext> </blist> <blist> <bibtext> Fauth, B., Wagner, W., Bertram, C., Göllner, R., Roloff, J., Lüdtke, O., Polikoff, M. S., Klusmann, U., & Trautwein, U. (2020). Don't blame the teacher? The need to account for classroom characteristics in evaluations of teaching quality. Journal of Educational Psychology, 112, 1284 – 1302. https://doi.org/10.1037/edu0000416</bibtext> </blist> <blist> <bibtext> Floridi, L., & Chiriatti, M. (2020). GPT‐3: Its nature, scope, limits, and consequences. Minds and Machines, 30, 681 – 694. https://doi.org/10.1007/s11023‐020‐09548‐1</bibtext> </blist> <blist> <bibtext> Gencoglu, B., Helms‐Lorenz, M., Maulana, R., Jansen, E. P. W. A., & Gencoglu, O. (2023). Machine and expert judgments of student perceptions of teaching behavior in secondary education: Added value of topic modeling with big data. Computers & Education, 193, 1 – 22. https://doi.org/10.1016/j.compedu.2022.104682</bibtext> </blist> <blist> <bibtext> Göllner, R., Fauth, B., Lenske, G., Praetorius, A.‐K., & Wagner, W. (2020). Do student ratings of classroom management tell us more about teachers or about classroom composition? Zeitschrift Für Pädagogik, 66, 156 – 172. https://doi.org/10.25656/01:25871</bibtext> </blist> <blist> <bibtext> Goodwin, A. P., Cho, S.‐J., Reynolds, D., Silverman, R., & Nunn, S. (2021). Explorations of classroom talk and links to reading achievement in upper elementary classrooms. Journal of Educational Psychology, 113, 27 – 48. https://doi.org/10.1037/edu0000462</bibtext> </blist> <blist> <bibtext> Grünkorn, J., Klieme, E., Praetorius, A.‐K., & Schreyer, P. (Eds.). (2020). Mathematikunterricht im internationalen Vergleich. Ergebnisse aus der TALIS‐Videostudie Deutschland [Mathematics Instruction in an International Comparison: Results from the TALIS Video Study Germany]. Leibniz‐Institut für Bildungsforschung und Bildungsinformation.</bibtext> </blist> <blist> <bibtext> Hamre, B. K., & Pianta, R. C. (2010). Classroom environments and developmental processes: Conceptualization and measurement. In J. L. Meece & J. S. Eccles (Eds.), Handbook of research on schools, schooling and human development (pp. 25 – 41). Routledge.</bibtext> </blist> <blist> <bibtext> Hou, R., Fütterer, T., Bühler, B., Bozkir, E., Gerjets, P., Trautwein, U., & Kasneci, E. (2024). Automated assessment of encouragement and warmth in classrooms leveraging multimodal emotional features and ChatGPT. In International conference on artificial intelligence in education (pp. 60 – 74). Springer Nature Switzerland.</bibtext> </blist> <blist> <bibtext> Hu, Q., Yuille, A., & Zhou, Z. (2023). Synthetic data as validation. arXiv preprint arXiv:2310.16052.</bibtext> </blist> <blist> <bibtext> Kane, T. J., & Staiger, D. O. (2012). Gathering feedback for teachers: Combining high‐quality observations with student surveys and achievement gains. Bill & Melinda Gates Foundation.</bibtext> </blist> <blist> <bibtext> Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., & Stadler, M. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274</bibtext> </blist> <blist> <bibtext> Klette, K., & Blikstad‐Balas, M. (2018). Observation manuals as lenses to classroom teaching: Pitfalls and possibilities. European Educational Research Journal, 17, 129 – 146. https://doi.org/10.1177/1474904117703228</bibtext> </blist> <blist> <bibtext> Kunter, M., Klusmann, U., Baumert, J., Richter, D., Voss, T., & Hachfeld, A. (2013). Professional competence of teachers: Effects on instructional quality and student development. Journal of Educational Psychology, 105, 805 – 820. https://doi.org/10.1037/a0032583</bibtext> </blist> <blist> <bibtext> Lazarides, R., Frenkel, J., Petković, U., Göllner, R., & Hellwich, O. (2025). 'No words'—Machine‐learning classified nonverbal immediacy and its role in connecting teacher self‐efficacy with perceived teaching and student interest. British Journal of Educational Psychology, 95 (Suppl. 1), S15 – S31. https://doi.org/10.1111/bjep.12732</bibtext> </blist> <blist> <bibtext> Leiss, D., Plath, J., & Schwippert, K. (2019). Language and mathematics—Key factors influencing the comprehension process in reality‐based tasks. Mathematical Thinking and Learning, 21, 131 – 153. https://doi.org/10.1080/10986065.2019.1570835</bibtext> </blist> <blist> <bibtext> Marder, J., Thiel, F., & Göllner, R. (2023). Classroom management and students' mathematics achievement: The role of students' disruptive behavior and teacher classroom management. Learning and Instruction, 86, 101746. https://doi.org/10.1016/j.learninstruc.2023.101746</bibtext> </blist> <blist> <bibtext> Martikainen, J. (2020). How students categorize teachers based on visual cues: Implications of nonverbal communication for classroom management. Scandinavian Journal of Educational Research, 64, 569 – 588. https://doi.org/10.1080/00313831.2019.1595713</bibtext> </blist> <blist> <bibtext> Marzano, R. J. (2004). The developing vision of vocabulary instruction. In J. Bauman & M. Kameenui (Eds.), Research and theory in vocabulary development (pp. 100 – 117). Guilford Press.</bibtext> </blist> <blist> <bibtext> Nainggolan, R., Peranginangin, R., Simarmata, E., & Tarigan, A. F. (2019). Improved the performance of the K‐means cluster using the sum of squared error (SSE) optimized by using the Elbow method. Journal of Physics: Conference Series, 1361 (1), 012015. https://doi.org/10.1088/1742‐6596/1361/1/012015</bibtext> </blist> <blist> <bibtext> Neelakantan, A., Xu, T., Puri, R., Radford, A., Han, J. M., Tworek, J., Yuan, Q., Tezak, N., Kim, J. W., Hallacy, C., & Hallacy, C. (2022). Text and code embeddings by contrastive pre‐training. arXiv 2022. arXiv preprint arXiv:2201.10005. https://arxiv.org/abs/2201.10005</bibtext> </blist> <blist> <bibtext> Nolen, S. B. (2024). Taking a situative approach to research in educational psychology. In P. Schutz & K. Muis (Eds.), Handbook of Educational Psychology (pp. 119 – 138). Routledge.</bibtext> </blist> <blist> <bibtext> OECD. (2020). Global teaching insights: A video study of teaching. OECD.</bibtext> </blist> <blist> <bibtext> Pianta, R. C., & Hamre, B. K. (2009). Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity. Educational Researcher, 38, 109 – 119. https://doi.org/10.3102/0013189X09332374</bibtext> </blist> <blist> <bibtext> Pianta, R. C., Hamre, B. K., & Allen, J. P. (2012). Teacher‐student relationships and engagement: Conceptualizing, measuring, and improving the capacity of classroom interactions. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 365 – 386). Springer.</bibtext> </blist> <blist> <bibtext> Putwain, D. W., Nicholson, L. J., & Kutuk, G. (2023). Warning students of the consequences of examination failure: An effective strategy for promoting student engagement? Journal of Educational Psychology, 115, 36 – 54. https://doi.org/10.1037/edu0000741</bibtext> </blist> <blist> <bibtext> Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI Blog, 1, 9.</bibtext> </blist> <blist> <bibtext> Reimers, N., & Gurevych, I. (2019). Sentence‐bert: Sentence embeddings using siamese bert‐networks. arXiv preprint arXiv:1908.10084. https://arxiv.org/abs/1908.10084</bibtext> </blist> <blist> <bibtext> SAS Institute Inc. (2024). SAS/STAT software (Version X.X) [Computer software]. SAS Institute Inc.</bibtext> </blist> <blist> <bibtext> Schleppegrell, M. (2004). The language of schooling: A functional linguistics perspective. Lawrence Erlbaum Associates.</bibtext> </blist> <blist> <bibtext> Schlotterbeck, D., Uribe, P., Araya, R., Jimenez, A., & Caballero, D. (2021, April). What classroom audio tells about teaching: A cost‐effective approach for detection of teaching practices using spectral audio features. In LAK21: 11th international learning analytics and knowledge conference (pp. 132 – 140). ACM.</bibtext> </blist> <blist> <bibtext> Shin, T., Razeghi, Y., Logan, R. L., IV, Wallace, E., & Singh, S. (2020). Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980. https://arxiv.org/abs/2010.15980</bibtext> </blist> <blist> <bibtext> Snijders, T. A. B., & Bosker, R. J. (2011). Multilevel analysis. An introduction to basic and advanced multilevel modeling. SAGE Publications Inc.</bibtext> </blist> <blist> <bibtext> Wagner, W., Göllner, R., Werth, S., Voss, T., Schmitz, B., & Trautwein, U. (2016). Student and teacher ratings of instructional quality: Consistency of ratings over time, agreement, and predictive power. Journal of Educational Psychology, 108, 705 – 721. https://doi.org/10.1037/edu0000075</bibtext> </blist> <blist> <bibtext> White, M., & Ronfeldt, M. (2024). Monitoring rater quality in observational systems: Issues due to unreliable estimates of rater quality. Educational Assessment, 29, 124 – 146. https://doi.org/10.1080/10627197.2024.2354311</bibtext> </blist> <blist> <bibtext> Zhang, G., Zhou, Y., & Bollegala, D. (2024). Evaluating unsupervised dimensionality reduction methods for pretrained sentence embeddings. arXiv preprint arXiv:2403.14001.</bibtext> </blist> </ref> <aug> <p>By Richard Göllner; Rebecca Lazarides and Philipp Stark</p> <p>Reported by Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib25" firstref="ref2"></nolink> <nolink nlid="nl2" bibid="bib27" firstref="ref3"></nolink> <nolink nlid="nl3" bibid="bib39" firstref="ref4"></nolink> <nolink nlid="nl4" bibid="bib15" firstref="ref6"></nolink> <nolink nlid="nl5" bibid="bib40" firstref="ref7"></nolink> <nolink nlid="nl6" bibid="bib21" firstref="ref8"></nolink> <nolink nlid="nl7" bibid="bib14" firstref="ref9"></nolink> <nolink nlid="nl8" bibid="bib16" firstref="ref10"></nolink> <nolink nlid="nl9" bibid="bib19" firstref="ref11"></nolink> <nolink nlid="nl10" bibid="bib22" firstref="ref12"></nolink> <nolink nlid="nl11" bibid="bib28" firstref="ref13"></nolink> <nolink nlid="nl12" bibid="bib20" firstref="ref14"></nolink> <nolink nlid="nl13" bibid="bib44" firstref="ref15"></nolink> <nolink nlid="nl14" bibid="bib33" firstref="ref16"></nolink> <nolink nlid="nl15" bibid="bib38" firstref="ref17"></nolink> <nolink nlid="nl16" bibid="bib29" firstref="ref19"></nolink> <nolink nlid="nl17" bibid="bib32" firstref="ref21"></nolink> <nolink nlid="nl18" bibid="bib36" firstref="ref22"></nolink> <nolink nlid="nl19" bibid="bib11" firstref="ref23"></nolink> <nolink nlid="nl20" bibid="bib31" firstref="ref24"></nolink> <nolink nlid="nl21" bibid="bib48" firstref="ref25"></nolink> <nolink nlid="nl22" bibid="bib23" firstref="ref28"></nolink> <nolink nlid="nl23" bibid="bib45" firstref="ref29"></nolink> <nolink nlid="nl24" bibid="bib18" firstref="ref30"></nolink> <nolink nlid="nl25" bibid="bib41" firstref="ref33"></nolink> <nolink nlid="nl26" bibid="bib17" firstref="ref34"></nolink> <nolink nlid="nl27" bibid="bib12" firstref="ref35"></nolink> <nolink nlid="nl28" bibid="bib26" firstref="ref36"></nolink> <nolink nlid="nl29" bibid="bib13" firstref="ref38"></nolink> <nolink nlid="nl30" bibid="bib24" firstref="ref41"></nolink> <nolink nlid="nl31" bibid="bib46" firstref="ref42"></nolink> <nolink nlid="nl32" bibid="bib37" firstref="ref45"></nolink> <nolink nlid="nl33" bibid="bib10" firstref="ref46"></nolink> <nolink nlid="nl34" bibid="bib35" firstref="ref49"></nolink> <nolink nlid="nl35" bibid="bib42" firstref="ref50"></nolink> <nolink nlid="nl36" bibid="bib50" firstref="ref53"></nolink> <nolink nlid="nl37" bibid="bib34" firstref="ref54"></nolink> <nolink nlid="nl38" bibid="bib43" firstref="ref55"></nolink> <nolink nlid="nl39" bibid="bib47" firstref="ref56"></nolink> <nolink nlid="nl40" bibid="bib30" firstref="ref64"></nolink> <nolink nlid="nl41" bibid="bib49" firstref="ref66"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1483482
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Revealing Teaching Quality through Lesson Semantics: A GPT-Assisted Analysis of Transcripts
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Richard+Göllner%22">Richard Göllner</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-9442-7616">0000-0002-9442-7616</externalLink>)<br /><searchLink fieldCode="AR" term="%22Rebecca+Lazarides%22">Rebecca Lazarides</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-0392-4981">0000-0003-0392-4981</externalLink>)<br /><searchLink fieldCode="AR" term="%22Philipp+Stark%22">Philipp Stark</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22British+Journal+of+Educational+Psychology%22"><i>British Journal of Educational Psychology</i></searchLink>. 2025 95(1):300-315.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 16
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Teacher+Effectiveness%22">Teacher Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Quality%22">Educational Quality</searchLink><br /><searchLink fieldCode="DE" term="%22Semantics%22">Semantics</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Uses+in+Education%22">Technology Uses in Education</searchLink><br /><searchLink fieldCode="DE" term="%22Transcripts+%28Written+Records%29%22">Transcripts (Written Records)</searchLink><br /><searchLink fieldCode="DE" term="%22Lesson+Plans%22">Lesson Plans</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Indicators%22">Educational Indicators</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/bjep.70001
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0007-0998<br />2044-8279
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: Existing conceptions of teaching quality assume that classroom interactions serve as the foundation for effective teaching. The resulting data necessitates analytical approaches capable of extracting the semantics of these interactions. Aim: This study investigates whether and to what extent lesson semantics provide insights into teaching quality (i.e., cognitive engagement, encouragement and warmth, multiple approaches, and the nature of discourse). To achieve this, GPT-4 was applied as a tool for analysing lesson transcripts. Sample: The study is based on data from the TALIS Video study, which included N = 50 teachers delivering two consecutive mathematics lessons in 9th grade. Teaching quality was annotated by trained observers across multiple dimensions. Method: The analysis involved embedding segmented lesson transcripts to examine their semantic characteristics and associations with human annotations of teaching quality. Additionally, we applied content-informed prompting to evaluate the interpretability of semantic characteristics for the considered dimensions. Results: GPT-4 identified five distinct semantic representations of transcripts, varying at both the teacher and lesson levels. These representations were related to teaching quality, accounting for up to 20% of variance in teaching quality annotations. Content-informed prompting aligned lesson segments more closely with semantic representations, supporting their interpretability. Conclusion: The findings suggest that lesson semantics serve as indicators of teaching quality, offering a promising approach to understanding effective classroom learning.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1483482
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1483482
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/bjep.70001
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 16
        StartPage: 300
    Subjects:
      – SubjectFull: Teacher Effectiveness
        Type: general
      – SubjectFull: Educational Quality
        Type: general
      – SubjectFull: Semantics
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Technology Uses in Education
        Type: general
      – SubjectFull: Transcripts (Written Records)
        Type: general
      – SubjectFull: Lesson Plans
        Type: general
      – SubjectFull: Educational Indicators
        Type: general
    Titles:
      – TitleFull: Revealing Teaching Quality through Lesson Semantics: A GPT-Assisted Analysis of Transcripts
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Richard Göllner
      – PersonEntity:
          Name:
            NameFull: Rebecca Lazarides
      – PersonEntity:
          Name:
            NameFull: Philipp Stark
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 09
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0007-0998
            – Type: issn-electronic
              Value: 2044-8279
          Numbering:
            – Type: volume
              Value: 95
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: British Journal of Educational Psychology
              Type: main
ResultId 1