Search Results - "Interrater Reliability"

Seeing Is Solving: MLLMs, Reasoning, and Refusal in Visual Math

Authors: Ethan Croteau Neil Heffernan

Source: Journal of Educational Data Mining. 2026 18(1):244-285.

Peer Reviewed: Y

Page Count: 42

Descriptors: Artificial Intelligence, Natural Language Processing, Models, Intelligent Tutoring Systems, Middle School Mathematics, Visual Aids, Graphs, Decision Making, Problem Solving, Computation, Accuracy, Error Patterns, Interrater Reliability

Automating Self-Affirmation Essay Coding: Fine-Tuned BERT Performance Comparable to Human Coders and Comparison with GPT-4

Authors: Cong Ye Trisha H. Borman Geoffrey D. Borman

Source: Journal of Educational Data Mining. 2026 18(1):66-88.

Peer Reviewed: Y

Page Count: 23

Sponsoring Agency: Institute of Education Sciences (ED)

Descriptors: Automation, Artificial Intelligence, Technology Uses in Education, Coding, Essays, Accuracy, Interrater Reliability

IES Funded: Yes

Examining Rater Effects in Assessing Learners' Speech and Writing: Insights from ICNALE

Authors: Hyunwoo Kim

Source: English Teaching. 2026 81(1):59-80.

Peer Reviewed: Y

Page Count: 22

Descriptors: Writing Evaluation, Speech Evaluation, Interrater Reliability, Bias, Scoring, English Teachers, English (Second Language), Experience, Evaluators, College Students, Essays, Foreign Countries

Geographic Terms: Asia

Introducing Sentinel Assessment: An AI-Powered Functionality for Evaluating Writing Competence in French as a Foreign Language

Authors: Abdelghani Es-sarghini Abdelaziz Boumahdi

Source: Language Education & Assessment. 2026 9.

Peer Reviewed: Y

Page Count: 22

Descriptors: Artificial Intelligence, French, Second Language Learning, Writing Skills, Writing Evaluation, Elementary School Students, Grade 6, Foreign Countries, Technology Uses in Education, Formative Evaluation, Interrater Reliability

Geographic Terms: Morocco

Examining Behavioral Ratings between Preschool Lead and Assistant Teachers Using Dyadic Multigroup Invariance

Authors: Ruiqin Gao Fred Greer Christine DiStefano et al.

Source: Journal of Psychoeducational Assessment. 2026 44(4):418-433.

Peer Reviewed: Y

Page Count: 16

Sponsoring Agency: Institute of Education Sciences (ED)

Descriptors: Preschool Teachers, Teacher Aides, Preschool Children, Student Behavior, Behavior Rating Scales, Screening Tests, Factor Structure, Interrater Reliability

Geographic Terms: South Carolina

IES Funded: Yes

Evaluating Rater Effects of Large Language Models in Automated Essay Scoring: GPT, Claude, Gemini, and DeepSeek

Authors: Hong Jiao Dan Song Won-Chan Lee

Source: Educational Measurement: Issues and Practice. 2026 45(2).

Peer Reviewed: Y

Page Count: 17

Descriptors: Artificial Intelligence, Natural Language Processing, Automation, Computer Assisted Testing, Scoring, Evaluators, Interrater Reliability, Accuracy, Item Response Theory

Parent-Teacher Discrepancies about the Mental Health of Siblings of Children with Chronic Disorders: A Latent Profile Analysis

Authors: Caitlin M. Prentice Susanne Wehrli Erica Zahl et al.

Source: Journal of Emotional and Behavioral Disorders. 2026 34(2):106-116.

Peer Reviewed: Y

Page Count: 11

Descriptors: Parent Attitudes, Teacher Attitudes, Siblings, Mental Disorders, Child Behavior, Interrater Reliability, Foreign Countries, Gender Differences, Age Differences, Behavior Problems, Children, Adolescents, Mental Health, Well Being

Geographic Terms: Norway

Assessment and Survey Identifiers: Strengths and Difficulties Questionnaire

The Connecticut State Department of Education's Implementation of Selected Components of Connecticut's Statewide Accountability System. ED-OIG/A24IL0199

Authors: Office of Inspector General (ED)

Source: Office of Inspector General, US Department of Education. 2025.

Peer Reviewed: N

Page Count: 29

Descriptors: Elementary Secondary Education, Educational Legislation, Federal Legislation, Accountability, Program Implementation, School Effectiveness, Academic Achievement, Achievement Gap, State Aid, Public Schools, Identification, Educational Needs, Educational Assessment, Evaluation Criteria, Interrater Reliability

Geographic Terms: Connecticut

Laws, Policies and Program Identifiers: Elementary and Secondary Education Act Title I, Elementary and Secondary Education Act, Every Student Succeeds Act 2015

Let's Measure Ready: A 50-State Analysis of College, Career, Military, and Civic Readiness Indicators. Research Report

Authors: Urban Institute Alliance for Excellent Education (All4Ed) Anne Hyslop

Source: Urban Institute. 2025.

Peer Reviewed: N

Page Count: 82

Sponsoring Agency: National Governors Association (NGA)

Descriptors: College Readiness, Career Readiness, High School Students, High Schools, State School District Relationship, Accountability, Aptitude Tests, Armed Forces, Vocational Aptitude, College Entrance Examinations, Student Evaluation, Outcome Measures, College Enrollment, Enrollment Rate, Interrater Reliability, Dual Enrollment, Citizenship Responsibility, Benchmarking

Assessment and Survey Identifiers: Armed Services Vocational Aptitude Battery, Armed Forces Qualification Test, ACT Assessment, SAT (College Admission Test)

Limitations of QWK in Evaluating Automated and Human Scoring Systems

Authors: Jennifer Lewis Jodi M. Casabianca

Source: Educational Measurement: Issues and Practice. 2026 45(1).

Peer Reviewed: Y

Page Count: 14

Descriptors: Interrater Reliability, Evaluation Methods, Measurement Techniques, Automation, Artificial Intelligence, Scores, Responses

Exploring Reading Rate, Miscues and Prosody in the Fluent Reading of Adults in a Transparent Language

Authors: Hale Hancer Pinar Civak Tan

Source: European Journal of Education. 2026 61(1).

Peer Reviewed: Y

Page Count: 15

Descriptors: Reading Rate, Miscue Analysis, Suprasegmentals, Reading Fluency, Adults, Reading Skills, Turkish, Reading Tests, Content Validity, Factor Analysis, Test Reliability, Factor Structure, Interrater Reliability

Aşşeşşorş' Enğağement wıth Vıdeo-Recorded Performance Aşşeşşmentş ın Nurşınğ: A Qualıtatıve Study

Authors: Conor Scully Mıchael O’Leary Zıta Lyşağht et al.

Source: Research & Practice in Assessment. 2025 20(2):105-117.

Peer Reviewed: Y

Page Count: 13

Descriptors: Nursing Students, Undergraduate Students, Performance Based Assessment, Video Technology, Student Evaluation, Nursing Education, Evaluators, Validity, Interrater Reliability, Cognitive Processes

Optimizing Inter-Rater Reliability in Foreign Language Constructed-Response Assessments

Authors: Jose Fabian Elizondo-Gonzalez (ORCID 0000-0003-4819-0213)

Source: Research in Pedagogy. 2025 15(2):471-482.

Peer Reviewed: Y

Page Count: 12

Descriptors: Interrater Reliability, Second Language Learning, English (Second Language), Language Tests, Language Proficiency, Foreign Countries, Test Reliability, Error Patterns, Scoring, Prompting, Responses, Test Construction

Geographic Terms: Costa Rica

Assessment and Survey Identifiers: Test of English as a Foreign Language

Systematic Comparison of Two Approaches for Evaluating and Using Rater-Mediated Performance Assessments

Authors: Chunling Niu Kelly Bradley Rui Jin et al.

Source: Practical Assessment, Research & Evaluation. 2025 30(1).

Peer Reviewed: Y

Page Count: 37

Descriptors: Evaluation Methods, Interrater Reliability, Test Bias, Research Problems, Performance Based Assessment, Scoring Rubrics, Factor Analysis, Error of Measurement, Evaluators

AIGC-Assisted Evaluation of Teachers' Tour-Guide Scripts: Construction and Practice of the Six-Step Method

Authors: Lingman Zhou Ting Xu Yunyao Zhang

Source: Asian Journal of Education and Training. 2025 11(4):224-232.

Peer Reviewed: Y

Page Count: 9

Descriptors: Artificial Intelligence, Scripts, Pedagogical Content Knowledge, Technological Literacy, Writing Evaluation, Career and Technical Education, Secondary School Students, Tourism, Computer Uses in Education, Foreign Countries, Career and Technical Education Teachers, Secondary School Teachers, Teacher Attitudes, Interrater Reliability

Geographic Terms: China

AI-Based L2 Pragmatic Speaking Assessment: Evidence from Many-Facet Rasch Measurement

Authors: Soo Jung Youn

Source: English Teaching. 2025 80(4):3-28.

Peer Reviewed: Y

Page Count: 26

Descriptors: Artificial Intelligence, Second Language Learning, Computer Assisted Testing, Pragmatics, Language Tests, Oral Language, Item Response Theory, English (Second Language), Evaluators, Test Items, Difficulty Level, Evaluation Criteria, Scoring, Interrater Reliability

Can AI Grade Like a Human? Validity, Reliability, and Fairness in University Coursework Assessment

Authors: Georgios Zacharis Stamatios Papadakis

Source: Educational Process: International Journal. Article e2025591 2025 19.

Peer Reviewed: Y

Page Count: 22

Descriptors: Artificial Intelligence, Technology Uses in Education, Computer Assisted Testing, Grading, Student Evaluation, College Students, Evaluators, Scoring, Foreign Countries, Interrater Reliability, High Stakes Tests, Construct Validity, Test Reliability, Formative Evaluation, Feedback (Response), Test Bias

Geographic Terms: Greece

Artificial Intelligence in International English Language Testing System Writing Assessments: A Comparative Study of Human Ratings and DeepAI

Authors: Somayeh Fathali Fatemeh Mohajeri

Source: Technology in Language Teaching & Learning. 2025 7(4).

Peer Reviewed: Y

Page Count: 18

Descriptors: English (Second Language), Language Tests, Second Language Learning, Artificial Intelligence, Writing Tests, High Stakes Tests, Foreign Countries, Scoring, Interrater Reliability, Automation

Geographic Terms: Iran (Tehran)

Assessment and Survey Identifiers: International English Language Testing System

Can AI Assess Writing Skills Like a Human? A Reliability Analysis

Authors: Hüseyin Ataseven Ömay Çokluk-Bökeoglu Fazilet Tasdemir

Source: Journal of Theoretical Educational Science. 2025 18(4):736-754.

Peer Reviewed: Y

Page Count: 19

Descriptors: Artificial Intelligence, Technology Uses in Education, Writing Skills, Student Evaluation, Scoring, English (Second Language), Interrater Reliability, College Students, Foreign Countries

Geographic Terms: Turkey

The Impact of Online Peer Assessment on Academic Performance in Higher Education: A Meta-Analytic Review

Authors: Kübra Karakaya Özyer

Source: Journal of Educators Online. 2025 22(3).

Peer Reviewed: Y

Page Count: 15

Descriptors: Electronic Learning, Peer Evaluation, Higher Education, Meta Analysis, Academic Achievement, Student Evaluation, College Students, Interrater Reliability, Effect Size, Training, Outcome Measures, Psychometrics