Large Language Models for Educational Task Authoring: A Bebras Challenge Case Study
Saved in:
| Title: | Large Language Models for Educational Task Authoring: A Bebras Challenge Case Study |
|---|---|
| Language: | English |
| Authors: | Leonard Busuttil (ORCID |
| Source: | Informatics in Education. 2026 25(1):37-57. |
| Availability: | Vilnius University Institute of Mathematics and Informatics, Lithuanian Academy of Sciences. Akademjos str. 4, Vilnius LT 08663 Lithuania. Tel: +37-5-21-09300; Fax: +37-5-27-29209; e-mail: info@mii.vu.lt; Web site: https://infedu.vu.lt/journal/INFEDU |
| Peer Reviewed: | Y |
| Page Count: | 21 |
| Publication Date: | 2026 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Artificial Intelligence, Natural Language Processing, International Programs, Information Science, Computation, Thinking Skills, Computer Science, Educational Assessment, Test Content |
| ISSN: | 1648-5831 2335-8971 |
| Abstract: | This study explores the application of large language models (LLMs) to create computational thinking tasks for the Bebras International Challenge through a single-case study approach. Using exemplar-based prompting with seven authentic Bebras tasks from the 2024 cycle as contextual input, a task was developed that was subsequently accepted for inclusion in the 2025 international Bebras challenge. Comparison with the exemplar tasks confirmed that the generated content drew from multiple sources rather than replicating any single task, combining grid-based constraint satisfaction, rule-based filtering, and logical deduction into a novel navigation puzzle with engaging narrative context. International expert reviewers evaluated the task using established Bebras quality criteria, confirming successful alignment with core pedagogical requirements including age-appropriateness, clarity, and cultural neutrality. However, two significant gaps emerged in the broader authoring workflow: accessibility compliance in the researcher-authored visual components and technical inaccuracies in the LLM-generated informatics framing. Following collaborative revision by international editors that addressed these concerns while preserving the LLM's creative contributions, the task achieved acceptance for international use. The findings reveal a collaborative pipeline comprising contextual preparation, LLM-guided generation, human technical implementation, expert community review, and collaborative revision. Results from this case suggest that LLMs can efficiently generate educationally sound creative foundations while requiring integrated human expertise to meet specialised standards and ensure inclusive design, with the task's acceptance providing encouraging evidence for the viability of this collaborative approach. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | EJ1506550 |
| Database: | ERIC |
| Abstract: | This study explores the application of large language models (LLMs) to create computational thinking tasks for the Bebras International Challenge through a single-case study approach. Using exemplar-based prompting with seven authentic Bebras tasks from the 2024 cycle as contextual input, a task was developed that was subsequently accepted for inclusion in the 2025 international Bebras challenge. Comparison with the exemplar tasks confirmed that the generated content drew from multiple sources rather than replicating any single task, combining grid-based constraint satisfaction, rule-based filtering, and logical deduction into a novel navigation puzzle with engaging narrative context. International expert reviewers evaluated the task using established Bebras quality criteria, confirming successful alignment with core pedagogical requirements including age-appropriateness, clarity, and cultural neutrality. However, two significant gaps emerged in the broader authoring workflow: accessibility compliance in the researcher-authored visual components and technical inaccuracies in the LLM-generated informatics framing. Following collaborative revision by international editors that addressed these concerns while preserving the LLM's creative contributions, the task achieved acceptance for international use. The findings reveal a collaborative pipeline comprising contextual preparation, LLM-guided generation, human technical implementation, expert community review, and collaborative revision. Results from this case suggest that LLMs can efficiently generate educationally sound creative foundations while requiring integrated human expertise to meet specialised standards and ensure inclusive design, with the task's acceptance providing encouraging evidence for the viability of this collaborative approach. |
|---|---|
| ISSN: | 1648-5831 2335-8971 |
