GENERATION OF MULTIPLE-CHOICE GRAMMAR EXERCISES WITH LLMs: A LANGUAGE-ORIENTED PERSPECTIVE

Multiple-Choice Cloze (MCC) exercises have long played a fundamental role in language teaching and testing alike. However, the creation of high-quality MCC items is often a labor-intensive process that necessitates expert knowledge. These exercises must not only be relevant to test language competencies, but also be engaging enough to keep the learners motivated. Distractors are also central: they must instill doubt to the learner, form a wrong sentence if keyed into the exercise, focus solely on the object of testing and rule out external factors. Despite the increasing interest in applying LLMs to educational scenarios, their experimentation with MCC exercises generation is scarce. Previous attempts have hardly gone further than deterministic approaches, such as using fixed part-of-speech sequences for the solution of the exercise, or generating distractors with rule-based, context-agnostic mechanisms. The present analysis of existing MCC exercise methods reveals significant issues related to diversity, reliability and consistency, alongside a critical gap in standardized automatic metrics for evaluating the quality of generated exercises. To address these challenges, it is herein hypothesized that LLMs can produce self-contained sentences offering creative items and context-aware distractors without pre-defined constraints. Herein is presented an LLM-based solution for generating MCC exercises, the curation a comprehensive dataset covering 19 distinct grammar topics, and the proposal for an automatic evaluation metric for the structural well-formedness of items, validated against human expert assessment. This research aims to advance the automatic generation of English grammar MCC exercises, enhancing both their quality and creativity, thereby contributing to more effective language education and improved learner engagement.