Full Text (PDF)
Abstract
This exploratory field study investigates how human-screened exam tasks generated by the large language model (LLM) GPT-4o perform in comparison with conventionally developed exam tasks with regard to selected quality criteria. Using two business administration modules at a German distance-learning university, human-screened LLM-generated tasks were integrated into submission assignments (SA) alongside instructor-created tasks. Four selected task-quality indicators were examined: objective difficulty, subjective difficulty, context relevance, and item discrimination. The analysis combines descriptive statistics with paired-sample t-tests and Wilcoxon signed-rank tests, applied at three aggregation levels: SA-specific, module-level, and cross-module. The results revealed mixed and task-specific patterns. While selected indicators were numerically similar for some task pairs, statistically significant and, in one comparison, substantial differences were observed in others, particularly in objective difficulty. Given that the study examined only four task pairs that were not systematically matched and that prior human screening was conducted, the findings establish neither equivalence nor general comparability. However, the study provides initial exploratory insights from two business administration modules that can serve as a foundation for further research.
License
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Article Type: Research Article
Journal of Digital Educational Technology, Volume 6, Issue 2, October 2026, Article No: ep2622
https://doi.org/10.29333/jdet/19699
Publication date: 29 Oct 2026
Article Views: 10
Article Downloads: 6
Open Access References How to cite this article