Learning to Rank Question Difficulty from Text-Question Alignment using Deep Representations (#1910)

Authors

Cordova, Vicente

Nicolis, Orietta

Valdenegro-Toro, Matias

Peralta, Billy

Abstract

In a context where artificial intelligence and large language models (LLMs) are radically transforming education, there is a pressing need to automatically evaluate and organize the content generated by these technologies—particularly assessment questions. This work addresses the technical challenge of classifying LLM-generated questions based on their difficulty, a problem often overlooked by traditional approaches that do not explicitly model the semantic relationship between the question and its source text. We propose a system that generates questions from input passages using LLMs such as Gemini, and classifies them via deep learning models trained with embeddings and regularization techniques, implemented in TensorFlow and PyTorch. Our methodology includes the creation of a custom dataset derived from SQuAD passages, the vectorization of texts and questions using various embedding strategies, and a comparative evaluation of multiple classification architectures. Experimental results show that the model based on paraphrase-MiniLM-L6-v2 achieves 90% bi-class accuracy. This supports the hypothesis that more difficult questions, due to their less ambiguous patterns, are classified with higher precision.