Assessment of Artificial Intelligence Literacy in Universities: A Critical Review (#1460)

Authors

Rojas Contreras, Mauricio

Villamizar Carrillo, Laura

Mendoza Bejarano, José Liviston

Quintana Suárez, Leydi Carolina

Villamizar Peña, Oscar Albeiro

Abstract

Artificial intelligence (AI) has become a transformative technology that redefines professional profiles and employability conditions, making it imperative for higher education institutions (HEIs) to prepare graduates to face these changes. This critical review examines the literature on the assessment of AI literacy (AIL) in the university context, supported by a multi-method approach that integrates critical review and hermeneutic triangulation. Existing conceptual frameworks, such as those from UNESCO and Ng et al., are analyzed, which converge on key dimensions: technical knowledge, application, critical evaluation, and ethics. A critical gap is evident between the high dependence on self-report instruments, which may be biased by individual overconfidence, and the emerging need for performance-based assessments that measure actual competence. Validated tools such as the AILQ (Artificial Intelligence Literacy Questionnaire) and the GLAT (Generative AI Literacy Assessment Test) performance test are highlighted, analyzing their structure and applicability. Institutional responses are explored, from "AI Across the Curriculum" curricular integration to the development of ethical policies. The findings indicate that while there is consensus on the dimensions of AIL, challenges persist in assessment standardization, academic integrity, and algorithmic bias. The conclusion underscores the urgency of adopting a multimodal approach that combines self-reports with performance tests to obtain a reliable measure of AIL, ensuring that graduates are truly prepared for an AI-driven labor market.