LLMs in Higher Education: A Comparative Evaluation of Mathematical Problem-Solving Capabilities and a Bloom-Aligned Human-AI Collaboration Framework

Koceska, Natasa and Koceski, Saso and Potes Barbas, Maria and Tales Vieira, Andreia (2026) LLMs in Higher Education: A Comparative Evaluation of Mathematical Problem-Solving Capabilities and a Bloom-Aligned Human-AI Collaboration Framework. Calitatea Vieții (Quality of Life), 37 (3). ISSN 1844-5292

Full text not available from this repository.

Abstract

This paper presents a comparative evaluation of three freely accessible Large Language Models (LLMs) such as ChatGPT-4o, Gemini 2.5 Flash, and Claude 4.5 Sonnet, with respect to their capability to solve ordinary difference equations spanning linear homogeneous, non-homogeneous, and system-type formulations. A structured dataset of twenty problems was developed, and solutions were independently assessed by three domain experts using a standardised rubric; inter-rater reliability was confirmed through intraclass correlation coefficients exceeding 0.9. Quantitative evaluation employed precision, recall, F1-score, and singlefactor ANOVA, which confirmed statistically significant inter-model differences at p < 0.05. All models achieved median performance scores above 7 out of 10, with ChatGPT recording the highest precision and F1-score. A systematic qualitative analysis, framed within Bloom’s Taxonomy, reveals that LLMs demonstrate reliable autonomous competence at the foundational cognitive levels, Remember, Understand, and partially Apply and Analyse, while the Evaluate and Create levels remain exclusively within the domain of human cognitive contribution. This boundary, empirically identified in the present study, provides the theoretical basis for the Human-AI Collaboration framework for higher-education STEM instruction grounded in Bloom’s Inverted Taxonomy, proposed herein: instruction should begin at the generative and evaluative apex of the cognitive hierarchy, where human contribution is indispensable, with LLMs serving as scaffolding infrastructure for the lower cognitive levels. The study further identifies a recurring failure mode common to all models − correct procedural narration coupled with arithmetic execution errors and discusses its value as a deliberate instructional resource. The results have direct relevance for educators, curriculum designers, and researchers seeking to integrate LLMs responsibly into mathematical learning environments.

Item Type: Article
Subjects: Natural sciences > Computer and information sciences
Divisions: Faculty of Computer Science
Depositing User: Natasa Koceska
Date Deposited: 30 Oct 2026 09:08
Last Modified: 30 Oct 2026 09:08
URI: https://eprints.ugd.edu.mk/id/eprint/39339

Actions (login required)

View Item
View Item