How AraBERTv2 Models Revolutionize Automatic Short Answer Grading in Arabic?

In recent years artificial intelligence has started to help even in schools, especially for grading short written answers. In Arabic, however, the task is tougher: the language has complex morphology, right‑to‑left script, and many word forms. To tackle these challenges, researchers have developed Arabic‑specific BERT versions, called AraBERTv2, trained on massive Arabic text corpora.

The new study compares several fine‑tuning strategies of these models, i.e., small adjustments made specifically for the grading task. Some experiments increased the transformer depth, others changed the number of attention heads or token size. The goal? Find the best trade‑off between accuracy (how close the model is to a human grader) and speed (how fast it evaluates an answer).

Results show that a model with 8 layers and 12 attention heads, trained for 3 epochs on a dataset of 10,000 answers, reaches an F1 score of 87 %, almost on par with teachers, while cutting inference time by 30 % compared to larger versions. Moreover, analysis highlighted that questions containing rare keywords are the hardest to grade, suggesting new ways to enrich the model’s vocabulary.

The research proves that, with the right architectural choices, fast and reliable automatic grading systems can be built, ready to support teachers and make learning more interactive.

AraBERTv2 is a BERT variant pre‑trained on over 70 GB of Arabic text, covering both Modern Standard Arabic and the most common dialects. The model uses WordPiece tokenization, which splits words into sub‑tokens to handle the language's rich morphology. During pre‑training, AraBERTv2 learns two tasks: Masked Language Modeling (MLM) and Next Sentence Prediction (NSP), allowing it to capture both intra‑sentence context and inter‑sentence relationships.

For fine‑tuning on short‑answer grading, the authors experimented with three main configurations: (1) increasing depth from 6 to 12 layers, (2) varying the number of attention heads from 8 to 16, and (3) changing the hidden size from 768 to 1024. Each configuration was trained for 2‑4 epochs on a balanced dataset of 10,000 answers, with scores ranging from 0 to 5 provided by experts.

Evaluation metrics included F1‑score, accuracy, and average inference time per answer. Results show that deeper models improve F1 by about 2 points but increase inference time by 45 %. Adding more attention heads yields a marginal F1 gain (≈0.8 points) with minimal computational overhead. The larger hidden size offers the best trade‑off, achieving an 87 % F1 and a 30 % reduction in inference time compared to the deepest model.

Error analysis revealed that answers containing technical terms or neologisms absent from AraBERTv2's original vocabulary tend to receive lower scores. To mitigate this, the authors suggest periodically updating the vocabulary with tokens extracted from new educational corpora. Additionally, data‑augmentation techniques such as automatic paraphrasing can boost model robustness across linguistic variants. Future work may integrate symbolic reasoning models to better assess answers requiring logical inference, paving the way for more intelligent, context‑aware grading systems.