This shared task at ArabicNLP 2025 focuses on evaluating Arabic cultural understanding through two subtasks: General Culture and Islamic Culture. Using a high-quality, MCQ dataset in MSA, the task promotes the development of culturally aware models tailored to the diverse heritage of the Arab world.
Two complementary tasks addressing core challenges in Arabic cultural understanding.
This subtask aims to assess language models' understanding of key elements of Islamic culture, which plays a foundational role in Arabic societies. It covers topics such as Islamic rituals and practices (e.g., prayers and fasting), Quranic knowledge, Hadith literature, historical developments in Islam, and religious holidays. Models are expected to answer multiple-choice questions that reflect both religious literacy and contextual sensitivity, ensuring their ability to handle culturally and theologically significant content with accuracy and respect.
Key milestones for the ArabicNLP 2025 Shared Task.
Release of training/dev data and evaluation scripts.
Final registration deadline and test set release.
Test submission deadline via CodaLab.
Final results released to participants.
System description papers due.
ArabicNLP 2025 Workshop in Suzhou, China.
How to participate in the ArabicNLP 2025 Shared Task.
If you have any questions regarding your team's registration, please email us at:
palmx2025@gmail.comFor more updates or inquiries, join the PalmX Google group:
Researchers in Arabic NLP.
Fakhraddin Alwajih, The University of British Columbia (Canada)
Abdellah EL Mekki, The University of British Columbia (Canada)
Hamdy Mubarak, Qatar Computing Research Institute (Qatar)
Majd Hawasly, Computing Research Institute (Qatar)
Abubakr Mohamed, Qatar Computing Research Institute (Qatar)
Muhammad Abdul-Mageed (LinkedIn), The University of British Columbia (Canada)