Falcon-Emirati: When an LLM Learns the Dialect, the Culture, and the Nuance

The Falcon-Emirati model was developed to understand Emirati Arabic, a Gulf dialect distinct from Modern Standard Arabic (MSA). While MSA is used in formal contexts, Emirati Arabic is crucial for daily conversation, humor, and storytelling in the UAE. The model's performance was evaluated using open-ended generation on 1,173 Alyah questions, scored by an LLM judge (Gemini 3.7 Flash) against models like ALLaM-7B-Instruct-preview and Jais-2-8B-Chat, to assess its ability to produce Emirati Arabic rather than defaulting to MSA.

This report uniquely details the evaluation of an LLM's dialect fidelity using open-ended generation and an LLM judge, unlike typical multiple-choice accuracy metrics.