Agnes-AI/Agnes-3.0-Flash 33B Multimodal, AA score: 36

"Built for demanding work. A 262 144-token context window, adjustable reasoning effort, tool calling, and text, image and video understanding.

Agnes-3.0-Flash is a hybrid-attention decoder: three of every four layers run a gated delta rule (recurrent, with per-layer state independent of sequence length), and the fourth runs standard global attention. Only 18 of the 72 layers therefore hold a KV cache that grows with context."

Context length 262 144 tokens Decoder layers 72 = 54 delta-rule recurrent + 18 global attention, alternating 3: 1 Hidden size 5120 Global attention 24 query heads / 4 KV heads (6: 1 GQA), head dim 256; RMS-norm on q and k, sigmoid-gated output Delta-rule layers 16 key heads / 48 value heads, head dim 128; causal conv (kernel 4) in front, gated RMS-norm; recurrent state in fp32 Feed-forward SwiGLU, intermediate size 17408; plus a parallel SwiGLU 2048 branch in every layer Positions 3-axis rotary (text / height / width), interleaved mrope sections 11: 11: 10, base 1e7, applied to the first 25 % of each head dim (64 dims) Vocabulary 248 320 Vision tower 27 layers, hidden 1152, patch 16, 2 × 2 spatial merge, projected to 5120Architecture Agnes-3.0-Flash is a hybrid-attention decoder: three of every four layers run a gated delta rule (recurrent, with per-layer state independent of sequence length), and the fourth runs standard global attention. Only 18 of the 72 layers therefore hold a KV cache that grows with context. Context length 262 144 tokensDecoder layers 72 = 54 delta-rule recurrent + 18 global attention, alternating 3: 1Hidden size 5120Global attention 24 query heads / 4 KV heads (6: 1 GQA), head dim 256; RMS-norm on q and k, sigmoid-gated outputDelta-rule layers 16 key heads / 48 value heads, head dim 128; causal conv (kernel 4) in front, gated RMS-norm; recurrent state in fp32Feed-forward SwiGLU, intermediate size 17408; plus a parallel SwiGLU 2048 branch in every layerPositions 3-axis rotary (text / height / width), interleaved mrope sections 11: 11: 10, base 1e7, applied to the first 25 % of each head dim (64 dims)Vocabulary 248 320Vision tower 27 layers, hidden 1152, patch 16, 2 × 2 spatial merge, projected to 5120 Edit: AA shows its 'Proprietary model'. The name is same as at HF but benchmarks results and context are different. So maybe it's not same model - https://artificialanalysis.ai/models/agnes-3-0-flash

Edit2: As they edit readme at HF to clarify: both models are totally different and AA score isn't correct for HF model (I can't edit title post to remove it tho).