CYBER WATCH / EQUINOXE INTELLIGENCE

← All articles

b12293 : cpu : accumulate flash attention V in F32 (#29810)

llama.cpp Releases · 2026-11-12 00:01:18

The one‑chunk kernel previously accumulated the V cache in F16, causing overflow past 65504 on long contexts and resulting in inf and NaN values. The update now accumulates V in F32, matching the tiled kernel. F16 rows use ggml_vec_mad_f16_f32, avoiding a temporary F32 row and keeping master speed on x86 while losing only about 4 % on ARM at 4 k context. The code also widens F16 rows to follow ggml_vec_dot_f16_f32 guards, using AVX512, AVX with F16C, NEON, RVV, and falling back to a scalar loop if needed.

Read original source ↗