CYBER WATCH / EQUINOXE INTELLIGENCE

← All articles

b12170: moe-cache: LRFU/MQ-like hybrid cache, ClassLRU (#31233)

llama.cpp Releases · 2026-11-06 16:02:01

The commit introduces a hybrid cache inspired by LRFU/MQ, advances the clock by N tokens, uses the actual token count, explains key calculation, scores based on age, removes a branch, clarifies the exp2/log2 relationship, rewrites the header comment, details unused slots and sorting, avoids copying, reverts a previous commit, adds key and iteration helpers, extracts decayed_crf, bumps the number of classes to 8, documents the formula, displays the cache size in token‑worth count, and asserts that the computed number of classes matches the half‑life.

Read original source ↗