Xiaomi and DeepSeek Just Solved the Biggest Problem in AI

Core problem / loop being optimized (agent with long logs)

The video describes an agent loop where each step:

1. runs an NPM test,
2. receives ~20,000 tokens of logs/stack traces as observations,
3. then must run the next command.

Because the model must read all those returned tokens before it can act again, the context grows continuously, repeatedly triggering heavy prefill compute and KV-cache growth.

---

Xiaomi’s main contribution: Hypar Sparse 2 in Mimo V3

Xiaomi’s Fully Lu presents Hypar Sparse 2 as the core of Mimo V3’s architecture, addressing three bottlenecks at once:

1. Compute needed to read new long chunks (prefill cost)
2. Memory growth from the KV cache
3. Whether sparsity/reuse still lets the model find the few important lines in long tool output

Reported scaling improvements (at 1 million tokens)

• KV cache size reduced from 12.09 GB (Xiaomi prior) to 2.69 GB
• Prefill compute reduced by about 5× vs Hybrid SWA
• Retrieval quality on Xiaomi’s 256K long-context test improved—contrary to what you’d expect if context handling got more approximate

---

The “building” explanation: what attention/KV caching costs in long contexts

A metaphor is used: the model is a 49-layer “building.”

• Each token read causes each relevant layer to store:
  • KV cache entries (“filing cabinet” cards) with:
    • a key (token-related label)
    • a value (token contribution)

During generation, sparse/dense selection retrieves from those stored entries, but prefill still must populate them—which is especially expensive when repeated inside agent loops.

---

Two key tricks in Hypar Sparse 2

Trick 1: KV Bridging (reducing prefill work with an early exit)

Hypar Sparse 2 splits layers into two sections:

• Self-decoder (lower section):
  • reads all tokens normally
  • builds the KV cache/“notes”
• Cross-decoder (upper section):
  • does not re-read the raw tokens during prefill
  • instead fills its KV/cache from “handoff notes” produced by the lower section

Result: prefill can exit early at the handoff boundary.

The idea is linked to “Yoko” (decoder-decoder / “you-only-cache-once”) from 2024.

Tradeoff

Upper layers don’t construct a full independent view of every past token during prefill; they rely on summaries/notes from the handoff layer.

---

Trick 2: KV Reuse inside sparse attention blocks

Within each layer block:

• the first layer is dense/full-attention
  • computes importance
  • stores a proper KV cabinet
• above it are sparse layers that:
  • don’t build their own cabinets
  • borrow the dense layer’s KV data, but only for a selected list of important tokens (token-level picks)

This builds on earlier Xiaomi work (HighSparse) where sparse layers reused dense KV structures.

Claimed changes in HighSparse 2

1. Token-level selection (choose individual KV cards rather than whole groups)
2. Recent-context handling moved to a fixed shelf in the borrowed cabinet:
   • keep last 28 tokens + 1,024 selected ones

Architectural outcome (the “49 floors” view)

• Only 5 of 49 layers do full attention (down from 9 in Xiaomi’s earlier Hybrid SWA)

• fewer KV cabinets built → lower memory and compute

---

Evidence: retrieval test numbers (where the “borrowing” didn’t hurt)

On RULER DAJI 2 (256K tokens retrieval after light post-training):

• Hypar Sparse 2: 58.45
• Original HighSparse: 32.61
• Hybrid SWA: 35.74

Important caveat

KV/compute claims are shown at 1M tokens, but the retrieval table shown is only up to 256K. So the “no quality loss” evidence is not fully proven at the maximum context length in the presented results.

---

Relationship to DeepSeek: same two-trick pattern, close release timing

The video connects Xiaomi’s approach to DeepSeek V4 (OneFlash):

• DeepSeek released Sep 10
• Xiaomi’s paper appeared Sep 22

DeepSeek’s V4 uses the same conceptual pair:

1. KV bridging via a “causal encoder-decoder” (prefill cached once, then reused upward)
2. A sparse attention reuse mode where upper layers borrow KV and top-K token lists from lower layers

Naming differences (same idea under different labels)

• Xiaomi: KV bridging
• DeepSeek: causal encoder-decoder
• “Yoko”: decoder-decoder

The video frames these as the same idea with different names.

Citation trail (why it likely isn’t “copying”)

• DeepSeek cites Yolo (Microsoft + Tsinghua, May 2024) for the early-exit decoder/cache reuse.
• DeepSeek also credits Xiaomi’s HighSparse (Feb paper) for KV reuse in sparse layers.
• Xiaomi, in turn, cites DeepSeek’s report for another related trick (not necessarily the exact same pair).

---

“Which is better?”—what’s answerable from the papers

The video says you can’t directly compare “fairly” from the presented metrics because:

• DeepSeek reports bytes-of-cache-per-token (including FP4 assumptions)
• Xiaomi reports GB at 1M tokens under different model/testing setups

So the video suggests waiting for true production-equivalent comparisons once model weights and full evaluation details are available.

---

Practical takeaway / what to watch next

• If you’re building agent systems that feed back long tool outputs into context, this “KV bridging + KV reuse in sparse borrowed caching” line of work is positioned as important.
• The open question the video ends on:
  • When Mimo V3 weights ship and retrieval is tested at full 1 million tokens, does it still hold up?

---

Main speakers / sources

• Speaker: Xiaomi’s Fully Lu (announcement presentation)

• Referenced model/paper authors/systems:

  • DeepSeek V4 (OneFlash)
  • Xiaomi Mimo V3 / Hypar Sparse 2
  • Xiaomi HighSparse / HighSparse 2
  • Hybrid SWA (Mimo B2 series)
  • Yoko (2024)
  • Yolo (Microsoft + Tsinghua, May 2024)
  • RULER DAJI 2 retrieval test results (dataset/task)