Cloudflare Cooked

Summary: Cloudflare “Cooked” — saving memory in 1.1.1.1 DNS caching

The video explains how Cloudflare optimized its DNS caching for 1.1.1.1 to reduce fleet-wide RAM usage, ultimately claiming ~100 terabytes saved by focusing on Rust data-structure and memory-layout inefficiencies. It starts with a primer on DNS resolution and caching, then walks through several memory optimizations.

DNS background (why caching matters)

• A DNS query for a hostname like example.com involves asking higher-level DNS authorities until the authoritative nameserver provides records (e.g., an A record).
• After Cloudflare receives the DNS answer, it caches results so subsequent clients can be served faster.
• The video emphasizes that because caching is persistent, even small per-entry memory waste scales massively.

Key scaling claim

• Cloudflare DNS caches 250+ billion DNS cache entries “at any given time.”
• Even a single wasted byte per entry becomes ~250 GB across the fleet, motivating extremely granular savings.

---

Memory optimizations discussed

1) Replace Vec capacity waste with slices/boxed slices

• In Rust, a Vec stores pointer + length + capacity.
• Vec may allocate extra reserved capacity beyond what’s actually used.
• Since cached DNS entries are not updated after insertion, the video suggests using a slice or boxed slice instead of Vec, eliminating unused capacity.

Claimed savings:

• Turning vectors into boxed slices saves about 80 bytes of wasted capacity in the illustrative example.
• For the DNS cache entry structure holding multiple vectors/strings, this adds up to ~15+ TB saved (with 250B+ entries).

---

2) Collapse repeated nested structures using offsets

The cache entries repeatedly store similar substructures, such as:

• A boxed slice of “records,” plus repeated pointer/length pairs for fields like authority and additional.

Optimization idea:

• Instead of storing multiple small allocations/structures, represent everything in one slice and keep offsets for where authority/additional data begins.

Impact (as described in the video):

• Reduces per-entry representation size (claimed roughly 48 bytes down to ~20 bytes, noting alignment could affect the exact figure).
• The video frames this as another tens-of-terabytes-scale optimization (smaller than the biggest win, but still significant).

---

3) Remove redundant strings when the cache key already contains the name

• DNS records include an owner/name string and TTL/type/RDATA.
• But the video notes that the cache key already includes the DNS name.

Therefore:

• If the record name equals what’s already in the cache key, the record can omit the redundant owner string.
• Otherwise, it uses an optional boxed string.

This reduces memory by avoiding repeated storage of identical names.

---

4) Biggest win: change enum sizing via boxing large variants

• The video points to a Rust enum that stores multiple DNS record types.
• Rust enums require all variants to be the same size, so the enum becomes as large as the largest variant.
  • Example: an IPv4/A payload is small, while a NAPTR variant is very large (video claims ~136 bytes).

Why it wastes memory:

• Traffic is dominated by A and AAAA, so the enum wastes padding sized for variants that are mostly unused.

Fix:

• Box the enum variant’s large payload (move it to the heap), shrinking the enum’s in-memory size.

Trade-off:

• Boxing can hurt cache locality (data lives in multiple memory locations).
• The video says the associated blog post addresses compensating performance impacts.

Reported results from the blog (as stated in the video):

• 99th percentile memory: 9.3 GB → 5.3 GB (~43% reduction)
• P90 memory: 6.5 GB → 3.8 GB (~42% reduction)
• Cache-heavy instances saw larger absolute savings
• Insert throughput: 625k entries/sec → ~900k entries/sec
• Lookup latency: ~150 ns faster (~19%)

Final claimed aggregate impact:

• ~100 TB saved across the entire fleet

---

Review / guide / tutorial angle

This is not presented as a product review, but as a technical walkthrough of:

• a DNS primer
• a set of Rust memory-optimization techniques
• and results/benchmarks reportedly from Cloudflare’s blog (the video recommends reading it)

---

Main speakers / sources

• Source: Cloudflare engineering content (video narration referencing a Cloudflare blog about DNS cache optimizations, specifically 1.1.1.1).
• In-video persona: Subtitles reference a speaker who jokes about “rust” and presents the optimizations, including mention of sponsor voices (e.g., Jippy and Prime) during the Sentry ad segment.