DeepSeek Is Free Now but What’s the Catch?

Summary

The video argues that DeepSeek’s low-cost AI service is not explained by unusually cheap GPU compute alone. A key advantage is its high cache-hit rate: much of the context users send is retrieved from storage rather than recomputed by GPUs. DeepSeek released parts of its inference software but kept important infrastructure for managing that cache across machines.

Releases and product details

• V4.1 Flash 1017, released October 17, is a minor update to the September model. The narrator describes it as a 552-billion-parameter mixture-of-experts model that activates 16 billion parameters when generating and 8 billion when reading. It supports up to one million tokens of context, uses MIT-licensed weights, and kept the same pricing. Its DeepSWE coding benchmark score rose from 74.2 to 77.0.
• DeepSeek also launched a Search API at $0.75 per 1,000 requests. The video says free access to built-in web search was scheduled to end November 1.
• On October 20, DeepSeek released DeepInfer, its inference engine, under Apache 2.0. The video also mentions the Rust DeepSeek Recipe library for matching the official API behavior and the previously released 3FS distributed file system.

Why the headline GPU-profit calculation misses the point

DeepSeek’s documentation cited a cluster serving 155 trillion tokens per day and earning $38 per GPU-hour. Comparing that figure with GPU rental costs led many commentators to claim DeepSeek had enormous margins.

The narrator checks the workload’s scale against DeepSeek’s own benchmark: one GPU generates about 26,300 tokens per second, implying that roughly 68,000 GPUs would be needed to compute 155 trillion tokens daily from scratch. At $38 per GPU-hour, that would imply about $22.7 billion in annual revenue—far above the roughly $1 billion revenue run rate cited in the video.

The proposed explanation is that most of the reported tokens are cache hits, not newly computed tokens.

KV cache and the economics of repeated context

A model processes prior conversation, files, and tool results along with a user’s latest message. A KV cache stores the intermediate key/value data from already-processed text, so repeated context can be loaded instead of recalculated.

• A matching prompt can produce a cache hit. New or unmatched context is a miss and must be computed.
• The video says DeepSeek’s February 2025 cache-hit rate was 56.3%, versus 96% in the new DeepInfer material. That means the share of input tokens needing fresh computation fell from about 44 in every 100 to about four—roughly 11 times less input compute.
• DeepSeek’s V4.1 Flash technical report is described as reducing cache size to 890 bytes per token, about one-quarter of the previous Flash model’s footprint.
• The system reportedly uses about 100 pebibytes of SSD storage and retains agent conversation caches for three days and ordinary chatbot caches for seven days.

The video’s illustrative token mix is 95 cached input tokens, four computed input tokens, and one generated output token per 100 served. In that example, the single output token contributes about as much revenue as all 95 cached tokens combined. The GPU-hour figure therefore reflects not just GPU work, but an infrastructure system that stores and retrieves context efficiently.

What DeepSeek did—and did not—open-source

The release includes model weights, DeepInfer, and GPU kernels, among other components. However, the README lists several pieces needed for a complete inference service that are not included:

• The distributed KV cache system
• Separation of prefill and decode work across machines
• Load balancing across replicas

The narrator’s interpretation is that DeepSeek has shared much of the software that performs inference while keeping the systems that make cache reuse work at scale. SemiAnalysis is cited as finding that DeepInfer supports offline, single-turn inference rather than being a ready-to-run live service, and that the existing open-source engine vLLM had already matched its published benchmark results. The video frames the release partly as a way to encourage third-party hosting of DeepSeek’s weights.

Practical pricing and provider differences

The video says providers on OpenRouter list the same V4.1 Flash weights at the same headline rates:

• $0.30 per million input tokens
• $1.20 per million output tokens
• $0.006 per million cached input tokens

However, providers can differ in how long they preserve cached context and whether it can be reused across machines. Using those prices, the video estimates that one million input tokens cost:

• About $0.018 at a 96% cache-hit rate
• About $0.065 at an 80% cache-hit rate
• About $0.135 at a 56.3% cache-hit rate

Thus, the same listed price can result in substantially different bills depending on cache performance. The video contrasts DeepSeek’s reported multi-day, cross-machine cache with providers whose caching may be limited to one replica or offered as a paid retention guarantee. It also recounts a posted bill of 89.5 million tokens costing $0.54, mostly from cache hits, versus an estimated $13.42 if those tokens had been misses.

Self-hosting, restrictions, and model trade-offs

The video cautions that self-hosting DeepSeek’s stack is not a simple way to avoid API costs. DeepInfer is described as supporting Nvidia Blackwell and Huawei Ascend 950—not H100s, H200s, or gaming cards—and DeepSeek’s FAQ reportedly calls for a minimum efficient deployment of 128 GPUs: 64 for prefill and 64 for decode. The model weights are about 510 GB, and the cache storage and distributed software add further requirements.

The video identifies three cases where the cost argument may not decide the choice:

• Federal projects: The video says a US Department of Homeland Security order restricts DeepSeek products and third-party products based on them on civilian federal computers. Federal-related projects should review the order.
• Data residency and retention: Users who cannot send data to a Chinese-operated API may consider DeepInfra, which the video describes as offering the model from US-based infrastructure with zero data retention. Cache performance should still be checked against the actual bill.
• Long autonomous coding tasks: For tasks where accuracy matters, the video says Claude Opus 5 scored ahead of V4.1 Flash on the cited Program Bench and Terminal Bench results. A cheaper token may not be cheaper overall if it requires retries or corrections.

Practical takeaway: For long-context, multi-turn, or agent workloads, ask providers how long they retain cache data and whether cache hits survive requests landing on different machines—not just what their per-token price sheet says.

Main speakers and sources

• Main speaker: The video’s narrator, identified with the Kai channel.
• Sources cited: DeepSeek documentation, benchmarks, and technical reports; SemiAnalysis; Eugene Wei; Bloomberg; The Information; the US Department of Homeland Security order; OpenRouter; and provider documentation from Fireworks and DeepInfra.