Video summary
Fable is Mythos, and it is really good.
Main summary
Key takeaways
Technological concepts / product features / analysis (key points)
Model access & “safeguards”
- The widely available model is Fable 5, not Mythos 5.
- Fable 5 is positioned as “still Mythos,” but with front-loaded safeguards that:
- refuse or block more requests in certain scenarios
- reduce performance on some benchmarks
- Despite restrictions, the speaker reports heavy practical usage (sub accounts + inference).
Compute + cost realities (inference and session limits)
- Reported spending on inference over ~24 hours is substantial: ~$600–$2,000, plus team activity.
- 5-hour session limits are hit quickly; the speaker describes burning through limits on multiple accounts simultaneously.
- A key pain point: after hitting limits, switching to usage-based billing led to unexpectedly fast spend (claimed $100 in ~8 minutes).
- Rough estimate provided:
- one 5-hour session ≈ 25% of weekly limit
- after maxing a session, continuation is only “a few times per week” (max described as “four times in a week”)
- Pricing stated:
- $10 per 1M input tokens
- $50 per 1M output tokens
- Overall implication: Fable/Mythos can burn through limits and cost quickly, and the speaker expects future tiering/subsidization as demand patterns become clearer.
Benchmark reporting methodology and “Mythos 5 vs Fable 5” mapping
- On some charts, labels appear as “Mythos 5 / Fable 5” because:
- some benchmark questions get refused by Fable
- refused questions plummet scores
- Example impact:
- Terminal Bench: large drop (e.g., “20 point drop”) attributed to blocked requests under safeguards.
Coding benchmark outcomes
swe-bench pro
- The speaker frames it as partly flawed/weak now because much of it involves:
- existing PR text + metadata
- the model “recreating” details
- Fable 5 strongly outperforms (contrasted with Opus models).
- Claim: the model can “recreate those PRs,” implying strong training/data coverage.
Frontier Code (general / harder / diamond tiers)
- Discussion focuses on the hardest subset (e.g., diamond tier = 50 hardest).
- Skepticism about Frontier Code scoring behavior:
- charts allegedly show inconsistent performance across reasoning settings
- suspected “random generator” pattern
- A more plausible internal/early dataset (“Deep SWE”) is highlighted:
- Fable XHigh ≈ GPT 5.5
- better value (fewer tokens spent) even if price is higher
- Token efficiency point:
- it may be a better deal per task because it uses fewer tokens despite being “more expensive.”
Knowledge work + vision / spatial reasoning
- Vision is described as meaningfully improved versus the “Opus line.”
- The speaker claims it may lead GPT in vision (and possibly beat Gemini 3.1 Pro, per their summary), with emphasis on spatial reasoning.
- Weaker performance reported on Skatebench:
- Google reportedly dominates (e.g., “98% often” cited)
- Fable improves relative to earlier versions but still lags
- the speaker says it tries to avoid reasoning on parts of the task.
“Fallback” / refusal routing and safety tradeoffs
- Safety interventions aren’t only refusals:
- for sensitive domains, the system may route to another model (e.g., Opus 4.8)
- or apply efficiency/prompt-steering/PEFT-like changes that reduce effectiveness
- Privacy/safety measurement section:
- fallback rate described as low (“fewer than 5% sessions”)
- but the speaker claims benchmark evidence suggests ~8% task-level fallback, especially on scientific questions
- the model may score very low on some hard exam-like questions when constrained
- Concern emphasized:
- users may pay full price for an effectively “dumber” model without clear disclosure about when downgrades occur.
Artificial Analysis benchmark claim
- The speaker claims the model is best-in-class / smartest on that benchmark:
- including a “~5 point lead over 5.5”
- Still, they stress that fallback/denial behavior affects real implications.
UI + template quality for code-generated interfaces
- Strong opinion: UI output quality is improved and looks less template-y than earlier Opus experiences.
- Hypothesis: improved templates or better design “instinct.”
Real-world coding / porting successes (not just benchmarks)
Reported tasks include:
- Modernizing a ~15,000-line codebase into a working version via iterative debugging using:
- error screenshots
- corrections
- Architecture modernization moves, including examples like:
- Prisma → (TanStack Start, Convex, Clerk)-style modernization
- some choices described as “auto-selected” by the model based on prompts/system instructions
- Workflow debugging/diagnosis tasks, including edge cases involving auth/UI
- Cases where the model breaks core functionality in more complex UI flows (progress mixed).
Software creation demos
- The team reportedly produced:
- a terminal-based 2.5D adventure via prompts (claimed working interactions)
- a Rust port of T3 code into a terminal UI with clickable thread handling
- an “Addie made a Minecraft clone” claim: fully generated assets/textures without image generation (inferred programmatic generation)
Fuzzers / test generation
- The speaker describes prompting the model to write fuzzers and test beds to find issues (e.g., “Lakebed”).
- They report finding concrete bugs during database/architecture overhauls.
- Overall argument: the model is more trustworthy/more capable at doing this end-to-end (though not perfect).
Data retention / privacy policy
- Major operational constraint:
- Fable 5 requires 30-day retention for all traffic
- Even with Anthropic setups that claim not to save data, the speaker warns this policy implies data is stored when using Fable 5.
- Consequence: the speaker warns it may violate some org policies/laws for real-world use cases.
How to use in day-to-day engineering (practical guide statements)
Recommended workflow principles:
- Treat AI as first-line defense for bug detection and feedback.
- Test the hardest UI flows.
- Have AI create screen recordings of PRs for review before humans look.
- Humans should focus on PR review after AI has analyzed.
- Use agents/workflows to self-prompt “most of the time”, with caution since agent prompting can be low quality.
Strategic stance:
- Let AI explore broader options:
- give vague goals
- synthesize, test, validate, then summarize results
- Push harder problems because the ceiling is meaningfully higher.
Economic argument
- The speaker claims software economics shifted:
- spend AI budget on many PRs so that some become mergeable
- avoid paying high costs repeatedly to make a single PR work
- Additional advice:
- temporarily “squeeze usage” within current subscriptions due to limited capacity and fast market changes
- expect pricing competition and improvements.
Reviews / tutorial / guide content explicitly present
- Benchmark walkthrough + critique, especially:
- Terminal Bench
- SWE-bench pro
- Frontier Code
- Skatebench
- Artificial Analysis
- fallback behavior
- Usage guide / process recommendations for engineering teams:
- AI as first-line QA
- review pipeline changes
- agent prompting guidance
- fuzzers/test beds
- Practical build notes from porting/modernizing real codebases:
- iterative debugging
- architecture migration examples
- Operational cautions:
- cost traps from usage billing
- refusal/safety downgrades
- 30-day data retention requirements
Main speakers / sources
- Main speaker: the video narrator/reviewer (speaks in first person throughout; no explicit name given in subtitles).
- Referenced sources/quotes:
- Walden (cited as a starting point for AI usage in engineering workflows)
- Simon (another tester mentioned in passing)
- Bench providers/platforms mentioned:
- GitHub / GitHub Actions
- Blacksmith sponsor
- benchmark suites: swe-bench pro, Terminal Bench, Frontier Codebench, Skatebench, Artificial Analysis / intelligence index