Video summary

Fable is Mythos, and it is really good.

Main summary

Key takeaways

Technology

Technological concepts / product features / analysis (key points)

Model access & “safeguards”

  • The widely available model is Fable 5, not Mythos 5.
  • Fable 5 is positioned as “still Mythos,” but with front-loaded safeguards that:
    • refuse or block more requests in certain scenarios
    • reduce performance on some benchmarks
  • Despite restrictions, the speaker reports heavy practical usage (sub accounts + inference).

Compute + cost realities (inference and session limits)

  • Reported spending on inference over ~24 hours is substantial: ~$600–$2,000, plus team activity.
  • 5-hour session limits are hit quickly; the speaker describes burning through limits on multiple accounts simultaneously.
  • A key pain point: after hitting limits, switching to usage-based billing led to unexpectedly fast spend (claimed $100 in ~8 minutes).
  • Rough estimate provided:
    • one 5-hour session ≈ 25% of weekly limit
    • after maxing a session, continuation is only “a few times per week” (max described as “four times in a week”)
  • Pricing stated:
    • $10 per 1M input tokens
    • $50 per 1M output tokens
  • Overall implication: Fable/Mythos can burn through limits and cost quickly, and the speaker expects future tiering/subsidization as demand patterns become clearer.

Benchmark reporting methodology and “Mythos 5 vs Fable 5” mapping

  • On some charts, labels appear as “Mythos 5 / Fable 5” because:
    • some benchmark questions get refused by Fable
    • refused questions plummet scores
  • Example impact:
    • Terminal Bench: large drop (e.g., “20 point drop”) attributed to blocked requests under safeguards.

Coding benchmark outcomes

swe-bench pro

  • The speaker frames it as partly flawed/weak now because much of it involves:
    • existing PR text + metadata
    • the model “recreating” details
  • Fable 5 strongly outperforms (contrasted with Opus models).
  • Claim: the model can “recreate those PRs,” implying strong training/data coverage.

Frontier Code (general / harder / diamond tiers)

  • Discussion focuses on the hardest subset (e.g., diamond tier = 50 hardest).
  • Skepticism about Frontier Code scoring behavior:
    • charts allegedly show inconsistent performance across reasoning settings
    • suspected “random generator” pattern
  • A more plausible internal/early dataset (“Deep SWE”) is highlighted:
    • Fable XHigh ≈ GPT 5.5
    • better value (fewer tokens spent) even if price is higher
  • Token efficiency point:
    • it may be a better deal per task because it uses fewer tokens despite being “more expensive.”

Knowledge work + vision / spatial reasoning

  • Vision is described as meaningfully improved versus the “Opus line.”
  • The speaker claims it may lead GPT in vision (and possibly beat Gemini 3.1 Pro, per their summary), with emphasis on spatial reasoning.
  • Weaker performance reported on Skatebench:
    • Google reportedly dominates (e.g., “98% often” cited)
    • Fable improves relative to earlier versions but still lags
    • the speaker says it tries to avoid reasoning on parts of the task.

“Fallback” / refusal routing and safety tradeoffs

  • Safety interventions aren’t only refusals:
    • for sensitive domains, the system may route to another model (e.g., Opus 4.8)
    • or apply efficiency/prompt-steering/PEFT-like changes that reduce effectiveness
  • Privacy/safety measurement section:
    • fallback rate described as low (“fewer than 5% sessions”)
    • but the speaker claims benchmark evidence suggests ~8% task-level fallback, especially on scientific questions
    • the model may score very low on some hard exam-like questions when constrained
  • Concern emphasized:
    • users may pay full price for an effectively “dumber” model without clear disclosure about when downgrades occur.

Artificial Analysis benchmark claim

  • The speaker claims the model is best-in-class / smartest on that benchmark:
    • including a “~5 point lead over 5.5”
  • Still, they stress that fallback/denial behavior affects real implications.

UI + template quality for code-generated interfaces

  • Strong opinion: UI output quality is improved and looks less template-y than earlier Opus experiences.
  • Hypothesis: improved templates or better design “instinct.”

Real-world coding / porting successes (not just benchmarks)

Reported tasks include:

  • Modernizing a ~15,000-line codebase into a working version via iterative debugging using:
    • error screenshots
    • corrections
  • Architecture modernization moves, including examples like:
    • Prisma → (TanStack Start, Convex, Clerk)-style modernization
    • some choices described as “auto-selected” by the model based on prompts/system instructions
  • Workflow debugging/diagnosis tasks, including edge cases involving auth/UI
  • Cases where the model breaks core functionality in more complex UI flows (progress mixed).

Software creation demos

  • The team reportedly produced:
    • a terminal-based 2.5D adventure via prompts (claimed working interactions)
    • a Rust port of T3 code into a terminal UI with clickable thread handling
    • an “Addie made a Minecraft clone” claim: fully generated assets/textures without image generation (inferred programmatic generation)

Fuzzers / test generation

  • The speaker describes prompting the model to write fuzzers and test beds to find issues (e.g., “Lakebed”).
  • They report finding concrete bugs during database/architecture overhauls.
  • Overall argument: the model is more trustworthy/more capable at doing this end-to-end (though not perfect).

Data retention / privacy policy

  • Major operational constraint:
    • Fable 5 requires 30-day retention for all traffic
  • Even with Anthropic setups that claim not to save data, the speaker warns this policy implies data is stored when using Fable 5.
  • Consequence: the speaker warns it may violate some org policies/laws for real-world use cases.

How to use in day-to-day engineering (practical guide statements)

Recommended workflow principles:

  • Treat AI as first-line defense for bug detection and feedback.
  • Test the hardest UI flows.
  • Have AI create screen recordings of PRs for review before humans look.
  • Humans should focus on PR review after AI has analyzed.
  • Use agents/workflows to self-prompt “most of the time”, with caution since agent prompting can be low quality.

Strategic stance:

  • Let AI explore broader options:
    • give vague goals
    • synthesize, test, validate, then summarize results
  • Push harder problems because the ceiling is meaningfully higher.

Economic argument

  • The speaker claims software economics shifted:
    • spend AI budget on many PRs so that some become mergeable
    • avoid paying high costs repeatedly to make a single PR work
  • Additional advice:
    • temporarily “squeeze usage” within current subscriptions due to limited capacity and fast market changes
    • expect pricing competition and improvements.

Reviews / tutorial / guide content explicitly present

  • Benchmark walkthrough + critique, especially:
    • Terminal Bench
    • SWE-bench pro
    • Frontier Code
    • Skatebench
    • Artificial Analysis
    • fallback behavior
  • Usage guide / process recommendations for engineering teams:
    • AI as first-line QA
    • review pipeline changes
    • agent prompting guidance
    • fuzzers/test beds
  • Practical build notes from porting/modernizing real codebases:
    • iterative debugging
    • architecture migration examples
  • Operational cautions:
    • cost traps from usage billing
    • refusal/safety downgrades
    • 30-day data retention requirements

Main speakers / sources

  • Main speaker: the video narrator/reviewer (speaks in first person throughout; no explicit name given in subtitles).
  • Referenced sources/quotes:
    • Walden (cited as a starting point for AI usage in engineering workflows)
    • Simon (another tester mentioned in passing)
  • Bench providers/platforms mentioned:
    • GitHub / GitHub Actions
    • Blacksmith sponsor
    • benchmark suites: swe-bench pro, Terminal Bench, Frontier Codebench, Skatebench, Artificial Analysis / intelligence index

Original video