Elon promised this one would be good...

Summary of Technological/Product Analysis in the Subtitles

• “Grok 4.7” is framed as a major upgrade in coding/knowledge-work, heavily hyped by Elon Musk, including claims that it exceeds all current models and that SpaceX’s training corpus makes it especially strong at real-world coding.
• The reviewer disputes this framing, arguing that benchmarks don’t reflect real-world performance and that the release includes misleading or low-signal benchmark figures.

---

Review: Grok 4.7 Capabilities vs. Grok 4.6

• The speaker describes Grok 4.7 as one of their favorite models of the year and says it can outperform others in real work.
• However, they claim benchmark results are contradictory.

Serve Speed & Pricing Parity Claims

• The official story (and Elon’s messaging in the subtitles) is that Grok 4.7 has the same price and speed as Grok 4.6.
• The reviewer challenges this with:
  • Their own cost measurements (based on API costs and observed usage).
  • Concerns that artificial benchmark cost analyses show Grok 4.7 as more expensive.

---

Key Contradiction: Token Efficiency / Cost

What the Official Claim Says

• Grok 4.7 would be:
  • slightly slower than Grok 4.6, but
  • more token-efficient than Grok 4.6.

What the Reviewer Claims

• Benchmark/analysis numbers imply much higher tokens per task, such as:
  • Grok 4.6: ~38k tokens/task
  • Grok 4.7: ~81k tokens/task
• In at least one cost framing example (artificial analysis context):
  • Grok 4.7 costs ~$3–$4 per task, while some competitors cost less.

Response from Michael Truel (Cursor CEO)

• Michael Truel’s response (as presented in the subtitles) is that in production:
  • Grok 4.7 uses only ~5% more tokens for median requests
  • and 20–30% more for P99 requests.

Reviewer’s Rebuttal: “Tokens per request” Can Mislead

• The reviewer argues the comparison may be misleading because token usage depends on agentic behavior:
  • Token usage is tied to “requests” and tool-calling patterns.
  • Even small per-request token differences may not reflect overall prompt cost if the agent triggers more tool calls / more internal requests.

---

Token Usage Can Depend on Agent/Tool Behavior

The subtitles emphasize an important technical point for agentic workloads:

• A single user prompt may lead to many tool calls.
• Each tool call corresponds to additional internal “requests.”
• As a result, overall cost can rise sharply even if “tokens per request” look similar.

---

Benchmark Ecosystem and “Which Benchmarks Matter”

The reviewer critiques both benchmark selection and benchmark interpretation:

• Cursor Bench
  • Previously included training-data concerns for Grok.
  • The speaker says these concerns were “repaired,” making the scores more meaningful.
• Coding/math/engineering-style benchmarks
  • Mixed results: Grok 4.7 does well on some, not on others.
• Safety/cybersecurity/legal/medical reasoning benches
  • Called “weird,” possibly implying cherry-picking or mismatched relevance.
• The reviewer also points out benchmark noise, including tests where scores “spike” depending on reasoning level.

---

Artificial Analysis “Agent Index” Concerns

• The reviewer discusses Artificial Analysis aggregation and an “agent index” style scoring approach.
• They claim:
  • Grok 4.7 scores below expectations relative to some competitors.
  • Grok 4.7 is behind Astra/Fable in overall coding-agent capability, even if it sometimes ranks well in specific benchmarks.
• They stress that even small point differences on such indexes can translate into large practical reliability gaps—e.g., how often the model makes “stupid” mistakes over a time window.

---

Pricing Plan / Product Limits Critique

• The speaker cites usage-limit pain:
  • Grok 4.7 is described as burning through higher-cost quotas quickly.
  • They compare the “feel” of Grok limits versus other labs’ plans (including mentions of OpenAI/Anthropic-style token economics in passing).
• Overall verdict in the subtitles:
  • Subscription economics and token pricing don’t justify the move, especially compared to competitors.

---

“Grokbot” and Agent Tooling

• The subtitles mention that Grok 4.7 is “trained to natively understand the Grokbot harness.”
• Grokbot usage is described as popular among users.
• There’s no deep evaluation beyond that implied integration advantage.

---

Front-End / UX Regression Claim

• The reviewer makes a strong claim that Grok 4.7’s front-end design capability is notably worse than other models.
• They name Claude/Astra as better at UI/design outcomes.
• The speaker implies Grok 4.7 may show presentation-quality regressions, including “worst front-end designs.”

---

Training / Rationale Behind Delays and Behavior Changes

• The speaker claims the release was delayed and links it to RL/behavior tuning:
  • Possibly earlier penalization of response length caused premature stopping on difficult tasks.
  • Fix involved RL to allow longer reasoning/output and better verification.
• They speculate this RL change may explain token-efficiency loss and increased cost.

---

Practical “Real World” Demo: Fish Slop (Game Generation)

The reviewer runs a game-generation task (“Fish Slop”) using Grok 4.7 and reports:

• Worse results than Astra with the same prompt/environment.
• Examples of failure modes:
  • wrong animation direction
  • incorrect mouse tracking/cursor behavior
  • poor “submarine” visuals
• They use this as evidence Grok 4.7 may be less reliable in certain non-coding creative/agentic tasks.

---

New Custom Benchmark: T3 Code Improvement Suggestions

The speaker describes a benchmark where:

• multiple models review a codebase
• suggest improvements
• then a judge evaluates the results

Reported outcomes:

• Astra and Grok 4.7 are described as identifying novel, viable improvement areas.
• Grok 4.7 performs well on:
  • suggestion viability/quality
• But cost is high:
  • best-case cost: comparable to Astra/Fable
  • worst-case cost: can be ~2x, due to excessive verification/looping
• Conclusion:
  • Grok 4.7’s capability gains don’t match its cost increase.

---

Overall Conclusion (As Stated in the Subtitles)

• Final stance: Grok 4.7 is “cool and fun” and can be good for users already using Cursor.
• But it’s hard to recommend broadly, primarily due to:
  • higher token/cost
  • agentic inefficiency
  • regressions/UX/front-end issues
  • limited economic justification versus competitors
• Framing from the subtitles:
  • Grok 4.7 is seen as a stepping stone model,
  • with hope that the next release improves efficiency/pricing to reach “frontier” performance.

---

Main Speakers / Sources Referenced

• Video narrator/reviewer: “nerds” (primary speaker; also references their co-host)
• Michael Truel — CEO/founder of Cursor; described as defending Grok 4.7 token-efficiency claims
• Elon Musk — source of hype/claims about Grok 4.7
• Ben Davis (“Nerdsip”) — co-host; described as defending Grok, with emphasis on pricing/limits concerns
• Artificial Analysis — third-party benchmark/evaluation source
• Blacksmith — sponsor mentioned for accelerating CI (GitHub runner replacement) and lowering CI cost/time