State of Agentic Coding #6 with Armin and Ben

Overview / Themes

• The episode discusses agentic coding/engineering (e.g., “vibe coding/vibe engineering”) and how real-world practices are shifting as agents take on more responsibilities: security testing, CI/tool use, code review, and workflow automation.
• A recurring emphasis is that the industry is moving from “AI as a fun accelerator” to cost-constrained, infrastructure-aware engineering.
• Token/compute economics are forcing companies to rethink how they deploy coding agents.

Key Points: Technology, Product Features, and Analysis

1) AI Engineering Conferences as “Fast Feedback Loops”

• The discussion highlights AI Engineer Europe and AI Engineer Miami as major gatherings.
• Positive evaluation criteria:
  • Talks are streamed and typically appear online relatively quickly.
  • The format/content is consistent enough to deliver value even without attending.
• They also stress the hallway track, noting that many AI tech ideas have short “shelf lives” (months/weeks).

2) Hardware Cost Pressures (Tokens → RAM/SSD/GPU Downstream)

• They revisit earlier compute-cost predictions, arguing that:
  • RAM sizes/densities (e.g., 64/128) are getting more expensive; 256GB is also rising.
  • Hard drives / NVMe SSDs have become unexpectedly expensive, especially when downloading LLMs.
• Technical framing and causal links:
  • Semiconductor manufacturing costs are influenced by upstream supply constraints (e.g., helium, oil/gas, power).
  • Prompt caching and “dumping GPU output to disk” increase the importance of fast local storage, driving higher NVMe demand.
  • As inference traffic becomes more repetitive (longer sessions, repeated requests), caching becomes a more significant architectural concern.

3) Security Vulnerability Discovery Is Escalating via “Harnesses/Agents”

• The premise: newer “security agents” may not be finding more bugs solely because models are smarter.
• Instead, the real change is that people are building better harnesses that orchestrate model-assisted search loops.
• Examples/themes mentioned:
  • Entropic “Myth”: it leaked to the community before broader release; despite model/provider withholding, vulnerabilities still emerged.
  • “Copyfail”: described as Linux root execution / privilege escalation found with AI assistance.
  • Warden: David (ex-Sentry) released a vulnerability-discovery harness that can run as an “agent loop,” using an SDK plus injected skills to find ~hundreds of vulnerabilities.
  • Company reactions:
    • Some vendors choose not to open source, since open sourcing can make security issues easier to find (e.g., Cal.com reportedly closes source).

4) Agentic Coding Economics: Token Spend Drives Standardization (“Clamp Down”)

• Core analysis: companies increasingly refuse “subsidized” token costs and will budget/control usage.
• Evidence of enterprise behavior:
  • Teams standardize on tools/harnesses due to cash efficiency (API pricing vs subscriptions, preventing runaway usage).
  • Token-maxing “got old quickly,” described as an “end of subsidies” phase.
• Product/pricing consequences:
  • Provider pricing shifts toward per-use / usage-based billing.
  • Code review tools and other AI SaaS products (example: Grapple) can show cost spikes under per-use pricing.
  • Once agents are used by agents (not humans), older pricing assumptions break: costs rise and value must be re-justified.

5) “Clamp Down” on Provider Models/Tools (Example: Claude Code Behavior)

• Examples include:
  • Restrictions that remove features on lower-tier plans.
  • Experiments where certain prompt phrases could trigger token overage charges even on “max” plans—suggesting provider-side throttling/discrimination to manage cost.

6) GitHub Is “Up for Grabs,” and GitHub Integration Doesn’t Scale with Agent Traffic

• The episode discusses:
  • GitHub instability/performance issues under agent-driven PR/issue traffic.
  • Mitchell Hashimoto’s view that dissatisfaction with GitHub’s state contributed to moving away from it.
• Infrastructure constraints connect back to cost pressure:
  • Increased storage/disks needs and migration work.
• Alternatives and speculation:
  • Mentions of Tangled and other projects.
  • Speculation about federation and tooling abstraction over time (e.g., generic interfaces using Go dependency URLs and model recognition of GH CLI).

“Pi” and “Leos”: Product Direction and Agent-Harness Design

• Arendelle is described as an umbrella company with two products:
  • Leos: an agent that sits in email for “normies.”
  • Pi: a coding agent—described as an “umbrella” coding agent designed to act as a component for building other agents.
• Background and motivation:
  • Pi emerged from broader community interest in coding agents and “software extending itself.”
  • Pi is described as adaptable to different project contexts (it “behave[s] differently because it adjusts”).
• Acquisition detail:
  • The episode states Arendelle acquired Pi.
• Product philosophy:
  • They want Pi to stay aligned with open-source stewards’ values while still generating revenue.
  • Emphasis on transparency, choice of models, and explicit data-sharing expectations (for Leos/Pi architecture decisions).

Traces / Agent Trace Sharing (Open Models vs Closed Ecosystems)

• Traces (agent run logs) are framed as becoming valuable “training/reward” signals.
  • Coding traces are especially valuable because they can include human input and have mechanically verifiable outcomes (e.g., whether the user committed).
• Debate over trace-sharing incentives:
  • Skepticism that individuals will share traces (a “chicken-and-egg” problem: unclear ROI; privacy/security concerns).
  • Suggestion: awareness must start on the model-lab side; users also need consent and opt-out options.

Cursor / X Acquisition and “Data vs Compute vs Training”

• They discuss SpaceX (X) pursuing/structuring an acquisition of Cursor (framed as an escape strategy/transaction structure).
• Analytical angle:
  • Synthetic-data-only training isn’t a winning strategy currently.
  • Companies with real coding traces/data may outperform those relying on compute alone.
• “Trace value” theme:
  • Coding traces function as actionable reinforcement learning signals.

Side Projects Mentioned (Agent Tooling Primitives)

• Ben
  • Built a game with his kids.
  • Built Pi extensions, including:
    • A kid-explainer extension (“PI extension that helps you explain to a kid what’s going on”).
    • Pi Draw: a terminal/browser drawing interface that sends images back to Pi for interpretation.
• Armen
  • Built “terminal diagramming” tooling:
    • A PI extension enabling terminal-based ASCII/diagram layout (described as a “TL Draw” / “term draw”-style concept).
    • Supports shapes, resize, and grouping, used to communicate desired UI/layout to the agent.
    • Notes suggest agents may understand pictures better than ASCII/braille characters, and that different model versions improve diagram quality.

Speaker / Source Identification (As Mentioned)

• Ben Vinegar (Modem; previously worked with Armen at Sentry; interviewed in conference context)
• Armin / Armen (Arendelle; building Leos and Pi; previously connected to Sentry and coding agent work)
• Other referenced people/sources:
  • Sean Wang / Swyx (conference/podcast organizer; credited with popularizing “AI engineering”)
  • David Kerm(it)/Sentry (mentioned for the Warden vulnerability harness)
  • Mitchell Hashimoto (GitHub creator; discussed leaving/moving away from GitHub)
  • Mario (frequently referenced as a major contributor tied to Pi history and trace-sharing/code agent work)
  • Eric / CodeRabbit (panel interviewer)
  • Jacob Thornton (mentioned in relation to a platform story; likely “pierre.com” / Forge-like platform discussion)
  • Claude / OpenAI / Entropic / GitHub / Microsoft referenced as providers/infrastructure/data sources