Lauren Tan - SpaceXAI engineer

Core thesis: AI for software engineering should be “pair programming,” not code replacement

• The speaker argues that the best use of AI is as a pair programming partner—augmenting engineering work—rather than a tool that replaces thinking or offloads judgment.
• There’s a risk of “AI slop”: optimizing for raw productivity metrics (e.g., shipping large volumes of PRs without understanding) can backfire.
• Even if AI speeds up code generation, the speaker reports spending more time verifying and reviewing what the AI produced, not less.

---

Main analytical theme: trust is the bottleneck for agent-based coding

• The key question is: How do you trust code-writing agents?
• Untrusted agents can:
  • hallucinate or guess
  • “confidently” find the wrong issue
• When trust drops, teams fall into micromanagement:
  • staying constantly in the loop
  • watching outputs closely
  • avoiding scaling automation
• Over time, by improving verification, the speaker describes moving up a “trust curve,” enabling higher automation and parallelism.

---

Product/feature focus: verification-driven agents and “control glass”

1) Verification skill (run code / traces / simulator)

A key enabling capability is verification, meaning the agent can actually run the app and observe real behavior:

• execute code
• capture CPU traces / heap snapshots
• run iOS simulators (or equivalent platform tools)

Verification doesn’t automatically guarantee good code quality, but it supports correctness by closing the feedback loop.

2) “Control Glass” / “agents window” (agents that can operate UIs)

The speaker references a Cursor internal code name (“Glass”) for an agents window—an agent-controlled UI workflow.

Verification is implemented via tool protocols:

• for web/electron: Chrome DevTools Protocol
• for Apple/iOS: simulator utilities and programmatic tracing

Without verification, performance debugging with agents becomes extremely slow because humans become the bottleneck (e.g., collecting screenshots, copying console errors, rerunning cycles manually).

3) “Feature map” (UI navigation knowledge for agents)

A further required component is a feature map.

• Agents could run traces, but often didn’t know how to navigate the UI to reach the relevant feature (e.g., laggy sidebar, broken PR tab).
• The feature map teaches agents how to locate and interact with UI elements, including:
  • routes to major features and sub-features
  • keyboard shortcuts
  • relevant DOM elements/attributes (via CDP selectors)
  • practical navigation context that turns vague user reports into actionable steps

A plugin (“Pstack”) includes setup and a create verification skill feature that helps build/maintain this feature map by exploring the codebase.

---

“Pstack” and how skills are incrementally built

The speaker describes building “Pstack” incrementally by:

• observing common agent failure modes
• turning recurring issues into explicit skills (e.g., preventing hallucinations by forcing the agent to “search/read code,” rather than guess)

Conceptually, skills are markdown-based but include instructions that can shift agent reasoning behavior (analogized to a “latent space” effect).

---

How to maintain and evaluate skills (evals as unit tests for agents)

Eval / eval playbook

• Evals are described as unit tests for agent skills.
• Cursor includes an “eval playbook” (“under potato mode”) featuring:
  • spawning sub-agents
  • a coordinator agent that creates a rubric
  • sub-agents running in isolated directories
  • precautions so sub-agents don’t detect they’re being evaluated (since that could change behavior)
• Cursor supports evaluating across a model matrix (multiple models).

Hill-climbing / looping until scores reach targets

• Evals return scores.
• The system can hill climb by iterating until targets (e.g., 10/10) are reached.
• The speaker describes doing similar iterative improvements for verification and control skills.

---

Scaling strategy: start local → trust → then move to cloud agents

Recommended workflow:

1. start locally to build and observe verification behavior
2. once trusted locally, scale to cloud agents
3. cloud agents can handle signals like bug reports—reproduce issues and produce PRs
4. later, enable higher automation (e.g., automerging PRs after verification)

The speaker cautions against jumping immediately to massive cloud agent counts because:

• it’s expensive (tokens/cost)
• without sufficient trust/verification, it’s wasteful

---

Automation example: cloud “Benny” agent reproduces bugs

“Benny” is described as a cloud agent that:

• takes incoming bug reports
• reproduces them in an isolated environment (using Cursor plus control/verification skills)
• sometimes confirms a bug is already fixed on main

Benefit: it saves human time by quickly determining whether fixes already exist.

---

Software architecture and constraints to help agents write safely

The speaker argues strong constraints reduce trust friction and improve agent code quality.

They discuss multiple layers of enforcement:

• architecture conventions (e.g., “Dune” for Grockbot)
• static analysis, CI failures, and import dependency rules
• lints / bug-finding tools (“bugbot”) as guardrails

Examples of CI hard rules:

• banning React useEffect
• banning code comments (agents often generate irrelevant/unreliable comments)

Core principle: don’t rely only on soft rules or human review—convert recurring failure patterns into hard CI constraints.

---

“Refactoring vs rewriting”: where agents make rewriting feasible (especially for greenfield)

The speaker’s position:

• rewrites may be justified depending on context
• brownfield apps might already be solid if well-structured
• greenfield/vibe-coded apps are the biggest risk due to missing guardrails

Their reasoning:

• with an unconstrained codebase, agents optimize for shortcuts
• the system can spiral into architectural chaos

They reference “Grockbot,” initially “vibe coded,” then refactored into a stricter architecture with CI constraints. After that refactor, the speaker claims they often inspect code less because guardrails + CI + verification make the process safer.

---

PR velocity and review automation outcomes (with skepticism acknowledged)

The speaker claims major productivity growth:

• agents auto-emit PRs (examples include “20 PRs landed”)
• rapid reviews landing on main
• sustained high PR velocity over months (thousands of landed PRs discussed)

Acknowledged skepticism:

• people may question how much of the code is genuinely good
• the speaker argues verification + constraints make enough of it trustworthy to scale

---

Product expansion: Grockbot as an agent interface for non-technical roles

• Grockbot is positioned as a more accessible, chat-like UI for agents (vs dev-only IDE/CLI workflows).
• It’s described as useful for:
  • product managers (summarizing work and requesting bug fixes)
  • designers/PMs shipping features
  • non-engineers contributing via agent orchestration
• “Dune” constraints are presented as what allow non-experts to contribute safely.

---

Main speakers / sources

• Lauren Tan (primary speaker; described as a “SpaceXAI engineer”)
• Cursor (referenced throughout: agents window, verification skills, eval playbook, cloud agents)
• Pstack (speaker-built plugin/tool)
• Grockbot (Cursor/SpaceXAI application referenced)
• Benny (named cloud agent example)
• Pretext (mentioned as powering virtualization in Grockbot/Cursor)