How I Shipped 2000 PRs Last Month — Trusting AI Agents｜Grok Bot｜Lauren Tan

Overview

Lauren Tan (from Grok Bot at SpaceX AI) explains how she enabled ~2,000 pull requests to production in a month by building a high-trust environment for AI agents. The approach turns agent-based development into something like a “software factory,” using a “Michelin kitchen” analogy:

• Agents are treated like kitchen staff.
• The team’s setup—equipment, policies, and constraints—determines output quality.

---

Core technological thesis: trust via layered agents

To scale from “babysitting agents” to running many autonomous sub-agents, you must earn trust by making agents reliably correct and able to produce high-quality results.

She frames this as an escalation path (“trust graph”):

• Increasing automation is only safe when verification + guardrails are strong.

---

Key story & motivation (performance + verification)

• Six months before SpaceX AI (after joining Cursor), Cursor’s agent window had performance issues.
• Early work was largely manual, involving:
  • Chrome DevTools
  • performance traces
  • heap snapshots
  • continued PR flow while performance investigations happened in parallel
• Frustrated by the manual “wall of PRs,” she shifted focus to building agent verification skills:
  • letting agents run the app themselves
  • collect traces
  • identify hot spots
  • iteratively improve performance
• The outcome:
  • her productivity increased sharply
  • cumulative infrastructure + skill improvements helped her ship thousands of PRs

---

How to build trust (verification first, then quality)

1) Verification skills (correctness + measurable evidence)

She distinguishes:

• Verification skills (practical):
  • Teach agents to run the application and gather evidence using tooling such as the Chrome DevTools Protocol
  • Examples include:
    • taking performance traces
    • collecting heap snapshots
    • debugging and meeting performance bars
• Formal verification (hard/open):
  • Using tools like Lean or TLA+ to prove invariants
  • Often not realistically feasible in this context

Example: “control glass”

A key verification skill is “control glass”:

• Provide a CLI that lets agents reproducibly run the app and collect traces
• Avoids fragile, ad-hoc scripts per session
• Emphasizes reliability and repeatable evidence gathering

---

2) “Feature map” (materialized memory for UI/navigation understanding)

Auto agents may receive vague or partial UI info (e.g., Slack screenshots with question marks). Without a structured understanding of the app, they can’t infer the intended navigation path.

To solve this, she introduced a feature map stored in the skill/codebase:

• how the application is structured
• how users reach features (keyboard shortcuts, DOM elements, what to click)
• what each feature does

This map is maintained via automation, allowing agents to translate user reports into the correct UI paths.

Combined impact

Agents need both:

• control/verification (can the agent run and measure the system?)
• intent/UI understanding (where should it act?)

That’s why control skills are treated as critical infrastructure.

---

3) Quality skills beyond correctness

Verification ensures features work, but doesn’t guarantee:

• code quality
• performance beyond basic checks

She describes building skills that teach agents to behave like real software engineers via playbooks, including workflows such as:

• debugging
• feature development
• prototyping
• code-writing processes

(An example mentioned is a Pystack plugin.)

---

Agent-friendly codebase architecture (preventing anti-patterns)

She argues that codebases must be designed for agent autonomy, not just human development.

Continuum of increasing trust/enforcement

From weakest to strongest:

1. Codebase as memory Agents extend existing patterns.

2. Static analysis Linters/compiler/CI make mistakes harder or impossible.

3. Guidance layer Rules, “Bug bot,” skills—less enforceable.

4. Style guide / human review Human-only; too slow at high PR volumes.

Her message: don’t rely only on style guides—invest in earlier layers to avoid scaling failure.

---

“Dune” (Grok Bot’s agent-friendly framework)

In the Grok Bot codebase, her team uses Dune, an agent-oriented framework aimed at fixing performance and autonomy issues.

Key ideas:

• Agents love shortcuts.
• Dune makes the easy path for agents also the correct path.
• The framework is intentionally annoying for humans, but ideal for agents—especially when non-engineers (designers, PMs, CEOs) contribute.

Concrete example: strict boundaries to protect UI performance

She describes a boundary rule:

• Code on the main thread must not be imported into the renderer thread (Electron context)
• Because it causes performance regressions

Dune enforces this using an import/independency graph—motivated by observed slowdowns in Cursor’s agent window.

---

Anti-pattern containment: stop “virus-like” propagation

A major warning: when humans patch agent mistakes with workarounds or comments, agents may replicate those behaviors, spreading bad patterns quickly.

Special anti-pattern: agents leaving comments

She initially viewed comments as reasonable (humans do it too), but found agents using comments to justify avoiding real fixes—i.e., band-aid behavior.

In Dune/Grok Bot, they ban comments to prevent propagation.

---

“Gardener” role

She recommends a “gardener” mindset/team function:

• monitor and remove weeds/tech debt early
• enforce a single “paved path”
• when bad patterns appear, write lint rules to “stop the bleeding,” even if you can’t fully refactor immediately

Dune principles align with this:

• remove tech debt
• enforce a paved path
• prevent expansion of bad patterns by turning them into enforceable constraints

---

Outer loop orchestration with Grok Bot

The system is positioned as:

• Grok Bot = outer loop
• It aggregates signals from tools/services, such as:
  • Slack
  • DataDog
  • Sentry
  • PlanetScale

Those signals can trigger automated actions, for example:

• starting cloud agents
• running routines (e.g., subscribing Slack threads to Sentry alerts)
• automatically reproducing bug reports and opening PRs

She claims you can build a “personal mission control kitchen” with limited extra infrastructure by chaining agent workflows through Grok Bot.

Integration with Cursor

• Cursor automations and an SDK can create additional bots that reuse agent infrastructure.

---

Takeaway: order investments to raise agent trust

Her actionable sequence focuses on choosing where to intervene first:

1. Make bad patterns categorically impossible via architecture/codebase/data structures
2. Apply static analysis
3. Add rules / bug bot / skills
4. Improve quality via skills and playbooks

Goal: reach a state where agents can operate “freely” and the team can parallelize confidently while maintaining high code quality.

---

Main speakers/sources

• Lauren Tan — speaker; works on Grok Bot at SpaceX AI (with Cursor context mentioned)