Video Summary - SpaceXAI engineer Lauren Tan part1

Key technological ideas (AI engineering + agent reliability)

Best use of AI for coding = pair programming, not a tool to blindly replace thinking or writing. The speaker warns against “AI everywhere” productivity pressure that leads to shipping many PRs without understanding.
Core problem: trust in code-writing agents. Agents can “wing it,” hallucinate, and confidently claim the wrong root cause—this destroys engineering trust and prevents scaling to many agents in parallel.
Trust curve concept: Over months, the workflow moves from being in the loop (reviewing/repairing each agent output) to eventually auto-merging PRs once the agent system is verified and reliable.

The most important skill for using coding agents is verification—having agents run and test real code (e.g., execute the app, use CPU traces/heap snapshots, or control simulators) to close the loop.
Verification does not guarantee “good” engineering, but it can ensure correct behavior (or at least catch failures), which is a major prerequisite for trust.

Cursor agent system and internal “agents window” work (code name referenced as Glass / “agents window”).
A key part of verification is building control skills so agents can interact with tools:
- Agents can use Chrome DevTools Protocol (for web/electron debugging) or Apple simulator utilities (for iOS), and then capture traces/events to validate behavior.
Feature map (a unique file/asset described in the “control glass” skill):
- Problem: after the agent could launch/use the app and take traces, it still couldn’t locate UI features reliably and “flailed” trying to find screens/buttons.
- Solution: a “feature map” teaches the agent how to navigate the UI features (including sub-features, entry points, DOM selectors/attributes usable via CDP).
- The speaker mentions a plugin called Pstack / “PAC cursor” (auto-generated subtitle spellings vary), which includes a create verification skill that can generate/update such a feature map by exploring the codebase/UI.

The speaker describes building verification and anti-hallucination skills incrementally:
- Early phase: agents wrote plausible claims but didn’t actually inspect relevant code.
- Response: create skills that force tool-based grounding (search/read code; use sub-agents; verify through execution instead of guessing).
Naming: “Pstack” is said to be a playful reference to “Gary Stack / GStack” (Y Combinator CEO Gary Tan).

Evals are described as unit tests for agent skills.
In Pstack under “Potato mode,” there is an eval playbook:
- Uses a main coordinator agent to create a rubric.
- Spawns many sub-agents and isolates their context (directory per run) while trying to prevent the sub-agents from detecting they’re being evaluated.
- Supports running/evaluating across multiple model choices (model matrix comparisons).
The speaker describes hill-climbing improvements:
- Use eval scoring and judge agents (possibly different models) to reduce bias.
- Iterate until targets like “10/10” are reached.
Maintenance is acknowledged as hard, requiring taste, observation, and “backseat driver” review.

Suggested progression:

Build verification skills locally (so you can observe agent/app interactions and tool calls).
Once trust is established, scale to cloud agents handling tasks like bug reproduction from reports.
Ultimately, reach stages like auto-merged PRs (where the system has enough verification/guardrails).

Warns against jumping too quickly to massive parallelism (e.g., spawning “hundreds/thousands”) due to token cost and expense.

The speaker argues that strong guardrails are especially important for agent-written code:

“Brownfield” apps (already structured) are safer.
“Greenfield/vibe-coded” apps lack constraints and can spiral into messy architecture because agents optimize for shortcuts.

Example architecture for their app (“Grockbot”):

CI is strict and enforces rules like banning React’s useEffect.
Even style/process rules are enforced (e.g., banning code comments as a guard against irrelevant/incorrect commentary by agents).
Uses CI/static analysis to prevent invalid dependency mixing (e.g., separation of electron main vs renderer via dependency graph checks).
Notes that performance regressions are recurring, and the framework encodes electron performance/isolation rules (to avoid blocking the renderer and causing jank).

Personal metrics described:
- After trusting agents more, the speaker claims high PR velocity (hundreds to thousands merged over time).
- With properly set up agents and verification, outputs become reviewable quickly and sometimes auto-land on main.
PR sizing guidance:
- No hard cap, but encourages splitting into multiple atomic PRs for easier reverts and better git-history context.

The speaker argues that depending on the situation, refactoring/restructuring can be valuable:

Large companies can have “human slop” and insufficient code quality despite scale.
Agents can help when infrastructure and guardrails exist.
Strong constraints early can prevent architecture drift when agents contribute heavily.

The speaker can’t claim unlimited-token scenarios apply to everyone (mentions working at an AI lab with more resources).
Frames the decision as ROI and engineering trade-offs:
- Upfront token spend to build constraints/verification may cost money, but reduces long-term human effort and helps teams scale contributions.

Lauren Tan (SpaceX AI engineer; primary speaker)
Q&A interviewer / chat participant (non-identified in subtitles; asks follow-ups)
Mentions of external references:
- Gary Tan (Y Combinator CEO) as the inspiration for naming (“GStack”)