How The Best Software Engineers Are Solving the Code Review Bottleneck

Main idea: scaling code review when AI generates code faster

• As AI speeds up code generation, the bottleneck shifts to the review phase.
• Senior engineers can become overloaded, leading to burnout and cognitive debt—the difficulty of understanding or tracking large/older codebases.
• The framing “Don’t do any code reviews” is treated as a loaded suggestion. The practical alternative is to reduce how much human review is needed by:
  • moving checks earlier, and
  • automating guardrails.

How top teams respond (policies + automation)

• AI-generated code increases review pressure, so companies apply different scrutiny levels depending on how critical the code/system is.
  • Example: Amazon reportedly uses policies that require senior engineer review for certain critical parts, after incidents involving outages or revenue loss attributed to AI-generated code.
• Another approach is automation at the workflow layer (horizontal scaling):
  • e.g., tools that scale PR/review mechanics.
  • However, quality impact is not always clearly communicated.
• A deeper approach is vertical scaling:
  • building specialized tooling/environments for coding agents (custom harnesses, refined workflows, project-specific delivery constraints).

Spec-driven / TDD + guardrails to reduce human-in-the-loop review

• The discussion emphasizes spec-driven development and tests as feedback.
• Early attempts using “specs” alone were often insufficient, because the model may deviate from intent unless specs/prompting are tightly defined.
• Better results came from combining:
  • behavioral specifications
  • automated tests (TDD-like) as the primary feedback loop
  • guardrails that constrain what the agent is allowed to produce
• Core claim: behavioral tests + fine-grained specifications create a loop where the system can validate whether generated code matches desired behavior.
• Key reliability insight: generating small, focused tests is more reliable for LMs than generating large, monolithic implementations (LMs are less likely to fail when the unit is small).

Automated feedback cycle (“environment engineering”)

• Instead of humans reviewing after the fact, the environment provides machine-executable feedback during coding.
• Examples of feedback mechanisms:
  • formatters
  • security scanners (e.g., SonarQube)
  • other static checks
• Feedback should be as close as possible to code generation:
  • ideally on the developer’s laptop, not only after PR/merge.
• Harnesses can trigger hooks/events (e.g., a “stop hook”) that:
  • run tests/guardrails via shell scripts
  • produce machine-usable feedback
  • feed that feedback back to the LM so it iterates
• The speakers reference looping behaviors such as:
  • Robo/REPL-style loops
  • goal/continuous run commands in tools like Codex/Claude Code (e.g., “keep running until fixed”)

Guardrails examples (what to enforce)

• Semantic grep (highlighted as a favorite):
  • enforces regex/pattern-based rules for code constructs
  • example: “No default values in method parameters” (Python), to avoid future review/debug frustration
  • used to “shape the environment” so the agent follows team/project style and policies
• Security controls and static analysis
• Architectural constraints via fast architecture unit tests:
  • enforce module boundaries (e.g., “UI must not access DB directly; go through business logic layer”)
  • catch coupling/entanglement patterns that humans might not design that way, but AI might introduce

Harness vs model: harness matters a lot

• The discussion directly answers: “Does it matter which harness I use?”
• Yes—harness matters more than the model in their experience.
• A harness provides capabilities such as:
  • prompting strategy
  • memory/context management
  • tool-calling and execution
  • integration with test/guardrail systems
• They describe an experiment where the same model behaved differently depending on the harness, especially for spec/TDD-style workflows.
• Since tools/models evolve quickly, they argue against rigid “only use X harness” rules and recommend continuous experimentation.

Review strategy evolves: architecture + up-front understanding

• Engineers still need strong architecture understanding—how components interact.
• AI can automate implementation, but teams must invest more upfront in:
  • defining what to build
  • sketching system architecture
  • encoding architectural constraints as guardrails
• This shifts how junior engineers learn:
  • from “just type” toward system thinking and specification skills.

Burnout and task switching mitigation

• Burnout causes mentioned include:
  • constant stimuli
  • context switching
  • LM latency
• Mitigation ideas:
  • treat “agent-environment work” as a separate project
  • interleave tasks while waiting
  • use tools (e.g., Cloud Code) that auto-summarize session state if you pause too long

Practical “starting steps” for developers/teams

1. Start with static checks:
   • formatter
   • linter
   • add semantic grep rules for team preferences and anti-patterns
2. Add guardrails incrementally:
   • run locally first
   • measure/improve based on outcomes (how often/how much the agent needs correction)
3. If adopting broadly and resistance appears:
   • hold a team discussion
4. Use logs to improve guardrails:
   • analyze session logs for recurring model misunderstandings
   • convert those patterns into new static checks
5. If limited to one harness:
   • choose harness strengths by use case (e.g., docs generation vs debugging)
   • don’t assume one tool stays optimal indefinitely

---

Main speakers / sources (as mentioned)

• Florian Buetow — AI engineer at Xebia
• Another main interviewer/host: the podcast host (name not provided in the subtitles)