"Software engineering is not about writing code" — Benoit Schillings, Google DeepMind VP of Research

Main ideas / lessons

• Software engineering is shifting away from “writing code” toward designing, verifying, and managing complexity.

  • The speaker argues that automated/AI-assisted code generation is already progressing to the point where the act of producing code is no longer the primary bottleneck.
  • Future progress is about ensuring code is correct, secure, modular, maintainable, and architecturally sound.

• Historical perspective: software bottlenecks changed over eras

  • Machine-limited era (early days / assembly): The hard part was extracting performance from hardware; accuracy mattered deeply.
  • Cloud & modularity era: Performance became less critical; modular design and building long-term maintainable systems became central.
  • Human cognitive limits shaped software processes: Humans can only juggle roughly 7–9 tokens of context at a time (as stated), which historically constrained how software was broken into manageable parts.
  • Now an “AI frontier” era: Writing code is increasingly easy; the bottlenecks move to:
    • specifying what the code should do (problem definition),
    • ensuring correctness,
    • architecture/implications beyond the immediate snippet.

• What AI progress has already made clear

  • The speaker claims superhuman syntax generation is largely “solved” relative to past difficulty (e.g., models writing functions humans would improve).
  • Remaining hard problems include:
    • multi-step reasoning in large codebases
    • managing extreme complexity (e.g., very large real-world repositories)

• Why code was a unique early ML success case

  • Code is data-rich (e.g., public repositories like GitHub provide training signal).
  • Code is also verifiable:
    • compile/run,
    • unit tests,
    • deterministic checks provide feedback on correctness.

• Why code-learning is now facing a data ceiling

  • The speaker says a large fraction of new code is increasingly machine-generated, reducing the “human-authored” signal.

• Self-play is positioned as the next leap

  • Inspired by DeepMind’s AlphaZero idea:
    • models create their own challenges,
    • judge validity of results,
    • and can (to some extent) evaluate architecture.
  • The claim: massive compute and self-play time can enable “superhuman coding” progress even without endless human-written training data.

• Economic and cultural shift in software

  • The cost assumption “writing code is hard/expensive” is breaking down.
  • If code generation becomes cheap/free:
    • the volume of code will explode,
    • “design and adequacy” become crucial to keep systems reliable at scale.

• New security and process requirements (“guard rails”)

  • AI-generated code can introduce vulnerabilities.
  • The speaker references a vision where automated analysis (e.g., tools/models like “Mythos” in the talk) keeps discovering issues, with fixes ongoing.
  • Proposed emphasis:
    • security becomes as important as code writing
    • move from “detect vulnerability + patch” toward teaching models to write correct code from the start (hard because correctness is context-dependent)

• Inductive architecture & planning as human strengths to teach to models

  • The speaker distinguishes:
    • inductive thinking: understanding broader system context, detecting patterns, deciding what to do.
    • deductive thinking / transfer & decomposition: applying knowledge across contexts and planning correct decompositions.
  • The talk argues models need training for:
    • correct planning
    • right decomposition of large problems into implementable parts

• Evaluation must move beyond “does it run?”

  • The speaker criticizes benchmarks that verify only runtime behavior/output (e.g., “threebench”).
  • Calls for more open-ended evaluation, where models must innovate algorithmically.
  • Example: text compression benchmark
    • ask for a lossless compressor producing minimal size:
      • objective = compressed size + source size,
      • scaled with large inputs,
      • encourages discovering new algorithms rather than only emitting syntactically correct code.

• Reasoning style changes: from token-by-token to richer representations

  • The speaker argues code writing can be like visual reasoning (e.g., block diagrams, data flow).
  • They claim Gemini was designed as a multimodal model, not just text:
    • enabling thinking using spatial/dynamic representations,
    • and expecting this to become necessary for complex problem solving.

• Possible future: new “languages” or stronger correctness guarantees

  • Existing languages (e.g., Python) work well for humans but not necessarily for safe/reliable code.
  • Suggests exploring:
    • strongly typed languages
    • inspiration from Lean and proof-oriented approaches
    • shifting the correctness burden using more formal methods (even if not human-readable)

• Beyond software: scientific discovery via rapid coding experimentation

  • Faster coding experimentation can accelerate other domains (science becomes easier to iterate on).
  • Exciting fronts mentioned:
    • chemistry: increasing molecule size (beyond what humans can model directly) to reveal emergent behavior
    • biology: models uncover relationships not easily found by humans; framed as nature’s engineering with poor documentation
    • “the gold we cannot see”: humans may be biased toward perceived solutions; ML can reveal breakthroughs outside human intuition (examples omitted)

---

Methodologies / instructions presented (detailed)

1) Shift software engineering focus away from token emission

• Treat “writing code” as mostly solved (syntax generation).
• Reallocate effort to:
  • problem specification (“what we need”),
  • verifying the result is correct,
  • architectural choices and long-term maintainability,
  • reasoning about implications beyond the snippet.

2) Use self-play to generate training challenges without relying on infinite human code

• Let code models:
  • generate candidate solutions,
  • create or propose their own tasks/challenges,
  • check validity of answers,
  • increasingly assess architecture.
• Increase progress by scaling:
  • compute,
  • self-play time.

3) Implement “active guard rails” for security and reliability in an AI-codegen world

• Assume vulnerability discovery will be an ongoing loop.
• Use two complementary strategies:
  • Detect + patch vulnerabilities found by models/tools.
  • Prevent vulnerabilities by training models to generate correct-by-construction code.
• Shift security emphasis to match the importance of code generation.

4) Redesign evaluation benchmarks

• Move beyond “runs and produces right output.”
• Include open-ended tasks requiring novel algorithmic work.
• Example benchmark design:
  • take a ~10MB input (e.g., code as compressible text),
  • ask for the best lossless compressor,
  • define objective as compressed file size + source code size,
  • reward improvements that force new algorithms.

5) Train models for complex decomposition/planning

• Teach models to:
  • plan correctly before generating code,
  • decompose complex problems into a structure that yields correctness/clarity,
  • transfer/compose knowledge across contexts (addressing weak deductive transfer).

6) Develop richer representations and possibly new coding “languages”

• Prefer multimodal reasoning (e.g., spatial/dynamic representations) over pure token streams for complex tasks.
• Consider more correct programming systems:
  • strongly typed designs,
  • proof-oriented approaches (inspired by Lean),
  • languages not necessarily intended for human readability, but aimed at correctness guarantees.

---

Speakers / sources featured

• Benois (Benoit) Schillings / Benois Schillings — Vice President of Research, Google DeepMind (main speaker)
• Jeff Dean — referenced as a key architect associated with Google architecture (source by mention)
• AlphaZero — referenced as an example of self-play enabling superhuman performance without human knowledge
• Mythos — referenced as an example of code/vulnerability analysis detecting vulnerabilities
• Gemini (Google Gemini) — referenced as a multimodal model and as an AI coding agent concept
• Threebench — referenced as a benchmark that focuses on “runs and produces correct output”
• Lean — referenced as inspiration for proof-oriented correctness approaches
• GitHub — referenced as a source of code data (scraping/mining context)