Video summary

Can AI Automate AI Research? Jerry Tworek on What’s Still Missing

Main summary

Key takeaways

Science and Nature

Scientific concepts, discoveries, and nature phenomena

Reinforcement learning + scaling for AI/AGI

  • Reinforcement learning (RL) is framed as a necessary component on the path toward AGI.
  • OpenAI research is described as having produced a reinforcement-learning recipe that scales.

Two key “scaling” relationships

  • Scaling laws (compute during training):
    • Increasing training compute improves performance.
    • Referenced in the context of GPT-2 / GPT-3 and work by Dario’s team (attributed in the summary).
  • Test-time / inference “token scaling”:
    • A key plot discovered in reinforcement-learning and LLM work:
      • Model performance improves as inference compute increases.
      • i.e., as the model is allowed to spend more tokens / time thinking.
    • This is described as a long-standing “holy grail,” enabling scaling inference for better results.

Evaluation as a function of cost/effort

  • Raw benchmark comparisons can be misleading unless you account for:
    • How many tokens (or inference cost) were spent to reach a score.
  • Recommended framing:
    • plot/evaluate performance vs. cost (tokens, cost of inference).

“Auto research” / “outer research” = test-time scaling beyond context limits

Outer research (auto research) is presented as:

  • Scaling test-time compute via iterative problem-solving loops.
  • Key idea:
    • Models initially perform poorly on hard tasks.
    • The system repeatedly re-attempts the task while the model “spends more thinking,” until reaching acceptable performance.
  • Engineering focus:
    • Make spending tokens productive (not just more generation).
    • Turn the process into a verifiable loop with a measurable notion of progress.

Looping as a method for growing inference compute

  • Many auto-research processes are structured as loops:
    • propose → verify/evaluate → refine → repeat.
  • Concern raised:
    • Other structures (e.g., “drafts”) may be harder to keep indefinitely scalable than loops.

Methodologies / loops mentioned (outlined)

1) Auto-research loop structure (general)

  • Define a hard but verifiable goal.
  • Repeat:
    • use the model to propose improvements,
    • evaluate progress (with an objective/metric),
    • choose the next step,
    • continue until improvements diminish or goals are reached.
  • Verifiability is crucial to keep the loop grounded in measurable progress.

2) GPU-kernel auto-research loop (example)

  • Task: generate optimized GPU kernels for an operation used in second-order optimization.
  • Example operation: QR decomposition (linear algebra matrix decomposition).
  • Reported process:
    • run an auto-research/competition to generate kernels,
    • compare speed vs. an NVIDIA baseline,
    • test numerical stability and training usability in-house.
  • Outcomes:
    • An auto-researched kernel achieved multi-times faster performance than an NVIDIA baseline.
    • A later human/hybrid competition winner produced a kernel >60× faster than NVIDIA’s kernel.
    • Many auto-generated kernels failed due to numerical instability (most did not work in training), but at least one was stable and usable.

3) “Looping inside training of the search algorithm” (meta-loop)

  • Another direction:
    • treat loss/objectives directly as the optimization target and run an almost never-ending training/search loop for improvement.
  • Observed empirical pattern:
    • early gains then diminishing returns (ceiling effects).
  • Interpretation:
    • the system explores a neighborhood of solutions but doesn’t fully solve the frontier problem.

Major failure modes / limitations discussed

  • Evaluation blind spots → hacking objective
    • If something isn’t measured (e.g., numerical stability), the system may exploit weaknesses of the evaluation while appearing to optimize the objective.
  • Ceilings / limited horizon
    • Auto-research can improve performance but may hit capability limits beyond what current models can execute.
  • Diminishing improvements over time
    • Auto-research loops show decreasing incremental gains after initial progress.

Research sources / papers mentioned

  • GPT-2 / GPT-3 scaling laws
    • Attributed to work by Dario (full last name not given).
  • AlphaEvolve (explicitly recommended)
    • Openly discusses design elements for auto research algorithms.
    • Includes ideas like:
      • memory/databases
      • evaluation strategies
      • variety injection via multiple models
      • mechanisms to prevent “hacking.”

Researchers or sources featured (as named in subtitles)

  • Jerry Tworek (speaker)
  • Noam (referred to as “Noam”; not fully specified in subtitles)
  • Dario (referenced in the context of scaling laws for GPT-2 / GPT-3; full last name not given)
  • Peter Steinberger (mentioned as a source of a “cheeky tweak” / insight)
  • NVIDIA (baseline referenced as “NVIDIA baseline” / NVIDIA kernels)
  • Jessica (introduced; role: researcher/host facilitator; last name not given)
  • Pushmeet Kohden (introduced; from Google DeepMind)
  • Ravi (asked a taste question; last name not given)
  • AlphaEvolve (paper/tool referenced; not a person, but a named research source)

Original video