Google DeepMind Pre-Training Lead: How To Land a Job at a Frontier Lab | Vlad Feinberg

Main ideas, concepts, and lessons

• Frontier labs value research ability that is grounded in strong engineering

  • Pre-training/research work at “frontier labs” (e.g., DeepMind) is tightly connected to both:
    • research
    • product deployment
  • Success requires being effective across a spectrum:
    • research vs. applied engineering
    • software engineering vs. AI research
  • Key shift: people must be able to handle uncertainty and stochastic outcomes typical of research—not just deterministic engineering progress.

• A standout in-demand skill: kernel development + low-level engineering

  • Vlad highlights kernel development and low-level runtime acceleration as a high-demand specialization across frontier labs.
  • Why it matters: when changing LLM architectures or serving strategies (e.g., KV caching), teams need to implement techniques efficiently at scale with:
    • high throughput
    • low latency
  • The output of this work must behave like software artifacts in classical backend systems—just at LLM scale.

• Applied “product” work still requires deep research

  • Example: making LLM outputs factual, grounded in sources, and filtering out low-quality responses (e.g., “sarcastic or a joke” content).
  • Even “applied” teams do substantive research (e.g., quality evaluation and grounding).

• Infrastructure investment drives research breakthroughs (example: distillation)

  • Vlad’s team focuses on pre-training-related research such as distillation.
  • Distillation transfers “teacher” knowledge (statistics derived from massive models) into “student” models.
  • Because computation/token scale implies enormous cost (“millions of dollars” scale), efficiency at every step is crucial—requiring strong infrastructure and distributed-systems engineering.
  • He describes iterative generations of distillation infrastructure (3–4 generations), each time redesigning abstractions and storage/distributed handling to enable new scaling experiments faster.
  • Outcome: infrastructure rewrites translate into new understanding (e.g., distillation scaling laws) and improved models.

• How to think about “software engineering vs research”

  • Software engineering
    • milestone dependency graph (DAG) is mostly deterministic
    • you can make monotone progress (build service → build next layer → implement storage, etc.)
  • Research
    • graph is stochastic
    • some ideas may fail; intermediate nodes/milestones may be hidden or unknown
    • requires “research taste”: intuition about which path is likely to work and what it will cost in time/risk
  • Vlad frames research as a Markov Decision Process (MDP)
    • estimate success probabilities and time investment for each “research step”
    • optimal strategies differ from deterministic planning

• What backend engineers may struggle with when joining research teams

  • Vlad gives concrete examples of likely gaps:
    • Context for the research landscape: understanding prior work and reading the right papers
    • Efficient traversal of citation trees (identify high-value papers; decide whether to read without fully reading)
    • Prerequisites: strong ML/CS background and enough math to understand methods
    • Without methodology depth, it’s hard to meaningfully improve existing approaches

• LLM pre-training is driven by scaling laws and “recipes”

  • Scaling laws predict test loss/generalization as FLOPs increase.
  • Difference vs classical vision training:
    • In vision, you can iterate across datasets and reliably validate.
    • In LLMs, each pre-training run consumes unprecedented FLOPs—so you can’t “train everything” and then iterate later.
  • A recipe is described as:
    • a function mapping target training compute (FLOPs) → a training routine
  • If paired with a prediction rule, you can choose better recipes based on predicted outcomes.

• Additional frontier-relevant areas beyond kernels

  • Programming language / domain-specific abstractions for kernels
    • example: “Thunderkittens”, aiming to enable kernel authoring with higher-level abstractions (e.g., fewer functions rather than large C++ blobs)
  • Reinforcement learning literature
    • Vlad notes post-RHF, DRL algorithms (e.g., PO) increasingly show up in production
    • but you still need strong RL foundations before advanced methods
  • Distributed systems + optimization for multi-GPU training
    • topics: asynchrony, gradient staleness, pipelining effects on convergence/quality
    • even if not LLM-specific, these translate to training systems at scale

• Why research demand will grow

  • Vlad argues research skill will become increasingly important; uncertainty and stochastic planning matter more over time.
  • Building “systems around LLMs” to make them useful in practice is also a transferable skill across roles.

• AI won’t replace all roles: humans retain accountability

  • He rejects doomer/fear-mongering narratives (FUD).
  • Humans remain responsible for key resource-allocation and accountability decisions.
  • Example: even if LLMs help with legal precedent, humans can’t be removed from roles that require accountability (e.g., they can be held liable and can’t be “disbarred” out of existence).

---

Methodologies / instruction-like content (detailed)

A) How to prepare for a job at a frontier lab (as conveyed by Vlad)

• Build the right “fitness” for frontier research/pre-training

  • Develop mathematical maturity to read/understand top ML research papers (especially scaling-law-era papers as “table stakes”).
  • Be able to:
    • deeply understand a paper at that level
    • take a research idea from a paper and implement it yourself
    • iterate and adapt the idea into the frontier lab’s domain

• Demonstrate specific technical capabilities (explicitly mentioned exercises)

  • Do handwritten “scaling book” exercises (details not fully listed, but described as part of the application).
  • Also do a transformer exercise as listed in Vlad’s post.
  • Prepare a video showing yourself doing these exercises.

• Show engineering “in the weeds” ability

  • Willingness to implement a real transformer and do the practical engineering + math needed to “size” LLMs.
  • Grit: willingness to do intensive, time-consuming work.

B) How to “signal” you’re valuable (what to show outside interviews)

• Provide evidence of creating something useful to others, especially in LLM efficiency/runtime:

  • kernel-related improvements
  • not necessarily “universal” improvements, but targeted improvements for specific settings

• Contribute to or demonstrate work on open-source serving stacks and inference systems, including examples Vlad mentions:

  • VLM
  • SGLANG
  • demonstrations with TensorRT
  • systems like “Dynamo” for disaggregated serving

• Look for work that improves:

  • inference efficiency
  • KV cache handling
  • request queuing/load balancing (backend distributed-systems concerns)

C) How to adapt if you’re doing an internal transfer (not external hiring)

• If your current org isn’t generating frontier models, Vlad suggests:
  • focus on the gap of applying LLMs effectively inside your organization
  • become the person who helps product areas serve and integrate LLMs effectively
  • this can naturally create partnerships with frontier research teams and may lead to transfer (or you may already be “doing cutting edge” work by integrating models well)

---

Examples / stories used to reinforce points

• Distillation infrastructure rewrite → faster research → scaling-law progress

  • Infrastructure redesign reduces experimentation time and unlocks new distillation scaling understanding.

• Flash 2.0 war story

  • A key latency/serving optimization decision:
    • route computation differently (pipeline prefill across layers rather than experts across machines)
  • Training described as extremely intense:
    • ~40 days
    • small team rotation
    • keeping jobs stable
    • quickly fixing data/iterator issues
  • Narrative mismatch noted vs. a Wall Street Journal article and benchmark leaderboard timing.

• Spot bonus story

  • Jeff Dean gave a spot bonus early in the Gemini/Bard rollout, tied to engineering contributions (helping with supervised fine-tuning).
  • Overall lesson: engage with product-critical work.

---

Sources / speakers featured

• Vlad Feinberg (Google DeepMind; pre-training area lead; main interviewee)
• Ryan (podcast host; asks questions)
• Jeff Dean (mentioned as giving a spot bonus; Google leader)
• Roana Neil (Vlad’s manager; encouraged broader engagement beyond pure research)
• Nate Linds (internal transfer example; now owns inference code design for Flash/Flashlight)
• Jacob Steinhart (professor) (author of a post framed in terms of research as an MDP)
• Professor “Todd Lipkin” (mentor/friend mentioned; “computer science” origin for Vlad and a model colleague)
• Barbara Liskoff, Mike Stonereaker, Mark Brooker (named by the host as past podcast guests)
• Dee (author of a referenced tweet about fear-mongering / “AI permanent underclass”)
• OpenAI, Anthropic, Cursor, Vercel (examples of companies / interview/post readership and/or ecosystem tools)
• WorkOS (sponsor; mentioned via ad segment)
• “Kaplan paper,” “Chinchilla,” “GPT,” Google PaLM papers (scaling-law literature referenced)
• Thunderkittens (programming language abstraction example)
• VLM, SGLANG, TensorRT, Dynamo (tools/systems referenced)
• Bard, Gemini, Flash 1.5, Flash 2.0, Flashlight, Ads, YouTube, OneP models (products/models referenced)
• DeepSeek V3 (compared in the Wall Street Journal / leaderboard narrative)
• Wall Street Journal (referenced article/source)
• Princeton (where Vlad gave a lecture mentioned in context)