I lead a Google DeepMind team at 26. If you want to work at an AI company... | Neel Nanda (Part 2)

Main ideas, concepts, and lessons

1) Mechanistic interpretability: what it is and why it matters

Mechanistic interpretability is research aimed at understanding:

• Why AI models behave the way they do
• How they internally perform their computations
• Why they select one behavior/feature over another

The speaker credits the field’s growth from a small effort to a large, many-researcher effort (hundreds contributing).

2) Career acceleration early (being young, producing impact)

Neel argues that early impact came from:

• Luck / timing (“right place at the right time”)
• Opportunity capture: being good at taking opportunities as they arise and making them
• Management/supervision as leverage: supervising and improving many researchers increases overall impact

He also highlights avoiding a long PhD as a major time-saving decision (he has an undergraduate degree).

3) Cold email strategy (for getting responses and mentorship)

Key principle: the recipient is busy and will stop reading at an unpredictable point—so optimize for fast skimming and clarity.

Guidance / method:

• Be concise; put the key info first.
• Prioritize key content and bold important keywords/phrases.
• Check “can you answer/respond within a minute?”
  • If yes, make the ask extremely prominent and turn it into a specific question that can be answered quickly.
  • If not, expect lower response rates.

• Signal competence:

  • Include impressive achievements so they can prioritize you quickly.
  • Avoid sounding arrogant; use framing like: “I’m sure you get many emails—here are the key impressive things about me so you can prioritize.”

• Email people more likely to reply:

  • Don’t always target the most senior/famous person (they’re usually busiest).
  • For mentorship, consider more junior people:
    • they may have more availability while still offering useful guidance (including people recently joined to a team).
  • Prefer paper first authors over later senior/canonical authors for mentorship requests.
• Use structure for skimmability:
  • Prefer bullet points.
  • If it’s a long document, include a one-paragraph blurb plus a link.
• Mentorship channel disclosure:
  • For mentorship requests, his main route is via the MATS Program (applications open; put this prominently).

4) Using LLMs/AI to skill up faster (and how to do it effectively)

Neel claims that not using LLMs when trying to enter a field is “a mistake,” but emphasizes using them as a tool, not blindly.

Core mental model

• LLMs lower the barrier to entry:
  • They’re good at junior-level performance
  • They’re not perfectly reliable, but reliability isn’t required for reaching the “right bar” (similar to how juniors don’t need perfect reliability).

Practical prompting/workflow tips

• Use strong system prompts / detailed instructions to define the task.
• Use provider “projects” features (multiple prompts + contextual docs in one place).
• If you’re bad at prompting, have the LLM write the prompt for you.
• Voice dictation / messy input handling:
  • Dictate rambly thoughts; ask the LLM to transcribe/neaten into coherent output.
  • He recommends Gemini for audio input support.
• Feedback-driven prompt iteration:
  • If output is bad, give specific feedback on what you didn’t like, then ask for a rewritten prompt and reuse.
• Add the right context:
  • Put key papers/literature reviews into the context window.
  • For mechanistic interpretability, include relevant key works and literature reviews.
• Don’t be passive—test comprehension:
  • Instead of only “summarize the paper,” ask for exercises/questions that verify understanding.
  • Try: ask the LLM to summarize the whole paper in your own words, then ask what you missed.
• Reduce sycophancy (LLMs flattering the user) using anti-sycophancy prompts:
  • Instruct it to be “brutal but truthful” / “most brutally honest constructive criticism.”
  • Note: this can become emotionally harsh if used on emotionally invested topics.
  • Soften tone when needed: “honest but sensitive growth-oriented feedback.”
• Use tools/interfaces for prompt comparison:
  • He recommends Google AI Studio (Gemini interface) for side-by-side comparison across models and prompts (including “brutal” vs “lighter”).
• Improve responses via extra effort and iteration:
  • Run the same question across top models.
  • Ask one model to:
    • assess strengths/weaknesses of each
    • combine them into a single best response
  • Optionally iterate via critique-and-rewrite loops.
  • Save prompts and use snippet tools (e.g., Alfred on Mac) to reduce friction.
• For coding: use LLMs differently depending on goals:
  • Learning by experience: write code yourself, using LLMs mainly as a tutor.
  • Finishing a task: use command-line agents (e.g., Gemini CLI / Claude Code).
  • High code quality or hard debugging: use an IDE-integrated tool like Cursor.

Bonus uses

• Catch “unknown unknowns”:
  • e.g., “I want to get into mech interp—here’s my rough plan; what am I missing?”
• Literature reviews:
  • ask for all relevant papers; tools with search (“Deep Research” etc.) can help.

5) Medical advice caution (and an accuracy-improving trick)

• He agrees LLMs can hallucinate symptoms/treatments, including plausible but incorrect details.
• He distinguishes “life utility/shopping” from “medical treatment.”
  • Example: for medical posture devices, he used LLMs for product discovery, not prescription.

Accuracy technique:

• Feed one LLM’s answer to a second LLM with an anti-sycophancy prompt asking it to validate/scrutinize the first answer.
• Use different providers when possible to reduce bias.

Still emphasized: not perfectly reliable; don’t treat as high-stakes medical guidance.

6) Safety work should advance capabilities—argument and nuance

Provocative take defended:

• If safety work doesn’t improve capabilities at least somewhat, it’s probably not “good” safety work.

Defense logic:

• Safety aims for models that do what we want and avoid what we don’t want (control/avoidance).
• Many safety issues are already commercially harmful (reward hacking, hallucinations, jailbreaks).
• Criticizing safety work as “capabilities work” doesn’t make sense if it advances the ability to produce desired behavior.

Nuance he adds:

• Some work advances safety more than general capability.
• Work without a plan for making systems better at desired behavior is questionable.

Why market incentives won’t fully ensure safety techniques

• Companies often fix urgent issues with the default cheapest solution (e.g., more fine-tuning data).
• Some safety problems (notably deception) may not be solved this way.
• Interpretability/evals/monitoring may be necessary earlier, but won’t be pursued if waiting for commercial incentives.

7) Safety “confidence and timelines” problem (overconfidence + humility)

He argues:

• “p(doom)” and timelines are extremely hard forecasting problems.
• People anchor on prominent figures too much.

He avoids public p(doom)/timelines to prevent anchoring and because it’s not action-relevant enough.

Recommended mindset under uncertainty:

• Act as though scenarios are uncertain, and build a portfolio of safety actions.
• Use a framing like “AGI tomorrow”:
  • ensure you’d be in a good state if AGI arrives quickly
  • but also allocate resources for longer horizons (6 months+, possibly 5 years+).
• Avoid burning bridges or destroying credibility if AGI takes longer.

8) How he chose technical AI safety (a path story + updated rationale)

• Early interest influenced by EA via Harry Potter fanfiction (humorous origin).
• Quant finance internships (Jane Street), but he realized he didn’t truly understand AI safety research.
• He tried multiple AI safety internships during/around COVID and didn’t love the direction, but learned enough to become convinced real work existed.
• He joined Anthropic early (interpretability team) via luck/opportunity, then transitioned to DeepMind leadership later.

9) Working in large organizations: not a monolith; internal “inefficiency”

Large organizations:

• delegate decisions, have stakeholders and bureaucracy
• don’t behave like a single coherent decision-maker

He uses an analogy: markets aren’t perfectly efficient.

• Markets have specialists who find inefficiencies.
• Companies don’t always have people dedicated to safety-relevant inefficiencies across the org.

Cultural prioritization also differs:

• ML culture often optimizes for problems visible today.
• Safety requires earlier handling of subtler, slower-to-fix issues.

10) Applied mechanistic interpretability for production models: what matters

For tools deployed to large-scale products, he emphasizes four evaluation criteria:

1. Effectiveness
   • Does it actually solve the problem?
   • Is it solving a problem stakeholders value?
2. Side effects
   • Potential harms to model performance or user traffic outcomes.
   • Example: a harmful-prompt monitor should not trigger often on normal traffic (and this is often not checked in papers).
3. Expense
   • Runtime/cost impact (cheap fine-tuning vs expensive per-query large model processing).
4. Implementation cost
   • Even “mathematically small” ideas may require deep code/serving stack changes.
   • Integration complexity and stakeholder resistance matter.

Additional adoption insight

• Build common ground so stakeholders with current product priorities can form a coalition.
• Incremental adoption is easier than adding new components.
• Submitting “pull requests” to existing infrastructure is easier than asking busy engineers to evaluate a brand-new architecture.

11) Advice for external researchers (outside frontier labs)

External work should become compelling enough that lab safety teams decide it’s worth evaluating.

Practical approach

• Labs can do more end-to-end testing:
  • check trigger rates
  • evaluate on private benchmarks
  • implement directly
• External researchers should aim for:
  • enough evidence to justify deeper lab investigation
  • anticipation of production constraints:
    • number of stakeholders needed
    • complexity/cost/side effects
  • solid evaluations, while keeping lab teams actively engaged during research (not only at the end)

Outreach pattern

• Maintain contact with relevant lab staff throughout the process (not only top leadership).
• Junior team members can provide valuable context and escalate if it looks promising.

12) Steering internal governance in big AI labs

He suggests impact comes from being:

• a neutral, trusted technical advisor
• building credibility with decision-makers (not necessarily “playing politics”)

Calibrated communication

• Avoid scaremongering, but call out real issues so people listen when it matters.

He notes skepticism may be justified, but impact can be real:

• many changes come through senior interfaced leaders,
• supported by research evidence from teams.

Team-level leverage

• Engineering efforts that accelerate research iteration speed can be highly impactful.
• He praises an engineering lead and invites engineers to email him if interested.

13) Safety and crisis-preparation: “managed crises”

He favors risk management frameworks that create managed crises, including:

• early warning tests
• thresholds for mitigations
• pre-prepared mitigations

Examples mentioned include concepts from Frontier Safety Framework / Responsible Scaling Policy ideas, such as:

• Gemini 2.5 Pro offensive cyber warning
• deceptive alignment inclusion
• alignment faking

14) Career skills for frontier AI hiring

He provides a checklist of traits.

Two key skills frontier companies care about

• Research ability, especially proven research experience
• Engineering skill, including:
  • deep experience working in large codebases (not just LeetCode-style puzzles)
  • ML engineering (efficient training + debugging messy failures)
  • ability to use LLMs to help navigate/maintain systems

Building research credibility

• Papers are portable credentials, regardless of PhD prestige.
• Work relevant to frontier language models matters more than institution alone.
• Show lab “fit” by meeting researchers and contacting people whose work you admire.

15) What makes a good researcher + how research process should be managed

Key researcher skills

• Good enough coding to run experiments (often hacky/small-scale in notebooks)
• Fast iteration
• Ability to operate under uncertainty
• Empirical scientific mindset + skepticism + robustness checks
• Research taste:
  • choosing good projects/directions
  • designing tractable experiments that test the right hypothesis

Research taste feedback-loop timing

• Coding feedback: minutes/hours
• Conceptual understanding/interpretability: hours
• Prioritization: weeks/months
• Research taste: increasingly slower feedback (months)

Misconception: research taste is only choosing the project

He argues research taste applies across levels:

• experiment design
• direction selection

Three stages of the research process

1. Explore
   • Goal: gain information; expand problem understanding
   • Mindset: open-ended learning; maximize “surface area”
   • Practical: use tools/techniques that yield rich information quickly
   • Checkpoint rule: if you learn nothing after ~1–2 days, consider changing tools
2. Understand
   • Goal: test a hypothesis; get evidence for/against specific claims
   • Mindset: directed experiments with a plan
   • Practical: design experiments that manipulate variables and test predicted effects
3. Distil
   • Goal: communicate what you believe and why, rigorously and convincingly
   • Mindset: craft narrative + clarity for outsiders
   • Practical: get feedback on confusion points; fix misunderstandings
   • Paper structure emphasis:
     • explain narrative + motivation early
     • then provide evidence/details supporting only key remembered ideas

16) Communication: tweeting and paper hype vs limitations

• He emphasizes distillation into a single narrative.
• For tweet threads:
  • ~95% won’t read beyond tweet 1, so tweet 1 must contain the whole narrative
  • avoid excessively long tweets that force clicks
  • use a single eye-catching figure (often the first figure) that conveys meaning with minimal context
  • include motivation + recognizable keywords; avoid jargon for non-experts
• Integrity matters:
  • limitations should be prominent (ideally in abstract/intro)
  • hype without substance damages reputation with experienced researchers

17) Supervising researchers efficiently

MATS scholar mentorship system

• Takes on 8 new scholars every ~6 months
• Pairs them, with:
  • weekly 1.5-hour check-ins per pair

Why it works:

• Scholars get fast feedback loops from coding they do themselves
• He provides “expensive-to-learn” expertise (prioritization, research taste) efficiently

18) PhD skepticism + mentorship as entry route

• PhD isn’t mandatory; it’s treated as a skills environment.
• You can leave early if better industry research opportunities appear.
• Mentorship matters more than institutional credentialing.
• He warns supervisors can be harmful (information asymmetry) and advises:
  • talk to students candidly
  • ask about supervisor time, benefits, and how they’d change things

19) Final meta-lesson and motivation

Closing advice:

• Doing things is a skill.

Increase “luck surface area” by:

• meeting people
• saying yes to opportunities
• doing “weird” projects
• accepting low-risk downside opportunities

He gives examples like:

• writing a blog post daily for a month
• accidentally stepping into leadership

---

Methodologies / instructions explicitly provided (collected)

Cold email methodology

• Assume quick skimming and early stopping.
• Be concise; lead with the actionable ask.
• Bold key phrases; use bullet points.
• If possible, make the request answerable within ~1 minute.
• Signal competence early, with an anti-boast framing.
• Prefer:
  • first authors over senior-end authors
  • junior people over leads (for mentorship questions)
• For longer docs: short paragraph + link pattern.
• Mention MATS when asking for mentorship.

LLM upskilling workflow

• Use system prompts / detailed instructions.
• Use “projects”/context features where available.
• Generate prompts via the LLM if needed.
• Use voice dictation; ask the LLM to transcribe/neaten.
• Iteratively improve prompts with explicit feedback.
• Provide relevant documents/literature in the context window.
• Actively verify comprehension (exercises/questions; summarize in your own words).
• Prevent sycophancy:
  • apply anti-sycophancy prompts for honest critique
  • optionally soften tone depending on stakes
• Use AI Studio compare mode for side-by-side response comparison.
• For important questions:
  • gather multiple model answers
  • ask one model to compare and combine
  • iterate with critique-and-redraft loops
• For coding:
  • write code yourself when learning by experience
  • use agents/IDE tools when you need completion or strong debugging help
• Use LLMs for:
  • discovering unknown unknowns
  • literature search/review

Research process stages (explore → understand → distil)

• Explore:
  • maximize information gain per time
  • use rich tools; zoom out/iterate
  • change tools if not learning after 1–2 days
• Understand:
  • form hypotheses and plan experiments to test them
• Distil:
  • communicate narrative + evidence clearly
  • seek external feedback; fix confusion
  • prioritize a small set of ideas readers will remember

---

Speakers / sources featured

• Neel Nanda (speaker; runs Google DeepMind mechanistic interpretability team; also discusses career and safety views)
• Rob Wiblin (podcast interviewer; 80,000 Hours podcast)
• Arthur Conmy (credited as running the applied mechanistic interpretability team; also identified as a MATS alumnus)
• Rohin