Garry Tan: Own Your Intelligence

Main ideas / concepts / lessons

• Spinoza as a model of “owning your intelligence”

  • Baruch Spinoza is presented as an extremely “canceled” figure: excommunicated by his Amsterdam Sephardic Jewish community at age 23, with a ban that included social/financial exclusion and no repentance clause.
  • He refuses offers of money to stop building and instead pursues truth and usefulness.
  • He survives an attempted stabbing, keeps the scar, and turns his exile into creation—grinding lenses by day and writing a dangerous philosophical work by night.
  • Key lesson: when deprived of comfort, compensation, and permission, keep building tools and knowledge.

• Conatus: the internal engine

  • Spinoza’s concept Conatus is framed as the drive in living things to persist and increase capability.
  • Success isn’t attributed to resumes/titles; it comes from amplifying conatus with tools.

• A “Spinoza update” about intelligence (AGI is not an event)

  • Traditional view: people wait for AGI like a singular threshold/event (“a god in a data center”).
  • Reframed view: intelligence is diffused/infrastructure, appearing through everyday systems and agents working continuously in your context.
  • Personal AGI is introduced:
    • Not a rented subscription chatbot.
    • Not corporate AGI you don’t own.
    • Yes to an agent that runs on your infrastructure, uses your owned memory, executes your procedures, and improves with your usage.

• Personal AGI as an owned “asset,” not consumed “product”

  • Corporate AGI (described as subscription/rent):
    • resets with tab closure,
    • relies on corporate knowledge,
    • can be “lobotomized” by the provider’s changes.
  • Personal AGI (desired):
    • an agent that reads from your memory,
    • runs your tools,
    • composes/improves daily based on your life.

• Why this is happening now: agent leverage, especially for coding

  • Speaker anecdote:
    • As a YC partner in 2013: shipped ~14 useful lines/day.
    • In a current YC full-time role: output is argued to be ~400× (with caveats about verbosity).
  • Generalized beyond coding: design, PM, growth.
  • YC portfolio observation:
    • Winter 25 batch shows many companies with ~95% AI-generated codebases and rapid growth.
  • Core takeaway:
    • Winners treat AI as a workforce, not autocomplete.
    • Leverage comes less from model weights and more from context relevance and choosing the right step in the workflow.

• Working memory vs “agent memory”

  • Human working memory: ~7 items (±2).
  • AI agents can handle far larger contexts (tokens/pages), enabling needle-finding and synthesis across many documents at once.
  • Implication: organizations/processes built for “seven-digit brains” are becoming outdated.

• “G brain” / personal agent architecture

  • Speaker describes a personal system (“G brain”):
    • a knowledge wiki plus an agent that compiles/curates/searches personal history,
    • avoids re-asking what it already answered,
    • uses context before drafting/responding.
  • Example workflow:
    • Inbox triage while sleeping
    • Morning briefing rather than an email pile
    • Pre-meeting prep docs referencing prior decisions and changes
    • Research cross-referenced against personal interests and filed automatically

• Skill files: the core unit of execution

  • A skill file is described as:
    • a page of clear instructions in English (Markdown),
    • specific enough that a smart intern could follow it (clarity test),
    • executable by an agent (not magic).
  • Principle: “Fat skills, thin harness”
    • Skills contain most of the task logic; the harness is lightweight glue.

• Latent vs deterministic computation

  • Emphasized separation:
    • Latent space computation: judgment, preference, taste—steered by Markdown instructions
    • Deterministic space computation: arithmetic/SQL/logistics—run via databases/scripts
  • Failure mode: letting the model do deterministic work poorly.
  • Fix: use a model + database/script split.

• Hygiene / provenance to prevent confident wrongness

  • Warning: a brain without curation becomes a garbage dump—retrieval can return stale/wrong facts with confidence.
  • Mitigation:
    • memory hygiene,
    • provenance on facts,
    • contradiction checks when new info conflicts,
    • a “librarian” role that prunes.

• How to turn leverage into organizations

  • A skill file is likened to an “employee.”
  • A resolver/task routing layer resembles an org chart.
  • Implication: you can “incorporate” an organization of one (plus agents) before traditional company assets (co-founders, logo, deck).
  • Anecdotes:
    • YC summer 24: startup goes from launch to nine-figure revenue in eight months with very small headcount at high revenue levels.
    • YC winter 24: high annualized revenue with relatively few people vs historical norms.

• Political thesis: ownership determines your future

  • Main caution: skill files can represent either:
    • ownership (you keep them when you change jobs), or
    • extraction (company keeps them under IT policy; you leave without your compounded judgment).
  • Fictional support engineer example (“Maya”):
    • Version 1: skills live in her repo → she moves jobs with accumulated expertise.
    • Version 2: skills live in company repo → skills continue without her; her judgment is effectively removed.
  • Doctrine statement:
    • Own your skills; otherwise your job becomes the skill file you don’t control.
  • Historical framing:
    • Spinoza was offered money to stop; modern equivalents are framed as arrangements that let you “stop building comfortably.”
    • Leibniz visiting Spinoza illustrates how powerful ideas can be contested and even misrepresented; the talk focuses more on adoption curves for agent-enabled workflows.

• Objections addressed

  • Objection 1: models improve so harness becomes obsolete
    • Response: differentiation shifts toward context and library ownership; better models amplify what your library does.
  • Objection 2: isn’t this just RAG?
    • Response: retrieval is a primitive; the product is what makes you “worth retrieving from,” including enrichment, linking, and contradiction arbitration.
  • Objection 3: what if your life leaks?
    • Response: mitigate via custody/keys—run on your own infra under your own keys rather than scattering context across corporate cloud accounts.

• Why open source

  • Claimed because it can be done (“because I can”).
  • Powerful leverage tools should be distributed to avoid creating a private priesthood.
  • Building in public and adoption curves lead to copying/clone dynamics.

• Final moral / closing vision

  • Personal AGI is framed not as a benchmark/demo, but as aimed architecture for one person’s real needs (example: a father building a library/agent for a child’s rare epilepsy).
  • Closing credo:
    • Build what others won’t.
    • Share tools.
    • Create institutions/leverages so effort goes straight to work—not through decades of hiring, funding, and permission.

---

Methodology / “how-to” instructions (detailed bullet list)

The 5-step setup the speaker recommends (in order)

1. Step 1 (tonight): pick a harness and run an agent on your own machine

   • Example harness/agent options mentioned:
     • OpenCloak (as named in subtitles)
     • Hermes agent with “G brain”
     • Hosted/open alternatives (example: gbrain.io as a hosted option)
     • Claims Codex / Cloud Code-type options can work for “99%” of the approach
   • Principle: concepts matter more than any single repo/product.
   • Goal: get an agent working with your local workflow ASAP.

2. Step 2 (this weekend): start your library

   • Build one folder of Markdown files.
   • Export notes and (if possible) email.
   • For each:
     • project you’re working on
     • person you work with
   • Create one page that captures what you actually know:
     • what you’re building together
     • what they care about
     • what you owe them
     • what they said last time
   • Emphasis:
     • models don’t have this; it exists in your head
     • agents become valuable when they answer using your context instead of the internet

3. Step 3: write your first skill file

   • Choose the task you do every week that you hate the most (e.g., expense reports, meeting notes, weekly status updates, competitor research).
   • Write plain-English instructions:
     • explain the job as you would to a smart friend on their first day
   • Allow the agent to try (even if it gets it wrong):
     • correct it
     • include rules, exceptions, and “oh and also” details
   • Result: the skill file becomes an “employee” you can run.

4. Step 4: wire it up as a recurring job

   • Convert the skill into scheduled automation, such as:
     • “Every morning at 7:00, do this”
     • “Every Friday, summarize that”
   • Expected shift: work finishing while you slept changes your sense of what “work” and “time” mean.

5. Step 5 (discipline for compounding): never do one-off work

   • Common mistake: run one agent task, then throw away the context/output.
   • Instead, at the end of each task:
     • have the agent skillify what it did (turn it into a reusable skill file)
   • If you have to ask for something twice, you “failed” (YC-style maxim).
   • Goal: knowledge becomes real memory (skills), not transient execution.

What progress should look like (time horizons given)

• Week 1: it’s “a toy”
  • library thin; skills clumsy; you’re fixing more than saving
• Week 4: the “flywheel catches”
  • agent answers using your context
  • morning job produces something you actually read
  • you write additional skill files because earlier ones worked
• Week 12: you have a functioning library + many skills
  • a dozen skill files cover recurring week parts
  • a small set of tools others ask to borrow (“which in this room is called a startup”)

Guardrails / hygiene requirement (important caveat)

A brain without curation is a garbage dump that retrieves confidently wrong facts.

Include:

• provenance for facts
• contradiction checks
• pruning via a “librarian” role
• treat the system as production infrastructure

---

Speakers / sources featured (as named in the subtitles)

• Garry Tan (speaker)
• Baruch Spinoza (philosopher; excommunicated “canceled man”)
• Albert Einstein (telegram exchange with a rabbi; quotes invoked)
• A New York rabbi (unnamed; challenges Einstein by telegram)
• Vannevar Bush (Memex idea)
• Karl (or Paul) McLuhan / Marshall McLuhan (“technology is an extension of man”)
• Steve Jobs (“bicycle for the mind”)
• Paul Graham (“make something people want” and “do things that don’t scale”)
• Gottfried Wilhelm Leibniz (visit to Spinoza; notes and later denial/cover-up theme)
• Nadler (Spinoza biography source)
• Goldstein (Spinoza biography source)
• Stewart (Spinoza biography source)
• CircleBack (referenced in a sample skill)
• GitHub (referenced via stars/open source ranking)
• Y Combinator (YC) (organization; referenced repeatedly)
• OpenAI Codex / Cloud Code (coding harness examples)
• Anthropic Claude (example harness: “Claude coder”)
• Hermes agent (part of harness/example)
• G brain / G stack (speaker’s system/tools)
• Postgres (analogy: retrieval ≈ index structure)
• B-trees (mentioned in the Postgres analogy)
• RAG (retrieval augmented generation; indirectly referenced)
• Memex (Vannevar Bush’s concept)
• “Ethics” (Spinoza’s work; referenced)
• “The Seminal paper in cognitive psychology” (working memory claim; paper not named)