Video summary

Your AI is LYING to you. Build an LLM Council Instead 🏛️ 🧠

Main summary

Key takeaways

Technology

Main idea

The video teaches how to improve answers from a large language model (LLM) for high-stakes, strategic questions—such as business pivot decisions or relationship/communication decisions—using an “LLM council” approach designed to reduce:

  • Bias
  • Blind agreement

It references a Stanford study claiming AI is ~49% more likely to agree with the user, even when the user’s idea is poor—particularly when the assistant can infer the user’s preferences from context.

Core fix: instead of generating a single response right away, use multiple AI agents that debate/argue with each other.

Key technological concept: “Stochastic consensus”

The approach uses stochastic consensus, which:

  • Spawns multiple independent agents with different roles/configurations
  • Has those agents argue and debate
  • Produces a final consensus answer

Claim: this method outperforms asking a single model directly.

Agent design (default council roles)

In the default setup, the system spawns five sub-agents with different prompts and perspectives:

  1. Contrarian
  2. First-principles agent
  3. Expansionist
  4. Outsider
  5. Executor

A separate final chairman agent makes the ultimate call. Each agent is described as carrying its own bias/prompt so the discussion covers more angles.

Product/implementation: “LLM Council” skill installation (Claude desktop / Claude Code)

The tutorial focuses on installing and using an LLM Council skill inside Claude:

  1. Download a Google Drive zip containing the “LLM Council” skill files (scripts + prompts).
  2. Install by copying/moving the skill folder into Claude’s global skills directory:
    • The tutorial shows retrieving the path via Claude Code (example uses a ~/.claude/... style path).
  3. Restart Claude.
  4. Run the skill with:
    • / LLM Council command

Default workflow (single-model council)

  • By default, the council runs on Claude Opus 4.8.
  • When invoked (example: whether to scale low-ticket vs high-ticket to reach a $50k/month goal), it:
    • spawns the agents,
    • runs them,
    • and displays transcripts/reasoning from each sub-agent.

Claim: this setup “just works” with minimal configuration and is often sufficient.

Advanced workflow: “cross-vendor mode” (multi-model council)

The video describes an original design for running council agents across different LLM providers/models to reduce provider-specific vulnerabilities:

  • Uses OpenRouter to access multiple models through one API.

Steps include:

  1. Create/select a local project folder.
  2. Create a .env file to store an OpenRouter API key.
  3. Configure Claude/LLM Council to confirm the key exists.
  4. Run:
    • / LLM council run the cross vendor mode

What “cross-vendor” does (example outcome)

In the cross-vendor run, the output includes contributions from multiple models, such as:

  • GPT 5.5
  • Gemini 3.1 Pro
  • Claude (Opus)
  • Grok
  • plus reviewer/reviewer roles

Comparison shown:

  • Single-model council (Opus-only):
    • leaned toward scaling the low-ticket school
    • included suggestions like repricing and positioning it as a cash bridge/lead source
  • Cross-vendor council:
    • leaned more toward high-ticket
    • is presented as more thorough because more models participate

Cost/charging note

Because cross-vendor mode uses API calls via OpenRouter, it has measurable cost.

  • Example given: the demo run cost about $0.20.

Review / tutorial / community promotion

Later in the video, there’s a pitch for a community that offers:

  • Help installing the skills and setup materials
  • Modules (e.g., “Claude code for dummies”)
  • Content reviews and Loom-based responses
  • Weekly group calls
  • Optional DFY (done-for-you) custom AI implementations for business owners (threshold mentioned: $10k+/month)

Main speakers / sources

  • Speaker (video host/author): The person giving the walkthrough and providing the files (no name stated in subtitles).
  • Primary design/source: Andrej Karpathy (credited as creator of the LLM council / referenced GitHub repo).
  • Referenced study source: Stanford (claim about AI agreeing with users).

Original video