Video summary

Construyo mi propio arnés de IA… y te enseño como hacer el tuyo

Main summary

Key takeaways

Technology

What the Video Builds (Goal)

  • The author teaches how to build an “AI harness” from scratch: a system that upgrades a basic chatbot into a real agent.
  • The harness supports:
    • Reasoning in a loop
    • Tool usage
    • Delegation to sub-agents
    • Operating with manageable context
  • A tutorial repository is provided so viewers can run and extend the system (educational and meant for tinkering).

Core Concepts: Harness Layers (“Onion” Model)

The harness is described as layers:

  1. Core / brain layer
    • Code that calls the LLM (connectivity + “control” of the model).
  2. Tool layer / skill layer
    • Creation of tools, sub-agents, and behaviors that influence the core loop.

Earlier videos modified an existing harness (e.g., Cloud Code). This one builds the core from scratch (conceptually like creating your own “Cloud Code”-style binary).

The Main Technological Pattern: RPL Loop + Internal Agent Loop

The architecture is explained using a game-loop analogy, then mapped to the agent’s loop.

External Loop: RPL (REPL-style)

  • Read user input (chat/UI command).
  • Print/Evaluate results (UI updates).
  • Loop back waiting for new input.

Often called REPL / Read-Eval-Print-Loop, sometimes “RPL”.

Internal Loop: EVAL (Inner Agent Loop)

When a user triggers something, evaluation may require multiple steps:

  1. The agent calls the LLM (via provider SDK).
  2. The model returns responses that may include:
    • Direct text answers, or
    • Tool-use requests (e.g., “read a file”, “run bash”).
  3. The harness executes requested tools and feeds the results back to the model.
  4. The inner loop ends when the model signals completion (“stop reason”).

Minimal Code Demonstration (What’s Implemented)

The repo implements a small harness (about ~175 lines in Go) showing:

  • A main function with the outer RPL loop.

  • An internal agent loop calling an LLM (example mentions Anthropic SDK and “Clou SDK / Opus”).

  • Handling arrays of model response messages, since the model may chain multiple tool uses.

  • A tool executor:
    • The model requests a tool.
    • The harness executes it locally on the machine.
    • Results are returned to the model.

Demo run: the harness reads prompts, calls the LLM, and can instantiate tools like a bash tool (e.g., “how many files are in this folder?” triggers a bash command).

Tooling System: Harness Executes Tools (LLM Chooses)

Tools are implemented by the developer:

  • Each tool includes:
    • a definition (description + input schema) provided to the LLM, and
    • an execution function that performs the action.

The LLM uses inference over tool descriptions to decide which tool to call. Note: poor tool descriptions can prevent correct tool selection.

Provider Abstraction with Polymorphism (Swap LLM Vendors)

A major design feature is polymorphism + interfaces to abstract differences between LLM providers.

  • The repo defines generic interface/types (e.g., provider sends messages and returns generic responses).
  • Provider implementations:
    • Anthropic provider: translates Anthropic message formats into generic harness responses.
    • OpenAI provider: does the same translation for OpenAI SDK structures.
    • Local/mock provider: usable for testing without charges (a “MOCK model” is mentioned).

System Prompt + agent.md Integration

The harness:

  • Loads a system prompt (hardcoded in the example).
  • Optionally incorporates an agent.md file into the system prompt (or as user context), depending on harness rules.
  • Can be extended to read repository files and inject extra context.

Safety: Permissions Gate for Tool Execution

Tools run without supervision by default, which creates a security risk.

Proposed mitigation:

  • Add a permissions/approval mechanism in the tool execution path.
  • Before executing a tool, the harness prompts the user:
    • if approved → execute,
    • if not approved → skip/deny.

This is positioned as a security gateway around tool runs.

Sub-Agents: Delegation Implemented as Another Tool

Delegation uses the same tool mechanism:

  • The main agent can call a delegate tool.
  • The delegate tool instantiates a new agent instance (spawns another internal loop / “character”).

The sub-agent:

  • can have its own context strategy (e.g., research agent may start with empty context or inherit context),
  • runs its own internal loop,
  • returns results back to the parent flow.

Extensibility Topics Mentioned

MCP Support (Model Context Protocol)

  • Implemented as wrappers/tools that can register MCP servers via configuration (loaded from an mcp.json-like file).
  • MCP tool definitions are loaded via HTTP fetching.
  • MCP loading is done in parallel (goroutines) to avoid blocking UI.

Slash Commands

  • Example: barrah help for harness commands.

Context Compaction Strategies

  • “None”
  • “Sliding window”
  • “Summary”
    • “Summary” uses the LLM to summarize history to control token growth.

Strategies are modular: you can add a new one by implementing a compaction interface.

Cost / Token Tracking

  • Providers expose token usage.
  • The harness estimates cost using per-model price configuration.

Memory Systems (as Tools)

  • Added as tools:
    • remember (store facts)
    • recall (retrieve by query/tags)
  • Example stores memory in JSON files with indexing/session metadata.
  • The harness can recall events like “what happened yesterday” using stored memories.

Debug / Visualization UI: “debook mode”

The repo includes a viewer to inspect:

  • system prompt size and growth,
  • payloads sent to the provider (JSON),
  • provider responses,
  • how memory and context management affect calls.

It demonstrates:

  • agent.md causing prompt bloat,
  • context compaction showing before/after message state,
  • options like verbose mode.

Tutorial Repository Contents (What to Learn)

The author links a repository covering:

  • full explanations of components:
    • context manager
    • memory manager
    • tools
    • subagents
    • MCP servers
    • tutorial chapters
  • English and Spanish versions
  • step-by-step “chapters,” including building the RPL/agent loop from scratch
  • exercises (at least six), including:
    • modifying the agent loop for tool error retries
    • adding sub-agents via Markdown (Cloud Code-like dynamic loading)

Main Speakers / Sources

  • Primary speaker: the video author (unnamed in subtitles; repeatedly uses “I” while guiding the implementation).
  • Primary technical sources referenced/used:
    • Anthropic SDK (mentions “Opus”)
    • OpenAI SDK (referred to as “openi”)
    • MCP (Model Context Protocol)
  • Repository: the author’s accompanying Git repository (publicly linked in the description).

Original video