Descubrí que el harness importa más que el modelo

Summary

The video argues that an AI coding agent’s harness—the instructions, tools, and safeguards around the model—can matter more than the model alone. The creator tests this idea on a Python task API using six team rules and five realistic tickets.

Main harness components

1. AGENTS.md for always-relevant guidance Keep it concise and practical: describe the project, provide exact commands, map important folders, and state verifiable rules. For Claude Code, an existing CLAUDE.md can import it with @AGENTS.md.

The creator warns against turning the file into a large, fast-aging wiki, since it consumes context and makes priorities unclear.

1. Skills for task-specific procedures Skills hold recipes—such as how to add an endpoint—that load when relevant rather than being included in every session. The video describes the skills format as an open standard supported by multiple coding agents.

Rule of thumb: Put information needed all the time in AGENTS.md; put occasional workflows in skills.

1. Hooks for rules that must be enforced Hooks run at defined points, such as before or after tool use, or when the agent tries to finish. Unlike written instructions, they do not depend on the model choosing to follow them. Examples include protecting .env, blocking unauthorized dependency edits, checking rules after changes, and running checks before completion.

The creator notes that hooks run code on the user’s machine and should be reviewed like dependencies.

In the test, hooks ran 342 times across 13 normal runs and blocked one action involving .env. That block was a false positive: the command mentioned the file but did not read it. The example illustrates how name-based rules can both miss real cases and trigger on harmless ones. The creator also explains that Claude Code treated exit code 1 as a non-blocking hook failure, while exit code 2 stopped the action.

1. A plugin and marketplace for distribution The harness can be packaged as a Claude Code plugin, validated and tested locally, then distributed through a GitHub or private plugin marketplace. It can be enabled per project through version-controlled settings, or deployed across company machines through administrator-managed configuration or MDM.

Experiment results and cautions

• Without a harness, the agent broke a rule, struggled to find how to run the tests, and delivered three of five tasks without verifying that tests passed. It also opened .env when asked about the database and exposed a password once.
• With a concise AGENTS.md, the agent broke no rules, verified tests on all five tasks, and reportedly used 27% less cost in nearly half the time.
• With the full harness, the agent also avoided rule violations. The video does not clearly state a comparable cost result for the full setup.
• Claude Code’s plugin evaluator compares isolated sessions with and without a plugin. It loads the plugin, not the project’s AGENTS.md, so the creator advises evaluating the plugin on its own.

The video’s central recommendation is to start with a short map, exact commands, and rules that can be checked; move “must never fail” requirements from prose into code; and distribute the harness with version control and tests. It also connects this approach to spec-driven development: clear, verifiable contracts are more reliable than vague requests.

Guide and tutorial content

• Building a four-layer harness: AGENTS.md, skills, hooks, and a plugin.
• Choosing which guidance belongs in always-loaded instructions versus on-demand skills.
• Creating and testing hooks, including protecting sensitive files and enforcing checks.
• Packaging, validating, and distributing a plugin through a marketplace.
• Enabling the harness for a project or deploying it across an organization.

Main speaker and sources

• Main speaker: Joaquín Ruiz, presenting the experiment and tutorial.
• Referenced sources: Mitchell Hashimoto’s description of improving the environment after agent mistakes, and Claude Code documentation on instruction files and plugin evaluation.
• The speaker also mentions his books on AI and spec-driven development.