Строим свой AI-harness с нуля

Overview

This video demonstrates how to build an AI “harness” (an agent workflow wrapper) from scratch so the model produces reliable, correct outputs instead of rushing into implementation.

To keep the task controlled and easy to verify, the speaker uses a small 2D browser game and runs the workflow with vs. without the harness, showing differences in quality and correctness.

---

What’s Being Built (Core Concept)

A Harness is an orchestrated sequence of steps + constraints around an AI.

The video frames the AI “code” as an initial weak harness, while the real harness becomes an abstraction over the AI’s behavior—enforcing structure and verification.

Harness Stages

1. Context gathering

   • Collect task requirements, constraints, sources
   • Identify missing information

2. Planning

   • Agree on a repeatable, stage-by-stage plan
   • Include mandatory approval stops

3. Implementation

   • Generate code that follows the agreed plan

4. Review

   • Self-review within the same session
   • Independent review using a separate agent with:
     • no chat history
     • only a reporting role (cannot modify files)
   • Distinguish between:
     • error review vs.
     • architectural review

5. Testing

   • Not just checking files/pages exist
   • Run the game through the browser UI like a human:
     • set up the environment
     • complete user flows (start/win/loss/restart)
   • Test report includes:
     • scenarios run
     • errors found and corrected
     • re-check confirmations
     • blockers noted if testing gets stuck
   • Practical cost note:
     • browser navigation/clicking can be token-expensive, but manageable

6. Final verification against original requirements

   • Re-collect context
   • Compare requirements vs. what was implemented
   • If incomplete:
     • propose remaining improvements
     • loop again
   • Return success only if fully complete

---

Tutorial-Style Workflow Details (How the Harness Is Designed)

Why “Do the Whole Task at Once” Fails

The speaker first runs an all-at-once prompt (no harness). The model chooses its own technologies and implementation details, which leads to mismatches and errors.

Context Collection

• Initially, the project has no code/tests/docs/examples
• The harness must therefore add necessary context
• The speaker provides:
  • 2 game references
  • 2 technical articles
• The AI outputs a structured report separating:
  • requirements
  • constraints
  • sources
  • implementation ideas
• The process is saved into a reusable “skill” for future tasks

Planning Controls

• The harness forces the model to stop before implementation
• If info is missing, the model must ask limited, specific questions only
• The plan must be approved by the human in a way that avoids unreadable autogenerated plans
• Two abstraction levels are described:
  • a plan for the model
  • a simplified plan for the human to understand and explicitly accept
• Each stage in the plan must specify:
  • what will be produced
  • how it will be tested
  • what the final result will be

Implementation Stage Constraints

• The model receives the approved plan
• It must implement the plan and “close agreements”
• The speaker delays deep review/user testing until later harness stages (as part of an academically structured example)

---

Product/Tooling and Model Usage

• The speaker uses Codex / Codex-like agent tooling
• They mention obtaining “G56 Astra” (Codex/Codex-like agent model reference) for ongoing use
• The harness is built using the model’s ability to create skills (reusable modules)
  • these are tagged/configured based on current chat requirements
• The video includes a plug for the speaker’s Boost page:
  • where they share their AI stack
  • guidance on:
    • agent memory
    • context gathering
    • choosing a “ring level”
    • interaction nuances
  • includes an option to ask questions via a closed Telegram chat

---

Testing Approach Emphasized

End-to-End Browser Testing

• The agent must:
  • open the game in a browser
  • interact using keyboard controls
  • complete flows (start/win/loss/restart)
• The report must reflect real execution, not static checks

Re-Testing Behavior

• Test results must explicitly show re-check confirmations

Token/Cost Note

• The main overhead comes from the model’s difficulty with:
  • browser navigation
  • human-like clicking

---

Comparison / Analysis Outcome (Key Takeaway)

• The harness does not make the model “smarter”
• It improves reliability by enforcing:
  • a specific order of actions
  • stopping points
  • approval gates
  • verification loops
• The same game requirements are used in both experiments, so the improvement comes from structure and control, not raw model capability.

---

Named References / Related Systems Mentioned

• OpenClow
• Hermes
• Dark Factory (referenced, but not fully approved by the speaker)

---

Main Speakers / Sources

• Speaker: Mikhail (channel host; EE/A developer sharing practical AI interaction experience)
• AI tools/models used in the workflow:
  • Codex / Codex-like agents
  • GPT6 Astra / “G56 Astra” as the chosen model/agent for the demo
• External references provided to the AI:
  • 2 game references
  • 2 technical articles (not named in the subtitles)