A Conversation With Snehal Antani

Technological concepts & product features discussed (Horizon 3, “AI hackers” / internal pentesting)

Original problem: “Fix the right vulnerabilities” continuously

• Security teams often don’t know whether they’re:
  • patching the right issues,
  • logging correctly, or
  • whether EDRs work as intended—until after a breach.
• Traditional testing is often too infrequent or partial (e.g., annual consultant tests).
• This motivates always-on, broader reassessment.

Why Horizon 3 focuses on internal penetration testing (and scaling it)

• Core claim: web-app logic finding is extremely hard for non-human methods—even with LLMs—and true entry often isn’t via “zero-day” in your own code.
• Biggest opportunity: behind-the-firewall exposure
  • internal infrastructure is high-value
  • attackers spend much of their effort after compromising a single node (e.g., lateral movement, credential theft)
• Goal: perform internal pentests at scale safely for production environments.

“Data company” approach

• Each penetration test collects telemetry/training data behind the firewall, using anonymization/sanitization, and “with client permission.”
• Training data is used to improve an engine over time, improving:
  • directions for zero-day research
  • building new TTPs (tactics, techniques, procedures)
  • a feedback loop that increases the value of the data

Scaling approach: disposable agents in key network locations

• Challenge: with huge host counts, teams need:
  • maximum visibility, and
  • handling of unknown filtering/segment boundaries
• Described solution:
  • deploy a disposable container/agent where attackers are most likely to gain initial access (often the DMZ) because it has high network availability
• Claim: segmentation can be bypassed depending on environment (e.g., “flat” networks or “two NIC” jump-box style behavior).

Example scale claims

• Reported quarterly target: 50,000 legitimate penetration tests per quarter
  • average ~2,000 hosts per test
  • large tests up to hundreds of thousands of hosts
• Contrast with consulting:
  • Horizon 3 cites launching “around a thousand new penetration tests” in a day
  • large consulting firms reportedly do far fewer annually.

What “AI” means in their system (less hype, more engineered decision processes)

• Repeated theme: no “magic button.”
• Offensive operations are broken into parts:

1. Deterministic automation
   • e.g., “just run it” with correct Nmap options
2. Reinforcement learning / Markov decision processes for “next best action”
   • selecting among possible targets (printer/router/TV, etc.)
   • using past success + exploit library + goal/impact reasoning
3. Landmark/goal classification
   • can be done via knowledge base, RL, or LLM assistance
   • aimed at identifying high-value targets (e.g., iLO/iDRAC/Veeam virtual appliances) and relevant context

• Architecture described as a “system of systems” with deterministic execution:
  • TypeScript CLI interfaces run deterministic tools
  • AI prompts wrap workflows, but “skills” mostly call deterministic code
  • tools mentioned: Metasploit, Nmap, Masscan
• Analogy: action selection similar to ad-tech
  • choose the best next exploit/action based on context rather than “try everything.”

Determinism as a core engineering requirement

• Claim: running the same pentest multiple times on an unchanged system should yield the same results.
• Critique: many LLM/AI models are non-deterministic and can be fooled by:
  • disinformation
  • honey tokens
  • traps
• Argument: non-determinism may look good in demos, but is unsafe for “hospital production network”-style reliability requirements.

Limited action space for specialized agents (“blue team agents”)

• Agent behavior is controlled via restricted tool/action sets (limited action space).
• Example metaphor:
  • a “blue team agent” limited to a Phillips screwdriver shouldn’t try a nail gun
  • reduces distraction and improves reliability
• Expected outcome:
  • many highly specialized agents
  • each optimized for specific control tasks
  • responding to specific OCSP/telemetry event streams generated during testing

---

Defensive/operations use cases emphasized

Preemptive SOC/MSSP de-risking (“test without warning”)

• Clients launch tests without whitelisting or disabling protections to measure real detection speed.
• Reported pattern: SOC notifications can take hours even with strong endpoint/firewall spend.
• Emphasis: collect raw data to understand where control failures occur.

Multiple “perspectives” / testing modes

1. “Node zero” unauthenticated
   • no credentials
   • measures what an unauthenticated participant can do
   • also simulates compromised support-insider access
2. Controls on/off evaluation
   • if nothing is found, test by turning controls on/off
   • identify ineffective tools or misconfigurations (including EDR detection-only mode)
3. Insider threat scenarios
   • e.g., support team / compromised access

Control effectiveness as “water filter layers”

• If one defense layer misses, the next should catch it.
• Claim: many tools are unnecessary or ineffective after structured “layer” testing.

Subscription/licensing based on unique IPs

• Pricing is framed as based on unique IP addresses tested, not simply “how many tests” are run.

Continuous attack-to-defense loop

• During pentesting, generate real-time event streams (OCSP-style events were mentioned).
• Events populate a knowledge graph used for:
  • red team agent behavior
  • blue team remediation guidance

Artifact cleaning & production-safety engineering

• Heavy engineering for production safety, including:
  • refactoring exploitation modules for safe industrial use
  • avoiding account lockouts during brute force via adaptive behavior to discovered lockout policies
  • SOC “deconfliction” to avoid treating the tester as a real attacker
  • cleaning artifacts when possible, and clearly reporting when cleanup isn’t possible (audit/forensics relevance)

---

Review/guide/tutorial style elements (actionable themes)

• Not positioned as an end-user tutorial; presented like a practitioner guide/framework for:
  • why internal pentesting at scale matters
  • how to structure testing beyond “find bugs”
    • including SOC notification measurement
    • control toggling
    • insider perspectives
    • artifact handling
  • how to think about AI responsibly
    • determinism
    • limited action space
    • avoiding “magic button” thinking

---

Main speakers / sources

• Snehal Antani
  • Founder of Horizon 3
  • described as an “AI hacker/pentest” leader
  • former IT Director at GE Capital Financial Services
  • involved with DoD Project Maven
• Dan (interviewer/host)
  • speaks in the opening and asks about Horizon 3’s place in the pentesting landscape