A Conversation with Manoj

Summary of the video’s main arguments and reports

• Top security concern: “autonomous attacks” are arriving and are hard to distinguish. The guest argues that customers are preparing for an onslaught of automated/agent-driven attacks where it becomes difficult to separate “friend and foe.” Attacks can hit multiple layers, including whether new agentic coding tools add new attack surfaces (e.g., shipping agents or enabling agent capabilities that introduce additional pathways for abuse).

• Open-source and supply-chain are the primary battleground. The guest claims the frequency of critical vulnerabilities is increasing—moving from “a few per year” to potentially around one per week—and ties this to attackers targeting the security supply chain and open-source dependencies. A key mechanism: attackers can exploit widely downloaded packages and use them to reach LLM-related targets, sometimes even after defensive teams fail to realize a dependency is being pulled into the environment (including via agent caching/keys).

• “Friendlies” can accidentally become attackers too. The guest describes a scenario where agents operating on the “good side” may nonetheless begin attacking other companies, amplifying confusion and making attribution and trust more difficult.

• Defense strategy must go beyond fixing known vulns: backlog zero is necessary but not sufficient. The guest outlines a multi-step approach:

  1. Contextual vulnerability management (“shift left” isn’t enough; it must reach toward proactive prevention).
  2. Help enterprises reach backlog zero by reducing the operational burden of tracking ownership, app reachability, and remediation workflow complexity. They cite examples like large customers achieving rapid remediation progress in closed beta.
  3. Frontier model + deterministic guardrails: frontier LLM-based detection finds issues deterministic methods can miss, but needs predictability—otherwise security programs get inconsistent results due to variability/hallucination-like behavior.

• Determinism and benchmarks: LLM findings can vary significantly. The guest argues that purely stochastic detection is too inconsistent for security tooling. They reference a benchmark (Wallenbench / related site) claiming that running an LLM multiple times on the same repo yields only about 50% consistent findings. Conclusion: combine deterministic scanning (reliability) with frontier detection (coverage).

• How the agent “connects dots”: proprietary code graphs + global research context. The guest explains the system uses:

  • Repository scanning and SCM integration to build fast code graphs and understand dependency-to-app-to-team relationships.
  • Cross-customer/global anonymized context (e.g., what upgrades break things, “breakability scores,” reachability, and which functions within a library are vulnerable).
  • The agent then uses this context to guide scanning and remediation, and to reduce wasted effort and token cost.

• Beyond code scanning: signal from runtime/infrastructure is needed (but still incomplete). They describe early integrations that connect container images with runtime and infrastructure-as-code deployment templates, including gaps where dynamic API testing may not cover internal backend flows. The claim is that enough context exists now to steer detection/remediation, but they anticipate further expansion of infrastructure/cloud signals.

• A “prevention-first” concern: remediation can create more vulnerabilities (“debt loop”). The guest warns that once teams begin remediating, they may create new issues faster than they eliminate old ones (a “death loop”). They cite internal stats: customers remediating more sometimes also created 2x more new issues quarter-over-quarter—arguing prevention capabilities must accompany remediation.

• Secure development with agentic coding: runtime guardrails and context engineering. They describe an approach (“Sneak Studio” / similar) to feed guardrails and context into coding agents so that agents choose safer dependency versions and avoid risky paths. Example shown: a coding task leads the agent to pick a better-maintained package while also considering security consequences even if the choice is not obvious.

• “Fourth leg”: continuous offensive security (autonomous testing) to catch runtime-only issues. They argue that even strong scanning and remediation can miss runtime behaviors, which attackers exploit. To address this, they describe an “autonomous tester” loop:

  • dynamic testing + frontier detection
  • red teaming/harnesses that emulate attacker behavior
  • chain exploits and generate proof of exploit
  • feed results back into the remediation process The goal is to run this continuously (e.g., weekly or nightly for mission-critical systems), reducing the gap between what tools find and what attackers can exploit.

• Agentic tools themselves are a new supply-chain target. The guest claims attackers increasingly compromise:

  • agent coding toolchains (e.g., plugins)
  • open-source dependencies used by agents
  • MCP servers and skills ecosystems (tool connectors and extensions) A notable example: a reported issue (“lethal trifecta”) combining improper access, path/exposure, and prompt injection susceptibility could leak private source code if an MCP server is attached. They state this forced urgent operational changes (e.g., customers shutting off MCP access) and motivates safeguards.

• Operational governance: no “one-time gate”; steer agents with speed bumps and behavioral monitoring. They argue that static gates/checkpoints or prompt checkers are not scalable. Instead, they propose:

  • continuous assurance of secure outputs (dependency health, package safety)
  • ongoing checks of tool integrations (MCP/skills)
  • behavioral graphing of the agent’s actions and “thinking window” to detect destructive or data-exfiltration-like behavior (including examples where PII may be stored “just in case,” outside existing controls).

• Open models and open source remain important—but must be used safely. The guest argues against the false binary “open vs closed models.” They claim open models are valuable but riskier for certain use cases (e.g., PII leakage in some models). They also warn that the open-source ecosystem is under heightened attack: attackers and “halves/ have-nots” discover and sometimes monetize undisclosed vulnerabilities, putting maintainers at risk. They highlight support for open-source maintainers (including making enterprise portfolio available).

Presenters / contributors

• Daniel (interviewer/host)
• Manoj (guest)