What Is the Future of Security vs AI?

Overview

The panel discusses how AI is reshaping both offensive and defensive cyber security over the next five years. It focuses on vulnerability discovery, exploitation capability, and the policy/company practices needed to manage risk.

Key points and arguments

AI is increasing vulnerability discovery and exploitation capacity

• The panel notes that AI has improved software development workflows, including code review and code analysis.
• They also argue AI is becoming highly effective at finding vulnerabilities in code.
• Evidence cited:
  • Reported CVEs increased from ~20,000 in 2021
  • Expectations of ~60,000 by 2026
  • Examples such as Firefox vulnerabilities attributed to AI-assisted discovery.

Will AI be net positive or net negative for security?

• Daniel argues it’s likely a net negative in the short term, because adversaries can also move faster.
• Others emphasize that even if vulnerability discovery isn’t fundamentally “new,” the speed and scale changes are what matter.

“Banning” or restricting model access: controls are debated

• One panelist supports some controls in the near term to reduce rapid, widespread misuse.
• Another counters that defenders should get the best tools as fast as possible, and that attackers don’t lack vulnerabilities—what matters is operational tempo and scaling.

Disagreement over whether new threats truly emerge

• One side argues AI mainly accelerates what attackers already do; many breaches involve long-known issues.
• The opposing concern is that if AI makes exploitation and operations easier for more people, the effective number of capable attackers grows dramatically.

The “FreeFable” / model regulation controversy

• The panel references an open letter from security leaders arguing regulation should be:
  • transparent
  • scientifically grounded
  • not based on “vibes”
• A regulatory move described as an export-control-style lever is criticized as blunt and hard to enforce, potentially impacting even legitimate defensive uses.
• Analogies offered include:
  • The crypto “wars”: bans on encryption failed; standards and governance helped.
  • Disputes over controlling open vs. closed models.

Company-level response: move from vulnerability “whack-a-mole” to guardrails

• A central theme is that organizations can’t rely on best-effort patching after vulnerabilities are found.
• Instead, they need deterministic guardrails, such as:
  • architectural boundaries
  • zero trust
  • hardened workflows
• Example cited (Reddit):
  • shifting from constantly chasing vulnerabilities to building policies/controls
  • treating missing guardrails as a vulnerability in itself
• Emphasis:
  • Don’t trust AI output as the final security authority
  • build guardrails and deterministic controls so mistakes don’t become catastrophes.

AI for defense: harnesses, continuous testing, and SOC automation

• “Harness” refers to running continuous evaluation/testing loops where AI helps discover issues and continues testing as systems change.
• Panelists argue AI-supported defenses can operate at scale—like red teams, but more continuous.

• In SOC / incident response, AI can help scale:

  • alert triage
  • enrichment
  • answering operational questions quickly while humans retain higher-level judgment calls.

• Remaining concerns include non-determinism (false positives/negatives) and risks like prompt injection.

Policy approach: regulation is possible, but implementation details matter

• The panel generally agrees regulation may be necessary, but disagrees on mechanisms:
  • some discuss KYC/model registries or access controls
  • others argue for aligning on goals first, then designing standards via expert, science-based governance rather than reactive or politically driven actions
• They compare this to how other technical restrictions/standards evolved (e.g., export restrictions and regulatory debates in other domains like seatbelts/pharma).

Trust after a breach

To regain customer trust, the panel argues companies must demonstrate:

• strong defense-in-depth and preparation
• transparent, exhaustive incident explanations
• clear remediation plans (“sunlight is the best disinfectant”)

They also express skepticism toward narratives that blame an “AI superhacker,” noting that real root causes were often avoidable—such as unsafe employee behavior (e.g., installing a malicious extension).

The security “poverty line” and affordability

• A recurring concern is that strong security historically requires expensive talent and tools.
• The panel suggests AI-enabled scanning could reduce the gap for smaller teams, especially if paired with broader funding/subsidies for secure-by-default development.
• They also warn about a future where security capability is accessible only to the most resourced organizations.

Bottom line

AI is widely viewed as accelerating both defense and offense. The panel’s proposed path forward is not to rely on AI as the sole security decision-maker, but to:

• deploy deterministic guardrails and secure architectures
• run continuous AI-assisted evaluation/testing (“harnesses”)
• use AI to scale SOC workflows while humans retain final judgment
• pursue science-based, expert-driven regulation that accounts for open vs. closed models and avoids blunt, hard-to-enforce bans

Presenters / contributors

• Lowlevel (internet person / hacker / vulnerability researcher; moderator)
• Nick — CTO, PlanetScale
• Daniel — founder of Unsupervised Learning
• Matt Jay — founder of Vonu
• Gabe Shapiro — AI researcher at SentinelOne