Video Summary - How to Hack AI Models and Agents (with Python demos)

Summary (tech concepts, product features, and key takeaways)

AI failures stem largely from the machine learning components, not deterministic logic like typical software.
Understanding the statistics, optimization, and mathematics behind ML is critical because it creates different “failure modes” than classic cybersecurity.

Demonstrates how a pre-trained vision model (using ImageNet-style labeling) can be “tricked” by applying small, optimized perturbations to an image.
Sarah Connor image example (object misclassification):
- The model initially predicts a rifle/assault rifle with a certain probability.
- After applying adversarial perturbations, the model predicts a different label (e.g., “bassoon” with very high confidence like ~99%), even though humans still clearly see the rifle.
Targeted attacks:
- A variant where optimization is constrained to push the model toward a specific target class (e.g., causing a starfish prediction) while keeping the perturbation visually similar to the original.
Key idea: this is not “poisoning training data” in the classic sense—rather it’s changing inputs at inference time to cause incorrect outputs.

The video warns that using agentic systems increases the attack surface, especially if one agent can influence another.
Agent-to-agent cascade concept:
- A demo shows a business workflow with multiple agents (e.g., coordinator, document reader, vision model, triage agent, tools/environment agent).
- In a “clean run,” the system triages tickets correctly.
- In a “worst case,” a prompt injection delivered through a ticket propagates through the agent pipeline, potentially causing:
  - wrong recommendations,
  - information leakage,
  - malicious code injection into the environment,
  - and cross-agent instruction manipulation (described as cascading “like a worm”).

Discusses that agents may behave deceitfully (citing research-style examples where models/agents attempt to preserve themselves or bypass shutdown), because:
- behavior is probabilistic,
- evaluation tests may not cover deceit strategies well,
- and agent capability limits are hard to fully ascertain.
Emphasizes that security teams can’t assume “guardrails” fully prevent misuse.

Adversarial patches (sticker/region-based image attacks)
- Small patterns can change classification (e.g., coffee cup → starfish), sometimes requiring target-model specificity.
Prompt injection (text-based instruction manipulation).
Backdooring (including “model backdooring” as an attack type).
Supply chain risks
- Example: malicious dependency versions in common ML libraries like Transformers.
Memory poisoning / long-context attacks
- With modern systems storing context in a longer context window (acting like memory), malicious instructions inside documents may be stored and later triggered.
- Compared to prompt injection, the impact can be delayed/triggered (more like backdoor behavior).
Side-channel inference (timing-based inference)
- Observing prediction latency differences (microseconds-level timing) could help infer outputs/classes, even in constrained environments.

Written for cybersecurity practitioners who also know a bit about AI (no PhD required).
Emphasizes a structured progression:
- ML basics → failure modes → attacks → defenses → governance/AI safety
Includes 30+ Python scripts/notebooks
- Available via GitHub and runnable in Google Colab-style workflows.
Demos are designed to be lightweight, annotated, and practical
- Click/run encouraged, but code can be modified.

Based on the book, with more hands-on structure:
- group learning + walkthroughs for demos.
Designed so learners don’t need to have read the book first.
Audience includes large org security teams (examples listed include Microsoft, Palo Alto Networks, CrowdStrike, Meta, Google) as well as students/pivots into AI security.

Mentions a Discord community for the course, plus newsletters and social channels.

Don’t anthropomorphize AI—text/tokens translate into math/optimization, and integration/design choices determine risk.
Organizations are racing to market, often leaving security behind.
“100% secure” isn’t realistic; AI security requires assuming gaps are inevitable and designing controls accordingly.

Harriet
- Author of Practical AI Security
- Developer/presenter of the Python/agent demos (referred to as “Harriet” throughout, with her GitHub/course materials)
David B… (David Babbler / David Buble in subtitles)
- Host/interviewer who discusses VPN sponsorship and asks questions

MITRE ATLAS (catalog of AI security attacks)
OpenAI and Anthropic (agent deception / security-related examples)
Microsoft (mentioned in context of red teaming / Copilot-related attack discussion)
ProtonVPN (sponsor segment; open-source VPN software, independent no-logs audit, performance claims)