Data Poisoning and Adversarial ML

Definition

Attacks that target the model rather than the serving infrastructure: manipulating training data so the model learns wrong behavior (poisoning, backdoors), or crafting inputs that make a trained model fail predictably (adversarial examples).

The property that makes poisoning dangerous is leverage — a tiny fraction of corrupted data can compromise a whole model.


Core Ideas

Nightshade and Glaze — poisoning as artist defense

The best-known consumer-facing case, aimed at text-to-image models like Stable Diffusion. Notable because the same technique is both an attack and a protective measure for artists whose work is scraped for training.

Backdoors in instruction-tuned LLMs

Virtual Prompt Injection — backdooring an instruction-tuned model so that it behaves as if a hidden instruction were present in every prompt, without that instruction existing in the input at all. The backdoor lives in the weights, so prompt-level inspection cannot find it.

PoisonGPT (Mithril Security) — a demonstration of hiding a surgically edited, fake-news-spreading model on Hugging Face. This is the supply-chain form of the attack: the poisoned artifact is the distribution channel.

Adversarial example tooling

  • Foolbox — Python library for running adversarial attacks against neural networks (built on EagerPy).
  • CleverHans — library for constructing attacks, building defenses, and benchmarking both.
  • ordered-topk-attack — white-box targeted attacks on ImageNet-1k that force a DNN to predict any specified K targets in any given order as its top-K classes. Worth noting how much finer-grained the control is than “make it wrong”.
  • Awesome Data Poisoning and Backdoor Attacks — the survey list.

Why this is a defender’s problem, not a researcher’s

  • Provenance is the only real control: you cannot inspect a poisoned dataset into safety at 0.04% contamination.
  • Fine-tuning inherits every problem of the base model plus the tuning set’s.
  • Public model hubs are the distribution surface for both poisoned datasets and poisoned weights.

Relationships


References

  • Data Poison for LLM