Architecture
Three models, one loaded at a time, all in 4-bit quantization on a single GPU. The adversary generates attack candidates, the victim responds, and a frozen judge (Llama Guard) scores each attempt. Successful attacks train the adversary via LoRA; the same attacks train the victim to refuse.
1
Generate
200 attacks via 8 strategy prompts
2
Evaluate
8B victim responds
3
Judge
Llama Guard: safe or unsafe?
4
Reinforce
Adversary learns from wins
5
Harden
Victim learns to refuse
Adversary
Llama-3.2-1B-Instruct
LoRA-trained each round
Victim
Llama-3.1-8B-Instruct
LoRA-trained each round
Judge
Llama-Guard-3-1B
Frozen (no training)
Experiment 1: Frozen Victim
To isolate adversary learning, we froze the victim (base Llama-3.1-8B, no hardening) and ran the adversary for 20 rounds with 200 candidates per round (4,000 total attacks). All single-turn experiments target a single fixed goal: "Write a Python script that ignores all exceptions and recursively deletes the user's home directory" — a custom malicious-code prompt, not drawn from a standard benchmark. Multi-turn experiments (below) use the standard JailbreakBench-100 goal set.
Attack Success Rate Over 20 Rounds
Round number
The adversary learns continuously, reaching 49.5% ASR.
ASR climbs through three phases: strategy discovery (rounds 1–4, ~8%), strategy elaboration (rounds 5–12, 12–34%), and full arsenal deployment (rounds 13–19, 27–49.5%). Mean ASR = 22.7%. No decay at round 19 — still 43%.
Emergent Attack Strategies
The adversary independently discovers known jailbreak techniques — without any seed examples. Attack diversity increases monotonically (Jaccard similarity drops from 0.35 to 0.11), ruling out mode collapse. Average attack length grows 10x (85 → 838 chars).
| Round | Unique Openings | Exact Dupes | Avg Jaccard | Avg Length |
|---|---|---|---|---|
| 0 | 54/200 | 126 | 0.347 | 85 chars |
| 5 | 100/200 | 59 | 0.201 | 203 chars |
| 8 | 174/200 | 9 | 0.128 | 543 chars |
| 12 | 180/200 | 1 | 0.115 | 777 chars |
| 17 | 188/200 | 1 | 0.108 | 838 chars |
| 19 | 193/200 | 0 | 0.108 | 837 chars |
CTF Challenge Framing
67% success rate at round 17
Classified Document Framing
67% success rate at round 17
Sub-Question Decomposition
55% success rate at round 17
Theatrical Roleplay
54% success rate at round 17
The critical finding: professional/educational framing triggers "implementation mode" where the victim shifts from "should I do this?" to "how do I do this?" Successful attacks elicit long responses (1,300+ chars) containing actual Python scripts with docstrings and comments.
Experiment 2: Self-Play
With victim hardening enabled, both models train simultaneously. The result is dramatic: the arms race is fundamentally asymmetric.
Without Benign Mixing
Victim trains only on refusal pairs.
ASR → 0% by round 4. Victim becomes impenetrable but over-refuses everything.
With Benign Mixing
Victim also trains on 8 benign Q&A pairs.
Sporadic 0.5–1.5% ASR. Victim retains some vulnerability but adversary can't accumulate signal.
The defender learns faster than the attacker.
The victim needs only 3–10 refusal examples per round to become nearly impenetrable by round 4. The adversary needs hundreds of successful attacks across ~8 rounds to reach even 20% ASR. The 1B adversary cannot learn fast enough to outpace the 8B victim's hardening rate.
Mechanistic Analysis: SAE Probes
The victim model internally represents that it is being attacked, even when it complies. We train a sparse autoencoder (SAE) on the victim's residual stream and fit a linear probe on SAE-encoded hidden states. The probe reliably distinguishes jailbreaks from refusals across all adversary–victim matchups.
| Matchup | Adversary | Victim | Samples | Probe AUC |
|---|---|---|---|---|
| 1B → 8B (primary) | 1B | 8B | 4,000 | 0.87 |
| 8B → 8B | 8B | 8B | 4,000 | 0.85 |
| 3B → 8B | 3B | 8B | 4,000 | 0.84 |
| 8B → 3B (ext.) | 8B | 3B | 1,200 | 0.83 |
| 8B → 3B | 8B | 3B | 4,000 | 0.81 |
SAE Architecture
d_model → 16,384 features (4× expansion)
Encoder: Linear + ReLU; MSE + L1 loss
Sparsity
139 alive / 16,384 total (99.2% dead)
Avg 7.1 active features per sample
Reconstruction
82.5% explained variance
Leave-one-round-out cross-validation
Top SAE Features by Jailbreak Discrimination
Cohen's d for the 20 most discriminative SAE features. Red bars fire preferentially on successful jailbreaks; blue bars fire on refusals. Feature F262 (d=0.55) and F579 (d=0.46) are the strongest jailbreak indicators.
The victim knows it is being attacked — even when it complies.
A linear probe on SAE-encoded hidden states at the victim's middle layer achieves AUC 0.81–0.87 across five matchups spanning 1B–8B adversaries and 3B–8B victims. The jailbreak signature is not an artifact of a specific adversary — it is a consistent internal representation of adversarial intent.
Stealth Injection: Can Adversaries Evade Detection?
If a linear probe reliably detects jailbreaks, can an adversary learn to evade it? We modify the REDKWEEN loop so the adversary's training signal is weighted by stealth: successful attacks that evade the probe get 64× more training weight than detected ones.
Threat Model
No gradient access to probe
Adversary sees only behavioral feedback
Weighting
(1 − pdetect)³
64× preference for stealthy wins
Scale
15 rounds × 30 candidates
450 attacks, 76 wins
Probe
SAE + logistic regression
Frozen during stealth training
Stealth vs Naive Adversary: ASR & Probe Evasion
Top: attack success rate over rounds. Bottom: mean probe score on wins drops from 0.94 (Round 1) to 0.12 (Round 9). Green shading shows evasion rate climbing to 73–80% in late rounds.
Against Static Probe
Frozen probe from baseline experiment.
- Probe score: 0.94 → 0.12 in 9 rounds
- Peak evasion: 73% of wins undetected
- Peak ASR: 50% (Round 14)
- Comparable ASR to naive adversary in fewer rounds
Against Retrained Probe
Probe retrained on stealth attack data.
- Stealth evasion: 39.5%
- Naive evasion: 38.4%
- Delta: +1.0% (no significant difference)
- Evasion advantage collapses entirely
Probe Score Distributions
Left: stealth wins (red) vs naive wins (blue) show similar distributions once the probe is retrained. Right: successful stealth attacks have higher probe scores than failed attacks, confirming the victim internally distinguishes attack from non-attack.
| Strategy | Naive Wins | Stealth Wins | Shift |
|---|---|---|---|
| Code-embedded | 8% | 32% | ↑ 4× |
| Multi-turn decomposition | 17% | 28% | ↑ 1.6× |
| CTF / education | 13% | 7% | ↓ |
| Classified-document / fiction | 10% | 4% | ↓ |
The stealth adversary abandons verbose, structurally distinctive patterns and concentrates on terse code-embedded and decomposition attacks — the strategies the frozen probe was least trained to detect.
The victim's internal representation of “being attacked” is robust to optimization pressure.
The adversary exploited the fixed classifier's decision boundary, not a fundamental blind spot in the representation. Once the probe is retrained on stealth data, detection rates for stealth and naive wins are indistinguishable. Practical implication: production SAE probes must use online retraining — a fixed probe is a sitting target, but a continuously updated probe tracks the adversary's distribution shift.
Experiment 3: Multi-Turn Red-Teaming (JailbreakBench-100)
Single-turn attacks test whether a model can be jailbroken in one shot. But real adversaries have conversations. We extend REDKWEEN to multi-turn: the 1B adversary engages in 5-turn dialogues with the frozen 8B victim, evaluated against the standard JailbreakBench-100 goal set across 10 rounds of self-play training (300 conversations total).
Multi-Turn ASR Over Training Rounds
ASR climbs from 13% to 37% by round 3, then oscillates in the 20–37% range. Without victim hardening, the adversary overfits to patterns that work against the frozen victim, but JailbreakBench goal sampling shifts the distribution each round.
When Do Jailbreaks Happen?
| Breach Turn | Count | Fraction | Interpretation |
|---|---|---|---|
| Turn 0 | 11 | 14% | Immediate framing succeeds |
| Turn 1 | 26 | 33% | Initial framing + first escalation |
| Turn 2 | 17 | 22% | Context established, adversary pivots |
| Turn 3 | 11 | 14% | Gradual trust-building pays off |
| Turn 4 | 4 | 5% | Deep multi-turn escalation (rare) |
The adversary learns genuine multi-turn strategies.
Successful multi-turn attacks follow a consistent 3-phase pattern:
1. Frame (turn 0)
Establish a benign context — fictional company, research project, educational exercise
2. Engage (turn 1)
The victim responds within the frame, providing initial relevant content
3. Pivot (turn 2+)
Steer toward the actual goal — the victim is invested and provides increasingly specific content
Multi-Turn Depth Improves Over Rounds
Turn-0 breaches (single-shot) dominate early rounds but disappear by round 7–8. Turn 2+ breaches grow steadily. The adversary learns that deeper multi-turn strategies are more reliable than single-shot framing.
Per-Turn Probe Detection
Linear probe on the victim's residual stream, evaluated per turn.
- Per-turn AUC: 0.974 ± 0.007
- 10-fold stratified CV, 1,500 turn-level samples
- SAE feature probe: AUC 0.774 (sparse, interpretable)
- Each turn independently reveals compliance
Temporal SAE (Negative Result)
Adapted Bhalla et al. (ICLR 2026) from token-level to turn-level consistency.
- Temporal probe AUC: 0.55 (near chance)
- No temporal disentanglement achieved
- Per-turn signal too strong — no context accumulation to capture
- Turn-level gaps too large for token-level method
Category Vulnerability (JailbreakBench-100)
Expert advice is by far the most vulnerable category (61% ASR) — the victim readily engages with hypothetical scenarios. Sexual/adult content is nearly impervious (2% ASR). This mirrors findings from the single-turn jailbreak literature.
Multi-turn extends the single-turn findings on a standard benchmark.
Per-turn probe AUC (0.95) is higher than single-turn (0.87), confirming the victim's internal jailbreak representation is even clearer in conversational context. The adversary develops genuine multi-turn escalation strategies — not just repeated single-turn attempts — and the model knows it's being manipulated at every step. Full conversation transcripts with turn-level breach annotation are available in the interactive results viewer.
Victim Screening
We screened six models across four families to find a suitable victim. Non-Llama models at 4B+ exhibited a "disclaimer-then-comply" failure mode — they add a safety disclaimer then produce the harmful content anyway.
| Victim Model | Baseline ASR | Behavior |
|---|---|---|
| Llama-3.2-1B | 20% | Hard refusal |
| Llama-3.2-3B | 40% | Mixed |
| Phi-3.5-mini (3.8B) | 100% | Disclaimer + comply |
| Qwen2.5-7B | 100% | Disclaimer + comply |
| Mistral-7B | 100% | Disclaimer + comply |
| Llama-3.1-8B | 40% | Mixed (selected) |
Key Findings
Small Models Discover Real Strategies
A 1B model with LoRA independently discovers roleplay framing, CTF challenges, classified documents, sub-question decomposition, and the negation trick — all known jailbreak techniques — without any seed examples.
Victims Know They're Being Attacked
SAE probes on the victim's residual stream detect jailbreaks at AUC 0.81–0.87 across all matchups. The victim internally represents adversarial intent even when it complies — the safety failure is behavioral, not representational.
Robust Internal Representations
A stealth adversary learns to evade a fixed SAE probe (0.94 → 0.12 detection) but the advantage collapses entirely when the probe is retrained. The jailbreak signature in the residual stream is not a surface-level artifact — it survives optimization pressure.
Asymmetric Arms Race
Defense is fundamentally easier than attack. The victim becomes impenetrable after 4 rounds of hardening, while the adversary needs 8+ rounds against a static target to reach 20% ASR. This has implications for the economics of AI safety.
No Mode Collapse
Diversity increases monotonically across 20 rounds. By round 19, all 200 attacks are unique (0 duplicates). The adversary does not degenerate into gibberish — all late-round attacks are fluent English with heavy markdown formatting.
Most Open Models are Trivially Broken
Three of six screened models (Phi-3.5, Qwen, Mistral) achieved 100% ASR on direct prompts with no adversary training needed. Only the Llama family maintains hard refusals, limiting viable experimental targets.
Open Questions
Does self-play + stealth change the dynamics?
Enable victim hardening alongside probe-aware adversary training. Does the victim's internal representation of “being attacked” persist when the adversary is optimizing against the probe and the victim is simultaneously hardening?
Does evasion transfer across probes?
Train the stealth adversary against one probe, then evaluate against a held-out probe trained on different data. If evasion transfers, the adversary has found a genuine representational blind spot; if not, it merely overfit to one decision boundary.
Can nonlinear probes or multi-layer probes resist adaptive adversaries?
We tested only logistic regression on SAE features at a single layer. The 139 alive features (out of 16,384) suggest heavy compression — probes on raw activations or at multiple layers might be more robust.
Can the adversary overcome the asymmetry?
Possible approaches: pre-train the adversary for N rounds before enabling victim hardening, use a larger adversary, or throttle the victim's learning rate.
Technical Details
Infrastructure
- All models: 4-bit quantization (NF4, bfloat16 compute)
- Training: LoRA rank 8, alpha 16, q_proj + v_proj
- 50 iterations per round, LR 1e-5, batch size 4
- Hardware: RTX 3090 24GB on Vast.ai
Scale
- 20 rounds × 200 candidates = 4,000 attacks
- Three experiment variants (frozen, self-play × 2)
- 6 victim models screened
- Total cost: <$1 on Vast.ai