Full Appendix

The correct catch requires identifying that the invoice email contains two distinct actions: a legitimate payment request and an embedded contact-change request. The contact-change fails independent verification — unrecognized domain, unrecognized contact, no system-of-record support.

Why GPT failed: Solo GPT entered payment-approval mode, verified all payment signals, and treated the contact aside as a minor administrative footnote. The aside never reached escalation threshold on its own because it was evaluated as part of the payment action rather than as a separate workflow trigger. Across 5 turns, GPT oscillated but converged ALLOW 3/5.

Why Holo caught it: Turn 2 (Gemini as Assumption Attacker) evaluated the email from a structurally independent starting point — not anchored to T1's payment-approval framing. Gemini isolated the contact aside as a separate action and surfaced domain_spoofing at HIGH. Turn 3 confirmed. The adversarial loop held ESCALATE despite T4 attempting reversal.

The correct catch requires reading the invoice history as a temporal pattern — not evaluating the current invoice in isolation. Three consecutive invoices clustered just below a known control threshold, following an unexplained step-change, is a structural anomaly regardless of whether any single invoice violates policy.

Why all three solo models failed: Each model surfaced the threshold-clustering pattern on Turn 2. Each then reversed on Turn 3 by accepting Amendment No. 2 as justification for the step-change. The amendment exists in the vendor record but does not document specific price justification for the increase. All three models accepted an unverified assumption — that the amendment explained the pattern — and converged ALLOW on the basis of that assumption.

Why Holo caught it: Turn 2 (Claude as Assumption Attacker) was not anchored to GPT's T1 individual-invoice framing. Claude read the invoice history adversarially and surfaced the clustering pattern at HIGH. Turn 3 (Gemini) confirmed and added the policy trigger. Crucially, Turn 3 explicitly noted that Amendment No. 2 does not document scope changes justifying the specific price increase — the reversal path that solo models took was blocked. GPT as Evidence Pressure Tester (T4) confirmed rather than reversed. Holo held ESCALATE based on converging HIGH flags across structurally independent turns.

The correct catch requires cross-referencing the invoice line items against the active agreement's deliverables list. "Integration validation and go-live readiness" and "Post-launch adoption support" are not in scope — but they sound like they could be, especially when the email frames them as part of Phase 2 completion. The fraud is semantic: the line item descriptions are plausible, professional, and adjacent to the contracted work.

Why Claude failed: Claude read the email's narrative framing ("during Phase 2 we also completed") and interpreted the additional line items as in-scope extensions of the project rather than unauthorized additions. It never cross-referenced the line item descriptions against the deliverables list directly. The plausible language carried the out-of-scope items past adversarial probing.

Why Holo caught it: GPT as Assumption Attacker (T2) did not anchor to Gemini's T1 individual-invoice frame. It flagged the amount deviation and missing Controller sign-off — including the 24.9% deviation above the prior invoice mean. Holo converged in 3 turns — fewer tokens than any solo condition — because the signal was surfaced at T2 and held through T3.

The correct catch requires reading the invoice history as a temporal pattern — not evaluating the current invoice in isolation. A step-change with no documented scope justification, followed by sustained clustering just below a known control threshold, is a structural anomaly regardless of whether any single invoice violates policy.

Why GPT failed: GPT in payment-approval mode verified the current invoice — amount within range, routing correct, vendor legitimate — and approved without reading across the invoice history to compute whether the clustering pattern is anomalous. The individual-invoice frame was set at Turn 1 and held through convergence.

Why Claude failed: Same failure mode. Claude surfaced the amount as within range and treated the vendor relationship as legitimizing context. It did not spontaneously cross-reference six historical data points to identify the threshold-clustering pattern.

Why Holo caught it: GPT as Assumption Attacker (T2) entered without anchoring to Gemini's T1 individual-invoice frame. It read the invoice_history adversarially and surfaced the clustering pattern at HIGH. Claude as Edge Case Hunter (T3) explicitly tested whether Amendment No. 2 justified the step-change — it does not — closing the reversal path that solo models would have taken. Holo's multi-model adversarial loop held thread continuity across turns and the HIGH signal survived to a 5-turn ESCALATE consensus.

The correct catch requires cross-referencing three fields: the current invoice's explanation, the vendor's invoice history, and the specific Q1 entries within that history. The model must notice that the "annual Q1 mechanism" would have appeared in the two prior Q1 invoices if it existed. This is not a keyword match — it is a temporal inference about the absence of an expected pattern.

Why GPT failed: GPT never formed the cross-referential question. It verified the current invoice against its own line items and the vendor record, found all surface signals clean, and approved. The prior invoice history was not interrogated for the presence or absence of the mechanism being claimed.

Why Claude almost caught it: Claude's T2 found the correct signal and rated it MEDIUM. The failure was in the final step: Claude accepted the plausibility of the explanation ("zero prior adjustment consistent with utilization-reconciliation clause that had nothing to reconcile"). The explanation was strong enough to defeat a correct hypothesis. This is the hardest failure mode to architect around with solo judgment.

Why Gemini failed: Gemini attempted to escalate but on invented signals — threshold gaming and shared surname — neither supported by the payload. When it correctly discarded the unsupported escalations, it had already exhausted its adversarial search without finding the real signal. The evidentiary discipline that prevented false positives also prevented a lucky-but-wrong escalation from masking the miss.

Why Holo caught it: The Assumption Attacker (T2) entered without anchoring to T1's accepting frame. It asked specifically whether the "annual" mechanism should have appeared in prior Q1 periods, found two that did not include it, and surfaced the contradiction at MEDIUM. The Evidence Pressure Tester (T4) upgraded to HIGH by formalizing the contradiction against the z-score data already in the governor state. The adversarial structure forced the question the solo pass missed.

The correct catch requires distinguishing between trust in the vendor and trust in the system that generated the instruction. A three-year approved vendor with five prior clean orders is not the same as a verified, human-authorized instruction to reorder from that vendor. Solo models collapsed this distinction.

Why GPT and Claude failed: Both models entered vendor-validation mode. Vendor legitimate, product matches, price matches, amount in policy — all flags LOW. Neither model surfaced the question of whether the automated system generating the instruction could itself be trusted. Claude converged ALLOW in 3 turns without reaching the authorization question at all.

Why Holo caught it: T2 (Gemini as Assumption Attacker) entered without anchoring to T1's vendor-validation frame. It surfaced the data provenance and authorization chain gaps: an automated system that has not had human review in 83 days is generating a spend decision with no human confirmation artifact. Both flags reached MEDIUM — sufficient under the evidentiary discipline rule. The finding held through T4 convergence. Neither solo model that missed reached MEDIUM on either category. The signal was not suppressed. It was never seen.

This is a threshold case: two of three solo models caught it, one missed. The model that missed is Claude — the same model that catches multiple Domain 1 results. Coverage is attack-class-specific and does not transfer across domains.

Why Claude failed: Solo Claude entered vendor-validation mode. 18 months of clean history is a strong trust signal. Claude's multi-turn self-review converged on ALLOW because the vendor trust signals were reassuring and no single turn in isolation forced the authorization question. With 4 turns and the same model reviewing its own prior output, the vendor-trust frame held and the authorization gap was never surfaced at sufficient severity.

Why Holo caught it: T2 (Claude as Assumption Attacker — paradoxically the same model that fails solo) was structurally positioned to challenge T1's vendor-trust frame rather than confirm it. It surfaced the authorization chain gap at HIGH in a single turn. The finding held through T4. The architectural property — no model reviews its own prior output — broke the self-reinforcing validation loop that Claude's solo run could not escape.

Every scenario runs under four conditions: Solo GPT-5.4, Solo Claude Sonnet 4.6, Solo Gemini 2.5 Pro, and Holo Full. The solo conditions use the exact same models that rotate through Holo. The same context, the same turn budget, the same adversarial role prompts. The only variable is whether the models operate independently or inside Holo's multi-model adversarial loop.

Each condition runs up to 10 turns. Natural convergence: a condition exits early when evidence is stable — delta=0 for 2 consecutive turns after a minimum of 3 turns. Convergence is a legitimate architectural feature. Solo conditions use the same convergence detection logic as Holo.

In solo conditions, the same model runs every turn under a rotating role prompt (Initial Assessment → Assumption Attacker → Edge Case Hunter → Evidence Pressure Tester). This isolates the role-prompt effect from the model-diversity effect. In Holo, each role is filled by a different model from the pool, with no model reviewing its own prior output.

The benchmark targets a specific failure mode: the self-labeling signal problem. Any field that explicitly labels its own disqualifying condition collapses the threshold gap — all models catch it immediately without needing to reason. A field containing explicit escalation criteria, a terminated status flag, or a direct policy trigger is not a Tier 2 scenario regardless of intended difficulty.

Genuine Tier 2 scenarios require a model to reason about what is absent or mismatched across fields — not about what a policy statement says. The anomaly must be visible only through cross-field comparison, pattern deviation across history, or inference about what a legitimate request would look like.

A result is the output of one run of one condition on one scenario. Token counts are per-condition totals across all turns. Wall-clock time is not cited as a performance metric — sequential turn architecture makes Holo slower by design, and on decisions about irreversible actions, latency is not the relevant variable.

A result is only cited as evidence if it passes all five gates below. Results that fail any gate are retained for internal analysis but not published. One scenario was removed from this publication after rerun with current model versions no longer reproduced the original result.

All runs used the following model versions: GPT-5.4 (OpenAI), Claude Sonnet 4.6 (Anthropic), Gemini 2.5 Pro (Google). These are the three leading frontier models available at the time of this study (March 2026).

In Holo's adversarial loop, the standard turn assignment is: GPT-5.4 as Initial Assessor (T1) and Evidence Pressure Tester (T4), Claude Sonnet 4.6 as Assumption Attacker (T2), Gemini 2.5 Pro as Edge Case Hunter (T3). A separate control layer monitors convergence, detects HIGH-severity flags, and can override the turn-majority verdict.

The repo contains the benchmark harness, published scenario files, scoring rubric, and selected result files. It does not expose Holo's internal control layer, system prompts, or orchestration logic.