~/joshua-small

cat notes/2026-08-25-de-trap-case.md

Haiku and Opus agreed on the category and disagreed on whether to fight

Aug 25, 2026 · Dispute Engine

The test case is filed as Visa 10.4 (fraud), but the narrative describes a cancelled subscription. Both models ignored the filed code and answered subscription_canceled at 0.85 confidence.

They differed on judgment:

Haiku 4.5 Opus 5
Winnability likely winnable uncertain
Required evidence 3 items 5 items
Missing evidence 2 3

Opus asked for the customer communication log and usage logs. Haiku didn’t. This is one case, so it’s a hypothesis, not a result. If it holds across the corpus, the cheaper model is systematically too optimistic about winnability, which would set the threshold for routing cases to the larger model.

Neither model flagged a $249 case full of contradictions for human review. The labeling guide needs a rule for what should trigger needs_human_review.

← notes/