Claude threat detection follows the newest listed Haiku and fails open when it is unpriced; GH_AW_MODEL_DETECTION_CLAUDE never reaches the detection step #66852

Summary

On the Claude engine, threat detection runs on model detection unless the workflow pins a model. AWF resolves that alias against the live Anthropic /v1/models listing and picks the highest version number. On 2026-10-07 Anthropic listed claude-haiku-5-5. From the next run onward, detection resolved to claude-haiku-5-5 instead of claude-haiku-4-5-20251001. The AWF pricing table has no entry for that model, so every detection request failed with HTTP 400. The lock file had not changed, and neither had the CLI pin.

Detection defaults to continue-on-error: true, so the detection job, the safe-outputs job, and the run all concluded success. Every affected run therefore lost its threat verdict without any workflow change. The only signals were a warning annotation and an agent_failure comment on the [aw] Detection Runs tracking issue.

The org variables documented for choosing the detection model (GH_AW_MODEL_DETECTION_CLAUDE, GH_AW_DEFAULT_MODEL_CLAUDE) do not reach the detection step, so they cannot steer it off the floating alias (details below).

The missing price is tracked in github/gh-aw-firewall#9695, which concerns workflows that opt into Haiku 5.5. This issue covers the gh-aw side: the default detection model floats onto any newly listed Haiku, and the documented overrides do not apply to it.

Environment

• gh-aw compiler v0.89.21 (default AWF v0.28.23), engine: claude, Claude Code CLI pinned via engine.version (2.1.285; also reproduced on 2.1.293)
• Anthropic auth via GitHub OIDC (WIF), external threat-detect v0.5.2
• The frontmatter models: overlay priced claude-opus-5-5 and claude-sonnet-5-5 only. default-ai-credits-pricing was not set.
• Org variables GH_AW_DEFAULT_MODEL_CLAUDE=sonnet and GH_AW_MODEL_AGENT_CLAUDE=sonnet were set.

Observed

Runs on 2026-10-04 and 2026-10-07 02:49 UTC: detection resolved to claude-haiku-4-5-20251001 and the requests returned HTTP 200. The run on 2026-10-08 02:47 UTC used the same lock file:

[engine] [claude-code:unrecognized_model] {"model":"detection","query_source":"sdk"}
API Error: 400 Model "claude-haiku-5-5" has no AI credits pricing and no default pricing is configured. Set apiProxy.defaultAiCreditsPricing in the AWF config (e.g. {"input": 3.0, "output": 15.0}) to provide a fallback rate, or add the model to the pricing table.
⚠️  Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.
##[error]ERR_SYSTEM: ❌ Detection result file not found at: /tmp/gh-aw/threat-detection/detection_result.json

Job conclusions were agent=success detection=success safe_outputs=success conclusion=success. Two runs of a second workflow later the same day show the same pattern. In all three runs the agent's only safe output was noop, so nothing was published without a verdict. A run that produced a PR or issue would have taken the same path.

The models.json that the detection job's api-proxy wrote at startup shows both Haiku models in the live listing:

providers.anthropic.models: [..., "claude-haiku-4-5-20251001", "claude-haiku-5-5", ...]   (source: provider, observed_at 2026-10-08T02:50:22Z)

Adding a claude-haiku-5-5 cost entry to the frontmatter models: overlay fixed the problem: detection then ran on claude-haiku-5-5 with HTTP 200, on both CLI versions.

Cause

1. gh-aw always gives Claude detection a concrete model string, defaulting to the detection alias.

• threat_detection_external.go (v0.89.21 L410-L422) resolves the model in this order:
  1. the main workflow model, via inheritedDetectionModel;
  2. the threat-detection engine model;
  3. compile-time GH_AW_DEFAULT_DETECTION_MODEL;
  4. engine.GetDefaultDetectionModel();
  5. the literal "detection".
• threatDetectionData.Model is therefore never empty. The inline path does the same at threat_detection_inline_engine.go L105-L122.
• Because the model is non-empty, applyClaudeModelEnvVars (claude_engine.go L531-L555) takes its configured-model branch and emits ANTHROPIC_MODEL: detection as a literal.
• The ${{ vars.GH_AW_MODEL_DETECTION_CLAUDE || vars.GH_AW_DEFAULT_MODEL_CLAUDE || ... }} expression is emitted only when the model is empty, so it never appears in the detection step.
• The compiled lock confirms this. The detection step contains ANTHROPIC_MODEL: detection and no reference to either variable.
• This contradicts reference/environment-variables.md ("Detection runs" table) and compiler-enterprise-environment-controls.md. It also contradicts ADR 35286, which says the precedence MUST be engine.model → vars.GH_AW_MODEL_* → vars.GH_AW_DEFAULT_MODEL_<ENGINE> → compiler fallback.
• The code is unchanged in v0.91.5 (threat_detection_external.go L498, claude_engine.go L562-L590) and on main.

2. In model_aliases.json, detection resolves to "the newest Haiku the provider lists".

• The chain is detection → small → mini → haiku → anthropic/*haiku* (model_aliases.json L7, L37-L40).
• In AWF, the candidates for anthropic/*haiku* come from the startup GET /v1/models fetch (providers/anthropic.js L154, adapter-factory.js L158-L170).
• The candidates are sorted by their extracted digit sequences, highest first, and the first one wins (model-resolver.js v0.28.23 L298-L302, compareByVersion in model-utils.js L55). [5,5] sorts above [4,5,20251001].
• Alias candidates are not filtered by priceability. The middle-power fallback pool is (model-fallback.js L94-L100), but the alias path is not. So the resolver picks an unpriced model over a priced one.

3. Nothing in the price lookup catches claude-haiku-5-5.

• ai-credits-pricing.js has claude-haiku-4-5, claude-sonnet-5 and claude-opus-5. It has no claude-haiku-5 entry (v0.28.23 L26-L35).
• The prefix match in resolveLowerPriorityPricing (guards/ai-credits-guard.js L175-L185) therefore prices claude-sonnet-5-5 and claude-opus-5-5 from their -5 entries but finds nothing for claude-haiku-5-5.
• The bundled models.dev catalog has claude-haiku-4.5 only.
• checkUnknownModelRejection (L267-L271) rejects when maxAiCredits is set and no default pricing exists. The detection step always sets maxAiCredits (it defaults to 400).
• The table is unchanged in AWF v0.28.44 (the default for gh-aw v0.91.5), v0.28.49 and main (8bc872d).

When it triggers

All of the following must hold:

• The Claude engine with the Anthropic provider.
• No engine.model in the workflow. If the workflow sets one, detection inherits it via inheritedDetectionModel (threat_detection_helpers.go L266).
• No safe-outputs.threat-detection.engine.model, and no GH_AW_DEFAULT_DETECTION_MODEL in the environment at compile time.
• No default-ai-credits-pricing, and no overlay price for the resolved model.
• Anthropic's /v1/models lists a Haiku that has a higher version number than every priced one and that the AWF table cannot price (currently claude-haiku-5-5).

It does not trigger when any of the following is set: engine.model, a pinned detection model, default-ai-credits-pricing, or an overlay price for the resolved model.

Not verified:

• The Copilot engine. copilot/*haiku* should resolve to claude-haiku-5.5 there. AWF reads runtime pricing only from Copilot-format billing.token_prices (runtime-model-catalog.js L56, L106). [model-inventory] Model alias inventory update - 2026-10-07 #66711 indicates the Copilot SDK returns prices for that model, but I have not run it.
• Bedrock and Vertex providers.
• Whether the same float could, in principle, affect evals (also small) or any workflow that sets model: haiku / mini / small on the Anthropic provider.

Workarounds

Any one of these works. The first is verified; the others follow from the code above.

# 1. Price the model the alias now resolves to (per-token, $0.10 / $0.01 / $0.125 / $0.50 per 1M)
models:
  providers:
    anthropic:
      models:
        claude-haiku-5-5:
          cost:
            input: "0.0000001"
            cache_read: "0.00000001"
            cache_write: "0.000000125"
            output: "0.0000005"

2. Set default-ai-credits-pricing (see api-proxy: add claude-haiku-5-5 to the AI-credits price table gh-aw-firewall#9695).
3. Pin detection: safe-outputs.threat-detection.engine: { id: claude, model: claude-haiku-4-5 }. Alternatively, set GH_AW_DEFAULT_DETECTION_MODEL in the environment of gh aw compile.

Setting vars.GH_AW_MODEL_DETECTION_CLAUDE does not work today, for the reason given in Cause 1.

Suggested fixes

1. gh-aw: make vars.GH_AW_MODEL_DETECTION_CLAUDE / vars.GH_AW_DEFAULT_MODEL_CLAUDE apply to detection when the detection model comes from the built-in default, as the docs and the ADR describe. For example, emit ${{ vars.GH_AW_MODEL_DETECTION_CLAUDE || vars.GH_AW_DEFAULT_MODEL_CLAUDE || 'detection' }} instead of the literal. The other engines' GH_AW_MODEL_DETECTION_* variables may need the same check.
2. gh-aw / AWF: restrict alias-candidate selection to priceable models, as the fallback pool already is. A priced claude-haiku-4-5-* would then win over an unpriced newer model, and new provider releases would stop breaking pinned workflows.
3. AWF: add claude-haiku-5-5 to the pricing table (api-proxy: add claude-haiku-5-5 to the AI-credits price table gh-aw-firewall#9695).
4. Optional: in the detection conclusion, report an engine failure caused by an HTTP 400 configuration error such as unknown_model_ai_credits differently from a transient tooling failure. It is deterministic and will repeat on every run until the configuration changes. Model routing: threat detection fails in routed workflows, so safe outputs are applied without a threat verdict #66224 raised a similar visibility point for routed workflows.

Related: github/gh-aw-firewall#9695 (missing price), #66711 (Copilot claude-haiku-5.5 pricing), #66224 and #59622 (other causes of detection engine failure with safe outputs still applied), #66492 (unpriced-model handling).