Skip to content

Docs: rewrite the Model Routing reference page for workflow authors #68857

Closed
#68860
Closed
Docs: rewrite the Model Routing reference page for workflow authors#68857
#68860
Assignees
SivaKesava1copilot-swe-agent
Labels
ai-inspecteddocumentationImprovements or additions to documentationmodel-routing

Description

@rabeckett
rabeckett
opened on Oct 14, 2026

Summary

Rewrite the Model Routing reference page (docs/src/content/docs/reference/model-routing.md) so it is aimed at workflow authors. The full proposed page is included at the end of this issue and should replace the current page as written, apart from fixes for any facts that turn out to be wrong.

Why

The current page reads like a design note for the feature, not a guide for someone who wants to use it:

  • It opens with how the classifier and router work, and never says why a workflow author would want routing: right-sizing models to tasks to save AI Credits and improve results, and not hard-coding model names that later get retired.
  • Much of the text is implementation detail that authors don't need. Examples: AWF configuration fields (apiProxy.allowedModels, routing.candidateModels), the /reflect verification endpoint, a table of AWF versions and capabilities, image-pin history, a list of tested smoke workflows, log file paths and record schemas, per-engine log-line tables, and steering notices.
  • The difference between engine.model-routing.allowed-models and the workflow model policy (models.allowed and models.blocked) is spread across an 8-row table and several paragraphs. Authors ask directly whether they can write gpt-* in allowed-models. They can't, because it only accepts exact names; patterns belong in models.allowed. The page doesn't say so.
  • The description of goal: cost-speed is wrong. The page and the schema say it "also weighs execution time". In fact it picks the fastest model and effort that meet the mode's target (see routing.py in gh-aw-router-training).
  • It doesn't explain that routing currently works only with models served through GitHub Copilot, or that this limitation is temporary.
  • Several behaviours that authors run into are missing: how routing settings merge from imports, that routing ignores the AI Credits budget, and that claude-haiku-4.5 fails on the Claude engine (Claude on Copilot: pass Claude Code a modelOverrides map for Copilot's dotted Claude model ids (smoke-claude-on-copilot 400 invalid_reasoning_effort) #68834).

Changes

1. Replace docs/src/content/docs/reference/model-routing.md

Replace the page with the content in "Proposed page" at the end of this issue. Keep the frontmatter title and sidebar.order. The new page is organized as follows:

  1. Why use model routing: right-sizing, not hard-coding model names, and when a fixed model is still the better fit.
  2. Get started: a minimal routed workflow and what it needs.
  3. Choose a goal and mode: meanings and guidance.
  4. Choose which models the router can use: the model policy compared with allowed-models, plus worked examples (one model family, blocking expensive models, a fixed shortlist, a policy and a shortlist together, shared settings through imports, keeping up with new models).
  5. Sub-agents.
  6. Use routing with Claude, Codex, or pi.
  7. How routing works, moved near the end and kept short.
  8. Check which model was used. This links to the Audit and Cost Management pages instead of repeating their content.
  9. Known limitations, including "GitHub Copilot models only", which says the limitation is temporary.
  10. Troubleshooting.

Every example in the proposed page was compiled with gh-aw v0.91.14. All of them compile, and the one that combines a policy with a shortlist gives the warning the page describes.

2. Make related pages consistent

  • docs/src/content/docs/reference/frontmatter.md, "Model Routing (engine.model-routing:)": it says routing is "for Copilot". Say it works with the Copilot, Claude, Codex, and pi engines on GitHub Copilot inference. State that allowed-models takes exact model names and that patterns go in models.allowed or models.blocked.
  • docs/src/content/docs/reference/engines.md: the sentence "Copilot also supports experimental per-run model and reasoning-effort selection" should name all four engines that support routing.
  • docs/src/content/docs/reference/cost-management.md: the closing sentence of the section that links to the Model Routing guide says that page explains "classifier overhead, selected-route costs, and deviations in gh aw audit and gh aw logs". The new page no longer covers that in detail, so point readers to the Audit page for interpreting routing output, and to the Model Routing page for configuration.
  • docs/src/content/docs/reference/sandbox.md, "Custom infrastructure images (sandbox.agent.images)": the "Supported image roles" list leaves out router, although the compiler accepts it and the new page links here for replacing the router image. Add router, and note that it applies when engine.model-routing is set.

3. Fix the schema descriptions

In pkg/parser/schemas/main_workflow_schema.json, under engine.model-routing:

  • Rewrite the goal description: cost prefers the cheapest model and effort that meet the mode's target; cost-speed prefers the fastest that meet it.
  • Shorten the allowed-models description to what an author needs. It takes exact model names, which narrow the router's choice for the main agent only. It is not a request policy, and patterns belong in models.allowed or models.blocked. When it is omitted, the router chooses from the models the policy allows (Claude and Codex only from their own family). Expressions are allowed only when no policy is set. Drop the AWF version history.

Then regenerate docs/src/content/docs/reference/frontmatter-full.md with make generate-schema-docs.

Out of scope

  • Behaviour changes to routing, the compiler, or AWF. Problems found while writing the page will be filed separately.
  • Changes to the Audit and Cost Management pages beyond the link fix above.

Acceptance criteria

  • docs/src/content/docs/reference/model-routing.md matches the proposed page, apart from any factual corrections, and has no AWF-internal configuration fields, log file paths, or version history tables.
  • The related-page and schema edits above are made, and frontmatter-full.md is regenerated.
  • Statements on the page agree with pkg/workflow/model_routing.go, pkg/workflow/engine_validation.go, and the schema. If one doesn't, correct the page and say so in the PR.
  • make build-docs passes, and every internal link and anchor resolves.
Proposed page 
---
title: Model Routing
description: Let each workflow run use a model and reasoning effort suited to its task, chosen from the models the workflow allows.
sidebar:
  order: 605
---

> [!WARNING]
> Model routing is experimental, and its configuration and behavior may change. Review the [known limitations](#known-limitations) before enabling it.

Model routing chooses the model and reasoning effort for each run of a workflow, based on the task that run receives. Instead of naming one model in `engine.model` and using it for everything, you state what to optimize for and which models are acceptable, and a router makes the choice for each run.

## Why use model routing

**Right-size the model for each task.** Most workflows get tasks of varying difficulty. An issue-triage workflow might get a typo report one day and a bug that spans several services the next. A single fixed model is a compromise: a large model spends AI Credits on easy tasks, and a small model struggles with hard ones. With routing, routine tasks run on small, inexpensive models, and harder tasks get stronger models or more reasoning effort. The workflow can cost less than it would on one large model, and produce better results on hard tasks than it would on one small model.

**Stop hard-coding model names.** Models are added and retired regularly. A workflow that pins a model in `engine.model` fails when that model is retired, and it keeps using that model after better or cheaper ones appear, until someone edits it. A routed workflow describes which models are acceptable, for example every GPT model, or every model except the most expensive ones. The router skips models that are no longer available, and new models become candidates when you upgrade gh-aw. See [Keeping up with new models](#keeping-up-with-new-models).

A fixed model is still the better fit when you need the same behavior on every run, because routing can choose differently for similar tasks, or when the task depends on one model's specific capabilities.

## Get started

To route a workflow, add `model-routing` under `engine` with a `goal` and a `mode`:

```aw wrap
---
on:
  issues:
    types: [opened]
permissions:
  contents: read
  issues: read
  copilot-requests: write
engine:
  id: copilot
  model-routing:
    goal: cost
    mode: auto
safe-outputs:
  add-comment:
---

Find the likely cause of issue #${{ github.event.issue.number }} and comment with a suggested fix.
```

This configuration is a good starting point for most workflows. `goal: cost` prefers the cheapest choice that is likely to succeed, and `mode: auto` lets the router judge how demanding each task is. Because the workflow doesn't restrict models, the router can choose any model it supports that your organization's Copilot plan offers. [Choose which models the router can use](#choose-which-models-the-router-can-use) shows how to narrow that.

Routing currently works only with models served through GitHub Copilot, so the workflow needs Copilot authentication, here through the [`copilot-requests: write`](/gh-aw/reference/auth/#copilot-requests-write-permission) permission. Workflows that call model providers such as Anthropic or OpenAI directly, with their own API keys, can't use routing yet; see [GitHub Copilot models only](#github-copilot-models-only). Routing also needs the AWF firewall, which is enabled by default. The Copilot and pi engines need nothing else. Claude and Codex need one more setting; see [Use routing with Claude, Codex, or pi](#use-routing-with-claude-codex-or-pi).

Compile and run the workflow as usual. To see which model a run used, see [Check which model was used](#check-which-model-was-used).

If the workflow already sets `engine.model`, or another fixed model or reasoning effort setting, routing overrides it and the compiler warns. Remove the fixed setting once routing is enabled.

## Choose a goal and mode

`goal` and `mode` are both required. Together they tell the router how to trade cost against the chance that the task succeeds.

The **mode** sets how likely the chosen model and effort should be to complete the task. Higher targets lead to stronger models and more reasoning effort, and so to higher cost:

| `mode` | Target | Use it for |
|---|---|---|
| `economy` | Lowest | Routine, low-risk tasks such as triage, labeling, and summaries. |
| `balanced` | Medium | General-purpose work. |
| `robust` | Highest | Tasks where a failed run is costly, such as workflows that open pull requests. |
| `auto` | Chosen per run | Workflows whose tasks vary. The router picks `economy`, `balanced`, or `robust` for each task. |

The **goal** decides which of the choices that meet the target the router prefers:

| `goal` | Prefers |
|---|---|
| `cost` | The cheapest model and effort that meet the target. |
| `cost-speed` | The fastest model and effort that meet the target, even when a slower choice would cost less. |

If you're unsure, start with `goal: cost` and `mode: auto`. Set a fixed mode when you know what kind of tasks the workflow gets, for example `economy` for a labeling workflow. Choose `cost-speed` when someone is waiting on the result, such as a workflow that replies to a comment.

Targets are success rates estimated from benchmark results on similar tasks, not guarantees for a particular repository. The router can choose below the target when no candidate meets it or, with `goal: cost`, when a choice just below the target costs much less.

## Choose which models the router can use

By default, the router chooses from every model it supports that your organization's Copilot plan offers. Two settings narrow that, and they do different jobs:

| Setting | What it does | Applies to | Entries |
|---|---|---|---|
| `models.allowed` and `models.blocked` | Sets which models the workflow may use at all. The firewall rejects requests for other models. | Every model request in the run: the main agent, sub-agents, and built-in agents | Model names or patterns with `*` |
| `engine.model-routing.allowed-models` | Gives the router a shortlist to choose from. | Only the router's choice for the main agent | Exact model names |

`models.allowed` and `models.blocked` are the workflow's [model policy](/gh-aw/reference/glossary/#model-policy-modelsallowed-modelsblocked), and they work the same way with or without routing. Use them for rules such as "only GPT models" or "never Opus models". Use `allowed-models` when you want the router to choose among a few specific models. Most workflows need only one of the two. The examples below show common setups.

### Only one model family

To let the router choose among all GPT models, put a pattern in the model policy and omit `allowed-models`:

```aw wrap
models:
  allowed: ["gpt-*"]
engine:
  id: copilot
  model-routing:
    goal: cost
    mode: auto
```

New GPT models become candidates without editing the workflow. Because the policy applies to the whole run, sub-agents are also limited to GPT models.

> [!NOTE]
> `allowed-models` accepts exact model names only. A pattern such as `allowed-models: ["gpt-*"]` fails to compile. Put patterns in `models.allowed` or `models.blocked` instead.

### Every model except the most expensive

To keep the router away from the most expensive models, block them in the model policy:

```aw wrap
models:
  blocked: ["claude-opus-*"]
engine:
  id: copilot
  model-routing:
    goal: cost
    mode: robust
```

This is useful with `mode: robust`, which chooses stronger models more often.

### A fixed shortlist

To let the router choose only among models you've tested the workflow with, list them in `allowed-models`:

```aw wrap
engine:
  id: copilot
  model-routing:
    goal: cost
    mode: auto
    allowed-models: [gpt-5.4-mini, gpt-5.6-luna, gpt-5.6-sol]
```

The router never considers other models, including new ones, so update the list as models change. If a listed model is retired, the router chooses among the others, and if none is available, the run fails. The shortlist doesn't affect sub-agents.

### A policy and a shortlist together

When a workflow sets both, the router chooses from the `allowed-models` entries that the policy allows. `allowed-models` can't permit a model that the policy excludes:

```aw wrap
models:
  allowed: ["gpt-*"]
engine:
  id: copilot
  model-routing:
    goal: cost
    mode: balanced
    allowed-models: [gpt-5.4-mini, gpt-5.6-sol, claude-sonnet-5]
```

Here the router chooses `gpt-5.4-mini` or `gpt-5.6-sol`, and the compiler warns that `claude-sonnet-5` won't be selected. If the policy excludes every entry, compilation fails.

`allowed-models` entries can be GitHub Actions expressions, such as `${{ vars.ROUTING_MODEL }}`, but only when the workflow sets neither `models.allowed` nor `models.blocked`, because the compiler can't check an expression against the policy.

### Shared routing settings

To use the same routing settings in several workflows, put them in a shared file and import it. For example, `.github/workflows/shared/routing.md`:

```aw wrap
---
engine:
  id: copilot
  model-routing:
    goal: cost
    mode: auto
    allowed-models: [gpt-5.4-mini, gpt-5.6-sol]
---
```

A workflow that imports it is routed with those settings:

```aw wrap
---
on: workflow_dispatch
permissions:
  contents: read
  copilot-requests: write
imports:
  - shared/routing.md
---

Summarize the open pull requests.
```

The importing workflow must either omit `engine` or use its object form, such as `engine: { id: copilot }`. If it also enables routing for the same engine, its `goal` and `mode` apply, and the imported `allowed-models` entries are added to its own. If it enables routing without `allowed-models`, the router chooses from only the imported entries.

### Keeping up with new models

The router chooses only among models it has benchmark data for. That data is part of the router version that gh-aw pins when it compiles the workflow. When gh-aw updates the router with data for new models, run [`gh aw upgrade`](/gh-aw/guides/working-with-workflows/#upgrading-workflows) to recompile your workflows, and the new models become candidates if your settings allow them. Models that have been retired, or that your organization's Copilot plan doesn't offer, are skipped at run time.

To get the most from this, describe models with patterns in the model policy, or don't restrict them, rather than listing exact names in `allowed-models`.

## Sub-agents

Routing chooses the model for the main agent only. Declared sub-agents use the model they declare, and built-in agents such as `explore` use their default models, as long as the model policy allows them. The compiler warns when a sub-agent's model is outside the policy, because the firewall will reject its requests. `gh aw audit` reports sub-agent requests as deviations from the selected route, which is expected.

A sub-agent's model must also work with the engine and with the main agent's selected model:

| Engine | Sub-agent models |
|---|---|
| Copilot in CLI mode (default) | Same family as the main agent's selected model, such as `gpt-*` or `claude-*` |
| Copilot in SDK mode (`engine.copilot-sdk: true`) | Any family |
| pi | Any family |
| Claude, Codex | The engine's own family |

In Copilot CLI mode, a sub-agent from a different family than the selected main model fails ([copilot-cli#5103](https://github.com/github/copilot-cli/issues/5103)), and the run logs a warning at startup. If a CLI-mode workflow declares sub-agents, either keep every candidate in the sub-agents' family or use SDK mode. In this example, the router can choose a GPT or Claude model for the main agent, and the Claude sub-agent works with either:

```aw wrap
---
on: workflow_dispatch
permissions:
  contents: read
  copilot-requests: write
engine:
  id: copilot
  copilot-sdk: true
  model-routing:
    goal: cost
    mode: auto
    allowed-models: [gpt-5.4-mini, gpt-5.6-sol, claude-sonnet-5]
---

Review the open issues and ask the `triager` agent to suggest a label for each one.

## agent: `triager`
---
description: Suggests one label for an issue
model: claude-haiku-4.5
---
Reply with the single best label for the issue you are given.
```

## Use routing with Claude, Codex, or pi

Routing works with the `copilot`, `claude`, `codex`, and `pi` engines, as long as they use models served through GitHub Copilot. Claude and Codex call the Anthropic and OpenAI APIs by default, so they need `model-provider: github` to use Copilot instead, and they can only use models from their own family:

| Engine | Required setting | Candidate models |
|---|---|---|
| `copilot` | None | Any |
| `pi` | None | Any |
| `claude` | `model-provider: github` | `claude-*` only |
| `codex` | `model-provider: github` | `gpt-*` only |

For example, to route a Claude workflow between two Claude models:

```aw wrap
engine:
  id: claude
  model-provider: github
  model-routing:
    goal: cost
    mode: balanced
    allowed-models: [claude-sonnet-5, claude-opus-5]
```

Without `allowed-models`, Claude and Codex choose from every model in their family that the policy allows. The compiler rejects other engines and providers, and `allowed-models` entries from another family.

Routing also has these requirements:

- It needs the AWF firewall, which is the default sandbox.
- It can't be combined with `engine.fallback-models`.
- gh-aw's default AWF version supports routing. If a workflow pins AWF with `sandbox.agent.version`, use v0.28.33 or later.
- Organizations that require images from approved registries can replace the router image with `sandbox.agent.images.router`; see [Custom infrastructure images](/gh-aw/reference/sandbox/#custom-infrastructure-images-sandboxagentimages).

## How routing works

Routing happens once per run, before the agent starts:

1. A small model classifies the task by type (such as fix, refactor, or explain), scope, and complexity. It reads the workflow's prompt, including interpolated content such as the triggering issue's text, and treats that text as data, not instructions.
2. The [router](https://github.com/githubnext/gh-aw-router) ranks the candidate models and efforts for that kind of task, using benchmark results, and chooses one according to the workflow's `goal` and `mode`.
3. The main agent runs the whole task with the chosen model and effort.

The router chooses only from the workflow's candidates, whatever the task text says. Classification runs on one of the candidates, preferring a small one, and typically costs a small fraction of an AI Credit per run. If routing can't choose a model, the run fails rather than falling back to a fixed model. Threat detection isn't routed and uses its configured detection model.

## Check which model was used

The agent step's log shows the selection, for example:

```text
inference routing: mode=awf-routed model=gpt-5.6-sol effort=high
```

`gh aw audit <run-id>` includes a Model Routing section with the task classification, the selected model and effort, the ranked alternatives, and what the classifier, the selected model, and any other models cost. `gh aw logs` aggregates routing across runs, showing which model each kind of task received and what it cost. See [Audit](/gh-aw/reference/audit/) and [Cost Management](/gh-aw/reference/cost-management/).

## Known limitations

### GitHub Copilot models only

The router currently chooses only among models served through GitHub Copilot, because its benchmark data and model availability checks cover Copilot's models. The compiler rejects routing in workflows that use another provider, such as Claude or Codex with their own Anthropic or OpenAI API keys, and in workflows on engines other than `copilot`, `claude`, `codex`, and `pi`. This limitation is temporary, and support for more providers and engines is planned.

### Selection varies between runs

The classifier is a language model, so the same task can get a different model or effort on another run, especially with `mode: auto`.

### Reasoning effort with Claude and Codex

The router doesn't yet account for which reasoning efforts each engine supports. If it chooses an effort the engine can't use (`none` or `minimal` with Claude, `none` or `max` with Codex), the run fails.

With the Claude engine, requests to `claude-haiku-4.5` fail with HTTP 400, because Claude Code sends a reasoning effort that this model doesn't accept ([gh-aw#68834](https://github.com/github/gh-aw/issues/68834)). The router can choose it when `allowed-models` is omitted, so list Claude candidates explicitly and leave it out.

### AI Credits budget

Routing doesn't consider the run's AI Credits budget (`max-ai-credits`, 1,000 by default). Runs routed to larger models, which `robust` chooses more often, use more of it. See [Cap AI Credits per Run](/gh-aw/reference/cost-management/#cap-ai-credits-per-run).

### Sub-agents in Copilot CLI mode

Sub-agents from a different model family than the selected main model fail. See [Sub-agents](#sub-agents).

## Troubleshooting

| Symptom | Cause and fix |
|---|---|
| Compilation fails because routing requires GitHub Copilot inference | A Claude or Codex workflow uses its default provider. Set `engine.model-provider: github`. See [GitHub Copilot models only](#github-copilot-models-only). |
| Compilation fails when `allowed-models` contains a pattern such as `gpt-*` | `allowed-models` takes exact names. Put the pattern in `models.allowed` instead. See [Only one model family](#only-one-model-family). |
| Compilation fails because all `allowed-models` are excluded by `models.allowed` or `models.blocked` | The policy excludes every candidate or, for Claude or Codex without `allowed-models`, the engine's whole family. Change either setting. |
| Compilation fails with an incompatible model-routing candidate | An `allowed-models` entry isn't in the engine's model family. Remove it. |
| Compilation fails with "multiple engine fields found" after importing routing settings | The importing workflow uses `engine: copilot`. Remove `engine`, or use `engine: { id: copilot }`. |
| The agent step fails and `gh aw audit` reports a routing failure | Routing couldn't produce a usable selection: no candidate was available to the organization, the router failed (exit code `78`), or the engine can't use the selected model or effort. The audit's Model Routing section shows the failure code and details. |
| A sub-agent's requests fail with HTTP 400 in Copilot CLI mode | The sub-agent's model family differs from the selected main model. See [Sub-agents](#sub-agents). |
| `gh aw audit` reports deviated requests | Requests used a model other than the selected one, usually from sub-agents or built-in agents. This is expected and isn't a policy violation. |
| `gh aw audit` reports degraded classification | Classification failed, so routing continued without it; with `mode: auto`, the router used `balanced`. |
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

ai-inspecteddocumentationImprovements or additions to documentationmodel-routing

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions