> ## Documentation Index
> Fetch the complete documentation index at: https://docs.inputapi.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Model routing and failover

> Distinguish model IDs, provider selection, channel retries, and application fallback.

## How a request is routed

The client sends one deployment-published `model` ID. The gateway filters candidate channels using the token's permitted provider category, automatic or manual routing settings, current grants, model, request path, and channel availability. Manual bindings and automatic routing follow their respective configuration; automatic routing does not grant access to every provider.

| Concept | What an integrator needs to know |
| - | - |
| Model ID | Select from the target deployment's model list; preserve prefixes and case |
| Model mapping | Operators can map a public ID to an upstream ID; the request ID need not equal the upstream name |
| Providers and channels | A model can have multiple candidate channels; deployment policies control ordering, weights, health, and capacity |
| Channel retry | The gateway retries within one request according to policy; budgets, status codes, and channel overrides affect the result |
| Application model fallback | The application sends a new request using another model; revalidate capabilities, context, and output structure |

Visibility in the model list does not prove every channel supports tools, JSON Schema, media, or state recovery. Start with [model capabilities](/en/guides/model-capabilities). The request's `group` is not a general provider-selection field; current code handles it only on the dedicated Playground route.

## Retry boundaries

The gateway considers cancellation, retry budgets, error classification, channel constraints, and session policy. Once an ordinary HTTP response has started writing, the corresponding retry path stops; strict sessions or pinned channels can also prevent switching. Do not promise that every failure selects another channel.

Client SDK retries and internal gateway attempts are separate counters. Local examples disable automatic SDK retries for generation. Before production, confirm the internal budget with your operator, then configure application deadlines, queue limits, and recovery policy.

## When a backup model is required

1. Validate both models' protocols, tool schemas, context capacity, and output constraints in advance.
2. Specify eligible failures and a total attempt limit; fix authentication, permission, and parameter errors first.
3. Record the model and gateway request ID for each new request. If output has already been shown, clearly begin a new attempt instead of concatenating outputs into one answer.
4. Check the state of executed tools, submitted media tasks, and external writes first. A network timeout does not prove the upstream did no work.
5. Do not assume Responses IDs, compacted state, native signatures, or WebSocket sessions transfer between channels or models. Use reviewed portable conversation history where appropriate.

This site does not provide a generic automatic generation-resubmission script: the application must handle duplicate work and state compatibility. A read-only retry example is available in [rate limits and retries](/en/guides/rate-limits).

## Migrating from OpenRouter

OpenRouter's `models` fallback array, `provider.order`, routing suffixes, and dedicated limit-inspection API are outside the general integration contract verified here. Interface compatibility alone does not make those settings portable. These examples use a single `model`; confirm any deployment-specific extension separately.

Verified against project revision `30a6d88a4`: `middleware/distributor.go`, `service/channel_select.go`, `service/group.go`, `service/relay_error.go`, and `relay/helper/model_mapped.go`. Operators manage routing settings; this guide does not change administration configuration.

[SDK migration](/en/guides/sdk-migration) · [Production integration](/en/guides/production)

[阅读中文](/zh/guides/model-routing)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.