How a request is routed
The client sends one deployment-publishedmodel ID. The gateway filters candidate channels using the token’s permitted provider category, automatic or manual routing settings, current grants, model, request path, and channel availability. Manual bindings and automatic routing follow their respective configuration; automatic routing does not grant access to every provider.
Visibility in the model list does not prove every channel supports tools, JSON Schema, media, or state recovery. Start with model capabilities. The request’s
group is not a general provider-selection field; current code handles it only on the dedicated Playground route.
Retry boundaries
The gateway considers cancellation, retry budgets, error classification, channel constraints, and session policy. Once an ordinary HTTP response has started writing, the corresponding retry path stops; strict sessions or pinned channels can also prevent switching. Do not promise that every failure selects another channel. Client SDK retries and internal gateway attempts are separate counters. Local examples disable automatic SDK retries for generation. Before production, confirm the internal budget with your operator, then configure application deadlines, queue limits, and recovery policy.When a backup model is required
- Validate both models’ protocols, tool schemas, context capacity, and output constraints in advance.
- Specify eligible failures and a total attempt limit; fix authentication, permission, and parameter errors first.
- Record the model and gateway request ID for each new request. If output has already been shown, clearly begin a new attempt instead of concatenating outputs into one answer.
- Check the state of executed tools, submitted media tasks, and external writes first. A network timeout does not prove the upstream did no work.
- Do not assume Responses IDs, compacted state, native signatures, or WebSocket sessions transfer between channels or models. Use reviewed portable conversation history where appropriate.
Migrating from OpenRouter
OpenRouter’smodels fallback array, provider.order, routing suffixes, and dedicated limit-inspection API are outside the general integration contract verified here. Interface compatibility alone does not make those settings portable. These examples use a single model; confirm any deployment-specific extension separately.
Verified against project revision 30a6d88a4: middleware/distributor.go, service/channel_select.go, service/group.go, service/relay_error.go, and relay/helper/model_mapped.go. Operators manage routing settings; this guide does not change administration configuration.
SDK migration · Production integration
阅读中文