> ## Documentation Index
> Fetch the complete documentation index at: https://docs.inputapi.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Production integration

> Define timeouts, concurrency, observability, and real-service acceptance.

Start with a minimal request, then add failure handling and resource bounds to your application. This page provides integration guidance, not universal deployment limits, latency, or billing guarantees.

## Timeouts and request lifecycle

| Layer | Define | Acceptance check |
| - | - | - |
| Connection | DNS, TLS, and proxy connection timeouts | Unreachable services fail promptly with a useful error |
| Ordinary requests | SDK read timeout and application deadline | Long requests cannot occupy workers indefinitely |
| SSE | First-event wait, idle timeout, total deadline | Network chunks are not events; truncation is not completion |
| Media tasks | Submission timeout, polling interval, maximum attempts | Save the task ID and query it after timeout instead of resubmitting |
| WebSocket | Upgrade, idle connections, event completion | Verify proxy and channel support and identify per-event failures |

Example timeout values are for demonstration. An SDK read timeout may not bound the entire operation; set an application deadline separately. Disconnection or client cancellation does not guarantee upstream work has stopped, especially for durable media tasks.

## Concurrency and retries

Bound concurrency and queue lengths per application, tenant, and model. Return a retryable message when queues fill. One RAG workflow can involve embedding, reranking, and generation calls, so account for the whole workflow. Bound tool-call rounds and calls per round too.

Examples disable automatic SDK retries to expose the result of one request. For production, see [errors and retries](/en/guides/errors): consider 429, transient 5xx, jittered backoff, and any gateway retries already configured. Avoid unbounded retries at the client, application queue, and gateway simultaneously. Correct invalid parameters on HTTP 400. After partial output, do not silently concatenate a second generation into the first answer.

## Proxies and output handling

For SSE, check reverse-proxy buffering, compression, idle timeouts, and whether events arrive incrementally. Client `stream: true` alone cannot remove proxy buffering. Verify WebSocket upgrades and connection lifetimes. Media upload sizes must be allowed by the client, proxy, and gateway together.

Validate final text completion, refusals, and content types. Check business schemas for structured output and call IDs for tools. Verify media MIME types and actual content. HTTP 200, valid JSON, and successful download are not complete business acceptance criteria.

## Observability and data

Record an application request ID, returned `X-Gateway-Request-Id` (compatibility header: `X-Oneapi-Request-Id`), time and timezone, model, endpoint, status, first-event latency, total latency, and retry count. Responses intercepted by a front proxy may lack gateway IDs. Do not assume SDK `_request_id` reads these custom headers.

Exclude keys, cookies, complete inputs, media Base64, and compaction ciphertext from default logs. Decide whether to retain history, tool results, and task IDs, and confirm gateway logging, upstream retention, and artifact URL expiry with the deployment operator. `store: false` is only a request option.

## Real-service acceptance

1. Discover a model and make a minimal call with the same address and token; verify the actual protocol.
2. Test each capability the application uses: text, SSE, tools, structured output, and multimodal inputs or outputs.
3. Exercise denied access, rate limits, timeouts, truncated streams, refusals, and failed tasks; ensure no infinite retries or duplicate publication.
4. Evaluate representative inputs for accuracy, latency, and resource use. Record versions and routing settings for comparison after upgrades.
5. Start with limited traffic and recoverable settings. Fall back to a verified application path instead of switching models arbitrarily.

This repository's `npm run check` checks documentation and local simulated responses only. These real-service checks require a deployment and test access and have not been performed. GitHub, hosting accounts, and domain connections remain deferred to publishing.

[阅读中文](/zh/guides/production)

[Model routing](/en/guides/model-routing) · [Rate limits](/en/guides/rate-limits) · [Observability](/en/guides/observability) · [AI SDK](/en/guides/ai-sdk)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.