Rerank documents

Rerank reorders candidate texts already retrieved for a query, commonly as a second stage in RAG. It returns relevance scores and original document indices. It does not generate embeddings, search the entire knowledge base, or directly generate an answer. POST /v1/rerank is a gateway endpoint, not an official OpenAI API. Confirm channel support and select a reranking model. Embedding and chat models are not automatically rerankers.

Python: rerank candidate texts

Configure API_BASE_URL (including /v1), API_KEY, and a reranking model’s API_MODEL using the Python SDK guide. Save as rerank.py and run python rerank.py. It reuses the SDK’s generic HTTP client.post helper, not a nonexistent client.rerank method, and needs no additional vendor SDK. The example explicitly sends top_n: 2 and return_documents: false, validates result indices and scores, sorts descending by score, and maps each index back to the original text. It uses application-owned text even if a channel still returns document.

import json
import math
import os
from openai import OpenAI, APIConnectionError, APIStatusError

query = "How can I reset my password?"
documents = [
    "Invoices are available in the billing dashboard.",
    "Use the password reset link on the sign-in page.",
    "You can change the interface language in settings.",
]
top_n = 2

with OpenAI(base_url=os.environ["API_BASE_URL"].rstrip("/"),
            api_key=os.environ["API_KEY"], timeout=60.0, max_retries=0) as client:
    try:
        # Generic HTTP helper: /rerank is a gateway extension, not an OpenAI API method.
        result = client.post("/rerank", cast_to=dict, body={
            "model": os.environ["API_MODEL"], "query": query, "documents": documents,
            "top_n": top_n, "return_documents": False,
        })
        if not isinstance(result, dict) or not isinstance(result.get("results"), list):
            raise ValueError("Expected a results array.")
        seen = set()
        matches = []
        for item in result["results"]:
            if not isinstance(item, dict):
                raise ValueError("Expected a result object.")
            index = item.get("index")
            score = item.get("relevance_score")
            if type(index) is not int or not 0 <= index < len(documents) or index in seen:
                raise ValueError("Invalid or duplicate document index.")
            if type(score) not in (int, float) or not math.isfinite(score):
                raise ValueError("Invalid relevance score.")
            seen.add(index)
            # Preserve application-owned text and IDs using the original input index.
            matches.append({"index": index, "score": score, "text": documents[index]})
        matches.sort(key=lambda item: item["score"], reverse=True)
        print(json.dumps(matches[:top_n], ensure_ascii=False, allow_nan=False))
    except APIStatusError as error:
        raise SystemExit(f"Request failed (HTTP {error.status_code}). Check rerank model access and parameters.")
    except APIConnectionError:
        raise SystemExit("Connection failed. Check the API address and network.")
    except (ValueError, TypeError, OverflowError):
        raise SystemExit("Invalid rerank response; check result indices and numeric scores.")

Indices, scores, and result counts

results[].index is a zero-based position in this request’s documents array, not a rank or database ID. After filtering, chunking, or selecting candidates, retain the mapping from that final array to application IDs. Never use a result’s position in the score list to fetch the source text. Higher scores usually indicate greater relevance within a query. They are not universally calibrated probabilities or directly comparable across models. The example requires finite scores without forcing a 0–1 range. Evaluate thresholds with your model and representative application data. Empty results produces an empty list rather than invented candidates. A model may still rank irrelevant candidates; apply application thresholds before answering. Set top_n explicitly to a positive integer no larger than the candidate count. Some channels may ignore the requested count. The example locally keeps the first two results, which does not reduce documents already processed upstream. top_n controls returned results, not knowledge-base retrieval size.

Channel differences

Parameter	Differences in the current implementation
`return_documents`	The Cohere adaptor always requests `true`; Ali defaults to `true` when omitted. Neither omission nor `false` is a universal guarantee that text will not be returned
`top_n`	Cohere maps omitted/nonpositive values to 1; other channels may differ. The example explicitly requests 2
`documents`	The DTO accepts objects, but fields and support depend on the channel. String arrays are the portable baseline here
`max_chunk_per_doc` / `overlap_tokens`	These are the gateway field spellings, but not every adaptor forwards them. Do not copy a provider’s plural spelling or assume uniform long-document chunking

The query and documents are sent to the configured gateway and upstream. return_documents: false does not prevent uploading source text. For structured documents, serialize stable text in the application and keep separate application IDs. Bound candidate counts and text lengths before tuning reranking parameters; do not send the entire corpus at once.

Combine with Embeddings

The flow is: Embeddings for a vector index → application candidate retrieval → Rerank → selected source text to a chat model. Rerank can also operate on candidates from keyword search. For errors, check model capability, nonempty query/documents, length, and optional parameters. Even on HTTP 200, validate the results shape, duplicate/out-of-range indices, and invalid scores. This example exits on malformed output and disables automatic retries. Score interpretation: Cohere Rerank best practices. That provider’s request limits are not universal channel limits. Next: combine retrieval, optional reranking, and cited answers in the RAG guide. 简体中文

Getting started

Integration guides

Application guides

Multimodal guides

Retrieval and reranking

Production and troubleshooting

Additional protocols and tasks

API reference

Python: rerank candidate texts

Indices, scores, and result counts

Channel differences

Combine with Embeddings

​Python: rerank candidate texts

​Indices, scores, and result counts

​Channel differences

​Combine with Embeddings

Python: rerank candidate texts

Indices, scores, and result counts

Channel differences

Combine with Embeddings