Production checklist

Before you send production traffic to Munito AI, set retries and timeouts in the client, choose a service tier and a region, set spend limits, and handle the answer of a reasoning model. Each item below links to the page with the details.

The base URL is https://api.munito.ai/inference/v1 for the OpenAI SDK, and https://api.munito.ai/inference for the Anthropic SDK. The key is in MUNITO_API_KEY.

A production clientCopy Anchor Link

pip install openai
export MUNITO_API_KEY="munitoai_..."

# pip install openai
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.eu.munito.ai/inference/v1",
    api_key=os.environ["MUNITO_API_KEY"],
    max_retries=5,  # retries 408, 409, 429 and 5xx, and follows Retry-After
    timeout=900,  # seconds; longer than your longest generation
)
response = client.chat.completions.create(
    model="qwen/qwen3.8-27b",
    messages=[{"role": "user", "content": "Write one sentence about Lisbon."}],
    max_tokens=2048,
    extra_headers={"munito-service-tier": "priority", "munito-overflow": "model"},
)
message = response.choices[0].message
if message.content is None:
    print("No answer text. finish_reason:", response.choices[0].finish_reason)
else:
    print(message.content.strip())

ChecklistCopy Anchor Link

RetriesCopy Anchor Link

The OpenAI and Anthropic SDKs retry 429 and 5xx answers, and wait for the Retry-After header. The default is 2 retries, so set max_retries higher for batch work. In your own HTTP client, use this rule:

The inference API has no idempotency key. A retried POST is a new request, and Munito AI bills each request that it answers. For OCR and background requests, poll the location that you already have. Do not send the POST again.

Timeouts and background requestsCopy Anchor Link

A long generation can take minutes. Set the client timeout longer than your longest generation: timeout=900 in the OpenAI and Anthropic SDKs, for example. A background request does not hold the connection open. Send long or batch work as a background request:

# pip install openai
import os
import time

from openai import OpenAI

client = OpenAI(base_url="https://api.munito.ai/inference/v1", api_key=os.environ["MUNITO_API_KEY"])
response = client.responses.create(
    model="qwen/qwen3.8-27b",
    input="Summarize the history of the printing press in five sentences.",
    max_output_tokens=4096,
    background=True,
    service_tier="flex",
)
while response.status in ("queued", "in_progress"):
    time.sleep(2)
    response = client.responses.retrieve(response.id)
print(response.status)
print(response.output_text)

A flex request waits in line for capacity when you send it in the background. Overflow does not apply to flex or background requests.

Spend limitsCopy Anchor Link

Owners and administrators set a daily and a monthly spend limit on the Billing page of the console. When your spend reaches a limit, each request gets 429 with The spend limit is reached. Access resumes at ..., and no Retry-After header. The OpenAI error body has the code organization_spend_limit_exceeded. To read the spend of today and of this month in your own tools, call GET /admin/v1/limits with an Admin API key. See Administration.

StreamingCopy Anchor Link

Set stream=True for an answer that a user reads, so that the first words show at once. A stream also keeps the connection active during a long generation. The usage of a streamed request comes in the last chunk when you send stream_options={"include_usage": True}.

Reasoning models and null contentCopy Anchor Link

qwen/qwen3.8-27b and zai-org/glm-5.3-flash reason before they answer. The reasoning counts against max_tokens. With a small max_tokens, the request can end before the answer starts. The message then has "content": null, the reasoning is in reasoning, and finish_reason is length:

{
  "finish_reason": "length",
  "message": {
    "role": "assistant",
    "content": null,
    "reasoning": "We need to respond to user..."
  }
}

To prevent it, use a large max_tokens, such as 2048 or more, or send reasoning_effort: "low". Always check content for null before you use it.

KeysCopy Anchor Link

• Create one key for each service, so that you can rotate one key without a stop of the others.
• Give a key only the permissions that it needs. A key without the Inference permission gets 403 with This API key does not have the inference permission.
• Keep the key on your server. Do not put it in a browser or a mobile app.