# Production checklist
URL: https://docs.munito.ai/production-checklist

Before you send production traffic to Munito AI, set retries and timeouts in the client, choose a service tier and a region, set spend limits, and handle the answer of a reasoning model.
Each item below links to the page with the details.

The base URL is `https://api.munito.ai/inference/v1` for the OpenAI SDK, and `https://api.munito.ai/inference` for the Anthropic SDK. The key is in `MUNITO_API_KEY`.

## A production client [#a-production-client]

```bash
pip install openai
export MUNITO_API_KEY="munitoai_..."
```

```python
# pip install openai
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.eu.munito.ai/inference/v1",
    api_key=os.environ["MUNITO_API_KEY"],
    max_retries=5,  # retries 408, 409, 429 and 5xx, and follows Retry-After
    timeout=900,  # seconds; longer than your longest generation
)
response = client.chat.completions.create(
    model="qwen/qwen3.8-27b",
    messages=[{"role": "user", "content": "Write one sentence about Lisbon."}],
    max_tokens=2048,
    extra_headers={"munito-service-tier": "priority", "munito-overflow": "model"},
)
message = response.choices[0].message
if message.content is None:
    print("No answer text. finish_reason:", response.choices[0].finish_reason)
else:
    print(message.content.strip())
```

## Checklist [#checklist]

| Item             | What to do                                                                                        | Details                                   |
| ---------------- | ------------------------------------------------------------------------------------------------- | ----------------------------------------- |
| Retries          | Retry `429`, `502` and `503` after the `Retry-After` time. Do not retry other `4xx` answers.      | [Errors](https://docs.munito.ai/errors)               |
| Timeouts         | Set a client timeout longer than your longest generation. Send long work as a background request. | Below                                     |
| Service tier     | Send `priority` for traffic that a user waits for, and `flex` for batch work.                     | [Service tiers](https://docs.munito.ai/service-tiers) |
| Overflow         | Opt in with `munito-overflow` if another provider or model may answer when capacity is full.      | [Overflow](https://docs.munito.ai/overflow)           |
| Region           | Use `api.eu.munito.ai` or `api.us.munito.ai` to keep processing in one region.                    | [Regions](https://docs.munito.ai/regions)             |
| Spend limits     | Set a daily and a monthly limit in the console.                                                   | Below                                     |
| Streaming        | Stream answers that a user reads.                                                                 | Below                                     |
| Reasoning models | Use a large `max_tokens`, and handle `content: null`.                                             | Below                                     |
| Rate limits      | Increase traffic gradually, and read the `x-ratelimit-*` headers.                                 | [Rate limits](https://docs.munito.ai/rate-limits)     |

## Retries [#retries]

The OpenAI and Anthropic SDKs retry `429` and `5xx` answers, and wait for the `Retry-After` header. The default is 2 retries, so set `max_retries` higher for batch work.
In your own HTTP client, use this rule:

| Status                                                                                        | Retry?                                           |
| --------------------------------------------------------------------------------------------- | ------------------------------------------------ |
| `429` with `Too Many Requests` or `Rate limit exceeded and no spare capacity. Retry shortly.` | Yes, after `Retry-After`                         |
| `429` with `The spend limit is reached. Access resumes at ...`                                | No. Raise the spend limit, or wait for the date. |
| `502`, `503`                                                                                  | Yes, after `Retry-After`, with backoff           |
| `400`, `401`, `403`, `404`, `413`, `421`, `422`                                               | No. Fix the request.                             |

The inference API has no idempotency key. A retried `POST` is a new request, and Munito AI bills each request that it answers.
For OCR and background requests, poll the `location` that you already have. Do not send the `POST` again.

## Timeouts and background requests [#timeouts-and-background-requests]

A long generation can take minutes. Set the client timeout longer than your longest generation: `timeout=900` in the OpenAI and Anthropic SDKs, for example.
A background request does not hold the connection open. Send long or batch work as a background request:

| API              | Start                                                                  | Poll                                                                                     |
| ---------------- | ---------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| Chat Completions | Header `Prefer: respond-async`. The answer is `202` with a `location`. | `GET /chat/completions/{id}` answers `202` while it runs, then `200` with the completion |
| Responses        | Field `background: true`                                               | `GET /responses/{id}` until `status` is not `queued` or `in_progress`                    |
| OCR              | Header `Prefer: respond-async`                                         | `GET /ocr/{request_id}`. See [PDF to Markdown](https://docs.munito.ai/pdf-to-markdown).              |

```python
# pip install openai
import os
import time

from openai import OpenAI

client = OpenAI(base_url="https://api.munito.ai/inference/v1", api_key=os.environ["MUNITO_API_KEY"])
response = client.responses.create(
    model="qwen/qwen3.8-27b",
    input="Summarize the history of the printing press in five sentences.",
    max_output_tokens=4096,
    background=True,
    service_tier="flex",
)
while response.status in ("queued", "in_progress"):
    time.sleep(2)
    response = client.responses.retrieve(response.id)
print(response.status)
print(response.output_text)
```

A flex request waits in line for capacity when you send it in the background. Overflow does not apply to flex or background requests.

## Spend limits [#spend-limits]

Owners and administrators set a daily and a monthly spend limit on the Billing page of the [console](https://console.munito.ai/billing).
When your spend reaches a limit, each request gets `429` with `The spend limit is reached. Access resumes at ...`, and no `Retry-After` header. The OpenAI error body has the code `organization_spend_limit_exceeded`.
To read the spend of today and of this month in your own tools, call [`GET /admin/v1/limits`](https://docs.munito.ai/api/getSpendLimits) with an Admin API key. See [Administration](https://docs.munito.ai/administration).

## Streaming [#streaming]

Set `stream=True` for an answer that a user reads, so that the first words show at once. A stream also keeps the connection active during a long generation.
The usage of a streamed request comes in the last chunk when you send `stream_options={"include_usage": True}`.

## Reasoning models and null content [#reasoning-models-and-null-content]

`qwen/qwen3.8-27b` and `zai-org/glm-5.3-flash` reason before they answer. The reasoning counts against `max_tokens`.
With a small `max_tokens`, the request can end before the answer starts. The message then has `"content": null`, the reasoning is in `reasoning`, and `finish_reason` is `length`:

```json
{
  "finish_reason": "length",
  "message": { "role": "assistant", "content": null, "reasoning": "We need to respond to user..." }
}
```

To prevent it, use a large `max_tokens`, such as 2048 or more, or send `reasoning_effort: "low"`. Always check `content` for `null` before you use it.

## Keys [#keys]

- Create one key for each service, so that you can rotate one key without a stop of the others.
- Give a key only the permissions that it needs. A key without the **Inference** permission gets `403` with `This API key does not have the inference permission.`
- Keep the key on your server. Do not put it in a browser or a mobile app.