# Prompt caching
URL: https://docs.munito.ai/prompt-caching

Munito AI caches prompts automatically. When a request starts with the same tokens as an earlier request, the model reads that prefix from the cache, and you pay the lower cached-input price for it.
You change nothing in the request, and Munito AI charges no fee to write the cache.

The OpenAI SDK uses the base URL `https://api.munito.ai/inference/v1`, and the Anthropic SDK uses `https://api.munito.ai/inference`. Both read the key from `MUNITO_API_KEY`.

## Price [#price]

| Tokens                      | Price                                   |
| --------------------------- | --------------------------------------- |
| Input tokens from the cache | The cached-input price of the model     |
| Other input tokens          | The input price of the model            |
| Tokens written to the cache | No extra fee. They cost the input price |

[Models and pricing](https://docs.munito.ai/models#prices) lists the input, cached-input and output price of each model.

## The usage fields [#the-usage-fields]

| API              | Cached input tokens                         | Included in                                      |
| ---------------- | ------------------------------------------- | ------------------------------------------------ |
| Chat Completions | `usage.prompt_tokens_details.cached_tokens` | `usage.prompt_tokens`                            |
| Responses        | `usage.input_tokens_details.cached_tokens`  | `usage.input_tokens`                             |
| Messages         | `usage.cache_read_input_tokens`             | Not in `usage.input_tokens`. Add the two fields. |

On Messages, `usage.cache_creation_input_tokens` counts the tokens that the request wrote to the cache. Munito AI bills them at the normal input price.

## See the cache on Chat Completions [#see-the-cache-on-chat-completions]

```bash
pip install openai
```

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.munito.ai/inference/v1",
    api_key=os.environ["MUNITO_API_KEY"],
)
rules = " ".join(f"Rule {i}: part P-{i:04d} fits frame size {i % 7 + 1}." for i in range(400))
system = "You answer questions about bicycle parts. " + rules

for question in ["Which part fits frame size 3?", "Which part fits frame size 5?"]:
    response = client.chat.completions.create(
        model="qwen/qwen3.8-27b",
        messages=[
            {"role": "system", "content": system},
            {"role": "user", "content": question},
        ],
        prompt_cache_key="bicycle-parts",
        reasoning_effort="low",
        max_tokens=2048,
    )
    usage = response.usage
    print(usage.prompt_tokens, "input tokens,", usage.prompt_tokens_details.cached_tokens, "from the cache")
```

The first request usually reads 0 tokens from the cache. The second request reads a large part of the system prompt from the cache.

## See the cache on Messages [#see-the-cache-on-messages]

```bash
pip install anthropic
```

```python
import os
from anthropic import Anthropic

client = Anthropic(
    base_url="https://api.munito.ai/inference",
    api_key=os.environ["MUNITO_API_KEY"],
)
rules = " ".join(f"Rule {i}: part P-{i:04d} fits frame size {i % 7 + 1}." for i in range(400))
system = "You answer questions about bicycle parts. " + rules

for question in ["Which part fits frame size 3?", "Which part fits frame size 5?"]:
    message = client.messages.create(
        model="qwen/qwen3.8-27b",
        max_tokens=2048,
        system=system,
        messages=[{"role": "user", "content": question}],
        metadata={"user_id": "bicycle-parts"},
    )
    usage = message.usage
    print(usage.input_tokens, "new,", usage.cache_read_input_tokens, "from the cache,", usage.cache_creation_input_tokens, "written")
```

You do not need `cache_control` markers. The cache works without them.

## Get more cache hits [#get-more-cache-hits]

The cache matches a prompt from its first token. One changed token ends the match, and the model reads all tokens after it again.

- Put the parts that stay the same first: the system prompt, the tool definitions, long documents and examples.
- Put the parts that change last: the question of the user, the date, a request ID.
- Keep the order and the text of the fixed parts exactly the same in each request. Do not put a timestamp in the system prompt.
- In a conversation, add new turns at the end, and send the earlier turns unchanged.
- Send the same `prompt_cache_key` (or `user`) with each request that shares a prefix. On Messages, send the same `metadata.user_id`. Munito AI uses this value to send the requests to where the prefix is cached.
  Agent harnesses such as pi and OpenClaw send the header `x-session-affinity` instead, and Munito AI uses it when a request has no `prompt_cache_key`.

The cache keeps a prefix for each model separately. A cache hit is likely but not certain, so the cached count can change from one request to the next, and it can be lower than the length of the shared prefix.
`prompt_cache_retention` returns `400`, because the cache retention is not configurable.

See the [chat reference](https://docs.munito.ai/api/createChatCompletion) and the [Messages reference](https://docs.munito.ai/api/createMessage).