Prompt caching

Munito AI caches prompts automatically. When a request starts with the same tokens as an earlier request, the model reads that prefix from the cache, and you pay the lower cached-input price for it. You change nothing in the request, and Munito AI charges no fee to write the cache.

The OpenAI SDK uses the base URL https://api.munito.ai/inference/v1, and the Anthropic SDK uses https://api.munito.ai/inference. Both read the key from MUNITO_API_KEY.

PriceCopy Anchor Link

Models and pricing lists the input, cached-input and output price of each model.

The usage fieldsCopy Anchor Link

On Messages, usage.cache_creation_input_tokens counts the tokens that the request wrote to the cache. Munito AI bills them at the normal input price.

See the cache on Chat CompletionsCopy Anchor Link

pip install openai

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.munito.ai/inference/v1",
    api_key=os.environ["MUNITO_API_KEY"],
)
rules = " ".join(f"Rule {i}: part P-{i:04d} fits frame size {i % 7 + 1}." for i in range(400))
system = "You answer questions about bicycle parts. " + rules

for question in ["Which part fits frame size 3?", "Which part fits frame size 5?"]:
    response = client.chat.completions.create(
        model="qwen/qwen3.8-27b",
        messages=[
            {"role": "system", "content": system},
            {"role": "user", "content": question},
        ],
        prompt_cache_key="bicycle-parts",
        reasoning_effort="low",
        max_tokens=2048,
    )
    usage = response.usage
    print(usage.prompt_tokens, "input tokens,", usage.prompt_tokens_details.cached_tokens, "from the cache")

The first request usually reads 0 tokens from the cache. The second request reads a large part of the system prompt from the cache.

See the cache on MessagesCopy Anchor Link

pip install anthropic

import os
from anthropic import Anthropic

client = Anthropic(
    base_url="https://api.munito.ai/inference",
    api_key=os.environ["MUNITO_API_KEY"],
)
rules = " ".join(f"Rule {i}: part P-{i:04d} fits frame size {i % 7 + 1}." for i in range(400))
system = "You answer questions about bicycle parts. " + rules

for question in ["Which part fits frame size 3?", "Which part fits frame size 5?"]:
    message = client.messages.create(
        model="qwen/qwen3.8-27b",
        max_tokens=2048,
        system=system,
        messages=[{"role": "user", "content": question}],
        metadata={"user_id": "bicycle-parts"},
    )
    usage = message.usage
    print(usage.input_tokens, "new,", usage.cache_read_input_tokens, "from the cache,", usage.cache_creation_input_tokens, "written")

You do not need cache_control markers. The cache works without them.

Get more cache hitsCopy Anchor Link

The cache matches a prompt from its first token. One changed token ends the match, and the model reads all tokens after it again.

• Put the parts that stay the same first: the system prompt, the tool definitions, long documents and examples.
• Put the parts that change last: the question of the user, the date, a request ID.
• Keep the order and the text of the fixed parts exactly the same in each request. Do not put a timestamp in the system prompt.
• In a conversation, add new turns at the end, and send the earlier turns unchanged.
• Send the same prompt_cache_key (or user) with each request that shares a prefix. On Messages, send the same metadata.user_id. Munito AI uses this value to send the requests to where the prefix is cached. Agent harnesses such as pi and OpenClaw send the header x-session-affinity instead, and Munito AI uses it when a request has no prompt_cache_key.

The cache keeps a prefix for each model separately. A cache hit is likely but not certain, so the cached count can change from one request to the next, and it can be lower than the length of the shared prefix. prompt_cache_retention returns 400, because the cache retention is not configurable.

See the chat reference and the Messages reference.