Overflow
Let another model or provider answer a chat request when capacity is full.
Overflow is off by default, and Munito never uses it without your permission.
When the capacity for a model is full, Munito answers with 503 Service Overloaded. With overflow, Munito can answer the request with the same model at a vetted provider, or with another model.
Opt in for a request with the munito-overflow header:
| Value | What can answer |
|---|---|
off | Nothing. This is the default. |
model | The same model, at a provider in your region. |
class | Also a similar model. |
any | Also any other model. |
The request keeps its region, zero data retention, and the price of the model that you asked for. When overflow answers, the response carries the munito-overflow header, and its model field names the model that answered.
Overflow applies to Chat Completions, Messages and Responses, except on the free tier and for flex and background requests.