Use Munito AI models in LlamaIndex through OpenAILike, with chat, streaming, tools and structured output.
To use Munito AI in LlamaIndex, create an OpenAILike model from llama-index-llms-openai-like with the Munito AI base URL and your Munito AI key.
Completion, chat, streaming, tool calls and structured output then work as with any LlamaIndex LLM.
Munito AI has no embeddings endpoint. Take the embeddings for a vector index from another source, and use Munito AI for the LLM.
pip install llama-index-llms-openai-like
export MUNITO_API_KEY="munitoai_..."Complete script
# pip install llama-index-llms-openai-like
import os
from llama_index.core.llms import ChatMessage
from llama_index.core.prompts import PromptTemplate
from llama_index.core.tools import FunctionTool
from llama_index.llms.openai_like import OpenAILike
from pydantic import BaseModel
llm = OpenAILike(
model="qwen/qwen3.8-27b",
api_base="https://api.munito.ai/inference/v1",
api_key=os.environ["MUNITO_API_KEY"],
context_window=262144,
max_tokens=2048,
is_chat_model=True,
is_function_calling_model=True,
additional_kwargs={"reasoning_effort": "low"},
timeout=120,
)
# 1. Complete
print(llm.complete("Name the capital of Portugal in one word.").text.strip())
# 2. Chat
messages = [
ChatMessage(role="system", content="Answer in one short sentence."),
ChatMessage(role="user", content="What is OCR?"),
]
print(llm.chat(messages).message.content.strip())
# 3. Stream
for delta in llm.stream_complete("Count from 1 to 5, one number on each line."):
print(delta.delta, end="", flush=True)
print()
# 4. Tools
def get_weather(city: str) -> str:
"""Return the current weather for a city."""
return f"It is 18 degrees and sunny in {city}."
response = llm.predict_and_call([FunctionTool.from_defaults(fn=get_weather)], "What is the weather in Lyon?")
print(response.response)
# 5. Structured output
class City(BaseModel):
name: str
country: str
population: int
print(llm.structured_predict(City, PromptTemplate("Give the facts for {city}."), city="Munich"))The output:
Lisbon
OCR (Optical Character Recognition) is a technology that converts scanned or image-based text into machine-readable, editable digital text.
1
2
3
4
5
It is 18 degrees and sunny in Lyon.
name='Munich' country='Germany' population=1500000Settings
| Setting | Value |
|---|---|
api_base | https://api.munito.ai/inference/v1, or a regional host such as https://api.eu.munito.ai/inference/v1 |
model | A chat model from the model catalog |
context_window | The context of the model, from its model page. qwen/qwen3.8-27b has 262,144 tokens. |
is_chat_model | True, so that LlamaIndex calls /chat/completions |
is_function_calling_model | True for a model that supports tools, such as qwen/qwen3.8-27b |
max_tokens | A large value, such as 2048. The model reasons before it answers, and a small value can end the request before the answer. |
additional_kwargs | Extra request fields, such as {"reasoning_effort": "low"} |
default_headers | Munito AI headers, such as {"munito-service-tier": "flex"}. See Service tiers. |
Retrieval
LlamaIndex needs an embedding model for a VectorStoreIndex. Munito AI does not serve embeddings, so set Settings.embed_model to an embedding model from another source, and set Settings.llm to the OpenAILike model above.
For retrieval with no embeddings, see RAG, which uses a keyword index.