# LlamaIndex
URL: https://docs.munito.ai/llamaindex

To use Munito AI in LlamaIndex, create an `OpenAILike` model from `llama-index-llms-openai-like` with the Munito AI base URL and your Munito AI key.
Completion, chat, streaming, tool calls and structured output then work as with any LlamaIndex LLM.

Munito AI has no embeddings endpoint. Take the embeddings for a vector index from another source, and use Munito AI for the LLM.

```bash
pip install llama-index-llms-openai-like
export MUNITO_API_KEY="munitoai_..."
```

## Complete script [#complete-script]

```python
# pip install llama-index-llms-openai-like
import os

from llama_index.core.llms import ChatMessage
from llama_index.core.prompts import PromptTemplate
from llama_index.core.tools import FunctionTool
from llama_index.llms.openai_like import OpenAILike
from pydantic import BaseModel

llm = OpenAILike(
    model="qwen/qwen3.8-27b",
    api_base="https://api.munito.ai/inference/v1",
    api_key=os.environ["MUNITO_API_KEY"],
    context_window=262144,
    max_tokens=2048,
    is_chat_model=True,
    is_function_calling_model=True,
    additional_kwargs={"reasoning_effort": "low"},
    timeout=120,
)

# 1. Complete
print(llm.complete("Name the capital of Portugal in one word.").text.strip())

# 2. Chat
messages = [
    ChatMessage(role="system", content="Answer in one short sentence."),
    ChatMessage(role="user", content="What is OCR?"),
]
print(llm.chat(messages).message.content.strip())

# 3. Stream
for delta in llm.stream_complete("Count from 1 to 5, one number on each line."):
    print(delta.delta, end="", flush=True)
print()


# 4. Tools
def get_weather(city: str) -> str:
    """Return the current weather for a city."""
    return f"It is 18 degrees and sunny in {city}."


response = llm.predict_and_call([FunctionTool.from_defaults(fn=get_weather)], "What is the weather in Lyon?")
print(response.response)


# 5. Structured output
class City(BaseModel):
    name: str
    country: str
    population: int


print(llm.structured_predict(City, PromptTemplate("Give the facts for {city}."), city="Munich"))
```

The output:

```text
Lisbon
OCR (Optical Character Recognition) is a technology that converts scanned or image-based text into machine-readable, editable digital text.
1
2
3
4
5
It is 18 degrees and sunny in Lyon.
name='Munich' country='Germany' population=1500000
```

## Settings [#settings]

| Setting                     | Value                                                                                                                          |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| `api_base`                  | `https://api.munito.ai/inference/v1`, or a [regional host](https://docs.munito.ai/regions) such as `https://api.eu.munito.ai/inference/v1` |
| `model`                     | A chat model from the [model catalog](https://docs.munito.ai/models)                                                                       |
| `context_window`            | The context of the model, from its model page. `qwen/qwen3.8-27b` has 262,144 tokens.                                          |
| `is_chat_model`             | `True`, so that LlamaIndex calls `/chat/completions`                                                                           |
| `is_function_calling_model` | `True` for a model that supports tools, such as `qwen/qwen3.8-27b`                                                             |
| `max_tokens`                | A large value, such as 2048. The model reasons before it answers, and a small value can end the request before the answer.     |
| `additional_kwargs`         | Extra request fields, such as `{"reasoning_effort": "low"}`                                                                    |
| `default_headers`           | Munito AI headers, such as `{"munito-service-tier": "flex"}`. See [Service tiers](https://docs.munito.ai/service-tiers).                   |

## Retrieval [#retrieval]

LlamaIndex needs an embedding model for a `VectorStoreIndex`. Munito AI does not serve embeddings, so set `Settings.embed_model` to an embedding model from another source, and set `Settings.llm` to the `OpenAILike` model above.
For retrieval with no embeddings, see [RAG](https://docs.munito.ai/rag), which uses a keyword index.