# Models and providers

> Server-side provider configuration with encrypted keys, a model registry with no hardcoded names, per-agent overrides, and a measured per-turn token profile

Source: https://aipril.dev/docs/harness/models-and-providers

## Providers

Configured server-side by an administrator, per organisation:

| Provider | Notes |
|---|---|
| OpenAI | Hosted |
| OpenRouter | Hosted, multi-model |
| Ollama | On-prem |
| Custom | Any OpenAI-compatible endpoint |

API keys are AES-256 encrypted at rest and never rendered back into the interface.

The Custom entry is what makes a self-hosted checkpoint behind vLLM a first-class provider. See
[Privacy and on-prem](https://aipril.dev/docs/harness/privacy.md).

## Model registry

No model identifier is compiled into the product. Models are registry entries an administrator
manages. Each carries:

- **Type** - `chat`, `embedding`, `decision` or `transcription`. Speech models are covered in
  [Transcription](https://aipril.dev/docs/harness/transcription.md).
- **Context window** - 128K default
- **Temperature** - around 0.7 for the final response, 0 for tool selection. Tool choice is a
  classification task, not a generative one.
- **Price per million tokens**, which is what lets cost be computed instead of reconciled
- **Per-agent override**, so the process-design persona and the chart persona need not share a model

Adding a model released this week is a registry entry; moving one agent onto it is one field.

## Embeddings

Standard is 1536 dimensions; a new organisation is provisioned at it. An existing organisation can
migrate to a different dimension through a dedicated, organisation-scoped path.

Without a configured embedding model, dataset scanning and contextual search report themselves
unavailable instead of silently degrading to text-only retrieval. See
[Data scanning and retrieval](https://aipril.dev/docs/harness/knowledge.md).

## Per-turn token profile

The registry accepts a 128K context window. That is a configurable ceiling, not a requirement.
Measured against the live pipeline, a chat turn is assembled from:

| Component | Typical | Peak |
|---|---|---|
| Base system prompt | ~4K | - |
| Tool schemas in scope | ~2-2.5K | ~4K |
| Retrieved knowledge context | ~0.5-1K | ~2-3K |
| Conversation history | ~2-6K | 12K cap |
| This turn's tool results | ~0.5-2K | ~6-10K |
| **Total** | **~10-14K** | **~28-32K** |

Two mechanisms hold it there as the catalogue grows:

- **Progressive tool disclosure.** A small core set is pre-activated per session; the rest is
  discovered on demand through a search tool with a clamped result count. See
  [Tools and MCP](https://aipril.dev/docs/harness/tools-and-mcp.md).
- **History budget with automatic summarisation.** History is trimmed to a token budget, and
  summarisation fires while the unsummarised tail is still below the ceiling, so steady state stays
  under the cap instead of repeatedly hitting it.

A native 32K context is sufficient for this pipeline; extended-context configuration adds nothing to
it.

## Rate limits

Per dimension, not one global number: chat requests per IP and per user, MCP requests per IP,
analysis requests per user, message size, rows returned per call, tool-calling rounds per turn, and
session history length with a TTL.

The tool-call round cap bounds a turn regardless of what the model decides to do inside it.