# Privacy and on-prem

> Agents run against storage inside the contour, on-prem and OpenAI-compatible providers are first-class, personal data has two separate rules, and self-hosting has a measured sizing answer

Source: https://aipril.dev/docs/harness/privacy

## What stays inside

Agents work against the platform's own storage - the local columnar data lake and the document
processing layer - not a copy exported to a cloud service. There is no export step in the analysis
path.

What reaches a provider is a prompt: instructions, retrieved context, and the rows a tool returned
within its cap. The [engine](https://aipril.dev/docs/engine/under-the-hood.md) applies row policies and column masks for
the asking identity before anything becomes a tool result, so a prompt cannot contain what its caller
may not see.

## Provider choice

All providers are configured server-side by an administrator: hosted providers, **Ollama** for
on-prem, and a **Custom** OpenAI-compatible endpoint for anything else - the entry a self-hosted
checkpoint behind vLLM uses. Keys are AES-256 encrypted and never rendered back.

Per-agent model overrides make the choice granular: the persona reading sensitive records can run on
a local model while another runs hosted.

## Two rules for personal data

- **Pre-dispatch:** a guardrail blocks an attempt to send personal data outward to a third-party
  channel - a messenger notification, a proxied external tool.
- **Post-dispatch:** arguments and output pass through redaction before they reach the audit trail.

Two mechanisms, because transmitting outward and recording internally are different risks with
different correct answers.

## Self-hosted sizing

**Measured context usage.** Real per-turn consumption against this pipeline is ~10-14K tokens
typical, ~28-32K peak (base prompt, tool schemas in scope, retrieved knowledge, trimmed history,
this turn's tool results). A native 32K context is sufficient; extended-context configuration adds
nothing. Sizing off the registry's 128K default would over-provision.

**Concurrency.** 5-100 registered users with irregular usage produces ~5-10 concurrent sessions, not
the full roster.

| Tier | GPU | Model | Serving |
|---|---|---|---|
| Recommended | 1 x 48 GB (L40S / A6000 Ada) | 32B-class instruct, AWQ INT4 | vLLM, FP8 KV cache |
| Budget | 1 x 24 GB (L4) | 14B-class instruct, AWQ INT4 | vLLM |

**Why FP8 KV cache is the deciding lever.** 10 sessions x ~20K tokens = 200K tokens of cache. At
FP16 that is ~51 GB and does not fit a 48 GB card with weights; at FP8 it is ~26 GB, ~44 GB total
with INT4 weights, which fits with headroom.

The 14B-class budget tier is measurably weaker on multi-step tool-call reliability, which is the
dominant pattern in this pipeline.

**Upgrading is a checkpoint swap.** CPU, RAM, storage and network do not scale with model size here;
GPU memory is the only differentiator between tiers.

## The resulting configuration

Own model, own hardware, own data, with the same agents, tool plane, gate and
[trace](https://aipril.dev/docs/harness/traces.md) as a hosted deployment. It connects as a Custom provider - the same
integration seam as any other.

See [Deployment](https://aipril.dev/docs/platform/environments/deployment.md) for how a version reaches that installation,
and [Environments](https://aipril.dev/docs/platform/environments.md) for what it is licensed to run.