What stays inside
Agents work against the platform's own storage - the local columnar data lake and the document processing layer - not a copy exported to a cloud service. There is no export step in the analysis path.
What reaches a provider is a prompt: instructions, retrieved context, and the rows a tool returned within its cap. The engine applies row policies and column masks for the asking identity before anything becomes a tool result, so a prompt cannot contain what its caller may not see.
Provider choice
All providers are configured server-side by an administrator: hosted providers, Ollama for on-prem, and a Custom OpenAI-compatible endpoint for anything else - the entry a self-hosted checkpoint behind vLLM uses. Keys are AES-256 encrypted and never rendered back.
Per-agent model overrides make the choice granular: the persona reading sensitive records can run on a local model while another runs hosted.
Two rules for personal data
- Pre-dispatch: a guardrail blocks an attempt to send personal data outward to a third-party channel - a messenger notification, a proxied external tool.
- Post-dispatch: arguments and output pass through redaction before they reach the audit trail.
Two mechanisms, because transmitting outward and recording internally are different risks with different correct answers.
Self-hosted sizing
Measured context usage. Real per-turn consumption against this pipeline is ~10-14K tokens typical, ~28-32K peak (base prompt, tool schemas in scope, retrieved knowledge, trimmed history, this turn's tool results). A native 32K context is sufficient; extended-context configuration adds nothing. Sizing off the registry's 128K default would over-provision.
Concurrency. 5-100 registered users with irregular usage produces ~5-10 concurrent sessions, not the full roster.
| Tier | GPU | Model | Serving |
|---|---|---|---|
| Recommended | 1 x 48 GB (L40S / A6000 Ada) | 32B-class instruct, AWQ INT4 | vLLM, FP8 KV cache |
| Budget | 1 x 24 GB (L4) | 14B-class instruct, AWQ INT4 | vLLM |
Why FP8 KV cache is the deciding lever. 10 sessions x ~20K tokens = 200K tokens of cache. At FP16 that is ~51 GB and does not fit a 48 GB card with weights; at FP8 it is ~26 GB, ~44 GB total with INT4 weights, which fits with headroom.
The 14B-class budget tier is measurably weaker on multi-step tool-call reliability, which is the dominant pattern in this pipeline.
Upgrading is a checkpoint swap. CPU, RAM, storage and network do not scale with model size here; GPU memory is the only differentiator between tiers.
The resulting configuration
Own model, own hardware, own data, with the same agents, tool plane, gate and trace as a hosted deployment. It connects as a Custom provider - the same integration seam as any other.
See Deployment for how a version reaches that installation, and Environments for what it is licensed to run.