On-premise RAG and AI assistant for companies

DocsGPT runs on-premises in your data centre or private cloud with Docker Compose or Kubernetes manifests, using a fully local stack or, in hybrid mode, a cloud LLM provider of your choice while documents stay inside your network. It runs on the GPUs you already have — the bank reference runs on existing NVIDIA T4s, in the stricter air-gapped posture — under the MIT licence, with no per-seat licence.

Book a demo Free 30-minute on-prem review

docker compose · kubernetes manifests · 8 local engines: llama.cpp / ollama / tgi / sglang / vllm / aphrodite / friendliai / lmdeploy · pgvector · oidc + scim · MIT

Last updated: 2026-09-19

Deployment posturesoperator · data boundary · LLM

Cloud
operatorArc53
data boundaryArc53 multi-tenant cloudEU region
network boundary
llm locationCloud provider
Managed
operatorArc53
data boundaryDedicated VPSyour region
network boundary
llm locationCloud provider or local engineor your own key (BYOM)
On-premises
operatorCustomer
data boundaryYour network
egress optional (hybrid)
llm locationLocal engineor your cloud LLM (hybrid)
Air-gapped
operatorCustomer
data boundaryYour networkno egress
no external network
llm locationLocal engine only

Text version

Cloud: Arc53 operates a multi-tenant service and the LLM runs at a cloud provider. Managed: Arc53 operates a dedicated instance in your chosen region; the LLM runs at a cloud provider, on a local engine on the instance, or with your own key (BYOM). On-premises: you operate DocsGPT inside your network; the LLM runs locally, or (hybrid) prompts cross the boundary to a cloud provider you choose. Air-gapped: zero external network; LLM, embeddings, vector store and storage all run inside your perimeter.

Fully local stack

Every layer has a local option; nothing in the platform requires an external service.

Every layer of an on-premises DocsGPT deployment has a local option: inference through any of the eight engines setup.sh offers; embeddings on FastEmbed with ONNX Runtime, sentence-transformers optional; pgvector, Qdrant, Milvus, Elasticsearch, LanceDB or MongoDB for vectors; local disk for files, with S3-compatible storage optional; PostgreSQL for user data; Celery and Redis for jobs; OIDC and SCIM for identity; OpenTelemetry for observability, off by default.

Components of a fully local on-premises DocsGPT stack
	Components	Notes
Inference engines	llama.cpp · Ollama · TGI · SGLang · vLLM · Aphrodite · FriendliAI · LMDeploy	Eight engines offered by setup.sh, plus any OpenAI-compatible endpoint
Embeddings	FastEmbed on ONNX Runtime	Default granite-311m on fresh installs; installs with an existing index stay on all-mpnet-base-v2 via sentence-transformers. Any Sentence Transformers model works — see the dimension note below
Vector store	pgvector · Qdrant · Milvus · Elasticsearch · LanceDB · MongoDB	pgvector is required for hybrid search and GraphRAG; every store honours score_threshold
File storage	Local disk	Default STORAGE_TYPE=local — no object-storage service required. Any S3-compatible store (MinIO, Ceph, an internal S3) can be used instead
User-data store	PostgreSQL	Sole store since v0.17; managed Postgres supported
Background jobs	Celery + Redis	Ingestion, scheduled agents, workflows
Identity	OIDC + SCIM 2.0	Okta, Keycloak, Authentik, Entra ID documented
Observability	OpenTelemetry	Off by default; exports to Grafana, Jaeger, Datadog and others

Embedding dimensions must stay consistent. Changing the embedding model changes the vector dimension, so an existing index cannot be read by the new model — every document has to be re-ingested. Choose the embedding model before the first ingest, and keep EMBEDDINGS_NAME fixed afterwards.

Hybrid: on-premises platform, your cloud LLM

DocsGPT runs inside the customer's network and calls a cloud LLM provider chosen by the customer; documents, embeddings and the vector store stay on-premises, prompts and retrieved passages leave to the provider.

Stays inside your network

Documents, retrieval and users

Source documents, embeddings, the vector store, user accounts, roles and logs never leave the customer's network.

documents · embeddings · vector store · users · logs

Crosses to your provider

Prompt and retrieved passages

Only the prompt and the passages retrieved for it are sent to the cloud LLM provider, under the customer's own key.

prompt · retrieved passages · your API key

providers: OpenAI · Azure OpenAI · Anthropic · Google Vertex · Groq · OpenRouter · any OpenAI-compatible endpoint

Hybrid is not air-gapped. If policy forbids any external network, see Air-gapped.

How to run AI on premise: installation

Five steps from an empty environment to a benchmarked deployment.

01
Install with the one-line installer or Kubernetes
Run curl -fsSL https://docs.ac/install | bash on the host and answer “OpenAI-compatible server” to the provider question with the local engine's endpoint; a server sets up without questions with bash -s -- --yes --domain … --provider openai-compatible …. Or apply the raw Kubernetes manifests in deployment/k8s with kubectl apply -k (no Helm chart; -f is not recursive). ARM builds are available.
curl -fsSL https://docs.ac/install | bash · kubectl apply -k deployment/k8s
02
Point the model registry at the local engine
Register the engine's endpoint; models hot-swap without a restart.
LLM_PROVIDER=ollama · vllm · llama.cpp · tgi · sglang · aphrodite · friendliai · lmdeploy
03
Connect the IdP and enable SCIM
OIDC single sign-on with group allowlists; SCIM 2.0 for provisioning.
AUTH_TYPE=oidc · scim 2.0
04
Connect sources
SharePoint, Google Drive, Confluence, S3 and uploads, with OCR for scanned documents.
sharepoint · google drive · confluence · s3 · ocr
05
Benchmark before rollout
Run a benchmark suite as the acceptance test.
docsgpt-cli bench

Settings › Models

model registry3 models · hot-swap, no restart

Qwen3.8-Flash-Next
Ollama · http://ollama:11434
defaultlocal
GLM-5.3-Flash
vLLM · http://gpu-01:8000/v1
local
GPT-5.6 Sol
DocsGPT · api key · via Arc53
cloud
+ Add model bring your own · openai-compatible
base url https://…/v1api key ••••••••model id my-model

Documentation

Runs on the GPUs you already have

Model choice is benchmark-driven, so the pipeline is sized to the hardware in the rack rather than to a procurement request.

For constrained hardware the model is chosen by running the customer's own benchmark suite against the candidates. In one bank proof of concept, Qwen 3.6 was selected from five-plus open-source models on custom benchmarks; in production the classification pipeline is sized to legacy NVIDIA T4 GPUs.

That bank runs air-gapped — the same on-premises stack with a zero-egress rule and a local-only model. The same sizing applies to either posture.

Hardware fit and sizing by model class →

See also Models & BYOM · Banking · Government.

24 h → 16 min

A large Central European commercial bank processes 10,000+ inbound documents a day; time-to-action fell from 24 hours to 16 minutes, with over 99.5 % classification accuracy, running fully air-gapped on the bank's existing NVIDIA T4 GPUs.

Source: EU bank, name under NDA · air-gapped deployment

Government infrastructure, local inference

UK Department for Work and Pensions: an internal knowledge assistant on government infrastructure, with local inference.

Source: DWP

Identity, roles and audit exports

SSO, provisioning, roles, append-only audit, guardrail events and approval gates — all in the open-source build.

SSO

OIDC single sign-on

Authorization Code + PKCE, group allowlists, admin-group mapping and back-channel logout against your identity provider.

AUTH_TYPE=oidc · Okta · Keycloak · Authentik · Entra ID

Provisioning

SCIM 2.0

User lifecycle driven by the IdP: accounts are created, deactivated and reactivated from your directory. Deprovisioning revokes sessions immediately.

scim 2.0 · users · immediate revocation

Roles

Global roles plus per-object sharing

User and admin roles across the workspace; Viewer / Editor sharing per agent and per source, applied at retrieval time.

admin · user · viewer · editor

Audit

Append-only audit table

Authentication events are recorded append-only and can be exported for compliance review.

auth-events · append-only · exportable

Guardrails

Guardrail events audit

PII, secrets, denylist and prompt-injection checks with flag, redact or block actions; every event lands in an audit panel.

pii · secrets · denylist · prompt-injection · flag/redact/block

Approval

Human-approval gates

Tool calls and responses can require a person to approve before they run or leave the system.

approve · reject · tool-call gate

Admin › Users: nine accounts filterable by user id, each with its tags (Admin, Inactive), when it was last seen and when it was created. The row menu offers View details, Make admin, Deactivate and Force logout. — app · Admin › Users

Detail: Admin, Security & Analytics · Security.

The one-week POC

Your environment, your data, your model. A one-week proof of concept with a benchmark suite as the acceptance test — not a twelve-month consulting engagement.

The one-week POCday 1 → day 5

Text version

Day 1 — Install: DocsGPT installed in the customer's environment. Days 2–3 — Connect: Two or three sources and the identity provider connected. Days 3–5 — Build: First agent or workflow, with a benchmark suite. Day 5 — Review: Reviewed against the acceptance checks. Your environment, your data, your model — a one-week proof of concept, not a twelve-month consulting engagement.

1–2 days → ~5 min

Transfer-document review that took 1–2 working days (P95) now completes in about 5 minutes, with 12 deterministic and agent checks and human approval for uncertain cases.

Source: Bank (POC), name under NDA

Frequently asked questions

How to run AI on premise?: Install with the one-line installer or Kubernetes, point it at a local inference engine (llama.cpp, Ollama, vLLM, TGI and others), use pgvector or Qdrant for retrieval and connect your identity provider. A working proof of concept typically takes one week.
Can I deploy LLM locally?: Yes. The installer offers eight local inference engines — llama.cpp, Ollama, TGI, SGLang, vLLM, Aphrodite, FriendliAI and LMDeploy — as well as any OpenAI-compatible endpoint; the model is registered in the model registry and can be swapped without a restart.
Which GPU is best for running LLM?: It depends on the model size you need. A large bank runs DocsGPT's document classification in production, air-gapped, on existing NVIDIA T4 GPUs (16 GB); larger models need more VRAM. The hardware guide gives sizing by model class.
Is hybrid deployment the same as air-gapped?: No. Hybrid keeps documents and retrieval on-premises but sends prompts to a cloud model provider you choose. Air-gapped means nothing leaves the network.
Do you charge per user for on-premises?: No. On-premises is priced from $10,000, either as a one-time fee for the one-week POC and implementation or as a recurring fee that adds production support; the software is MIT-licensed and there is no per-seat licence.

Bring your infrastructure team

A 30-minute demo on your documents, or a free on-premises review: hardware and model recommendation and a stack blueprint from an Arc53 engineer.

Book a demo Free 30-minute on-prem review Air-gapped deployment