Skip to content
Start free

The private RAG backend for your product

DocsGPT is an open-source (MIT) RAG and agent backend you run in your own cloud behind your product: an OpenAI-compatible API, streaming, webhooks, white-label widgets, any model, per-domain benchmarks. Arc53 offers it as a subscription plus an engineering retainer; LexEU, a legal-tech platform, has run on it since 2023. The same team builds the platform and does the engineering, so where configuration is not enough, we change the platform.

Book a demoSelf-host in 5 minutes

Last updated:

Product architecture
  1. 01Your app / widgetReact · HTML · search bar
  2. 02DocsGPT API/v1/chat/completions · SSE · webhooks
  3. 03Ingestion & retrievalyour corpus · hybrid · GraphRAG
  4. 04Model of your choiceBYOM · registry · fallback
  5. 05 · humanReview & benchmarksyour feedback loop · docsgpt-cli bench
Text version

Five stages: your application or an embedded widget calls the DocsGPT API (OpenAI-compatible, server-sent events, webhooks); DocsGPT ingests and retrieves over your corpus; the model you registered generates the answer; your team's review loop and the benchmark suite keep quality measurable on every release.

The situation

  • Where the value is

    A vertical AI product's value is in domain quality — citations, multilingual retrieval, evals — not in re-implementing ingestion, chunking, retrieval and authentication.

  • Customer demands

    Customers in legal, medical and government demand private cloud or on-premises deployment and no lock-in to a single model vendor.

  • Team cost

    A two-person retrieval team is expensive to run and slow to replace; the backend they built leaves with them.

What DocsGPT does here

Workflow 01

API surface

Each agent exposes OpenAI-compatible /v1/chat/completions and /v1/models under its own API key — a drop-in for OpenAI SDKs with streaming, JSON-Schema structured outputs, multimodal images, client-side tool calling and Idempotency-Key. The native /api/answer endpoint returns the answer and its sources as one JSON response; /stream emits answer, source, tool_calls and thought events over SSE. Both accept attachments and pass variables into prompt templates.

/v1/chat/completions · /v1/models · /api/answer · /stream (SSE) · JSON Schema · Idempotency-Key · attachments

API, Widgets & Channels →

Workflow 02

Async and events

Per-agent webhooks with task polling and idempotency for long-running jobs; a realtime SSE channel for live updates; interrupted streams resume where they stopped.

Webhooks · task polling · idempotency · realtime SSE · stream resume

API, Widgets & Channels →

Workflow 03

White-label front ends

A React or plain-HTML chat widget on npm, themable to your product; a search-bar widget; a Nextra wrapper for documentation sites. All open source.

npm docsgpt · React / HTML chat widget · search widget · Nextra wrapper

API, Widgets & Channels →

Workflow 04

Any model, your evals

Bring your own model with a key or base URL; hot-swap in the Model Registry; set a fallback LLM and override the model per workflow node. Use any Sentence Transformers or remote embedding model. Run docsgpt-cli bench with YAML suites, JUnit output and baseline diffs in CI.

BYOM · Model Registry · fallback LLM · per-node model · Sentence Transformers · docsgpt-cli bench · JUnit · baseline diff

Built for the client · fine-tuned legal embeddings, as for LexEU.

Models & BYOM →Benchmarks & Evals →CLI →

Workflow 05

Tenancy and control

Teams and workspaces, per-agent API keys, token and rate limits per user and per agent, guardrails on input and output, OpenTelemetry export, and agent import/export as YAML so configurations travel between environments.

Teams & workspaces · per-agent API keys · token / rate limits · guardrails · OpenTelemetry · agent YAML import/export

Admin, Security & Analytics →Guardrails & Human-in-the-loop →

Drop-in for OpenAI SDKs

Each agent has its own API key and answers on /v1/chat/completions. Existing client code changes a base URL.

# Your agent, OpenAI-compatible; point any OpenAI SDK at this base URL
curl -N https://your-docsgpt.example/v1/chat/completions \
  -H "Authorization: Bearer $AGENT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"docsgpt-agent","stream":true,"messages":[{"role":"user","content":"Which clause governs termination?"}]}'
# → SSE stream: answer tokens, then sources with title and page. The agent behind the key sets the model; "model" is accepted and ignored.

Build vs buy, by layer. Your product keeps the domain corpus, prompts, fine-tuned embeddings, UI and the customer relationship; the rest comes from an MIT-licensed backend you can fork. Cost comparison: vs Building in-house.

Build vs buy: what a product team maintains in-house against what DocsGPT provides
 You maintain in-houseDocsGPT provides
IngestionParsers for 30+ formats, OCR for scans, connectors (Drive, SharePoint, Confluence, S3)Ingestion pipeline, parser + OCR, connectors with sync
RetrievalChunking, embeddings, vector store, hybrid/graph retrievalFive chunking strategies, hybrid BM25 + vector, GraphRAG, 7 vector stores
Answers and APICitation plumbing, streaming, structured outputsCited answers, SSE streaming, OpenAI-compatible API with JSON Schema
AgentsAgent runtime, tools, scheduling, workflowsFour agent types, MCP/API/DB tools, cron schedules, visual workflows
Auth and tenancyAuth, SSO, tenancy, audit, quotasOIDC, SCIM, RBAC, teams, per-agent keys, audit log, token limits
EvalsEval harness and regression testsdocsgpt-cli bench (assertions, LLM-judge, JUnit, baseline diffs)
OwnershipEverything above, forever, with a team that can leaveMIT code you can fork; Arc53 engineering retainer optional

Open source on GitHub: 18,273 stars, 226 contributors, MIT licence.

your-product.example
Ask the docsdocsgpt

How do I rotate an API key?

Open Settings › Agents, pick the agent and choose Regenerate key. The previous key stops working immediately and the new one is shown once, so update clients first 1.

→ Admin guide · API keys · p. 12

Settings › Models
model registry3 models · hot-swap, no restart

  • Qwen3.8-Flash-Next

    Ollama · http://ollama:11434

    defaultlocal

  • GLM-5.3-Flash

    vLLM · http://gpu-01:8000/v1

    local

  • GPT-5.6 Sol

    DocsGPT · api key · via Arc53

    cloud

  • + Add model bring your own · openai-compatible

    base url https://…/v1api key ••••••••model id my-model
zsh — curl

Evidence

Results from named deployments, with the source.

2.5M+ pages · 6 languages

Multilingual EU-law research over 2.5M+ pages, answers in 6 EU languages with full source citations, 100–200 ms retrieval.

Source: LexEU

6+ h → under 5 min

IP application drafting: 6+ hours to under 5 minutes per application, with lawyer review as the final step.

Source: Balt Alnoor

How it is deployed

Typical posture: Self-hosted or On-premises in your own cloud, with your own operators — or a Managed instance if you'd rather not operate it.

Product teams run DocsGPT themselves in their own cloud — Kubernetes manifests + kustomize or Docker Compose, PostgreSQL with pgvector, S3-compatible storage — which is the Self-hosted or On-premises posture with your own operators. Teams that would rather not operate it take a dedicated managed instance from Arc53.

Documentation

What a first project looks like, and the commercial model

Scope, measure, then decide.

  1. 01

    One-week integration spike

    Run DocsGPT in your cloud, point your client at the OpenAI-compatible endpoint, load one corpus.

    base_url → /v1/chat/completions

  2. 02

    Write the domain benchmark suite together

    Source and content assertions, LLM-judge rubrics, in YAML.

    docsgpt-cli bench

  3. 03

    Retrieval tuning and embedding choice

    Fine-tuning only if the benchmark says so.

  4. 04

    Production

    SSO and tenancy, webhooks, widgets, observability.

  5. 05

    Retainer

    Monthly engineering plus upgrades; the benchmark suite is re-run on every release.

Subscription plus an engineering retainer. LexEU, a legal-tech platform, has run on one since 2023. Self-hosting is free under MIT. See pricing and build vs buy.

Related

Frequently asked questions

Should I build or buy a RAG pipeline?

Build the parts that differentiate your product (corpus, prompts, domain evals, UI) and take the rest from an open-source backend you can fork. DocsGPT covers ingestion, retrieval, agents, auth and evals under the MIT licence, so “buy” here does not mean lock-in.

Is the API OpenAI-compatible?

Yes. Each agent exposes /v1/chat/completions and /v1/models with its own API key, including streaming, structured outputs and tool calling, so existing OpenAI SDK code points at DocsGPT with a base-URL change.

Can we use our own models and embeddings?

Yes. Register any provider or local engine (vLLM, Ollama, llama.cpp and others) in the Model Registry, override per workflow node, and use any Sentence Transformers or remote embedding model, including fine-tuned ones.

How do we know retrieval quality holds after an upgrade?

Run your benchmark suite with docsgpt-cli bench in CI: assertions on cited sources, content and latency, JUnit output, and a diff against the last baseline before you deploy.

What does the commercial model look like?

Self-hosting is free under MIT. Arc53 offers a subscription plus an engineering retainer for teams that want the platform team on call; LexEU, a legal-tech platform, has run this way since 2023.

See DocsGPT on your documents

A 30-minute demo on your use case, or a free on-premises review with an engineer.

Book a demoSelf-host in 5 minutes