You Can't Review Every PR Anymore: How DocsGPT Strengthened Security in the GitHub Secure Open Source Fund
Three lessons from the GitHub Secure Open Source Fund: threat-model before you ship, treat dependencies as attack surface, let CI catch the rest.
Pavel Torbin6 min readsecurityopen-source
AI writes a serious chunk of the PRs landing in open-source repos now, and no maintainer can hand-review all of it. The GitHub Secure Open Source Fund gave us a few weeks alongside working security engineers to catch up with that reality, and every session pointed the same direction: prepare in advance, then automate. Three lessons stuck. Here’s what we shipped for each, and what you can steal.
Model the Threat Before You Ship the Feature
The threat-modeling sessions reframed security as a design step, not a cleanup job. Break the feature down, find the threats, prioritize, mitigate - STRIDE gives you a checklist for the “find” step (spoofing, tampering, repudiation, info disclosure, denial of service, elevation of privilege). We turned that into a permanent artifact: a public THREAT_MODEL.md covering DocsGPT’s trust boundaries, attack surfaces, key threats with expected mitigations, and baseline controls for public deployments.
The proof it works: our sandboxed code execution, where the assistant writes and runs its own Python. That’s about as sharp as attack surfaces get, and because we modeled it first, the mitigations were designed in rather than bolted on. The kernel env is scrubbed of API keys, tokens, and the DB URL; a NetworkPolicy blocks the internal IP ranges and the cloud-metadata endpoint that SSRF loves; and the whole thing ships off by default. Threat-modeling a feature before you build it is cheaper than patching it after someone else finds the hole.
A public threat model also has a second job we didn’t plan for: it’s a map for automated reviewers. An AI scanner or a security researcher reading it skips the surfaces we’ve already closed and spends their time on real attack vectors instead of flooding us with false positives.
Text version
The assistant writes Python and hands it to a sandbox kernel that sits inside its own trust boundary. Inside the boundary the kernel environment is scrubbed: no API keys, tokens or database URL. A NetworkPolicy blocks egress to internal IP ranges and to the cloud-metadata endpoint that SSRF attacks target. The whole feature ships off by default. These mitigations were designed in from the STRIDE threat-modelling session rather than bolted on afterwards.
Your Dep Tree Is Attack Surface (and a License Problem)
The licensing and compliance session paid off fastest. It sent us straight into our dependency tree: we audited what we actually ship, cut dormant deps we hadn’t touched in ages, and resolved license conflicts. Every package you forgot about is both a supply-chain risk and a legal liability, and trimming the tree fixes both at once. It also keeps us honestly compliant with our MIT license and everything underneath it.
Generating an SBOM turned out to be a button, not a project. GitHub exports one in SPDX format straight from the repo’s dependency graph, so there’s no real excuse for not having one. Dependabot now keeps the tree current instead of letting it rot between releases.
Let CI Catch What Humans Can’t
This is the shift that changed how we work day to day. When a real fraction of incoming PRs are AI-authored and the volume keeps climbing, reading every diff line by line stops scaling. Uncle Bob Martin put the uncomfortable version out loud:
“I don’t review code written by agents. I measure things like test coverage, dependency structure, cyclomatic complexity, module sizes, mutation testing… Humans are slow at code. To get productivity we humans need to disengage from code and manage from a higher level.”
You don’t have to go that far to take the point: the baseline has to be automatic. So we wired CI to catch the common problems on every PR without anyone remembering to look. Custom CodeQL queries inventory every untrusted-input point and dangerous sink in the codebase, so a new attack surface has to be consciously allow-listed instead of slipping in. Bandit scans the Python backend, zizmor hardens the Actions workflows themselves (pinned action SHAs, persist-credentials: false, least-privilege permissions), and secret scanning plus Dependabot cover leaked creds and vulnerable deps. Human attention goes where scanners can’t: the threat model, the architecture, test coverage.
Steal This Baseline
Most of it is a settings toggle away: flip on secret scanning, Dependabot, and CodeQL, add a workflow scanner like zizmor, and export the SBOM while you’re on the dependency graph page. Then write a threat model, even a short one, so your tools and your researchers have a map. And document how someone reports a hole, our SECURITY.md sets a private reporting path and a 48-hour acknowledgment target.
None of this makes DocsGPT unbreakable, and the threat model says so out loud. What changed is that the next risky feature already has a process waiting for it. Thanks to GitHub and the Secure Open Source Fund for the room to do the work.
- Try DocsGPT Cloud
- Read the threat model & security policy
- Join our Community on Discord, X and LinkedIn
- Check out Github’s blog post about it