
If your company has shipped more than one AI feature, you have probably already run into the same problem: nobody can answer simple questions about your LLM traffic. Which team is calling which model? What is it costing this month? Did a prompt just send a customer’s social security number to a third-party API? Is anyone even allowed to send that data there in the first place?
This is not a new problem. It is the same problem the software industry solved a decade ago for microservices, when every internal service call started passing through an API gateway that could log it, meter it, and enforce policy on it. LLM calls need the same layer, and most companies do not have one yet.
That is the gap Chokepoint is built to close.
What Chokepoint does
Chokepoint is a self-hosted, open-source gateway written in Go. You run it once, point your existing OpenAI or Anthropic SDK at it instead of the provider directly, and every call your application makes now passes through a single, observable checkpoint before it reaches the model.
At that checkpoint, four things happen automatically.
First, every prompt is scanned for sensitive data. Chokepoint detects email addresses, phone numbers, social security numbers, credit card numbers, IP addresses, AWS keys, API keys, and JWTs before they leave your infrastructure.
Second, a policy engine decides what happens next. You define ordered rules that can block a call outright, redact the sensitive parts before forwarding it, or reroute it to a cheaper model entirely. Chokepoint ships with sensible defaults out of the box: block anything containing a leaked secret, redact common PII, and route unusually long prompts to a lower-cost model. You can change these rules live, without redeploying anything.

Third, every call is priced. Chokepoint calculates the cost of each request from its token usage against a configurable price list, and rolls that up by team, feature, and model, so you can finally answer “what is this feature actually costing us” without digging through a provider’s billing dashboard.
Fourth, everything is logged. Every call gets an immutable audit record: which team made it, which model it hit, how many tokens it used, how long it took, and what the policy engine decided to do with it. Prompt and response content is redacted in the log by default, because an audit trail should not become its own privacy liability. Retention is configurable, and a background process prunes old records automatically.
All of this is visible in a dashboard that ships with the gateway itself. There is no separate frontend to stand up. Open the built-in page and you get real-time cost totals, error rates, average latency, and a live feed of recent calls with their policy outcomes.

Why this matters now
Two things are true at once right now. AI features are shipping faster than the tooling around them is maturing, and the compliance bar for handling customer data through third-party APIs is not getting any lower. Regulated industries in particular, fintech, healthtech, legal, cannot treat an LLM call the way they might have treated an experimental feature two years ago. It needs an audit trail. It needs policy enforcement. It needs to be provably governed, not just monitored after the fact.
Cost is the other half of the equation. Token pricing varies enormously across models, and it is easy for a single feature to quietly become the most expensive line item in an infrastructure budget without anyone noticing until the invoice arrives. Per-feature, per-team cost attribution is not a nice-to-have once you have more than a couple of AI features in production; it is the only way to make sensible tradeoffs about which model to use where.
Built to be multi-provider from day one
Most companies are not betting on a single model vendor anymore. Chokepoint was designed around that reality: the same policy engine, the same audit log, and the same cost tracking work identically whether a call is going to OpenAI or Anthropic, with more providers planned. You are not locked into a single vendor’s tooling just to get visibility into your own traffic.
Getting started
Chokepoint is a single Go binary with no external dependencies to run. Build it, start it, point your existing SDK’s base URL at it, and you have a governance layer in front of your LLM traffic in minutes, not weeks.

It is open source, self-hosted, and designed to be the kind of infrastructure you set up once and mostly forget about, until the day someone asks “did we log that call” and the answer is yes.

Explore the project on GitHub – https://github.com/AIHelpers/chokepoint
Leave a Reply