How to Stop Using Virtual API Keys for Your AI Gateway
If your team runs an AI gateway like LiteLLM, Bifrost, or Pangolin's own public resources then there's a good chance every engineer has a key sitting in a config file, an environment variable, or ~/.claude/settings.json right now. That's the default setup for any gateway reachable over plain HTTP because something has to prove who's calling!
It can cause a lot of maintenance debt. For example keys need to be generated, sent to the right person through some channel that isn't really built for secrets, tracked so you know who has what, and rotated or revoked when someone leaves or a key leaks. None of that is really about authentication. It more about key logistics, and it scales badly as the team and the number of tools each person uses both grow.
This is a practical walkthrough for moving human users off virtual keys entirely, onto access authenticated by their existing SSO identity instead. For the reasoning behind why this matters more than it might seem, see Why Virtual API Keys Are a Bad Fit for LiteLLM and Bifrost Deployments.
Virtual Keys vs. Identity-Based Access
| Virtual API Key | Identity-Based Access | |
|---|---|---|
| What proves who's calling | A secret string in the request | An SSO login plus a device-level tunnel |
| Onboarding a new user | Generate a key, send it somewhere, hope it lands only with them | Grant them the resource; they log in with the account they already have |
| Offboarding | Find and revoke every key they were issued | Their IdP account is disabled; access stops with it |
| If it leaks | Whoever has the string can use it until someone notices and rotates it | Nothing to leak - there's no bearer secret for the client to hold |
| Endpoint reachability | Needs to be public, or everyone needs a VPN to reach it | Reachable only over the tunnel the client builds - never public |
| Usage attribution | Tied to whichever key was used, not necessarily the person | Tied to the authenticated user on every call |
Step 1: Find Out What's Actually Using a Key
Before changing anything, separate your callers into two groups: People who are engineers using coding agents, chat clients, or scripts they run themselves. And Everything else which are CI pipelines, scheduled jobs, backend services, agents that run unattended. Only the first group benefits from this migration. The second group still needs a credential, because there's no human present to authenticate interactively which is fine, and covered in Step 5.
Step 2: Set Up a Private AI Gateway Resource
Identity-based access in Pangolin comes from a private AI Gateway resource - reachable only through the Pangolin client, with no key checked at all. If you're migrating from a self-hosted LiteLLM or Bifrost deployment, or from one of Pangolin's own public resources, you're adding this alongside what exists today, not editing it in place:
- You need a Newt site with connectivity to wherever your models are actually served - your existing LiteLLM/Bifrost deployment, a cloud provider, or a self-hosted cluster. Private AI Gateway resources can only be created on a Newt site.
- Resources > Create > Type AI Gateway > Private.
- Attach the same providers your team already uses - point one at your existing LiteLLM or Bifrost deployment as a Custom provider if you're keeping its routing logic, or attach OpenAI, Anthropic, and self-hosted providers directly.
- Grant users and roles the same way you would for any private resource.
No virtual key exists for this resource. Access is entirely a function of who's connected and what they're granted.
Step 3: Move Everyone to the Pangolin Client
Each engineer installs the Pangolin client - desktop app or CLI - and logs in with your existing identity provider:
pangolin login
pangolin up
That's the entire credential step. There's no key to request from an admin, no wiki page to find, nothing to paste that could end up somewhere it shouldn't.
Step 4: Point Clients at the New Resource
Reconfigure coding agents and other tools to use the private resource's URL, with none in the key field - most clients still need something typed there, but Pangolin doesn't check it:
pangolin configure claude --resource <resource>
For anything that speaks the OpenAI API directly:
export OPENAI_BASE_URL=<endpoint>/v1
export OPENAI_API_KEY=none
Full per-client setup: Claude Code, Codex, OpenCode, Gemini CLI, Open WebUI.
Run both resources in parallel for as long as it takes people to switch over - there's no cutover moment required.
Step 5: Keep Keys Only Where a Person Isn't Present
CI jobs, scheduled tasks, and unattended agents still need a credential, and that's a legitimate use for one - it just shouldn't be the same credential a person uses day to day. Create a manual virtual key scoped to exactly the resource(s) that job needs, and attach a budget so a misbehaving job can't run up unbounded spend. Unlike a key handed to a person, a scoped service key is easy to reason about: it's not going in anyone's laptop config, and revoking it only affects the one job it was issued for.
Step 6: Revoke What's Left
Once your engineers are running through the private resource, retire the keys they were using before. If you're moving off a self-hosted LiteLLM or Bifrost deployment entirely, that means decommissioning the keys minted there. If you're moving off a Pangolin public resource, delete the identity keys tied to those users, or remove their access to the public resource - their identity key stops working for it immediately.
FAQ
Can I get rid of API keys entirely?
For human users, yes - private AI Gateway resources don't check a key at all; access comes from the authenticated client connection. Services and unattended jobs still need something to authenticate with, since there's no person to log in. Use a scoped manual key for those instead of reusing a person's key.
Do I have to give up LiteLLM or Bifrost to do this?
No. Point Pangolin at your existing LiteLLM or Bifrost deployment as a Custom provider and keep its routing, budgeting, and fallback logic. Pangolin sits in front and handles who's allowed to call in the first place.
What if some of my team can't install the desktop client?
The Pangolin CLI is a lighter-weight option that works the same way - log in, connect, and the same identity-based access applies. For a case where neither is possible, a scoped manual key is the fallback, the same as it would be for a service.
Does this work if my models are self-hosted, not just cloud providers?
Yes - that's the more common case, in fact. A site connector reaches your self-hosted models over an outbound tunnel, and the private resource sits in front the same way it would for a cloud provider. See How to Connect Any Self-Hosted AI Model to Pangolin for that setup specifically.
How is this different from just rotating keys more often?
Rotation reduces how long a leaked key stays valid; it doesn't remove the key from the loop. Identity-based access removes the bearer credential for the common case entirely - there's nothing for a person to hold that could leak, get copied into a repo, or outlive the reason it was issued.
See Also
- Why Virtual API Keys Are a Bad Fit for LiteLLM and Bifrost Deployments: the reasoning behind this migration
- Virtual API Keys: the full reference for identity keys and manual keys
- Private AI Gateway: how private resources authenticate without a key
- How to Connect Any Self-Hosted AI Model to Pangolin: attaching your own models to a gateway resource
- vLLM and Ollama Clusters: the same identity model applied to a self-hosted cluster
Pangolin is an open-source infrastructure company that provides secure, zero trust remote access for teams of all sizes. Built to simplify user workflows and protect critical systems, Pangolin helps companies and individuals connect to their networks, applications, and devices safely without relying on traditional VPNs. With a focus on device security, usability, and transparency, Pangolin empowers organizations to manage access efficiently while keeping their infrastructure secure.
Keep reading
- SSH in the Browser: Web-Based Terminal Access Without a Client
SSH in the Browser: Web-Based Terminal Access Without a ClientRun a full SSH session in any modern browser. Users connect with a URL and authentication, without installing an SSH client, VPN, or desktop app.
Guides - NVIDIA DGX Spark Remote Access: Run Inference From Anywhere
NVIDIA DGX Spark Remote Access: Run Inference From AnywhereAccess your NVIDIA DGX Spark from any laptop without SSH, a VPN, or an open port. Run Ollama on the Spark and reach it through Pangolin's identity-aware AI gateway, from the same endpoint you use for cloud models.
Guides - How to SSH with Pangolin: Browser Access and Private CLI
How to SSH with Pangolin: Browser Access and Private CLISSH to private servers through Pangolin with a browser terminal or private CLI over a scoped tunnel. Automatic user provisioning via PAM, without opening port 22 or distributing static keys.
Guides
