Self-Host DeepSeek, Qwen, and Kimi K2: Private Access From Anywhere

DeepSeek V3 and R1, Alibaba's Qwen series, Moonshot AI's Kimi K2, and Zhipu's GLM family are competitive with frontier closed models on a lot of benchmarks, and the weights are yours to download and run. For teams with data sovereignty requirements, or anyone who'd rather not send every prompt to a third party, that's the whole appeal: the model runs on hardware you control, and nothing leaves your network unless you decide it does.

That guarantee only holds if "your network" is actually private. A model server with no auth in front of it, reachable from anywhere on your LAN or worse, defeats the purpose almost as thoroughly as calling a hosted API would. And most teams still want to reach these models from outside the office with their laptop at home, or a remote engineer, or a CI pipeline - which is where self-hosting usually gets complicated.

Running the Models

These models are large enough that serving them well takes real GPU infrastructure, not a laptop. The two common paths are vLLM for production-grade throughput - DeepSeek-R1, Qwen2.5/Qwen3, and Kimi K2 all have day-one or near-day-one vLLM support, with an OpenAI-compatible server built in. Or Ollama for quantized variants that fit on a single GPU or a box like an NVIDIA DGX Spark - smaller Qwen and DeepSeek-distill checkpoints run comfortably this way.

Either way, you end up with an OpenAI-compatible (or in Kimi K2's case, also Anthropic-compatible) HTTP endpoint on your network. The question is how you and your team reach it without punching a hole in your firewall or standing up a VPN just for model traffic.

Keep It Private, Reach It From Anywhere

Pangolin is an AI gateway built around the same principle that makes self-hosting these models worth doing in the first place: nothing should have to leave your network to work. A lightweight connector runs next to the model server and opens an outbound tunnel to Pangolin - the GPU host never accepts inbound connections, never gets a public IP, and is never one misconfigured iptables rule away from being reachable by anyone.

1. Serve the Model

vLLM example, serving DeepSeek-R1 with its OpenAI-compatible server:

vllm serve deepseek-ai/DeepSeek-R1 --port 8000

Or Qwen through Ollama:

ollama pull qwen2.5:32b

2. Install a Site Connector on the Same Network

In the dashboard, go to Sites > Create, choose Newt as the site type, and name it. Pangolin generates a site ID, secret, and a ready-to-copy install command. Run that command on the model server itself, or any host that can reach it - the connector is the only thing that talks to Pangolin's control plane:

curl -fsSL https://static.pangolin.net/get-newt.sh | bash
newt --id <site-id> --secret <site-secret> --endpoint https://app.pangolin.net

See Install Sites for the full reference.

3. Add the Model as a Custom Provider

AI Gateway > Providers > Create, type Custom:

• vLLM: select OpenAI Chat Completions, Site Targets, target the host on port 8000, allow deepseek-ai/DeepSeek-R1.
• Ollama: select OpenAI Chat Completions, Site Targets, target the host on port 11434, allow qwen2.5:32b.
• Kimi K2 via its own Anthropic-compatible server: select Anthropic Messages, point routing at the upstream, auth type x-api-key.

See Custom providers for the full field reference, and the worked vLLM and Ollama examples.

4. Attach to a Resource and Connect

Create an AI Gateway resource, attach the provider, and choose private (reached only through the Pangolin client, no key) or public (reached from anywhere with a virtual API key). Install the Pangolin client, log in with your identity provider, and point Open WebUI, a coding agent, or your own tooling at the resource URL.

curl <endpoint>/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "deepseek-ai/DeepSeek-R1", "messages": [{"role": "user", "content": "hello"}]}'

Mix Local and Cloud Without Changing Endpoints

Most teams don't run open-weight models to the total exclusion of frontier ones - they want the option to use DeepSeek or Qwen for the bulk of traffic and reach for Claude or GPT when a task calls for it. Attach a cloud Anthropic or OpenAI provider to the same resource as your self-hosted models, and switching is a model-name change in the client, not a new endpoint or a new key. See Multiple Gateway Resources if different teams should see different model sets from the same URL.

Why This Matters for Data Sovereignty

The reason to self-host DeepSeek, Qwen, or Kimi K2 in the first place is usually some combination of cost, control, and not wanting prompts to leave your infrastructure. That last point is worth taking seriously end to end:

By restricting the model server to outbound connections only, Newt eliminates inbound ports, public IPs, and exposed attack surfaces for external scanners. Access is governed through identity-based authentication—using Pangolin's SSO and a device-level tunnel—eliminating the security risks of easily leaked static API keys. Every interaction remains fully attributable through session logs and usage analytics to verify data destinations, while organizations with strict compliance requirements can self-host the control plane using Pangolin's AGPLv3 Community Edition.

FAQ

Can I self-host DeepSeek, Qwen, or Kimi K2 and still access them remotely?

Yes. Run the model with vLLM or Ollama on your own GPU hardware, install a Pangolin site connector on the same network, and attach the model as a provider on an AI Gateway resource. You reach it through the Pangolin client from anywhere, without exposing the model server to the internet.

Is it safe to run these models without exposing a port?

That's the point of the setup here. The site connector makes an outbound-only connection to Pangolin, so the model server never listens for inbound traffic and never needs a public IP or forwarded port.

Does Kimi K2 work with Pangolin?

Yes. Kimi K2 speaks the Anthropic Messages API, so a Custom provider with Anthropic capabilities selected works whether Kimi K2 is self-hosted or reached through Moonshot's hosted endpoint. Point Claude Code or Claude Desktop at the resulting resource the same way you would for real Anthropic.

Can I run DeepSeek or Qwen alongside Claude or GPT in the same setup?

Yes. Attach both the self-hosted provider and a cloud provider to the same AI Gateway resource. Clients call one endpoint and pick the model by name; Pangolin routes each request to the right upstream.

Do I need an API key to reach a self-hosted model through Pangolin?

Not for a private resource - identity comes from your SSO login through the Pangolin desktop client. Public resources use a virtual API key that's separate from any real provider credential, useful for CI or services that can't run the client.

What GPU hardware do I need to self-host these models?

It depends on the model and quantization. Full-precision DeepSeek-R1 or Kimi K2 needs multi-GPU serving infrastructure; quantized or distilled variants of DeepSeek and mid-sized Qwen checkpoints run on a single high-memory GPU or a unified-memory box like an NVIDIA DGX Spark. The remote access setup in this article is the same regardless of scale.

See Also

• AI Gateway Overview: providers, resources, and virtual keys
• Custom Providers: the general pattern for any self-hosted or vendor endpoint
• vLLM and Ollama Clusters: scaling this to a multi-node cluster
• NVIDIA DGX Spark Remote Access: running smaller checkpoints on a single box
• How to Connect Any Self-Hosted AI Model to Pangolin: the general-purpose version of this guide
• Why Virtual API Keys Are a Bad Fit for LiteLLM and Bifrost Deployments: why identity beats keys for this kind of deployment