How to Scrape Prometheus Metrics Behind a Firewall With No Open Ports

Prometheus scrapes over plain HTTP. That works when the exporter and the Prometheus server sit on the same network, but it is harder when a target is private and the Prometheus instance collecting it is not, or the reverse. A node exporter on a customer site, an application's /metrics endpoint in a segmented VPC, or a Kubernetes cluster behind NAT all need a path to the collector that doesn't involve opening an inbound port and pinning it to a source IP that can change.

Pangolin gives that endpoint a public, authenticated URL instead. The exporter stays on its private network, a site connector tunnels the traffic out, and header auth authenticates every scrape with HTTP Basic credentials.

What You Get

Any process that serves Prometheus-format metrics on an HTTP endpoint (a client library's /metrics handler, node_exporter, cAdvisor, blackbox_exporter, a custom collector) can be published as an HTTPS public resource. Pangolin assigns it a domain, terminates TLS, and proxies requests through to the target address on your private network.

Header auth sits in front of that proxy. It checks incoming requests for an Authorization: Basic header (or the equivalent embedded credentials in the URL) and rejects anything that doesn't match the username and password configured on the resource. Prometheus's basic_auth scrape config field sends exactly that header on every request, so no additional client, sidecar, or plugin is needed on the Prometheus side.

Why Tunneling Matters

Metrics targets are usually the least exposed part of a network on purpose. A node exporter binds to every host in a fleet; an application's metrics port often has no authentication of its own because it was only ever meant to be reachable from inside the cluster. Putting any of that on a public IP so a remote Prometheus server can reach it is dangerous.

A site connector installed inside the private network maintains an outbound tunnel to Pangolin. When Prometheus scrapes the resource URL, the request travels in through that tunnel to the exporter. The exporter's port is not bound to a public interface and does not need an inbound firewall rule. The connector only requires outbound internet access, which is why this works from behind NAT, inside customer environments, and in cloud VPCs with no public subnet. One connector can front every exporter on its network and you can install a second on the same network for a redundant path. Read more about sites and connectors and installing a site connector.

Step by Step

1. Identify the metrics endpoint

Note the internal host and port the exporter listens on. This guide uses a node_exporter instance at 10.20.0.15:9100 serving /metrics as the working example. Confirm a site connector is installed on that network and can reach the address.

2. Create an HTTP/HTTPS resource

In the Pangolin dashboard, create a new public resource and choose HTTPS. Assign it a domain, for example node1-metrics.example.com, and add a target pointing to 10.20.0.15:9100, assigned to the site running the connector.

Leave the path at the default unless the exporter serves metrics somewhere other than /metrics.

3. Turn off Platform SSO and enable Header Auth

Public resources default to Pangolin (Platform) SSO, which redirects browsers to a login page. Prometheus cannot complete that flow, so on the resource's Authentication tab, disable Platform SSO and enable Header Auth as the only authentication method.

Set a username and password. Keep them specific to this resource, or to a small group of related exporters, rather than reusing credentials across unrelated targets.

Leave Extended Compatibility off for this use case. It forces the resource to answer unauthenticated requests with a 401 so browsers know to prompt for credentials. Prometheus's basic_auth sends the Authorization header on the first request without waiting for a challenge, so the extra round trip isn't needed. Turn it on only if you also want to open the URL directly in a browser during setup and be prompted interactively.

4. Verify the endpoint from the command line

Before touching the Prometheus config, confirm the resource answers with credentials and rejects requests without them:

# should return metrics
curl -u node1:supersecret https://node1-metrics.example.com/metrics

# equivalent, sending the header directly
curl -H "Authorization: Basic $(echo -n node1:supersecret | base64)" \
  https://node1-metrics.example.com/metrics

# should fail with a 401 or an auth error page
curl https://node1-metrics.example.com/metrics

5. Add the target to Prometheus

Prometheus's basic_auth block on a scrape job sends the same header curl -u sends. Point the job at the resource's domain over HTTPS:

scrape_configs:
  - job_name: node1
    scheme: https
    metrics_path: /metrics
    basic_auth:
      username: node1
      password: supersecret
    static_configs:
      - targets: ["node1-metrics.example.com"]

Keep the password out of version control by using password_file instead of password, pointing at a file Prometheus can read at startup:

scrape_configs:
  - job_name: node1
    scheme: https
    basic_auth:
      username: node1
      password_file: /etc/prometheus/secrets/node1.pass
    static_configs:
      - targets: ["node1-metrics.example.com"]

Reload Prometheus (SIGHUP, the /-/reload endpoint, or a restart, depending on how it's deployed) to pick up the change.

6. Confirm the target is up

In the Prometheus UI, under Status > Targets, the job should show state UP with a recent last-scrape time. A 403 or 401 here almost always means the username or password in the scrape config doesn't match what's set on the resource, or Platform SSO is still enabled and intercepting the request before it reaches header auth.

Scaling to Many Targets

A single site connector can front every exporter on its network, so adding a second or third host is the same three steps: a target on the existing site, a resource with its own domain, header auth turned on with its own credentials. For a fleet, apply a shared resource policy so authentication settings and access rules stay consistent across every metrics resource instead of being reconfigured one at a time.

Each resource is a distinct scrape job in Prometheus, which keeps the target list explicit and makes it obvious in the Prometheus UI which host stopped responding.

Layering Additional Access Rules

Header auth authenticates the request; it does not by itself restrict where requests can come from. If the Prometheus server has a stable egress IP, add an IP allow rule on the resource so only that address can reach the endpoint at all, with header auth as the second check behind it. This is the same rule mechanism used for geo-blocking, applied to a single known source instead of a country list.

FAQ

What is header auth in Pangolin?

Header auth authenticates requests to a public resource using an Authorization: Basic header, or a username and password challenge in the browser. Prometheus scrape jobs send this header automatically when basic_auth is set in the scrape config, with no extra client or plugin required.

Do I need to open the exporter's port or run a VPN?

No. The exporter stays on its private network. A site connector on that network maintains an outbound tunnel to Pangolin, and Prometheus reaches the exporter through the resource's public URL over that tunnel. No inbound firewall rule or public IP is needed on the host running the exporter.

Does Prometheus support HTTP Basic auth natively?

Yes. The basic_auth field on a scrape config accepts username and either password or password_file, and Prometheus sends the resulting header on every scrape request without a challenge-response round trip.

Should I leave Platform SSO enabled alongside header auth?

For a metrics endpoint, no. Platform SSO redirects browsers to a login page, which a Prometheus scrape request cannot complete. Disable Platform SSO on the resource and use header auth as the sole authentication method for that endpoint.

Can I restrict scraping to my Prometheus server's IP as well?

Yes. Add an IP allow rule on the resource in addition to header auth. Requests then have to match the allowed source and present valid credentials before reaching the exporter.

Can more than one Prometheus server scrape the same resource?

Yes. Header auth checks the credentials on each request; it doesn't limit how many clients can present them. Give every legitimate collector the same credentials, or issue different resources with different credentials per collector if you want to tell scrapers apart in logs.

How is this different from a Prometheus push gateway?

A push gateway requires the target to actively push metrics out, which changes the metric semantics for anything meant to be scraped (like per-scrape gauges and process-level metrics). This approach keeps the standard pull model. Prometheus scrapes on its normal interval; Pangolin just provides the authenticated path to a target it otherwise couldn't reach.

Is Pangolin open source?

Yes. Pangolin is open source and self-hostable. You can deploy the entire platform yourself, use Pangolin Cloud, or mix hosted and self-hosted components. Public resources and header auth are available in both Pangolin Cloud and self-hosted editions.

See Also

• Authentication for public resources: the full list of authentication methods, including header auth and access rules
• Metrics and Observability: native Prometheus and OTLP metrics exposed by Pangolin's own components
• How to Geo-block with Pangolin: ranked allow/deny rules for restricting access by IP, region, and path
• Resource Policies: share authentication and access rule settings across many resources
• HTTP/HTTPS public resources: full configuration reference for targets, path routing, and proxy settings
• How Pangolin Works: architecture overview for sites, resources, and policy