Implementing TLS for Internal Services the Correct Way

By Jakub Kołodziejczak (mrl5@tuxnet.dev) Date: 2026-07-09

The title might seem a bit like clickbait, but bear with me—I'm about to explain why this specific architecture is "the way" to handle internal encryption.

The Scenario

Imagine a server hosting several HTTP services. To access these internal tools, a user must first connect via a VPN.

When deciding how to handle the domain and encryption, we generally face two primary paths:

Option 1: The Private Route

We use a top-level domain (TLD) reserved by ICANN for private networks (e.g., .home.arpa).

Setup:

1. Server internal IP: 10.0.1.10
2. VPN DNS resolver creates an "A" record: grafana.home.arpa $\rightarrow$ 10.0.1.10

The Trade-off: To avoid plain-text HTTP, you must generate a self-signed certificate. While tutorials for this are abundant, the reality is unpleasant:

The Ugly Part: Every single client device must be manually configured to trust your custom root CA. Otherwise, you're forced to tell users to just ignore the browser security warnings $\text{ (which is a terrible habit) }$.

Option 2: The "Right" Route (Split-Horizon DNS)

Instead of a private TLD, we use a Split-Horizon DNS configuration. In this setup, the domain grafana.tuxnet.dev resolves differently depending on where the request originates:

• Public DNS: Resolves to a public IP.
• VPN DNS: Resolves to the internal IP.

The "ugly" part here is that since the domain is public, we need a Web Application Firewall (WAF) to ensure that only VPN traffic actually reaches the service. However, configuring one WAF on a server is significantly easier than installing certificates on every employee's laptop.

---

Technical Implementation

The Toolset

To get this running, I utilize the following stack:

• NetBird: A VPN with robust DNS resolver capabilities (I prefer this over Tailscale).
• Nginx: Acting as the reverse proxy and WAF.
• acme.sh: For automated certificate management.

Visualizing the Traffic Flow

Obtaining the Certificate

I use the http-01 challenge. Using acme.sh in standalone mode is highly efficient because Nginx doesn't need to occupy port 80 permanently.

acme.sh --issue -d grafana.tuxnet.dev --server letsencrypt --standalone

Nginx Configuration

The "magic" happens in the listen directive. By binding to the VPN-specific hostname/IP, we create an implicit WAF.

upstream grafana { 
    server localhost:3000; 
} 

map $http_upgrade $connection_upgrade { 
    default upgrade; 
    '' close; 
} 

server { 
    # This is the WAF: only traffic on the NetBird interface is accepted
    listen our-server.netbird.cloud:443 ssl; 
    server_name grafana.tuxnet.dev; 
    
    http2 on; 
    ssl_certificate /etc/ssl/certs/grafana.tuxnet.dev.crt; 
    ssl_certificate_key /etc/ssl/private/grafana.tuxnet.dev.key; 
    
    access_log /var/log/nginx/grafana.tuxnet.dev.access.log main; 
    error_log /var/log/nginx/grafana.tuxnet.dev.error.log warn; 

    location / { 
        proxy_pass http://grafana; 
        proxy_set_header Host $host; 
    } 

    # Handling Grafana Live WebSocket connections
    location /api/live/ { 
        proxy_pass http://grafana; 
        proxy_http_version 1.1; 
        proxy_set_header Upgrade $http_upgrade; 
        proxy_set_header Connection $connection_upgrade; 
        proxy_set_header Host $host; 
    } 
}

Security Logic: $\text{Total Security} = \text{Split-Horizon DNS} + \text{Interface Binding (WAF)}$

If the DNS is bypassed, the listen directive acts as the second line of defense.

---

Automation and Maintenance

To keep certificates fresh, I run a daily cron job executing acme.sh --cron. I use a custom script to synchronize the certificates and reload Nginx.

main () { 
    refresh_certs 
    sync_api_tuxnet_dev_certs 
    sync_internal_tuxnet_dev_certs 
    reload_nginx 
} 

refresh_certs () { 
    # Allow socat to bind to port 80 without root
    setcap CAP_NET_BIND_SERVICE=+ep /usr/bin/socat1 
    sudo -u acmesh /home/acmesh/acme.sh/acme.sh --cron 
    setcap -r /usr/bin/socat1 
} 

sync_api_tuxnet_dev_certs () { 
    local green = $( get_checksum "$API_SRC_KEY" ) 
    local blue = $( get_checksum "$API_DST_KEY" ) 
    local key_allowed_group = www-data 
    if [[ "$green" != "$blue" ]]; then 
        sync_certs "$API_SRC_KEY" "$API_DST_KEY" "$API_SRC_CERT" "$API_DST_CERT" "$key_allowed_group" 
    fi 
} 

sync_internal_tuxnet_dev_certs () { 
    local green = $( get_checksum "$INTERNAL_SRC_KEY" ) 
    local blue = $( get_checksum "$INTERNAL_DST_KEY" ) 
    local key_allowed_group = www-data 
    if [[ "$green" != "$blue" ]]; then 
        sync_certs "$INTERNAL_SRC_KEY" "$INTERNAL_DST_KEY" "$INTERNAL_SRC_CERT" "$INTERNAL_DST_CERT" "$key_allowed_group" 
    fi 
} 

reload_nginx () { 
    nginx -t 
    systemctl reload nginx 
} 

get_checksum () { 
    sha256sum "$1" | cut -d ' ' -f1 
} 

sync_certs () { 
    local src_key = "$1" 
    local dst_key = "$2" 
    local src_cert = "$3" 
    local dst_cert = "$4" 
    local key_allowed_group = "$5" 
    cp -v "$src_key" "$dst_key" 
    logger "synced $dst_key" 
    cp -v "$src_cert" "$dst_cert" 
    logger "synced $dst_cert" 
    chown root: ${key_allowed_group} "$dst_key" 
    chown root:ssl-cert "$dst_cert" 
    chmod 640 "$dst_key" "$dst_cert" 
} 

main "$@"

A Note on Permissions

The command setcap CAP_NET_BIND_SERVICE=+ep /usr/bin/socat1 is crucial. Because acme.sh in standalone mode relies on socat to listen on port 80, this command grants socat the specific privilege to bind to a privileged port without requiring full root access.