Skip to content

Traefik Install

Traefik is installed with the official Traefik helm package.

The dashboard is handled by a separate package, so that it can be enabled on demand.

Creating the Traefik Package

Create a K8s / K8s Package with:

  • Application: Traefik
  • Type: Helm Chart
  • Namespace required: Checked
  • Package Source: Helm repository
  • Helm repo. name: traefik-charts
  • Helm repo. URL: https://traefik.github.io/charts
  • Helm package name: traefik

Click the 'Refresh Package from source' button.

If everything is fine, Muppy refreshes the versions (Packages and App).

Creating a Traefik Package Release

In Muppy, create a Package Release and fill in:

  • Cluster
  • Namespace: traefik-pack8s
  • Qualifier: Infrastructure
  • Package: Traefik v2.10.6 - 26.0.0 # or later
  • Helm Instance: traefik
  • Values:
    • ingressRoute.dashboard.enabled: false
    • providers.kubernetesCRD.allowCrossNamespace: true
      • required to allow Traefik Middleware to call TocToc endpoints
    • ports.web.redirectTo.port: websecure
    • ports.web.proxyProtocol.trustedIPs: ['{{server public ip}}']
    • ports.websecure.proxyProtocol.trustedIPs: ['{{server public ip}}']
    • service.type: LoadBalancer (this is the default)

Click [helm install], then wait about 20 seconds.

Updating the LoadBalancer (managed clusters)

Getting the LoadBalancer's external IP

On the Objects tab:

  1. Click Update Info
  2. Open the Kind: Service object and check that ExternalIP is set.
  3. If it is not, wait and repeat from step 1.

Updating the LoadBalancer (Self Hosted K0s or microk8s clusters only)

In a singlenode cluster, if you create a Traefik service of type LoadBalancer, then:

  1. If you patch the loadbalancer with the public IP, k8s exposes ports 80/http, 443/https and every NodePort on the public IP.
    Note: it is the patch command that triggers the publication of ports 80/http and 443/https on the public IP.

  2. If you patch the loadbalancer with a private IP on a mesh network (Tailscale, Nebula, …), k8s exposes services 80/http and 443/https and every NodePort on the Tailscale IP.
    The services are indeed published on the Tailnet — you can reach them with a browser from a machine on the Tailnet. But NOTHING is publicly reachable.
    To expose ports 80/http and 443/https on the public IP, you must add a LoadBalancer.

  3. If you patch the loadbalancer with a Private Network (non-mesh) private IP, k8s exposes services http/80 and https/443 (and the NodePorts) on the private IP, on the private network — but nothing is publicly reachable, and a LoadBalancer with a public IP is needed to expose the traffic.

Conclusion:

The Muppy setup is to:

  • publish K8s on a private IP (mesh or private), always
  • use a LoadBalancer external to the Cluster (as a Service, or managed by Muppy) to expose the services publicly
  • filter access at 2 levels:
    • external LoadBalancer (Layer 4)
    • IngressRoutes (Layer 7)

With that setup, services can be configured to be exposed publicly or only on the private network. For instance, the Traefik and K8s dashboards are published on a Tailnet while the applications are exposed publicly.

Publishing private services

There are 2 options for publishing on the private network:

  1. every service of the cluster is private.
  2. the cluster hosts both private and public services.
Option 1: every service is private

In this case, the simplest is to install no external load balancer, and to publish every service on the private IP.

Option 2: the cluster hosts private and public services
  • define a DNS record pointing at the private IP
  • restrict access to the ingressRoute, by configuring ipWhitelisting to allow only the node's public IP

Todo

  • integrate an ultralight LoadBalancer (Envoy?)
  • use an ingress on the Tailnet

Configuration

Check that the LoadBalancer is indeed waiting for its external IP: 

➜  kubectl --kubeconfig=k0s-multinode-ctb.kubeconfig get -A svc
# or
➜  microk8s kubectl get -A svc
# or use muppy GUI in tab `Objects`of traefik K8s Package Release
NAMESPACE        NAME                   TYPE           CLUSTER-IP       EXTERNAL-IP       PORT(S)                      AGE
cert-manager     cert-manager           ClusterIP      10.103.233.62    <none>            9402/TCP                     14h
cert-manager     cert-manager-webhook   ClusterIP      10.103.156.229   <none>            443/TCP                      14h
default          kubernetes             ClusterIP      10.96.0.1        <none>            443/TCP                      15h
kube-system      kube-dns               ClusterIP      10.96.0.10       <none>            53/UDP,53/TCP,9153/TCP       15h
kube-system      metrics-server         ClusterIP      10.100.70.192    <none>            443/TCP                      15h
traefik-pack8s   traefik                LoadBalancer   10.97.226.173    <pending>         80:31432/TCP,443:30168/TCP   54m

Installing the external LoadBalancer

We will use Traefik as the external loadbalancer.

Info

If you wonder how Traefik performs relative to other LBs, we suggest reading https://www.loggly.com/blog/benchmarking-5-popular-load-balancers-nginx-haproxy-envoy-traefik-and-alb/

Traefik install as TCP LB

We install a second Traefik on the server, used only as a TCP LoadBalancer (L4).

Use Network / Traefik Servers to install a Traefik instance with the following parameters:

  • Host: the Kubernetes Node
  • Domain: select any domain — the field is mandatory but the Domain will not be used
  • Traefik version: select the latest version
  • Tab [Main Configuration]
    • Main Config Template: select the latest version of Traefik2 - K8s TCP Router v? - Muppy
  • Tab [SSL & TLS]
    • Use Let's Encrypt: uncheck it. Traefik is used to load balance TCP, not HTTP, and certificate management happens inside the Kubernetes cluster.
  • Click [Save], then [Edit]
  • On the [Applications] tab, click Add a line and enter the following values:
    • Host Name: enter not-used — the hostname is not used in L4 LB mode.
    • Tab [Services]: add 2 services (use the [Create] button) with these values:
    • Host=the Kubernetes node, Address=the private IP, Network Port=80-tcp and Service=k8s-http
    • Host=the Kubernetes node, Address=the private IP, Network Port=443-tcp and Service=k8s-https
    • Tab [Config Template], in App Config Template:
    • select the latest version of Traefik2 K8s Loadbalancer (L4) App Config vx Yaml - Muppy
    • click the [(Re)load] button
    • Click [Refresh] to regenerate the application's configuration.
    • Click [Save and Close] to save and close the application.
  • The configuration is done — click [Save]

  • Click the [Reconfigure] button and check:

    • Force Traefik Program (re)installation
    • Reset Certificate Store ​
    • Reconfigure Applications

The next step is creating a K8s Object of kind LoadBalancer, which is what lets you choose the IP used to expose a Package Release.

Create a K8s / K8s Object with:

  • Cluster:
  • name: tcp-router@{{hostname}}
  • Kind: Service
  • namespace: n/a
  • Managed by: hand
  • type: LoadBalancer
  • External IP: the public ip
  • a "Managed manually" flag, to exclude it from the sync (along with the sync change)
  • (Later) a button that creates the object from the Traefik Server

From here there are 2 possibilities, depending on whether you want the HTTP and HTTPS ports publicly reachable:

1. HTTP and HTTPS ports publicly reachable

In this configuration, we tell Kubernetes to use the public IP as the External IP. Kubernetes — or rather Docker — then uses iptables to expose ports 80 and 443 on the public interface through the prerouting table. The Docker configuration therefore takes precedence over muppy's UFW configuration, but only for ports 80 and 443, which Traefik handles.

Change the service to assign the Node's public IP (or the gateway's, on Multinode clusters). For example, for 158.220.99.99:

export MPY_K8S_NODE_PUBLIC_IP=158.220.99.99
kubectl patch svc -n traefik-pack8s traefik -p "{\"spec\": {\"externalIPs\": [\"$MPY_K8S_NODE_PUBLIC_IP\"]}}"

Check in Muppy that the LoadBalancer has picked up an externalIp.

2. HTTP and HTTPS ports with restricted access

In this configuration we expose Traefik on the private IP, and install a second LoadBalancer on the public IP which balances the TCP traffic onto the private IP.

This configuration has 2 advantages:

  • UFW can filter which IPs may reach the public interface
  • the external LoadBalancer adds a layer of traffic filtering

Note

This is the same configuration as for Multinode clusters — except that the LoadBalancer is installed on a separate server, and the traffic is spread over several Kubernetes Worker Nodes.

Change the service to assign the Node's private IP. For example, for 172.16.0.2:

# Private Network case
export MPY_K8S_NODE_PRIVATE_IP=10.0.1.15

# Tailscale case
export MPY_K8S_NODE_PRIVATE_IP=$(sudo ip --json a | jq -r '.[] | select(.ifname == "tailscale0" ) | .addr_info[] | select(.family == "inet") | .local')

# Nebula case
export MPY_K8S_NODE_PRIVATE_IP=$(sudo ip --json a | jq -r '.[] | select(.ifname == "defined1") | .addr_info[] | select(.family == "inet") | .local')

kubectl patch svc -n traefik-pack8s traefik -p "{\"spec\": {\"externalIPs\": [\"$MPY_K8S_NODE_PRIVATE_IP\"]}}"

Network configuration of the K8s Cluster

  • Open the cluster's form in Muppy (K8s / K8s Clusters)
  • On the Network tab, update:
    • Default LoadBalancer
    • DNS Domain
    • Hostname generator
    • Registration email

Checking the host's exposure

Connect over SSH to a server that is not allowed to reach the host (in Host / UFW) and check the open ports:

nmap -p- {{node_public_ip}}
nmap -Pn {{node_public_ip}}