Skip to content

Installing a SingleNode Kubernetes Cluster — with Muppy Tasks

This document describes how to configure a secured SingleNode Cluster managed with Muppy Pack8s (Multi Cloud HA).

A single node cluster is a k8s cluster with only one node, which runs:

  • the Control Plane
  • Kubelet (worker)
  • PostgreSQL (see the options)

Info

There are three options for PostgreSQL:

  • do not install it
  • install it on the Node
  • install it on a separate host.

Security strategies

Muppy currently supports 2 security strategies for Kubernetes Clusters:

  • Private
  • Public (WAF or Whitelist mode)

Private

In this mode everything is published on the Private IP (Tailscale) and the firewall is configured to let only administrative traffic through.

This strategy suits development servers, and servers holding sensitive information. They are, however, completely isolated.

In this configuration, no service is exposed on the Public IP.

Public

In this mode, each Service can be published on the Public or on the Private interface.

To do so, Muppy installs a TCP Router (Level 4) that forwards TCP traffic from the Public IP to the Private IP.

In addition, the Services published on the Public interface can be filtered with:

  • ipWhiteListing — access is restricted to a list of IP addresses that can be updated dynamically (CIDR Dynamic Range)
  • TocToc — access is managed by Muppy TocToc, which handles access by email and other means.
  • HTTP Basic Auth — adds rudimentary user/password protection (to be avoided at all costs).

These 3 filtering systems suit sites reserved for internal use; they are useless for a public site.

Warning

For maximum security, the infrastructure packages Muppy deploys are always deployed on the Private IP and filtered by IP Whitelisting.

Public internet sites

For fully public sites, Muppy can filter every request with the OWASP ModSecurity Core Rule Set.

Danger

If your public site is open to significant volume, we advise deploying an additional level of filtering / security.

Conclusion

Before launching the installation, you must have:

  • selected a security strategy: private or public
  • [Optional] defined a list of fixed IPs allowed to reach the services the cluster publishes

Prerequisites

  1. The Host must have at least one Private interface.

  2. You have chosen the security strategy to apply to the Cluster, among these options:

    • Public IP + Mesh VPN (Tailscale, Nebula, Wireguard, …)
    • Public IP + Private IP
    • Private IP

  3. You have created a CIDR Dynamic Range containing at least the Host's Public IP. This is mandatory, because Muppy protects the Dashboards by publishing them on the Private network and whitelisting them with the content of that CIDR Dynamic Range.
    You must add the Host's public IP to let the Tailscale traffic through.

  4. If the cluster is publicly reachable, you must allow incoming traffic on port(s) 443 (and 80).
    Use the UFW tab of Muppy's Host object for that.

Warning

For now, only the Public IP + Tailscale option is covered by support.

Installation workflow

The installation steps are:

  • Provision a Host with a supported Ubuntu LTS and enrol it in Muppy.
  • Install Tailscale (Task tailscale:tailscale_install)
  • Install microk8s (Task: k8s_cluster:install_single_node_microk8s)
  • Install PostgreSQL
  • Install the infrastructure packages (Task: k8s_cluster:install_single_node_infra_packages)

Tailscale

microk8s

Create a Task Run with the install_single_node_microk8s Task and fill in the parameters:

  • k8s_private_cidr_obj — the IPv4 CIDR of the private interface (e.g. the Tailscale IP)
  • k8s_public_cidr_obj — the IPv4 CIDR of the public interface, i.e. the host's public IP
  • system_resources_reservation

The Task also accepts microk8s_channel, setup_ufw_routed_default_policy, kubeconfig_server_cidr_obj and qualifier_obj, each with a usable default.

Warning

This Task reboots and reconfigures the host, which destroys the SSH connection it runs on. That is expected.

Lean on the help attached to each parameter (and tell us when the help is rubbish).

system_resources_reservation

This is the trickiest parameter!

A SingleNode cluster hosts every service on a single machine. The K8s cluster and the database in particular compete for resources (ram, cpu, disk).

The system_resources_reservation parameter restricts the resources allocated to the cluster, by declaring the resources reserved for the system. Kubernetes will then arrange to leave the system the resources you define.

The parameter is a JSON object made of the following (optional) keys: 

{
  "cpu": "2", // (or "200m") cores reserved for system and non k8s daemons
  "memory": "56G", // (or 1500M) size of RAM reserved for system and non k8s daemons
  "ephemeral-storage": "200G", // disk space reserved for system and non k8s daemons
  "pid": "2000" // number of pid reserved for system and non k8s daemons
}

Some documentation, if you want to dig into the subject: Kubernetes docs

Danger

On a Production server, you must set values for system_resources_reservation.

Tip

On a Test or Dev server you can leave these values out, but reserving a few resources is still better — it keeps SSH working should anything go wrong.

At the end of the Task Run you get a K8s Cluster with the same Qualifier as the Host it is installed on.

PostgreSQL

If you want to use PostgreSQL, install it now.

Follow the Muppy documentation (https://docs.muppy.io).

The only specific point is the UFW configuration. You must:

  • allow traffic from the private network, or from the cluster if PostgreSQL is on a dedicated server
  • allow traffic from the Pods if the database is on the kubernetes worker.

Info

To find the Pods' CIDR, run this command on the node: sudo grep -R 'cluster-cidr=' /var/snap/microk8s/current/args/. Unless it has been changed, the Pods' CIDR is 10.1.0.0/16 on microk8s (the default Calico Pods Range) and 10.244.0.0/24 on K0s.

  • allow traffic from a Replica, if there is one

Infrastructure Packages

Create a Task Run with the install_single_node_infra_packages Task and fill in the parameters:

  • k8s_cluster_obj
  • k8s_private_cidr_obj
  • k8s_public_cidr_obj
  • acme_registration_email
  • default_dns_domain_id
  • cluster_security_mode
  • allowed_cidr_dynamic_range_run_obj
  • ...
  • http_basic_auth_vault_obj

Please check Param. Description of these params for help.

References information

  • https://microk8s.io/docs/configure-host-interfaces -