Kubernetes

The cluster runs Talos Linux on Hetzner Cloud, provisioned through OpenTofu using the hcloud-k8s/kubernetes/hcloud module. Everything below the application layer — VMs, network, storage, certificates — is defined in flux-clusters/stefanzhelev/terraform/hcloud/.

Cluster topology

Node pool	Server type	Count	Role
control	CX33	3	Control plane (etcd, API server)
stateful-ha	CX43	3	HA workloads (Vault, CNPG, ClickHouse)
stateful-single	CX43	1	Single-instance stateful apps
stateless-single	CX53	1	Stateless workloads

Workloads are placed on pools by node selectors so HA databases stay on the three-node pool while one-off services share the cheaper single-node pools.

Networking

Setting	Value
Network CIDR	10.10.0.0/16
CNI	Cilium
Encryption	WireGuard (node-to-node)
Gateway API	Enabled
kube-proxy	Replaced by Cilium

Storage

Storage class	Encrypted	Default	Reclaim policy
hcloud-volumes-encrypted	Yes	Yes	Retain
hcloud-volumes	No	No	Retain

Talos state and ephemeral partitions are encrypted at the OS layer in addition to the CSI-level volume encryption.

Security posture

• Talos state and ephemeral partition encryption enabled
• Hetzner firewall rules auto-configured against the operator’s current IP
• OIDC authentication for the Kubernetes API delegated to Authentik
• TLS certificates issued by cert-manager with the Let’s Encrypt production issuer
• etcd backups written to S3-compatible storage

Provisioning with OpenTofu

Infrastructure lives in flux-clusters/stefanzhelev/terraform/hcloud/. Terraform state is stored remotely in Cloudflare R2:

backend "s3" {
  bucket = "stefanzhelev-tf-state"
  key    = "clusters/stefanzhelev-hcloud.tfstate"
  endpoints = { s3 = "https://<account-id>.r2.cloudflarestorage.com" }
  region = "auto"
}

Initialize and plan

cd flux-clusters/stefanzhelev/terraform/hcloud
tofu init
tofu plan

Apply infrastructure changes

tofu apply

Destroy the cluster

tofu destroy

Caution

Talos machine resets can timeout during destroy. If a talos_machine_configuration_apply resource gets stuck, remove it from state manually:

tofu state rm 'module.kubernetes.talos_machine_configuration_apply["<node-name>"]'

Then re-run tofu destroy.

Talos commands

Talos is managed entirely through its API — there is no SSH access. The talosctl CLI handles every node operation.

Configuration

# Set talosconfig from the Terraform output
export TALOSCONFIG=$(pwd)/talosconfig

# Verify connectivity
talosctl health

Node information

talosctl get members
talosctl dashboard
talosctl logs kubelet
talosctl logs etcd
talosctl services

Cluster operations

# Get kubeconfig
talosctl kubeconfig ./kubeconfig

# Upgrade Talos on a node
talosctl upgrade --nodes <node-ip> \
  --image ghcr.io/siderolabs/installer:<version>

# Upgrade Kubernetes
talosctl upgrade-k8s --to <k8s-version>

# Reset a node
talosctl reset --nodes <node-ip> --graceful

Troubleshooting

talosctl etcd status
talosctl etcd members
talosctl usage /var
talosctl get machineconfig -o yaml
talosctl reboot --nodes <node-ip>