Flux CD

Flux CD is the GitOps engine that keeps the cluster in sync with this repository. Every application, configuration change, and infrastructure update flows through Git — there is no out-of-band kubectl apply.

Git push → Flux detects change → Kustomization reconciles → HelmRelease installs/upgrades

Repository structure

flux-clusters/stefanzhelev/
├── cluster.yaml              # Cluster metadata
├── kustomization.yaml         # Root kustomization
├── flux-system/
│   ├── gotk-components.yaml   # Flux controllers (from `flux bootstrap`)
│   └── gotk-sync.yaml         # GitRepository + root Kustomization
├── terraform/
│   └── hcloud/                # OpenTofu for the underlying cluster
└── apps/
    ├── apps.yaml              # App orchestration (sync waves)
    ├── airflow/               # one folder per Flux Kustomization
    ├── authentik/
    ├── ...

The cluster-specific apps/<name>/ folders are thin Kustomization wrappers — each one points at a base in flux-apps/<name>/ at the repo root, where the actual HelmReleases, ExternalSecrets, and Terraform CRs live. This split keeps cluster-specific overlays separate from reusable app definitions.

Bootstrap

flux-system/gotk-sync.yaml declares two resources that together form the GitOps root:

• A GitRepository named flux-system that polls this repo’s main branch every minute.
• A Kustomization that reconciles ./flux-clusters/stefanzhelev against the cluster every ten minutes.

Once those exist, Flux is self-managing — every other Kustomization in the cluster is a child of the root.

Sync waves

Applications are deployed in dependency order through apps/apps.yaml. Each entry is a Kustomization CR with explicit dependsOn so a downstream service waits for its prerequisites to become ready.

Wave	Apps	Depends on
1	tofu-controller, external-secrets-crds, vault, airflow, dremio	—
2	external-secrets-config	vault, external-secrets-crds
3	cloudnative-pg	external-secrets-config, tofu-controller
4	authentik, forgejo	cloudnative-pg, external-secrets-config
5	clickhouse, clickstack, grafana	cloudnative-pg, clickhouse, external-secrets-config

The dependency graph effectively reads: Vault must be up before secrets can be pulled, secrets must be available before databases can be provisioned, databases must exist before any consuming app can boot.

Tofu Controller

Tofu Controller sits inside the cluster and reconciles Terraform CRs alongside Flux. It is what lets the GitOps loop reach beyond Kubernetes manifests:

• Vault paths and policies
• CloudNative-PG databases and users
• Authentik / Forgejo OIDC client secrets

allowCrossNamespaceRefs: true is set so a Terraform CR in one namespace can reference secrets and configs from another.

Key commands

# Check Flux system status
flux check

# View all Kustomizations and their status
flux get kustomizations

# View all HelmReleases
flux get helmreleases -A

# Force reconciliation of a specific app
flux reconcile kustomization vault

# Force reconciliation of a HelmRelease
flux reconcile helmrelease vault -n vault

# Suspend reconciliation (e.g., for manual debugging)
flux suspend kustomization grafana

# Resume reconciliation
flux resume kustomization grafana

# View Flux logs
flux logs --level=error

# Export current state of a resource
flux export kustomization vault > vault-kustomization.yaml