kuhn-cloud

This uses Kube-Hetzner to setup the k8s cluster on hetzner cloud.

To get started you need to copy secrets.auto.tfvars.example to secrets.auto.tfvars and fill the secrets.

Dashboards

Kubernetes Dashboard

You need to create a token to get access:

kubectl -n kube-system create token admin-user

Longhorn

Bla

Setup

Install

You need to have opentofu and kubectl installed.

Update images

export HCLOUD_TOKEN=$(echo "nonsensitive(var.hcloud_token)" | terraform console -var-file secrets.auto.tfvars | sed -e 's/^"//' -e 's/"$//')

cd cluster/kube
curl -sL https://raw.githubusercontent.com/kube-hetzner/terraform-hcloud-kube-hetzner/master/packer-template/hcloud-microos-snapshots.pkr.hcl -o "hcloud-microos-snapshots.pkr.hcl"
packer init hcloud-microos-snapshots.pkr.hcl
packer build hcloud-microos-snapshots.pkr.hcl

Run

OpenTofu

tofu init --upgrade
tofu validate
tofu apply -auto-approve

kubectl

Normally the kube config is created automatically. Otherwise you can get it via

terraform output --raw kubeconfig > k3s_kubeconfig.yaml

Now you can use kubectl like this:

export KUBECONFIG="$(pwd)/k3s_kubeconfig.yaml"

kubectl version

You might need to rerun tofu, because for now the kube api is blocked by the firewall and only the local IP is allowed. Since it will change regularly you also need to update the ip by applying tofu agian.

Setup

To setup the base line of cluster services like monitoring, storange, etc. you need to run setup.sh to make sure all are setup and installed in the correct order.

Secrets

Unfortunately it's not possible to provide the secrets to all places via the kubernetes secrets. Therefore the with_secrets.sh script exists which populates the secrets correctly. The secrets have to be stored in secrets.env. Copy .secrets.env.example and fill it with the correct secrets.

Now you can create those via

./with_secrets.sh apply service/hello

and delete them via

./with_secrets.sh delete service/hello

Development

Work with helm

Use kustomize to keep helm charts as IaC (see setup/ for examples). To see the different helm chart config values you can use following or check out the values.yaml of the source code (in this exampe kube-prometheus-stack/values.yaml)

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm show values prometheus-community/kube-prometheus-stack | less

Useful commands

Delete

To delete a whole namespace use:

kubectl -n monitoring delete all --all

List

List all services

kubectl -n prometheus get services

List all ingress

kubectl -n prometheus get ingress

List all pods

kubectl -n prometheus get pods

If you can't find it you can normally use --all-namespaces instead of -n ... (Must be after get pods to work).

Describe

You can use describe to get more details whats going on.

kubectl -n prometheus describe ingress

Logs

You can read the logs with the following command. To get the pod id use the list command from above.

kubectl -n prometheus logs <pod name>