All posts
Kubernetes for Absolute Beginners: A Gentle Introduction (with ClickHouse® in Mind)

Kubernetes for Absolute Beginners: A Gentle Introduction (with ClickHouse® in Mind)

April 28, 20269 min readSanjeev Kumar G
Share:

This series covers running the ClickHouse® database on Kubernetes with the Altinity® Kubernetes Operator, and it begins with Kubernetes itself. If you have never touched Kubernetes, start here. The article builds the platform up from first principles and keeps one eye on what a database needs from it, which is not always what a stateless web service needs.

By the end you will know what Kubernetes is, the handful of building blocks that show up again and again, and why a database leans on some of them harder than an ordinary application does.

What Kubernetes is, and the problem it solves

Say you have an application packaged as a container and you want to run it reliably. On a single laptop, that is trivial. Now require ten copies spread across five machines, automatic restarts when one crashes, new versions rolled out with no downtime, and storage that outlives any single machine. Coordinating all of that by hand is slow and easy to get wrong.

Kubernetes does that coordination for you. You describe the state you want in a text file, and it works continuously to make the running system match. A container dies, it starts another. A machine drops off the network, it reschedules that machine's work elsewhere. This idea, declaring the desired state and letting the system reconcile toward it, is the one thing to internalize before anything else. Everything else is built on top of it.

Containers, in one paragraph

Kubernetes runs containers, so a quick recap earns its place. A container bundles an application with its libraries and dependencies into a single isolated unit; Docker is the usual tool for building and running them. The image is the blueprint, and a running container is a live instance of that image. ClickHouse ships official images, so putting it in a container takes no special effort. Kubernetes takes those containers and runs them across a fleet of machines.

Clusters: the control plane and the nodes

A Kubernetes cluster is a group of machines that work together, split into two roles.

The control plane is the brain. It holds the desired state, schedules work, and runs the reconciliation loops that keep the system on course. Every request you make lands on the control plane's API server.

The nodes are the muscle. A node is a machine, virtual or physical, that actually runs your containers, and a cluster usually has several of them. Kubernetes decides which node each piece of work runs on. You will not manage the control plane yourself early on; tools like minikube set it up for you, which is where the next article picks up.

Pods: the smallest thing you deploy

You might expect to hand Kubernetes a container directly, but it wraps one or more containers in a slightly larger unit called a pod, and the pod is the smallest thing it schedules. Usually a pod holds a single main container, say one ClickHouse server. Containers in the same pod share a network address and can share storage, which suits helper containers that run alongside the main one.

The catch is that pods are disposable. Kubernetes can delete a pod and start a replacement at any moment, on any node, and that replacement comes up with a new network address. For a stateless app, fine. For a database it raises two immediate questions: how does anything find a pod whose address keeps changing, and what happens to the data on disk when the pod is replaced? Services and persistent storage are the answers, and both come up shortly.

Controllers: holding the right number of pods

You rarely create pods directly. You tell a higher-level object how many pods you want and what they should look like, and that object, a controller, keeps that many healthy pods running. Two controllers matter here.

A Deployment manages a set of identical, interchangeable pods. It fits stateless applications such as a web front end, where one pod is as good as another and rolling updates and scaling come for free.

A StatefulSet is the one databases need. Where a Deployment treats its pods as anonymous, a StatefulSet gives each pod a stable identity that survives restarts: the pods are numbered from zero, and each keeps its own dedicated storage. Replace a StatefulSet pod and the new one inherits the same identity and reattaches the same data. ClickHouse depends on exactly this, because a given replica has to stay the same replica, with the same data, across restarts. It is why the Altinity Kubernetes Operator builds ClickHouse clusters on StatefulSets rather than Deployments.

Services: a fixed address for moving pods

Because pods come and go with changing addresses, Kubernetes gives you a Service: a stable name and address that always points at the right set of pods, however often they are replaced. Clients talk to the Service, and Kubernetes routes them to a healthy pod behind it.

A few kinds come up. A ClusterIP Service is reachable only from inside the cluster, and it is the default. A NodePort opens a port on every node for outside access. A LoadBalancer asks your cloud provider for an external load balancer with a public address. A headless Service gives each pod its own stable DNS name instead of load-balancing across them, which is how a StatefulSet lets you address one database replica directly. The point to hold onto is simple: a Service is the stable front door to pods that are otherwise in constant flux.

Storage: keeping data when pods vanish

By default, whatever a container writes to its own filesystem disappears when the pod is replaced. For a database that is unacceptable, so Kubernetes keeps storage separate from pods through three linked ideas.

A PersistentVolume (PV) is a real piece of storage in the cluster, a cloud disk for instance. A PersistentVolumeClaim (PVC) is a workload's request for storage of a given size and type. Kubernetes binds the claim to a volume and attaches it to the pod. Because the claim is tied to the StatefulSet pod's identity rather than to the pod itself, the data outlives the pod: the replacement reattaches the same claim and the same data.

A StorageClass ties it together by describing a kind of storage the cluster can create on demand. With one in place, you do not pre-create disks; when a claim asks for storage, the cluster provisions a matching volume automatically, which is called dynamic provisioning. When you deploy ClickHouse later, you hand it a StorageClass and a size, and the storage appears. A later article is devoted entirely to ClickHouse storage, because getting it right is most of what separates a demo from a real deployment.

Configuration: ConfigMaps and Secrets

Applications need configuration, and some of it is sensitive. Kubernetes splits the two. A ConfigMap holds ordinary configuration such as settings files. A Secret holds sensitive values such as passwords and certificates, kept apart so they can be handled with more care. When you set up ClickHouse users later, their passwords live in Secrets rather than in plain text, and the operator wires them in.

Namespaces: keeping things separate

A namespace partitions a cluster into separate areas, much like folders. You might keep the database in one and monitoring in another. Namespaces stop names from colliding and make access control simpler. In this series the operator and ClickHouse each go into their own namespace to keep things orderly.

kubectl: talking to the cluster

You drive a cluster with a command-line tool called kubectl. You apply a YAML file describing what you want with kubectl apply, list things with kubectl get, and inspect one object in detail with kubectl describe. Nearly every step in this series runs through kubectl, and the handful of commands that matter become second nature fast.

Here is the shape of a minimal manifest, so the YAML feels familiar when it matters. Every object carries an apiVersion, a kind, some metadata such as a name, and a spec that describes what you want:

apiVersion: v1
kind: Pod
metadata:
  name: hello
spec:
  containers:
    - name: hello
      image: busybox
      command: ["sh", "-c", "echo Hello from Kubernetes && sleep 3600"]

You will rarely write a bare pod like this in practice, but it shows the four-part structure every Kubernetes object shares.

The operator pattern, and why this series exists

Everything so far is plain Kubernetes. Running a single container with these pieces is easy; running a correct, replicated ClickHouse cluster is not. That means coordinating StatefulSets, headless Services, persistent volumes, configuration, users, and a coordination service called ClickHouse Keeper, and holding them all consistent as you scale and upgrade. By hand, that is a great deal of fiddly, error-prone YAML.

The operator pattern is the answer. An operator is a program that runs inside the cluster and teaches Kubernetes about one specific application. You give it a single high-level description of the cluster you want, and it creates and manages the underlying objects the way an experienced operator would. The Altinity Kubernetes Operator does this for ClickHouse: instead of dozens of manifests, you describe a ClickHouseInstallation and it handles the rest. It arrives a few articles from now, once you have built ClickHouse the manual way and felt the work it removes.

Why run ClickHouse on Kubernetes at all

The payoff is worth stating plainly. On Kubernetes you get self-healing pods, declarative configuration you can keep in version control, straightforward scaling of shards and replicas, rolling upgrades without downtime, and the same setup across clouds and on-premises. For an analytical database that tends to grow and shift over time, that operational leverage compounds. The price is the learning curve you are on right now, and the rest of the series is built to make it a gentle one.

Where this goes next

You now have the vocabulary: cluster, node, pod, Deployment, StatefulSet, Service, PersistentVolume, PersistentVolumeClaim, StorageClass, ConfigMap, Secret, namespace, and the operator pattern. The next article sets up a real Kubernetes cluster on your own machine with minikube and k3s, so you have somewhere to practice everything that follows.

References

Share: