This is the twelfth article in our series on running the ClickHouse® database on Kubernetes with the Altinity® Kubernetes Operator. A cluster you can see still needs to be locked down. This article covers the security essentials: who can connect, how credentials are stored, and how to encrypt traffic.
The default security model
It helps to know what the operator gives you out of the box. By default it deploys two users, both protected by network rules. The default user exists for connections from within the pod and for distributed queries; historically it ships with an empty password, and the operator restricts it to the cluster's own pods so it cannot be reached from outside. The clickhouse_operator user is used by the operator itself to manage replicas and collect metrics; its credentials live in a Kubernetes Secret and its access is restricted to the operator's own pod by IP.
This is a sensible baseline, but for production you should go further: remove or properly password-protect the default user, give real users strong hashed credentials, encrypt connections, and tighten the network.
Hardening the default user
The cleanest hardening step is to disable the default user entirely, since your applications should connect as named users. You can do that by adding a small configuration file through the operator's files block that removes the user:
spec:
configuration:
files:
users.d/remove_default.xml: |
<clickhouse>
<users>
<default remove="1"/>
</users>
</clickhouse>If you do keep the default user for internal use, leave the operator's network restrictions in place so it stays unreachable from outside the cluster, and never expose it publicly.
Strong credentials for real users
We covered users in an earlier article; here is the security-focused summary. Never store a plain password in a CHI that lives in version control. Prefer a hash, computed locally, or better, keep the secret in a Kubernetes Secret and reference it. ClickHouse accepts password_sha256_hex and password_double_sha1_hex hashes, and the operator can read any of them from a Secret with valueFrom.secretKeyRef:
spec:
configuration:
users:
analyst/password_sha256_hex:
valueFrom:
secretKeyRef:
name: ch-credentials
key: analyst_hash
analyst/networks/ip:
- 10.0.0.0/8Two things make this secure: the password never appears in the manifest, and networks/ip restricts where the user may connect from. Tighten that range to your application's network rather than leaving it open to the world.
Encrypting connections with TLS
By default ClickHouse listens on plaintext ports. For production you want TLS so that traffic between clients and servers, and between servers, is encrypted. The operator turns this on with a single secure flag on the cluster, which opens the secure native port 9440 and the secure HTTP port 8443:
spec:
configuration:
clusters:
- name: "main"
secure: "yes"
layout:
shardsCount: 1
replicasCount: 2You can also set a TLS policy, for example requiring at least TLS 1.3 and strict certificate verification, under the cluster's security block:
spec:
configuration:
clusters:
- name: "main"
secure: "yes"
security:
clickhouse:
tls:
verify: Strict
minVersion: "1.3"
layout:
shardsCount: 1
replicasCount: 2Providing certificates
TLS needs a certificate and private key. The production pattern is to store them in a Kubernetes Secret and mount them into the ClickHouse pods, then point ClickHouse at the mounted files. You create a TLS Secret from your certificate and key:
kubectl create secret tls clickhouse-tls -n ch \
--cert=server.crt --key=server.keyThen in the pod template you mount that Secret as a volume, and you supply a small OpenSSL configuration through the files block that references the mounted certificate and key paths. Many teams generate and rotate these certificates automatically with cert-manager, which integrates cleanly with this approach. The key idea is that certificates, like passwords, live in Secrets and are mounted in, never baked into the image or the manifest.
Connecting over TLS
Once the secure ports are open, clients connect with TLS by using the secure port and the --secure flag:
kubectl exec -it -n ch chi-ch-main-0-0-0 -- \
clickhouse-client --secure --port 9440 -u analyst --password '...' -q 'SELECT 1'For applications, point your driver at port 9440 (native) or 8443 (HTTPS) and enable TLS in its connection settings.
Securing Keeper traffic too
The coordination layer deserves encryption as well. The Keeper resource accepts the same secure flag, which opens the TLS Keeper port 2281 so that ClickHouse and the Keeper nodes talk over TLS. You can run both the plaintext and secure ports during a migration, then lock down to TLS only once every client has moved over. Apply the same certificate-from-Secret pattern to Keeper.
Network hardening as defense in depth
Encryption and user rules protect the database; Kubernetes NetworkPolicies protect the surrounding network. A NetworkPolicy lets you declare exactly which pods may talk to your ClickHouse pods, so even if something is misconfigured, only approved clients can reach the database ports. As a final layer, avoid exposing ClickHouse with a public LoadBalancer unless you genuinely need external access, and when you do, put it behind TLS and tight network rules.
A practical security checklist
In short, for a production cluster: disable or lock down the default user, create named users with hashed credentials stored in Secrets, restrict each user's networks/ip, enable secure: "yes" with a sane TLS minimum version, mount certificates from Secrets, secure Keeper with TLS, and add NetworkPolicies. Each step is a few lines in the CHI, and together they move you from a demo to something you can run safely.
What is next
Your cluster is encrypted and access-controlled. In the next article we tune how the pods themselves run: CPU and memory requests and limits, node placement, disruption budgets, and the pod template features that make a cluster production-ready.
