Encrypt etcd Secrets at Rest

By Kokil Thapa | Last reviewed: September 2026

When you need to encrypt etcd secrets at rest, you are really protecting two layers: Kubernetes Secret objects written into etcd, and the physical or virtual disks that store etcd data. A plain etcd snapshot or a stolen volume can expose base64-encoded credentials, TLS keys, and database passwords unless encryption is configured before those values ever hit disk. This guide walks through the Kubernetes API encryption path, provider choices, rotation, and the operational checks I use on production Linux infrastructure.

If you run workloads on managed Kubernetes, some controls are platform-owned. On self-managed clusters—the kind teams often pair with Linux system administration and custom deployment pipelines—you own the full stack. That includes how secrets leave the API server and how etcd backups are stored. The goal is defense in depth: encrypt at the API layer, lock down etcd access, and encrypt underlying storage.

What does it mean to encrypt etcd secrets at rest?

etcd is the key-value store behind Kubernetes. It holds cluster state: Deployments, ConfigMaps, RBAC rules, and Secret resources. Secrets are only base64-encoded in etcd by default. Base64 is encoding, not encryption. Anyone with etcd read access or a raw snapshot file can decode them in seconds.

Encrypt etcd secrets at rest by configuring the Kubernetes API server to encrypt resource fields before persistence. etcd then stores ciphertext. The API server decrypts on read using keys you control. This is separate from TLS, which protects data in transit between components.

A third layer—full-disk or volume encryption on etcd member nodes—protects against physical theft or cloud volume leaks. It does not replace API-level encryption. A backup taken from an unencrypted API path still contains readable secret payloads inside the etcd snapshot format.

Related reading: how etcd works as the Kubernetes cluster data store and Kubernetes Secrets and ConfigMaps done right cover storage mechanics and naming conventions that affect what you encrypt.

What etcd actually stores

Each Secret is a Kubernetes object serialized as JSON or protobuf, then written under a key such as /registry/secrets/default/my-tls. The data map holds base64 blobs. Without encryption at rest, those blobs decode to plaintext passwords, API tokens, or private keys.

ConfigMaps can also hold sensitive values by mistake. EncryptionConfiguration lets you encrypt secrets, configmaps, or other resources selectively. Most teams start with secrets only to limit performance overhead.

How do you enable Kubernetes encryption at rest for etcd?

Enable encryption on the API server, not inside etcd itself. The API server reads EncryptionConfiguration, picks the first matching provider for each resource, and encrypts before the etcd write path.

1. Create an encryption config file on every control plane node.
2. Generate a strong random key for local providers such as aescbc or aesgcm.
3. Add the --encryption-provider-config flag to the API server manifest or static pod.
4. Restart the API server and confirm the config loaded without errors.
5. Re-encrypt existing secrets so old plaintext entries are rewritten as ciphertext.

Step 1: Create EncryptionConfiguration

On a control plane node, create /etc/kubernetes/encryption-config.yaml. Use aesgcm or aescbc for self-managed clusters. Keep identity as the last provider so the API server can still read legacy plaintext during migration.

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
    providers:
      - aescbc:
          keys:
            - name: key1
              secret: <base64-encoded-32-byte-key>
      - identity: {}

Generate a 32-byte key and base64-encode it. On Linux you can use OpenSSL:

head -c 32 /dev/urandom | base64 -w0

Restrict file permissions. I set 600 and root ownership, similar to how I protect TLS keys on Ubuntu servers.

chmod 600 /etc/kubernetes/encryption-config.yaml
chown root:root /etc/kubernetes/encryption-config.yaml

Step 2: Wire the API server

For kubeadm clusters, edit the static pod at /etc/kubernetes/manifests/kube-apiserver.yaml. Add these flags under spec.containers[0].command:

--encryption-provider-config=/etc/kubernetes/encryption-config.yaml
--encryption-provider-config-automatic-reload=true

The automatic reload flag lets you rotate keys without a full API server restart when using supported provider changes. After saving, kubelet restarts the pod. Watch logs until the apiserver reports healthy.

kubectl -n kube-system logs kube-apiserver-$(hostname) | grep -i encryption

Step 3: Verify and re-encrypt existing secrets

New secrets encrypt immediately. Existing objects may still be stored with the identity provider until rewritten. Force re-encryption cluster-wide:

kubectl get secrets --all-namespaces -o json | kubectl replace -f -

For large clusters, batch by namespace to reduce API load. Confirm encryption by reading raw etcd data only on a test cluster. In production, rely on audit logs and documented runbooks instead of direct etcd reads.

Pair this with Kubernetes disaster recovery and etcd backup practices. Encrypted etcd snapshots are useless to attackers without your encryption keys, but restore drills still matter.

How do you choose an encryption provider for etcd secrets?

Kubernetes supports several providers in EncryptionConfiguration. The first listed provider that matches a resource type wins for writes. Reads walk the list until one provider can decrypt the stored value.

For cloud-native teams, KMS v2 integrates with AWS KMS, Google Cloud KMS, Azure Key Vault, or a KMS plugin backed by Vault. The data encryption key is wrapped by the KMS master key. Compromise of an etcd snapshot without KMS decrypt permission blocks secret recovery.

External secret managers still help at the application layer. See External Secrets Operator with Vault and secrets management with HashiCorp Vault for syncing credentials into the cluster. Encryption at rest protects what lands in etcd; Vault reduces how many long-lived secrets you store there at all.

Example KMS v2 snippet (conceptual)

Exact plugin names vary by vendor. The structure follows the upstream Kubernetes format documented at kubernetes.io/docs/tasks/administer-cluster/encrypt-data/:

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
    providers:
      - kms:
          apiVersion: v2
          name: my-kms-plugin
          endpoint: unix:///var/run/kms-plugin/socket.sock
          cachesize: 1000
          timeout: 3s
      - identity: {}

Run the KMS plugin as a DaemonSet or static pod on control plane nodes. Test failover: if KMS is unreachable, the API server cannot decrypt existing secrets. That is correct security behavior, but it becomes an availability incident without HA key services.

What is the difference between etcd disk encryption and API encryption?

Teams often conflate these two controls. Both belong in a hardened cluster, but they solve different problems.

• API encryption at rest — Kubernetes encrypts Secret fields before etcd persistence. Backups taken via etcdctl snapshot save contain encrypted payloads when this layer is active and secrets have been re-encrypted.
• etcd disk / volume encryption — LUKS, BitLocker, or cloud volume encryption protects every byte on the etcd data directory at the block level. It helps if someone steals a disk or clones a volume without OS credentials.
• etcd peer TLS — Protects replication traffic between members. It does not encrypt stored values on disk.

On Ubuntu servers I maintain, full-disk encryption and tight filesystem permissions are baseline steps. They mirror the same mindset as locking down .env files on Laravel deployments. Neither replaces application- or API-level secret handling.

Database teams face a parallel split. Our database encryption at rest and in transit guide covers MySQL and PostgreSQL patterns that application teams expect when secrets point to managed databases.

How do you rotate encryption keys for etcd secrets safely?

Key rotation is not optional for long-lived clusters. Compliance frameworks and internal policy often require annual—or faster—rotation. Kubernetes supports multiple named keys in one provider block. The first key encrypts new writes. Older keys remain for decrypting existing data until you re-encrypt.

Rotation procedure

1. Add a new key entry at the top of the keys list with a unique name.
2. Reload or restart the API server so it picks up the config.
3. Run the cluster-wide secret replace command to rewrite objects with the new key.
4. Remove retired key entries only after every secret has been re-encrypted and backups are updated.
5. Document the rotation date and store offline key material in your break-glass vault.

providers:
  - aescbc:
      keys:
        - name: key2
          secret: <new-base64-key>
        - name: key1
          secret: <old-base64-key>
  - identity: {}

With KMS, rotation often means updating the KMS key version or plugin config. Cloud providers expose automatic key rotation. Confirm your plugin uses the current primary key for wrap operations. See encrypt data with AWS KMS keys, policies, and rotation for the AWS-side mechanics that pair with EKS control planes.

Backup and restore implications

An etcd restore brings back ciphertext only when API encryption was active at backup time. You must restore the same EncryptionConfiguration keys or maintain KMS access with equivalent permissions. I treat encryption keys as part of disaster recovery assets, alongside backup schedules and restore runbooks.

Never store encryption keys in the same S3 bucket as etcd snapshots. Use a separate secrets vault, hardware security module, or password manager with audit trails. A password generator helps create strong break-glass credentials for vault access, but encryption keys themselves should come from cryptographically secure random sources.

Operational hardening checklist

Encryption at rest is one control in a wider secrets program. These items reduce the chance that etcd becomes your weakest link.

• Restrict etcd client access to control plane nodes; firewall port 2379 from worker subnets.
• Enable audit logging on the API server for Secret read and write events.
• Prefer short-lived credentials via external managers instead of static Secret blobs.
• Scan Git and CI for leaked kubeconfigs or encryption config files — see secrets scanning in Git and CI with Gitleaks.
• Integrate pipeline secrets separately — handle secrets in CI/CD pipelines safely and CI/CD secrets management best practices cover build-time exposure.
• Use Sealed Secrets or GitOps patterns where manifests live in Git — manage secrets in GitOps with Sealed Secrets.

For enterprise platforms that mix on-prem and cloud, read multi-cloud secrets management before centralizing keys. A client portal on Laravel—such as work I've done on secure document-sharing platforms—still depends on infrastructure teams protecting cluster credentials that unlock application secrets.

Managed Kubernetes on EKS, GKE, or AKS may enable encryption at rest by default with cloud KMS. Self-managed kubeadm clusters on VPS or bare metal—common for budget-sensitive teams in Nepal and elsewhere—require explicit configuration. Budget roughly Rs 15,000–40,000/month (~USD 110–295) for a three-node control plane footprint before KMS fees, if you self-host on cloud VMs.

Official etcd security guidance at etcd.io/docs/latest/op-guide/security/ covers mTLS, certificate rotation, and member isolation. Pair that with the Kubernetes Secret concept docs at kubernetes.io/docs/concepts/configuration/secret/ so developers understand what actually gets encrypted.

If your platform team needs help designing the wider stack—not only etcd—see enterprise application development and testing and optimization for how application and infrastructure security fit together. For day-two operations, support and maintenance covers the ongoing runbooks encryption rotation depends on.

Key Takeaways

• Encrypt etcd secrets at rest via Kubernetes EncryptionConfiguration on the API server—not by expecting base64 encoding to protect data.
• Always keep identity last during migration, then re-encrypt existing Secret objects so nothing stays in plaintext.
• Use KMS v2 in production; local aesgcm or aescbc keys are acceptable only when you can guard the config file like a root CA.
• Combine API encryption with disk encryption and strict etcd network policy for defense in depth.
• Include encryption keys in backup and disaster recovery docs; restores fail silently without matching keys.
• Reduce etcd secret volume with External Secrets, Vault, or Sealed Secrets so fewer credentials persist in the data store at all.

People Also Ask

Does Kubernetes encrypt secrets in etcd by default?

No. By default, Secret values are base64-encoded and stored as plaintext inside etcd's key-value payload. You must enable encryption at rest explicitly through the API server EncryptionConfiguration. Managed offerings may enable KMS-backed encryption by default—check your provider's documentation.

Can I encrypt etcd secrets without restarting the cluster?

You can avoid a full cluster restart. Adding --encryption-provider-config-automatic-reload=true lets the API server reload config changes. Static pod edits still restart the API server process on that node. Plan maintenance windows for multi-control-plane clusters and roll one node at a time.

Is etcd encryption the same as Sealed Secrets?

No. Sealed Secrets encrypt Secret manifests before they reach the API, using a controller key pair. API encryption at rest protects data after admission, at etcd persistence time. Many teams use both: Sealed Secrets for GitOps storage, API encryption for defense if etcd or backups leak.

What happens if I lose the encryption key?

Encrypted Secret objects become unreadable. Applications lose access to credentials stored only in those secrets. Recovery requires key backup or KMS access you configured during setup. There is no backdoor. Treat key backup as mandatory, not optional.

Next steps for your cluster

Encrypt etcd secrets at rest before your first production backup, not after an audit finding. Start in staging: apply EncryptionConfiguration, confirm new secrets encrypt, run the re-encryption pass, and restore from an etcd snapshot to validate keys travel with your DR plan. Then extend the same discipline to pipelines, Git repos, and application-level secret stores covered in Kubernetes secrets management done right.

Need help hardening infrastructure, deployment pipelines, or the applications that depend on cluster secrets? Contact us to discuss Linux administration, API design, and production security for your stack. You can also browse the blog for related guides or learn more about my infrastructure and development work.