August 17, 2026
10 min read
Table of Contents
By Kokil Thapa | Last reviewed: August 2026
Managing application configuration securely is one of the most common failure points in container orchestration. Getting Kubernetes Secrets and ConfigMaps done right requires understanding that native Kubernetes objects are not encrypted by default and that treating them like simple environment variables leads to security leaks and operational fragility. This guide provides the concrete patterns, tooling choices, and architectural decisions needed to manage sensitive data safely in production clusters during 2026.
Quick answer: Kubernetes Secrets and ConfigMaps done right means never storing raw credentials in Git, enabling encryption at rest for etcd, using External Secrets Operator or Sealed Secrets for GitOps workflows, and injecting sensitive values as mounted volumes rather than environment variables to prevent leakage through process listings and logs.
What is the difference between ConfigMaps and Secrets in Kubernetes?
While both objects store key-value pairs for pod configuration, they serve fundamentally different security domains. Understanding this distinction is the first step toward getting server and application security correct in a clustered environment. A common mistake I see on client projects is developers storing API keys in ConfigMaps because "it’s just a string," completely bypassing the access control and audit mechanisms reserved for Secrets.
ConfigMaps are designed for non-sensitive configuration: feature flags, database hostnames, logging levels, and application properties. They are stored as plaintext in etcd and have no special handling in the Kubernetes API server beyond basic namespace scoping. Secrets, despite their name, are only base64-encoded by default. Base64 is an encoding scheme, not encryption. Anyone with read access to the namespace can decode them instantly. The real security value of Secrets comes from three capabilities that ConfigMaps lack:
- Encryption at rest: When properly configured via
EncryptionConfiguration, the API server encrypts Secret data before writing to etcd. ConfigMaps never receive this treatment. - RBAC granularity: You can grant read access to ConfigMaps while denying Secret access, implementing least-privilege principles essential for multi-team clusters.
- Audit logging: Kubernetes audit policies can be configured to log Secret access at metadata or request-body level, providing compliance trails that ConfigMaps don’t typically warrant.
In practice, I treat the distinction as a security contract: if losing the data would cause a breach, regulatory violation, or financial loss, it belongs in a Secret with encryption enabled. Everything else goes in a ConfigMap. This binary decision simplifies code review and prevents the gradual drift where "temporary" test credentials end up committed to ConfigMaps.
How do you securely manage Kubernetes Secrets in GitOps workflows?
The fundamental tension in GitOps is that Git repositories should be the single source of truth, but you must never commit plaintext secrets. In 2026, two mature patterns dominate for achieving Kubernetes Secrets and ConfigMaps done right within declarative workflows: External Secrets Operator (ESO) and Sealed Secrets. Each solves the problem differently, and choosing correctly depends on your infrastructure maturity and team size.
External Secrets Operator Pattern
ESO synchronizes secrets from external providers (AWS Secrets Manager, HashiCorp Vault, Azure Key Vault, GCP Secret Manager) into native Kubernetes Secrets. The Git repository contains only an ExternalSecret manifest referencing the remote key, never the value itself.
<!-- ExternalSecret manifest example -->
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: payment-gateway-creds
namespace: ecommerce-prod
spec:
refreshInterval: 1h
secretStoreRef:
name: aws-secrets-manager
kind: ClusterSecretStore
target:
name: payment-gateway-creds
creationPolicy: Owner
data:
- secretKey: stripe_api_key
remoteRef:
key: prod/payment/stripe
property: apiKey
- secretKey: webhook_secret
remoteRef:
key: prod/payment/stripe
property: webhookSecretThis pattern offers several advantages for production systems. Secret rotation happens externally without redeploying manifests. Access control lives in the cloud provider’s IAM system, which is typically more mature than Kubernetes RBAC alone. Audit logs are centralized in the provider’s console. For teams already invested in AWS or Azure, this avoids introducing additional infrastructure.
Sealed Secrets Pattern
Sealed Secrets uses asymmetric cryptography: developers encrypt secrets locally with a public key, commit the encrypted SealedSecret to Git, and the in-cluster controller decrypts them using a private key that never leaves the cluster. This works well for smaller teams or clusters without cloud provider integration, but introduces key management overhead and makes secret rotation more cumbersome since each change requires re-encryption and a new commit.
| Criteria | External Secrets Operator | Sealed Secrets | SOPS + Age/GPG |
|---|---|---|---|
| Secret storage location | Cloud KMS / Vault | In-cluster (encrypted) | Git (encrypted file) |
| Rotation complexity | Low (external update) | High (re-encrypt + commit) | Medium (decrypt + edit + re-encrypt) |
| Multi-cluster support | Native via ClusterSecretStore | Per-cluster controller | Manual key distribution |
| Infrastructure dependency | Cloud provider or Vault | None (in-cluster only) | None (git-only) |
| Best for | Production multi-team clusters | Small teams, air-gapped | Individual devs, simple setups |
For most production workloads I’ve deployed in 2026, External Secrets Operator is the default recommendation. The operational overhead of maintaining a Vault instance or managing Sealed Secrets keys across environments outweighs the benefits unless you have specific constraints like air-gapped networks or strict data residency requirements that prevent cloud KMS usage.
Why should you mount secrets as volumes instead of environment variables?
Environment variable injection is the default pattern in most tutorials, but it creates multiple attack surfaces that volume mounts eliminate. When building secure Laravel APIs or any application handling sensitive data, understanding this distinction prevents entire classes of vulnerabilities.
Environment variables are visible through /proc/<pid>/environ, meaning any command execution vulnerability, debugging session, or misconfigured monitoring agent can exfiltrate credentials. Child processes inherit the full environment, so sidecar containers or init containers running with different trust levels gain access to secrets they shouldn’t see. Crash handlers and error reporting tools frequently dump environment variables, sending credentials to third-party services. Most critically, environment variables are immutable after container start; rotating a secret requires restarting every pod.
Volume mounts solve all four problems. Files can be permissioned to restrict access to the application user only. They don’t appear in process listings or environment dumps. The kubelet automatically updates projected volumes when the underlying Secret changes (with a configurable sync period), enabling zero-downtime rotation. SubPath mounts allow exposing individual keys rather than entire Secrets, reducing blast radius.
# Secure volume mount pattern for Laravel/PHP apps
volumes:
- name: app-secrets
secret:
secretName: payment-gateway-creds
defaultMode: 0400 # Owner-read only
containers:
- name: laravel-app
volumeMounts:
- name: app-secrets
mountPath: /var/run/secrets/stripe-key
subPath: stripe_api_key
readOnly: true
- name: app-secrets
mountPath: /var/run/secrets/webhook-secret
subPath: webhook_secret
readOnly: trueThe tradeoff is application compatibility. Your code must read files instead of getenv(). For PHP/Laravel applications, this means configuring the framework to load secrets from file paths or using a custom config loader. The migration cost is real but finite, and the security payoff compounds over the application’s lifetime.
How do you enable encryption at rest for Kubernetes Secrets?
Without encryption at rest, Secrets are stored as base64-encoded plaintext in etcd. Any backup of etcd, any compromised etcd node, or any administrator with direct etcd access can read every secret in your cluster. Enabling encryption is non-negotiable for production and is a prerequisite for claiming Kubernetes Secrets and ConfigMaps done right.
- Generate an encryption key: Use
head -c 32 /dev/urandom | base64to create a 256-bit AES-CBC key. Store this key securely outside the cluster initially (password manager, cloud KMS). - Create EncryptionConfiguration: Define the encryption provider chain. Always include
aescbcorsecretboxas the first provider andidentityas fallback for reading unencrypted legacy data during migration. - Configure API server: Add
--encryption-provider-config=/etc/kubernetes/encryption/config.yamlto the kube-apiserver manifest. On managed clusters (EKS, GKE, AKS), this is typically a cluster setting rather than a manual flag. - Re-encrypt existing secrets: After enabling encryption, existing Secrets remain unencrypted until rewritten. Run
kubectl get secrets --all-namespaces -o json | kubectl replace -f -to force re-encryption. Verify withetcdctl get /registry/secrets/<namespace>/<name> | hexdump -C— encrypted data won’t show readable base64.
# EncryptionConfiguration example (Kubernetes 1.30+)
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
- secrets
providers:
- aescbc:
keys:
- name: key1
secret: <base64-encoded-32-byte-key>
- identity: {} # Fallback for reading unencrypted data during migrationIn 2026, prefer secretbox (XSalsa20-Poly1305) over aescbc for new deployments. It’s faster, doesn’t require padding, and has stronger authenticated encryption properties. Reserve aescbc for compatibility with older clusters. For highest-security environments, integrate with cloud KMS via the kms provider so the encryption key never touches disk in plaintext.
What are the common mistakes when using Kubernetes Secrets and ConfigMaps?
Even experienced teams fall into predictable traps. Recognizing these patterns early prevents incidents during audits or breaches. I’ve encountered each of these on real production systems, often during post-incident reviews or security assessments for clients migrating legacy applications to Kubernetes.
Mistake 1: Confusing base64 encoding with encryption. This bears repeating because it causes real breaches. Base64 is reversible by anyone. If your threat model includes "anyone with etcd read access," base64 provides zero protection. Always verify encryption is active before storing production credentials.
Mistake 2: Missing namespace-scoped RBAC. By default, many clusters grant broad Secret read access to service accounts. Apply least-privilege RBAC: application service accounts should only read Secrets in their own namespace, and only specific named Secrets when possible. Cross-namespace Secret access is almost always a design flaw.
Mistake 3: Hardcoding Secret names in application code. Applications should reference configuration paths, not specific Secret names. This decouples deployment from application logic and enables testing with dummy values. Use ConfigMaps for non-sensitive defaults and override with Secrets at deploy time.
Mistake 4: Ignoring Secret size limits. Kubernetes Secrets are limited to 1MB. Large certificates, key bundles, or configuration files exceeding this limit fail silently or cause pod startup failures. Split large payloads across multiple Secrets or use volume-mounted ConfigMaps for non-sensitive portions.
Mistake 5: Not testing backup restoration. Encrypted Secrets are useless if the encryption key is lost. Regularly test etcd backup restoration in an isolated environment. Document key recovery procedures. Teams that skip this discover during disasters that their backups contain gibberish.
Implementing Kubernetes Secrets and ConfigMaps Done Right in Production
Getting Kubernetes Secrets and ConfigMaps done right is not a one-time setup but an ongoing discipline combining encryption, access control, secure injection patterns, and operational verification. Start by enabling encryption at rest today if you haven’t already. Migrate environment-variable-based Secret consumption to volume mounts during your next maintenance window. Adopt External Secrets Operator or Sealed Secrets to remove plaintext credentials from Git entirely. Audit RBAC bindings quarterly to catch permission creep. These steps compound into a configuration management posture that survives audits, scales across teams, and actually protects the sensitive data your applications depend on.
If your team needs hands-on implementation support for secure Kubernetes configuration management, reach out to discuss your specific cluster architecture and security requirements.
Frequently Asked Questions
ConfigMaps store non-sensitive configuration data as plain text key-value pairs or files, while Secrets are specifically designed for sensitive credentials like passwords, API keys, and TLS certificates. Although both use base64 encoding by default, Secrets receive additional security controls including encryption at rest options, restricted RBAC policies, and audit logging capabilities that ConfigMaps do not provide natively.
No, they are only base64 encoded.
You must configure an EncryptionConfiguration resource on the API server specifying a provider like aescbc, secretbox, or kms. Without this explicit configuration, etcd stores Secret values in plaintext regardless of their object type. On managed clusters like EKS or GKE, enable KMS envelope encryption through the cloud console rather than editing API server flags directly. Always verify encryption status using etcdctl to read raw bytes after enabling the provider.
Both are limited to 1MB per object by the API server. Exceeding this causes creation failures or silent truncation during updates. For larger configurations, mount external storage via PersistentVolumeClaims, use CSI drivers for secret injection, or split content across multiple objects with application-level assembly. In my experience deploying Laravel applications, large .env files often hit this boundary when containing embedded certificates or lengthy third-party service tokens requiring refactoring.
Prefer volume mounts for production workloads. Environment variables leak into process listings, crash dumps, child processes, and container logs. Mounted files can be updated without pod restarts when using subPath or projected volumes with atomic writes. However, many frameworks including Laravel expect specific env var names. A practical compromise is mounting a file and sourcing it in an init container, keeping runtime containers free of direct secret exposure while maintaining framework compatibility.
Use projected volumes instead of direct secretKeyRef references. Projected volumes automatically propagate updates to mounted files within the kubelet sync period, typically 60 seconds. Note that subPath mounts do not receive automatic updates; you must remove subPath and mount the entire secret directory. Applications must watch filesystem changes or periodically re-read files since most frameworks cache configuration at boot. This pattern works reliably for PHP-FPM configurations I have deployed where graceful reload suffices.
Native Secrets store credentials in etcd, creating a single high-value target. External Secrets Operator synchronizes secrets from AWS Secrets Manager, HashiCorp Vault, or Azure Key Vault into Kubernetes objects without persisting source values permanently. It enables rotation without redeployment, centralizes access control outside cluster boundaries, and provides audit trails independent of Kubernetes RBAC. For Nepal-based clients using local infrastructure without managed KMS, I recommend starting with sealed-secrets before adopting full external secret management.
Never commit raw Secrets manifests. Use SealedSecrets, SOPS, or git-crypt to encrypt sensitive YAML before version control. Configure pre-commit hooks with tools like gitleaks or trufflehog to scan staged files. In CI pipelines, validate manifests against policy engines like OPA Gatekeeper rejecting unencrypted Secret resources. On projects I maintain using Deployer 7 and GitLab CI, secrets live exclusively in CI variable stores or external vaults, never in repository history even in encrypted form.
Follow least privilege strictly. Grant get and list only on specific named secrets, never wildcard access to all secrets in a namespace. Service accounts running application pods rarely need direct Secret API access when using volume mounts; restrict permissions to deployment controllers instead. Audit all Secret access using Kubernetes audit logs filtered for secrets resource. Separate RBAC roles for developers versus operators prevents routine debugging sessions from exposing production credentials unnecessarily across team members.
Not directly, as Secrets are namespace-scoped objects. Options include copying secrets via automation tools like Reflector or Kyverno, using ClusterSecretStore with External Secrets Operator, or implementing a custom controller watching source namespaces. Avoid granting cross-namespace RBAC bindings as this defeats isolation guarantees. For multi-tenant Laravel deployments I have architected, each tenant namespace receives its own synchronized copy managed centrally, ensuring credential rotation propagates consistently without compromising namespace security boundaries.
Check Events first using kubectl describe pod to identify FailedMount or CreateContainerConfigError messages referencing specific secret keys. Verify the secret exists in the correct namespace with exact spelling and case sensitivity. Confirm RBAC allows the pod service account access if using projected service account tokens. Validate base64 encoding lacks trailing newlines which cause silent decoding failures. In my troubleshooting experience, mismatched key names between manifest references and actual secret data account for most startup failures after deployments.
Existing pods continue running with cached values until restart. New pods or rescheduled replicas fail immediately with CreateContainerConfigError. Volume-mounted secrets become inaccessible upon next kubelet sync cycle causing application errors. Deletion is irreversible without backups. Before removing any Secret, grep cluster manifests and external secret stores for references. Implement soft deletion patterns marking secrets deprecated before removal, allowing observation periods to catch hidden dependencies in cron jobs or batch processors.
Adopt blue-green secret naming conventions appending version suffixes like db-password-v2. Update deployments referencing new versions before deleting old ones. Use External Secrets Operator refresh intervals for automated rotation from upstream providers. Test rotation procedures in staging environments matching production topology. Applications must support dynamic credential reloading or tolerate brief dual-validity windows. For payment integrations I have built connecting eSewa and Khalti, overlapping validity periods prevent transaction failures during gateway credential transitions.
Absolutely not. Base64 is encoding, not encryption, trivially reversible by anyone with read access. It exists solely to allow binary data in YAML/JSON formats. Relying on base64 alone provides zero security benefit beyond obscuring plaintext visually. Enable encryption at rest, enforce strict RBAC, implement network policies limiting API server access, and integrate external secret managers for defense in depth. Treat base64-encoded values in manifests as effectively public documentation requiring redaction before sharing logs or screenshots.
Native Secrets with encryption at rest costs nothing beyond cluster overhead. External Secrets Operator is open-source but requires backend infrastructure like Vault Enterprise ($5,000+ USD annually) or cloud KMS fees (~Rs 3,000–8,000 NPR monthly depending on usage). Managed solutions like AWS Secrets Manager charge $0.40 USD per secret monthly plus API calls. For Nepal startups, sealed-secrets offers free GitOps-compatible encryption without external dependencies, balancing security needs against budget constraints common in regional development contexts.
