We were hardcoding configuration in Docker images. Different image for dev, staging, production. Changing a config value meant rebuilding and redeploying.

I learned about ConfigMaps and Secrets. Now we use one image across all environments, with configuration injected at runtime. Config changes deploy in seconds.

Table of Contents

The Problem

Our old approach:

# Dockerfile
FROM node:8
COPY . /app
WORKDIR /app

# Hardcoded config!
ENV DATABASE_HOST=prod-db.example.com
ENV API_KEY=abc123

RUN npm install
CMD ["node", "server.js"]

Issues:

  • Different image per environment
  • Secrets in image layers
  • Config changes require rebuild
  • Can’t reuse images

ConfigMaps

Store non-sensitive configuration.

Create from literal values:

kubectl create configmap app-config \
  --from-literal=DATABASE_HOST=postgres.default.svc.cluster.local \
  --from-literal=DATABASE_PORT=5432 \
  --from-literal=LOG_LEVEL=info

Create from file:

# config.properties
database.host=postgres.default.svc.cluster.local
database.port=5432
log.level=info

kubectl create configmap app-config --from-file=config.properties

Create from YAML:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  DATABASE_HOST: postgres.default.svc.cluster.local
  DATABASE_PORT: "5432"
  LOG_LEVEL: info
  config.json: |
    {
      "feature_flags": {
        "new_ui": true,
        "beta_features": false
      }
    }

kubectl apply -f configmap.yaml

Using ConfigMaps as Environment Variables

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
      - name: web-app
        image: web-app:1.0.0
        env:
        - name: DATABASE_HOST
          valueFrom:
            configMapKeyRef:
              name: app-config
              key: DATABASE_HOST
        - name: DATABASE_PORT
          valueFrom:
            configMapKeyRef:
              name: app-config
              key: DATABASE_PORT
        - name: LOG_LEVEL
          valueFrom:
            configMapKeyRef:
              name: app-config
              key: LOG_LEVEL

Or import all keys:

envFrom:
- configMapRef:
    name: app-config

Using ConfigMaps as Volume Mounts

Mount config files:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  template:
    spec:
      containers:
      - name: web-app
        image: web-app:1.0.0
        volumeMounts:
        - name: config
          mountPath: /etc/config
          readOnly: true
      volumes:
      - name: config
        configMap:
          name: app-config

Files appear in /etc/config/:

  • /etc/config/DATABASE_HOST
  • /etc/config/DATABASE_PORT
  • /etc/config/config.json

Mount specific keys:

volumes:
- name: config
  configMap:
    name: app-config
    items:
    - key: config.json
      path: app-config.json

File appears at /etc/config/app-config.json

Secrets

Store sensitive data (passwords, tokens, keys).

Create from literal:

kubectl create secret generic db-credentials \
  --from-literal=username=admin \
  --from-literal=password=super_secret_password

Create from file:

kubectl create secret generic tls-cert \
  --from-file=tls.crt=server.crt \
  --from-file=tls.key=server.key

Create from YAML:

apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
type: Opaque
data:
  username: YWRtaW4=        # base64 encoded "admin"
  password: c3VwZXJfc2VjcmV0X3Bhc3N3b3Jk  # base64 encoded

Note: Values must be base64 encoded!

echo -n "admin" | base64
# YWRtaW4=

Using Secrets as Environment Variables

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  template:
    spec:
      containers:
      - name: web-app
        image: web-app:1.0.0
        env:
        - name: DB_USERNAME
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: username
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: password

Or import all:

envFrom:
- secretRef:
    name: db-credentials

Using Secrets as Volume Mounts

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  template:
    spec:
      containers:
      - name: web-app
        image: web-app:1.0.0
        volumeMounts:
        - name: db-creds
          mountPath: /etc/secrets
          readOnly: true
      volumes:
      - name: db-creds
        secret:
          secretName: db-credentials

Files appear in /etc/secrets/:

  • /etc/secrets/username
  • /etc/secrets/password

Read in application:

with open('/etc/secrets/username') as f:
    username = f.read().strip()

with open('/etc/secrets/password') as f:
    password = f.read().strip()

TLS Secrets

Special type for TLS certificates:

kubectl create secret tls tls-secret \
  --cert=server.crt \
  --key=server.key

Use in Ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-app-ingress
spec:
  tls:
  - hosts:
    - example.com
    secretName: tls-secret
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-app
            port:
              number: 80

Docker Registry Secrets

Pull images from private registry:

kubectl create secret docker-registry regcred \
  --docker-server=registry.example.com \
  --docker-username=user \
  --docker-password=password \
  --docker-email=user@example.com

Use in pod:

apiVersion: v1
kind: Pod
metadata:
  name: private-app
spec:
  containers:
  - name: app
    image: registry.example.com/private-app:1.0.0
  imagePullSecrets:
  - name: regcred

Environment-Specific Configs

Different ConfigMap per environment:

Development:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: development
data:
  DATABASE_HOST: postgres-dev
  LOG_LEVEL: debug
  FEATURE_FLAGS: '{"new_ui": true}'

Production:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: production
data:
  DATABASE_HOST: postgres-prod
  LOG_LEVEL: warn
  FEATURE_FLAGS: '{"new_ui": false}'

Same deployment YAML, different namespace = different config!

Updating ConfigMaps

Update ConfigMap:

kubectl edit configmap app-config

Or apply new YAML:

kubectl apply -f configmap.yaml

Important: Pods don’t automatically reload!

Options:

  1. Restart pods manually
  2. Use rolling update
  3. Watch for changes in app code

Rolling update:

kubectl rollout restart deployment web-app

Immutable ConfigMaps

Kubernetes 1.19+ (we’re on 1.7, but good to know):

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
immutable: true
data:
  DATABASE_HOST: postgres

Can’t be modified. Must delete and recreate.

Benefits:

  • Prevents accidental changes
  • Better performance (kubelet doesn’t watch)

Best Practices

  1. Separate config from code - Never hardcode
  2. Use Secrets for sensitive data - Not ConfigMaps
  3. One ConfigMap per app - Don’t share between apps
  4. Version ConfigMaps - app-config-v1, app-config-v2
  5. Limit Secret access - Use RBAC
  6. Don’t log Secrets - Avoid printing env vars
  7. Encrypt Secrets at rest - Enable encryption in etcd

Our Setup

ConfigMap (app-config):

  • Database host/port
  • Log level
  • Feature flags
  • API endpoints

Secret (db-credentials):

  • Database username/password

Secret (api-keys):

  • Third-party API keys
  • JWT signing key

Secret (tls-cert):

  • TLS certificate/key

Real-World Example

Complete deployment:

apiVersion: v1
kind: ConfigMap
metadata:
  name: web-app-config
data:
  DATABASE_HOST: postgres.default.svc.cluster.local
  DATABASE_PORT: "5432"
  DATABASE_NAME: production
  REDIS_HOST: redis.default.svc.cluster.local
  LOG_LEVEL: info
---
apiVersion: v1
kind: Secret
metadata:
  name: web-app-secrets
type: Opaque
data:
  DB_PASSWORD: c3VwZXJfc2VjcmV0
  JWT_SECRET: and0X3NlY3JldF9rZXk=
  API_KEY: YXBpX2tleV8xMjM=
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
      - name: web-app
        image: web-app:1.0.0
        envFrom:
        - configMapRef:
            name: web-app-config
        - secretRef:
            name: web-app-secrets
        ports:
        - containerPort: 3000

Results

Before:

  • Config hardcoded in images
  • Different image per environment
  • Config changes = rebuild + redeploy (30 min)
  • Secrets in image layers

After:

  • Config in ConfigMaps/Secrets
  • One image for all environments
  • Config changes = kubectl apply (30 sec)
  • Secrets properly managed

Lessons Learned

  1. Use ConfigMaps early - Don’t hardcode config
  2. Secrets for sensitive data - Always
  3. Test config changes - In dev first
  4. Document config keys - What each key does
  5. Version control - Keep ConfigMap YAMLs in git

Conclusion

ConfigMaps and Secrets are essential for Kubernetes apps. They separate configuration from code, enabling true portability.

Key takeaways:

  1. ConfigMaps for non-sensitive config
  2. Secrets for passwords, keys, tokens
  3. One image, multiple environments
  4. Update config without rebuilding
  5. Use RBAC to protect Secrets

Stop hardcoding configuration. Use ConfigMaps and Secrets properly.