Our security team ran a vulnerability scan on our Docker images. The results were alarming: 247 CVEs in our production images.

I spent three weeks hardening our images. We’re now down to 12 CVEs - a 95% reduction. Here’s how.

Table of Contents

The Wake-Up Call

Initial scan results:

$ docker scan myapp:latest

Testing myapp:latest...

 High severity vulnerabilities: 43
 Medium severity vulnerabilities: 128  
 Low severity vulnerabilities: 76

Total: 247 vulnerabilities

Most vulnerabilities came from:

  • Outdated base images
  • Unnecessary packages
  • Running as root
  • Secrets in images

Step 1: Use Minimal Base Images

Original Dockerfile:

FROM ubuntu:18.04

RUN apt-get update && apt-get install -y \
    python3 \
    python3-pip \
    curl \
    git \
    vim \
    && rm -rf /var/lib/apt/lists/*

Vulnerabilities: 187

Why? Ubuntu includes hundreds of packages we don’t need.

New Dockerfile:

FROM python:3.7-alpine

# Only install what we need
RUN apk add --no-cache \
    ca-certificates

Vulnerabilities: 23 (88% reduction)

Alpine is minimal - only 5MB base image.

Step 2: Multi-Stage Builds

Separate build and runtime environments:

# Build stage - can have build tools
FROM python:3.7 AS builder

WORKDIR /app
COPY requirements.txt .
RUN pip install --user -r requirements.txt

# Runtime stage - minimal
FROM python:3.7-alpine

# Copy only dependencies, not build tools
COPY --from=builder /root/.local /root/.local
COPY app.py .

ENV PATH=/root/.local/bin:$PATH

USER nobody
CMD ["python", "app.py"]

Build tools (gcc, make, etc.) stay in build stage, not in production image.

Step 3: Don’t Run as Root

Running as root is dangerous:

# Bad - runs as root
FROM alpine
COPY app.py .
CMD ["python", "app.py"]

If container is compromised, attacker has root access.

Fix:

FROM alpine

# Create non-root user
RUN addgroup -g 1000 appuser && \
    adduser -D -u 1000 -G appuser appuser

# Change ownership
COPY --chown=appuser:appuser app.py .

# Switch to non-root user
USER appuser

CMD ["python", "app.py"]

Step 4: Scan Images in CI/CD

Add scanning to GitLab CI:

# .gitlab-ci.yml
scan:
  stage: test
  image: aquasec/trivy:latest
  script:
    - trivy image --exit-code 1 --severity HIGH,CRITICAL myapp:$CI_COMMIT_SHA
  allow_failure: false

This fails the build if high/critical vulnerabilities are found.

Step 5: Keep Base Images Updated

Old base images have known vulnerabilities.

Bad:

FROM python:3.6  # Old version

Good:

FROM python:3.7.12-alpine3.14  # Specific, recent version

Update regularly:

# Check for updates
docker pull python:3.7-alpine

# Rebuild
docker build -t myapp:latest .

Step 6: Remove Secrets from Images

Never put secrets in Dockerfiles:

# NEVER DO THIS
ENV DATABASE_PASSWORD=supersecret
ENV API_KEY=abc123

These are visible in image history:

$ docker history myapp:latest
IMAGE          CREATED BY
abc123         ENV DATABASE_PASSWORD=supersecret

Use environment variables at runtime:

docker run -e DATABASE_PASSWORD=$DB_PASS myapp:latest

Or use Docker secrets (Swarm) or Kubernetes secrets.

Step 7: Use .dockerignore

Prevent sensitive files from being copied:

# .dockerignore
.git
.env
*.log
secrets/
.aws/
.ssh/
node_modules/
__pycache__/

Step 8: Sign and Verify Images

Use Docker Content Trust:

# Enable content trust
export DOCKER_CONTENT_TRUST=1

# Push signed image
docker push myapp:latest

# Only signed images can be pulled
docker pull myapp:latest

Step 9: Read-Only Filesystem

Make filesystem read-only:

FROM alpine

# App doesn't need to write to filesystem
USER nobody

CMD ["python", "app.py"]

Run with read-only flag:

docker run --read-only myapp:latest

If app needs to write, use volumes:

docker run --read-only -v /tmp myapp:latest

Step 10: Limit Resources

Prevent resource exhaustion:

docker run \
  --memory=512m \
  --cpus=1 \
  --pids-limit=100 \
  myapp:latest

In Kubernetes:

resources:
  limits:
    memory: "512Mi"
    cpu: "1000m"
  requests:
    memory: "256Mi"
    cpu: "500m"

Security Scanning Tools

I use multiple scanners:

1. Trivy (my favorite):

trivy image myapp:latest

2. Clair:

clairctl analyze myapp:latest

3. Docker Scan (uses Snyk):

docker scan myapp:latest

4. Anchore:

anchore-cli image add myapp:latest
anchore-cli image vuln myapp:latest all

Hardened Dockerfile Template

Here’s my production template:

# Build stage
FROM python:3.7 AS builder

WORKDIR /app

# Install dependencies
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

# Runtime stage
FROM python:3.7-alpine

# Install only runtime dependencies
RUN apk add --no-cache ca-certificates

# Create non-root user
RUN addgroup -g 1000 appuser && \
    adduser -D -u 1000 -G appuser appuser

WORKDIR /app

# Copy dependencies from builder
COPY --from=builder --chown=appuser:appuser /root/.local /root/.local

# Copy application
COPY --chown=appuser:appuser app.py .

# Set PATH
ENV PATH=/root/.local/bin:$PATH

# Switch to non-root user
USER appuser

# Health check
HEALTHCHECK --interval=30s --timeout=3s \
  CMD python -c "import requests; requests.get('http://localhost:5000/health')" || exit 1

# Expose port
EXPOSE 5000

# Run application
CMD ["python", "app.py"]

Results

After hardening all images:

ImageBeforeAfterReduction
API Gateway247 CVEs12 CVEs95%
User Service198 CVEs8 CVEs96%
Frontend312 CVEs15 CVEs95%

Image sizes also decreased:

ImageBeforeAfterReduction
API Gateway1.2GB85MB93%
User Service800MB65MB92%
Frontend950MB120MB87%

Compliance

Our images now pass:

  • CIS Docker Benchmark
  • NIST Container Security Guidelines
  • PCI DSS requirements

Automated Scanning

I set up automated daily scans:

#!/bin/bash
# scan-images.sh

IMAGES=$(docker images --format "{{.Repository}}:{{.Tag}}" | grep myapp)

for IMAGE in $IMAGES; do
  echo "Scanning $IMAGE..."
  trivy image --severity HIGH,CRITICAL $IMAGE
done

Run via cron:

0 2 * * * /usr/local/bin/scan-images.sh

Monitoring

Track security metrics:

  • Number of vulnerabilities per image
  • Time to patch critical CVEs
  • Images running as root
  • Images without health checks

Best Practices Summary

  1. Use minimal base images (Alpine, distroless)
  2. Multi-stage builds (separate build/runtime)
  3. Don’t run as root (create non-root user)
  4. Scan in CI/CD (fail builds on vulnerabilities)
  5. Keep images updated (rebuild regularly)
  6. No secrets in images (use env vars or secrets management)
  7. Use .dockerignore (exclude sensitive files)
  8. Sign images (Docker Content Trust)
  9. Read-only filesystem (when possible)
  10. Limit resources (prevent DoS)

Lessons Learned

  1. Security is ongoing - New CVEs appear daily
  2. Automate scanning - Manual scans don’t scale
  3. Fail fast - Block vulnerable images in CI
  4. Educate team - Everyone needs to understand security
  5. Balance security and usability - Don’t make it too hard

Conclusion

Docker security isn’t optional. With containers running in production, vulnerabilities can be exploited quickly.

Key takeaways:

  1. Start with minimal base images
  2. Scan images in CI/CD pipeline
  3. Never run as root
  4. Keep images updated
  5. Automate security checks

Our security posture is now much stronger. We catch vulnerabilities before they reach production.

If you’re not scanning your images, start today. Your security team will thank you.