Docker BuildKit: Faster Builds with Better Caching

Our Docker builds were slow - 8 minutes for a full build, 3 minutes even with cache. CI/CD pipeline was bottlenecked by build time.

Docker 18.09 introduced BuildKit. I migrated our builds and got 3x speedup. Builds now take 2.5 minutes, sometimes under 1 minute with cache.

Table of Contents

The Problem

Original build time:

$ time docker build -t myapp:latest .

real    8m15s
user    0m2s
sys     0m1s

Even with cache:

$ time docker build -t myapp:latest .

real    3m20s  # Still slow!

Why? Sequential layer building, poor cache utilization, no parallelization.

What is BuildKit?

BuildKit is Docker’s new build engine with:

1. Parallel builds - Build independent stages concurrently
2. Better caching - More intelligent cache invalidation
3. Build secrets - Secure secret handling
4. SSH forwarding - Access private repos during build
5. Faster - Overall performance improvements

Enabling BuildKit

Temporary:

DOCKER_BUILDKIT=1 docker build -t myapp:latest .

Permanent:

// /etc/docker/daemon.json
{
  "features": {
    "buildkit": true
  }
}

Restart Docker:

sudo systemctl restart docker

Parallel Multi-Stage Builds

Before BuildKit, stages build sequentially:

FROM node:12 AS frontend
WORKDIR /app
COPY frontend/ .
RUN npm install && npm run build

FROM golang:1.13 AS backend
WORKDIR /app
COPY backend/ .
RUN go build -o server

FROM alpine:3.10
COPY --from=frontend /app/dist /var/www
COPY --from=backend /app/server /usr/local/bin/
CMD ["server"]

Without BuildKit: frontend builds, then backend builds (sequential) With BuildKit: frontend and backend build in parallel!

Result: 8 minutes → 4.5 minutes

Better Cache Mounting

Old way (cache invalidated on file change):

FROM node:12

WORKDIR /app

# Cache invalidated if ANY file changes
COPY . .
RUN npm install

BuildKit way (cache only package.json):

# syntax=docker/dockerfile:1

FROM node:12

WORKDIR /app

# Cache dependencies separately
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm install

# Copy source (doesn't invalidate npm cache)
COPY . .

Now npm install only re-runs if package.json changes.

Cache Mounts for Package Managers

Node.js:

RUN --mount=type=cache,target=/root/.npm \
    npm install

Python:

RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

Go:

RUN --mount=type=cache,target=/go/pkg/mod \
    go mod download

apt-get:

RUN --mount=type=cache,target=/var/cache/apt \
    --mount=type=cache,target=/var/lib/apt \
    apt-get update && apt-get install -y curl

Build Secrets

Never put secrets in Dockerfile:

# BAD - secret in image history
ARG GITHUB_TOKEN
RUN git clone https://${GITHUB_TOKEN}@github.com/private/repo.git

BuildKit secrets (not stored in image):

# syntax=docker/dockerfile:1

FROM alpine

RUN --mount=type=secret,id=github_token \
    git clone https://$(cat /run/secrets/github_token)@github.com/private/repo.git

Build with secret:

docker build --secret id=github_token,src=$HOME/.github-token -t myapp .

Secret is never stored in image layers!

SSH Forwarding

Access private repos without copying SSH keys:

# syntax=docker/dockerfile:1

FROM alpine

RUN apk add --no-cache git openssh-client

RUN --mount=type=ssh \
    git clone git@github.com:private/repo.git

Build with SSH:

docker build --ssh default -t myapp .

Uses your local SSH agent, no keys in image.

Bind Mounts

Mount files during build without copying:

# syntax=docker/dockerfile:1

FROM golang:1.13

WORKDIR /app

# Mount source code (not copied into image)
RUN --mount=type=bind,source=.,target=/app \
    go build -o /out/server

Useful for large codebases.

Output to Local Filesystem

Export build artifacts:

# syntax=docker/dockerfile:1

FROM golang:1.13 AS builder

WORKDIR /app
COPY . .
RUN go build -o server

FROM scratch AS export
COPY --from=builder /app/server /

Build and export:

docker build --output type=local,dest=./bin -t myapp .

Binary is now in ./bin/server on host.

Real-World Example

Before BuildKit:

FROM node:12

WORKDIR /app

COPY package*.json ./
RUN npm install

COPY . .
RUN npm run build

FROM nginx:alpine
COPY --from=0 /app/dist /usr/share/nginx/html

Build time: 5m30s

After BuildKit:

# syntax=docker/dockerfile:1

FROM node:12 AS builder

WORKDIR /app

# Cache npm packages
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci

# Copy source
COPY . .

# Cache build output
RUN --mount=type=cache,target=/app/.cache \
    npm run build

FROM nginx:alpine
COPY --from=builder /app/dist /usr/share/nginx/html

Build time: 1m45s (3x faster!)

Python Example

Before:

FROM python:3.8

WORKDIR /app

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .

CMD ["python", "app.py"]

After:

# syntax=docker/dockerfile:1

FROM python:3.8

WORKDIR /app

# Cache pip packages
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

COPY . .

CMD ["python", "app.py"]

Go Example

Before:

FROM golang:1.13

WORKDIR /app

COPY go.mod go.sum ./
RUN go mod download

COPY . .
RUN go build -o server

CMD ["./server"]

After:

# syntax=docker/dockerfile:1

FROM golang:1.13

WORKDIR /app

# Cache Go modules
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
    go mod download

COPY . .

# Cache build cache
RUN --mount=type=cache,target=/root/.cache/go-build \
    go build -o server

CMD ["./server"]

CI/CD Integration

GitLab CI:

# .gitlab-ci.yml
build:
  image: docker:19.03
  services:
    - docker:19.03-dind
  variables:
    DOCKER_BUILDKIT: 1
  script:
    - docker build -t myapp:$CI_COMMIT_SHA .
    - docker push myapp:$CI_COMMIT_SHA

GitHub Actions:

# .github/workflows/build.yml
- name: Build with BuildKit
  env:
    DOCKER_BUILDKIT: 1
  run: docker build -t myapp:${{ github.sha }} .

Cache Backends

Export cache for CI:

# Build and export cache
docker build \
  --cache-to type=local,dest=/tmp/cache \
  -t myapp .

# Build using cache
docker build \
  --cache-from type=local,src=/tmp/cache \
  -t myapp .

Registry cache:

# Push cache to registry
docker build \
  --cache-to type=registry,ref=myregistry/myapp:cache \
  -t myapp .

# Pull cache from registry
docker build \
  --cache-from type=registry,ref=myregistry/myapp:cache \
  -t myapp .

Results

After migrating all Dockerfiles:

Service	Before	After	Improvement
Frontend	5m30s	1m45s	68%
API Gateway	8m15s	2m30s	70%
User Service	4m20s	1m20s	69%
Email Service	3m50s	1m10s	70%

Average improvement: 69%

With cache:

Service	Before	After	Improvement
Frontend	2m10s	25s	81%
API Gateway	3m20s	45s	77%
User Service	1m50s	20s	82%
Email Service	1m30s	18s	80%

Average improvement: 80%

CI/CD Pipeline Impact

Before:

• Build stage: 8-10 minutes
• Total pipeline: 15-20 minutes

After:

• Build stage: 2-3 minutes
• Total pipeline: 8-10 minutes

50% faster deployments!

Best Practices

1. Use cache mounts - For package managers
2. Order layers wisely - Least to most frequently changed
3. Parallel stages - Independent stages build concurrently
4. Use secrets - Never put secrets in Dockerfile
5. Export cache - Share cache between CI runs

Common Pitfalls

1. Forgetting syntax directive:

# Required for BuildKit features
# syntax=docker/dockerfile:1

FROM alpine

2. Cache mount permissions:

# May need to match user
RUN --mount=type=cache,target=/root/.cache,uid=1000 \
    pip install -r requirements.txt

3. Not using .dockerignore:

# .dockerignore
.git
node_modules
__pycache__
*.log

Monitoring Build Performance

Track build times:

#!/bin/bash
# build-and-time.sh

START=$(date +%s)
docker build -t myapp:latest .
END=$(date +%s)

DURATION=$((END - START))
echo "Build took ${DURATION}s"

# Send to metrics
curl -X POST metrics-server/build-time -d "duration=${DURATION}"

Future: BuildKit Features

Upcoming features I’m excited about:

• Better cache sharing - Across builds
• Distributed builds - Build on multiple machines
• More output formats - OCI, tar, etc.

Conclusion

BuildKit transformed our build process. 3x faster builds, better caching, more secure.

Key takeaways:

1. Enable BuildKit for all builds
2. Use cache mounts for package managers
3. Leverage parallel multi-stage builds
4. Use secrets for sensitive data
5. Export cache in CI/CD

If you’re not using BuildKit, enable it today. Your CI/CD pipeline will thank you.

Build time matters. Faster builds mean faster iterations and happier developers.