Docker Compose V2: Multi-Container Development

Introduction

Docker Compose V2 represents a fundamental rearchitecture of the tool that changed how developers build multi-container applications. Rewritten in Go and integrated directly into the Docker CLI as a plugin rather than a separate Python binary, Compose V2 delivers significant performance improvements, better compatibility with the broader Docker ecosystem, and powerful new features like profiles and watch mode that were impossible in the V1 architecture.

The migration from V1 to V2 isn't just a version bump—it's a paradigm shift. The command changed from docker-compose (hyphen) to docker compose (space), the configuration schema expanded with new capabilities, and the runtime engine was completely replaced. For teams still running V1, understanding these changes is critical for maintaining efficient development workflows.

Understanding Docker Compose V2: Core Concepts

What Changed from V1 to V2

The most visible change is the command syntax. Docker Compose V1 was a standalone binary invoked as docker-compose. V2 is a Docker CLI plugin invoked as docker compose (space-separated). This seemingly minor change integrates Compose into Docker's plugin architecture, enabling shared configuration, context management, and consistent behavior across Docker tools.

Under the hood, the rewrite from Python to Go eliminated several long-standing issues:

Startup overhead: V1's Python interpreter added 1-2 seconds to every command. V2 starts in milliseconds.
Memory usage: V2 uses significantly less memory, important when running large compose stacks.
Compose Spec compliance: V2 fully implements the Compose Specification, the open standard that defines the file format.
BuildKit integration: V2 uses BuildKit by default for image builds, enabling parallel build stages, build cache mounts, and secret mounting.

The Compose Specification

The Compose Specification is the open standard that defines every field available in a Compose file. V2 fully implements this spec, which means your Compose files are portable across any compliant tool—Docker, Podman, OrbStack, or cloud platforms.

# compose.yml - Modern Compose file (no version field needed)
name: myapp
 
services:
  web:
    image: nginx:alpine
    ports:
      - "80:80"
    develop:
      watch:
        - action: sync
          path: ./html
          target: /usr/share/nginx/html
 
  api:
    build:
      context: ./api
      dockerfile: Dockerfile
      target: development
      args:
        NODE_ENV: development
    environment:
      DATABASE_URL: postgres://user:pass@db:5432/myapp
    develop:
      watch:
        - action: sync
          path: ./api/src
          target: /app/src
        - action: rebuild
          path: ./api/package.json
 
  db:
    image: postgres:16-alpine
    profiles:
      - backend
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user -d myapp"]
      interval: 5s
      timeout: 5s
      retries: 5
 
volumes:
  pgdata:

BuildKit as Default

Compose V2 uses BuildKit by default for all docker compose build operations. BuildKit provides several advantages over the legacy builder:

services:
  api:
    build:
      context: ./api
      dockerfile: Dockerfile
      # BuildKit-specific features
      cache_from:
        - type=gha
      cache_to:
        - type=gha,mode=max
      secrets:
        - npmrc
      ssh:
        - default
 
secrets:
  npmrc:
    file: ~/.npmrc

BuildKit enables parallel execution of independent build stages, reducing build times for complex Dockerfiles by 50-70%. The cache mount feature eliminates redundant dependency installations:

# Dockerfile with BuildKit cache mounts
FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci
COPY . .

Architecture and Design Patterns

Profile-Based Service Organization

Profiles let you group services that should only run under specific conditions. This solves the common problem of optional services (debug tools, monitoring agents, documentation servers) cluttering your default docker compose up:

services:
  # Always running
  api:
    build: ./api
    ports:
      - "4000:4000"
    depends_on:
      db:
        condition: service_healthy
 
  # Always running
  db:
    image: postgres:16-alpine
    profiles:
      - backend
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user"]
      interval: 5s
      timeout: 5s
      retries: 5
 
  # Only with 'debug' profile
  adminer:
    image: adminer:latest
    profiles:
      - debug
    ports:
      - "8080:8080"
 
  # Only with 'monitoring' profile
  prometheus:
    image: prom/prometheus:latest
    profiles:
      - monitoring
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
 
  grafana:
    image: grafana/grafana:latest
    profiles:
      - monitoring
    ports:
      - "3000:3000"
    environment:
      GF_SECURITY_ADMIN_PASSWORD: admin

# Default: only api and db start
docker compose up -d
 
# With debug tools
docker compose --profile debug up -d
 
# With monitoring stack
docker compose --profile monitoring up -d
 
# Multiple profiles
docker compose --profile debug --profile monitoring up -d

The Compose Watch Pattern

Compose Watch (introduced in v2.22) replaces the need for external file-watching tools like nodemon or chokidar for container synchronization. It operates at the Compose level, watching your local filesystem and automatically syncing changes to containers or triggering rebuilds:

services:
  frontend:
    build:
      context: ./frontend
      target: development
    develop:
      watch:
        # Sync source files immediately
        - action: sync
          path: ./frontend/src
          target: /app/src
          ignore:
            - '**/*.test.ts'
            - '**/*.spec.ts'
        # Sync public assets
        - action: sync
          path: ./frontend/public
          target: /app/public
        # Rebuild image when dependencies change
        - action: rebuild
          path: ./frontend/package.json
        # Rebuild when Dockerfile changes
        - action: rebuild
          path: ./frontend/Dockerfile
 
  api:
    build:
      context: ./api
      target: development
    develop:
      watch:
        - action: sync
          path: ./api/src
          target: /app/src
        - action: rebuild
          path: ./api/package.json
        - action: rebuild
          path: ./api/tsconfig.json

The sync action performs incremental file copies without restarting the container, while rebuild triggers a full image rebuild when configuration or dependency files change. The ignore array excludes files from triggering syncs, reducing unnecessary container updates.

Multi-Environment Configuration

Compose V2 supports multiple configuration file layers, letting you maintain clean separation between environments:

# compose.yml (shared base)
services:
  api:
    build: ./api
    environment:
      NODE_ENV: ${NODE_ENV:-development}
 
# compose.override.yml (auto-loaded in development)
services:
  api:
    volumes:
      - ./api/src:/app/src
      - /app/node_modules
    command: npm run dev
 
# compose.prod.yml (explicitly loaded in production)
services:
  api:
    restart: unless-stopped
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 1G

# Development (loads compose.yml + compose.override.yml automatically)
docker compose up -d
 
# Production (loads compose.yml + compose.prod.yml)
docker compose -f compose.yml -f compose.prod.yml up -d
 
# Testing (loads compose.yml + compose.test.yml)
docker compose -f compose.yml -f compose.test.yml up --abort-on-container-exit

Step-by-Step Implementation

Step 1: Install Docker Compose V2

# Verify Compose V2 is installed
docker compose version
# Docker Compose version v2.24.0
 
# If using Docker Desktop, V2 is included since v4.15
# For standalone installation on Linux:
mkdir -p ~/.docker/cli-plugins/
curl -SL https://github.com/docker/compose/releases/latest/download/docker-compose-linux-x86_64 \
  -o ~/.docker/cli-plugins/docker-compose
chmod +x ~/.docker/cli-plugins/docker-compose

Step 2: Migrate from V1

# Remove V1
sudo rm /usr/local/bin/docker-compose
 
# Translate hyphen to space in all scripts and documentation
# docker-compose up → docker compose up
# docker-compose down → docker compose down
# docker-compose build → docker compose build
 
# Verify the migration
docker compose version
docker compose config  # Validate your compose file

Step 3: Build a Full-Stack Application

# compose.yml
name: fullstack-app
 
services:
  proxy:
    image: traefik:v3.0
    command:
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
    ports:
      - "80:80"
      - "8080:8080"  # Traefik dashboard
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    labels:
      - "traefik.enable=true"
 
  frontend:
    build:
      context: ./frontend
      target: development
    develop:
      watch:
        - action: sync
          path: ./frontend/src
          target: /app/src
        - action: rebuild
          path: ./frontend/package.json
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.frontend.rule=PathPrefix(`/`)"
      - "traefik.http.services.frontend.loadbalancer.server.port=3000"
 
  api:
    build:
      context: ./api
      target: development
    environment:
      DATABASE_URL: postgres://app:secret@db:5432/appdb
      REDIS_URL: redis://cache:6379
    develop:
      watch:
        - action: sync
          path: ./api/src
          target: /app/src
        - action: rebuild
          path: ./api/package.json
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.api.rule=PathPrefix(`/api`)"
      - "traefik.http.services.api.loadbalancer.server.port=4000"
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_started
 
  db:
    image: postgres:16-alpine
    profiles:
      - backend
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: appdb
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
      interval: 5s
      timeout: 5s
      retries: 5
 
  cache:
    image: redis:7-alpine
    profiles:
      - backend
 
volumes:
  pgdata:

Step 4: Use Watch Mode

# Start services and enable watch mode
docker compose up --watch
 
# This opens an interactive session where file changes are
# detected and automatically synced to containers
# Press Ctrl+C to stop watching (containers keep running)

Step 5: Environment Management

# .env file for default values
POSTGRES_USER=app
POSTGRES_PASSWORD=secretpassword
POSTGRES_DB=myapp
API_PORT=4000
LOG_LEVEL=info
 
# Override for specific environments
# .env.staging, .env.production, etc.
# Use --env-file to specify
docker compose --env-file .env.staging up -d

Real-World Use Cases

Use Case 1: Full-Stack Development with Hot Reload

A team building a SaaS product uses Compose Watch to create a seamless development experience. Frontend developers see React component changes reflected in under 500ms without losing application state. Backend developers get automatic TypeScript compilation and server restart when API code changes. The rebuild action handles dependency updates, so npm install runs automatically when package.json changes.

Use Case 2: Multi-Tenant Development Environments

A company serving enterprise clients uses Docker Compose profiles to create isolated tenant environments for testing. Each client has a dedicated profile that spins up a database instance with tenant-specific seed data, a configured API instance, and a mock payment gateway. Developers switch between tenant environments with docker compose --profile client-acme up -d.

Use Case 3: CI/CD Pipeline with BuildKit Caching

A DevOps team leverages BuildKit's GitHub Actions cache integration to speed up CI builds. Compose V2's native BuildKit support means Dockerfile cache mounts, multi-stage parallel builds, and GHA cache export/import work seamlessly. Their average build time dropped from 8 minutes to 2.5 minutes after migrating to Compose V2 with BuildKit caching.

Use Case 4: Developer Onboarding Automation

A startup documented their entire development setup as a single compose.yml with profiles. New developers install Docker Desktop, clone the repository, and run docker compose --profile full up -watch. The stack provisions the database, runs migrations, seeds test data, and starts all services with hot-reload enabled. Onboarding time dropped from a full day to 15 minutes.

Best Practices for Production

Remove the version field: Compose V2 ignores the version field and emits a deprecation warning. Remove it from all Compose files—it's no longer needed as the Compose Specification is self-versioning.
Use docker compose config to validate: Run docker compose config before deployment to catch syntax errors, missing variables, and invalid configurations. Add this to your CI pipeline as a linting step.
Leverage BuildKit cache mounts: Use RUN --mount=type=cache,target=/path in Dockerfiles for package manager caches. This eliminates redundant downloads during builds without polluting the final image.
Use profiles for optional services: Don't include debug tools, monitoring agents, or documentation servers in your default docker compose up. Assign them to profiles and activate explicitly.
Set COMPOSE_PROJECT_NAME for isolation: Prevent container and network name collisions when running multiple Compose projects by setting a unique project name.
Use docker compose watch for development: Replace external file-watching tools with Compose Watch. It's integrated, faster, and handles both sync and rebuild actions.
Pin Compose file version to a specific Docker Desktop version: If your team uses Docker Desktop, document the minimum required version that supports all Compose features you use.
Use docker compose cp for file transfers: Instead of bind mounts for one-off file transfers, use docker compose cp service:container_path host_path to copy files between containers and the host.

Common Pitfalls and Solutions

Pitfall	Impact	Solution
Using `docker-compose` (hyphen)	Command not found or V1 behavior	Use `docker compose` (space) everywhere
Keeping `version` field	Deprecation warning, confusion	Remove the `version` field from all files
Not using profiles for optional services	Cluttered default startup, resource waste	Add `profiles` to debug tools, monitoring, etc.
Ignoring BuildKit cache mounts	Slow builds, redundant downloads	Use `RUN --mount=type=cache` in Dockerfiles
Not validating with `config`	Runtime errors in CI/CD	Run `docker compose config` in CI pipeline
Mixing V1 and V2 in team	Inconsistent behavior, CI failures	Document minimum version, update CI scripts
Not using `develop.watch`	Manual file sync or external tools	Adopt Compose Watch for development
Forgetting `depends_on` conditions	Race conditions on startup	Use `condition: service_healthy` for databases

Performance Optimization

Compose V2's Go runtime delivers measurable performance improvements over V1's Python runtime:

# Benchmark: starting a 10-service stack
# V1 (Python): ~4.2 seconds
# V2 (Go): ~0.8 seconds
 
# Benchmark: parsing a large compose file
# V1: ~1.1 seconds
# V2: ~0.05 seconds

BuildKit integration provides additional performance wins:

# Parallel build stages with BuildKit
FROM node:18-alpine AS deps
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci --omit=dev
 
FROM node:18-alpine AS dev-deps
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci
 
FROM deps AS production
COPY --from=dev-deps /app/node_modules /app/node_modules
COPY . .
RUN npm run build

These stages execute in parallel where possible, reducing total build time. The cache mount ensures npm packages are downloaded once and reused across builds.

# Resource limits to prevent runaway containers
services:
  api:
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 1G
        reservations:
          cpus: '0.5'
          memory: 256M

Comparison with Alternatives

Feature	Docker Compose V2	Docker Compose V1	Podman Compose	OrbStack	Tilt
Runtime	Go	Python	Python/Go	Native	Go
Startup overhead	~50ms	~2s	~1s	~30ms	~100ms
BuildKit support	Default	No	Varies	Yes	Yes
Profiles	Yes	No	Partial	Yes	N/A
Watch mode	Built-in	No	No	Yes	Built-in
Compose Spec	Full	Partial	Partial	Full	Partial
Docker Desktop plugin	Native	Legacy	No	Alternative	Separate
Containerd support	Yes	No	Yes	Yes	Yes

Advanced Patterns

Conditional Service Dependencies with Profiles

services:
  app:
    build: ./app
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_started
        required: false  # Don't fail if cache profile isn't active
 
  db:
    image: postgres:16-alpine
    healthcheck:
      test: ["CMD-SHELL", "pg_isready"]
      interval: 5s
      timeout: 5s
      retries: 5
 
  cache:
    image: redis:7-alpine
    profiles:
      - with-cache

Compose with Docker Contexts for Remote Development

# Create a context for a remote development server
docker context create remote-dev \
  --docker "host=ssh://developer@dev-server.example.com"
 
# Use the context with Compose
docker --context remote-dev compose up -d
 
# All Compose operations now target the remote server
docker --context remote-dev compose logs -f

Merging Multiple Compose Files

# Combine base, team-specific, and local overrides
docker compose \
  -f compose.yml \
  -f compose.${TEAM:-backend}.yml \
  -f compose.local.yml \
  up -d

Testing Strategies

# compose.test.yml
services:
  test-runner:
    build:
      context: ./api
      target: development
    command: npm test -- --watchAll=false
    environment:
      NODE_ENV: test
      DATABASE_URL: postgres://test:test@test-db:5432/test
      REDIS_URL: redis://test-cache:6379
    depends_on:
      test-db:
        condition: service_healthy
    profiles:
      - test
 
  test-db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: test
      POSTGRES_PASSWORD: test
      POSTGRES_DB: test
    tmpfs:
      - /var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U test"]
      interval: 2s
      timeout: 2s
      retries: 10
    profiles:
      - test
 
  test-cache:
    image: redis:7-alpine
    tmpfs:
      - /data
    profiles:
      - test
 
  e2e:
    image: mcr.microsoft.com/playwright:v1.40.0-jammy
    working_DIR: /app
    volumes:
      - ./e2e:/app
      - ./playwright-report:/app/playwright-report
    command: npx playwright test
    profiles:
      - e2e

# Run unit/integration tests
docker compose -f compose.test.yml --profile test up --build --abort-on-container-exit
 
# Run E2E tests
docker compose -f compose.test.yml --profile e2e up --build --abort-on-container-exit
 
# Clean up test volumes
docker compose -f compose.test.yml down -v

Future Outlook

Docker Compose V2 is actively developed with monthly releases introducing new Compose Specification features. The roadmap includes improved multi-platform build support, tighter integration with Docker Desktop's Kubernetes environment, and enhanced secret management. The Compose Specification's adoption by competing tools (Podman, OrbStack, cloud platforms) ensures your Compose files remain portable as the ecosystem evolves.

The develop.watch feature is particularly transformative, positioning Compose as a complete development environment tool rather than just a container orchestrator. Future enhancements will likely include more sophisticated sync strategies, IDE integration, and build artifact caching across team members.

Conclusion

Docker Compose V2 is a complete reimagining of the tool that defined container-based development. The Go rewrite delivers performance improvements, BuildKit integration provides faster and smarter builds, and new features like profiles and watch mode transform the developer experience.

Key takeaways:

Migrate from docker-compose to docker compose and remove the version field
Use profiles to organize optional services and keep default startup lean
Adopt Compose Watch (docker compose up --watch) for automatic file synchronization
Leverage BuildKit's cache mounts and parallel builds for faster image construction
Layer multiple Compose files for environment-specific configuration
Validate configurations with docker compose config in CI pipelines
Use condition: service_healthy for reliable startup ordering
Set resource limits to prevent development containers from consuming excessive host resources

The combination of profiles, watch mode, and BuildKit makes Compose V2 the most productive container development tool available. Start by upgrading your existing Compose files, then progressively adopt the new features to streamline your development workflow.

Minh Vo

Slaying code & making it lit fr fr 🔥 tagline