Compose Profiles, Override Files and Multiple Environments

One compose.yaml rarely fits development, CI and production at once: development wants bind mounts and published ports, CI wants a test runner, production wants restart policies and limits. Compose has three mechanisms for this: variable interpolation, file merging, and profiles. After this lesson you will layer them so a single committed base file serves every environment without copy-pasting.

Variable Interpolation and .env

Any value in the file can reference environment variables with ${VAR}. Compose reads variables from the shell first, then from a .env file in the project directory. Defaults and required-value errors are supported:

yaml

services:
  api:
    image: ghcr.io/team/shop-api:${TAG:-latest}   # default if unset or empty
    environment:
      DATABASE_URL: ${DATABASE_URL:?DATABASE_URL must be set}   # hard error if missing
      LOG_LEVEL: ${LOG_LEVEL-info}                 # default only if unset

bash

TAG=1.4.2 docker compose up -d
docker compose --env-file .env.staging config      # use a different file for interpolation

Interpolation happens in the Compose file itself. It is different from env_file: on a service, which passes variables into the container without affecting the file. docker compose config prints the result and is the first thing to run when a value looks wrong.

Override Files and Merging

When you run docker compose up, Compose automatically loads compose.yaml and compose.override.yaml if it exists, merging the second onto the first. Commit the base file and keep development-only settings in the override. For other environments, list files explicitly with -f; later files win:

bash

docker compose -f compose.yaml -f compose.prod.yaml up -d
COMPOSE_FILE=compose.yaml:compose.prod.yaml docker compose up -d   # same via env var

The merge rules per key:

| Key type | Behaviour | |----------|-----------| | Scalars (image, command, restart) | Later value replaces earlier | | Maps (environment, labels) | Merged key by key | | Lists (ports, volumes, expose) | Concatenated (duplicates for the same target are deduplicated) | | depends_on, networks | Merged |

Because lists concatenate, you cannot remove a port or volume in an override; you can only add. The !reset and !override YAML tags handle that case: ports: !reset [] clears the list, and ports: !override ["8080:80"] replaces it entirely.

Profiles

Profiles mark services that should start only on request: debugging tools, seed scripts, or optional components such as a mail catcher. A service without profiles always starts; a service with one starts only when its profile is active.

yaml

services:
  api:
    build: ./api
  mailpit:
    image: axllent/mailpit
    ports: ["8025:8025"]
    profiles: ["debug"]
  seed:
    build: ./api
    command: node scripts/seed.js
    depends_on: [db]
    profiles: ["seed"]

bash

docker compose up -d                       # api and db only
docker compose --profile debug up -d       # adds mailpit
COMPOSE_PROFILES=debug,seed docker compose up -d
docker compose run --rm seed               # running a profiled service by name activates it

docker compose down stops all profiles, so nothing is left behind. Profiles are usually cleaner than override files for optional services, while override files are better for changing how core services run.

include and extends

For large projects, include: composes several files as if they were one, each with its own project directory and .env, which suits monorepos:

yaml

include:
  - path: ./services/payments/compose.yaml
  - path: ./infra/compose.monitoring.yaml
services:
  api:
    build: ./api
    depends_on: [payments]     # service defined in the included file

extends: reuses a single service definition from another file (extends: { file: common.yaml, service: base-api }) and then adds or overrides keys. Prefer include for whole stacks and extends for repeated service templates.

A Typical Layout

compose.yaml: images, networks, volumes, healthchecks, depends_on. Works everywhere.
compose.override.yaml: build:, bind mounts, published ports, debug variables. Developers get it automatically.
compose.prod.yaml: restart:, resource limits, logging options, secrets. Used explicitly by deployment scripts.
compose.test.yaml: a tests service with depends_on conditions, run in CI with docker compose -f compose.yaml -f compose.test.yaml run --rm tests.
.env: local values, ignored by Git; .env.example: committed template.

Key Takeaways

${VAR:-default} and ${VAR:?error} interpolate values from the shell and .env; docker compose config shows the result.
compose.override.yaml merges automatically for development; other environments use -f file lists where later files win.
Maps merge and lists concatenate; use !override or !reset to replace or clear a list.
Profiles gate optional services; --profile or COMPOSE_PROFILES activates them.
include: assembles multi-file stacks and extends: reuses service templates.

Next lesson: Running Databases in Containers: PostgreSQL, MySQL and MongoDB — configure, persist, initialise and back up databases in Docker.