Compose & env conventions

Two conventions do most of the work in docker/: one compose file per service, plus a dev override, and three env layers under docker/.config/. Together they are why make dev s=api can start a subset without a per-project runbook.

Compose layering

A baseline compose file describes the service everywhere; a dev override adds the source mount, the HMR port and a relaxed healthcheck. make dev merges the two, make up uses the baseline alone.

Every service gets two files. docker/compose/<service>.yml is the baseline, describing the service as it ships. docker/compose/<service>.dev.yml is the override, describing what it needs while you are working on it.

make dev                # baseline + .dev.yml override, merged
make dev s=api          # just this service's files
make dev s="api site"   # a named subset: both services and their deps
make up                 # baseline alone — the same file that ships

The quotes in s="api site" are required. Make splits on whitespace, so an unquoted second name is read as a second goal rather than as part of the value. dev, up, down, stop, restart and logs all take a list, because the compose verb underneath accepts several service operands. build, run, exec and sh/bash act on exactly one image or container, and they refuse a list out loud rather than silently using the first name.

make dev brings the existing services down first, so it doubles as “restart into a known state”.

A typical dev override adds four things and nothing else:

services:
  api:
    build:
      target: development        # the dev stage of the same dockerfile
    environment:
      - RUST_LOG=debug
    volumes:
      - ../../apps/api:/app      # source mount for hot reload
    healthcheck:
      disable: true              # or a relaxed one with room to boot
  db:
    ports:
      - "${DB_HOST_PORT:-0}:5432"  # host 0 = Docker picks a free port

Note what is not there: the application service publishes no HTTP host port in either file. That is the router’s job. Backing services are the exception, because psql and Redis GUIs want a host port.

Host ports

Every host-side publish in a shipped compose file is ${VAR:-default}, in one of two classes, and a conformance test refuses a bare number in either.

Ports tooling dials on demand default to 0, which tells Docker to allocate a free one. A pinned 5432 is a claim on the whole workstation, and a second mx stack booting into it dies on Bind for 0.0.0.0:5432 failed: port is already allocated. Host port 0 cannot collide. Discover the number when you need it:

docker compose -p "$COMPOSE_PROJECT_NAME" port db 5432
# 0.0.0.0:32789

Or pin one for a session while its siblings stay ephemeral:

DB_HOST_PORT=5432 make dev

The names: DB_HOST_PORT, REDIS_HOST_PORT, NODE_DEBUG_HOST_PORT, METRICS_HOST_PORT, VITE_HOST_PORT, INERTIA_SSR_HOST_PORT.

Ports a browser dials from a URL the page already holds cannot move silently, so they keep today’s number as the default and gain an override: LEPTOS_RELOAD_HOST_PORT (3001), ZOLA_LIVERELOAD_PORT (1024), NGINX_HTTP_PORT / NGINX_HTTPS_PORT, TRAEFIK_HTTP_PORT / TRAEFIK_HTTPS_PORT / TRAEFIK_DASHBOARD_PORT, MX_ROUTER_HTTP_PORT / MX_ROUTER_HTTPS_PORT. A lone stack is unchanged; a second stack overrides rather than failing to start.

The shape alone is not the fix. ${DB_HOST_PORT:-5432} is parameterized and still collides out of the box, which is why the test asserts the default per class as well as the syntax, and fails deliberately on a container port it has never been told how to classify. Publishing a host port is a design call.

A script or GUI connection profile pointed at localhost:5432, localhost:6379, localhost:9229, localhost:9090, localhost:5173 or localhost:13714 of an mx dev stack no longer finds anything there. Discover the port with docker compose -p <project> port <service> <container-port>, or pin it with the variable above.

Port 24678 is gone entirely rather than parameterized. The astro and nuxt dev overrides published it for “Vite HMR” with nothing listening behind it: a live astro dev container has 4321 open and nothing else, nuxt has 3000 and nothing else. Both frameworks carry HMR over the app’s own port and the router upgrades that websocket same-origin. Docker binds a published port whether or not anything answers, so this held a real port for nothing, and an astro stack holding 24678 killed a nuxt stack’s boot. HMR still works, through the router, at the app’s own hostname.

Composition, not duplication

Compose’s include: is how a service pulls in the backing services it needs without owning them:

include:
  - path: ./db.yml
  - path: ./redis.yml

Two services that both need Postgres include the same db.yml. One database, declared once.

Include only a file the recipe actually ships. Compose’s long-form include accepts path, project_directory and env_file and nothing else, so an optional: true key is silently ignored rather than honoured, and a missing sibling is a hard error instead of a skip. Every installed recipe’s assembled dev config has to pass docker compose config straight after mx add, which is the gate that now catches this class.

Container names

Services are named by you; containers are named by Compose. No compose file mx ships sets container_name, and a recipe may not either, so the name on disk is always Compose’s own <COMPOSE_PROJECT_NAME>-<service>-<index>. In a project directory called myproj, the database container is myproj-db-1.

The shipped compose files used to pin container_name: db, container_name: api and so on. Container names are a Docker-daemon-wide namespace, so once every project pinned its own COMPOSE_PROJECT_NAME, two mx projects that both declared db still collided with Conflict. The container name "/db" is already in use. The pins are gone.

Service names did not change. depends_on, make sh s=db, the Traefik labels and the include: graph all address services, and they behave exactly as before. What breaks is a script that addressed a container by a fixed name:

docker exec db psql -U postgres        # no longer resolves
docker logs api                        # no longer resolves

Use the project’s own verbs, or name the project explicitly:

make sh s=db                           # shell into it
make exec s=db c=psql                  # one command in it
make logs s=api                        # tail it
docker compose -p "$COMPOSE_PROJECT_NAME" exec db psql -U postgres

One container keeps a fixed name on purpose: the global router, mx-router. It is installed once per machine rather than per project, so there is nothing for it to collide with, and mx router addresses it by that name.

A related change rides along. The Postgres healthcheck now authenticates as the role the container is actually configured with, read from the container’s own environment. It used to fall back to postgres and report healthy while the role the app connects as did not exist. A project whose credentials are wrong now shows the db container as UNHEALTHY, which is the honest answer, where before it went green and the application failed later with an authentication error.

The three env layers

All under docker/.config/:

File	Holds	Committed
`.env.shared`	values every service needs: project name, ports, shared hostnames	yes
`.env.secrets`	credentials, API keys, database passwords	no (gitignored)
`.env.<service>`	one service’s settings	yes

.env.secrets.template is committed, with the keys and no values. make init copies it to .env.secrets if that file does not exist, and then fills it: it generates a real development value for every credential that is still empty or still one of the __GENERATE_*__ / CHANGE_ME placeholders the templates ship. A value that is already real, whether an earlier run generated it or you typed it, is never touched, so make init cannot rotate credentials out from under a running database. make dev runs init.sh on every start, which is why a service added with mx add after mx new comes up with working credentials without a hand edit.

One key stays deliberately blank: REDIS_PASSWORD. The shipped Redis runs without auth, and a generated password there would break every client that builds redis://:$REDIS_PASSWORD@redis:6379 against a server that wants none.

make doctor reports on the same file, cheaply and without Docker. It names any key that has no value yet, and any value still holding a ${VAR} reference, which is a latent empty: Compose interpolates a ${…} inside an env_file value from the compose project directory’s own .env, never from a sibling env file, so such a value reaches the container as an empty string. scripts/generate-secrets.sh resolves those references into literals for that reason. Generate a one-off secret of your own with make make-key BYTES=32 FORMAT=hex.

Precedence, precisely

Compose applies env_file entries in order and later entries win. Then the environment: block wins over all of them, and a value already exported in your shell wins over the compose file’s interpolation defaults.

Every compose file mx ships lists them in one order, and always has the same one:

env_file:
  - ../.config/.env.shared
  - ../.config/.env.secrets
  - ../.config/.env.<service>

So the effective precedence, highest first, is environment: → .env.<service> → .env.secrets → .env.shared. A credential in .env.secrets beats the same key in .env.shared, and one service’s .env.<service> beats both. That holds for application services and for the backing services alike: db.yml and redis.yml follow it too.

This used to be true of some files and not others. The application services in four recipes, plus all four of laravel’s compose fragments, listed .env.secrets first, which inverted it for exactly the layer most likely to collide, and the zola recipe omitted .env.secrets altogether. They were corrected, and the rule is now held by a conformance test over every compose file in templates/ rather than by authoring habit: every_env_file_list_follows_the_documented_precedence_order and every_env_file_list_carrying_shared_also_carries_secrets in crates/mx-lib/tests/templates_env_precedence.rs. You can read the order off any shipped file and trust that the next one says the same thing.

A worked example

What mx add api --recipe rust-api lands in docker/compose/api.yml is a plain instance of all of it:

include:
  - path: ./db.yml

services:
  api:
    build:
      context: ../..
      dockerfile: docker/dockerfiles/api/app
      target: production
    env_file:
      - ../.config/.env.shared
      - ../.config/.env.secrets
      - ../.config/.env.api
    environment:
      - RUST_ENV=production
      - RUST_LOG=info,api=debug
    networks:
      - default
      - devmesh-traefik
    depends_on:
      db:
        condition: service_healthy

The database URL is not in that environment: block, and that placement is the point. docker/.config/.env.api carries it:

DATABASE_URL=postgres://${DB_USER}:${DB_PASSWORD}@${DB_HOST:-db}:${DB_PORT:-5432}/${DB_NAME}

and scripts/generate-secrets.sh resolves those references into literals before anything starts, reading the credentials it generated in .env.secrets. It has to: Compose does not interpolate an env_file value from a sibling env file, so a ${DB_PASSWORD} left as a reference here would reach the container empty and Postgres would refuse to initialize. Literals are the only form that arrives.

One consequence worth knowing. Once resolved, a derived value is a copy, so changing DB_PASSWORD in .env.secrets later does not re-derive DATABASE_URL in .env.api. Blank the derived value and re-run make init to have it regenerated. make doctor reports anything left unset.

Dockerfiles

docker/dockerfiles/<service>/app is the development image and app.prod the shipped one. Both are usually multi-stage, and the compose files select a stage with build.target. That is how one dockerfile serves make dev and make build s=api prod=1.