# Build, Deploy & Release Recipe

URL: https://mechcrate.dev/docs/corpus/infra/build-deploy-recipe/
Source: docs/development/BUILD_DEPLOY_RECIPE.md

---

> A batteries-included blueprint for monorepo infrastructure: modular Makefiles, Docker organization, semantic versioning, Cloudflare deployment, and Terraform IaC. Use this to bootstrap any project with production-grade patterns.

---

## Table of Contents

1. [Project Structure Overview](#project-structure-overview)
2. [Makefile Module System](#makefile-module-system)
3. [Docker Organization](#docker-organization)
4. [Scripts Architecture](#scripts-architecture)
5. [Release & Version Bumping](#release--version-bumping)
6. [Cloudflare Workers & Containers](#cloudflare-workers--containers)
7. [Terraform Infrastructure](#terraform-infrastructure)
8. [Complete Bootstrap Checklist](#complete-bootstrap-checklist)

---

## Project Structure Overview

```
project-root/
├── Makefile                      # Root loader - includes all make/*.mk modules
├── make/                         # Modular Makefile targets (one per concern)
│   ├── app.mk                    # App build/run targets
│   ├── cloudflare.mk             # CF Workers deployment
│   ├── dev.mk                    # Development server
│   ├── gpu-infra.mk              # Terraform GPU infrastructure
│   ├── rag.mk                    # RAG system targets
│   ├── release.mk                # Version bumping & releases
│   └── whitepaper.mk             # Documentation compilation
│
├── apps/                         # Application packages (monorepo style)
│   ├── main-app/
│   │   ├── package.json          # Version source of truth
│   │   ├── CHANGELOG.md          # Auto-generated by standard-version
│   │   └── ...
│   └── admin-app/
│
├── brain/                        # Internal tooling packages
│   └── rag/                      # RAG system with its own package.json
│
├── infra/                        # All infrastructure code
│   ├── dockerfiles/              # Docker images (one folder per image)
│   │   ├── main-app/
│   │   │   ├── Dockerfile
│   │   │   └── base.dockerfile   # Alternative/specialized Dockerfile
│   │   └── sidecar-service/
│   │       ├── Dockerfile
│   │       ├── package.json      # Self-contained buildable package
│   │       ├── src/
│   │       └── tsconfig.json
│   │
│   ├── cloudflare/               # Cloudflare Workers (one folder per worker)
│   │   ├── containers-worker/    # Main app container orchestration
│   │   │   ├── wrangler.toml
│   │   │   ├── package.json
│   │   │   └── src/index.ts
│   │   ├── api-worker/           # API endpoints
│   │   ├── websocket-relay/      # WebSocket Durable Objects
│   │   └── README.md
│   │
│   ├── compose/                  # Docker Compose files for local/atomic services
│   │   └── app.yml
│   │
│   └── terraform/                # Terraform modules
│       └── gpu-node/
│           ├── main.tf
│           ├── variables.tf
│           ├── outputs.tf
│           ├── versions.tf
│           ├── cloud-init.yaml.tpl
│           ├── env.example
│           └── terraform.tfvars.example
│
├── scripts/                      # Shell scripts & tooling
│   ├── app-version.mjs           # Extract version from package.json
│   ├── release-sync-versions.mjs # Sync versions across manifests
│   ├── simple-release.sh         # Fallback release script
│   ├── setup-gpu-infra.sh        # Interactive setup wizard
│   ├── validate-gpu-infra.sh     # Infrastructure validation
│   │
│   ├── rag-*.sh                  # RAG command wrappers (thin shells)
│   │
│   └── whitepaper/               # Documentation compiler (self-contained)
│       ├── package.json
│       ├── compile.ts
│       ├── md2pdf.ts
│       ├── style.css
│       └── template.latex
│
├── docs/                         # Documentation
├── artifacts/                    # Build outputs (gitignored)
│
├── release-please-config.json    # Optional: Google release-please
├── .release-please-manifest.json
└── package.json                  # Root package (minimal - workspace deps only)
```

### Key Organization Principles

1. **One folder per concern** - Each Dockerfile, Worker, Terraform module gets its own folder
2. **Self-contained builds** - `infra/dockerfiles/*/` can include source code for buildable sidecars
3. **Scripts as thin wrappers** - Shell scripts delegate to actual implementations
4. **Infrastructure isolation** - All infra code under `infra/`, never mixed with app code

---

## Makefile Module System

### Root Makefile Pattern

```makefile
# Makefile
.SILENT:
SHELL := /bin/bash
.SHELLFLAGS := -eu -o pipefail -c
ROOT_DIR := $(abspath $(CURDIR))
APP_DIR := $(ROOT_DIR)/apps/main-app
MAKEFILES := $(wildcard make/*.mk)

.PHONY: help
help: ## Show all available targets
	@if [ -z "$(MAKEFILES)" ]; then \
		printf "No module targets loaded yet.\n"; \
	else \
		printf "Available targets:\n"; \
		grep -hE '^[a-zA-Z0-9_.-]+:.*##' $(MAKEFILES) | awk 'BEGIN {FS = ":.*##"}; {printf "  %-25s %s\n", $$1, $$2}'; \
	fi

# Utility: Generate secrets
BYTES ?= 32
FORMAT ?= hex

.PHONY: make-key
make-key: ## Generate a secret key (BYTES=32, FORMAT=hex|base64|uuid)
	@case "$(FORMAT)" in \
		hex)    openssl rand -hex $(BYTES) ;; \
		base64) openssl rand -base64 $(BYTES) ;; \
		uuid)   uuidgen | tr '[:upper:]' '[:lower:]' ;; \
		*)      echo "Unknown format: $(FORMAT)" && exit 1 ;; \
	esac

# Auto-include all modules
ifneq ($(strip $(MAKEFILES)),)
include $(MAKEFILES)
endif
```

### Module Pattern

Each `make/*.mk` file follows this pattern:

```makefile
# make/example.mk

# Module-specific path variables
EXAMPLE_DIR := $(ROOT_DIR)/path/to/component
EXAMPLE_SCRIPTS := $(ROOT_DIR)/scripts

# Targets with ## for auto-documentation
.PHONY: example-build
example-build: ## Build the example component
	@echo "Building..."
	@cd $(EXAMPLE_DIR) && ./build.sh

.PHONY: example-deploy
example-deploy: example-build ## Deploy (chains to build first)
	@echo "Deploying..."
	@cd $(EXAMPLE_DIR) && ./deploy.sh

# Pass-through arguments pattern
EXAMPLE_ARGS := $(strip $(ARGS))

.PHONY: example-run
example-run: ## Run with arguments (ARGS="--flag value")
	$(EXAMPLE_SCRIPTS)/example-run.sh $(EXAMPLE_ARGS)
```

### Conventions

| Convention | Example |
|------------|---------|
| Every target has `## Description` | `build: ## Build the app` |
| Use `@` prefix for silent execution | `@echo "message"` |
| Chain dependencies left-to-right | `deploy: build test` |
| Use `$(ROOT_DIR)` for absolute paths | `cd $(ROOT_DIR)/apps/...` |
| Pass arguments via `ARGS` variable | `make target ARGS="--flag"` |
| Group related targets with comments | `# ═══ Section ═══` |

---

## Docker Organization

### Folder Structure Philosophy

```
infra/dockerfiles/
├── main-app/                    # App container
│   ├── Dockerfile               # Production multi-stage build
│   └── base.dockerfile          # Alternative (e.g., for different base image)
│
└── sidecar-service/             # Self-contained service
    ├── Dockerfile
    ├── package.json             # Its own dependencies
    ├── tsconfig.json
    └── src/
        └── index.ts             # Source lives HERE, not in apps/
```

**Why this structure?**
- Docker COPY paths are clean and predictable
- Each image is self-contained (no cross-folder dependencies)
- Sidecars can have their own package.json without polluting main app
- Build context is minimal (just that folder)

### Multi-Stage Dockerfile Pattern

```dockerfile
# infra/dockerfiles/main-app/Dockerfile
# syntax=docker/dockerfile:1.7-labs

ARG NODE_IMAGE=node:20-slim
ARG APP_VERSION="0.0.0"

# ─────────────────────────────────────────────────────────────────
# Base: Shared settings
# ─────────────────────────────────────────────────────────────────
FROM ${NODE_IMAGE} AS base
WORKDIR /app
ENV NODE_ENV=production
RUN corepack enable

# ─────────────────────────────────────────────────────────────────
# Deps: Install all dependencies
# ─────────────────────────────────────────────────────────────────
FROM base AS deps
ENV NODE_ENV=development
COPY package.json yarn.lock ./
RUN --mount=type=cache,target=/root/.cache/yarn \
    yarn install --frozen-lockfile --non-interactive

# ─────────────────────────────────────────────────────────────────
# Builder: Build the application
# ─────────────────────────────────────────────────────────────────
FROM base AS builder
ARG APP_VERSION
ENV PUBLIC_APP_VERSION=${APP_VERSION}
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN yarn build && yarn cache clean

# ─────────────────────────────────────────────────────────────────
# Prod-deps: Production dependencies only
# ─────────────────────────────────────────────────────────────────
FROM deps AS prod-deps
ENV NODE_ENV=production
RUN --mount=type=cache,target=/root/.cache/yarn \
    yarn install --frozen-lockfile --production --prefer-offline

# ─────────────────────────────────────────────────────────────────
# Runner: Final minimal image
# ─────────────────────────────────────────────────────────────────
FROM ${NODE_IMAGE} AS runner
WORKDIR /app
ENV NODE_ENV=production \
    HOST=0.0.0.0 \
    PORT=8080

COPY package.json yarn.lock ./
COPY --from=prod-deps /app/node_modules ./node_modules
COPY --from=builder /app/dist ./dist

EXPOSE 8080
USER node

HEALTHCHECK --interval=30s --timeout=5s --start-period=10s \
  CMD node -e "fetch('http://127.0.0.1:8080/_health').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1));"

CMD ["node", "./dist/server/entry.mjs"]
```

### Sidecar Dockerfile Pattern (Alpine, Self-Contained)

```dockerfile
# infra/dockerfiles/sidecar-service/Dockerfile
FROM node:20-alpine AS builder
WORKDIR /app

COPY package*.json ./
RUN npm ci

COPY src ./src
COPY tsconfig.json ./
RUN npm run build
RUN npm ci --omit=dev

# ─────────────────────────────────────────────────────────────────
FROM node:20-alpine
WORKDIR /app

# Security: Non-root user
RUN addgroup -g 1001 -S appuser && \
    adduser -S appuser -u 1001 -G appuser

COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package.json ./

HEALTHCHECK --interval=15s --timeout=5s --start-period=10s --retries=3 \
  CMD wget -q --spider http://localhost:8080/health || exit 1

USER appuser
EXPOSE 8080
CMD ["node", "dist/index.js"]
```

### Docker Compose for Development

```yaml
# infra/compose/app.yml
version: '3.8'

services:
  main-app:
    build:
      context: ../dockerfiles/main-app
      dockerfile: Dockerfile
    ports:
      - "8080:8080"
    environment:
      - NODE_ENV=development
    volumes:
      - ../../apps/main-app/src:/app/src:ro
    healthcheck:
      test: ["CMD", "curl", "-sf", "http://localhost:8080/_health"]
      interval: 30s
      timeout: 10s
      retries: 3

  sidecar:
    build:
      context: ../dockerfiles/sidecar-service
    ports:
      - "8081:8080"
    environment:
      - MAIN_APP_URL=http://main-app:8080
    depends_on:
      main-app:
        condition: service_healthy
```

---

## Scripts Architecture

### Pattern: Thin Shell Wrappers

Scripts in the root `scripts/` folder are **thin wrappers** that delegate to actual implementations:

```bash
#!/usr/bin/env bash
# scripts/rag-query.sh
set -euo pipefail

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
yarn --cwd "$ROOT_DIR/brain/rag" rag:query "$@"
```

**Why?**
- Makefile targets call these scripts
- Scripts handle path resolution
- Actual implementation lives in the package (brain/rag)
- Easy to call from anywhere

### Script Categories

| Category | Pattern | Example |
|----------|---------|---------|
| **Wrappers** | Delegate to package scripts | `rag-*.sh` |
| **Utilities** | Standalone tools | `app-version.mjs` |
| **Setup wizards** | Interactive configuration | `setup-gpu-infra.sh` |
| **Validation** | Pre-flight checks | `validate-gpu-infra.sh` |
| **Subsystems** | Self-contained tooling | `whitepaper/` folder |

### Version Extraction Script

```javascript
#!/usr/bin/env node
// scripts/app-version.mjs
import { readFileSync } from 'node:fs';
import { fileURLToPath } from 'node:url';
import { dirname, join } from 'node:path';

const __dirname = dirname(fileURLToPath(import.meta.url));
const pkgPath = join(__dirname, '..', 'apps', 'main-app', 'package.json');
const pkg = JSON.parse(readFileSync(pkgPath, 'utf8'));

if (!pkg.version) {
  console.error('Unable to read version');
  process.exit(1);
}

process.stdout.write(pkg.version);
```

### Setup Wizard Pattern

```bash
#!/usr/bin/env bash
# scripts/setup-something.sh
set -euo pipefail

# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
CYAN='\033[0;36m'
NC='\033[0m'
BOLD='\033[1m'

# Paths
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"

# Helpers
print_header() { echo -e "\n${BOLD}${CYAN}═══ $1 ═══${NC}\n"; }
print_step() { echo -e "${GREEN}▶${NC} $1"; }
print_success() { echo -e "${GREEN}✓${NC} $1"; }
print_warning() { echo -e "${YELLOW}⚠${NC} $1"; }
print_error() { echo -e "${RED}✗${NC} $1"; }

check_command() {
  if command -v "$1" &> /dev/null; then
    print_success "$1 is installed"
    return 0
  else
    print_error "$1 is not installed"
    return 1
  fi
}

# Main
main() {
  print_header "Setup Wizard"
  
  # Step 1: Check prerequisites
  print_header "Checking Prerequisites"
  check_command "docker" || exit 1
  check_command "terraform" || exit 1
  
  # Step 2: Collect input
  print_header "Configuration"
  read -p "Enter API token: " -s API_TOKEN
  echo
  
  # Step 3: Write config files
  print_header "Writing Configuration"
  cat > "$PROJECT_ROOT/infra/terraform/.env" << EOF
API_TOKEN=$API_TOKEN
EOF
  print_success "Created .env file"
  
  # Summary
  print_header "Setup Complete"
  echo "Next steps:"
  echo "  1. make terraform-init"
  echo "  2. make terraform-plan"
}

main "$@"
```

### Validation Script Pattern

```bash
#!/usr/bin/env bash
# scripts/validate-something.sh
set -euo pipefail

PASSED=0
FAILED=0
WARNINGS=0

check_pass() { echo -e "✓ $1"; PASSED=$((PASSED + 1)); }
check_fail() { echo -e "✗ $1"; FAILED=$((FAILED + 1)); }
check_warn() { echo -e "⚠ $1"; WARNINGS=$((WARNINGS + 1)); }

# Run checks
[[ -f "config.json" ]] && check_pass "Config exists" || check_fail "Config missing"
command -v docker &>/dev/null && check_pass "Docker installed" || check_fail "Docker not found"

# Summary
echo ""
echo "Passed: $PASSED | Failed: $FAILED | Warnings: $WARNINGS"
[[ $FAILED -eq 0 ]] && exit 0 || exit 1
```

---

## Release & Version Bumping

### Strategy: standard-version + Make Targets

The release workflow uses [standard-version](https://github.com/conventional-changelog/standard-version) for:
- Automatic version bumping based on conventional commits
- CHANGELOG.md generation
- Git tagging

**Version source of truth:** `apps/main-app/package.json`

### Release Makefile Module

```makefile
# make/release.mk

RELEASE_SYNC_SCRIPT := $(ROOT_DIR)/scripts/release-sync-versions.mjs
APP_DIR := $(ROOT_DIR)/apps/main-app

# Post-release hook: sync versions across manifests
define post_release
	@echo "📋 Syncing versions across manifest files..."
	@node $(RELEASE_SYNC_SCRIPT) 2>/dev/null || true
	@echo "📄 Copying CHANGELOG.md to public folder..."
	@cp $(APP_DIR)/CHANGELOG.md $(APP_DIR)/public/CHANGELOG.md 2>/dev/null || true
	@git add -A 2>/dev/null || true
	@git commit --amend --no-edit 2>/dev/null || true
endef

# ─────────────────────────────────────────────────────────────────
# Main Release Commands
# ─────────────────────────────────────────────────────────────────

.PHONY: release
release: ## Create release (auto-determines version from commits)
	@echo "🚀 Creating release..."
	@cd $(APP_DIR) && npx standard-version
	$(post_release)
	@git push --follow-tags origin main
	@echo "✅ Release pushed!"

.PHONY: release-patch
release-patch: ## Force patch release (x.y.Z)
	@cd $(APP_DIR) && npx standard-version --release-as patch
	$(post_release)

.PHONY: release-minor
release-minor: ## Force minor release (x.Y.0)
	@cd $(APP_DIR) && npx standard-version --release-as minor
	$(post_release)

.PHONY: release-major
release-major: ## Force major release (X.0.0)
	@cd $(APP_DIR) && npx standard-version --release-as major
	$(post_release)

.PHONY: release-dry
release-dry: ## Preview release (no changes)
	@cd $(APP_DIR) && npx standard-version --dry-run

.PHONY: release-first
release-first: ## Create first release (when starting fresh)
	@cd $(APP_DIR) && npx standard-version --first-release
	$(post_release)

# ─────────────────────────────────────────────────────────────────
# Combined Commands
# ─────────────────────────────────────────────────────────────────

.PHONY: release-push
release-push: ## Push release commit and tags
	@git push --follow-tags origin main

.PHONY: release-full
release-full: release ## Alias (release already pushes)
```

### Version Sync Script

Keeps versions in sync across:
- `apps/main-app/package.json`
- `.release-please-manifest.json`
- Any other manifests

```javascript
#!/usr/bin/env node
// scripts/release-sync-versions.mjs
import { readFileSync, writeFileSync, existsSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';

const __dirname = dirname(fileURLToPath(import.meta.url));
const ROOT = join(__dirname, '..');

// Files to sync
const PACKAGE_JSON = join(ROOT, 'apps/main-app/package.json');
const MANIFEST = join(ROOT, '.release-please-manifest.json');

// Get current version
const pkg = JSON.parse(readFileSync(PACKAGE_JSON, 'utf8'));
const version = pkg.version;

console.log(`[sync] Version: ${version}`);

// Update manifest
if (existsSync(MANIFEST)) {
  const manifest = JSON.parse(readFileSync(MANIFEST, 'utf8'));
  manifest['apps/main-app'] = version;
  writeFileSync(MANIFEST, JSON.stringify(manifest, null, 2) + '\n');
  console.log(`[sync] Updated ${MANIFEST}`);
}
```

### Fallback: Simple Release Script

For when you need to bypass standard-version:

```bash
#!/bin/bash
# scripts/simple-release.sh
set -e

RELEASE_TYPE=${1:-patch}
APP_PATH="apps/main-app"

cd "$APP_PATH"
CURRENT=$(node -p "require('./package.json').version")
echo "Current: $CURRENT"

npm version $RELEASE_TYPE --no-git-tag-version
NEW=$(node -p "require('./package.json').version")
echo "New: $NEW"

cd ../..
git add "$APP_PATH/package.json"
git commit -m "chore: bump to v$NEW"

TAG="v$NEW"
git tag -a "$TAG" -m "Release $TAG"
git push origin HEAD
git push origin "$TAG"

echo "✅ Released $TAG"
```

### Conventional Commits Quick Reference

| Prefix | Version Bump | Example |
|--------|--------------|---------|
| `feat:` | Minor | `feat: add dark mode` |
| `fix:` | Patch | `fix: resolve crash` |
| `feat!:` or `BREAKING CHANGE:` | Major | `feat!: new API` |
| `chore:`, `docs:`, `style:` | None | `docs: update readme` |

---

## Cloudflare Workers & Containers

Cloudflare Containers allow you to run Docker containers on Cloudflare's edge network, managed by Durable Objects. This gives you:
- **Edge-deployed containers** - Your Docker image runs close to users
- **Auto-scaling** - Containers spin up/down based on demand
- **Sleep mode** - Containers sleep after inactivity (cost savings)
- **Managed registry** - Push directly to Cloudflare's container registry

### Architecture Overview

```
┌─────────────────────────────────────────────────────────────────┐
│                     Cloudflare Edge                             │
│  ┌───────────────┐      ┌─────────────────────────────────┐    │
│  │    Worker     │──────│   Container Durable Object      │    │
│  │  (index.ts)   │      │  ┌───────────────────────────┐  │    │
│  │               │      │  │   Your Docker Container   │  │    │
│  │ • Routes      │      │  │   (main-app:v1.0.0)       │  │    │
│  │ • Health      │      │  │                           │  │    │
│  │ • Management  │      │  │   PORT 8080               │  │    │
│  └───────────────┘      │  └───────────────────────────┘  │    │
│                         └─────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────────┘
```

**Flow:**
1. Request hits Cloudflare Worker
2. Worker gets/creates Container Durable Object instance
3. DO manages container lifecycle (start, stop, sleep)
4. Worker proxies request to container
5. Container responds via Worker

### Directory Structure

```
infra/cloudflare/
├── containers-worker/           # Main app container orchestration
│   ├── wrangler.toml           # Wrangler + container config
│   ├── package.json
│   ├── tsconfig.json
│   └── src/
│       └── index.ts            # Worker + Container DO class
│
├── api-worker/                  # Separate API worker (optional)
├── websocket-relay/             # WebSocket DO (optional)
└── README.md
```

### Deployment Pipeline

```
┌──────────────┐    ┌──────────────┐    ┌──────────────┐    ┌──────────────┐
│   1. Build   │───▶│   2. Push    │───▶│   3. Sync    │───▶│  4. Deploy   │
│   (Docker)   │    │  (Registry)  │    │  (wrangler)  │    │   (Worker)   │
└──────────────┘    └──────────────┘    └──────────────┘    └──────────────┘
     docker            wrangler            perl -pi           wrangler
     buildx          containers push       (update toml)       deploy
```

**`make cf-deploy` runs all 4 steps:**
1. Build Docker image locally with `docker buildx`
2. Push to Cloudflare's managed registry via `wrangler containers push`
3. Update `wrangler.toml` with the new image tag
4. Deploy the Worker which references the new image

### Makefile Module

```makefile
# make/cloudflare.mk

CF_ROOT := $(ROOT_DIR)/infra/cloudflare
CF_WORKER_DIR := $(CF_ROOT)/containers-worker
CF_WRANGLER_CONFIG := $(CF_WORKER_DIR)/wrangler.toml
CF_DOCKER_CONTEXT := $(ROOT_DIR)/infra/dockerfiles/main-app
CF_DOCKERFILE := $(CF_DOCKER_CONTEXT)/Dockerfile
CF_DOCKER_PLATFORM := linux/amd64
CF_IMAGE_NAME := main-app

# Version from package.json (source of truth)
APP_VERSION := $(shell node $(ROOT_DIR)/scripts/app-version.mjs)
CF_IMAGE_TAG := v$(APP_VERSION)

# Cloudflare registry URL pattern:
# registry.cloudflare.com/<ACCOUNT_ID>/<IMAGE_NAME>:<TAG>
CF_REGISTRY := registry.cloudflare.com/YOUR_ACCOUNT_ID/$(CF_IMAGE_NAME)
CF_IMAGE_URI := $(CF_REGISTRY):$(CF_IMAGE_TAG)

# ═══════════════════════════════════════════════════════════════════
# Authentication
# ═══════════════════════════════════════════════════════════════════

.PHONY: cf-login
cf-login: ## Authenticate with Cloudflare (opens browser)
	cd $(ROOT_DIR) && npx wrangler login

# ═══════════════════════════════════════════════════════════════════
# Container Build & Push
# ═══════════════════════════════════════════════════════════════════

.PHONY: cf-container-build
cf-container-build: ## Build and push via Wrangler (simpler, but less control)
	cd $(ROOT_DIR) && npx wrangler containers build $(CF_DOCKER_CONTEXT) \
		--tag $(CF_IMAGE_NAME):$(CF_IMAGE_TAG) --push

.PHONY: cf-container-build-local
cf-container-build-local: ## Build container locally with Docker (more control)
	docker buildx build \
		--platform $(CF_DOCKER_PLATFORM) \
		-f $(CF_DOCKERFILE) \
		-t $(CF_IMAGE_NAME):$(CF_IMAGE_TAG) \
		--build-arg APP_VERSION=$(APP_VERSION) \
		--load \
		$(CF_DOCKER_CONTEXT)

.PHONY: cf-container-push
cf-container-push: ## Push local image to Cloudflare registry
	cd $(ROOT_DIR) && npx wrangler containers push $(CF_IMAGE_NAME):$(CF_IMAGE_TAG)

.PHONY: cf-container-publish
cf-container-publish: cf-container-build-local cf-container-push ## Build locally + push

.PHONY: cf-container-images
cf-container-images: ## List images in Cloudflare registry
	cd $(ROOT_DIR) && npx wrangler containers images list --filter $(CF_IMAGE_NAME)

# ═══════════════════════════════════════════════════════════════════
# Wrangler Config Sync
# ═══════════════════════════════════════════════════════════════════

.PHONY: cf-sync-wrangler-image
cf-sync-wrangler-image: ## Update wrangler.toml with latest image tag
	@echo "Updating image references to $(CF_IMAGE_URI)"
	@perl -0pi -e 's!image = "$(CF_REGISTRY):[^"]+"!image = "$(CF_IMAGE_URI)"!g' $(CF_WRANGLER_CONFIG)

# ═══════════════════════════════════════════════════════════════════
# Worker Deployment
# ═══════════════════════════════════════════════════════════════════

.PHONY: cf-worker-install
cf-worker-install: ## Install worker dependencies
	cd $(CF_WORKER_DIR) && npm install

.PHONY: cf-worker-dev
cf-worker-dev: cf-worker-install ## Run worker locally (dev mode)
	cd $(CF_WORKER_DIR) && npm run dev

.PHONY: cf-worker-deploy
cf-worker-deploy: cf-worker-install ## Deploy worker only (no container build)
	cd $(ROOT_DIR) && npx wrangler deploy --config $(CF_WRANGLER_CONFIG) --env production

# ═══════════════════════════════════════════════════════════════════
# Full Deploy (The Main Command)
# ═══════════════════════════════════════════════════════════════════

.PHONY: cf-deploy
cf-deploy: cf-container-publish cf-sync-wrangler-image cf-worker-install ## Full deploy: build → push → sync → deploy
	git checkout main && git pull
	cd $(ROOT_DIR) && npx wrangler deploy --config $(CF_WRANGLER_CONFIG) --env production
	@echo "✅ Deployed $(CF_IMAGE_URI) to production"
```

### Worker Code Pattern

The Worker extends `Container` from `@cloudflare/containers` and handles routing:

```typescript
// infra/cloudflare/containers-worker/src/index.ts
import { Container, getContainer } from '@cloudflare/containers';

// Health check paths that return container state instead of proxying
const HEALTH_PATHS = new Set(['/_health', '/_healthz', '/_readyz']);

// Default instance name (can be overridden via env or query param)
const DEFAULT_INSTANCE = 'main-app-ssr';
const CONTAINER_PORT = 8080;

/**
 * Container Durable Object class
 * Extends Container to configure your specific container settings
 */
export class MainAppContainerDO extends Container {
  // Port your container listens on
  defaultPort = CONTAINER_PORT;
  
  // Auto-sleep after 10 minutes of inactivity
  sleepAfter = '10m';
  
  // Environment variables passed to the container
  envVars = {
    NODE_ENV: 'production',
    HOST: '0.0.0.0',
    PORT: `${CONTAINER_PORT}`
  };
}

interface Env {
  // Durable Object binding (configured in wrangler.toml)
  APP_CONTAINER: DurableObjectNamespace<MainAppContainerDO>;
  
  // Optional: Pin requests to a specific instance
  CONTAINER_INSTANCE_ID?: string;
}

const worker = {
  async fetch(request: Request, env: Env): Promise<Response> {
    const url = new URL(request.url);
    
    // Determine which container instance to use
    // Priority: query param > env var > default
    const instanceId =
      url.searchParams.get('instance') ?? 
      env.CONTAINER_INSTANCE_ID ?? 
      DEFAULT_INSTANCE;
    
    // Get container handle (creates if doesn't exist)
    const container = getContainer(env.APP_CONTAINER, instanceId);

    // ─────────────────────────────────────────────────────────────
    // Health Check Endpoints - Return container state
    // ─────────────────────────────────────────────────────────────
    if (HEALTH_PATHS.has(url.pathname)) {
      const state = await container.getState();
      return Response.json({
        status: state.status,
        lastChange: new Date(state.lastChange).toISOString(),
        exitCode: 'exitCode' in state ? state.exitCode : undefined
      }, { 
        headers: { 'cache-control': 'no-store' } 
      });
    }

    // ─────────────────────────────────────────────────────────────
    // Management Endpoints - Container lifecycle control
    // ─────────────────────────────────────────────────────────────
    
    // Force restart
    if (url.pathname === '/_container/restart' && request.method === 'POST') {
      await container.destroy();
      return new Response('Restart requested', { status: 202 });
    }

    // Force start and wait for ready
    if (url.pathname === '/_container/start' && request.method === 'POST') {
      await container.startAndWaitForPorts({ ports: [CONTAINER_PORT] });
      const state = await container.getState();
      return Response.json({ message: 'Container started', state, instanceId });
    }

    // Get current status
    if (url.pathname === '/_container/status') {
      const state = await container.getState();
      return Response.json({ state, instanceId });
    }

    // Graceful stop
    if (url.pathname === '/_container/stop' && request.method === 'POST') {
      await container.stop();
      return new Response('Stop signal sent', { status: 202 });
    }

    // ─────────────────────────────────────────────────────────────
    // Default: Proxy to container
    // ─────────────────────────────────────────────────────────────
    return container.fetch(request);
  }
};

export default worker;
```

### Worker package.json

```json
{
  "name": "containers-worker",
  "private": true,
  "type": "module",
  "scripts": {
    "dev": "wrangler dev",
    "deploy": "wrangler deploy",
    "deploy:production": "wrangler deploy --env production"
  },
  "dependencies": {
    "@cloudflare/containers": "^0.0.30"
  },
  "devDependencies": {
    "@cloudflare/workers-types": "^4.20241112.0",
    "typescript": "^5.6.3",
    "wrangler": "^3.88.0"
  }
}
```

### Wrangler Configuration (Annotated)

```toml
# infra/cloudflare/containers-worker/wrangler.toml

# Worker name (appears in dashboard)
name = "main-app-worker"

# Entry point
main = "src/index.ts"

# Cloudflare API compatibility date
compatibility_date = "2024-11-16"

# Your Cloudflare account ID (find in dashboard URL or `wrangler whoami`)
account_id = "YOUR_ACCOUNT_ID"

# Enable workers.dev URL for testing
workers_dev = true

# ═══════════════════════════════════════════════════════════════════
# Environment Variables (non-secret)
# ═══════════════════════════════════════════════════════════════════
[vars]
CONTAINER_INSTANCE_ID = "main-app-dev"

# ═══════════════════════════════════════════════════════════════════
# Container Configuration
# ═══════════════════════════════════════════════════════════════════
[[containers]]
# Name for this container definition
name = "main-app"

# Must match the exported class name in your worker
class_name = "MainAppContainerDO"

# Full registry path: registry.cloudflare.com/<account_id>/<name>:<tag>
# This gets updated by `make cf-sync-wrangler-image`
image = "registry.cloudflare.com/YOUR_ACCOUNT_ID/main-app:v1.0.0"

# Max concurrent container instances
max_instances = 5

# ═══════════════════════════════════════════════════════════════════
# Durable Object Bindings
# ═══════════════════════════════════════════════════════════════════
[[durable_objects.bindings]]
# Name used in worker code: env.APP_CONTAINER
name = "APP_CONTAINER"

# Must match class_name in [[containers]]
class_name = "MainAppContainerDO"

# ═══════════════════════════════════════════════════════════════════
# Migrations (required for Durable Objects)
# ═══════════════════════════════════════════════════════════════════
[[migrations]]
tag = "v1"
new_sqlite_classes = ["MainAppContainerDO"]

# ═══════════════════════════════════════════════════════════════════
# Production Environment
# ═══════════════════════════════════════════════════════════════════
[env.production]
# Disable workers.dev URL in production
workers_dev = false

# Route configuration - attach to your domain
routes = [
  { pattern = "yourdomain.com/*", zone_name = "yourdomain.com" }
]

# Production container config (same structure, can have different settings)
[[env.production.containers]]
name = "main-app"
class_name = "MainAppContainerDO"
image = "registry.cloudflare.com/YOUR_ACCOUNT_ID/main-app:v1.0.0"
max_instances = 5

[[env.production.durable_objects.bindings]]
name = "APP_CONTAINER"
class_name = "MainAppContainerDO"

[env.production.vars]
CONTAINER_INSTANCE_ID = "main-app-prod"
```

### Management Endpoints

Once deployed, these endpoints are available:

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/_health` | GET | Container health status |
| `/_container/status` | GET | Detailed container state |
| `/_container/start` | POST | Force start container |
| `/_container/stop` | POST | Graceful stop |
| `/_container/restart` | POST | Destroy and recreate |

**Usage:**
```bash
# Check health
curl https://yourdomain.com/_health

# Get detailed status
curl https://yourdomain.com/_container/status

# Force start (useful for pre-warming)
curl -X POST https://yourdomain.com/_container/start

# Restart after deploy
curl -X POST https://yourdomain.com/_container/restart
```

### Bootstrap: New Project Setup

**1. Create the directory structure:**
```bash
mkdir -p infra/cloudflare/containers-worker/src
mkdir -p infra/dockerfiles/main-app
```

**2. Initialize the worker package:**
```bash
cd infra/cloudflare/containers-worker
npm init -y
npm install @cloudflare/containers
npm install -D @cloudflare/workers-types typescript wrangler
```

**3. Create tsconfig.json:**
```json
{
  "compilerOptions": {
    "target": "ES2022",
    "module": "ES2022",
    "moduleResolution": "bundler",
    "lib": ["ES2022"],
    "types": ["@cloudflare/workers-types"],
    "strict": true,
    "skipLibCheck": true
  },
  "include": ["src"]
}
```

**4. Copy the worker code pattern** (from above) to `src/index.ts`

**5. Copy the wrangler.toml pattern** and replace:
- `YOUR_ACCOUNT_ID` with your Cloudflare account ID
- `main-app` with your app name
- `yourdomain.com` with your domain

**6. Login and deploy:**
```bash
npx wrangler login
npx wrangler deploy  # deploys to workers.dev for testing
```

**7. First container push:**
```bash
# From project root
docker buildx build --platform linux/amd64 \
  -f infra/dockerfiles/main-app/Dockerfile \
  -t main-app:v1.0.0 \
  --load \
  infra/dockerfiles/main-app

npx wrangler containers push main-app:v1.0.0
```

**8. Deploy to production:**
```bash
npx wrangler deploy --env production
```

### Troubleshooting

| Issue | Solution |
|-------|----------|
| Container not starting | Check `/_container/status` for exitCode |
| Image not found | Run `wrangler containers images list` to verify push |
| Wrong version deployed | Run `make cf-sync-wrangler-image` then redeploy |
| Container sleeping | Hit any endpoint or POST to `/_container/start` |
| Auth errors | Run `wrangler login` again |

---

## Terraform Infrastructure

### File Organization

```
infra/terraform/gpu-node/
├── main.tf                  # Resources (droplet, firewall)
├── variables.tf             # Input variables with validation
├── outputs.tf               # Computed outputs
├── versions.tf              # Provider requirements
├── cloud-init.yaml.tpl      # VM bootstrap template
├── env.example              # Example environment file
├── terraform.tfvars.example # Example variables file
└── modules/                 # Optional sub-modules
    ├── monitoring/
    └── networking/
```

### Makefile Module

```makefile
# make/gpu-infra.mk

TF_DIR := $(ROOT_DIR)/infra/terraform/gpu-node
SIDECAR_DIR := $(ROOT_DIR)/infra/dockerfiles/sidecar-service

# Load environment if exists
-include $(TF_DIR)/.env

# ─────────────────────────────────────────────────────────────────
# Terraform Commands
# ─────────────────────────────────────────────────────────────────

.PHONY: gpu-init
gpu-init: ## Initialize Terraform
	@cd $(TF_DIR) && terraform init

.PHONY: gpu-validate
gpu-validate: ## Validate configuration
	@cd $(TF_DIR) && terraform validate

.PHONY: gpu-plan
gpu-plan: ## Plan changes
	@cd $(TF_DIR) && terraform plan -var-file=terraform.tfvars

.PHONY: gpu-apply
gpu-apply: ## Apply changes (creates resources)
	@echo "⚠️  This will create billable resources!"
	@cd $(TF_DIR) && terraform apply -auto-approve -var-file=terraform.tfvars

.PHONY: gpu-destroy
gpu-destroy: ## Destroy all resources
	@cd $(TF_DIR) && terraform destroy -auto-approve -var-file=terraform.tfvars

.PHONY: gpu-output
gpu-output: ## Show outputs
	@cd $(TF_DIR) && terraform output

# ─────────────────────────────────────────────────────────────────
# SSH Access
# ─────────────────────────────────────────────────────────────────

.PHONY: gpu-ssh
gpu-ssh: ## SSH into the instance
	@IP=$$(cd $(TF_DIR) && terraform output -raw ipv4_address 2>/dev/null) && \
	if [ -n "$$IP" ]; then ssh root@$$IP; \
	else echo "❌ No instance running"; fi

.PHONY: gpu-logs
gpu-logs: ## View container logs
	@IP=$$(cd $(TF_DIR) && terraform output -raw ipv4_address 2>/dev/null) && \
	if [ -n "$$IP" ]; then \
		ssh root@$$IP "docker compose logs -f --tail=100"; \
	else echo "❌ No instance running"; fi

# ─────────────────────────────────────────────────────────────────
# Sidecar Docker
# ─────────────────────────────────────────────────────────────────

.PHONY: sidecar-build
sidecar-build: ## Build sidecar image
	@cd $(SIDECAR_DIR) && docker buildx build --platform linux/amd64 -t sidecar:latest .

.PHONY: sidecar-push
sidecar-push: ## Push to GHCR
	@cd $(SIDECAR_DIR) && docker buildx build --platform linux/amd64 \
		-t ghcr.io/YOUR_ORG/sidecar:latest --push .

# ─────────────────────────────────────────────────────────────────
# Setup
# ─────────────────────────────────────────────────────────────────

.PHONY: gpu-setup
gpu-setup: ## Interactive setup wizard
	@$(ROOT_DIR)/scripts/setup-gpu-infra.sh

.PHONY: gpu-test
gpu-test: ## Validate infrastructure
	@$(ROOT_DIR)/scripts/validate-gpu-infra.sh
```

### Example Files Pattern

```bash
# infra/terraform/gpu-node/env.example
# Copy to .env and fill in values. NEVER commit .env!
DIGITALOCEAN_TOKEN=dop_v1_your_token_here
GHCR_TOKEN=ghp_your_github_token_here
```

```hcl
# infra/terraform/gpu-node/terraform.tfvars.example
# Copy to terraform.tfvars. NEVER commit terraform.tfvars!

session_id = "sess-example001"
user_id    = "user-example"
auth_token = "generate-with-openssl-rand-hex-32"

region   = "nyc2"
gpu_size = "gpu-h100x1-80gb"

admin_ssh_key_ids = []

ghcr_username = "your-github-username"
ghcr_token    = "ghp_your_token"
```

---

## Complete Bootstrap Checklist

### 1. Create Directory Structure

```bash
mkdir -p make infra/{dockerfiles,cloudflare,compose,terraform} scripts apps/main-app brain docs artifacts
```

### 2. Create Root Makefile

Copy the root Makefile pattern from [Makefile Module System](#makefile-module-system).

### 3. Create Initial Modules

At minimum:
- `make/dev.mk` - Development server
- `make/release.mk` - Versioning

### 4. Set Up Docker

1. Create `infra/dockerfiles/main-app/Dockerfile`
2. For sidecars: create self-contained folders with their own `package.json`

### 5. Set Up Release Tooling

```bash
cd apps/main-app
npm init -y  # or yarn init
npm install -D standard-version
```

Add to `package.json`:
```json
{
  "scripts": {
    "release": "standard-version",
    "release:minor": "standard-version --release-as minor",
    "release:major": "standard-version --release-as major",
    "release:patch": "standard-version --release-as patch",
    "release:dry": "standard-version --dry-run"
  }
}
```

### 6. Create Version Scripts

- `scripts/app-version.mjs`
- `scripts/release-sync-versions.mjs`

### 7. Add to .gitignore

```gitignore
# Terraform
*.tfstate
*.tfstate.backup
*.tfvars
!*.tfvars.example
.terraform/

# Environment
**/.env
!**/.env.example

# Artifacts
artifacts/
dist/
node_modules/
```

### 8. Verify Setup

```bash
make help              # Should list all targets
make release-dry       # Should preview release
```

---

## Quick Reference

| Command | Description |
|---------|-------------|
| `make help` | List all targets |
| `make dev` | Start dev server |
| `make release` | Create release (auto-bump) |
| `make release-patch` | Force patch bump |
| `make release-minor` | Force minor bump |
| `make release-major` | Force major bump |
| `make release-dry` | Preview release |
| `make cf-deploy` | Full Cloudflare deploy |
| `make cf-container-build-local` | Build Docker image |
| `make gpu-apply` | Create GPU infrastructure |
| `make gpu-destroy` | Tear down infrastructure |
| `make gpu-ssh` | SSH into instance |
| `make make-key` | Generate secret |
| `make make-key FORMAT=base64` | Generate base64 secret |