Platform Runbook

This runbook provides frequently used commands for operating the CTMS platform. Commands are organized by task with both native Docker Compose commands and Make-based shortcuts.

Environment Selection

Most Make commands default to .env.production. Override with ENV=:

make up                         # uses .env.production (default)
make up ENV=.env.zynomi.prod    # uses instance-specific env

Quick Reference

Task	Make Shortcut	Docker Compose Equivalent
Start core services	`make up`	`docker compose --env-file .env.production up -d`
Start all services	`make up-all`	`docker compose --env-file .env.production --profile all up -d`
Stop all services	`make down`	`docker compose --profile all down`
View service status	`make status`	`docker compose --profile all ps`
Health check	`make health`	`curl` per service (see below)
View all logs	`make logs`	`docker compose --profile all logs -f`
Pull latest images	`make pull`	`docker compose pull`
First-time setup	`make setup`	— (hosts + pull)

1. Service Management (ctms.devops)

Starting Services

Task	Make Command	Docker Compose Command
Core only (Caddy, KrakenD, Zynexa, Sublink, ODM)	`make up`	`docker compose --env-file .env.production up -d`
Core + Observability	`make up-obs`	`docker compose --env-file .env.production --profile core --profile observability up -d`
Core + Analytics	`make up-analytics`	`docker compose --env-file .env.production --profile core --profile analytics up -d`
All services	`make up-all`	`docker compose --env-file .env.production --profile all up -d`

Stopping & Restarting

Task	Make Command	Docker Compose Command
Stop all	`make down`	`docker compose --profile all down`
Restart all	`make restart`	`docker compose --env-file .env.production restart`
Stop + remove volumes (destructive)	`make clean`	`docker compose --profile all down -v --remove-orphans`

Instance-Specific Deployments

# Deploy for a specific instance
docker compose -f docker-compose.yml -f docker-compose.prod.yml \
  --env-file .env.<instance>.prod --profile all up -d

# Examples
docker compose -f docker-compose.yml -f docker-compose.prod.yml \
  --env-file .env.zynomi.prod --profile all up -d

Individual Service Management

restart vs up -d — Know the Difference

docker compose restart <service> only restarts the container process — it does NOT re-read .env.production or compose file changes. If you changed an environment variable (e.g., CUBEJS_DEV_MODE), you must use up -d to recreate the container with the new config:

# ❌ WRONG — env changes are NOT picked up
docker compose restart cube

# ✅ CORRECT — recreates container with latest env
docker compose -f docker-compose.yml -f docker-compose.prod.yml \
  --env-file .env.production up -d cube

Rule of thumb: Use restart only for a quick process bounce. Use up -d after any config change.

The canonical compose command for production (IP-based, before DNS) is:

DC="docker compose -f docker-compose.yml -f docker-compose.prod.yml --env-file .env.production"

Task	Start / Recreate (picks up env changes)	Quick Restart (no env reload)
API Gateway	`$DC up -d api-gateway`	`$DC restart api-gateway`
Zynexa	`$DC up -d zynexa`	`$DC restart zynexa`
Sublink	`$DC up -d sublink`	`$DC restart sublink`
Caddy (reload config)	`$DC exec caddy caddy reload --config /etc/caddy/Caddyfile`	—
OpenObserve + OTEL	`$DC --profile observability up -d openobserve otel-collector`	`$DC restart openobserve otel-collector`
Cube.dev	`$DC --profile analytics up -d cube`	`$DC restart cube`
MCP Server	`$DC up -d mcp-server`	`$DC restart mcp-server`
ODM API	`$DC up -d odm-api`	`$DC restart odm-api`

Common Scenarios

# Set the canonical compose command
DC="docker compose -f docker-compose.yml -f docker-compose.prod.yml --env-file .env.production"

# Scenario: Changed CUBEJS_DEV_MODE in .env.production
# Only 'cube' needs recreating — cubestore is unused when DEV_MODE=true
$DC --profile analytics up -d cube

# Scenario: Changed NEXTAUTH_SECRET or NEXTAUTH_URL
$DC up -d zynexa

# Scenario: Changed FRAPPE_API_TOKEN
$DC up -d zynexa mcp-server odm-api

# Scenario: Changed EC2_PUBLIC_IP (affects RUNTIME_* vars in prod overlay)
$DC --profile all up -d

# Scenario: Quick restart after OOM or crash (no config changes)
$DC restart zynexa

2. Logs

Task	Make Command	Docker Compose Command
All services	`make logs`	`docker compose --profile all logs -f`
Caddy	`make caddy-logs`	`docker compose logs -f caddy`
API Gateway	`make api-logs`	`docker compose logs -f api-gateway`
Zynexa	`make zynexa-logs`	`docker compose logs -f zynexa`
Sublink	`make sublink-logs`	`docker compose logs -f sublink`
OpenObserve	`make openobserve-logs`	`docker compose logs -f openobserve`
OTEL Collector	`make otel-logs`	`docker compose logs -f otel-collector`
Cube.dev	`make cube-logs`	`docker compose logs -f cube`

Useful Log Filters

# Last 100 lines of a service
docker compose logs --tail 100 zynexa

# Logs since a specific time
docker compose logs --since 1h zynexa

# Logs for multiple services at once
docker compose logs -f zynexa api-gateway caddy

3. Shell Access

Container	Make Command	Docker Compose Command
Caddy	`make caddy-shell`	`docker compose exec caddy sh`
API Gateway	`make api-shell`	`docker compose exec api-gateway sh`
Zynexa	`make zynexa-shell`	`docker compose exec zynexa sh`
Sublink	`make sublink-shell`	`docker compose exec sublink sh`
OpenObserve	`make openobserve-shell`	`docker compose exec openobserve sh`
Cube.dev	`make cube-shell`	`docker compose exec cube sh`

4. Health Checks

Make Command

make health

Manual Health Checks

Service	Command	Expected
Caddy	`curl -s https://zynexa.localhost`	200
API Gateway	`curl -s https://api.localhost/__health`	`{"status":"ok"}`
Zynexa	`curl -s http://localhost:3000/api/health`	200
Sublink	`curl -s http://localhost:3001`	200
Cube.dev	`curl -s http://localhost:4000/readyz`	`{"health":"HEALTH"}`
OpenObserve	`curl -s http://localhost:5080/healthz`	200
ODM API	`curl -s http://localhost:8000/health`	200
MCP Server	`curl -s http://localhost:8006/health`	200

Docker Health Status

# Check container health status
docker compose --profile all ps --format "table {{.Name}}\t{{.Status}}"

# Check a specific service
docker inspect --format='{{.State.Health.Status}}' ctms-zynexa

5. CTMS Init & Provisioning

Docker-Based (Recommended)

Task	Command
Run all 5 stages	`docker compose --env-file .env.production --profile init up ctms-init`
Run specific stages	`CTMS_INIT_STAGES=3,4 docker compose --env-file .env.production --profile init run --rm ctms-init`
Dry run	`CTMS_INIT_DRY_RUN=true docker compose --env-file .env.production --profile init run --rm ctms-init`
Seed Supabase tables	`docker compose --env-file .env.production --profile init run --rm ctms-supabase-seed`

Make-Based (Python Scripts)

Requires one-time setup: make frappe-seed-setup

Task	Make Command
Full 5-stage provisioning	`make frappe-provision ENV=../../.env.production`
Stage 1: Create DocTypes	`make frappe-setup-doctypes ENV=../../.env.production`
Stage 2: Create Custom Fields	`make frappe-setup-custom-fields ENV=../../.env.production`
Stage 3: Seed RBAC	`make frappe-seed-rbac ENV=../../.env.production`
Stage 4: Seed Master Data	`make frappe-seed-master-data ENV=../../.env.production`
Sync Permissions	`make frappe-sync-permissions ENV=../../.env.production`
Dry run Custom Fields	`make frappe-setup-custom-fields-dry-run ENV=../../.env.production`

6. Data Pipeline (Lakehouse)

Docker Commands

Task	Command
Start Lakehouse DB	`docker compose --env-file .env.production --profile lakehouse up -d lakehouse-db`
Run data ingestion	`docker compose --env-file .env.production --profile lakehouse run --rm lakehouse-ingester`
Run dbt daily pipeline	`docker compose --env-file .env.production --profile lakehouse run --rm lakehouse-dbt dbt-daily`
Run dbt full refresh	`docker compose --env-file .env.production --profile lakehouse run --rm lakehouse-dbt dbt-full-refresh`
Purge Lakehouse schemas	See below

Purge Lakehouse Schemas (Reset)

docker exec ctms-lakehouse-db psql -U ctms_user -d ctms_dlh -c "
DROP SCHEMA IF EXISTS raw CASCADE;
DROP SCHEMA IF EXISTS raw_staging CASCADE;
DROP SCHEMA IF EXISTS bronze CASCADE;
DROP SCHEMA IF EXISTS bronze_staging CASCADE;
DROP SCHEMA IF EXISTS silver CASCADE;
DROP SCHEMA IF EXISTS gold CASCADE;
"

7. Vendor Stacks (Self-Hosted)

Supabase

Task	Command
Start	`cd supabase && docker compose up -d`
Stop	`cd supabase && docker compose down`
Status	`cd supabase && docker compose ps`
Logs	`cd supabase && docker compose logs -f`

Frappe

Task	Command
Start	`cd frappe-marley-health && docker compose up -d`
Stop	`cd frappe-marley-health && docker compose down`
Status	`cd frappe-marley-health && docker compose ps`
Get API token	`docker logs frappe-marley-health-setup-1 2>&1 \| grep FRAPPE_API_TOKEN`
Regenerate API token	`docker exec -w /home/frappe/frappe-bench frappe-marley-health-backend-1 bash -c 'source env/bin/activate && python3 /setup/frappe-generate-token.py'`
Frappe shell (bench)	`docker exec -it frappe-marley-health-backend-1 bash`

Frappe Migration Tools

Task	Make Command
Setup migration venv	`make frappe-migration-setup`
Compare DocTypes (all)	`make frappe-compare`
Compare Custom module only	`make frappe-compare-custom`
Compare Custom Fields	`make frappe-compare-custom-fields`
List source DocTypes	`make frappe-list-source`
List target DocTypes	`make frappe-list-target`

8. Web Application (hb-life-science-web)

Development

Task	Make Command	Native Command
Install dependencies	`make setup`	`bun install`
Start dev server	`make dev`	`bun run dev`
Build for production	`make build`	`bun run build`
Start production server	`make start`	`bun run start`
Lint code	`make lint`	`bun run lint`
View configuration	`make status`	—

Vercel Deployment

Task	Make Command	Native Command
Link to Vercel project	`make vercel-link`	`vercel link`
Deploy preview	`make deploy`	`vercel`
Deploy production	`make deploy-prod`	`vercel --prod`
Pull env variables	`make vercel-env-pull`	`vercel env pull .env.local`
Add env variable	`make vercel-env-add KEY=name VALUE=value`	`vercel env add`
View logs	`make vercel-logs`	`vercel logs`

API Generation

Task	Command
Generate entity APIs	`bun run generate:apis`
Generate OpenAPI spec	`bun run openapi:generate`
Validate OpenAPI spec	`bun run openapi:validate`
Export Postman collection	`bun run openapi:postman`

9. Common Workflows

Full Platform Start (On-Prem)

cd ctms.devops

# 1. Start Supabase (creates ctms-network)
cd supabase && docker compose up -d && cd ..

# 2. Seed CTMS tables into Supabase
docker compose --env-file .env.local --profile init run --rm ctms-supabase-seed

# 3. Start Frappe (setup auto-completes wizard + generates API token)
cd frappe-marley-health && docker compose up -d && cd ..

# 4. Retrieve the generated API token → update .env.local
docker logs frappe-marley-health-setup-1 2>&1 | grep FRAPPE_API_TOKEN

# 5. Provision Frappe with CTMS data model (5 stages)
docker compose --env-file .env.local --profile init up ctms-init

# 6. Start CTMS core services
make up                             # or: docker compose --env-file .env.local up -d

# 7. (Optional) Start analytics + observability
make up-all                         # or: docker compose --env-file .env.local --profile all up -d

Daily Data Refresh

cd ctms.devops

# 1. Extract data from Frappe API → Lakehouse
docker compose --env-file .env.production --profile lakehouse run --rm lakehouse-ingester

# 2. Transform through dbt layers (Bronze → Silver → Gold)
docker compose --env-file .env.production --profile lakehouse run --rm lakehouse-dbt dbt-daily

# 3. Clear Cube.dev analytics cache
make cube-restart

Full Platform Stop (On-Prem, Reverse Order)

cd ctms.devops

# CTMS core
make down

# Frappe
cd frappe-marley-health && docker compose down && cd ..

# Supabase (last — owns ctms-network)
cd supabase && docker compose down && cd ..

10. URLs Reference

After `make up` (Core)

Service	URL
Zynexa	`https://zynexa.localhost`
Sublink	`https://sublink.localhost`
API Gateway	`https://api.localhost`
ODM API	`https://odm.localhost`

After `make up-obs` (+ Observability)

Service	URL
OpenObserve	`https://observe.localhost`

After `make up-analytics` (+ Analytics)

Service	URL
Cube.dev Playground	`https://cube.localhost`
MCP Server	`https://mcp.localhost`

Vendor Stacks (Self-Hosted)

Service	URL
Supabase Studio	`http://localhost:8000`
Frappe Dashboard	`http://localhost:8080/app`

11. Environment Files

File	Purpose
`.env.example`	Template (commit-safe, checked into Git)
`.env.production`	Default production configuration
`.env.<instance>.prod`	Instance-specific production (e.g., `.env.zynomi.prod`)
`.env.local`	Local development (gitignored)

Getting Help

Each component's Makefile includes built-in help:

cd <component-directory>
make help

This displays all available commands with descriptions.

Quick Reference​

1. Service Management (ctms.devops)​

Starting Services​

Stopping & Restarting​

Instance-Specific Deployments​

Individual Service Management​

Common Scenarios​

2. Logs​

Useful Log Filters​

3. Shell Access​

4. Health Checks​

Make Command​

Manual Health Checks​

Docker Health Status​

5. CTMS Init & Provisioning​

Docker-Based (Recommended)​

Make-Based (Python Scripts)​

6. Data Pipeline (Lakehouse)​

Docker Commands​

Purge Lakehouse Schemas (Reset)​

7. Vendor Stacks (Self-Hosted)​

Supabase​

Frappe​

Frappe Migration Tools​

8. Web Application (hb-life-science-web)​

Development​

Vercel Deployment​

API Generation​

9. Common Workflows​

Full Platform Start (On-Prem)​

Daily Data Refresh​

Full Platform Stop (On-Prem, Reverse Order)​

10. URLs Reference​

After make up (Core)​

After make up-obs (+ Observability)​

After make up-analytics (+ Analytics)​

Vendor Stacks (Self-Hosted)​

11. Environment Files​

Getting Help​