Operational runbook for rolling Mimic to RT infrastructure. Scope is
the application repo only; the Ansible playbook (D-010) and Caddy
reverse proxy (D-007) are referenced as out-of-scope dependencies.
Sections:
- Host prerequisites (Podman 5, rootless, linger, PostgreSQL 16 reach).
- Filesystem layout: blobs + evidence pools at 0750 under the deploy
user (D-012), log directory, Quadlet directory.
- Environment variables: split into "required in prod" (MIMIC_SECRET_KEY,
MIMIC_FERNET_KEY, MIMIC_DATABASE_URL, MIMIC_DATABASE_AUDIT_URL,
MIMIC_ENV) and "required with safe defaults" (cookie flags, log
format, CORS origins, blob/evidence roots). Explicit note that the
two database DSNs must point to two different Postgres roles to
preserve the audit append-only contract (NF-AUDIT, code-reviewer N5).
- Secrets management: dedicated section addressing PR3 code-reviewer M2.
File-based generation under ~/secrets with 0700 perms, systemd
EnvironmentFile or future MIMIC_*_FILE indirection, vault back-up,
Fernet key rotation requires re-encryption pass.
- Container images: pin policy `:X.Y.Z` (cross-references F-D1), exposed
ports per layer (backend 5000 as uid 1001, frontend 8080 as uid 101).
- PostgreSQL setup: bootstrap of mimic_audit_writer role with the SQL
the Ansible playbook runs, plus the fail-loud rationale if the role
is missing. Alembic upgrade head invocation.
- Quadlet units: backend example with PublishPort 127.0.0.1:5000 (the
external surface is Caddy, not the backend), EnvironmentFile,
blob+evidence bind-mounts with `:Z` SELinux relabel.
- Smoke validation: three curl checks (Caddy-fronted /healthz, direct
backend /healthz, audit DSN presence) with explicit "do not announce
the release" gate on failure.
- Upgrade procedure: 5-step rolling restart anchored on Quadlet image
tag edits + alembic upgrade as part of the entrypoint.
- Rollback procedure: image-only (additive schema) vs schema-affecting,
with alembic downgrade against an explicit revision.
- Open items: explicit pointers to FERNET-KEY, F-D1, F-D2, F-D3
trackers in tasks/todo.md so future operators see them.
No other file touched; no application code changed.
