Merge branch 'feature/backend-skeleton' into main

Sprint 0 backend skeleton: Python 3.12 / Flask / SQLAlchemy 2 / Pydantic 2
/ Alembic / pytest. Data model §8 complete, C2Connector ABC, Jinja2
sandbox with google-re2 regex_extract (D-011), CAS gzip blob storage
(D-012), local auth + group-based RBAC (D-003/D-008), F11 tenant scoping,
audit log infrastructure with hash chain anticipated (D-013).

Quality gates: ruff/mypy strict/56 unit tests passing. LGTM code-reviewer
after 2 rounds of remediation (B1 BLOCKER + 6 MAJOR addressed).

Co-Authored-By: backend <backend@mimic.local>

* origin/feature/backend-skeleton:
  docs(backend): track sprint-0 follow-ups + flag integration migration gap
  feat(backend): wire created_by_id, audit log, F11 scope into CRUD (MA4/5/6)
  fix(backend): freeze F11 matrix inline in the initial migration (MA3)
  fix(backend): stream store_blob and enforce max_bytes mid-write (MA2)
  fix(backend): stop seeding the audit-writer role via postgres-init (MA1)
  fix(backend): make google-re2 a hard dependency, drop re fallback (B1)
  chore(backend): mypy strict clean + ruff format pass
  feat(backend): add content-addressed gzip blob store (D-012)
  fix(backend): align regex_extract + outputs.blob() with D-011/D-012
  fix(backend): align with D-008/D-009 (drop ttp_version, seed F11 matrix)
  docs: update CHANGELOG + tasks for the backend skeleton sprint 0
  test(backend): add pytest baseline (B0.8)
  feat(backend): add Flask app factory, audit writer, flat CRUD + CLI (B0.7)
  feat(backend): add local auth + group-based RBAC matching F11 (B0.6)
  feat(backend): add Jinja2 sandbox + regex_extract filter (B0.5)
  feat(backend): add C2Connector ABC + payload mapping + factory (B0.4)
  feat(backend): add §8 data model + Alembic baseline (B0.2, B0.3)
  chore(backend): bootstrap Python 3.12+ project skeleton (B0.1)
This commit is contained in:
knacky
2026-05-22 11:45:17 +02:00
80 changed files with 5121 additions and 26 deletions

View File

@@ -2,24 +2,67 @@
Repo skeleton + foundational modules. Nothing that depends on PR1/PR2/PR3.
## Backend (`backend`)
## Backend (`backend`) — done in `feature/backend-skeleton`
- [ ] B0.1 — `backend/` Python project: `pyproject.toml` (ruff, mypy strict, pytest, coverage),
`Makefile`, `Dockerfile`, `docker-compose.yml` for Postgres dev DB.
- [ ] B0.2 — Alembic init + complete initial migration covering the §8 schema (incl.
`c2_credential`, `user`, `group`, `user_group`, `permission`, `group_permission`,
`soc_session`, audit_log with write-only Postgres role). **No `ttp_version` table** (D-009).
Seed groups `rt_operator`, `rt_lead`, `soc_analyst` with F11 permissions (D-008).
- [ ] B0.3 — SQLAlchemy 2 typed mapped classes for every table + repositories scaffold.
- [ ] B0.4`C2Connector` ABC + dataclasses (`Payload`, `TaskHandle`, `TaskResult`) + enum
`payload_type` + factory keyed on `c2_type`. **No real implementation.**
- [ ] B0.5 — Jinja2 SandboxedEnvironment + `regex_extract` filter via `google-re2` +
`{{outputs.text}}` and `{{outputs.blob(key)}}` accessors with 10 MB cap.
- [ ] B0.6 — Local auth (login/password bcrypt + Flask server-side sessions) + RBAC
group-based decorators + F11 permission matrix declared in code.
- [ ] B0.7 — Flat CRUD endpoints (engagements, hosts, TTPs, scenarios) — no orchestration,
no WebSocket, no reporting yet.
- [ ] B0.8 — pytest baseline: unit (SQLite) + integration scaffold (testcontainers Postgres).
- [x] B0.1 — `backend/` Python 3.12+ project: `pyproject.toml` (ruff, mypy strict, pytest,
coverage 70 %), `Makefile` (Docker/Podman auto), multi-stage `Dockerfile`,
`docker-compose.yml` for Postgres dev DB, `.env.example`.
- [x] B0.2 — Alembic baseline migration `202605210001_initial_schema` creates every table,
enum, index, and the idempotent grants for the audit write-only Postgres role. **No
`ttp_version` table** (D-009). Groups `rt_operator`, `rt_lead`, `soc_analyst` seeded
with the exact F11 permission matrix (D-008).
- [x] B0.3SQLAlchemy 2 typed mapped classes for every spec §8 aggregate (engagement,
host, user/group RBAC, ttp, scenario/scenario_step, run/run_step/cleanup, detection,
evidence, report, soc_session, c2_credential, audit_log).
- [x] B0.4 — `C2Connector` ABC + dataclasses + `payload_type` enum + factory keyed on
`c2_type`. Mythic payload map populated; Home stays empty until PR2.
- [x] B0.5 — Jinja2 SandboxedEnvironment, `regex_extract` filter (`google-re2` hard
dependency per D-011, raises `RuntimeError` at boot if absent — no `re` fallback),
fail-loud no-match, `{{ outputs.text }}` / `{{ outputs.blob() }}` accessors
reading gzip-compressed blobs with 10 MB cap.
- [x] B0.6 — bcrypt password helpers + SOC opaque token (256-bit url-safe, bcrypt-hashed) +
group-based RBAC matrix matching F11 + `@require_perm` decorator.
- [x] B0.7 — Flat CRUD blueprints for engagements / hosts / TTPs / scenarios (incl. step
composition with F3 invariant `host.c2_type == scenario.c2_type`).
- [x] B0.8 — pytest baseline: unit tests passing, integration scaffold ready
(testcontainers Postgres + `/healthz` smoke).
## Backend follow-ups (sprint 1+)
Tracked from code-review verdict on `feature/backend-skeleton` @ 12d131c:
### MINOR (8) — to schedule
- **M1** — Replace `parse_uuid` integer-ish lookup with `werkzeug` UUID converter on
the routes (`<uuid:eid>`) to avoid the 404 on malformed strings being hidden by
the 400 path.
- **M2** — Add OpenAPI generation (Pydantic 2 + `flask-pydantic-openapi` or hand-rolled).
- **M3** — Wire `flask-limiter` for `/auth/local/login` (NF-network).
- **M4** — Replace string-based `Engagement.status` setter with a typed transition method.
- **M5** — Introduce per-engagement read view that pre-joins `engagement_member` for
RT operator dashboards (current per-request join is fine for v1 traffic, but
re-evaluate at scale).
- **M6** — `mimic-cli user create` does not handle the SOC user-type (intended, but
document and gate explicitly with a clean error message).
- **M7** — Add a `mimic-cli` `engagement add-member <uid> --role rt_operator` shortcut so
the F11 scoping in MA6 is reachable from the CLI without manual SQL.
- **M8** — _(fixed in MA1 follow-up commit)_ Initial migration docstring no longer
references `ttp_version`.
### NIT (6) — opportunistic
- **N1** — Sort imports inside `mimic.db.models.__init__` alphabetically for diff
stability.
- **N2** — Extract the `_engagement_or_404` duplicated body into a shared helper.
- **N3** — Replace the inline `Permission.TTP_PROMOTE not in perms` check in `ttps.py`
with a second `@require_perm`-style decorator.
- **N4** — _(fixed)_ `gunicorn` added to `pyproject.toml` dependencies.
- **N5** — Replace bare `getattr(current_user, "groups", frozenset())` accesses by a
thin `current_groups()` helper.
- **N6** — `tests/integration/conftest.py` uses `db.create_all()` instead of running
Alembic. Marked with a TODO; switch over once the F11 seed must be exercised in
integration. Plan: convert to `alembic upgrade head` once the audit role
bootstrap lives in the playbook (D-010).
## Frontend (`ux-frontend`)