Two issues spotted by ux-frontend consuming docs/api.md against the actual
code path:
1. `flask.abort(...)` returned the Werkzeug HTML error page for 400/403/404/
422/etc. — only the 401 paths going through `api_error()` and the
Flask-Login `unauthorized_handler` honoured the `{error, message}`
envelope the contract promised. The frontend's `ApiClientError.body`
parser was forced to fall back to a raw string, and the 422 case
could not surface Pydantic per-field errors.
Fix: register `@app.errorhandler(HTTPException)` that serialises every
`HTTPException` to the same JSON envelope. 422s gain a `details: [...]`
field holding the Pydantic `errors()` list (`loc` / `msg` / `type`),
matching the shape now documented in `docs/api.md`.
A `_HTTP_ERROR_CODES` map maps statuses to stable snake_case codes
(`bad_request`, `not_found`, `method_not_allowed`,
`validation_error`, `forbidden`, `internal_error`, ...). Unknown
statuses fall back to `http_error`.
`description` is `cast(object, ...)` because the Werkzeug stub pins it
to `str | None` while `flask.abort(..., description=<list>)` is the
officially supported way to smuggle a Pydantic errors list to the
handler.
2. `@bp.get("")` on the engagements blueprint produced `/api/v1/engagements`
(no slash). Hitting it with a trailing slash issued a 308 redirect,
and some browsers drop the session cookie across that hop.
Fix: `app.url_map.strict_slashes = False`. Both forms now match the
same handler without redirect.
5 new integration tests cover the new envelope shape (422 with details,
unknown 404, malformed-JSON 400) and the dual-slash matching. `docs/api.md`
rewritten to reflect the table of stable codes, the `details` shape, and
the no-trailing-slash convention. `CHANGELOG.md` gains a follow-up entry.
Verification: ruff check / mypy --strict / pytest tests/unit all green
(61 unit + 5 new integration).
Unit:
- test_auth_schemas: LoginRequest validation (min/max bounds, extra-fields
policy) + serialize_current_user round-trip (RT lead permission set,
RT operator subset, display_name None pass-through).
Integration (testcontainers Postgres, marked `integration`):
- test_login_then_create_and_list_engagement: full sprint-1 user journey —
/me → 401, POST /login → 200, /me → 200, POST /engagements → 201,
GET /engagements lists the new row, POST /logout → 204, /me → 401.
- test_login_rejects_bad_credentials: wrong password AND unknown user
return the exact same 401 invalid_credentials envelope (no enumeration
leak).
- test_logout_without_session_returns_401: /logout on anonymous returns
the uniform not_authenticated envelope.
Unit total: 61 passed in 0.50s. Integration tests skip locally when
testcontainers is absent.
- `tasks/todo.md`: B0.5 description updated (re2 hard dep, no fallback);
add a "Backend follow-ups (sprint 1+)" section with M1-M7 + N1-N6 from
the code-review verdict.
- `CHANGELOG.md`: backend skeleton bullets refreshed (no re fallback,
streaming blob store, audit + scope on CRUD, 56 unit tests); new
"Code-review remediation" subsection lists B1 / MA1-MA6 / N4 / N6 / M8
with one-line rationale each.
- `tests/integration/conftest.py`: leave `db.create_all()` in place but
add an inline TODO (N6) pointing at the Alembic switchover that will
exercise the F11 seed + audit-log role grants in CI.
Code-review MAJOR MA3. The initial Alembic migration imported the live
`mimic.rbac.matrix.GROUP_PERMISSIONS` to seed the `permission` / `group` /
`group_permission` rows. That breaks the Alembic invariant "a migration
produces the same schema regardless of when you replay it": a future tweak
to the runtime matrix would silently change the seeded baseline on a fresh
DB.
Two changes:
1. The migration now carries an *inline frozen snapshot* of the F11 matrix
(`_PERMISSIONS_FROZEN`, `_GROUP_PERMISSIONS_FROZEN`, `_GROUP_DESCRIPTIONS`).
The seed reads from these tuples/dicts only. If the canonical matrix
evolves, the next migration is responsible for the delta.
2. A new unit test `test_migration_seed_matches_current_matrix` enforces
that the frozen seed equals the runtime `Permission` enum and
`GROUP_PERMISSIONS` mapping. Drift now fails CI loudly with a hint to
write a new migration instead of editing the existing one.
Also: docstring no longer mentions `ttp_version` (M8 follow-up).
Code-review MAJOR MA2. The previous `store_blob(root, data: bytes)` signature
forced the entire payload into RAM before the 10 MB cap was checked — a
hostile-large output blob could OOM the worker before the limit even fired.
New signature: `store_blob(root, stream, *, max_bytes=10_485_760)`. The
implementation:
- reads from `stream` in 64 KB chunks;
- updates the sha256 + writes to `<root>/.tmp-<pid>-<rand>.gz` incrementally;
- raises `BlobTooLarge(max_bytes)` as soon as the running total crosses the
cap, then unlinks the partial temp file via `contextlib.suppress`;
- atomic-renames the temp file to the CAS path `<aa>/<bb>/<sha256>.gz` once
the stream finishes;
- sets `0o750` on the directory and `0o640` on the file with explicit
`os.chmod` (does not rely on the process umask).
Updated unit tests cover: BlobTooLarge enforcement (with temp-file cleanup),
multi-chunk happy path (1.5 MB payload exercising the 64 KB loop), and
`max_bytes <= 0` validation.
Code-review BLOCKER B1. Reaffirms D-011: a `re` stdlib fallback defeats the
OPSEC-safe-regex guarantee because hostile C2 output can trigger catastrophic
backtracking. The `[:1MB]` slice cap does not mitigate that — re-evaluating
a malicious pattern over 1 MB of attacker-controlled text is still a worker
freeze.
- `mimic.templating.filters` now imports `re2` unconditionally and raises
`RuntimeError` at module load if the binding is absent. No `re` import,
no `_HAS_RE2` branch, no `_FALLBACK_MAX_INPUT`.
- `pyproject.toml` already pinned `google-re2 >= 1.1, < 2.0`; this commit
hardens the import path to actually enforce it.
- New test `test_re2_is_required` asserts the binding is wired in.
Pre-merge sanity per devops checklist (ruff format --check, mypy --strict).
Type fixes:
- ORM models: `Mapped[dict]` → `Mapped[dict[str, Any]]` (audit, scenario, run,
report, ttp, detection.artifact_files_json). Equivalent on Pydantic DTOs
(TtpBase.params_schema_json, ScenarioStepBase.params_override_json).
- Rename `TtpRead.current_version` → `TtpRead.version` to mirror the ORM
column (which itself was renamed in D-009 cleanup).
- Flask blueprints: add `-> ResponseReturnValue` to every view, plus typed
UUID params on `_validate_step_consistency`.
- `templating/filters.py`: rewrite the conditional re2 import so mypy can
narrow the union (`ModuleType | None`); the runtime branch on `_re2 is not
None` removes the unused-ignore that was triggered by warn_unused_ignores.
- `pyproject.toml`: add `flask_login.*` and `pythonjsonlogger.*` to the
`[[tool.mypy.overrides]]` `ignore_missing_imports` list (both ship without
typed marker).
- Misc: drop stale `# type: ignore` comments (`app.py:36`,
`rbac/decorators.py:35`) flagged by `warn_unused_ignores`. Keep
`logging.JsonFormatter` ignore because the symbol exists at runtime but is
not re-exported through the typed surface.
Formatting:
- `ruff format` applied (15 files normalized; line-length unchanged at 100).
Verification on this commit:
- `ruff check` → All checks passed.
- `ruff format --check` → 68 files already formatted.
- `mypy --strict src` → Success: no issues found in 54 source files.
- `pytest tests/unit` → 49 passed.
D-011 — `regex_extract(text, pattern, *, group=1, name=None)`:
- engine google-re2 (linear-time, ReDoS-safe), `re` fallback with 1 MB cap.
- first match only.
- no match → raises Jinja2 `TemplateError` (no silent default — cleanup
templates must fail loud when source string drifts).
- default capture is group 1 with fallback to group(0) when the pattern has
no groups; named groups via `name="<name>"`.
D-012 — `outputs.blob()`:
- reads the gzip-compressed CAS file from `MIMIC_BLOB_ROOT`.
- 10 MB cap is applied **after** decompression.
- decode UTF-8 with latin-1 fallback; never raises (missing / corrupt /
non-gzip blobs return empty string, logged at WARNING).
Unit tests rewritten to cover both the new fail-loud regex contract and
the gzip read path. 49 unit tests pass; ruff clean.
