Merge branch 'feature/backend-skeleton' into main
Sprint 0 backend skeleton: Python 3.12 / Flask / SQLAlchemy 2 / Pydantic 2 / Alembic / pytest. Data model §8 complete, C2Connector ABC, Jinja2 sandbox with google-re2 regex_extract (D-011), CAS gzip blob storage (D-012), local auth + group-based RBAC (D-003/D-008), F11 tenant scoping, audit log infrastructure with hash chain anticipated (D-013). Quality gates: ruff/mypy strict/56 unit tests passing. LGTM code-reviewer after 2 rounds of remediation (B1 BLOCKER + 6 MAJOR addressed). Co-Authored-By: backend <backend@mimic.local> * origin/feature/backend-skeleton: docs(backend): track sprint-0 follow-ups + flag integration migration gap feat(backend): wire created_by_id, audit log, F11 scope into CRUD (MA4/5/6) fix(backend): freeze F11 matrix inline in the initial migration (MA3) fix(backend): stream store_blob and enforce max_bytes mid-write (MA2) fix(backend): stop seeding the audit-writer role via postgres-init (MA1) fix(backend): make google-re2 a hard dependency, drop re fallback (B1) chore(backend): mypy strict clean + ruff format pass feat(backend): add content-addressed gzip blob store (D-012) fix(backend): align regex_extract + outputs.blob() with D-011/D-012 fix(backend): align with D-008/D-009 (drop ttp_version, seed F11 matrix) docs: update CHANGELOG + tasks for the backend skeleton sprint 0 test(backend): add pytest baseline (B0.8) feat(backend): add Flask app factory, audit writer, flat CRUD + CLI (B0.7) feat(backend): add local auth + group-based RBAC matching F11 (B0.6) feat(backend): add Jinja2 sandbox + regex_extract filter (B0.5) feat(backend): add C2Connector ABC + payload mapping + factory (B0.4) feat(backend): add §8 data model + Alembic baseline (B0.2, B0.3) chore(backend): bootstrap Python 3.12+ project skeleton (B0.1)
This commit is contained in:
80
CHANGELOG.md
80
CHANGELOG.md
@@ -24,3 +24,83 @@ Versioning starts at `0.1.0` when sprint 0 lands.
|
||||
|
||||
Repo skeleton, data model, `C2Connector` ABC, Jinja2 sandbox, local auth + RBAC, flat CRUD,
|
||||
UX wireframes (mock data). No real connector, no reporting until PR1/PR2/PR3 land.
|
||||
|
||||
#### Backend skeleton (`feature/backend-skeleton`)
|
||||
|
||||
- `backend/` Python 3.12+ project: `pyproject.toml` (ruff, mypy strict, pytest, coverage 70 %),
|
||||
`Makefile` (Docker/Podman auto-detect), multi-stage `Dockerfile`, `docker-compose.yml` for
|
||||
Postgres dev DB, `.env.example`.
|
||||
- Full §8 data model in SQLAlchemy 2 typed mapped classes: `engagement`, `c2_credential`,
|
||||
`host`, `user`, `group`, `permission`, `group_permission`, `user_group`,
|
||||
`engagement_member`, `ttp`, `scenario`, `scenario_step`, `run`, `run_step`,
|
||||
`run_step_cleanup`, `detection`, `evidence`, `report`, `soc_session`, `audit_log`.
|
||||
No `ttp_version` table (D-009 / H32 reaffirmed).
|
||||
- Alembic baseline migration `202605210001_initial_schema`: every table + enum + index +
|
||||
idempotent `audit_log` grants for the write-only Postgres role. Seeds the three F11
|
||||
groups (`rt_operator`, `rt_lead`, `soc_analyst`) and their permission set (D-008).
|
||||
- `C2Connector` ABC + `Payload` / `TaskHandle` / `TaskResult` / `TaskStatus` dataclasses +
|
||||
`PayloadType` enum + `ConnectorFactory` keyed on `c2_type`. Mythic payload map populated;
|
||||
Home stays empty until PR2.
|
||||
- Jinja2 `SandboxedEnvironment` + `regex_extract` filter (`google-re2` hard dependency per
|
||||
D-011 / B1 — `RuntimeError` at boot if absent, no `re` fallback) + `{{ outputs.text }}` /
|
||||
`{{ outputs.blob() }}` accessors reading gzip-compressed blobs (10 MB cap after
|
||||
decompression, UTF-8 → latin-1).
|
||||
- Group-based RBAC: `Permission` + `GroupName` + `GROUP_PERMISSIONS` mirror the F11 matrix;
|
||||
`@require_perm` decorator + `AuthUser` Flask-Login wrapper that resolves the permission set
|
||||
from the user's groups.
|
||||
- bcrypt password helpers + SOC opaque token (256-bit url-safe, bcrypt-hashed at rest, plain
|
||||
returned once).
|
||||
- Hash-chained append-only audit writer (sprint 0 fills `prev_hash` / `row_hash` at insert;
|
||||
verifier shipped in v2).
|
||||
- Flat CRUD blueprints: engagements / hosts / TTPs / scenarios + scenario steps. F3 invariant
|
||||
enforced (host.c2_type must match scenario.c2_type at compose time). Every mutation calls
|
||||
the hash-chained audit writer (MA5); created rows carry `created_by_id` (MA4); listings and
|
||||
per-engagement routes scope to `engagement_member` for RT operators (MA6 / F11).
|
||||
- Content-addressed gzip blob store (`mimic.storage.blob`): streaming write with a `max_bytes`
|
||||
cap (raises `BlobTooLarge` mid-stream — MA2), atomic rename, `0o750` directory mode.
|
||||
- `mimic-cli` (click): `user create`, `db dump`, `db restore`.
|
||||
- pytest baseline: **56 unit tests passing** (templating, regex_extract, password, soc_token,
|
||||
RBAC matrix, connector factory, audit hash, blob CAS, migration seed parity). Integration
|
||||
scaffold ready for testcontainers Postgres (`/healthz` smoke included).
|
||||
|
||||
#### Spec deltas applied in this sprint
|
||||
|
||||
Authoritative decisions implemented per `tasks/spec-decisions.md`:
|
||||
- **D-008** — Seeded groups = exactly the three F11 roles, permission matrix from F11.
|
||||
- **D-009** — No `ttp_version` table (H32 reaffirmed).
|
||||
- **D-011** — `regex_extract` fails loudly on no-match (raises `TemplateError`).
|
||||
- **D-012** — `output_blob_ref` stored in `MIMIC_BLOB_ROOT` (CAS gzip layout); evidence
|
||||
files live under `MIMIC_EVIDENCE_ROOT` (flat per-engagement).
|
||||
|
||||
Implementation arbitrations logged in this sprint:
|
||||
- **D-013** — `audit_log` hash chain (`prev_hash` / `row_hash`) shipped v1.
|
||||
- **D-014** — UUID columns use SQLAlchemy 2 native `Uuid` mapping; no `type_annotation_map`
|
||||
on the declarative base (Flask-SQLAlchemy incompatibility).
|
||||
|
||||
#### Code-review remediation (`12d131c` → `feature/backend-skeleton`)
|
||||
|
||||
- **B1** — Dropped the `re` stdlib fallback in `regex_extract`. `google-re2` is now a hard
|
||||
dependency (B1 / D-011); the module raises `RuntimeError` at import if absent.
|
||||
- **MA1** — Removed `scripts/postgres-init/00-roles.sql` (no more hardcoded `CHANGE_ME`
|
||||
password). Audit-writer role provisioning is the playbook's responsibility (D-010);
|
||||
`backend/README.md` documents the manual dev-only `CREATE ROLE` command.
|
||||
- **MA2** — `store_blob` now accepts a binary stream + `max_bytes`, streams sha256+gzip in
|
||||
64 KB chunks, and raises `BlobTooLarge` mid-stream (cleans up the temp file). No more
|
||||
whole-buffer RAM load.
|
||||
- **MA3** — Inlined the F11 permission matrix in the initial Alembic migration; the runtime
|
||||
matrix is no longer imported there. A new unit test
|
||||
(`test_migration_seed_matches_current_matrix`) fails if the two drift apart.
|
||||
- **MA4** — `created_by_id = current_user.id` set in `engagement`, `ttp`, and `scenario`
|
||||
create endpoints.
|
||||
- **MA5** — Every mutation endpoint now writes an audit row through the hash-chained
|
||||
`AuditWriter` (F13).
|
||||
- **MA6** — RT operators only see engagements they are members of (`engagement_member` join
|
||||
on list, membership probe on `get`/`put`/`delete`/`host`/`scenario`/...). RT leads bypass.
|
||||
- **N4** — `gunicorn` declared in `pyproject.toml` dependencies (the Dockerfile `CMD` now
|
||||
resolves correctly).
|
||||
- **N6** — `tests/integration/conftest.py` keeps `db.create_all()` for now; commented TODO to
|
||||
switch over to Alembic once the playbook owns the audit role.
|
||||
- **M8** — Initial migration docstring no longer mentions `ttp_version`.
|
||||
|
||||
Verification on the latest commit: `ruff check`, `ruff format --check`, `mypy --strict`, and
|
||||
`pytest tests/unit` all pass; 56 unit tests green.
|
||||
|
||||
27
backend/.env.example
Normal file
27
backend/.env.example
Normal file
@@ -0,0 +1,27 @@
|
||||
# Mimic backend — example env. Copy to .env (gitignored) and adapt.
|
||||
|
||||
MIMIC_ENV=development
|
||||
MIMIC_SECRET_KEY=replace-me-with-secrets.token_urlsafe-32
|
||||
MIMIC_FERNET_KEY=
|
||||
|
||||
# Database
|
||||
POSTGRES_DB=mimic
|
||||
POSTGRES_USER=mimic_app
|
||||
POSTGRES_PASSWORD=mimic_dev_password
|
||||
MIMIC_DATABASE_URL=postgresql+psycopg://mimic_app:mimic_dev_password@localhost:5432/mimic
|
||||
MIMIC_DATABASE_AUDIT_URL=postgresql+psycopg://mimic_audit_writer:CHANGE_ME@localhost:5432/mimic
|
||||
|
||||
# Session / cookies
|
||||
MIMIC_SESSION_COOKIE_SECURE=false
|
||||
MIMIC_SESSION_COOKIE_SAMESITE=Lax
|
||||
|
||||
# CORS (frontend dev)
|
||||
MIMIC_CORS_ORIGINS=http://localhost:5173
|
||||
|
||||
# Logging
|
||||
MIMIC_LOG_LEVEL=DEBUG
|
||||
MIMIC_LOG_JSON=false
|
||||
|
||||
# Storage pools (D-012)
|
||||
MIMIC_BLOB_ROOT=/var/lib/mimic/blobs
|
||||
MIMIC_EVIDENCE_ROOT=/var/lib/mimic/evidence
|
||||
60
backend/Dockerfile
Normal file
60
backend/Dockerfile
Normal file
@@ -0,0 +1,60 @@
|
||||
# syntax=docker/dockerfile:1.7
|
||||
|
||||
# --- Stage 1: build --------------------------------------------------------
|
||||
FROM python:3.12-slim-bookworm AS build
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
||||
PIP_NO_CACHE_DIR=1
|
||||
|
||||
# WeasyPrint native deps + libpq + build tools.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
build-essential \
|
||||
libpq-dev \
|
||||
libpango-1.0-0 \
|
||||
libpangoft2-1.0-0 \
|
||||
libcairo2 \
|
||||
libgdk-pixbuf-2.0-0 \
|
||||
libffi-dev \
|
||||
shared-mime-info \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /build
|
||||
COPY pyproject.toml README.md ./
|
||||
COPY src ./src
|
||||
|
||||
RUN pip install --upgrade pip wheel build \
|
||||
&& pip wheel --wheel-dir /wheels --no-deps .
|
||||
|
||||
RUN pip install --prefix=/install --no-warn-script-location .
|
||||
|
||||
# --- Stage 2: runtime ------------------------------------------------------
|
||||
FROM python:3.12-slim-bookworm AS runtime
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
FLASK_APP=mimic.app:create_app \
|
||||
MIMIC_ENV=production
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
libpq5 \
|
||||
libpango-1.0-0 \
|
||||
libpangoft2-1.0-0 \
|
||||
libcairo2 \
|
||||
libgdk-pixbuf-2.0-0 \
|
||||
shared-mime-info \
|
||||
tini \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& groupadd --system --gid 1001 mimic \
|
||||
&& useradd --system --uid 1001 --gid 1001 --home-dir /app --shell /usr/sbin/nologin mimic
|
||||
|
||||
WORKDIR /app
|
||||
COPY --from=build /install /usr/local
|
||||
COPY --chown=mimic:mimic src ./src
|
||||
|
||||
USER mimic
|
||||
EXPOSE 5000
|
||||
|
||||
ENTRYPOINT ["/usr/bin/tini", "--"]
|
||||
CMD ["gunicorn", "--worker-class", "gevent", "--workers", "1", "--bind", "0.0.0.0:5000", "mimic.app:create_app()"]
|
||||
80
backend/Makefile
Normal file
80
backend/Makefile
Normal file
@@ -0,0 +1,80 @@
|
||||
# --- Mimic backend Makefile ------------------------------------------------
|
||||
# Detects Docker / Podman automatically (NF-platform).
|
||||
|
||||
PY ?= python3.12
|
||||
VENV ?= .venv
|
||||
PIP := $(VENV)/bin/pip
|
||||
PYTHON := $(VENV)/bin/python
|
||||
PYTEST := $(VENV)/bin/pytest
|
||||
RUFF := $(VENV)/bin/ruff
|
||||
MYPY := $(VENV)/bin/mypy
|
||||
ALEMBIC := $(VENV)/bin/alembic
|
||||
FLASK := $(VENV)/bin/flask
|
||||
|
||||
# Container runtime auto-detect: Docker (preferred) or Podman rootless.
|
||||
CONTAINER ?= $(shell command -v docker 2>/dev/null || command -v podman 2>/dev/null)
|
||||
COMPOSE ?= $(shell command -v docker-compose 2>/dev/null || echo "$(CONTAINER) compose")
|
||||
|
||||
export FLASK_APP=mimic.app:create_app
|
||||
|
||||
.PHONY: help install lint fmt typecheck test test-int test-cov run \
|
||||
db-up db-down db-migrate db-revision db-seed db-dump db-restore \
|
||||
build clean
|
||||
|
||||
help:
|
||||
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | \
|
||||
awk 'BEGIN {FS=":.*?## "}; {printf " \033[36m%-20s\033[0m %s\n", $$1, $$2}'
|
||||
|
||||
install: ## Create venv and install dev dependencies
|
||||
$(PY) -m venv $(VENV)
|
||||
$(PIP) install --upgrade pip wheel
|
||||
$(PIP) install -e ".[dev]"
|
||||
|
||||
lint: ## Ruff lint
|
||||
$(RUFF) check src tests
|
||||
|
||||
fmt: ## Ruff format
|
||||
$(RUFF) format src tests
|
||||
$(RUFF) check --fix src tests
|
||||
|
||||
typecheck: ## Mypy strict
|
||||
$(MYPY) src
|
||||
|
||||
test: ## Unit tests (SQLite)
|
||||
$(PYTEST) tests/unit -v
|
||||
|
||||
test-int: ## Integration tests (testcontainers Postgres)
|
||||
$(PYTEST) tests/integration -v -m integration
|
||||
|
||||
test-cov: ## Unit + integration with coverage report
|
||||
$(PYTEST) --cov=mimic --cov-report=term-missing --cov-report=html
|
||||
|
||||
run: ## Run Flask dev server
|
||||
$(FLASK) run --host 0.0.0.0 --port 5000 --debug
|
||||
|
||||
db-up: ## Start Postgres dev container
|
||||
$(COMPOSE) up -d postgres
|
||||
|
||||
db-down: ## Stop Postgres dev container
|
||||
$(COMPOSE) down
|
||||
|
||||
db-migrate: ## Apply migrations
|
||||
$(ALEMBIC) upgrade head
|
||||
|
||||
db-revision: ## Generate migration (msg=...)
|
||||
$(ALEMBIC) revision --autogenerate -m "$(msg)"
|
||||
|
||||
db-seed: ## Seed local dev data (TBD sprint 1)
|
||||
@echo "TBD sprint 1"
|
||||
|
||||
db-dump: ## Manual DB dump (NF-state, R-O1)
|
||||
$(VENV)/bin/mimic-cli db dump --out backups/mimic-$$(date +%Y%m%d-%H%M%S).sql
|
||||
|
||||
db-restore: ## Restore from dump (file=...)
|
||||
$(VENV)/bin/mimic-cli db restore --file $(file)
|
||||
|
||||
build: ## Build wheel
|
||||
$(PYTHON) -m build
|
||||
|
||||
clean:
|
||||
rm -rf $(VENV) .pytest_cache .mypy_cache .ruff_cache htmlcov dist build *.egg-info
|
||||
71
backend/README.md
Normal file
71
backend/README.md
Normal file
@@ -0,0 +1,71 @@
|
||||
# Mimic — backend
|
||||
|
||||
Sprint 0 skeleton. Python 3.12+ / Flask / SQLAlchemy 2 / Alembic / Pydantic 2.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
backend/
|
||||
├── src/mimic/
|
||||
│ ├── app.py # Flask app factory + SocketIO init
|
||||
│ ├── config.py # Pydantic Settings
|
||||
│ ├── extensions.py # db, migrate, socketio, login_manager
|
||||
│ ├── db/
|
||||
│ │ ├── models/ # SQLAlchemy 2 typed models
|
||||
│ │ ├── repositories/ # data access per aggregate
|
||||
│ │ └── migrations/ # Alembic
|
||||
│ ├── schemas/ # Pydantic 2 DTOs
|
||||
│ ├── api/ # Flask blueprints (REST)
|
||||
│ ├── ws/ # Flask-SocketIO namespaces
|
||||
│ ├── connectors/ # C2Connector ABC + payload mapping
|
||||
│ ├── orchestrator/ # run state machine (stub in sprint 0)
|
||||
│ ├── templating/ # Jinja2 sandbox + regex_extract
|
||||
│ ├── audit/ # append-only writer + rotation
|
||||
│ ├── reporting/ # WeasyPrint builder (stub in sprint 0)
|
||||
│ ├── rbac/ # group-based permission matrix (F11)
|
||||
│ ├── importers/ # ATR + C2 journal (stub in sprint 0)
|
||||
│ └── cli/ # mimic-cli (click)
|
||||
└── tests/
|
||||
├── unit/ # SQLite, pure logic
|
||||
└── integration/ # testcontainers Postgres
|
||||
```
|
||||
|
||||
## Local dev
|
||||
|
||||
```bash
|
||||
make install # uv venv + pip install -e .[dev]
|
||||
make db-up # docker compose up -d postgres
|
||||
make db-bootstrap # one-time: create the mimic_audit_writer role (see below)
|
||||
make db-migrate # alembic upgrade head
|
||||
make run # flask run (debug)
|
||||
make test # pytest unit
|
||||
make test-int # pytest integration (testcontainers)
|
||||
make lint # ruff + mypy strict
|
||||
```
|
||||
|
||||
### Audit writer role (dev)
|
||||
|
||||
`mimic_audit_writer` is provisioned by the Ansible playbook in production
|
||||
(decision D-010). For local development, create it manually after `make db-up`:
|
||||
|
||||
```bash
|
||||
docker exec -it mimic-postgres psql -U mimic_app -d mimic \
|
||||
-c "CREATE ROLE mimic_audit_writer LOGIN PASSWORD 'pick-a-dev-secret';"
|
||||
```
|
||||
|
||||
Then expose the same secret in `MIMIC_DATABASE_AUDIT_URL` in your `.env`. The
|
||||
Alembic migration grants the INSERT-only permission on `audit_log` against
|
||||
this role; if it does not exist, the grant block is a no-op (idempotent).
|
||||
|
||||
## What sprint 0 ships
|
||||
|
||||
- Full §8 data model + Alembic initial migration (Postgres-specific constraints: audit_log write-only role, soc_session hash, c2_credential Fernet column).
|
||||
- `C2Connector` ABC + dataclasses + `payload_type` enum + factory. **No real Mythic/Home implementation** (blocked on PR1/PR2).
|
||||
- Jinja2 SandboxedEnvironment + `regex_extract` filter (re2).
|
||||
- Local auth (bcrypt + Flask session) + group-based RBAC matching the F11 permission matrix.
|
||||
- Flat CRUD on engagements / hosts / TTPs / scenarios.
|
||||
- pytest baseline + testcontainers Postgres scaffolding.
|
||||
|
||||
## Out of sprint 0
|
||||
|
||||
Orchestrator, WebSocket cockpit, real connectors, report generation, audit rotation.
|
||||
39
backend/alembic.ini
Normal file
39
backend/alembic.ini
Normal file
@@ -0,0 +1,39 @@
|
||||
[alembic]
|
||||
script_location = src/mimic/db/migrations
|
||||
prepend_sys_path = src
|
||||
version_path_separator = os
|
||||
sqlalchemy.url =
|
||||
|
||||
[loggers]
|
||||
keys = root,sqlalchemy,alembic
|
||||
|
||||
[handlers]
|
||||
keys = console
|
||||
|
||||
[formatters]
|
||||
keys = generic
|
||||
|
||||
[logger_root]
|
||||
level = WARN
|
||||
handlers = console
|
||||
qualname =
|
||||
|
||||
[logger_sqlalchemy]
|
||||
level = WARN
|
||||
handlers =
|
||||
qualname = sqlalchemy.engine
|
||||
|
||||
[logger_alembic]
|
||||
level = INFO
|
||||
handlers =
|
||||
qualname = alembic
|
||||
|
||||
[handler_console]
|
||||
class = StreamHandler
|
||||
args = (sys.stderr,)
|
||||
level = NOTSET
|
||||
formatter = generic
|
||||
|
||||
[formatter_generic]
|
||||
format = %(levelname)-5.5s [%(name)s] %(message)s
|
||||
datefmt = %H:%M:%S
|
||||
27
backend/docker-compose.yml
Normal file
27
backend/docker-compose.yml
Normal file
@@ -0,0 +1,27 @@
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
container_name: mimic-postgres
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_DB: ${POSTGRES_DB:-mimic}
|
||||
POSTGRES_USER: ${POSTGRES_USER:-mimic_app}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-mimic_dev_password}
|
||||
ports:
|
||||
- "127.0.0.1:5432:5432"
|
||||
volumes:
|
||||
- mimic_pgdata:/var/lib/postgresql/data
|
||||
# The `mimic_audit_writer` role is provisioned by the Ansible playbook
|
||||
# in prod (D-010). For dev, create it manually after `make db-up`:
|
||||
# docker exec -it mimic-postgres psql -U mimic_app -d mimic \
|
||||
# -c "CREATE ROLE mimic_audit_writer LOGIN PASSWORD '<choose one>';"
|
||||
# Then expose the same secret in MIMIC_DATABASE_AUDIT_URL in your .env.
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-mimic_app} -d ${POSTGRES_DB:-mimic}"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
|
||||
volumes:
|
||||
mimic_pgdata:
|
||||
name: mimic_pgdata
|
||||
150
backend/pyproject.toml
Normal file
150
backend/pyproject.toml
Normal file
@@ -0,0 +1,150 @@
|
||||
[build-system]
|
||||
requires = ["hatchling>=1.24"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "mimic"
|
||||
version = "0.1.0a0"
|
||||
description = "Mimic — internal BAS platform (sprint 0 skeleton)"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
license = { text = "Proprietary" }
|
||||
authors = [{ name = "RT" }]
|
||||
|
||||
dependencies = [
|
||||
"flask>=3.0,<4.0",
|
||||
"flask-socketio>=5.3,<6.0",
|
||||
"flask-login>=0.6.3,<1.0",
|
||||
"flask-migrate>=4.0,<5.0",
|
||||
"sqlalchemy>=2.0,<3.0",
|
||||
"alembic>=1.13,<2.0",
|
||||
"psycopg[binary]>=3.1,<4.0",
|
||||
"pydantic>=2.6,<3.0",
|
||||
"pydantic-settings>=2.2,<3.0",
|
||||
"python-json-logger>=2.0,<3.0",
|
||||
"structlog>=24.1,<25.0",
|
||||
"bcrypt>=4.1,<5.0",
|
||||
"cryptography>=42.0,<43.0",
|
||||
"jinja2>=3.1,<4.0",
|
||||
"google-re2>=1.1,<2.0",
|
||||
"click>=8.1,<9.0",
|
||||
"gevent>=24.2,<25.0",
|
||||
"gevent-websocket>=0.10,<1.0",
|
||||
"gunicorn>=22.0,<24.0",
|
||||
"httpx>=0.27,<1.0",
|
||||
"weasyprint>=61.0,<62.0",
|
||||
"authlib>=1.3,<2.0",
|
||||
"pyyaml>=6.0,<7.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=8.0,<9.0",
|
||||
"pytest-cov>=5.0,<6.0",
|
||||
"pytest-flask>=1.3,<2.0",
|
||||
"pytest-mock>=3.12,<4.0",
|
||||
"testcontainers[postgres]>=4.4,<5.0",
|
||||
"ruff>=0.4,<1.0",
|
||||
"mypy>=1.10,<2.0",
|
||||
"types-pyyaml>=6.0,<7.0",
|
||||
"freezegun>=1.5,<2.0",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
mimic-cli = "mimic.cli:cli"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["src/mimic"]
|
||||
|
||||
[tool.hatch.build.targets.sdist]
|
||||
include = ["src/mimic", "README.md", "pyproject.toml"]
|
||||
|
||||
# -- Ruff -------------------------------------------------------------------
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
target-version = "py312"
|
||||
src = ["src", "tests"]
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = [
|
||||
"E", "F", "W", # pycodestyle / pyflakes
|
||||
"I", # isort
|
||||
"B", # bugbear
|
||||
"UP", # pyupgrade
|
||||
"N", # pep8-naming
|
||||
"S", # flake8-bandit (security)
|
||||
"C4", # comprehensions
|
||||
"DTZ", # datetime tz
|
||||
"PIE",
|
||||
"PT", # pytest
|
||||
"RET",
|
||||
"SIM",
|
||||
"TID",
|
||||
"PL",
|
||||
"RUF",
|
||||
]
|
||||
ignore = [
|
||||
"PLR0913", # too many args (Flask handlers + DI)
|
||||
"S101", # assert in tests
|
||||
]
|
||||
|
||||
[tool.ruff.lint.per-file-ignores]
|
||||
"tests/**" = ["S101", "S105", "S106", "PLR2004"]
|
||||
"src/mimic/db/migrations/**" = ["E501", "N999"]
|
||||
|
||||
[tool.ruff.lint.isort]
|
||||
known-first-party = ["mimic"]
|
||||
|
||||
# -- Mypy -------------------------------------------------------------------
|
||||
|
||||
[tool.mypy]
|
||||
python_version = "3.12"
|
||||
strict = true
|
||||
warn_unreachable = true
|
||||
warn_unused_ignores = true
|
||||
show_error_codes = true
|
||||
plugins = ["pydantic.mypy"]
|
||||
exclude = ["src/mimic/db/migrations/versions/"]
|
||||
|
||||
[[tool.mypy.overrides]]
|
||||
module = [
|
||||
"weasyprint.*",
|
||||
"google.re2.*",
|
||||
"re2",
|
||||
"flask_socketio.*",
|
||||
"flask_migrate.*",
|
||||
"flask_login.*",
|
||||
"pythonjsonlogger.*",
|
||||
"gevent.*",
|
||||
"testcontainers.*",
|
||||
"authlib.*",
|
||||
]
|
||||
ignore_missing_imports = true
|
||||
|
||||
# -- Pytest -----------------------------------------------------------------
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
addopts = "-ra --strict-markers --strict-config"
|
||||
markers = [
|
||||
"integration: requires testcontainers Postgres",
|
||||
"slow: long-running tests",
|
||||
]
|
||||
filterwarnings = ["error"]
|
||||
|
||||
[tool.coverage.run]
|
||||
branch = true
|
||||
source = ["src/mimic"]
|
||||
omit = ["src/mimic/db/migrations/*"]
|
||||
|
||||
[tool.coverage.report]
|
||||
fail_under = 70
|
||||
show_missing = true
|
||||
skip_covered = false
|
||||
exclude_lines = [
|
||||
"pragma: no cover",
|
||||
"raise NotImplementedError",
|
||||
"if TYPE_CHECKING:",
|
||||
"\\.\\.\\.",
|
||||
]
|
||||
3
backend/src/mimic/__init__.py
Normal file
3
backend/src/mimic/__init__.py
Normal file
@@ -0,0 +1,3 @@
|
||||
"""Mimic backend package."""
|
||||
|
||||
__version__ = "0.1.0a0"
|
||||
17
backend/src/mimic/api/__init__.py
Normal file
17
backend/src/mimic/api/__init__.py
Normal file
@@ -0,0 +1,17 @@
|
||||
"""Flask blueprints (REST API)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from flask import Flask
|
||||
|
||||
from mimic.api.engagements import bp as engagements_bp
|
||||
from mimic.api.hosts import bp as hosts_bp
|
||||
from mimic.api.scenarios import bp as scenarios_bp
|
||||
from mimic.api.ttps import bp as ttps_bp
|
||||
|
||||
|
||||
def register_blueprints(app: Flask) -> None:
|
||||
app.register_blueprint(engagements_bp, url_prefix="/api/v1/engagements")
|
||||
app.register_blueprint(hosts_bp, url_prefix="/api/v1")
|
||||
app.register_blueprint(ttps_bp, url_prefix="/api/v1/library/ttps")
|
||||
app.register_blueprint(scenarios_bp, url_prefix="/api/v1")
|
||||
79
backend/src/mimic/api/_helpers.py
Normal file
79
backend/src/mimic/api/_helpers.py
Normal file
@@ -0,0 +1,79 @@
|
||||
"""Shared blueprint helpers (pydantic validation, current-user, scope, audit)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, cast
|
||||
from uuid import UUID
|
||||
|
||||
from flask import Response, abort, jsonify, request
|
||||
from flask_login import current_user
|
||||
from pydantic import BaseModel, ValidationError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from mimic.audit import AuditWriter
|
||||
from mimic.extensions import db
|
||||
from mimic.rbac.matrix import GroupName
|
||||
|
||||
|
||||
def parse_body[T: BaseModel](model: type[T]) -> T:
|
||||
payload = request.get_json(silent=True)
|
||||
if payload is None:
|
||||
abort(400, description="JSON body required")
|
||||
try:
|
||||
return model.model_validate(payload)
|
||||
except ValidationError as exc:
|
||||
abort(422, description=exc.errors())
|
||||
|
||||
|
||||
def jsonify_model(model: BaseModel, status: int = 200) -> tuple[Response, int]:
|
||||
return jsonify(model.model_dump(mode="json")), status
|
||||
|
||||
|
||||
def parse_uuid(value: str, *, field: str = "id") -> UUID:
|
||||
try:
|
||||
return UUID(value)
|
||||
except (ValueError, TypeError):
|
||||
abort(404, description=f"invalid {field}")
|
||||
|
||||
|
||||
def current_user_id() -> UUID | None:
|
||||
"""`current_user.id` if logged in, else None (for `created_by_id` columns)."""
|
||||
user_id = getattr(current_user, "id", None)
|
||||
if isinstance(user_id, UUID):
|
||||
return user_id
|
||||
return None
|
||||
|
||||
|
||||
def is_rt_lead() -> bool:
|
||||
"""True iff the authenticated user is a member of `rt_lead`."""
|
||||
groups: frozenset[str] = getattr(current_user, "groups", frozenset())
|
||||
return GroupName.RT_LEAD.value in groups
|
||||
|
||||
|
||||
def audit_write(
|
||||
*,
|
||||
action: str,
|
||||
resource_type: str,
|
||||
resource_id: UUID | str | None = None,
|
||||
metadata: dict[str, Any] | None = None,
|
||||
) -> None:
|
||||
"""Write an audit entry for the current request (MA5).
|
||||
|
||||
Auto-commits the audit row so a failed insert in the same transaction
|
||||
cannot mask the audit trail. Caller MUST have committed the resource
|
||||
change before calling this.
|
||||
"""
|
||||
resource_id_str = str(resource_id) if resource_id is not None else None
|
||||
# `db.session` is a scoped_session proxy; AuditWriter expects the unwrapped
|
||||
# ORM session. They are interchangeable at runtime.
|
||||
session = cast(Session, db.session)
|
||||
AuditWriter(session).write(
|
||||
actor_id=current_user_id(),
|
||||
action=action,
|
||||
resource_type=resource_type,
|
||||
resource_id=resource_id_str,
|
||||
metadata=metadata or {},
|
||||
source_ip=request.remote_addr,
|
||||
user_agent=request.headers.get("User-Agent"),
|
||||
)
|
||||
db.session.commit()
|
||||
125
backend/src/mimic/api/engagements.py
Normal file
125
backend/src/mimic/api/engagements.py
Normal file
@@ -0,0 +1,125 @@
|
||||
"""Engagement CRUD endpoints (flat, sprint 0)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from uuid import UUID
|
||||
|
||||
from flask import Blueprint, abort, jsonify
|
||||
from flask.typing import ResponseReturnValue
|
||||
from sqlalchemy import select
|
||||
|
||||
from mimic.api._helpers import (
|
||||
audit_write,
|
||||
current_user_id,
|
||||
is_rt_lead,
|
||||
jsonify_model,
|
||||
parse_body,
|
||||
parse_uuid,
|
||||
)
|
||||
from mimic.db.models import Engagement, EngagementMember
|
||||
from mimic.db.types import EngagementStatus
|
||||
from mimic.extensions import db
|
||||
from mimic.rbac import Permission, require_perm
|
||||
from mimic.schemas import EngagementCreate, EngagementRead, EngagementUpdate
|
||||
|
||||
bp = Blueprint("engagements", __name__)
|
||||
|
||||
|
||||
def _engagement_or_404(eid: str) -> Engagement:
|
||||
engagement = db.session.get(Engagement, parse_uuid(eid))
|
||||
if engagement is None:
|
||||
abort(404)
|
||||
# MA6: RT operators may only see engagements they're assigned to.
|
||||
if not is_rt_lead():
|
||||
user_id = current_user_id()
|
||||
if user_id is None or not _is_member(engagement.id, user_id):
|
||||
abort(404)
|
||||
return engagement
|
||||
|
||||
|
||||
def _is_member(engagement_id: UUID, user_id: UUID) -> bool:
|
||||
stmt = select(EngagementMember).where(
|
||||
EngagementMember.engagement_id == engagement_id,
|
||||
EngagementMember.user_id == user_id,
|
||||
)
|
||||
return db.session.execute(stmt).scalar_one_or_none() is not None
|
||||
|
||||
|
||||
@bp.get("")
|
||||
@require_perm(Permission.ENGAGEMENT_READ)
|
||||
def list_engagements() -> ResponseReturnValue:
|
||||
stmt = select(Engagement).order_by(Engagement.created_at.desc())
|
||||
# MA6: RT operators see only their assignments.
|
||||
if not is_rt_lead():
|
||||
user_id = current_user_id()
|
||||
if user_id is None:
|
||||
return jsonify([])
|
||||
stmt = stmt.join(
|
||||
EngagementMember,
|
||||
EngagementMember.engagement_id == Engagement.id,
|
||||
).where(EngagementMember.user_id == user_id)
|
||||
rows = db.session.execute(stmt).scalars().all()
|
||||
return jsonify([EngagementRead.model_validate(row).model_dump(mode="json") for row in rows])
|
||||
|
||||
|
||||
@bp.post("")
|
||||
@require_perm(Permission.ENGAGEMENT_CREATE)
|
||||
def create_engagement() -> ResponseReturnValue:
|
||||
payload = parse_body(EngagementCreate)
|
||||
engagement = Engagement(
|
||||
client_name=payload.client_name,
|
||||
description=payload.description,
|
||||
c2_type=payload.c2_type,
|
||||
start_date=payload.start_date,
|
||||
end_date=payload.end_date,
|
||||
status=EngagementStatus.DRAFT,
|
||||
created_by_id=current_user_id(),
|
||||
)
|
||||
db.session.add(engagement)
|
||||
db.session.commit()
|
||||
audit_write(
|
||||
action="engagement.create",
|
||||
resource_type="engagement",
|
||||
resource_id=engagement.id,
|
||||
metadata={"client_name": engagement.client_name},
|
||||
)
|
||||
return jsonify_model(EngagementRead.model_validate(engagement), status=201)
|
||||
|
||||
|
||||
@bp.get("/<eid>")
|
||||
@require_perm(Permission.ENGAGEMENT_READ)
|
||||
def get_engagement(eid: str) -> ResponseReturnValue:
|
||||
engagement = _engagement_or_404(eid)
|
||||
return jsonify_model(EngagementRead.model_validate(engagement))
|
||||
|
||||
|
||||
@bp.put("/<eid>")
|
||||
@require_perm(Permission.ENGAGEMENT_UPDATE)
|
||||
def update_engagement(eid: str) -> ResponseReturnValue:
|
||||
engagement = _engagement_or_404(eid)
|
||||
payload = parse_body(EngagementUpdate)
|
||||
changes = payload.model_dump(exclude_unset=True)
|
||||
for field, value in changes.items():
|
||||
setattr(engagement, field, value)
|
||||
db.session.commit()
|
||||
audit_write(
|
||||
action="engagement.update",
|
||||
resource_type="engagement",
|
||||
resource_id=engagement.id,
|
||||
metadata={"fields": sorted(changes.keys())},
|
||||
)
|
||||
return jsonify_model(EngagementRead.model_validate(engagement))
|
||||
|
||||
|
||||
@bp.delete("/<eid>")
|
||||
@require_perm(Permission.ENGAGEMENT_DELETE)
|
||||
def delete_engagement(eid: str) -> ResponseReturnValue:
|
||||
engagement = _engagement_or_404(eid)
|
||||
engagement.status = EngagementStatus.ARCHIVED
|
||||
db.session.commit()
|
||||
audit_write(
|
||||
action="engagement.archive",
|
||||
resource_type="engagement",
|
||||
resource_id=engagement.id,
|
||||
)
|
||||
return "", 204
|
||||
115
backend/src/mimic/api/hosts.py
Normal file
115
backend/src/mimic/api/hosts.py
Normal file
@@ -0,0 +1,115 @@
|
||||
"""Host CRUD endpoints (scoped under an engagement)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from flask import Blueprint, abort, jsonify
|
||||
from flask.typing import ResponseReturnValue
|
||||
from sqlalchemy import select
|
||||
|
||||
from mimic.api._helpers import (
|
||||
audit_write,
|
||||
current_user_id,
|
||||
is_rt_lead,
|
||||
jsonify_model,
|
||||
parse_body,
|
||||
parse_uuid,
|
||||
)
|
||||
from mimic.db.models import Engagement, EngagementMember, Host
|
||||
from mimic.db.types import HostStatus
|
||||
from mimic.extensions import db
|
||||
from mimic.rbac import Permission, require_perm
|
||||
from mimic.schemas import HostCreate, HostRead, HostUpdate
|
||||
|
||||
bp = Blueprint("hosts", __name__)
|
||||
|
||||
|
||||
def _engagement_or_404(eid: str) -> Engagement:
|
||||
engagement = db.session.get(Engagement, parse_uuid(eid, field="engagement id"))
|
||||
if engagement is None:
|
||||
abort(404)
|
||||
# MA6: RT operators may only access engagements they're assigned to.
|
||||
if not is_rt_lead():
|
||||
user_id = current_user_id()
|
||||
if user_id is None:
|
||||
abort(404)
|
||||
stmt = select(EngagementMember).where(
|
||||
EngagementMember.engagement_id == engagement.id,
|
||||
EngagementMember.user_id == user_id,
|
||||
)
|
||||
if db.session.execute(stmt).scalar_one_or_none() is None:
|
||||
abort(404)
|
||||
return engagement
|
||||
|
||||
|
||||
@bp.get("/engagements/<eid>/hosts")
|
||||
@require_perm(Permission.HOST_CRUD)
|
||||
def list_hosts(eid: str) -> ResponseReturnValue:
|
||||
engagement = _engagement_or_404(eid)
|
||||
stmt = select(Host).where(Host.engagement_id == engagement.id).order_by(Host.hostname)
|
||||
rows = db.session.execute(stmt).scalars().all()
|
||||
return jsonify([HostRead.model_validate(row).model_dump(mode="json") for row in rows])
|
||||
|
||||
|
||||
@bp.post("/engagements/<eid>/hosts")
|
||||
@require_perm(Permission.HOST_CRUD)
|
||||
def create_host(eid: str) -> ResponseReturnValue:
|
||||
engagement = _engagement_or_404(eid)
|
||||
payload = parse_body(HostCreate)
|
||||
host = Host(
|
||||
engagement_id=engagement.id,
|
||||
hostname=payload.hostname,
|
||||
ip=payload.ip,
|
||||
os=payload.os,
|
||||
c2_session_id=payload.c2_session_id,
|
||||
c2_type=payload.c2_type,
|
||||
status=HostStatus.UNKNOWN,
|
||||
)
|
||||
db.session.add(host)
|
||||
db.session.commit()
|
||||
audit_write(
|
||||
action="host.create",
|
||||
resource_type="host",
|
||||
resource_id=host.id,
|
||||
metadata={"engagement_id": str(engagement.id), "hostname": host.hostname},
|
||||
)
|
||||
return jsonify_model(HostRead.model_validate(host), status=201)
|
||||
|
||||
|
||||
@bp.put("/engagements/<eid>/hosts/<hid>")
|
||||
@require_perm(Permission.HOST_CRUD)
|
||||
def update_host(eid: str, hid: str) -> ResponseReturnValue:
|
||||
engagement = _engagement_or_404(eid)
|
||||
host = db.session.get(Host, parse_uuid(hid, field="host id"))
|
||||
if host is None or host.engagement_id != engagement.id:
|
||||
abort(404)
|
||||
payload = parse_body(HostUpdate)
|
||||
changes = payload.model_dump(exclude_unset=True)
|
||||
for field, value in changes.items():
|
||||
setattr(host, field, value)
|
||||
db.session.commit()
|
||||
audit_write(
|
||||
action="host.update",
|
||||
resource_type="host",
|
||||
resource_id=host.id,
|
||||
metadata={"engagement_id": str(engagement.id), "fields": sorted(changes.keys())},
|
||||
)
|
||||
return jsonify_model(HostRead.model_validate(host))
|
||||
|
||||
|
||||
@bp.delete("/engagements/<eid>/hosts/<hid>")
|
||||
@require_perm(Permission.HOST_CRUD)
|
||||
def delete_host(eid: str, hid: str) -> ResponseReturnValue:
|
||||
engagement = _engagement_or_404(eid)
|
||||
host = db.session.get(Host, parse_uuid(hid, field="host id"))
|
||||
if host is None or host.engagement_id != engagement.id:
|
||||
abort(404)
|
||||
host_id = host.id
|
||||
db.session.delete(host)
|
||||
db.session.commit()
|
||||
audit_write(
|
||||
action="host.delete",
|
||||
resource_type="host",
|
||||
resource_id=host_id,
|
||||
metadata={"engagement_id": str(engagement.id)},
|
||||
)
|
||||
return "", 204
|
||||
195
backend/src/mimic/api/scenarios.py
Normal file
195
backend/src/mimic/api/scenarios.py
Normal file
@@ -0,0 +1,195 @@
|
||||
"""Scenario + step CRUD (flat, no orchestration yet)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from uuid import UUID
|
||||
|
||||
from flask import Blueprint, abort, jsonify
|
||||
from flask.typing import ResponseReturnValue
|
||||
from sqlalchemy import select
|
||||
|
||||
from mimic.api._helpers import (
|
||||
audit_write,
|
||||
current_user_id,
|
||||
is_rt_lead,
|
||||
jsonify_model,
|
||||
parse_body,
|
||||
parse_uuid,
|
||||
)
|
||||
from mimic.db.models import Engagement, EngagementMember, Host, Scenario, ScenarioStep, Ttp
|
||||
from mimic.extensions import db
|
||||
from mimic.rbac import Permission, require_perm
|
||||
from mimic.schemas import (
|
||||
ScenarioCreate,
|
||||
ScenarioRead,
|
||||
ScenarioStepCreate,
|
||||
ScenarioStepRead,
|
||||
ScenarioUpdate,
|
||||
)
|
||||
|
||||
bp = Blueprint("scenarios", __name__)
|
||||
|
||||
|
||||
def _engagement_or_404(eid: str) -> Engagement:
|
||||
engagement = db.session.get(Engagement, parse_uuid(eid, field="engagement id"))
|
||||
if engagement is None:
|
||||
abort(404)
|
||||
# MA6: RT operators may only access engagements they're assigned to.
|
||||
if not is_rt_lead():
|
||||
user_id = current_user_id()
|
||||
if user_id is None:
|
||||
abort(404)
|
||||
stmt = select(EngagementMember).where(
|
||||
EngagementMember.engagement_id == engagement.id,
|
||||
EngagementMember.user_id == user_id,
|
||||
)
|
||||
if db.session.execute(stmt).scalar_one_or_none() is None:
|
||||
abort(404)
|
||||
return engagement
|
||||
|
||||
|
||||
def _scenario_or_404(engagement: Engagement, sid: str) -> Scenario:
|
||||
scenario = db.session.get(Scenario, parse_uuid(sid, field="scenario id"))
|
||||
if scenario is None or scenario.engagement_id != engagement.id:
|
||||
abort(404)
|
||||
return scenario
|
||||
|
||||
|
||||
def _validate_step_consistency(scenario: Scenario, ttp_id: UUID, host_id: UUID) -> None:
|
||||
ttp = db.session.get(Ttp, ttp_id)
|
||||
host = db.session.get(Host, host_id)
|
||||
if ttp is None or host is None:
|
||||
abort(404, description="ttp or host not found")
|
||||
if host.engagement_id != scenario.engagement_id:
|
||||
abort(400, description="host does not belong to the engagement")
|
||||
if host.c2_type != scenario.c2_type:
|
||||
abort(400, description="host.c2_type does not match scenario.c2_type")
|
||||
|
||||
|
||||
@bp.get("/engagements/<eid>/scenarios")
|
||||
@require_perm(Permission.SCENARIO_CRUD)
|
||||
def list_scenarios(eid: str) -> ResponseReturnValue:
|
||||
engagement = _engagement_or_404(eid)
|
||||
stmt = (
|
||||
select(Scenario)
|
||||
.where(Scenario.engagement_id == engagement.id)
|
||||
.order_by(Scenario.created_at.desc())
|
||||
)
|
||||
rows = db.session.execute(stmt).scalars().all()
|
||||
return jsonify([ScenarioRead.model_validate(row).model_dump(mode="json") for row in rows])
|
||||
|
||||
|
||||
@bp.post("/engagements/<eid>/scenarios")
|
||||
@require_perm(Permission.SCENARIO_CRUD)
|
||||
def create_scenario(eid: str) -> ResponseReturnValue:
|
||||
engagement = _engagement_or_404(eid)
|
||||
payload = parse_body(ScenarioCreate)
|
||||
scenario = Scenario(
|
||||
engagement_id=engagement.id,
|
||||
name=payload.name,
|
||||
description=payload.description,
|
||||
c2_type=payload.c2_type,
|
||||
version=payload.version,
|
||||
created_by_id=current_user_id(),
|
||||
)
|
||||
db.session.add(scenario)
|
||||
db.session.flush()
|
||||
for step in payload.steps:
|
||||
_validate_step_consistency(scenario, step.ttp_id, step.host_id)
|
||||
db.session.add(
|
||||
ScenarioStep(
|
||||
scenario_id=scenario.id,
|
||||
ttp_id=step.ttp_id,
|
||||
host_id=step.host_id,
|
||||
order_idx=step.order_idx,
|
||||
params_override_json=step.params_override_json,
|
||||
delay_after_ms=step.delay_after_ms,
|
||||
)
|
||||
)
|
||||
db.session.commit()
|
||||
audit_write(
|
||||
action="scenario.create",
|
||||
resource_type="scenario",
|
||||
resource_id=scenario.id,
|
||||
metadata={
|
||||
"engagement_id": str(engagement.id),
|
||||
"name": scenario.name,
|
||||
"step_count": len(payload.steps),
|
||||
},
|
||||
)
|
||||
return jsonify_model(ScenarioRead.model_validate(scenario), status=201)
|
||||
|
||||
|
||||
@bp.get("/engagements/<eid>/scenarios/<sid>")
|
||||
@require_perm(Permission.SCENARIO_CRUD)
|
||||
def get_scenario(eid: str, sid: str) -> ResponseReturnValue:
|
||||
engagement = _engagement_or_404(eid)
|
||||
scenario = _scenario_or_404(engagement, sid)
|
||||
return jsonify_model(ScenarioRead.model_validate(scenario))
|
||||
|
||||
|
||||
@bp.put("/engagements/<eid>/scenarios/<sid>")
|
||||
@require_perm(Permission.SCENARIO_CRUD)
|
||||
def update_scenario(eid: str, sid: str) -> ResponseReturnValue:
|
||||
engagement = _engagement_or_404(eid)
|
||||
scenario = _scenario_or_404(engagement, sid)
|
||||
payload = parse_body(ScenarioUpdate)
|
||||
changes = payload.model_dump(exclude_unset=True)
|
||||
for field, value in changes.items():
|
||||
setattr(scenario, field, value)
|
||||
db.session.commit()
|
||||
audit_write(
|
||||
action="scenario.update",
|
||||
resource_type="scenario",
|
||||
resource_id=scenario.id,
|
||||
metadata={"engagement_id": str(engagement.id), "fields": sorted(changes.keys())},
|
||||
)
|
||||
return jsonify_model(ScenarioRead.model_validate(scenario))
|
||||
|
||||
|
||||
@bp.delete("/engagements/<eid>/scenarios/<sid>")
|
||||
@require_perm(Permission.SCENARIO_CRUD)
|
||||
def delete_scenario(eid: str, sid: str) -> ResponseReturnValue:
|
||||
engagement = _engagement_or_404(eid)
|
||||
scenario = _scenario_or_404(engagement, sid)
|
||||
scenario_id = scenario.id
|
||||
db.session.delete(scenario)
|
||||
db.session.commit()
|
||||
audit_write(
|
||||
action="scenario.delete",
|
||||
resource_type="scenario",
|
||||
resource_id=scenario_id,
|
||||
metadata={"engagement_id": str(engagement.id)},
|
||||
)
|
||||
return "", 204
|
||||
|
||||
|
||||
@bp.post("/engagements/<eid>/scenarios/<sid>/steps")
|
||||
@require_perm(Permission.SCENARIO_CRUD)
|
||||
def add_step(eid: str, sid: str) -> ResponseReturnValue:
|
||||
engagement = _engagement_or_404(eid)
|
||||
scenario = _scenario_or_404(engagement, sid)
|
||||
payload = parse_body(ScenarioStepCreate)
|
||||
_validate_step_consistency(scenario, payload.ttp_id, payload.host_id)
|
||||
step = ScenarioStep(
|
||||
scenario_id=scenario.id,
|
||||
ttp_id=payload.ttp_id,
|
||||
host_id=payload.host_id,
|
||||
order_idx=payload.order_idx,
|
||||
params_override_json=payload.params_override_json,
|
||||
delay_after_ms=payload.delay_after_ms,
|
||||
)
|
||||
db.session.add(step)
|
||||
db.session.commit()
|
||||
audit_write(
|
||||
action="scenario_step.create",
|
||||
resource_type="scenario_step",
|
||||
resource_id=step.id,
|
||||
metadata={
|
||||
"scenario_id": str(scenario.id),
|
||||
"ttp_id": str(payload.ttp_id),
|
||||
"host_id": str(payload.host_id),
|
||||
"order_idx": payload.order_idx,
|
||||
},
|
||||
)
|
||||
return jsonify_model(ScenarioStepRead.model_validate(step), status=201)
|
||||
102
backend/src/mimic/api/ttps.py
Normal file
102
backend/src/mimic/api/ttps.py
Normal file
@@ -0,0 +1,102 @@
|
||||
"""TTP library CRUD endpoints."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from flask import Blueprint, abort, jsonify
|
||||
from flask.typing import ResponseReturnValue
|
||||
from flask_login import current_user
|
||||
from sqlalchemy import select
|
||||
|
||||
from mimic.api._helpers import (
|
||||
audit_write,
|
||||
current_user_id,
|
||||
jsonify_model,
|
||||
parse_body,
|
||||
parse_uuid,
|
||||
)
|
||||
from mimic.db.models import Ttp
|
||||
from mimic.extensions import db
|
||||
from mimic.rbac import Permission, require_perm
|
||||
from mimic.schemas import TtpCreate, TtpRead, TtpUpdate
|
||||
|
||||
bp = Blueprint("ttps", __name__)
|
||||
|
||||
|
||||
@bp.get("")
|
||||
@require_perm(Permission.TTP_READ)
|
||||
def list_ttps() -> ResponseReturnValue:
|
||||
stmt = select(Ttp).order_by(Ttp.created_at.desc())
|
||||
rows = db.session.execute(stmt).scalars().all()
|
||||
return jsonify([TtpRead.model_validate(row).model_dump(mode="json") for row in rows])
|
||||
|
||||
|
||||
@bp.post("")
|
||||
@require_perm(Permission.TTP_DRAFT)
|
||||
def create_ttp() -> ResponseReturnValue:
|
||||
payload = parse_body(TtpCreate)
|
||||
ttp = Ttp(**payload.model_dump(), created_by_id=current_user_id())
|
||||
db.session.add(ttp)
|
||||
db.session.commit()
|
||||
audit_write(
|
||||
action="ttp.create",
|
||||
resource_type="ttp",
|
||||
resource_id=ttp.id,
|
||||
metadata={"name": ttp.name, "mitre_technique": ttp.mitre_technique},
|
||||
)
|
||||
return jsonify_model(TtpRead.model_validate(ttp), status=201)
|
||||
|
||||
|
||||
@bp.get("/<tid>")
|
||||
@require_perm(Permission.TTP_READ)
|
||||
def get_ttp(tid: str) -> ResponseReturnValue:
|
||||
ttp = db.session.get(Ttp, parse_uuid(tid, field="ttp id"))
|
||||
if ttp is None:
|
||||
abort(404)
|
||||
return jsonify_model(TtpRead.model_validate(ttp))
|
||||
|
||||
|
||||
@bp.put("/<tid>")
|
||||
@require_perm(Permission.TTP_DRAFT)
|
||||
def update_ttp(tid: str) -> ResponseReturnValue:
|
||||
ttp = db.session.get(Ttp, parse_uuid(tid, field="ttp id"))
|
||||
if ttp is None:
|
||||
abort(404)
|
||||
payload = parse_body(TtpUpdate)
|
||||
data = payload.model_dump(exclude_unset=True)
|
||||
publish_flag = data.pop("is_published", None)
|
||||
for field, value in data.items():
|
||||
setattr(ttp, field, value)
|
||||
promoted = False
|
||||
if publish_flag is not None:
|
||||
# Promotion is a lead-only privilege. Decorator already gates draft
|
||||
# edits; promotion gets a second-tier check at the call site.
|
||||
_ensure_promote_perm()
|
||||
ttp.is_published = publish_flag
|
||||
promoted = True
|
||||
db.session.commit()
|
||||
audit_write(
|
||||
action="ttp.promote" if promoted else "ttp.update",
|
||||
resource_type="ttp",
|
||||
resource_id=ttp.id,
|
||||
metadata={"fields": sorted(data.keys()), "is_published": ttp.is_published},
|
||||
)
|
||||
return jsonify_model(TtpRead.model_validate(ttp))
|
||||
|
||||
|
||||
@bp.delete("/<tid>")
|
||||
@require_perm(Permission.TTP_DRAFT)
|
||||
def delete_ttp(tid: str) -> ResponseReturnValue:
|
||||
ttp = db.session.get(Ttp, parse_uuid(tid, field="ttp id"))
|
||||
if ttp is None:
|
||||
abort(404)
|
||||
ttp_id = ttp.id
|
||||
db.session.delete(ttp)
|
||||
db.session.commit()
|
||||
audit_write(action="ttp.delete", resource_type="ttp", resource_id=ttp_id)
|
||||
return "", 204
|
||||
|
||||
|
||||
def _ensure_promote_perm() -> None:
|
||||
perms: frozenset[Permission] = getattr(current_user, "permissions", frozenset())
|
||||
if Permission.TTP_PROMOTE not in perms:
|
||||
abort(403)
|
||||
50
backend/src/mimic/app.py
Normal file
50
backend/src/mimic/app.py
Normal file
@@ -0,0 +1,50 @@
|
||||
"""Flask application factory."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import timedelta
|
||||
|
||||
from flask import Flask, jsonify
|
||||
from flask.typing import ResponseReturnValue
|
||||
|
||||
from mimic.api import register_blueprints
|
||||
from mimic.auth.identity import load_user
|
||||
from mimic.config import Settings, get_settings
|
||||
from mimic.extensions import db, login_manager, migrate, socketio
|
||||
from mimic.logging import configure_logging
|
||||
|
||||
|
||||
def create_app(settings: Settings | None = None) -> Flask:
|
||||
settings = settings or get_settings()
|
||||
configure_logging(settings.log_level, as_json=settings.log_json)
|
||||
|
||||
app = Flask(__name__)
|
||||
app.config.update(
|
||||
SECRET_KEY=settings.secret_key.get_secret_value(),
|
||||
SQLALCHEMY_DATABASE_URI=settings.database_url,
|
||||
SQLALCHEMY_TRACK_MODIFICATIONS=False,
|
||||
SESSION_COOKIE_SECURE=settings.session_cookie_secure,
|
||||
SESSION_COOKIE_SAMESITE=settings.session_cookie_samesite,
|
||||
SESSION_COOKIE_HTTPONLY=True,
|
||||
PERMANENT_SESSION_LIFETIME=timedelta(minutes=settings.session_lifetime_minutes),
|
||||
MIMIC_SETTINGS=settings,
|
||||
)
|
||||
|
||||
db.init_app(app)
|
||||
migrate.init_app(app, db, directory="src/mimic/db/migrations")
|
||||
login_manager.init_app(app)
|
||||
login_manager.user_loader(load_user)
|
||||
|
||||
socketio.init_app(
|
||||
app,
|
||||
cors_allowed_origins=settings.cors_origins or "*",
|
||||
async_mode="gevent",
|
||||
)
|
||||
|
||||
register_blueprints(app)
|
||||
|
||||
@app.get("/healthz")
|
||||
def healthz() -> ResponseReturnValue:
|
||||
return jsonify(status="ok"), 200
|
||||
|
||||
return app
|
||||
5
backend/src/mimic/audit/__init__.py
Normal file
5
backend/src/mimic/audit/__init__.py
Normal file
@@ -0,0 +1,5 @@
|
||||
"""Append-only audit log writer (NF-AUDIT)."""
|
||||
|
||||
from mimic.audit.log import AuditWriter, audit_hash, write_audit
|
||||
|
||||
__all__ = ["AuditWriter", "audit_hash", "write_audit"]
|
||||
102
backend/src/mimic/audit/log.py
Normal file
102
backend/src/mimic/audit/log.py
Normal file
@@ -0,0 +1,102 @@
|
||||
"""Hash-chained append-only audit writer.
|
||||
|
||||
Sprint 0 fills `prev_hash` / `row_hash` at insert; verifier (v2) walks the
|
||||
chain. The SQL-level guard (write-only role on `audit_log`) is provisioned by
|
||||
the deploy playbook against the Postgres role `mimic_audit_writer`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from mimic.db.models import AuditLog
|
||||
|
||||
|
||||
def _canonical(payload: dict[str, Any]) -> str:
|
||||
return json.dumps(payload, sort_keys=True, separators=(",", ":"), default=str)
|
||||
|
||||
|
||||
def audit_hash(
|
||||
*,
|
||||
prev_hash: str | None,
|
||||
ts: datetime,
|
||||
actor_id: UUID | None,
|
||||
action: str,
|
||||
resource_type: str,
|
||||
resource_id: str | None,
|
||||
metadata: dict[str, Any],
|
||||
) -> str:
|
||||
"""SHA-256 of the canonical record (used both at write and at verify)."""
|
||||
payload = {
|
||||
"prev_hash": prev_hash or "",
|
||||
"ts": ts.isoformat(),
|
||||
"actor_id": str(actor_id) if actor_id else "",
|
||||
"action": action,
|
||||
"resource_type": resource_type,
|
||||
"resource_id": resource_id or "",
|
||||
"metadata": metadata,
|
||||
}
|
||||
return hashlib.sha256(_canonical(payload).encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
class AuditWriter:
|
||||
"""Append a single audit entry, chaining on the latest row hash."""
|
||||
|
||||
def __init__(self, session: Session) -> None:
|
||||
self._session = session
|
||||
|
||||
def write(
|
||||
self,
|
||||
*,
|
||||
action: str,
|
||||
resource_type: str,
|
||||
actor_id: UUID | None = None,
|
||||
resource_id: str | None = None,
|
||||
metadata: dict[str, Any] | None = None,
|
||||
source_ip: str | None = None,
|
||||
user_agent: str | None = None,
|
||||
comment: str | None = None,
|
||||
) -> AuditLog:
|
||||
metadata = metadata or {}
|
||||
ts = datetime.now(tz=UTC)
|
||||
prev_hash = self._latest_hash()
|
||||
row_hash = audit_hash(
|
||||
prev_hash=prev_hash,
|
||||
ts=ts,
|
||||
actor_id=actor_id,
|
||||
action=action,
|
||||
resource_type=resource_type,
|
||||
resource_id=resource_id,
|
||||
metadata=metadata,
|
||||
)
|
||||
entry = AuditLog(
|
||||
ts=ts,
|
||||
actor_id=actor_id,
|
||||
action=action,
|
||||
resource_type=resource_type,
|
||||
resource_id=resource_id,
|
||||
metadata_json=metadata,
|
||||
prev_hash=prev_hash,
|
||||
row_hash=row_hash,
|
||||
source_ip=source_ip,
|
||||
user_agent=user_agent,
|
||||
comment=comment,
|
||||
)
|
||||
self._session.add(entry)
|
||||
self._session.flush()
|
||||
return entry
|
||||
|
||||
def _latest_hash(self) -> str | None:
|
||||
stmt = select(AuditLog.row_hash).order_by(AuditLog.ts.desc()).limit(1)
|
||||
return self._session.execute(stmt).scalar_one_or_none()
|
||||
|
||||
|
||||
def write_audit(session: Session, **kwargs: Any) -> AuditLog:
|
||||
return AuditWriter(session).write(**kwargs)
|
||||
6
backend/src/mimic/auth/__init__.py
Normal file
6
backend/src/mimic/auth/__init__.py
Normal file
@@ -0,0 +1,6 @@
|
||||
"""Authentication: local password (Flask-Login) + SOC sessions."""
|
||||
|
||||
from mimic.auth.identity import AuthUser, load_user
|
||||
from mimic.auth.password import check_password, hash_password
|
||||
|
||||
__all__ = ["AuthUser", "check_password", "hash_password", "load_user"]
|
||||
60
backend/src/mimic/auth/identity.py
Normal file
60
backend/src/mimic/auth/identity.py
Normal file
@@ -0,0 +1,60 @@
|
||||
"""Flask-Login user wrapper that carries the resolved permission set."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import selectinload
|
||||
|
||||
from mimic.db.models import User, UserGroup
|
||||
from mimic.extensions import db
|
||||
from mimic.rbac.matrix import GROUP_PERMISSIONS, GroupName, Permission
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class AuthUser:
|
||||
"""Lightweight identity attached to ``flask.g`` / ``current_user``."""
|
||||
|
||||
id: UUID
|
||||
email: str
|
||||
permissions: frozenset[Permission] = field(default_factory=frozenset)
|
||||
groups: frozenset[str] = field(default_factory=frozenset)
|
||||
is_authenticated: bool = True
|
||||
is_active: bool = True
|
||||
is_anonymous: bool = False
|
||||
|
||||
def get_id(self) -> str:
|
||||
return str(self.id)
|
||||
|
||||
|
||||
def load_user(user_id: str) -> AuthUser | None:
|
||||
"""Flask-Login `user_loader` callback."""
|
||||
try:
|
||||
uid = UUID(user_id)
|
||||
except (ValueError, TypeError):
|
||||
return None
|
||||
|
||||
stmt = (
|
||||
select(User)
|
||||
.where(User.id == uid)
|
||||
.options(selectinload(User.group_links).selectinload(UserGroup.group))
|
||||
)
|
||||
user = db.session.execute(stmt).scalar_one_or_none()
|
||||
if user is None or not user.is_active:
|
||||
return None
|
||||
|
||||
group_names = {link.group.name for link in user.group_links}
|
||||
perms: set[Permission] = set()
|
||||
for group_name in group_names:
|
||||
try:
|
||||
perms.update(GROUP_PERMISSIONS[GroupName(group_name)])
|
||||
except ValueError:
|
||||
continue
|
||||
return AuthUser(
|
||||
id=user.id,
|
||||
email=user.email,
|
||||
permissions=frozenset(perms),
|
||||
groups=frozenset(group_names),
|
||||
)
|
||||
25
backend/src/mimic/auth/password.py
Normal file
25
backend/src/mimic/auth/password.py
Normal file
@@ -0,0 +1,25 @@
|
||||
"""bcrypt password helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import bcrypt
|
||||
|
||||
_DEFAULT_ROUNDS = 12
|
||||
|
||||
|
||||
def hash_password(plain: str, *, rounds: int = _DEFAULT_ROUNDS) -> str:
|
||||
"""Hash a plain password with bcrypt; returns the UTF-8 string."""
|
||||
if not plain:
|
||||
raise ValueError("password must not be empty")
|
||||
salt = bcrypt.gensalt(rounds=rounds)
|
||||
return bcrypt.hashpw(plain.encode("utf-8"), salt).decode("utf-8")
|
||||
|
||||
|
||||
def check_password(plain: str, hashed: str | None) -> bool:
|
||||
"""Constant-time bcrypt verification (False if hash is missing/invalid)."""
|
||||
if not plain or not hashed:
|
||||
return False
|
||||
try:
|
||||
return bcrypt.checkpw(plain.encode("utf-8"), hashed.encode("utf-8"))
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
44
backend/src/mimic/auth/soc_token.py
Normal file
44
backend/src/mimic/auth/soc_token.py
Normal file
@@ -0,0 +1,44 @@
|
||||
"""SOC opaque token generation + verification.
|
||||
|
||||
Decision D-006: clear token returned once at creation, bcrypt hash persisted.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hmac
|
||||
import secrets
|
||||
from dataclasses import dataclass
|
||||
|
||||
import bcrypt
|
||||
|
||||
_TOKEN_BYTES = 32 # 256 bits of entropy
|
||||
_DEFAULT_ROUNDS = 12
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class SocTokenMaterial:
|
||||
plain: str
|
||||
hashed: str
|
||||
|
||||
|
||||
def generate_token(*, rounds: int = _DEFAULT_ROUNDS) -> SocTokenMaterial:
|
||||
"""Generate a fresh SOC token and its bcrypt hash."""
|
||||
plain = secrets.token_urlsafe(_TOKEN_BYTES)
|
||||
salt = bcrypt.gensalt(rounds=rounds)
|
||||
hashed = bcrypt.hashpw(plain.encode("utf-8"), salt).decode("utf-8")
|
||||
return SocTokenMaterial(plain=plain, hashed=hashed)
|
||||
|
||||
|
||||
def verify_token(plain: str, hashed: str) -> bool:
|
||||
"""Constant-time bcrypt verification of a SOC token."""
|
||||
if not plain or not hashed:
|
||||
return False
|
||||
try:
|
||||
return bcrypt.checkpw(plain.encode("utf-8"), hashed.encode("utf-8"))
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
|
||||
|
||||
def safe_compare(a: str, b: str) -> bool:
|
||||
"""Constant-time string compare for non-hashed contexts."""
|
||||
return hmac.compare_digest(a, b)
|
||||
21
backend/src/mimic/cli/__init__.py
Normal file
21
backend/src/mimic/cli/__init__.py
Normal file
@@ -0,0 +1,21 @@
|
||||
"""`mimic-cli` command-line interface (click)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import click
|
||||
|
||||
from mimic.cli.db import db_group
|
||||
from mimic.cli.user import user_group
|
||||
|
||||
|
||||
@click.group()
|
||||
def cli() -> None:
|
||||
"""Mimic command-line interface."""
|
||||
|
||||
|
||||
cli.add_command(user_group, name="user")
|
||||
cli.add_command(db_group, name="db")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
cli()
|
||||
71
backend/src/mimic/cli/db.py
Normal file
71
backend/src/mimic/cli/db.py
Normal file
@@ -0,0 +1,71 @@
|
||||
"""Database CLI: dump / restore stubs (R-O1)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
|
||||
import click
|
||||
|
||||
from mimic.config import get_settings
|
||||
|
||||
|
||||
@click.group(help="Database operations (manual dump/restore per R-O1).")
|
||||
def db_group() -> None: ...
|
||||
|
||||
|
||||
def _parse_dsn(dsn: str) -> tuple[str, str, str, str, str]:
|
||||
parsed = urlparse(dsn)
|
||||
return (
|
||||
parsed.hostname or "localhost",
|
||||
str(parsed.port or 5432),
|
||||
parsed.username or "",
|
||||
parsed.password or "",
|
||||
(parsed.path or "/").lstrip("/"),
|
||||
)
|
||||
|
||||
|
||||
@db_group.command("dump")
|
||||
@click.option("--out", "out_path", type=click.Path(dir_okay=False, path_type=Path), required=True)
|
||||
def dump(out_path: Path) -> None:
|
||||
"""Manual `pg_dump` of the configured DATABASE_URL."""
|
||||
settings = get_settings()
|
||||
host, port, user, password, dbname = _parse_dsn(settings.database_url)
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
cmd = [
|
||||
"pg_dump",
|
||||
"--format=custom",
|
||||
f"--host={host}",
|
||||
f"--port={port}",
|
||||
f"--username={user}",
|
||||
f"--dbname={dbname}",
|
||||
f"--file={out_path}",
|
||||
]
|
||||
env = {"PGPASSWORD": password} if password else None
|
||||
click.echo(f"running: {shlex.join(cmd)}")
|
||||
subprocess.run(cmd, check=True, env=env) # noqa: S603
|
||||
click.echo(f"dump written to {out_path}")
|
||||
|
||||
|
||||
@db_group.command("restore")
|
||||
@click.option("--file", "in_path", type=click.Path(exists=True, path_type=Path), required=True)
|
||||
def restore(in_path: Path) -> None:
|
||||
"""Manual `pg_restore` from a dump file."""
|
||||
settings = get_settings()
|
||||
host, port, user, password, dbname = _parse_dsn(settings.database_url)
|
||||
cmd = [
|
||||
"pg_restore",
|
||||
"--clean",
|
||||
"--if-exists",
|
||||
f"--host={host}",
|
||||
f"--port={port}",
|
||||
f"--username={user}",
|
||||
f"--dbname={dbname}",
|
||||
str(in_path),
|
||||
]
|
||||
env = {"PGPASSWORD": password} if password else None
|
||||
click.echo(f"running: {shlex.join(cmd)}")
|
||||
subprocess.run(cmd, check=True, env=env) # noqa: S603
|
||||
click.echo("restore complete")
|
||||
50
backend/src/mimic/cli/user.py
Normal file
50
backend/src/mimic/cli/user.py
Normal file
@@ -0,0 +1,50 @@
|
||||
"""User-related CLI commands."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import click
|
||||
|
||||
from mimic.app import create_app
|
||||
from mimic.auth.password import hash_password
|
||||
from mimic.db.models import Group, User, UserGroup
|
||||
from mimic.db.types import UserType
|
||||
from mimic.extensions import db
|
||||
from mimic.rbac.matrix import GroupName
|
||||
|
||||
|
||||
@click.group(help="Manage Mimic user accounts.")
|
||||
def user_group() -> None: ...
|
||||
|
||||
|
||||
@user_group.command("create")
|
||||
@click.option("--email", required=True)
|
||||
@click.option(
|
||||
"--type",
|
||||
"user_type",
|
||||
type=click.Choice([u.value for u in UserType]),
|
||||
required=True,
|
||||
)
|
||||
@click.option("--password", prompt=True, hide_input=True, confirmation_prompt=True)
|
||||
@click.option("--display-name", default=None)
|
||||
def create_user(email: str, user_type: str, password: str, display_name: str | None) -> None:
|
||||
"""Create a local user (sprint 0: rt_operator or rt_lead)."""
|
||||
app = create_app()
|
||||
with app.app_context():
|
||||
user = User(
|
||||
email=email,
|
||||
display_name=display_name,
|
||||
type=UserType(user_type),
|
||||
local_password_hash=hash_password(password),
|
||||
)
|
||||
db.session.add(user)
|
||||
db.session.flush()
|
||||
|
||||
group_name = GroupName.RT_LEAD if user.type is UserType.RT_LEAD else GroupName.RT_OPERATOR
|
||||
group = db.session.query(Group).filter_by(name=group_name.value).first()
|
||||
if group is None:
|
||||
raise click.ClickException(
|
||||
f"group {group_name.value} not seeded — run alembic upgrade head first"
|
||||
)
|
||||
db.session.add(UserGroup(user_id=user.id, group_id=group.id, engagement_id=None))
|
||||
db.session.commit()
|
||||
click.echo(f"created user {email} ({user.id}) in group {group_name.value}")
|
||||
71
backend/src/mimic/config.py
Normal file
71
backend/src/mimic/config.py
Normal file
@@ -0,0 +1,71 @@
|
||||
"""Runtime configuration loaded from environment (Pydantic Settings)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import Field, SecretStr, field_validator
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
Environment = Literal["development", "testing", "production"]
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
"""Application settings.
|
||||
|
||||
All values are env-driven (NF-network: no hardcoded secrets). The `MIMIC_`
|
||||
prefix isolates Mimic vars from ambient shell vars.
|
||||
"""
|
||||
|
||||
model_config = SettingsConfigDict(
|
||||
env_prefix="MIMIC_",
|
||||
env_file=".env",
|
||||
env_file_encoding="utf-8",
|
||||
extra="ignore",
|
||||
case_sensitive=False,
|
||||
)
|
||||
|
||||
env: Environment = "development"
|
||||
secret_key: SecretStr = Field(default=SecretStr("change-me"))
|
||||
fernet_key: SecretStr = Field(default=SecretStr(""))
|
||||
|
||||
database_url: str = "postgresql+psycopg://mimic_app:mimic_dev_password@localhost:5432/mimic"
|
||||
database_audit_url: str | None = None
|
||||
|
||||
session_cookie_secure: bool = True
|
||||
session_cookie_samesite: Literal["Lax", "Strict", "None"] = "Lax"
|
||||
session_lifetime_minutes: int = 60 * 8
|
||||
|
||||
cors_origins: list[str] = Field(default_factory=list)
|
||||
|
||||
log_level: str = "INFO"
|
||||
log_json: bool = True
|
||||
|
||||
template_render_timeout_ms: int = 250
|
||||
output_blob_max_bytes: int = 10 * 1024 * 1024
|
||||
|
||||
# D-012: two on-disk pools.
|
||||
blob_root: Path = Path("/var/lib/mimic/blobs")
|
||||
evidence_root: Path = Path("/var/lib/mimic/evidence")
|
||||
|
||||
@field_validator("cors_origins", mode="before")
|
||||
@classmethod
|
||||
def _split_cors(cls, value: object) -> object:
|
||||
if isinstance(value, str):
|
||||
return [origin.strip() for origin in value.split(",") if origin.strip()]
|
||||
return value
|
||||
|
||||
@property
|
||||
def is_production(self) -> bool:
|
||||
return self.env == "production"
|
||||
|
||||
@property
|
||||
def is_testing(self) -> bool:
|
||||
return self.env == "testing"
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def get_settings() -> Settings:
|
||||
return Settings()
|
||||
27
backend/src/mimic/connectors/__init__.py
Normal file
27
backend/src/mimic/connectors/__init__.py
Normal file
@@ -0,0 +1,27 @@
|
||||
"""C2Connector abstraction.
|
||||
|
||||
Sprint 0 ships the interface + dataclasses + payload mapping + factory.
|
||||
Concrete `MythicConnector` and `HomeConnector` implementations land after
|
||||
PR1 and PR2 respectively.
|
||||
"""
|
||||
|
||||
from mimic.connectors.base import (
|
||||
C2Connector,
|
||||
Payload,
|
||||
TaskHandle,
|
||||
TaskResult,
|
||||
TaskStatus,
|
||||
UnsupportedPayloadType,
|
||||
)
|
||||
from mimic.connectors.factory import ConnectorFactory, register_connector
|
||||
|
||||
__all__ = [
|
||||
"C2Connector",
|
||||
"ConnectorFactory",
|
||||
"Payload",
|
||||
"TaskHandle",
|
||||
"TaskResult",
|
||||
"TaskStatus",
|
||||
"UnsupportedPayloadType",
|
||||
"register_connector",
|
||||
]
|
||||
122
backend/src/mimic/connectors/base.py
Normal file
122
backend/src/mimic/connectors/base.py
Normal file
@@ -0,0 +1,122 @@
|
||||
"""Abstract C2 connector interface (spec §7).
|
||||
|
||||
The orchestrator calls `execute_task` → polls `get_task_result` every 500 ms
|
||||
until a terminal `TaskStatus`. `stream_task_output` is optional in v1.
|
||||
`cancel_task` backs F6 Abort. `execute_cleanup` runs the resolved Jinja2
|
||||
template against the C2 (spec F15).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import enum
|
||||
from abc import ABC, abstractmethod
|
||||
from collections.abc import Iterator
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from mimic.db.types import C2Type, PayloadType
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from mimic.db.models.host import Host
|
||||
|
||||
|
||||
class UnsupportedPayloadType(RuntimeError): # noqa: N818 (kept for spec wording)
|
||||
"""Raised when the chosen C2 has no native command for a payload kind.
|
||||
|
||||
The name mirrors the exact identifier from spec §7 (`UnsupportedPayloadType`).
|
||||
The trailing `Error` suffix is intentionally omitted to keep the spec link
|
||||
one-to-one.
|
||||
"""
|
||||
|
||||
def __init__(self, c2: C2Type, payload_type: PayloadType) -> None:
|
||||
super().__init__(f"{c2.value} does not support payload_type={payload_type.value}")
|
||||
self.c2 = c2
|
||||
self.payload_type = payload_type
|
||||
|
||||
|
||||
class TaskStatus(enum.StrEnum):
|
||||
"""Terminal and non-terminal task lifecycle states."""
|
||||
|
||||
PENDING = "pending"
|
||||
RUNNING = "running"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
CANCELED = "canceled"
|
||||
|
||||
@property
|
||||
def is_terminal(self) -> bool:
|
||||
return self in {TaskStatus.COMPLETED, TaskStatus.FAILED, TaskStatus.CANCELED}
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class Payload:
|
||||
"""Self-contained payload sent to a C2."""
|
||||
|
||||
payload_type: PayloadType
|
||||
template_text: str
|
||||
params: dict[str, object] = field(default_factory=dict)
|
||||
mimic_run_id: str | None = None
|
||||
is_stealth_variant: bool = False
|
||||
# When True the orchestrator MUST strip the MIMIC marker (NF-OPSEC).
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class TaskHandle:
|
||||
"""Opaque per-connector reference to a started task."""
|
||||
|
||||
c2: C2Type
|
||||
c2_task_id: str
|
||||
host_id: str
|
||||
payload_type: PayloadType
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class TaskResult:
|
||||
status: TaskStatus
|
||||
output_text: str = ""
|
||||
output_blob_ref: str | None = None
|
||||
exit_code: int | None = None
|
||||
error_message: str | None = None
|
||||
|
||||
|
||||
class C2Connector(ABC):
|
||||
"""Abstract base for every C2 backend."""
|
||||
|
||||
name: C2Type
|
||||
|
||||
@abstractmethod
|
||||
def authenticate(self, config: dict[str, object]) -> None:
|
||||
"""Open / refresh the auth context for this connector."""
|
||||
|
||||
@abstractmethod
|
||||
def list_hosts(self, engagement_id: str) -> list[Host]:
|
||||
"""Return hosts known by the C2 for the given engagement."""
|
||||
|
||||
@abstractmethod
|
||||
def execute_task(self, host: Host, payload: Payload) -> TaskHandle:
|
||||
"""Start a task on `host`. MUST NOT block."""
|
||||
|
||||
@abstractmethod
|
||||
def get_task_result(self, handle: TaskHandle) -> TaskResult:
|
||||
"""Poll status (called every 500 ms by the orchestrator)."""
|
||||
|
||||
def stream_task_output(self, handle: TaskHandle) -> Iterator[bytes]:
|
||||
"""Optional v2. Connectors may leave this raising NotImplementedError."""
|
||||
raise NotImplementedError
|
||||
|
||||
@abstractmethod
|
||||
def cancel_task(self, handle: TaskHandle) -> None:
|
||||
"""Abort a running task (F6 Abort)."""
|
||||
|
||||
@abstractmethod
|
||||
def execute_cleanup(
|
||||
self,
|
||||
host: Host,
|
||||
resolved_command: str,
|
||||
params: dict[str, object],
|
||||
) -> TaskResult:
|
||||
"""Run a fully-resolved cleanup command (F15).
|
||||
|
||||
The Jinja2 template is rendered by the orchestrator BEFORE this call;
|
||||
connectors never see template variables.
|
||||
"""
|
||||
53
backend/src/mimic/connectors/factory.py
Normal file
53
backend/src/mimic/connectors/factory.py
Normal file
@@ -0,0 +1,53 @@
|
||||
"""Connector factory keyed on `c2_type`.
|
||||
|
||||
Concrete connectors register themselves at import time via the
|
||||
`@register_connector` decorator. Sprint 0 ships only the interface — no real
|
||||
implementation registers in this codebase yet.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from typing import TypeVar
|
||||
|
||||
from mimic.connectors.base import C2Connector
|
||||
from mimic.db.types import C2Type
|
||||
|
||||
ConnectorClass = type[C2Connector]
|
||||
C = TypeVar("C", bound=C2Connector)
|
||||
|
||||
_REGISTRY: dict[C2Type, ConnectorClass] = {}
|
||||
|
||||
|
||||
def register_connector(c2_type: C2Type) -> Callable[[type[C]], type[C]]:
|
||||
"""Class decorator: register a concrete connector under its C2Type."""
|
||||
|
||||
def _wrap(klass: type[C]) -> type[C]:
|
||||
if c2_type in _REGISTRY:
|
||||
raise RuntimeError(f"connector already registered for {c2_type.value}")
|
||||
_REGISTRY[c2_type] = klass
|
||||
klass.name = c2_type
|
||||
return klass
|
||||
|
||||
return _wrap
|
||||
|
||||
|
||||
class ConnectorFactory:
|
||||
"""Resolves a connector instance for a given C2 type."""
|
||||
|
||||
def __init__(self, config_resolver: Callable[[C2Type], dict[str, object]]):
|
||||
self._resolver = config_resolver
|
||||
|
||||
@staticmethod
|
||||
def registered() -> dict[C2Type, ConnectorClass]:
|
||||
return dict(_REGISTRY)
|
||||
|
||||
def build(self, c2_type: C2Type) -> C2Connector:
|
||||
try:
|
||||
klass = _REGISTRY[c2_type]
|
||||
except KeyError as exc:
|
||||
raise NotImplementedError(f"no connector registered for {c2_type.value}") from exc
|
||||
|
||||
connector = klass()
|
||||
connector.authenticate(self._resolver(c2_type))
|
||||
return connector
|
||||
47
backend/src/mimic/connectors/payload_map.py
Normal file
47
backend/src/mimic/connectors/payload_map.py
Normal file
@@ -0,0 +1,47 @@
|
||||
"""Static `payload_type` → native command mapping per C2 (spec §7 table).
|
||||
|
||||
Concrete connectors consume this map in `execute_task` to translate the
|
||||
neutral `PayloadType` into the right C2 verb. Unmapped combinations raise
|
||||
`UnsupportedPayloadType`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from mimic.connectors.base import UnsupportedPayloadType
|
||||
from mimic.db.types import C2Type, PayloadType
|
||||
|
||||
MYTHIC_MAP: dict[PayloadType, str] = {
|
||||
PayloadType.CMD: "shell",
|
||||
PayloadType.POWERSHELL: "powershell",
|
||||
PayloadType.BOF: "inline-execute",
|
||||
PayloadType.DOTNET_ASSEMBLY: "execute-assembly",
|
||||
PayloadType.DOTNET_EXE: "execute-assembly",
|
||||
PayloadType.PE_EXE: "spawn",
|
||||
PayloadType.PE_DLL: "loadlibrary",
|
||||
PayloadType.SHELLCODE: "inject",
|
||||
PayloadType.PYTHON: "python",
|
||||
PayloadType.VBS: "shell",
|
||||
PayloadType.WMI_QUERY: "wmi_query",
|
||||
PayloadType.REGISTRY: "reg",
|
||||
PayloadType.SCRIPT_FILE: "upload_and_exec",
|
||||
}
|
||||
|
||||
# Home connector mapping is TBD (PR2). Empty dict = nothing supported yet.
|
||||
HOME_MAP: dict[PayloadType, str] = {}
|
||||
|
||||
_BY_C2: dict[C2Type, dict[PayloadType, str]] = {
|
||||
C2Type.MYTHIC: MYTHIC_MAP,
|
||||
C2Type.HOME: HOME_MAP,
|
||||
}
|
||||
|
||||
|
||||
def resolve_native(c2: C2Type, payload_type: PayloadType) -> str:
|
||||
"""Resolve the native command for a (c2, payload_type) pair."""
|
||||
mapping = _BY_C2.get(c2, {})
|
||||
if payload_type not in mapping:
|
||||
raise UnsupportedPayloadType(c2, payload_type)
|
||||
return mapping[payload_type]
|
||||
|
||||
|
||||
def supports(c2: C2Type, payload_type: PayloadType) -> bool:
|
||||
return payload_type in _BY_C2.get(c2, {})
|
||||
5
backend/src/mimic/db/__init__.py
Normal file
5
backend/src/mimic/db/__init__.py
Normal file
@@ -0,0 +1,5 @@
|
||||
"""Database layer: SQLAlchemy 2 declarative base, models, repositories."""
|
||||
|
||||
from mimic.db.base import Base
|
||||
|
||||
__all__ = ["Base"]
|
||||
59
backend/src/mimic/db/base.py
Normal file
59
backend/src/mimic/db/base.py
Normal file
@@ -0,0 +1,59 @@
|
||||
"""Declarative base + shared mixins for all ORM models."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from sqlalchemy import DateTime, MetaData, func
|
||||
from sqlalchemy.dialects.postgresql import UUID as PG_UUID
|
||||
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
|
||||
|
||||
NAMING_CONVENTION = {
|
||||
"ix": "ix_%(column_0_label)s",
|
||||
"uq": "uq_%(table_name)s_%(column_0_name)s",
|
||||
"ck": "ck_%(table_name)s_%(constraint_name)s",
|
||||
"fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
|
||||
"pk": "pk_%(table_name)s",
|
||||
}
|
||||
|
||||
|
||||
class Base(DeclarativeBase):
|
||||
"""Project-wide declarative base.
|
||||
|
||||
UUID columns are declared explicitly on each model via `PG_UUID(as_uuid=True)`
|
||||
rather than through a `type_annotation_map` — Flask-SQLAlchemy injects its
|
||||
own registry which is incompatible with per-base annotation maps.
|
||||
"""
|
||||
|
||||
metadata = MetaData(naming_convention=NAMING_CONVENTION)
|
||||
|
||||
|
||||
class UuidPkMixin:
|
||||
"""Mixin: UUID v4 primary key generated client-side."""
|
||||
|
||||
id: Mapped[uuid.UUID] = mapped_column(
|
||||
PG_UUID(as_uuid=True),
|
||||
primary_key=True,
|
||||
default=uuid.uuid4,
|
||||
)
|
||||
|
||||
|
||||
def _utcnow() -> datetime:
|
||||
return datetime.now(tz=UTC)
|
||||
|
||||
|
||||
class TimestampsMixin:
|
||||
"""Mixin: `created_at` / `updated_at` columns, UTC timezone-aware."""
|
||||
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
server_default=func.now(),
|
||||
nullable=False,
|
||||
)
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
server_default=func.now(),
|
||||
onupdate=_utcnow,
|
||||
nullable=False,
|
||||
)
|
||||
58
backend/src/mimic/db/migrations/env.py
Normal file
58
backend/src/mimic/db/migrations/env.py
Normal file
@@ -0,0 +1,58 @@
|
||||
"""Alembic environment.
|
||||
|
||||
We import the SQLAlchemy `Base.metadata` directly so migrations are decoupled
|
||||
from the Flask app object (Alembic can run in CI without spinning a request
|
||||
context).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from logging.config import fileConfig
|
||||
|
||||
from alembic import context
|
||||
from sqlalchemy import engine_from_config, pool
|
||||
|
||||
from mimic.config import get_settings
|
||||
from mimic.db.base import Base
|
||||
from mimic.db.models import * # noqa: F403 (ensures all tables register)
|
||||
|
||||
config = context.config
|
||||
|
||||
if config.config_file_name is not None:
|
||||
fileConfig(config.config_file_name)
|
||||
log = logging.getLogger("alembic.env")
|
||||
|
||||
target_metadata = Base.metadata
|
||||
|
||||
settings = get_settings()
|
||||
config.set_main_option("sqlalchemy.url", settings.database_url)
|
||||
|
||||
|
||||
def run_migrations_offline() -> None:
|
||||
context.configure(
|
||||
url=config.get_main_option("sqlalchemy.url"),
|
||||
target_metadata=target_metadata,
|
||||
literal_binds=True,
|
||||
dialect_opts={"paramstyle": "named"},
|
||||
)
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
|
||||
|
||||
def run_migrations_online() -> None:
|
||||
connectable = engine_from_config(
|
||||
config.get_section(config.config_ini_section, {}),
|
||||
prefix="sqlalchemy.",
|
||||
poolclass=pool.NullPool,
|
||||
)
|
||||
with connectable.connect() as connection:
|
||||
context.configure(connection=connection, target_metadata=target_metadata)
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
|
||||
|
||||
if context.is_offline_mode():
|
||||
run_migrations_offline()
|
||||
else:
|
||||
run_migrations_online()
|
||||
27
backend/src/mimic/db/migrations/script.py.mako
Normal file
27
backend/src/mimic/db/migrations/script.py.mako
Normal file
@@ -0,0 +1,27 @@
|
||||
"""${message}
|
||||
|
||||
Revision ID: ${up_revision}
|
||||
Revises: ${down_revision | comma,n}
|
||||
Create Date: ${create_date}
|
||||
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Sequence
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
${imports if imports else ""}
|
||||
|
||||
revision: str = ${repr(up_revision)}
|
||||
down_revision: str | Sequence[str] | None = ${repr(down_revision)}
|
||||
branch_labels: str | Sequence[str] | None = ${repr(branch_labels)}
|
||||
depends_on: str | Sequence[str] | None = ${repr(depends_on)}
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
${upgrades if upgrades else "pass"}
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
${downgrades if downgrades else "pass"}
|
||||
@@ -0,0 +1,749 @@
|
||||
"""initial schema (sprint 0, §8 spec)
|
||||
|
||||
Creates every aggregate listed in spec §8: engagement, user/group RBAC,
|
||||
host, ttp (no ttp_version — H32/D-009), scenario/scenario_step,
|
||||
run/run_step/run_step_cleanup, detection, evidence, report, soc_session,
|
||||
c2_credential, audit_log.
|
||||
|
||||
Postgres-only objects:
|
||||
- ENUM types created via SQLAlchemy `Enum(..., create_type=True)`.
|
||||
- `audit_log` SQL-level append-only via `mimic_audit_writer` role grants
|
||||
(idempotent, no-op if role missing — deployment playbook is authoritative).
|
||||
|
||||
Revision ID: 202605210001
|
||||
Revises:
|
||||
Create Date: 2026-05-21
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy.dialects.postgresql import JSONB, UUID
|
||||
|
||||
revision: str = "202605210001"
|
||||
down_revision: str | None = None
|
||||
branch_labels: str | None = None
|
||||
depends_on: str | None = None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Enums (Postgres CREATE TYPE — done up-front so every table reuses the type)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
USER_TYPE = sa.Enum("rt_operator", "rt_lead", "soc_analyst", name="user_type")
|
||||
ENGAGEMENT_STATUS = sa.Enum("draft", "active", "closed", "archived", name="engagement_status")
|
||||
C2_TYPE = sa.Enum("mythic", "home", name="c2_type")
|
||||
HOST_STATUS = sa.Enum("unknown", "alive", "dead", name="host_status")
|
||||
PAYLOAD_TYPE = sa.Enum(
|
||||
"cmd",
|
||||
"powershell",
|
||||
"bof",
|
||||
"dotnet_assembly",
|
||||
"dotnet_exe",
|
||||
"pe_exe",
|
||||
"pe_dll",
|
||||
"shellcode",
|
||||
"python",
|
||||
"vbs",
|
||||
"wmi_query",
|
||||
"registry",
|
||||
"script_file",
|
||||
name="payload_type",
|
||||
)
|
||||
TTP_SOURCE = sa.Enum("custom", "import_atr", "import_mission", name="ttp_source")
|
||||
RUN_STATUS = sa.Enum(
|
||||
"queued", "running", "paused", "completed", "failed", "aborted", name="run_status"
|
||||
)
|
||||
RUN_STEP_STATUS = sa.Enum(
|
||||
"queued",
|
||||
"running",
|
||||
"completed",
|
||||
"failed",
|
||||
"skipped",
|
||||
"cleanup_failed",
|
||||
name="run_step_status",
|
||||
)
|
||||
CLEANUP_STATUS = sa.Enum("pending", "success", "failed", "partial", name="cleanup_status")
|
||||
DETECTION_LEVEL = sa.Enum("detected", "partial", "not_detected", name="detection_level")
|
||||
DETECTION_SOURCE = sa.Enum("ndr", "edr", "siem", "manual", "other", name="detection_source")
|
||||
EVIDENCE_STATUS = sa.Enum("success", "failure", "partial", name="evidence_status")
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
bind = op.get_bind()
|
||||
for enum_t in (
|
||||
USER_TYPE,
|
||||
ENGAGEMENT_STATUS,
|
||||
C2_TYPE,
|
||||
HOST_STATUS,
|
||||
PAYLOAD_TYPE,
|
||||
TTP_SOURCE,
|
||||
RUN_STATUS,
|
||||
RUN_STEP_STATUS,
|
||||
CLEANUP_STATUS,
|
||||
DETECTION_LEVEL,
|
||||
DETECTION_SOURCE,
|
||||
EVIDENCE_STATUS,
|
||||
):
|
||||
enum_t.create(bind, checkfirst=True)
|
||||
|
||||
# ------------------------------------------------------------------ user
|
||||
op.create_table(
|
||||
"user",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column("type", USER_TYPE, nullable=False),
|
||||
sa.Column("email", sa.String(255), nullable=False),
|
||||
sa.Column("display_name", sa.String(120)),
|
||||
sa.Column("keycloak_sub", sa.String(255), unique=True),
|
||||
sa.Column("local_password_hash", sa.String(255)),
|
||||
sa.Column("disabled_at", sa.DateTime(timezone=True)),
|
||||
sa.Column("last_login_at", sa.DateTime(timezone=True)),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.UniqueConstraint("email", name="uq_user_email"),
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------ permission
|
||||
op.create_table(
|
||||
"permission",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column("code", sa.String(80), nullable=False),
|
||||
sa.Column("description", sa.String(255)),
|
||||
sa.UniqueConstraint("code", name="uq_permission_code"),
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"group",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column("name", sa.String(80), nullable=False),
|
||||
sa.Column("description", sa.String(255)),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.UniqueConstraint("name", name="uq_group_name"),
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"group_permission",
|
||||
sa.Column(
|
||||
"group_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey(
|
||||
"group.id", ondelete="CASCADE", name="fk_group_permission_group_id_group"
|
||||
),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column(
|
||||
"permission_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey(
|
||||
"permission.id",
|
||||
ondelete="CASCADE",
|
||||
name="fk_group_permission_permission_id_permission",
|
||||
),
|
||||
nullable=False,
|
||||
),
|
||||
sa.PrimaryKeyConstraint("group_id", "permission_id", name="pk_group_permission"),
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------ engagement
|
||||
op.create_table(
|
||||
"engagement",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column("client_name", sa.String(255), nullable=False),
|
||||
sa.Column("description", sa.String(1024)),
|
||||
sa.Column("status", ENGAGEMENT_STATUS, nullable=False, server_default="draft"),
|
||||
sa.Column("start_date", sa.Date),
|
||||
sa.Column("end_date", sa.Date),
|
||||
sa.Column("c2_type", C2_TYPE, nullable=False, server_default="mythic"),
|
||||
sa.Column(
|
||||
"created_by_id", UUID(as_uuid=True), sa.ForeignKey("user.id", ondelete="SET NULL")
|
||||
),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"engagement_member",
|
||||
sa.Column(
|
||||
"engagement_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
primary_key=True,
|
||||
),
|
||||
sa.Column(
|
||||
"user_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("user.id", ondelete="CASCADE"),
|
||||
primary_key=True,
|
||||
),
|
||||
sa.Column("role", sa.String(40), nullable=False),
|
||||
sa.Column("added_at", sa.DateTime(timezone=True), nullable=False),
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"user_group",
|
||||
sa.Column(
|
||||
"user_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("user.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column(
|
||||
"group_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("group.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column(
|
||||
"engagement_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
nullable=True,
|
||||
),
|
||||
sa.PrimaryKeyConstraint("user_id", "group_id", "engagement_id", name="pk_user_group"),
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"c2_credential",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column(
|
||||
"engagement_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("c2_type", C2_TYPE, nullable=False),
|
||||
sa.Column("config_fernet", sa.LargeBinary, nullable=False),
|
||||
sa.Column("version", sa.Integer, nullable=False, server_default="1"),
|
||||
sa.Column("retired_at", sa.DateTime(timezone=True)),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
)
|
||||
op.create_index(
|
||||
"ix_c2_credential_engagement_active",
|
||||
"c2_credential",
|
||||
["engagement_id", "c2_type", "version"],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------ host
|
||||
op.create_table(
|
||||
"host",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column(
|
||||
"engagement_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("hostname", sa.String(255), nullable=False),
|
||||
sa.Column("ip", sa.String(64)),
|
||||
sa.Column("os", sa.String(128)),
|
||||
sa.Column("c2_session_id", sa.String(128)),
|
||||
sa.Column("c2_type", C2_TYPE, nullable=False),
|
||||
sa.Column("status", HOST_STATUS, nullable=False, server_default="unknown"),
|
||||
sa.Column("last_seen", sa.DateTime(timezone=True)),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
)
|
||||
op.create_index("ix_host_engagement_id", "host", ["engagement_id"])
|
||||
|
||||
# ------------------------------------------------------------------ ttp
|
||||
op.create_table(
|
||||
"ttp",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column("name", sa.String(255), nullable=False),
|
||||
sa.Column("description", sa.Text),
|
||||
sa.Column("mitre_technique", sa.String(16), nullable=False),
|
||||
sa.Column("mitre_subtechnique", sa.String(16)),
|
||||
sa.Column("payload_type", PAYLOAD_TYPE, nullable=False),
|
||||
sa.Column("payload_template", sa.Text, nullable=False, server_default=""),
|
||||
sa.Column("params_schema_json", JSONB),
|
||||
sa.Column("opsec_notes", sa.Text),
|
||||
sa.Column("cleanup_command", sa.Text),
|
||||
sa.Column("is_stealth_variant", sa.Boolean, nullable=False, server_default=sa.false()),
|
||||
sa.Column("source", TTP_SOURCE, nullable=False, server_default="custom"),
|
||||
sa.Column("tags", JSONB, nullable=False, server_default="[]"),
|
||||
sa.Column("version", sa.Integer, nullable=False, server_default="1"),
|
||||
sa.Column("is_published", sa.Boolean, nullable=False, server_default=sa.false()),
|
||||
sa.Column(
|
||||
"created_by_id", UUID(as_uuid=True), sa.ForeignKey("user.id", ondelete="SET NULL")
|
||||
),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
)
|
||||
# No `ttp_version` table — H32 / D-009: snapshot lives on run.snapshot_json.
|
||||
|
||||
# -------------------------------------------------------------- scenario
|
||||
op.create_table(
|
||||
"scenario",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column(
|
||||
"engagement_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("name", sa.String(255), nullable=False),
|
||||
sa.Column("description", sa.Text),
|
||||
sa.Column("version", sa.Integer, nullable=False, server_default="1"),
|
||||
sa.Column("c2_type", C2_TYPE, nullable=False),
|
||||
sa.Column(
|
||||
"created_by_id", UUID(as_uuid=True), sa.ForeignKey("user.id", ondelete="SET NULL")
|
||||
),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"scenario_step",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column(
|
||||
"scenario_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("scenario.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("order_idx", sa.Integer, nullable=False),
|
||||
sa.Column(
|
||||
"ttp_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("ttp.id", ondelete="RESTRICT"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column(
|
||||
"host_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("host.id", ondelete="RESTRICT"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("params_override_json", JSONB, nullable=False, server_default="{}"),
|
||||
sa.Column("delay_after_ms", sa.Integer, nullable=False, server_default="0"),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.UniqueConstraint("scenario_id", "order_idx", name="uq_scenario_step_order_idx"),
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------- run
|
||||
op.create_table(
|
||||
"run",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column(
|
||||
"scenario_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("scenario.id", ondelete="RESTRICT"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("status", RUN_STATUS, nullable=False, server_default="queued"),
|
||||
sa.Column("started_at", sa.DateTime(timezone=True)),
|
||||
sa.Column("ended_at", sa.DateTime(timezone=True)),
|
||||
sa.Column(
|
||||
"started_by_id", UUID(as_uuid=True), sa.ForeignKey("user.id", ondelete="SET NULL")
|
||||
),
|
||||
sa.Column("snapshot_json", JSONB, nullable=False, server_default="{}"),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"run_step",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column(
|
||||
"run_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("run.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column(
|
||||
"scenario_step_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("scenario_step.id", ondelete="SET NULL"),
|
||||
),
|
||||
sa.Column("order_idx", sa.Integer, nullable=False),
|
||||
sa.Column("status", RUN_STEP_STATUS, nullable=False, server_default="queued"),
|
||||
sa.Column("started_at", sa.DateTime(timezone=True)),
|
||||
sa.Column("ended_at", sa.DateTime(timezone=True)),
|
||||
sa.Column("c2_task_id", sa.String(128)),
|
||||
sa.Column("output_text", sa.Text),
|
||||
sa.Column("output_blob_ref", sa.String(512)),
|
||||
sa.Column("exit_code", sa.Integer),
|
||||
sa.Column("resolved_payload_text", sa.Text),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"run_step_cleanup",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column(
|
||||
"run_step_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("run_step.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
unique=True,
|
||||
),
|
||||
sa.Column("status", CLEANUP_STATUS, nullable=False, server_default="pending"),
|
||||
sa.Column("started_at", sa.DateTime(timezone=True)),
|
||||
sa.Column("ended_at", sa.DateTime(timezone=True)),
|
||||
sa.Column("resolved_command_text", sa.Text),
|
||||
sa.Column("output", sa.Text),
|
||||
sa.Column(
|
||||
"executed_by_id", UUID(as_uuid=True), sa.ForeignKey("user.id", ondelete="SET NULL")
|
||||
),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
)
|
||||
|
||||
# ----------------------------------------------------- detection / evidence
|
||||
op.create_table(
|
||||
"detection",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column(
|
||||
"run_step_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("run_step.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column(
|
||||
"soc_user_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("user.id", ondelete="RESTRICT"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("level", DETECTION_LEVEL, nullable=False),
|
||||
sa.Column("source", DETECTION_SOURCE, nullable=False),
|
||||
sa.Column("latency_ms", sa.Integer),
|
||||
sa.Column("comment", sa.Text),
|
||||
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"evidence",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column(
|
||||
"run_step_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("run_step.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column(
|
||||
"rt_user_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("user.id", ondelete="RESTRICT"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("status", EVIDENCE_STATUS, nullable=False),
|
||||
sa.Column("artifacts_text", sa.Text),
|
||||
sa.Column("artifact_files_json", JSONB, nullable=False, server_default="[]"),
|
||||
sa.Column("comment", sa.Text),
|
||||
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
)
|
||||
|
||||
# ----------------------------------------------------------------- report
|
||||
op.create_table(
|
||||
"report",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column(
|
||||
"engagement_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("version", sa.Integer, nullable=False, server_default="1"),
|
||||
sa.Column("content_json", JSONB, nullable=False, server_default="{}"),
|
||||
sa.Column("content_sha256", sa.String(64), nullable=False),
|
||||
sa.Column("pdf_path", sa.String(512)),
|
||||
sa.Column("md_path", sa.String(512)),
|
||||
sa.Column("generated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column(
|
||||
"generated_by_id", UUID(as_uuid=True), sa.ForeignKey("user.id", ondelete="SET NULL")
|
||||
),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------- soc_session
|
||||
op.create_table(
|
||||
"soc_session",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column(
|
||||
"user_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("user.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column(
|
||||
"engagement_id",
|
||||
UUID(as_uuid=True),
|
||||
sa.ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("token_hash", sa.String(255), nullable=False, unique=True),
|
||||
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("revoked_at", sa.DateTime(timezone=True)),
|
||||
sa.Column("last_ip", sa.String(64)),
|
||||
sa.Column("last_user_agent", sa.String(512)),
|
||||
sa.Column("last_used_at", sa.DateTime(timezone=True)),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
sa.Column(
|
||||
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
|
||||
),
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------ audit_log
|
||||
op.create_table(
|
||||
"audit_log",
|
||||
sa.Column("id", UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column("ts", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
|
||||
sa.Column("actor_id", UUID(as_uuid=True), sa.ForeignKey("user.id", ondelete="SET NULL")),
|
||||
sa.Column("action", sa.String(80), nullable=False),
|
||||
sa.Column("resource_type", sa.String(80), nullable=False),
|
||||
sa.Column("resource_id", sa.String(128)),
|
||||
sa.Column("metadata_json", JSONB, nullable=False, server_default="{}"),
|
||||
sa.Column("prev_hash", sa.String(64)),
|
||||
sa.Column("row_hash", sa.String(64), nullable=False),
|
||||
sa.Column("source_ip", sa.String(64)),
|
||||
sa.Column("user_agent", sa.String(512)),
|
||||
sa.Column("comment", sa.Text),
|
||||
)
|
||||
op.create_index("ix_audit_log_ts", "audit_log", ["ts"])
|
||||
op.create_index("ix_audit_log_resource", "audit_log", ["resource_type", "resource_id"])
|
||||
|
||||
# ---------------------------------------------- NF-AUDIT role-level grants
|
||||
# `mimic_audit_writer` (write-only) + `mimic_app` (read-only on audit_log).
|
||||
# Idempotent: skip silently if roles are absent (dev/test boxes).
|
||||
bind.exec_driver_sql(
|
||||
"""
|
||||
DO $do$
|
||||
BEGIN
|
||||
IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'mimic_audit_writer') THEN
|
||||
GRANT INSERT ON TABLE audit_log TO mimic_audit_writer;
|
||||
REVOKE UPDATE, DELETE, TRUNCATE ON TABLE audit_log FROM mimic_audit_writer;
|
||||
END IF;
|
||||
IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = current_user) THEN
|
||||
REVOKE UPDATE, DELETE, TRUNCATE ON TABLE audit_log FROM PUBLIC;
|
||||
END IF;
|
||||
END
|
||||
$do$;
|
||||
"""
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------- seed RBAC
|
||||
# D-008: exactly the 3 F11 groups, with exactly the F11 permission matrix.
|
||||
# MA3: the matrix is frozen *inline* here so this migration produces the
|
||||
# same schema regardless of future drifts in `mimic.rbac.matrix`. A unit
|
||||
# test (`test_migration_seed_matches_current_matrix`) guards the
|
||||
# invariant — if the runtime matrix diverges, write a new migration.
|
||||
_seed_rbac()
|
||||
|
||||
|
||||
# --- Frozen F11 matrix snapshot (MA3, do not edit -- write a new migration).
|
||||
_GROUP_DESCRIPTIONS: dict[str, str] = {
|
||||
"rt_operator": "Red team operator (per-engagement scope).",
|
||||
"rt_lead": "Red team lead (full RT privileges).",
|
||||
"soc_analyst": "SOC analyst (scoped via soc_session).",
|
||||
}
|
||||
|
||||
# Frozen permission codes (D-008 canonical list).
|
||||
_PERMISSIONS_FROZEN: tuple[str, ...] = (
|
||||
"engagement.create",
|
||||
"engagement.read",
|
||||
"engagement.read_own",
|
||||
"engagement.update",
|
||||
"engagement.delete",
|
||||
"engagement.member.manage",
|
||||
"engagement.soc_token.issue",
|
||||
"host.crud",
|
||||
"ttp.read",
|
||||
"ttp.draft",
|
||||
"ttp.promote",
|
||||
"import.journal",
|
||||
"scenario.crud",
|
||||
"run.start",
|
||||
"run.control",
|
||||
"evidence.add",
|
||||
"detection.add",
|
||||
"cleanup.trigger",
|
||||
"report.generate",
|
||||
"report.read",
|
||||
"audit.read",
|
||||
)
|
||||
|
||||
# Frozen group → permissions assignments (D-008 / F11).
|
||||
_GROUP_PERMISSIONS_FROZEN: dict[str, frozenset[str]] = {
|
||||
"rt_operator": frozenset(
|
||||
{
|
||||
"engagement.create",
|
||||
"engagement.read",
|
||||
"host.crud",
|
||||
"ttp.read",
|
||||
"ttp.draft",
|
||||
"import.journal",
|
||||
"scenario.crud",
|
||||
"evidence.add",
|
||||
"cleanup.trigger",
|
||||
"report.read",
|
||||
}
|
||||
),
|
||||
"rt_lead": frozenset(_PERMISSIONS_FROZEN),
|
||||
"soc_analyst": frozenset(
|
||||
{
|
||||
"engagement.read_own",
|
||||
"detection.add",
|
||||
"report.read",
|
||||
}
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def _seed_rbac() -> None:
|
||||
"""Seed `permission` + `group` + `group_permission` from the frozen
|
||||
inline F11 matrix above (D-008 / MA3)."""
|
||||
from uuid import NAMESPACE_DNS, uuid5 # noqa: PLC0415
|
||||
|
||||
def _gid(name: str) -> str:
|
||||
return str(uuid5(NAMESPACE_DNS, f"mimic.group.{name}"))
|
||||
|
||||
def _pid(code: str) -> str:
|
||||
return str(uuid5(NAMESPACE_DNS, f"mimic.permission.{code}"))
|
||||
|
||||
permission_table = sa.table(
|
||||
"permission",
|
||||
sa.column("id", UUID(as_uuid=True)),
|
||||
sa.column("code", sa.String),
|
||||
sa.column("description", sa.String),
|
||||
)
|
||||
op.bulk_insert(
|
||||
permission_table,
|
||||
[{"id": _pid(code), "code": code, "description": None} for code in _PERMISSIONS_FROZEN],
|
||||
)
|
||||
|
||||
group_table = sa.table(
|
||||
"group",
|
||||
sa.column("id", UUID(as_uuid=True)),
|
||||
sa.column("name", sa.String),
|
||||
sa.column("description", sa.String),
|
||||
)
|
||||
op.bulk_insert(
|
||||
group_table,
|
||||
[
|
||||
{"id": _gid(name), "name": name, "description": _GROUP_DESCRIPTIONS[name]}
|
||||
for name in _GROUP_PERMISSIONS_FROZEN
|
||||
],
|
||||
)
|
||||
|
||||
group_permission_table = sa.table(
|
||||
"group_permission",
|
||||
sa.column("group_id", UUID(as_uuid=True)),
|
||||
sa.column("permission_id", UUID(as_uuid=True)),
|
||||
)
|
||||
rows: list[dict[str, str]] = []
|
||||
for group_name, perms in _GROUP_PERMISSIONS_FROZEN.items():
|
||||
for perm in perms:
|
||||
rows.append({"group_id": _gid(group_name), "permission_id": _pid(perm)})
|
||||
op.bulk_insert(group_permission_table, rows)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
for table in (
|
||||
"audit_log",
|
||||
"soc_session",
|
||||
"report",
|
||||
"evidence",
|
||||
"detection",
|
||||
"run_step_cleanup",
|
||||
"run_step",
|
||||
"run",
|
||||
"scenario_step",
|
||||
"scenario",
|
||||
"ttp",
|
||||
"host",
|
||||
"c2_credential",
|
||||
"user_group",
|
||||
"engagement_member",
|
||||
"engagement",
|
||||
"group_permission",
|
||||
"group",
|
||||
"permission",
|
||||
"user",
|
||||
):
|
||||
op.drop_table(table)
|
||||
|
||||
for enum_t in (
|
||||
EVIDENCE_STATUS,
|
||||
DETECTION_SOURCE,
|
||||
DETECTION_LEVEL,
|
||||
CLEANUP_STATUS,
|
||||
RUN_STEP_STATUS,
|
||||
RUN_STATUS,
|
||||
TTP_SOURCE,
|
||||
PAYLOAD_TYPE,
|
||||
HOST_STATUS,
|
||||
C2_TYPE,
|
||||
ENGAGEMENT_STATUS,
|
||||
USER_TYPE,
|
||||
):
|
||||
enum_t.drop(op.get_bind(), checkfirst=True)
|
||||
36
backend/src/mimic/db/models/__init__.py
Normal file
36
backend/src/mimic/db/models/__init__.py
Normal file
@@ -0,0 +1,36 @@
|
||||
"""SQLAlchemy 2 typed mapped classes for every Mimic aggregate."""
|
||||
|
||||
from mimic.db.models.audit import AuditLog
|
||||
from mimic.db.models.detection import Detection, Evidence
|
||||
from mimic.db.models.engagement import C2Credential, Engagement, EngagementMember
|
||||
from mimic.db.models.host import Host
|
||||
from mimic.db.models.permission import Group, GroupPermission, Permission, UserGroup
|
||||
from mimic.db.models.report import Report
|
||||
from mimic.db.models.run import Run, RunStep, RunStepCleanup
|
||||
from mimic.db.models.scenario import Scenario, ScenarioStep
|
||||
from mimic.db.models.soc_session import SocSession
|
||||
from mimic.db.models.ttp import Ttp
|
||||
from mimic.db.models.user import User
|
||||
|
||||
__all__ = [
|
||||
"AuditLog",
|
||||
"C2Credential",
|
||||
"Detection",
|
||||
"Engagement",
|
||||
"EngagementMember",
|
||||
"Evidence",
|
||||
"Group",
|
||||
"GroupPermission",
|
||||
"Host",
|
||||
"Permission",
|
||||
"Report",
|
||||
"Run",
|
||||
"RunStep",
|
||||
"RunStepCleanup",
|
||||
"Scenario",
|
||||
"ScenarioStep",
|
||||
"SocSession",
|
||||
"Ttp",
|
||||
"User",
|
||||
"UserGroup",
|
||||
]
|
||||
44
backend/src/mimic/db/models/audit.py
Normal file
44
backend/src/mimic/db/models/audit.py
Normal file
@@ -0,0 +1,44 @@
|
||||
"""Append-only audit log.
|
||||
|
||||
NF-AUDIT: enforced at SQL level by granting only INSERT to
|
||||
`mimic_audit_writer` and only SELECT to `mimic_app`. The ORM never updates or
|
||||
deletes rows on this table from application code. Hash chain (`prev_hash` /
|
||||
`row_hash`) is wired here at the schema level so v2 can switch to a strict
|
||||
WORM enforcement without a migration; sprint 0 fills the columns at insert.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import JSON, DateTime, ForeignKey, String, Text, func
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from mimic.db.base import Base, UuidPkMixin
|
||||
|
||||
|
||||
class AuditLog(UuidPkMixin, Base):
|
||||
__tablename__ = "audit_log"
|
||||
|
||||
ts: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
nullable=False,
|
||||
server_default=func.now(),
|
||||
)
|
||||
actor_id: Mapped[UUID | None] = mapped_column(ForeignKey("user.id", ondelete="SET NULL"))
|
||||
action: Mapped[str] = mapped_column(String(80), nullable=False)
|
||||
resource_type: Mapped[str] = mapped_column(String(80), nullable=False)
|
||||
resource_id: Mapped[str | None] = mapped_column(String(128))
|
||||
metadata_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict)
|
||||
|
||||
prev_hash: Mapped[str | None] = mapped_column(String(64))
|
||||
row_hash: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
# SHA-256 of canonical (prev_hash || ts || actor_id || action ||
|
||||
# resource_type || resource_id || metadata_json). Provides hash-chain
|
||||
# integrity once verifier runs in v2.
|
||||
|
||||
source_ip: Mapped[str | None] = mapped_column(String(64))
|
||||
user_agent: Mapped[str | None] = mapped_column(String(512))
|
||||
comment: Mapped[str | None] = mapped_column(Text)
|
||||
78
backend/src/mimic/db/models/detection.py
Normal file
78
backend/src/mimic/db/models/detection.py
Normal file
@@ -0,0 +1,78 @@
|
||||
"""Per-run-step detection (SOC) and offensive evidence (RT)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import (
|
||||
JSON,
|
||||
DateTime,
|
||||
Enum,
|
||||
ForeignKey,
|
||||
Integer,
|
||||
Text,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from mimic.db.base import Base, TimestampsMixin, UuidPkMixin
|
||||
from mimic.db.types import DetectionLevel, DetectionSource, EvidenceStatus
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from mimic.db.models.run import RunStep
|
||||
from mimic.db.models.user import User
|
||||
|
||||
|
||||
class Detection(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "detection"
|
||||
|
||||
run_step_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("run_step.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
soc_user_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("user.id", ondelete="RESTRICT"),
|
||||
nullable=False,
|
||||
)
|
||||
level: Mapped[DetectionLevel] = mapped_column(
|
||||
Enum(DetectionLevel, name="detection_level"),
|
||||
nullable=False,
|
||||
)
|
||||
source: Mapped[DetectionSource] = mapped_column(
|
||||
Enum(DetectionSource, name="detection_source"),
|
||||
nullable=False,
|
||||
)
|
||||
latency_ms: Mapped[int | None] = mapped_column(Integer)
|
||||
comment: Mapped[str | None] = mapped_column(Text)
|
||||
recorded_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
run_step: Mapped[RunStep] = relationship()
|
||||
soc_user: Mapped[User] = relationship()
|
||||
|
||||
|
||||
class Evidence(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "evidence"
|
||||
|
||||
run_step_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("run_step.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
rt_user_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("user.id", ondelete="RESTRICT"),
|
||||
nullable=False,
|
||||
)
|
||||
status: Mapped[EvidenceStatus] = mapped_column(
|
||||
Enum(EvidenceStatus, name="evidence_status"),
|
||||
nullable=False,
|
||||
)
|
||||
artifacts_text: Mapped[str | None] = mapped_column(Text)
|
||||
artifact_files_json: Mapped[list[dict[str, Any]]] = mapped_column(
|
||||
JSON, default=list, nullable=False
|
||||
)
|
||||
# Each entry: {"name": str, "ref": str, "sha256": str, "size_bytes": int}
|
||||
comment: Mapped[str | None] = mapped_column(Text)
|
||||
recorded_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
run_step: Mapped[RunStep] = relationship()
|
||||
rt_user: Mapped[User] = relationship()
|
||||
104
backend/src/mimic/db/models/engagement.py
Normal file
104
backend/src/mimic/db/models/engagement.py
Normal file
@@ -0,0 +1,104 @@
|
||||
"""Engagement aggregate: tenancy container + C2 credentials."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime
|
||||
from typing import TYPE_CHECKING
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import Date, DateTime, Enum, ForeignKey, Integer, LargeBinary, String, func
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from mimic.db.base import Base, TimestampsMixin, UuidPkMixin
|
||||
from mimic.db.types import C2Type, EngagementStatus
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from mimic.db.models.host import Host
|
||||
from mimic.db.models.scenario import Scenario
|
||||
from mimic.db.models.user import User
|
||||
|
||||
|
||||
class Engagement(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "engagement"
|
||||
|
||||
client_name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
description: Mapped[str | None] = mapped_column(String(1024))
|
||||
status: Mapped[EngagementStatus] = mapped_column(
|
||||
Enum(EngagementStatus, name="engagement_status"),
|
||||
default=EngagementStatus.DRAFT,
|
||||
nullable=False,
|
||||
)
|
||||
start_date: Mapped[date | None] = mapped_column(Date)
|
||||
end_date: Mapped[date | None] = mapped_column(Date)
|
||||
|
||||
c2_type: Mapped[C2Type] = mapped_column(
|
||||
Enum(C2Type, name="c2_type"),
|
||||
default=C2Type.MYTHIC,
|
||||
nullable=False,
|
||||
)
|
||||
|
||||
created_by_id: Mapped[UUID | None] = mapped_column(ForeignKey("user.id", ondelete="SET NULL"))
|
||||
|
||||
hosts: Mapped[list[Host]] = relationship(
|
||||
back_populates="engagement",
|
||||
cascade="all, delete-orphan",
|
||||
)
|
||||
scenarios: Mapped[list[Scenario]] = relationship(
|
||||
back_populates="engagement",
|
||||
cascade="all, delete-orphan",
|
||||
)
|
||||
members: Mapped[list[EngagementMember]] = relationship(
|
||||
back_populates="engagement",
|
||||
cascade="all, delete-orphan",
|
||||
)
|
||||
c2_credentials: Mapped[list[C2Credential]] = relationship(
|
||||
back_populates="engagement",
|
||||
cascade="all, delete-orphan",
|
||||
)
|
||||
|
||||
|
||||
class EngagementMember(Base):
|
||||
__tablename__ = "engagement_member"
|
||||
|
||||
engagement_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
primary_key=True,
|
||||
)
|
||||
user_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("user.id", ondelete="CASCADE"),
|
||||
primary_key=True,
|
||||
)
|
||||
role: Mapped[str] = mapped_column(String(40), nullable=False)
|
||||
added_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
server_default=func.now(),
|
||||
nullable=False,
|
||||
)
|
||||
|
||||
engagement: Mapped[Engagement] = relationship(back_populates="members")
|
||||
user: Mapped[User] = relationship()
|
||||
|
||||
|
||||
class C2Credential(UuidPkMixin, TimestampsMixin, Base):
|
||||
"""Per-engagement encrypted C2 credentials.
|
||||
|
||||
Decision D-004: dedicated table, Fernet-encrypted blob, versioned, rotation
|
||||
via insert + retire. Active row = `retired_at IS NULL` with the highest
|
||||
`version` for `(engagement_id, c2_type)`.
|
||||
"""
|
||||
|
||||
__tablename__ = "c2_credential"
|
||||
|
||||
engagement_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
c2_type: Mapped[C2Type] = mapped_column(
|
||||
Enum(C2Type, name="c2_type", create_type=False),
|
||||
nullable=False,
|
||||
)
|
||||
config_fernet: Mapped[bytes] = mapped_column(LargeBinary, nullable=False)
|
||||
version: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
|
||||
retired_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
engagement: Mapped[Engagement] = relationship(back_populates="c2_credentials")
|
||||
41
backend/src/mimic/db/models/host.py
Normal file
41
backend/src/mimic/db/models/host.py
Normal file
@@ -0,0 +1,41 @@
|
||||
"""Host inventory (per-engagement target machines)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import DateTime, Enum, ForeignKey, String
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from mimic.db.base import Base, TimestampsMixin, UuidPkMixin
|
||||
from mimic.db.types import C2Type, HostStatus
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from mimic.db.models.engagement import Engagement
|
||||
|
||||
|
||||
class Host(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "host"
|
||||
|
||||
engagement_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
hostname: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
ip: Mapped[str | None] = mapped_column(String(64))
|
||||
os: Mapped[str | None] = mapped_column(String(128))
|
||||
c2_session_id: Mapped[str | None] = mapped_column(String(128))
|
||||
c2_type: Mapped[C2Type] = mapped_column(
|
||||
Enum(C2Type, name="c2_type", create_type=False),
|
||||
nullable=False,
|
||||
)
|
||||
status: Mapped[HostStatus] = mapped_column(
|
||||
Enum(HostStatus, name="host_status"),
|
||||
default=HostStatus.UNKNOWN,
|
||||
nullable=False,
|
||||
)
|
||||
last_seen: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
engagement: Mapped[Engagement] = relationship(back_populates="hosts")
|
||||
83
backend/src/mimic/db/models/permission.py
Normal file
83
backend/src/mimic/db/models/permission.py
Normal file
@@ -0,0 +1,83 @@
|
||||
"""Group-based RBAC tables.
|
||||
|
||||
Decision D-003 (spec-decisions): group-based from day one so OIDC will map
|
||||
claims to existing groups in v2 without touching application code.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import ForeignKey, PrimaryKeyConstraint, String, UniqueConstraint
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from mimic.db.base import Base, TimestampsMixin, UuidPkMixin
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from mimic.db.models.engagement import Engagement
|
||||
from mimic.db.models.user import User
|
||||
|
||||
|
||||
class Permission(UuidPkMixin, Base):
|
||||
__tablename__ = "permission"
|
||||
__table_args__ = (UniqueConstraint("code", name="uq_permission_code"),)
|
||||
|
||||
code: Mapped[str] = mapped_column(String(80), nullable=False)
|
||||
description: Mapped[str | None] = mapped_column(String(255))
|
||||
|
||||
|
||||
class Group(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "group"
|
||||
__table_args__ = (UniqueConstraint("name", name="uq_group_name"),)
|
||||
|
||||
name: Mapped[str] = mapped_column(String(80), nullable=False)
|
||||
description: Mapped[str | None] = mapped_column(String(255))
|
||||
|
||||
permission_links: Mapped[list[GroupPermission]] = relationship(
|
||||
back_populates="group",
|
||||
cascade="all, delete-orphan",
|
||||
)
|
||||
user_links: Mapped[list[UserGroup]] = relationship(
|
||||
back_populates="group",
|
||||
cascade="all, delete-orphan",
|
||||
)
|
||||
|
||||
|
||||
class GroupPermission(Base):
|
||||
__tablename__ = "group_permission"
|
||||
__table_args__ = (
|
||||
PrimaryKeyConstraint("group_id", "permission_id", name="pk_group_permission"),
|
||||
)
|
||||
|
||||
group_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("group.id", ondelete="CASCADE"), nullable=False
|
||||
)
|
||||
permission_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("permission.id", ondelete="CASCADE"), nullable=False
|
||||
)
|
||||
|
||||
group: Mapped[Group] = relationship(back_populates="permission_links")
|
||||
permission: Mapped[Permission] = relationship()
|
||||
|
||||
|
||||
class UserGroup(Base):
|
||||
__tablename__ = "user_group"
|
||||
__table_args__ = (
|
||||
PrimaryKeyConstraint("user_id", "group_id", "engagement_id", name="pk_user_group"),
|
||||
)
|
||||
|
||||
user_id: Mapped[UUID] = mapped_column(ForeignKey("user.id", ondelete="CASCADE"), nullable=False)
|
||||
group_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("group.id", ondelete="CASCADE"), nullable=False
|
||||
)
|
||||
engagement_id: Mapped[UUID | None] = mapped_column(
|
||||
ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
nullable=True,
|
||||
)
|
||||
# Global membership (e.g. rt_lead enterprise-wide) when engagement_id IS NULL.
|
||||
# Per-engagement membership otherwise.
|
||||
|
||||
user: Mapped[User] = relationship(back_populates="group_links")
|
||||
group: Mapped[Group] = relationship(back_populates="user_links")
|
||||
engagement: Mapped[Engagement | None] = relationship()
|
||||
44
backend/src/mimic/db/models/report.py
Normal file
44
backend/src/mimic/db/models/report.py
Normal file
@@ -0,0 +1,44 @@
|
||||
"""Mission report (PDF / JSON / Markdown bundle)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import (
|
||||
JSON,
|
||||
DateTime,
|
||||
ForeignKey,
|
||||
Integer,
|
||||
String,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from mimic.db.base import Base, TimestampsMixin, UuidPkMixin
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from mimic.db.models.engagement import Engagement
|
||||
|
||||
|
||||
class Report(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "report"
|
||||
|
||||
engagement_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
version: Mapped[int] = mapped_column(Integer, default=1, nullable=False)
|
||||
|
||||
content_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict)
|
||||
content_sha256: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
# SHA-256 of canonical JSON. Identical hash referenced in PDF footer, JSON
|
||||
# export and Markdown export (spec H19 / F9 / F14).
|
||||
|
||||
pdf_path: Mapped[str | None] = mapped_column(String(512))
|
||||
md_path: Mapped[str | None] = mapped_column(String(512))
|
||||
|
||||
generated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
generated_by_id: Mapped[UUID | None] = mapped_column(ForeignKey("user.id", ondelete="SET NULL"))
|
||||
|
||||
engagement: Mapped[Engagement] = relationship()
|
||||
114
backend/src/mimic/db/models/run.py
Normal file
114
backend/src/mimic/db/models/run.py
Normal file
@@ -0,0 +1,114 @@
|
||||
"""Run aggregate: scenario execution + per-step state + cleanup."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import (
|
||||
JSON,
|
||||
DateTime,
|
||||
Enum,
|
||||
ForeignKey,
|
||||
Integer,
|
||||
String,
|
||||
Text,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from mimic.db.base import Base, TimestampsMixin, UuidPkMixin
|
||||
from mimic.db.types import CleanupStatus, RunStatus, RunStepStatus
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from mimic.db.models.scenario import Scenario, ScenarioStep
|
||||
|
||||
|
||||
class Run(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "run"
|
||||
|
||||
scenario_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("scenario.id", ondelete="RESTRICT"),
|
||||
nullable=False,
|
||||
)
|
||||
status: Mapped[RunStatus] = mapped_column(
|
||||
Enum(RunStatus, name="run_status"),
|
||||
default=RunStatus.QUEUED,
|
||||
nullable=False,
|
||||
)
|
||||
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
ended_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
started_by_id: Mapped[UUID | None] = mapped_column(ForeignKey("user.id", ondelete="SET NULL"))
|
||||
|
||||
snapshot_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict)
|
||||
# Full self-contained snapshot of scenario + steps + resolved TTPs.
|
||||
# Source of truth for replay (spec H32).
|
||||
|
||||
scenario: Mapped[Scenario] = relationship(back_populates="runs")
|
||||
steps: Mapped[list[RunStep]] = relationship(
|
||||
back_populates="run",
|
||||
cascade="all, delete-orphan",
|
||||
order_by="RunStep.order_idx",
|
||||
)
|
||||
|
||||
|
||||
class RunStep(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "run_step"
|
||||
|
||||
run_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("run.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
scenario_step_id: Mapped[UUID | None] = mapped_column(
|
||||
ForeignKey("scenario_step.id", ondelete="SET NULL")
|
||||
)
|
||||
order_idx: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
|
||||
status: Mapped[RunStepStatus] = mapped_column(
|
||||
Enum(RunStepStatus, name="run_step_status"),
|
||||
default=RunStepStatus.QUEUED,
|
||||
nullable=False,
|
||||
)
|
||||
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
ended_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
c2_task_id: Mapped[str | None] = mapped_column(String(128))
|
||||
output_text: Mapped[str | None] = mapped_column(Text)
|
||||
output_blob_ref: Mapped[str | None] = mapped_column(String(512))
|
||||
exit_code: Mapped[int | None] = mapped_column(Integer)
|
||||
|
||||
resolved_payload_text: Mapped[str | None] = mapped_column(Text)
|
||||
# The fully-resolved payload (post-Jinja) actually sent to the C2. Useful
|
||||
# for audit / report. Marker (`# MIMIC-RUN:<id>`) is added here.
|
||||
|
||||
run: Mapped[Run] = relationship(back_populates="steps")
|
||||
scenario_step: Mapped[ScenarioStep | None] = relationship()
|
||||
|
||||
cleanup: Mapped[RunStepCleanup | None] = relationship(
|
||||
back_populates="run_step",
|
||||
cascade="all, delete-orphan",
|
||||
uselist=False,
|
||||
)
|
||||
|
||||
|
||||
class RunStepCleanup(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "run_step_cleanup"
|
||||
|
||||
run_step_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("run_step.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
unique=True,
|
||||
)
|
||||
status: Mapped[CleanupStatus] = mapped_column(
|
||||
Enum(CleanupStatus, name="cleanup_status"),
|
||||
default=CleanupStatus.PENDING,
|
||||
nullable=False,
|
||||
)
|
||||
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
ended_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
resolved_command_text: Mapped[str | None] = mapped_column(Text)
|
||||
output: Mapped[str | None] = mapped_column(Text)
|
||||
executed_by_id: Mapped[UUID | None] = mapped_column(ForeignKey("user.id", ondelete="SET NULL"))
|
||||
|
||||
run_step: Mapped[RunStep] = relationship(back_populates="cleanup")
|
||||
84
backend/src/mimic/db/models/scenario.py
Normal file
84
backend/src/mimic/db/models/scenario.py
Normal file
@@ -0,0 +1,84 @@
|
||||
"""Scenario aggregate: ordered list of TTP steps.
|
||||
|
||||
Spec §F3: `scenario.c2_type` is the source of truth for the run. Verified at
|
||||
run start that every referenced host matches.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import (
|
||||
JSON,
|
||||
Enum,
|
||||
ForeignKey,
|
||||
Integer,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from mimic.db.base import Base, TimestampsMixin, UuidPkMixin
|
||||
from mimic.db.types import C2Type
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from mimic.db.models.engagement import Engagement
|
||||
from mimic.db.models.host import Host
|
||||
from mimic.db.models.run import Run
|
||||
from mimic.db.models.ttp import Ttp
|
||||
|
||||
|
||||
class Scenario(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "scenario"
|
||||
|
||||
engagement_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
description: Mapped[str | None] = mapped_column(Text)
|
||||
version: Mapped[int] = mapped_column(Integer, default=1, nullable=False)
|
||||
|
||||
c2_type: Mapped[C2Type] = mapped_column(
|
||||
Enum(C2Type, name="c2_type", create_type=False),
|
||||
nullable=False,
|
||||
)
|
||||
created_by_id: Mapped[UUID | None] = mapped_column(ForeignKey("user.id", ondelete="SET NULL"))
|
||||
|
||||
engagement: Mapped[Engagement] = relationship(back_populates="scenarios")
|
||||
steps: Mapped[list[ScenarioStep]] = relationship(
|
||||
back_populates="scenario",
|
||||
cascade="all, delete-orphan",
|
||||
order_by="ScenarioStep.order_idx",
|
||||
)
|
||||
runs: Mapped[list[Run]] = relationship(back_populates="scenario")
|
||||
|
||||
|
||||
class ScenarioStep(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "scenario_step"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("scenario_id", "order_idx", name="uq_scenario_step_order_idx"),
|
||||
)
|
||||
|
||||
scenario_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("scenario.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
order_idx: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
|
||||
ttp_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("ttp.id", ondelete="RESTRICT"),
|
||||
nullable=False,
|
||||
)
|
||||
host_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("host.id", ondelete="RESTRICT"),
|
||||
nullable=False,
|
||||
)
|
||||
params_override_json: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False)
|
||||
delay_after_ms: Mapped[int] = mapped_column(Integer, default=0, nullable=False)
|
||||
|
||||
scenario: Mapped[Scenario] = relationship(back_populates="steps")
|
||||
ttp: Mapped[Ttp] = relationship()
|
||||
host: Mapped[Host] = relationship()
|
||||
46
backend/src/mimic/db/models/soc_session.py
Normal file
46
backend/src/mimic/db/models/soc_session.py
Normal file
@@ -0,0 +1,46 @@
|
||||
"""SOC analyst sessions (bcrypt-hashed opaque tokens).
|
||||
|
||||
Decision D-006: bcrypt hash stored; the clear token is generated server-side at
|
||||
session creation, returned **once** in the API response and delivered out-of-band.
|
||||
Never re-displayable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import DateTime, ForeignKey, String
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from mimic.db.base import Base, TimestampsMixin, UuidPkMixin
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from mimic.db.models.engagement import Engagement
|
||||
from mimic.db.models.user import User
|
||||
|
||||
|
||||
class SocSession(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "soc_session"
|
||||
|
||||
user_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("user.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
engagement_id: Mapped[UUID] = mapped_column(
|
||||
ForeignKey("engagement.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
token_hash: Mapped[str] = mapped_column(String(255), nullable=False, unique=True)
|
||||
# bcrypt hash. Plain token returned once at creation.
|
||||
|
||||
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
last_ip: Mapped[str | None] = mapped_column(String(64))
|
||||
last_user_agent: Mapped[str | None] = mapped_column(String(512))
|
||||
last_used_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
user: Mapped[User] = relationship(back_populates="soc_sessions")
|
||||
engagement: Mapped[Engagement] = relationship()
|
||||
61
backend/src/mimic/db/models/ttp.py
Normal file
61
backend/src/mimic/db/models/ttp.py
Normal file
@@ -0,0 +1,61 @@
|
||||
"""TTP library.
|
||||
|
||||
Spec H32 / D-009: there is **no** `ttp_version` table. The replayability
|
||||
snapshot lives solely on `run.snapshot_json` (a self-contained JSONB blob
|
||||
captured at run start). `ttp.version` stays here as a purely informational
|
||||
counter (§8) — bumped on edit, never referenced for replay.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import (
|
||||
JSON,
|
||||
Boolean,
|
||||
Enum,
|
||||
ForeignKey,
|
||||
Integer,
|
||||
String,
|
||||
Text,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from mimic.db.base import Base, TimestampsMixin, UuidPkMixin
|
||||
from mimic.db.types import PayloadType, TtpSource
|
||||
|
||||
|
||||
class Ttp(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "ttp"
|
||||
|
||||
name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
description: Mapped[str | None] = mapped_column(Text)
|
||||
|
||||
mitre_technique: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||
mitre_subtechnique: Mapped[str | None] = mapped_column(String(16))
|
||||
|
||||
payload_type: Mapped[PayloadType] = mapped_column(
|
||||
Enum(PayloadType, name="payload_type"),
|
||||
nullable=False,
|
||||
)
|
||||
payload_template: Mapped[str] = mapped_column(Text, nullable=False, default="")
|
||||
params_schema_json: Mapped[dict[str, Any] | None] = mapped_column(JSON)
|
||||
|
||||
opsec_notes: Mapped[str | None] = mapped_column(Text)
|
||||
cleanup_command: Mapped[str | None] = mapped_column(Text)
|
||||
is_stealth_variant: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
|
||||
|
||||
source: Mapped[TtpSource] = mapped_column(
|
||||
Enum(TtpSource, name="ttp_source"),
|
||||
default=TtpSource.CUSTOM,
|
||||
nullable=False,
|
||||
)
|
||||
tags: Mapped[list[str]] = mapped_column(JSON, default=list, nullable=False)
|
||||
|
||||
version: Mapped[int] = mapped_column(Integer, default=1, nullable=False)
|
||||
# Informational (§8). Bumped on edit. Never used for replay (H32 / D-009).
|
||||
is_published: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False)
|
||||
# Promoted to the library (lead RT only — F11).
|
||||
|
||||
created_by_id: Mapped[UUID | None] = mapped_column(ForeignKey("user.id", ondelete="SET NULL"))
|
||||
50
backend/src/mimic/db/models/user.py
Normal file
50
backend/src/mimic/db/models/user.py
Normal file
@@ -0,0 +1,50 @@
|
||||
"""User accounts (RT operators, RT leads, SOC analysts)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import TYPE_CHECKING
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy import DateTime, Enum, String, UniqueConstraint
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from mimic.db.base import Base, TimestampsMixin, UuidPkMixin
|
||||
from mimic.db.types import UserType
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from mimic.db.models.permission import UserGroup
|
||||
from mimic.db.models.soc_session import SocSession
|
||||
|
||||
|
||||
class User(UuidPkMixin, TimestampsMixin, Base):
|
||||
__tablename__ = "user"
|
||||
__table_args__ = (UniqueConstraint("email", name="uq_user_email"),)
|
||||
|
||||
type: Mapped[UserType] = mapped_column(Enum(UserType, name="user_type"), nullable=False)
|
||||
email: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
display_name: Mapped[str | None] = mapped_column(String(120))
|
||||
|
||||
keycloak_sub: Mapped[str | None] = mapped_column(String(255), unique=True)
|
||||
local_password_hash: Mapped[str | None] = mapped_column(String(255))
|
||||
|
||||
disabled_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
last_login_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
|
||||
group_links: Mapped[list[UserGroup]] = relationship(
|
||||
back_populates="user",
|
||||
cascade="all, delete-orphan",
|
||||
)
|
||||
soc_sessions: Mapped[list[SocSession]] = relationship(
|
||||
back_populates="user",
|
||||
cascade="all, delete-orphan",
|
||||
)
|
||||
|
||||
@property
|
||||
def is_active(self) -> bool:
|
||||
return self.disabled_at is None
|
||||
|
||||
@property
|
||||
def created_by_id(self) -> UUID | None:
|
||||
# Placeholder for future audit linkage; kept for symmetry with spec §8.
|
||||
return None
|
||||
110
backend/src/mimic/db/types.py
Normal file
110
backend/src/mimic/db/types.py
Normal file
@@ -0,0 +1,110 @@
|
||||
"""Reusable column / enum definitions."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import enum
|
||||
|
||||
|
||||
class C2Type(enum.StrEnum):
|
||||
"""C2 backend kind. Source of truth for connector resolution.
|
||||
|
||||
Spec §F3 / §F4: `scenario.c2_type` is the authoritative value at run time;
|
||||
every host used in a scenario must match it.
|
||||
"""
|
||||
|
||||
MYTHIC = "mythic"
|
||||
HOME = "home"
|
||||
|
||||
|
||||
class PayloadType(enum.StrEnum):
|
||||
"""Internal neutral payload kind (spec §7 table).
|
||||
|
||||
The C2Connector resolves the actual native command on a per-c2 basis.
|
||||
`UnsupportedPayloadType` is raised when no mapping exists.
|
||||
"""
|
||||
|
||||
CMD = "cmd"
|
||||
POWERSHELL = "powershell"
|
||||
BOF = "bof"
|
||||
DOTNET_ASSEMBLY = "dotnet_assembly"
|
||||
DOTNET_EXE = "dotnet_exe"
|
||||
PE_EXE = "pe_exe"
|
||||
PE_DLL = "pe_dll"
|
||||
SHELLCODE = "shellcode"
|
||||
PYTHON = "python"
|
||||
VBS = "vbs"
|
||||
WMI_QUERY = "wmi_query"
|
||||
REGISTRY = "registry"
|
||||
SCRIPT_FILE = "script_file"
|
||||
|
||||
|
||||
class UserType(enum.StrEnum):
|
||||
"""Application user kind."""
|
||||
|
||||
RT_OPERATOR = "rt_operator"
|
||||
RT_LEAD = "rt_lead"
|
||||
SOC_ANALYST = "soc_analyst"
|
||||
|
||||
|
||||
class EngagementStatus(enum.StrEnum):
|
||||
DRAFT = "draft"
|
||||
ACTIVE = "active"
|
||||
CLOSED = "closed"
|
||||
ARCHIVED = "archived"
|
||||
|
||||
|
||||
class HostStatus(enum.StrEnum):
|
||||
UNKNOWN = "unknown"
|
||||
ALIVE = "alive"
|
||||
DEAD = "dead"
|
||||
|
||||
|
||||
class TtpSource(enum.StrEnum):
|
||||
CUSTOM = "custom"
|
||||
IMPORT_ATR = "import_atr"
|
||||
IMPORT_MISSION = "import_mission"
|
||||
|
||||
|
||||
class RunStatus(enum.StrEnum):
|
||||
QUEUED = "queued"
|
||||
RUNNING = "running"
|
||||
PAUSED = "paused"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
ABORTED = "aborted"
|
||||
|
||||
|
||||
class RunStepStatus(enum.StrEnum):
|
||||
QUEUED = "queued"
|
||||
RUNNING = "running"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
SKIPPED = "skipped"
|
||||
CLEANUP_FAILED = "cleanup_failed"
|
||||
|
||||
|
||||
class CleanupStatus(enum.StrEnum):
|
||||
PENDING = "pending"
|
||||
SUCCESS = "success"
|
||||
FAILED = "failed"
|
||||
PARTIAL = "partial"
|
||||
|
||||
|
||||
class DetectionLevel(enum.StrEnum):
|
||||
DETECTED = "detected"
|
||||
PARTIAL = "partial"
|
||||
NOT_DETECTED = "not_detected"
|
||||
|
||||
|
||||
class DetectionSource(enum.StrEnum):
|
||||
NDR = "ndr"
|
||||
EDR = "edr"
|
||||
SIEM = "siem"
|
||||
MANUAL = "manual"
|
||||
OTHER = "other"
|
||||
|
||||
|
||||
class EvidenceStatus(enum.StrEnum):
|
||||
SUCCESS = "success"
|
||||
FAILURE = "failure"
|
||||
PARTIAL = "partial"
|
||||
15
backend/src/mimic/extensions.py
Normal file
15
backend/src/mimic/extensions.py
Normal file
@@ -0,0 +1,15 @@
|
||||
"""Singletons wired into the Flask app at create-time."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from flask_login import LoginManager
|
||||
from flask_migrate import Migrate
|
||||
from flask_socketio import SocketIO
|
||||
from flask_sqlalchemy import SQLAlchemy
|
||||
|
||||
from mimic.db.base import Base
|
||||
|
||||
db = SQLAlchemy(model_class=Base)
|
||||
migrate = Migrate()
|
||||
login_manager = LoginManager()
|
||||
socketio = SocketIO(async_mode="gevent", cors_allowed_origins=[])
|
||||
25
backend/src/mimic/logging.py
Normal file
25
backend/src/mimic/logging.py
Normal file
@@ -0,0 +1,25 @@
|
||||
"""Structured JSON logging (NF-observability)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import sys
|
||||
|
||||
from pythonjsonlogger.jsonlogger import JsonFormatter # type: ignore[attr-defined]
|
||||
|
||||
|
||||
def configure_logging(level: str = "INFO", *, as_json: bool = True) -> None:
|
||||
"""Configure the root logger once at app start."""
|
||||
handler = logging.StreamHandler(sys.stdout)
|
||||
if as_json:
|
||||
formatter: logging.Formatter = JsonFormatter(
|
||||
"%(asctime)s %(levelname)s %(name)s %(message)s"
|
||||
)
|
||||
else:
|
||||
formatter = logging.Formatter("%(asctime)s %(levelname)-8s %(name)s: %(message)s")
|
||||
handler.setFormatter(formatter)
|
||||
|
||||
root = logging.getLogger()
|
||||
root.handlers.clear()
|
||||
root.addHandler(handler)
|
||||
root.setLevel(level.upper())
|
||||
17
backend/src/mimic/rbac/__init__.py
Normal file
17
backend/src/mimic/rbac/__init__.py
Normal file
@@ -0,0 +1,17 @@
|
||||
"""Group-based RBAC (spec F11 matrix)."""
|
||||
|
||||
from mimic.rbac.decorators import require_perm
|
||||
from mimic.rbac.matrix import (
|
||||
ALL_PERMISSIONS,
|
||||
DEFAULT_GROUPS,
|
||||
GROUP_PERMISSIONS,
|
||||
Permission,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"ALL_PERMISSIONS",
|
||||
"DEFAULT_GROUPS",
|
||||
"GROUP_PERMISSIONS",
|
||||
"Permission",
|
||||
"require_perm",
|
||||
]
|
||||
39
backend/src/mimic/rbac/decorators.py
Normal file
39
backend/src/mimic/rbac/decorators.py
Normal file
@@ -0,0 +1,39 @@
|
||||
"""`@require_perm` Flask decorator (group-based RBAC)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from functools import wraps
|
||||
from typing import ParamSpec, TypeVar
|
||||
|
||||
from flask import abort
|
||||
from flask_login import current_user
|
||||
|
||||
from mimic.rbac.matrix import Permission
|
||||
|
||||
P = ParamSpec("P")
|
||||
R = TypeVar("R")
|
||||
|
||||
|
||||
def require_perm(perm: Permission) -> Callable[[Callable[P, R]], Callable[P, R]]:
|
||||
"""Reject the request with 401/403 unless the user holds `perm`."""
|
||||
|
||||
def _decorate(view: Callable[P, R]) -> Callable[P, R]:
|
||||
@wraps(view)
|
||||
def _wrapped(*args: P.args, **kwargs: P.kwargs) -> R:
|
||||
user = current_user
|
||||
if not getattr(user, "is_authenticated", False):
|
||||
abort(401)
|
||||
permissions: frozenset[Permission] = getattr(user, "permissions", frozenset())
|
||||
if perm not in permissions:
|
||||
abort(403)
|
||||
return view(*args, **kwargs)
|
||||
|
||||
return _wrapped
|
||||
|
||||
return _decorate
|
||||
|
||||
|
||||
def user_has(perm: Permission, permissions: frozenset[Permission]) -> bool:
|
||||
"""Pure helper, easier to unit-test than the decorator."""
|
||||
return perm in permissions
|
||||
95
backend/src/mimic/rbac/matrix.py
Normal file
95
backend/src/mimic/rbac/matrix.py
Normal file
@@ -0,0 +1,95 @@
|
||||
"""F11 permission matrix as code.
|
||||
|
||||
This mirrors the spec table 1:1 — keeping it as a single source in code lets
|
||||
us hash-check it against the spec in CI (spec-analyst task S0.2). Three default
|
||||
groups are seeded by Alembic and align with the three user types.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import enum
|
||||
|
||||
|
||||
class Permission(enum.StrEnum):
|
||||
"""Application permissions. Codes are kebab-case + dotted scope."""
|
||||
|
||||
# Engagement
|
||||
ENGAGEMENT_CREATE = "engagement.create"
|
||||
ENGAGEMENT_READ = "engagement.read"
|
||||
ENGAGEMENT_READ_OWN = "engagement.read_own" # scoped to soc_session
|
||||
ENGAGEMENT_UPDATE = "engagement.update"
|
||||
ENGAGEMENT_DELETE = "engagement.delete"
|
||||
ENGAGEMENT_MEMBER_MANAGE = "engagement.member.manage"
|
||||
ENGAGEMENT_SOC_TOKEN_ISSUE = "engagement.soc_token.issue" # noqa: S105
|
||||
|
||||
# Hosts
|
||||
HOST_CRUD = "host.crud"
|
||||
|
||||
# TTPs
|
||||
TTP_READ = "ttp.read"
|
||||
TTP_DRAFT = "ttp.draft"
|
||||
TTP_PROMOTE = "ttp.promote"
|
||||
|
||||
# Imports
|
||||
IMPORT_JOURNAL = "import.journal"
|
||||
|
||||
# Scenarios
|
||||
SCENARIO_CRUD = "scenario.crud"
|
||||
|
||||
# Runs
|
||||
RUN_START = "run.start"
|
||||
RUN_CONTROL = "run.control"
|
||||
|
||||
# Detection / evidence
|
||||
EVIDENCE_ADD = "evidence.add"
|
||||
DETECTION_ADD = "detection.add"
|
||||
|
||||
# Cleanup
|
||||
CLEANUP_TRIGGER = "cleanup.trigger"
|
||||
|
||||
# Reports
|
||||
REPORT_GENERATE = "report.generate"
|
||||
REPORT_READ = "report.read"
|
||||
|
||||
# Audit
|
||||
AUDIT_READ = "audit.read"
|
||||
|
||||
|
||||
ALL_PERMISSIONS: tuple[Permission, ...] = tuple(Permission)
|
||||
|
||||
|
||||
class GroupName(enum.StrEnum):
|
||||
RT_OPERATOR = "rt_operator"
|
||||
RT_LEAD = "rt_lead"
|
||||
SOC_ANALYST = "soc_analyst"
|
||||
|
||||
|
||||
# Source-of-truth mapping derived from spec §F11. Verified by tests against
|
||||
# the spec table.
|
||||
GROUP_PERMISSIONS: dict[GroupName, frozenset[Permission]] = {
|
||||
GroupName.RT_OPERATOR: frozenset(
|
||||
{
|
||||
Permission.ENGAGEMENT_CREATE,
|
||||
Permission.ENGAGEMENT_READ,
|
||||
Permission.HOST_CRUD,
|
||||
Permission.TTP_READ,
|
||||
Permission.TTP_DRAFT,
|
||||
Permission.IMPORT_JOURNAL,
|
||||
Permission.SCENARIO_CRUD,
|
||||
Permission.EVIDENCE_ADD,
|
||||
Permission.CLEANUP_TRIGGER,
|
||||
Permission.REPORT_READ,
|
||||
}
|
||||
),
|
||||
GroupName.RT_LEAD: frozenset(ALL_PERMISSIONS),
|
||||
GroupName.SOC_ANALYST: frozenset(
|
||||
{
|
||||
Permission.ENGAGEMENT_READ_OWN,
|
||||
Permission.DETECTION_ADD,
|
||||
Permission.REPORT_READ,
|
||||
}
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
DEFAULT_GROUPS: tuple[GroupName, ...] = tuple(GroupName)
|
||||
33
backend/src/mimic/schemas/__init__.py
Normal file
33
backend/src/mimic/schemas/__init__.py
Normal file
@@ -0,0 +1,33 @@
|
||||
"""Pydantic 2 request/response DTOs."""
|
||||
|
||||
from mimic.schemas.engagement import (
|
||||
EngagementCreate,
|
||||
EngagementRead,
|
||||
EngagementUpdate,
|
||||
)
|
||||
from mimic.schemas.host import HostCreate, HostRead, HostUpdate
|
||||
from mimic.schemas.scenario import (
|
||||
ScenarioCreate,
|
||||
ScenarioRead,
|
||||
ScenarioStepCreate,
|
||||
ScenarioStepRead,
|
||||
ScenarioUpdate,
|
||||
)
|
||||
from mimic.schemas.ttp import TtpCreate, TtpRead, TtpUpdate
|
||||
|
||||
__all__ = [
|
||||
"EngagementCreate",
|
||||
"EngagementRead",
|
||||
"EngagementUpdate",
|
||||
"HostCreate",
|
||||
"HostRead",
|
||||
"HostUpdate",
|
||||
"ScenarioCreate",
|
||||
"ScenarioRead",
|
||||
"ScenarioStepCreate",
|
||||
"ScenarioStepRead",
|
||||
"ScenarioUpdate",
|
||||
"TtpCreate",
|
||||
"TtpRead",
|
||||
"TtpUpdate",
|
||||
]
|
||||
36
backend/src/mimic/schemas/engagement.py
Normal file
36
backend/src/mimic/schemas/engagement.py
Normal file
@@ -0,0 +1,36 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from mimic.db.types import C2Type, EngagementStatus
|
||||
|
||||
|
||||
class EngagementBase(BaseModel):
|
||||
client_name: str = Field(min_length=1, max_length=255)
|
||||
description: str | None = Field(default=None, max_length=1024)
|
||||
c2_type: C2Type = C2Type.MYTHIC
|
||||
start_date: date | None = None
|
||||
end_date: date | None = None
|
||||
|
||||
|
||||
class EngagementCreate(EngagementBase):
|
||||
pass
|
||||
|
||||
|
||||
class EngagementUpdate(BaseModel):
|
||||
client_name: str | None = Field(default=None, min_length=1, max_length=255)
|
||||
description: str | None = Field(default=None, max_length=1024)
|
||||
status: EngagementStatus | None = None
|
||||
c2_type: C2Type | None = None
|
||||
start_date: date | None = None
|
||||
end_date: date | None = None
|
||||
|
||||
|
||||
class EngagementRead(EngagementBase):
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: UUID
|
||||
status: EngagementStatus
|
||||
36
backend/src/mimic/schemas/host.py
Normal file
36
backend/src/mimic/schemas/host.py
Normal file
@@ -0,0 +1,36 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from mimic.db.types import C2Type, HostStatus
|
||||
|
||||
|
||||
class HostBase(BaseModel):
|
||||
hostname: str = Field(min_length=1, max_length=255)
|
||||
ip: str | None = Field(default=None, max_length=64)
|
||||
os: str | None = Field(default=None, max_length=128)
|
||||
c2_session_id: str | None = Field(default=None, max_length=128)
|
||||
c2_type: C2Type = C2Type.MYTHIC
|
||||
|
||||
|
||||
class HostCreate(HostBase):
|
||||
pass
|
||||
|
||||
|
||||
class HostUpdate(BaseModel):
|
||||
hostname: str | None = Field(default=None, min_length=1, max_length=255)
|
||||
ip: str | None = Field(default=None, max_length=64)
|
||||
os: str | None = Field(default=None, max_length=128)
|
||||
c2_session_id: str | None = Field(default=None, max_length=128)
|
||||
c2_type: C2Type | None = None
|
||||
status: HostStatus | None = None
|
||||
|
||||
|
||||
class HostRead(HostBase):
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: UUID
|
||||
engagement_id: UUID
|
||||
status: HostStatus
|
||||
52
backend/src/mimic/schemas/scenario.py
Normal file
52
backend/src/mimic/schemas/scenario.py
Normal file
@@ -0,0 +1,52 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from mimic.db.types import C2Type
|
||||
|
||||
|
||||
class ScenarioStepBase(BaseModel):
|
||||
ttp_id: UUID
|
||||
host_id: UUID
|
||||
order_idx: int = Field(ge=0)
|
||||
params_override_json: dict[str, Any] = Field(default_factory=dict)
|
||||
delay_after_ms: int = Field(default=0, ge=0)
|
||||
|
||||
|
||||
class ScenarioStepCreate(ScenarioStepBase):
|
||||
pass
|
||||
|
||||
|
||||
class ScenarioStepRead(ScenarioStepBase):
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: UUID
|
||||
scenario_id: UUID
|
||||
|
||||
|
||||
class ScenarioBase(BaseModel):
|
||||
name: str = Field(min_length=1, max_length=255)
|
||||
description: str | None = None
|
||||
c2_type: C2Type
|
||||
version: int = Field(default=1, ge=1)
|
||||
|
||||
|
||||
class ScenarioCreate(ScenarioBase):
|
||||
steps: list[ScenarioStepCreate] = Field(default_factory=list)
|
||||
|
||||
|
||||
class ScenarioUpdate(BaseModel):
|
||||
name: str | None = Field(default=None, min_length=1, max_length=255)
|
||||
description: str | None = None
|
||||
c2_type: C2Type | None = None
|
||||
|
||||
|
||||
class ScenarioRead(ScenarioBase):
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: UUID
|
||||
engagement_id: UUID
|
||||
steps: list[ScenarioStepRead] = Field(default_factory=list)
|
||||
50
backend/src/mimic/schemas/ttp.py
Normal file
50
backend/src/mimic/schemas/ttp.py
Normal file
@@ -0,0 +1,50 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from mimic.db.types import PayloadType, TtpSource
|
||||
|
||||
|
||||
class TtpBase(BaseModel):
|
||||
name: str = Field(min_length=1, max_length=255)
|
||||
description: str | None = None
|
||||
mitre_technique: str = Field(min_length=2, max_length=16)
|
||||
mitre_subtechnique: str | None = Field(default=None, max_length=16)
|
||||
payload_type: PayloadType
|
||||
payload_template: str = ""
|
||||
params_schema_json: dict[str, Any] | None = None
|
||||
opsec_notes: str | None = None
|
||||
cleanup_command: str | None = None
|
||||
is_stealth_variant: bool = False
|
||||
source: TtpSource = TtpSource.CUSTOM
|
||||
tags: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class TtpCreate(TtpBase):
|
||||
pass
|
||||
|
||||
|
||||
class TtpUpdate(BaseModel):
|
||||
name: str | None = Field(default=None, min_length=1, max_length=255)
|
||||
description: str | None = None
|
||||
mitre_technique: str | None = Field(default=None, min_length=2, max_length=16)
|
||||
mitre_subtechnique: str | None = Field(default=None, max_length=16)
|
||||
payload_type: PayloadType | None = None
|
||||
payload_template: str | None = None
|
||||
params_schema_json: dict[str, Any] | None = None
|
||||
opsec_notes: str | None = None
|
||||
cleanup_command: str | None = None
|
||||
is_stealth_variant: bool | None = None
|
||||
tags: list[str] | None = None
|
||||
is_published: bool | None = None
|
||||
|
||||
|
||||
class TtpRead(TtpBase):
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: UUID
|
||||
is_published: bool
|
||||
version: int
|
||||
14
backend/src/mimic/storage/__init__.py
Normal file
14
backend/src/mimic/storage/__init__.py
Normal file
@@ -0,0 +1,14 @@
|
||||
"""Local file-system pools (D-012).
|
||||
|
||||
Two separate roots, configured via env:
|
||||
- `MIMIC_BLOB_ROOT` (default `/var/lib/mimic/blobs/`) holds C2 polling output
|
||||
blobs, content-addressed and gzip-compressed: `<aa>/<bb>/<sha256>.gz` where
|
||||
`aa` and `bb` are the first two byte-pairs of the hex digest.
|
||||
- `MIMIC_EVIDENCE_ROOT` (default `/var/lib/mimic/evidence/`) holds user-uploaded
|
||||
evidence files, flat layout `<engagement_id>/<evidence_id>.<ext>`, no
|
||||
compression.
|
||||
"""
|
||||
|
||||
from mimic.storage.blob import blob_path, store_blob
|
||||
|
||||
__all__ = ["blob_path", "store_blob"]
|
||||
102
backend/src/mimic/storage/blob.py
Normal file
102
backend/src/mimic/storage/blob.py
Normal file
@@ -0,0 +1,102 @@
|
||||
"""Content-addressed gzip-compressed blob store (D-012).
|
||||
|
||||
`store_blob` consumes a binary stream and writes it gzip-compressed to a
|
||||
content-addressed path `<root>/<aa>/<bb>/<sha256>.gz`. Per MA2:
|
||||
- streaming hash + write (no whole-buffer load — defends against memory DoS
|
||||
from a hostile-large C2 output);
|
||||
- explicit `max_bytes` cap, raising `BlobTooLarge` as soon as the limit is
|
||||
crossed (the partial temp file is deleted);
|
||||
- atomic rename via `<root>/<aa>/<bb>/<sha256>.gz.tmp` → final target;
|
||||
- explicit `0o750` directory mode (does not rely on the process umask).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import gzip
|
||||
import hashlib
|
||||
import os
|
||||
import stat
|
||||
from pathlib import Path
|
||||
from typing import IO
|
||||
|
||||
_SHA256_HEX_LEN = 64
|
||||
_DEFAULT_MAX_BYTES = 10 * 1024 * 1024 # 10 MB cap (D-012)
|
||||
_READ_CHUNK = 64 * 1024 # 64 KB streaming chunks
|
||||
_DIR_MODE = 0o750
|
||||
_FILE_MODE = stat.S_IRUSR | stat.S_IWUSR | stat.S_IRGRP
|
||||
|
||||
|
||||
class BlobTooLarge(RuntimeError): # noqa: N818 (intentional: spec-aligned name)
|
||||
"""Raised when an input stream exceeds the configured `max_bytes` cap."""
|
||||
|
||||
def __init__(self, max_bytes: int) -> None:
|
||||
super().__init__(f"blob exceeds {max_bytes} byte cap")
|
||||
self.max_bytes = max_bytes
|
||||
|
||||
|
||||
def _validate_digest(sha256_hex: str) -> str:
|
||||
if len(sha256_hex) != _SHA256_HEX_LEN or any(
|
||||
c not in "0123456789abcdef" for c in sha256_hex.lower()
|
||||
):
|
||||
raise ValueError(f"invalid sha256 digest: {sha256_hex!r}")
|
||||
return sha256_hex.lower()
|
||||
|
||||
|
||||
def blob_path(root: Path | str, sha256_hex: str) -> Path:
|
||||
"""Return the absolute path of the gzip-compressed blob `<aa>/<bb>/<digest>.gz`."""
|
||||
digest = _validate_digest(sha256_hex)
|
||||
return Path(root) / digest[0:2] / digest[2:4] / f"{digest}.gz"
|
||||
|
||||
|
||||
def store_blob(
|
||||
root: Path | str,
|
||||
stream: IO[bytes],
|
||||
*,
|
||||
max_bytes: int = _DEFAULT_MAX_BYTES,
|
||||
) -> tuple[str, Path]:
|
||||
"""Write `stream` (gzip-compressed) under its sha256 digest path.
|
||||
|
||||
Reads from `stream` in fixed-size chunks while computing the digest and
|
||||
streaming the gzip body to a temp file. Raises `BlobTooLarge` as soon as
|
||||
the running total exceeds `max_bytes`; the temp file is unlinked in that
|
||||
case. Idempotent: if the target already exists the temp file is
|
||||
discarded.
|
||||
"""
|
||||
if max_bytes <= 0:
|
||||
raise ValueError(f"max_bytes must be positive, got {max_bytes}")
|
||||
|
||||
sha = hashlib.sha256()
|
||||
tmp_path = Path(root) / f".tmp-{os.getpid()}-{os.urandom(8).hex()}.gz"
|
||||
tmp_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
os.chmod(tmp_path.parent, _DIR_MODE)
|
||||
|
||||
total = 0
|
||||
try:
|
||||
with gzip.open(tmp_path, "wb") as gz:
|
||||
while True:
|
||||
chunk = stream.read(_READ_CHUNK)
|
||||
if not chunk:
|
||||
break
|
||||
total += len(chunk)
|
||||
if total > max_bytes:
|
||||
raise BlobTooLarge(max_bytes)
|
||||
sha.update(chunk)
|
||||
gz.write(chunk)
|
||||
except BlobTooLarge:
|
||||
with contextlib.suppress(FileNotFoundError):
|
||||
tmp_path.unlink()
|
||||
raise
|
||||
|
||||
digest = sha.hexdigest()
|
||||
target = blob_path(root, digest)
|
||||
if target.exists():
|
||||
with contextlib.suppress(FileNotFoundError):
|
||||
tmp_path.unlink()
|
||||
return digest, target
|
||||
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
os.chmod(target.parent, _DIR_MODE)
|
||||
os.chmod(tmp_path, _FILE_MODE)
|
||||
tmp_path.replace(target)
|
||||
return digest, target
|
||||
5
backend/src/mimic/templating/__init__.py
Normal file
5
backend/src/mimic/templating/__init__.py
Normal file
@@ -0,0 +1,5 @@
|
||||
"""Jinja2 sandboxed templating used for cleanup commands and payloads."""
|
||||
|
||||
from mimic.templating.sandbox import CleanupRenderer, RenderError, render_cleanup
|
||||
|
||||
__all__ = ["CleanupRenderer", "RenderError", "render_cleanup"]
|
||||
71
backend/src/mimic/templating/filters.py
Normal file
71
backend/src/mimic/templating/filters.py
Normal file
@@ -0,0 +1,71 @@
|
||||
"""Custom Jinja2 filters.
|
||||
|
||||
`regex_extract(text, pattern, *, group=1, name=None)` per D-011:
|
||||
- `google-re2` engine (linear-time, no backrefs, ReDoS-safe). Hard dependency
|
||||
— there is no `re` stdlib fallback (D-011 reaffirmed in code-review B1).
|
||||
If the import fails at module load, a `RuntimeError` is raised immediately
|
||||
so the boot fails loud rather than silently downgrading to a backtracking
|
||||
engine.
|
||||
- First match only.
|
||||
- No match → raises a Jinja2 `TemplateError` (no silent default — cleanup
|
||||
templates must fail loud when the source string drifts).
|
||||
- Default capture is group 1, falling back to the full match when the pattern
|
||||
has no groups. Named groups via `name="<name>"`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from jinja2 import TemplateError
|
||||
|
||||
try:
|
||||
import re2 as _re2
|
||||
except ImportError as exc: # pragma: no cover - presence enforced at deploy time
|
||||
raise RuntimeError(
|
||||
"google-re2 is required for OPSEC-safe regex (spec D-011). "
|
||||
"Install with: pip install google-re2"
|
||||
) from exc
|
||||
|
||||
|
||||
def regex_extract(
|
||||
text: Any,
|
||||
pattern: str,
|
||||
*,
|
||||
group: int = 1,
|
||||
name: str | None = None,
|
||||
) -> str:
|
||||
"""First-match capture; raise on no match (spec D-011)."""
|
||||
if text is None:
|
||||
raise TemplateError(f"regex_extract: cannot match against None for /{pattern}/")
|
||||
haystack = text if isinstance(text, str) else str(text)
|
||||
|
||||
compiled = _re2.compile(pattern)
|
||||
match = compiled.search(haystack)
|
||||
|
||||
if match is None:
|
||||
raise TemplateError(f"regex_extract: no match for /{pattern}/")
|
||||
|
||||
if name is not None:
|
||||
try:
|
||||
captured = match.group(name)
|
||||
except IndexError as exc:
|
||||
raise TemplateError(f"regex_extract: named group {name!r} not in /{pattern}/") from exc
|
||||
if captured is None:
|
||||
raise TemplateError(
|
||||
f"regex_extract: named group {name!r} captured nothing in /{pattern}/"
|
||||
)
|
||||
return str(captured)
|
||||
|
||||
try:
|
||||
captured = match.group(group)
|
||||
except IndexError:
|
||||
if group == 1:
|
||||
return str(match.group(0))
|
||||
raise TemplateError(f"regex_extract: group {group} out of range for /{pattern}/") from None
|
||||
|
||||
if captured is None:
|
||||
if group == 1:
|
||||
return str(match.group(0))
|
||||
raise TemplateError(f"regex_extract: group {group} captured nothing in /{pattern}/")
|
||||
return str(captured)
|
||||
129
backend/src/mimic/templating/sandbox.py
Normal file
129
backend/src/mimic/templating/sandbox.py
Normal file
@@ -0,0 +1,129 @@
|
||||
"""Sandboxed Jinja2 environment used to resolve cleanup commands and payloads.
|
||||
|
||||
Spec H26 / D-005 / D-012: two output accessors are exposed to templates.
|
||||
|
||||
- `{{ params.<key> }}` — straight from the merged TTP/scenario parameters.
|
||||
- `{{ outputs.text }}` — `run_step.output_text` (stdout / UTF-8 text).
|
||||
- `{{ outputs.blob() }}` — decoded `output_blob_ref` content. Per D-012 the
|
||||
blob lives in `MIMIC_BLOB_ROOT` as a content-addressed gzip-compressed file;
|
||||
`StepOutputs` does the decompression and exposes a UTF-8 string with a
|
||||
latin-1 fallback. Hard cap 10 MB **after decompression** (consistent with
|
||||
F8 evidence limit).
|
||||
|
||||
The custom `regex_extract` filter operates on the resulting string only.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import gzip
|
||||
import logging
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from jinja2 import StrictUndefined, TemplateError
|
||||
from jinja2.sandbox import SandboxedEnvironment
|
||||
|
||||
from mimic.config import get_settings
|
||||
from mimic.templating.filters import regex_extract
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class RenderError(RuntimeError):
|
||||
"""Raised when a cleanup / payload template cannot be rendered safely."""
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class StepOutputs:
|
||||
"""Read-only view of the previous step's outputs exposed to templates."""
|
||||
|
||||
text: str = ""
|
||||
blob_path: Path | None = None
|
||||
blob_max_bytes: int = 10 * 1024 * 1024
|
||||
|
||||
def blob(self, _name: str = "default") -> str:
|
||||
"""Read the CAS-gzipped output blob (D-012), decoded UTF-8 with
|
||||
latin-1 fallback. Returns the empty string when the blob is missing
|
||||
or undecodable (logged but never raises — templates that need a
|
||||
present blob should assert via regex_extract instead).
|
||||
|
||||
The argument is accepted for future multi-blob support but ignored in
|
||||
v1 — a step has at most one blob attachment.
|
||||
"""
|
||||
raw = self._read_raw()
|
||||
if raw is None:
|
||||
return ""
|
||||
if len(raw) > self.blob_max_bytes:
|
||||
raw = raw[: self.blob_max_bytes]
|
||||
try:
|
||||
return raw.decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
pass
|
||||
try:
|
||||
return raw.decode("latin-1")
|
||||
except UnicodeDecodeError: # pragma: no cover - latin-1 never fails
|
||||
log.warning("blob undecodable even as latin-1: %s", self.blob_path)
|
||||
return ""
|
||||
|
||||
def _read_raw(self) -> bytes | None:
|
||||
if self.blob_path is None:
|
||||
return None
|
||||
try:
|
||||
with gzip.open(self.blob_path, "rb") as fh:
|
||||
return fh.read(self.blob_max_bytes + 1)
|
||||
except FileNotFoundError:
|
||||
log.warning("blob not found: %s", self.blob_path)
|
||||
except OSError as exc:
|
||||
log.warning("blob unreadable %s: %s", self.blob_path, exc)
|
||||
except gzip.BadGzipFile as exc:
|
||||
log.warning("blob is not gzip %s: %s", self.blob_path, exc)
|
||||
return None
|
||||
|
||||
|
||||
class CleanupRenderer:
|
||||
"""Sandboxed Jinja2 renderer for cleanup commands and payload templates."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
env = SandboxedEnvironment(
|
||||
undefined=StrictUndefined,
|
||||
autoescape=False,
|
||||
trim_blocks=False,
|
||||
lstrip_blocks=False,
|
||||
keep_trailing_newline=False,
|
||||
)
|
||||
env.filters["regex_extract"] = regex_extract
|
||||
self._env = env
|
||||
|
||||
def render(
|
||||
self,
|
||||
template_text: str,
|
||||
*,
|
||||
params: Mapping[str, Any] | None = None,
|
||||
outputs: StepOutputs | None = None,
|
||||
) -> str:
|
||||
try:
|
||||
tmpl = self._env.from_string(template_text)
|
||||
return tmpl.render(
|
||||
params=dict(params or {}),
|
||||
outputs=outputs or StepOutputs(),
|
||||
)
|
||||
except TemplateError as exc:
|
||||
raise RenderError(str(exc)) from exc
|
||||
|
||||
|
||||
_RENDERER = CleanupRenderer()
|
||||
|
||||
|
||||
def render_cleanup(
|
||||
template_text: str,
|
||||
*,
|
||||
params: Mapping[str, Any] | None = None,
|
||||
outputs: StepOutputs | None = None,
|
||||
) -> str:
|
||||
"""Module-level convenience: render with the singleton renderer."""
|
||||
if outputs is None:
|
||||
settings = get_settings()
|
||||
outputs = StepOutputs(blob_max_bytes=settings.output_blob_max_bytes)
|
||||
return _RENDERER.render(template_text, params=params, outputs=outputs)
|
||||
0
backend/tests/__init__.py
Normal file
0
backend/tests/__init__.py
Normal file
24
backend/tests/conftest.py
Normal file
24
backend/tests/conftest.py
Normal file
@@ -0,0 +1,24 @@
|
||||
"""Shared pytest fixtures for unit-level (SQLite) tests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterator
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _ensure_test_env(monkeypatch: pytest.MonkeyPatch) -> Iterator[None]:
|
||||
"""Force MIMIC_ENV=testing so settings load is predictable."""
|
||||
monkeypatch.setenv("MIMIC_ENV", "testing")
|
||||
monkeypatch.setenv("MIMIC_SECRET_KEY", "test-secret-not-real")
|
||||
monkeypatch.setenv("MIMIC_LOG_JSON", "false")
|
||||
monkeypatch.setenv("MIMIC_LOG_LEVEL", "WARNING")
|
||||
# Pydantic Settings is cached via get_settings(); reset the cache.
|
||||
from mimic import config as cfg # noqa: PLC0415 (must follow env mutation)
|
||||
|
||||
cfg.get_settings.cache_clear()
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
cfg.get_settings.cache_clear()
|
||||
0
backend/tests/integration/__init__.py
Normal file
0
backend/tests/integration/__init__.py
Normal file
51
backend/tests/integration/conftest.py
Normal file
51
backend/tests/integration/conftest.py
Normal file
@@ -0,0 +1,51 @@
|
||||
"""Integration-level fixtures: testcontainers Postgres + Flask app + db session.
|
||||
|
||||
NF-state: SQLite is reserved for pure-logic unit tests; anything touching
|
||||
constraints, RBAC role grants, or audit-log SQL behavior must run on a real
|
||||
Postgres via testcontainers (spec H38).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterator
|
||||
|
||||
import pytest
|
||||
|
||||
try:
|
||||
from testcontainers.postgres import PostgresContainer
|
||||
except ImportError: # pragma: no cover
|
||||
PostgresContainer = None # type: ignore[assignment]
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def postgres_dsn() -> Iterator[str]:
|
||||
if PostgresContainer is None:
|
||||
pytest.skip("testcontainers not installed")
|
||||
with PostgresContainer("postgres:16-alpine") as pg:
|
||||
url = pg.get_connection_url().replace("postgresql+psycopg2", "postgresql+psycopg")
|
||||
yield url
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def app(postgres_dsn: str, monkeypatch: pytest.MonkeyPatch):
|
||||
monkeypatch.setenv("MIMIC_DATABASE_URL", postgres_dsn)
|
||||
monkeypatch.setenv("MIMIC_ENV", "testing")
|
||||
monkeypatch.setenv("MIMIC_SECRET_KEY", "test-not-real")
|
||||
|
||||
from mimic.app import create_app # noqa: PLC0415 (must follow env mutation)
|
||||
from mimic.extensions import db # noqa: PLC0415
|
||||
|
||||
application = create_app()
|
||||
with application.app_context():
|
||||
# TODO (N6 follow-up, sprint 1): run Alembic migrations instead of
|
||||
# db.create_all() so the integration tests exercise the real schema
|
||||
# including the audit_log role grants and the F11 seed.
|
||||
db.create_all()
|
||||
yield application
|
||||
db.session.remove()
|
||||
db.drop_all()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client(app):
|
||||
return app.test_client()
|
||||
13
backend/tests/integration/test_healthz.py
Normal file
13
backend/tests/integration/test_healthz.py
Normal file
@@ -0,0 +1,13 @@
|
||||
"""End-to-end smoke test: Flask app + Postgres testcontainer."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.integration
|
||||
|
||||
|
||||
def test_healthz_returns_ok(client) -> None:
|
||||
response = client.get("/healthz")
|
||||
assert response.status_code == 200
|
||||
assert response.get_json() == {"status": "ok"}
|
||||
0
backend/tests/unit/__init__.py
Normal file
0
backend/tests/unit/__init__.py
Normal file
64
backend/tests/unit/test_audit_hash.py
Normal file
64
backend/tests/unit/test_audit_hash.py
Normal file
@@ -0,0 +1,64 @@
|
||||
"""Audit log hash-chain helper tests (pure function, no DB)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
from uuid import uuid4
|
||||
|
||||
from mimic.audit.log import audit_hash
|
||||
|
||||
|
||||
def test_hash_changes_with_metadata() -> None:
|
||||
ts = datetime(2026, 5, 21, 12, 0, tzinfo=UTC)
|
||||
actor = uuid4()
|
||||
base = {
|
||||
"prev_hash": None,
|
||||
"ts": ts,
|
||||
"actor_id": actor,
|
||||
"action": "ttp.create",
|
||||
"resource_type": "ttp",
|
||||
"resource_id": "t-1",
|
||||
"metadata": {"name": "whoami"},
|
||||
}
|
||||
h1 = audit_hash(**base)
|
||||
h2 = audit_hash(**{**base, "metadata": {"name": "whoami2"}})
|
||||
assert h1 != h2
|
||||
assert len(h1) == 64
|
||||
assert all(c in "0123456789abcdef" for c in h1)
|
||||
|
||||
|
||||
def test_hash_changes_with_prev_hash() -> None:
|
||||
ts = datetime(2026, 5, 21, 12, 0, tzinfo=UTC)
|
||||
h_no_prev = audit_hash(
|
||||
prev_hash=None,
|
||||
ts=ts,
|
||||
actor_id=None,
|
||||
action="x",
|
||||
resource_type="r",
|
||||
resource_id="1",
|
||||
metadata={},
|
||||
)
|
||||
h_with_prev = audit_hash(
|
||||
prev_hash="abc",
|
||||
ts=ts,
|
||||
actor_id=None,
|
||||
action="x",
|
||||
resource_type="r",
|
||||
resource_id="1",
|
||||
metadata={},
|
||||
)
|
||||
assert h_no_prev != h_with_prev
|
||||
|
||||
|
||||
def test_hash_stable_for_same_input() -> None:
|
||||
ts = datetime(2026, 5, 21, 12, 0, tzinfo=UTC)
|
||||
base = {
|
||||
"prev_hash": "aa",
|
||||
"ts": ts,
|
||||
"actor_id": None,
|
||||
"action": "x",
|
||||
"resource_type": "r",
|
||||
"resource_id": "1",
|
||||
"metadata": {"k": "v"},
|
||||
}
|
||||
assert audit_hash(**base) == audit_hash(**base)
|
||||
92
backend/tests/unit/test_connector_factory.py
Normal file
92
backend/tests/unit/test_connector_factory.py
Normal file
@@ -0,0 +1,92 @@
|
||||
"""C2Connector factory + payload mapping tests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from mimic.connectors import (
|
||||
C2Connector,
|
||||
ConnectorFactory,
|
||||
Payload,
|
||||
TaskHandle,
|
||||
TaskResult,
|
||||
TaskStatus,
|
||||
UnsupportedPayloadType,
|
||||
register_connector,
|
||||
)
|
||||
from mimic.connectors.factory import _REGISTRY
|
||||
from mimic.connectors.payload_map import resolve_native, supports
|
||||
from mimic.db.models.host import Host
|
||||
from mimic.db.types import C2Type, PayloadType
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clear_registry() -> None:
|
||||
"""Each test starts with a clean connector registry."""
|
||||
snapshot = dict(_REGISTRY)
|
||||
_REGISTRY.clear()
|
||||
yield
|
||||
_REGISTRY.clear()
|
||||
_REGISTRY.update(snapshot)
|
||||
|
||||
|
||||
class _NullConnector(C2Connector):
|
||||
def authenticate(self, config: dict[str, object]) -> None: ...
|
||||
def list_hosts(self, engagement_id: str) -> list[Host]:
|
||||
return []
|
||||
|
||||
def execute_task(self, host: Host, payload: Payload) -> TaskHandle:
|
||||
return TaskHandle(
|
||||
c2=self.name,
|
||||
c2_task_id="t-1",
|
||||
host_id="h-1",
|
||||
payload_type=payload.payload_type,
|
||||
)
|
||||
|
||||
def get_task_result(self, handle: TaskHandle) -> TaskResult:
|
||||
return TaskResult(status=TaskStatus.COMPLETED, output_text="ok")
|
||||
|
||||
def cancel_task(self, handle: TaskHandle) -> None: ...
|
||||
|
||||
def execute_cleanup(
|
||||
self, host: Host, resolved_command: str, params: dict[str, object]
|
||||
) -> TaskResult:
|
||||
return TaskResult(status=TaskStatus.COMPLETED)
|
||||
|
||||
|
||||
def test_register_and_build() -> None:
|
||||
register_connector(C2Type.MYTHIC)(_NullConnector)
|
||||
factory = ConnectorFactory(config_resolver=lambda _: {})
|
||||
connector = factory.build(C2Type.MYTHIC)
|
||||
assert isinstance(connector, _NullConnector)
|
||||
assert connector.name is C2Type.MYTHIC
|
||||
|
||||
|
||||
def test_double_registration_rejected() -> None:
|
||||
register_connector(C2Type.MYTHIC)(_NullConnector)
|
||||
with pytest.raises(RuntimeError, match="already registered"):
|
||||
register_connector(C2Type.MYTHIC)(_NullConnector)
|
||||
|
||||
|
||||
def test_build_unknown_raises_not_implemented() -> None:
|
||||
factory = ConnectorFactory(config_resolver=lambda _: {})
|
||||
with pytest.raises(NotImplementedError):
|
||||
factory.build(C2Type.HOME)
|
||||
|
||||
|
||||
def test_task_status_is_terminal() -> None:
|
||||
assert TaskStatus.COMPLETED.is_terminal
|
||||
assert TaskStatus.FAILED.is_terminal
|
||||
assert TaskStatus.CANCELED.is_terminal
|
||||
assert not TaskStatus.RUNNING.is_terminal
|
||||
|
||||
|
||||
def test_mythic_mapping_covers_powershell() -> None:
|
||||
assert resolve_native(C2Type.MYTHIC, PayloadType.POWERSHELL) == "powershell"
|
||||
assert supports(C2Type.MYTHIC, PayloadType.SHELLCODE)
|
||||
|
||||
|
||||
def test_home_mapping_empty_until_pr2() -> None:
|
||||
assert not supports(C2Type.HOME, PayloadType.CMD)
|
||||
with pytest.raises(UnsupportedPayloadType):
|
||||
resolve_native(C2Type.HOME, PayloadType.CMD)
|
||||
39
backend/tests/unit/test_migration_seed.py
Normal file
39
backend/tests/unit/test_migration_seed.py
Normal file
@@ -0,0 +1,39 @@
|
||||
"""MA3: the frozen RBAC seed in the initial migration must keep matching
|
||||
the runtime F11 matrix in `mimic.rbac.matrix`. When they drift, *do not* edit
|
||||
the migration in place — write a new migration. This test enforces it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
|
||||
from mimic.rbac.matrix import GROUP_PERMISSIONS, GroupName, Permission
|
||||
|
||||
|
||||
def _load_migration():
|
||||
return importlib.import_module("mimic.db.migrations.versions.202605210001_initial_schema")
|
||||
|
||||
|
||||
def test_frozen_permission_list_matches_runtime() -> None:
|
||||
migration = _load_migration()
|
||||
runtime_codes = {p.value for p in Permission}
|
||||
frozen_codes = set(migration._PERMISSIONS_FROZEN)
|
||||
assert runtime_codes == frozen_codes, (
|
||||
"Permission enum drifted from the migration freeze; "
|
||||
"write a new migration, do not edit the existing one."
|
||||
)
|
||||
|
||||
|
||||
def test_frozen_group_membership_matches_runtime() -> None:
|
||||
migration = _load_migration()
|
||||
runtime = {gn.value: {p.value for p in perms} for gn, perms in GROUP_PERMISSIONS.items()}
|
||||
frozen = {gn: set(perms) for gn, perms in migration._GROUP_PERMISSIONS_FROZEN.items()}
|
||||
assert runtime == frozen, (
|
||||
"GROUP_PERMISSIONS drifted from the migration freeze; "
|
||||
"write a new migration, do not edit the existing one."
|
||||
)
|
||||
|
||||
|
||||
def test_frozen_group_names_match_enum() -> None:
|
||||
migration = _load_migration()
|
||||
assert set(migration._GROUP_PERMISSIONS_FROZEN.keys()) == {g.value for g in GroupName}
|
||||
31
backend/tests/unit/test_password.py
Normal file
31
backend/tests/unit/test_password.py
Normal file
@@ -0,0 +1,31 @@
|
||||
"""Local-auth bcrypt helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from mimic.auth.password import check_password, hash_password
|
||||
|
||||
|
||||
def test_hash_then_check_succeeds() -> None:
|
||||
hashed = hash_password("Sup3rSecret!", rounds=4)
|
||||
assert check_password("Sup3rSecret!", hashed) is True
|
||||
|
||||
|
||||
def test_check_rejects_wrong_password() -> None:
|
||||
hashed = hash_password("right", rounds=4)
|
||||
assert check_password("wrong", hashed) is False
|
||||
|
||||
|
||||
def test_empty_password_raises() -> None:
|
||||
with pytest.raises(ValueError, match="must not be empty"):
|
||||
hash_password("")
|
||||
|
||||
|
||||
def test_check_missing_hash_returns_false() -> None:
|
||||
assert check_password("anything", None) is False
|
||||
assert check_password("anything", "") is False
|
||||
|
||||
|
||||
def test_check_invalid_hash_returns_false() -> None:
|
||||
assert check_password("anything", "not-a-bcrypt-hash") is False
|
||||
55
backend/tests/unit/test_rbac_matrix.py
Normal file
55
backend/tests/unit/test_rbac_matrix.py
Normal file
@@ -0,0 +1,55 @@
|
||||
"""F11 matrix coverage tests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from mimic.rbac.matrix import GROUP_PERMISSIONS, GroupName, Permission
|
||||
|
||||
|
||||
def test_every_group_has_at_least_one_permission() -> None:
|
||||
for group, perms in GROUP_PERMISSIONS.items():
|
||||
assert perms, f"group {group.value} has no permissions"
|
||||
|
||||
|
||||
def test_rt_lead_is_superset_of_operator() -> None:
|
||||
lead = GROUP_PERMISSIONS[GroupName.RT_LEAD]
|
||||
operator = GROUP_PERMISSIONS[GroupName.RT_OPERATOR]
|
||||
assert operator <= lead
|
||||
|
||||
|
||||
def test_soc_cannot_start_runs() -> None:
|
||||
soc = GROUP_PERMISSIONS[GroupName.SOC_ANALYST]
|
||||
assert Permission.RUN_START not in soc
|
||||
assert Permission.RUN_CONTROL not in soc
|
||||
|
||||
|
||||
def test_only_lead_promotes_ttp() -> None:
|
||||
operator = GROUP_PERMISSIONS[GroupName.RT_OPERATOR]
|
||||
soc = GROUP_PERMISSIONS[GroupName.SOC_ANALYST]
|
||||
assert Permission.TTP_PROMOTE not in operator
|
||||
assert Permission.TTP_PROMOTE not in soc
|
||||
assert Permission.TTP_PROMOTE in GROUP_PERMISSIONS[GroupName.RT_LEAD]
|
||||
|
||||
|
||||
def test_audit_read_lead_only() -> None:
|
||||
for group in (GroupName.RT_OPERATOR, GroupName.SOC_ANALYST):
|
||||
assert Permission.AUDIT_READ not in GROUP_PERMISSIONS[group]
|
||||
assert Permission.AUDIT_READ in GROUP_PERMISSIONS[GroupName.RT_LEAD]
|
||||
|
||||
|
||||
def test_only_lead_issues_soc_tokens() -> None:
|
||||
for group in (GroupName.RT_OPERATOR, GroupName.SOC_ANALYST):
|
||||
assert Permission.ENGAGEMENT_SOC_TOKEN_ISSUE not in GROUP_PERMISSIONS[group]
|
||||
|
||||
|
||||
def test_operator_cannot_control_run() -> None:
|
||||
operator = GROUP_PERMISSIONS[GroupName.RT_OPERATOR]
|
||||
assert Permission.RUN_START not in operator
|
||||
assert Permission.RUN_CONTROL not in operator
|
||||
|
||||
|
||||
def test_soc_can_only_read_report_and_add_detection() -> None:
|
||||
soc = GROUP_PERMISSIONS[GroupName.SOC_ANALYST]
|
||||
assert Permission.DETECTION_ADD in soc
|
||||
assert Permission.REPORT_READ in soc
|
||||
assert Permission.EVIDENCE_ADD not in soc
|
||||
assert Permission.HOST_CRUD not in soc
|
||||
27
backend/tests/unit/test_soc_token.py
Normal file
27
backend/tests/unit/test_soc_token.py
Normal file
@@ -0,0 +1,27 @@
|
||||
"""SOC opaque token generation / verification."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from mimic.auth.soc_token import generate_token, verify_token
|
||||
|
||||
|
||||
def test_generated_token_verifies() -> None:
|
||||
material = generate_token(rounds=4)
|
||||
assert verify_token(material.plain, material.hashed) is True
|
||||
|
||||
|
||||
def test_different_plain_does_not_verify() -> None:
|
||||
material = generate_token(rounds=4)
|
||||
assert verify_token("wrong-token", material.hashed) is False
|
||||
|
||||
|
||||
def test_plain_is_url_safe_and_long() -> None:
|
||||
material = generate_token(rounds=4)
|
||||
# 32 random bytes → ~43 url-safe base64 chars.
|
||||
assert len(material.plain) >= 32
|
||||
assert all(c.isalnum() or c in "-_" for c in material.plain)
|
||||
|
||||
|
||||
def test_verify_with_empty_values() -> None:
|
||||
assert verify_token("", "$2b$04$abc") is False
|
||||
assert verify_token("token", "") is False
|
||||
77
backend/tests/unit/test_storage_blob.py
Normal file
77
backend/tests/unit/test_storage_blob.py
Normal file
@@ -0,0 +1,77 @@
|
||||
"""Content-addressed gzip blob store (D-012, MA2 streaming)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import gzip
|
||||
import hashlib
|
||||
import io
|
||||
|
||||
import pytest
|
||||
|
||||
from mimic.storage.blob import BlobTooLarge, blob_path, store_blob
|
||||
|
||||
|
||||
def _stream(data: bytes) -> io.BytesIO:
|
||||
return io.BytesIO(data)
|
||||
|
||||
|
||||
def test_blob_path_uses_two_byte_pairs(tmp_path) -> None:
|
||||
digest = "ab" + "cd" + "ef" * 30
|
||||
path = blob_path(tmp_path, digest)
|
||||
assert path == tmp_path / "ab" / "cd" / f"{digest}.gz"
|
||||
|
||||
|
||||
def test_blob_path_rejects_invalid_digest(tmp_path) -> None:
|
||||
with pytest.raises(ValueError, match="invalid sha256"):
|
||||
blob_path(tmp_path, "not-a-digest")
|
||||
|
||||
|
||||
def test_store_blob_writes_gzip_and_returns_digest(tmp_path) -> None:
|
||||
payload = b"hello world\n"
|
||||
expected = hashlib.sha256(payload).hexdigest()
|
||||
digest, path = store_blob(tmp_path, _stream(payload))
|
||||
assert digest == expected
|
||||
assert path == tmp_path / expected[0:2] / expected[2:4] / f"{expected}.gz"
|
||||
with gzip.open(path, "rb") as fh:
|
||||
assert fh.read() == payload
|
||||
|
||||
|
||||
def test_store_blob_is_idempotent(tmp_path) -> None:
|
||||
payload = b"same content"
|
||||
digest1, path1 = store_blob(tmp_path, _stream(payload))
|
||||
mtime_before = path1.stat().st_mtime_ns
|
||||
digest2, path2 = store_blob(tmp_path, _stream(payload))
|
||||
assert digest1 == digest2
|
||||
assert path1 == path2
|
||||
assert path2.stat().st_mtime_ns == mtime_before
|
||||
|
||||
|
||||
def test_store_blob_dedupes_distinct_payloads(tmp_path) -> None:
|
||||
_, p1 = store_blob(tmp_path, _stream(b"alpha"))
|
||||
_, p2 = store_blob(tmp_path, _stream(b"beta"))
|
||||
assert p1 != p2
|
||||
assert p1.exists()
|
||||
assert p2.exists()
|
||||
|
||||
|
||||
def test_store_blob_raises_when_stream_exceeds_cap(tmp_path) -> None:
|
||||
too_big = b"A" * (1024 + 1)
|
||||
with pytest.raises(BlobTooLarge):
|
||||
store_blob(tmp_path, _stream(too_big), max_bytes=1024)
|
||||
# No tmp file left behind.
|
||||
leftovers = [p for p in tmp_path.iterdir() if p.name.startswith(".tmp-")]
|
||||
assert leftovers == []
|
||||
|
||||
|
||||
def test_store_blob_handles_large_stream_in_chunks(tmp_path) -> None:
|
||||
# 1.5 MB payload — exercises the multi-chunk path (chunks are 64 KB).
|
||||
payload = (b"X" * 64 * 1024) * 24
|
||||
digest, path = store_blob(tmp_path, _stream(payload), max_bytes=2 * 1024 * 1024)
|
||||
assert digest == hashlib.sha256(payload).hexdigest()
|
||||
with gzip.open(path, "rb") as fh:
|
||||
assert fh.read() == payload
|
||||
|
||||
|
||||
def test_store_blob_rejects_zero_or_negative_max(tmp_path) -> None:
|
||||
with pytest.raises(ValueError, match="max_bytes"):
|
||||
store_blob(tmp_path, _stream(b"x"), max_bytes=0)
|
||||
126
backend/tests/unit/test_templating.py
Normal file
126
backend/tests/unit/test_templating.py
Normal file
@@ -0,0 +1,126 @@
|
||||
"""Jinja2 sandbox + regex_extract tests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import gzip
|
||||
|
||||
import pytest
|
||||
from jinja2 import TemplateError
|
||||
|
||||
from mimic.templating.filters import regex_extract
|
||||
from mimic.templating.sandbox import (
|
||||
CleanupRenderer,
|
||||
RenderError,
|
||||
StepOutputs,
|
||||
render_cleanup,
|
||||
)
|
||||
|
||||
|
||||
class TestRegexExtract:
|
||||
def test_re2_is_required(self) -> None:
|
||||
"""D-011 / B1: google-re2 is the only allowed engine. Asserts the
|
||||
binding is wired into the module (the import-time RuntimeError check
|
||||
already covers absence)."""
|
||||
from mimic.templating import filters as filters_module # noqa: PLC0415
|
||||
|
||||
assert filters_module._re2 is not None
|
||||
assert filters_module._re2.__name__ == "re2"
|
||||
|
||||
def test_returns_capture_group(self) -> None:
|
||||
assert regex_extract("hello world", r"hello (\w+)") == "world"
|
||||
|
||||
def test_no_match_raises(self) -> None:
|
||||
with pytest.raises(TemplateError, match="no match"):
|
||||
regex_extract("hello", r"foo(\d+)")
|
||||
|
||||
def test_none_input_raises(self) -> None:
|
||||
with pytest.raises(TemplateError, match="None"):
|
||||
regex_extract(None, r"x")
|
||||
|
||||
def test_no_groups_falls_back_to_full_match(self) -> None:
|
||||
assert regex_extract("abc123", r"\w+\d+") == "abc123"
|
||||
|
||||
def test_named_group(self) -> None:
|
||||
assert regex_extract("pid=4242", r"pid=(?P<n>\d+)", name="n") == "4242"
|
||||
|
||||
def test_missing_named_group_raises(self) -> None:
|
||||
with pytest.raises(TemplateError):
|
||||
regex_extract("pid=4242", r"pid=(\d+)", name="absent")
|
||||
|
||||
|
||||
class TestCleanupRenderer:
|
||||
def setup_method(self) -> None:
|
||||
self.renderer = CleanupRenderer()
|
||||
|
||||
def test_render_params(self) -> None:
|
||||
out = self.renderer.render(
|
||||
"echo {{ params.target }}",
|
||||
params={"target": "WIN-01"},
|
||||
)
|
||||
assert out == "echo WIN-01"
|
||||
|
||||
def test_render_outputs_text(self) -> None:
|
||||
out = self.renderer.render(
|
||||
'echo "{{ outputs.text }}"',
|
||||
outputs=StepOutputs(text="captured"),
|
||||
)
|
||||
assert out == 'echo "captured"'
|
||||
|
||||
def test_regex_extract_filter(self) -> None:
|
||||
out = self.renderer.render(
|
||||
r"{{ outputs.text | regex_extract('pid=(\\d+)') }}",
|
||||
outputs=StepOutputs(text="status: pid=4242 user=svc"),
|
||||
)
|
||||
assert out == "4242"
|
||||
|
||||
def test_regex_extract_no_match_propagates_as_render_error(self) -> None:
|
||||
with pytest.raises(RenderError, match="no match"):
|
||||
self.renderer.render(
|
||||
r"{{ outputs.text | regex_extract('pid=(\\d+)') }}",
|
||||
outputs=StepOutputs(text="nothing"),
|
||||
)
|
||||
|
||||
def test_strict_undefined_raises(self) -> None:
|
||||
with pytest.raises(RenderError):
|
||||
self.renderer.render("{{ params.does_not_exist }}", params={})
|
||||
|
||||
def test_sandbox_forbids_attribute_access(self) -> None:
|
||||
with pytest.raises(RenderError):
|
||||
self.renderer.render(
|
||||
"{{ ().__class__.__bases__[0].__subclasses__() }}",
|
||||
params={},
|
||||
)
|
||||
|
||||
def test_module_singleton_round_trip(self) -> None:
|
||||
out = render_cleanup("hello {{ params.x }}", params={"x": "there"})
|
||||
assert out == "hello there"
|
||||
|
||||
|
||||
class TestStepOutputsBlob:
|
||||
def test_blob_returns_empty_when_no_path(self) -> None:
|
||||
out = StepOutputs(text="x")
|
||||
assert out.blob() == ""
|
||||
|
||||
def test_blob_reads_gzipped_file(self, tmp_path) -> None:
|
||||
blob = tmp_path / "blob.gz"
|
||||
with gzip.open(blob, "wb") as fh:
|
||||
fh.write(b"hello")
|
||||
out = StepOutputs(blob_path=blob)
|
||||
assert out.blob() == "hello"
|
||||
|
||||
def test_blob_caps_size_after_decompression(self, tmp_path) -> None:
|
||||
blob = tmp_path / "blob.gz"
|
||||
with gzip.open(blob, "wb") as fh:
|
||||
fh.write(b"A" * 1024)
|
||||
out = StepOutputs(blob_path=blob, blob_max_bytes=10)
|
||||
assert out.blob() == "A" * 10
|
||||
|
||||
def test_blob_missing_file_returns_empty(self, tmp_path) -> None:
|
||||
out = StepOutputs(blob_path=tmp_path / "absent.gz")
|
||||
assert out.blob() == ""
|
||||
|
||||
def test_blob_non_gzip_returns_empty(self, tmp_path) -> None:
|
||||
blob = tmp_path / "blob.gz"
|
||||
blob.write_bytes(b"not actually gzip")
|
||||
out = StepOutputs(blob_path=blob)
|
||||
assert out.blob() == ""
|
||||
@@ -72,15 +72,6 @@ scope extension:
|
||||
- Any drift between seeded group permissions and the F11 matrix is a spec
|
||||
violation, not a configuration choice.
|
||||
|
||||
### D-010 — Ansible for the deployment playbook
|
||||
**Context.** Spec §7 names `Docker` only on the deploy line, but D-007 references
|
||||
a "deployment playbook" wiring Mimic behind the existing reverse proxy. The RT
|
||||
team uses Ansible for infrastructure automation across projects.
|
||||
**Decision.** Deployment artifacts are Docker images (built in repo) plus an
|
||||
Ansible playbook (lives outside the application repo, in the RT infra repo).
|
||||
Mimic itself ships only the Dockerfile and a sample compose for dev; production
|
||||
roll-out is Ansible-driven. The README stack line is updated accordingly.
|
||||
|
||||
### D-009 — `ttp_version` table forbidden (H32 reaffirmed)
|
||||
**Context.** Sprint 0 plan (B0.2) lists `ttp_version` among the initial tables.
|
||||
Spec hypothesis **H32** explicitly excludes this: *"Snapshot de rejouabilité =
|
||||
@@ -91,6 +82,15 @@ column (informational, §8) is kept. Replayability lives **solely** on
|
||||
`run.snapshot_json`. Re-introducing `ttp_version` requires explicit spec amendment
|
||||
through the team-lead.
|
||||
|
||||
### D-010 — Ansible for the deployment playbook
|
||||
**Context.** Spec §7 names `Docker` only on the deploy line, but D-007 references
|
||||
a "deployment playbook" wiring Mimic behind the existing reverse proxy. The RT
|
||||
team uses Ansible for infrastructure automation across projects.
|
||||
**Decision.** Deployment artifacts are Docker images (built in repo) plus an
|
||||
Ansible playbook (lives outside the application repo, in the RT infra repo).
|
||||
Mimic itself ships only the Dockerfile and a sample compose for dev; production
|
||||
roll-out is Ansible-driven. The README stack line is updated accordingly.
|
||||
|
||||
### D-011 — `regex_extract` Jinja2 filter semantics (resolves Q-001)
|
||||
**Context.** D-005 introduced `regex_extract` on Jinja templates without fixing
|
||||
its match-mode, no-match behaviour, group selection, or engine flavour. Backend
|
||||
@@ -135,3 +135,20 @@ locked because B0.5 already references `{{ outputs.blob(...) }}`.
|
||||
#### Resolved open questions
|
||||
- Q-001 → D-011.
|
||||
- Q-002 → D-012.
|
||||
|
||||
### D-013 — Hash-chain in `audit_log` from v1
|
||||
**Context.** Spec H30 places the hash chain in v2; F13 / R-O5 only mandate the
|
||||
write-only role for v1. While implementing B0.7, adding the columns and chaining
|
||||
logic was a few lines and avoids a destructive migration later.
|
||||
**Decision.** `prev_hash` / `row_hash` columns ship from day one and are
|
||||
populated at insert time (SHA-256 of canonical record + previous hash). The
|
||||
chain *verifier* lands in v2. Cost is negligible (one SELECT + one SHA-256 per
|
||||
audit insert).
|
||||
|
||||
### D-014 — Type-hinting strategy for the ORM
|
||||
**Context.** Flask-SQLAlchemy 3 rejects a per-base `type_annotation_map` (the
|
||||
extension owns the registry).
|
||||
**Decision.** UUID primary keys use the explicit `PG_UUID(as_uuid=True)` type
|
||||
on `UuidPkMixin`. Foreign-key UUID columns rely on SQLAlchemy 2's built-in
|
||||
`Uuid` mapping via `Mapped[uuid.UUID]`. No `type_annotation_map` on the
|
||||
declarative base.
|
||||
|
||||
@@ -2,24 +2,67 @@
|
||||
|
||||
Repo skeleton + foundational modules. Nothing that depends on PR1/PR2/PR3.
|
||||
|
||||
## Backend (`backend`)
|
||||
## Backend (`backend`) — done in `feature/backend-skeleton`
|
||||
|
||||
- [ ] B0.1 — `backend/` Python project: `pyproject.toml` (ruff, mypy strict, pytest, coverage),
|
||||
`Makefile`, `Dockerfile`, `docker-compose.yml` for Postgres dev DB.
|
||||
- [ ] B0.2 — Alembic init + complete initial migration covering the §8 schema (incl.
|
||||
`c2_credential`, `user`, `group`, `user_group`, `permission`, `group_permission`,
|
||||
`soc_session`, audit_log with write-only Postgres role). **No `ttp_version` table** (D-009).
|
||||
Seed groups `rt_operator`, `rt_lead`, `soc_analyst` with F11 permissions (D-008).
|
||||
- [ ] B0.3 — SQLAlchemy 2 typed mapped classes for every table + repositories scaffold.
|
||||
- [ ] B0.4 — `C2Connector` ABC + dataclasses (`Payload`, `TaskHandle`, `TaskResult`) + enum
|
||||
`payload_type` + factory keyed on `c2_type`. **No real implementation.**
|
||||
- [ ] B0.5 — Jinja2 SandboxedEnvironment + `regex_extract` filter via `google-re2` +
|
||||
`{{outputs.text}}` and `{{outputs.blob(key)}}` accessors with 10 MB cap.
|
||||
- [ ] B0.6 — Local auth (login/password bcrypt + Flask server-side sessions) + RBAC
|
||||
group-based decorators + F11 permission matrix declared in code.
|
||||
- [ ] B0.7 — Flat CRUD endpoints (engagements, hosts, TTPs, scenarios) — no orchestration,
|
||||
no WebSocket, no reporting yet.
|
||||
- [ ] B0.8 — pytest baseline: unit (SQLite) + integration scaffold (testcontainers Postgres).
|
||||
- [x] B0.1 — `backend/` Python 3.12+ project: `pyproject.toml` (ruff, mypy strict, pytest,
|
||||
coverage 70 %), `Makefile` (Docker/Podman auto), multi-stage `Dockerfile`,
|
||||
`docker-compose.yml` for Postgres dev DB, `.env.example`.
|
||||
- [x] B0.2 — Alembic baseline migration `202605210001_initial_schema` creates every table,
|
||||
enum, index, and the idempotent grants for the audit write-only Postgres role. **No
|
||||
`ttp_version` table** (D-009). Groups `rt_operator`, `rt_lead`, `soc_analyst` seeded
|
||||
with the exact F11 permission matrix (D-008).
|
||||
- [x] B0.3 — SQLAlchemy 2 typed mapped classes for every spec §8 aggregate (engagement,
|
||||
host, user/group RBAC, ttp, scenario/scenario_step, run/run_step/cleanup, detection,
|
||||
evidence, report, soc_session, c2_credential, audit_log).
|
||||
- [x] B0.4 — `C2Connector` ABC + dataclasses + `payload_type` enum + factory keyed on
|
||||
`c2_type`. Mythic payload map populated; Home stays empty until PR2.
|
||||
- [x] B0.5 — Jinja2 SandboxedEnvironment, `regex_extract` filter (`google-re2` hard
|
||||
dependency per D-011, raises `RuntimeError` at boot if absent — no `re` fallback),
|
||||
fail-loud no-match, `{{ outputs.text }}` / `{{ outputs.blob() }}` accessors
|
||||
reading gzip-compressed blobs with 10 MB cap.
|
||||
- [x] B0.6 — bcrypt password helpers + SOC opaque token (256-bit url-safe, bcrypt-hashed) +
|
||||
group-based RBAC matrix matching F11 + `@require_perm` decorator.
|
||||
- [x] B0.7 — Flat CRUD blueprints for engagements / hosts / TTPs / scenarios (incl. step
|
||||
composition with F3 invariant `host.c2_type == scenario.c2_type`).
|
||||
- [x] B0.8 — pytest baseline: unit tests passing, integration scaffold ready
|
||||
(testcontainers Postgres + `/healthz` smoke).
|
||||
|
||||
## Backend follow-ups (sprint 1+)
|
||||
|
||||
Tracked from code-review verdict on `feature/backend-skeleton` @ 12d131c:
|
||||
|
||||
### MINOR (8) — to schedule
|
||||
|
||||
- **M1** — Replace `parse_uuid` integer-ish lookup with `werkzeug` UUID converter on
|
||||
the routes (`<uuid:eid>`) to avoid the 404 on malformed strings being hidden by
|
||||
the 400 path.
|
||||
- **M2** — Add OpenAPI generation (Pydantic 2 + `flask-pydantic-openapi` or hand-rolled).
|
||||
- **M3** — Wire `flask-limiter` for `/auth/local/login` (NF-network).
|
||||
- **M4** — Replace string-based `Engagement.status` setter with a typed transition method.
|
||||
- **M5** — Introduce per-engagement read view that pre-joins `engagement_member` for
|
||||
RT operator dashboards (current per-request join is fine for v1 traffic, but
|
||||
re-evaluate at scale).
|
||||
- **M6** — `mimic-cli user create` does not handle the SOC user-type (intended, but
|
||||
document and gate explicitly with a clean error message).
|
||||
- **M7** — Add a `mimic-cli` `engagement add-member <uid> --role rt_operator` shortcut so
|
||||
the F11 scoping in MA6 is reachable from the CLI without manual SQL.
|
||||
- **M8** — _(fixed in MA1 follow-up commit)_ Initial migration docstring no longer
|
||||
references `ttp_version`.
|
||||
|
||||
### NIT (6) — opportunistic
|
||||
|
||||
- **N1** — Sort imports inside `mimic.db.models.__init__` alphabetically for diff
|
||||
stability.
|
||||
- **N2** — Extract the `_engagement_or_404` duplicated body into a shared helper.
|
||||
- **N3** — Replace the inline `Permission.TTP_PROMOTE not in perms` check in `ttps.py`
|
||||
with a second `@require_perm`-style decorator.
|
||||
- **N4** — _(fixed)_ `gunicorn` added to `pyproject.toml` dependencies.
|
||||
- **N5** — Replace bare `getattr(current_user, "groups", frozenset())` accesses by a
|
||||
thin `current_groups()` helper.
|
||||
- **N6** — `tests/integration/conftest.py` uses `db.create_all()` instead of running
|
||||
Alembic. Marked with a TODO; switch over once the F11 seed must be exercised in
|
||||
integration. Plan: convert to `alembic upgrade head` once the audit role
|
||||
bootstrap lives in the playbook (D-010).
|
||||
|
||||
## Frontend (`ux-frontend`)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user