diff --git a/CHANGELOG.md b/CHANGELOG.md index ea2f1a0..37f8628 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,33 @@ All notable changes to this project will be documented here. Format: [Keep a Cha ## [Unreleased] +### Added — M5 (Test & scenario templates) +- **CRUD `test_templates`** (`app/services/test_templates.py` + `app/api/test_templates.py`): + - Fields: name, description, objective, procedure (markdown), prerequisites (markdown), expected result red, expected detection blue, OPSEC level (`low/medium/high`), free tags (TEXT[]), expected IOCs (TEXT[]). + - Polymorphic MITRE tag set (`(kind, external_id)` ↔ exactly one of `tactic_id`/`technique_id`/`subtechnique_id`). The wire payload uses ATT&CK external IDs — server resolves to UUIDs. + - Filters: `q` (LIKE on name/description), `tactic`/`technique`/`subtechnique` (joined via subquery on the polymorphic tag table), `opsec`, `tag` (array contains). + - REST: `GET /test-templates`, `GET /test-templates/{id}`, `POST /test-templates`, `PUT /test-templates/{id}` (partial, with explicit `_UNSET` sentinel so omitted fields stay untouched), `DELETE /test-templates/{id}` (soft). +- **CRUD `scenario_templates`** (`app/services/scenario_templates.py` + `app/api/scenario_templates.py`): + - Ordered list of test_templates with `position` (UNIQUE `scenario_template_id, position`). + - Reorder via full replace: `PUT /scenario-templates/{id}/tests` deletes the join rows and re-inserts at positions `0..N-1` — clean atomic op that respects the UNIQUE constraint without a 2-phase position shuffle. + - The same test can appear multiple times (chained operations). + - REST: `GET`/`POST`/`PATCH` (metadata) / `DELETE` (soft) on `/scenario-templates`. +- **Frontend**: + - `lib/templates.ts` — typed client + queryKey factory. + - `pages/AdminTestsPage.tsx` — list + filters (q, tactic, opsec, tag) + modal with full field set + embedded `` for tags. + - `pages/AdminScenariosPage.tsx` — list + modal with **@dnd-kit/sortable** vertical drag-and-drop on the ordered test list. New deps: `@dnd-kit/core`, `@dnd-kit/sortable`, `@dnd-kit/utilities`. + - `components/MarkdownField.tsx` — lean textarea with markdown hint (no heavy editor dep; rendering happens at display time in M7). + - Nav adds **Tests** and **Scenarios** links (admin-gated). +- **/diag/reset** truncates the 4 new tables before the MITRE block — the `scenario_template_tests.test_template_id` FK is `ON DELETE RESTRICT`, so the order matters. +- **Testing**: + - `backend/tests/test_templates.py` — **19 pytest** (create/list/filter by tactic+opsec+tag, MITRE tag resolution + replacement on update, soft-delete, perm gating, scenario create+reorder+delete, soft-deleted test linking semantics). + - `e2e/tests/m5-templates.spec.ts` — **4 Playwright** (API CRUD round-trip, scenario reorder, SPA list + opsec filter, SPA scenario list rendering with ordered tests). + - `tasks/testing-m5.md`. + +### Fixed (M5 implementation) +- **`LogRecord` key collision**: `log.info(..., extra={"name": ...})` raises `KeyError("Attempt to overwrite 'name' in LogRecord")` because `name` is reserved by Python's stdlib logging. Renamed to `template_name`. +- **React `currentTarget` null in deferred state updaters**: `onChange={(e) => setX((prev) => ({ ...prev, q: e.currentTarget.value }))}` blanked the page on the first user input because `currentTarget` is cleared after the listener bubble ends, before React invokes the updater. Switched all M5 handlers to `e.target.value`, which persists on the synthetic event. + ### Added — M4 (MITRE ATT&CK Enterprise) - **STIX 2.1 parser + upsert** (`app/services/mitre_seed.py`): stdlib-only (`urllib.request` + `hashlib`), pinned to Enterprise v19.0 (`enterprise-attack-19.0.json`, sha256 `df520ea0…`). Parses 25k+ STIX objects → 15 tactics, 222 techniques, 475 sub-techniques in ~1.1 s. Skips revoked + deprecated, resolves sub-technique parents via `relationship[subtechnique-of]` with a `T1003.001 → T1003` dotted-id fallback, copies kill-chain phases into the `mitre_technique_tactics` M2M. - **CLI**: `flask metamorph seed-mitre [--source ] [--checksum-sha256 ] [--skip-checksum]` (`app/cli.py`). `make seed-mitre` wraps it. diff --git a/README.md b/README.md index da324e7..59df5d2 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ Collaborative purple-team platform. Red team logs the tests they execute (procedure, command, timestamp); blue team annotates each test with detection evidence (alerts, logs, files). At the end of an engagement, Metamorph generates a standalone reveal.js slide deck classified by MITRE ATT&CK tactic. -> **Status**: M0–M4 delivered (bootstrap → DB schema → auth → RBAC → MITRE ATT&CK reference). See `tasks/spec.md` for the full specification and `tasks/todo.md` for the milestone-by-milestone plan. +> **Status**: M0–M5 delivered (bootstrap → DB schema → auth → RBAC → MITRE ATT&CK reference → test & scenario templates). See `tasks/spec.md` for the full specification and `tasks/todo.md` for the milestone-by-milestone plan. ## Stack @@ -11,6 +11,7 @@ Collaborative purple-team platform. Red team logs the tests they execute (proced - **Auth (M2+)**: JWT access (1h) + refresh (30d), Argon2id, invite-link enrollment. - **RBAC (M3+)**: atomic permissions (31 codes) bundled into custom groups; 3 system groups seeded (`admin` / `redteam` / `blueteam`). - **MITRE ATT&CK (M4+)**: Enterprise reference catalogue pinned to v19.0, seedable via `make seed-mitre`. +- **Template catalogue (M5+)**: reusable `test_templates` (markdown procedure, OPSEC level, free tags, expected IOCs, MITRE tags) + ordered `scenario_templates` with drag-and-drop reordering. Admin pages at `/admin/tests` and `/admin/scenarios`. - **Delivery**: docker-compose. TLS termination is expected to be handled by an external reverse proxy in production. ## Quickstart diff --git a/e2e/tests/m5-templates.spec.ts b/e2e/tests/m5-templates.spec.ts new file mode 100644 index 0000000..8a58963 --- /dev/null +++ b/e2e/tests/m5-templates.spec.ts @@ -0,0 +1,253 @@ +import { expect, test, type APIRequestContext, type Page } from '@playwright/test'; + +/** + * M5 — Test + Scenario template catalogue. + * + * Verifies CRUD on /test-templates and /scenario-templates plus the admin SPA + * pages. We do NOT seed the full MITRE bundle here — M4 already covers that + * suite. This spec only needs ONE technique resolvable from a STIX-like + * shape (we ride on the same `/diag/reset` then re-seed MITRE so tag refs + * resolve). + */ + +const ADMIN_EMAIL = `admin-${crypto.randomUUID().slice(0, 8)}@metamorph.local`; +const ADMIN_PASSWORD = 'AdminPass1234!'; + +async function resetAndMintToken(request: APIRequestContext): Promise { + const r = await request.post('/api/v1/diag/reset'); + expect(r.status()).toBe(200); + return (await r.json()).install_token as string; +} + +async function loginAndGetAccess( + request: APIRequestContext, + email: string, + password: string, +): Promise { + const r = await request.post('/api/v1/auth/login', { data: { email, password } }); + expect(r.status()).toBe(200); + return (await r.json()).access_token as string; +} + +async function loginViaSpa(page: Page, email: string, password: string) { + await page.goto('/login'); + await page.getByLabel(/email/i).fill(email); + await page.getByLabel(/password/i).fill(password); + await page.getByRole('button', { name: /sign in/i }).click(); + await expect(page.getByTestId('me-email')).toHaveText(email); +} + +test.describe.configure({ mode: 'serial' }); + +test.describe('M5 — Template catalogue', () => { + test.beforeAll(async ({ request }) => { + const installToken = await resetAndMintToken(request); + const setup = await request.post('/api/v1/setup', { + data: { install_token: installToken, email: ADMIN_EMAIL, password: ADMIN_PASSWORD }, + }); + expect(setup.status()).toBe(201); + // MITRE re-sync — picker + tag refs rely on the canonical bundle. + const access = await loginAndGetAccess(request, ADMIN_EMAIL, ADMIN_PASSWORD); + const sync = await request.post('/api/v1/mitre/sync', { + headers: { Authorization: `Bearer ${access}` }, + }); + expect(sync.status()).toBe(200); + }); + + test.afterAll(async ({ request }) => { + // Restore the stable admin (cf. memory feedback_metamorph_test_admin): + // any wipe should leave admin@metamorph.local / AdminPass1234! usable. + const installToken = await resetAndMintToken(request); + await request.post('/api/v1/setup', { + data: { + install_token: installToken, + email: 'admin@metamorph.local', + password: 'AdminPass1234!', + }, + }); + // Re-seed MITRE so subsequent manual sessions don't see an empty matrix. + const access = await loginAndGetAccess(request, 'admin@metamorph.local', 'AdminPass1234!'); + await request.post('/api/v1/mitre/sync', { + headers: { Authorization: `Bearer ${access}` }, + }); + }); + + // === API smoke ============================================================ + + test('CRUD test-templates via API', async ({ request }) => { + const access = await loginAndGetAccess(request, ADMIN_EMAIL, ADMIN_PASSWORD); + const auth = { Authorization: `Bearer ${access}` }; + + // Create + const r1 = await request.post('/api/v1/test-templates', { + headers: auth, + data: { + name: 'phish-link', + description: 'send a phishing email with tracked link', + objective: 'land a click', + procedure_md: '1. craft mail\n2. send\n3. await click', + opsec_level: 'low', + tags: ['phish', 'initial-access'], + expected_iocs: ['phish@example.com'], + mitre_tags: [ + { kind: 'tactic', external_id: 'TA0001' }, + { kind: 'technique', external_id: 'T1566' }, + ], + }, + }); + expect(r1.status(), await r1.text()).toBe(201); + const created = await r1.json(); + expect(created.name).toBe('phish-link'); + expect(created.mitre_tags.length).toBe(2); + expect(created.tags).toContain('phish'); + + // Update — partial: change opsec only + const r2 = await request.put(`/api/v1/test-templates/${created.id}`, { + headers: auth, + data: { opsec_level: 'high' }, + }); + expect(r2.status()).toBe(200); + const updated = await r2.json(); + expect(updated.opsec_level).toBe('high'); + expect(updated.name).toBe('phish-link'); // untouched + + // List + filter by tactic + const r3 = await request.get('/api/v1/test-templates?tactic=TA0001', { + headers: auth, + }); + expect(r3.status()).toBe(200); + const list = await r3.json(); + expect(list.items.map((it: { name: string }) => it.name)).toContain('phish-link'); + + // Reject unknown MITRE + const r4 = await request.post('/api/v1/test-templates', { + headers: auth, + data: { + name: 'bad', + mitre_tags: [{ kind: 'technique', external_id: 'T9999' }], + }, + }); + expect(r4.status()).toBe(400); + expect((await r4.json()).error).toBe('unknown_mitre_tag'); + + // Soft-delete + const r5 = await request.delete(`/api/v1/test-templates/${created.id}`, { + headers: auth, + }); + expect(r5.status()).toBe(200); + const r6 = await request.get('/api/v1/test-templates', { headers: auth }); + expect( + (await r6.json()).items.map((it: { name: string }) => it.name), + ).not.toContain('phish-link'); + }); + + test('Scenario template: create + reorder + soft-delete', async ({ request }) => { + const access = await loginAndGetAccess(request, ADMIN_EMAIL, ADMIN_PASSWORD); + const auth = { Authorization: `Bearer ${access}` }; + + async function mkTest(name: string): Promise { + const r = await request.post('/api/v1/test-templates', { + headers: auth, + data: { name }, + }); + expect(r.status()).toBe(201); + return (await r.json()).id as string; + } + + const a = await mkTest('scn-step-a'); + const b = await mkTest('scn-step-b'); + const c = await mkTest('scn-step-c'); + + // Create with [a, b, c] + const r1 = await request.post('/api/v1/scenario-templates', { + headers: auth, + data: { name: 'ordered-scenario', test_template_ids: [a, b, c] }, + }); + expect(r1.status()).toBe(201); + const sc = await r1.json(); + expect(sc.tests.map((t: { test_template_name: string }) => t.test_template_name)).toEqual([ + 'scn-step-a', + 'scn-step-b', + 'scn-step-c', + ]); + + // Reorder → [c, a, b] + const r2 = await request.put(`/api/v1/scenario-templates/${sc.id}/tests`, { + headers: auth, + data: { test_template_ids: [c, a, b] }, + }); + expect(r2.status()).toBe(200); + const after = await r2.json(); + expect(after.tests.map((t: { test_template_name: string }) => t.test_template_name)).toEqual([ + 'scn-step-c', + 'scn-step-a', + 'scn-step-b', + ]); + + // Soft-delete the scenario. + const r3 = await request.delete(`/api/v1/scenario-templates/${sc.id}`, { headers: auth }); + expect(r3.status()).toBe(200); + const list = await (await request.get('/api/v1/scenario-templates', { headers: auth })).json(); + expect(list.items.map((it: { name: string }) => it.name)).not.toContain('ordered-scenario'); + }); + + // === SPA smoke ============================================================ + + test('SPA — admin sees the test catalogue and can filter', async ({ page, request }) => { + // Seed two tests up front via the API — exercise the SPA list + filter + // pipeline without fighting the heavy create-modal (covered by API tests). + const access = await loginAndGetAccess(request, ADMIN_EMAIL, ADMIN_PASSWORD); + const auth = { Authorization: `Bearer ${access}` }; + await request.post('/api/v1/test-templates', { + headers: auth, + data: { name: 'spa-list-fast', opsec_level: 'low', tags: ['fast'] }, + }); + await request.post('/api/v1/test-templates', { + headers: auth, + data: { name: 'spa-list-slow', opsec_level: 'high' }, + }); + + await loginViaSpa(page, ADMIN_EMAIL, ADMIN_PASSWORD); + await page.goto('/admin/tests'); + + await expect(page.getByText('spa-list-fast')).toBeVisible(); + await expect(page.getByText('spa-list-slow')).toBeVisible(); + + await page.getByTestId('filter-opsec').selectOption('high'); + await expect(page.getByText('spa-list-slow')).toBeVisible(); + await expect(page.getByText('spa-list-fast')).toBeHidden(); + }); + + test('SPA — scenario list shows ordered tests with their position', async ({ page, request }) => { + // Seed a 3-test scenario via the API; the SPA must render the order as + // saved. Pointer-event drag is flaky in CI, and the API-level reorder + // test already covers the persistence pipeline. + const access = await loginAndGetAccess(request, ADMIN_EMAIL, ADMIN_PASSWORD); + const auth = { Authorization: `Bearer ${access}` }; + const ids: string[] = []; + for (const name of ['drag-1', 'drag-2', 'drag-3']) { + const r = await request.post('/api/v1/test-templates', { + headers: auth, + data: { name }, + }); + ids.push((await r.json()).id); + } + const scResp = await request.post('/api/v1/scenario-templates', { + headers: auth, + data: { + name: 'spa-rendered-scenario', + test_template_ids: [ids[2], ids[0], ids[1]], + }, + }); + const scId = (await scResp.json()).id; + + await loginViaSpa(page, ADMIN_EMAIL, ADMIN_PASSWORD); + await page.goto('/admin/scenarios'); + + const card = page.locator(`[data-testid="scenario-row-${scId}"]`); + await expect(card).toBeVisible(); + await expect(card.getByText('1. drag-3')).toBeVisible(); + await expect(card.getByText('2. drag-1')).toBeVisible(); + await expect(card.getByText('3. drag-2')).toBeVisible(); + }); +}); diff --git a/tasks/lessons.md b/tasks/lessons.md index 4386376..2143545 100644 --- a/tasks/lessons.md +++ b/tasks/lessons.md @@ -78,6 +78,18 @@ project: Metamorph - **`podman compose stop api` puis `up -d api` casse les dépendances** entre containers (`db` healthy → `api` depends on it) : podman-compose ne résout pas la chaîne de deps quand on cible un seul service. Pour un override d'env, mieux vaut `make down && APP_ENV=test make up`. - **`/diag/reset` test-only** : exposer un endpoint qui truncate la DB est tentant pour les e2e mais ouvre une grosse surface en cas de fuite. Compromise actuel : autorisé en `dev` ET `test` (pas en prod), avec un log `WARNING` à chaque appel. Si jamais on déploie une stack dev publique, **désactiver** l'endpoint via env var. +## 2026-05-12 — M5 templates + scenarios + +- **`extra={"name": ...}` dans `log.info()` crash silencieusement** — Python's `logging.LogRecord` réserve `name` (le logger name). Coût : 500 sur le POST, message peu parlant (`KeyError: "Attempt to overwrite 'name' in LogRecord"`). Fix : renommer la clé (`template_name`). Liste réservée à éviter : `name`, `msg`, `args`, `levelname`, `levelno`, `pathname`, `filename`, `module`, `funcName`, `created`, `msecs`, `lineno`, `thread`, `threadName`, `process`. Pattern : préfixer les clés extra par l'entité (`template_name`, `group_id`, `user_id` est OK mais `id` aussi est piégeux dans certains setups). +- **React 18 + `setX((prev) => ({...prev, val: e.currentTarget.value }))` → page blanche au 1er input.** `e.currentTarget` est cleared après la fin du bubble, AVANT que l'updater fonctionnel exécute. Le synthetic event survit (pas de pooling depuis React 17), mais `currentTarget` est setté/cleared par le dispatcher. Fix : `e.target.value` (qui persiste sur le synthetic event), ou capturer `const v = e.currentTarget.value;` avant le `setX`. À garder en tête : tout `onChange` qui passe par un updater fonctionnel doit lire `e.target`, pas `e.currentTarget`. +- **Sentinel `Any = object()` plutôt que `... (Ellipsis)`** pour les "field unset" optional en service Python. Pyright voit `... = object()` correctement comme `Any`, alors que `description: str | None | object = ...` rend `description.strip()` invalide. Pattern : `_UNSET: Any = object()` au top du module + `description: Any = _UNSET` dans la signature + `if description is not _UNSET: ...`. Net + typecheck-friendly. +- **Postgres UNIQUE(scenario_id, position) + position-swap = ON CONFLICT pendant l'UPDATE.** Pour réordonner, le pattern naïf (UPDATE position) viole la contrainte sur le 1er swap. Trois options : (a) full delete + re-insert dans la même tx [retenu, atomique + lisible], (b) shift d'offset (UPDATE position = position + 1000 puis renumérotation), (c) deferred constraint. (a) gagne en simplicité — la liste rarement >50 éléments, le coût est négligeable. +- **`@dnd-kit/sortable` requires `useSortable({ id })` IDs to be unique and stable across renders.** Si on utilise un index numérique comme id, drag-and-drop ne réagit pas. Utiliser `test_template_id` (UUID stable) marche directement. +- **Frontend deps ajoutés à `package.json` sans `package-lock.json`** : le Dockerfile fait `npm install --no-audit --no-fund` sur fallback. OK pour M5 (3 deps `@dnd-kit/*`). À l'avenir, freeze un lockfile avant M14 pour build reproductibles. +- **Playwright `getByTestId` est défini par `testIdAttributeName: 'data-testid'`** dans `playwright.config.ts`. Pour qu'un test-id descende sur l'input via TextField, il faut que `...rest` soit spread sur l'input (déjà OK dans `TextField.tsx`). Mais avec un wrapper `
`, `getByTestId` matche le DIV si le test-id est dessus. Bien le mettre sur l'élément interactif (input/button), pas sur le container. +- **`/diag/reset` truncate order matters** : `scenario_template_tests.test_template_id` est FK `ON DELETE RESTRICT`, donc il faut truncate `scenario_template_tests` AVANT `test_templates`. Hierarchy : `scenario_template_tests → scenario_templates → test_template_mitre_tags → test_templates → mitre_*`. Maintenant inscrite dans `diag.py`. +- **Modal embarquant le `MitreTagPicker` complet (15 cols × 50 techniques)** : le picker se charge via `/mitre/matrix` (~94 KB). Affichage instantané, OK. Pour de futurs modals lourds, considérer le lazy-render derrière un toggle ou tab. +