Compare commits
4 Commits
7a69f10f3e
...
2c85f9b57e
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2c85f9b57e | ||
|
|
8b1de6e258 | ||
|
|
54adfee690 | ||
|
|
63b48addc0 |
17
CHANGELOG.md
17
CHANGELOG.md
@@ -14,8 +14,8 @@ All notable changes to this project will be documented here. Format: [Keep a Cha
|
|||||||
- **Persisted metadata** in `settings`: `mitre_last_sync`, `mitre_version`, `mitre_source_url`.
|
- **Persisted metadata** in `settings`: `mitre_last_sync`, `mitre_version`, `mitre_source_url`.
|
||||||
- **Compose volume `metamorph_mitre`** mounted at `/data/mitre/` in the api container — caches the downloaded bundle across restarts. Owned by `metamorph:metamorph`.
|
- **Compose volume `metamorph_mitre`** mounted at `/data/mitre/` in the api container — caches the downloaded bundle across restarts. Owned by `metamorph:metamorph`.
|
||||||
- **Frontend**:
|
- **Frontend**:
|
||||||
- `<MitreTagPicker>` component: 3-column tactic → technique → sub-technique with multi-select chips, autocomplete on each column. Returns `MitreTag[]` (`kind`, `id`, `external_id`, `name`), ready for M5 templates.
|
- `<MitreTagPicker>` component: flat ATT&CK matrix matching `attack.mitre.org/#` — full-bleed beyond `max-w-page`, 15 equal-width columns via `grid-template-columns: repeat(N, minmax(7rem, 1fr))`, sans-serif 12px, **name-only cells** (external_id surfaces on hover via `title` and in selection chips), `▸/▾` chevron expands sub-techniques inline within the column, multi-select with chip-removal at the top. Returns `MitreTag[]` (`kind`, `id`, `external_id`, `name`), ready for M5 templates.
|
||||||
- `/mitre` showcase page with status card, admin-gated **Trigger sync** button, picker preview, and `<pre>` payload preview.
|
- `/mitre` showcase page with status card, admin-gated **Trigger sync** button, the picker, and a JSON `<pre>` preview of the current selection.
|
||||||
- Nav adds **MITRE** link for any logged-in user.
|
- Nav adds **MITRE** link for any logged-in user.
|
||||||
- **Testing**:
|
- **Testing**:
|
||||||
- `backend/tests/test_mitre.py` — **12 pytest** (parser, idempotence, checksum mismatch, persisted settings, endpoint variants, perm enforcement) using a hand-crafted minimal STIX bundle (no network in tests).
|
- `backend/tests/test_mitre.py` — **12 pytest** (parser, idempotence, checksum mismatch, persisted settings, endpoint variants, perm enforcement) using a hand-crafted minimal STIX bundle (no network in tests).
|
||||||
@@ -27,6 +27,17 @@ All notable changes to this project will be documented here. Format: [Keep a Cha
|
|||||||
- **`/diag/reset` consistency**: now truncates the `mitre_*` tables alongside `settings` so `GET /mitre/status` and `GET /mitre/tactics` agree after a reset (previously: catalogue rows persisted, but `mitre_last_sync` got wiped → status lied).
|
- **`/diag/reset` consistency**: now truncates the `mitre_*` tables alongside `settings` so `GET /mitre/status` and `GET /mitre/tactics` agree after a reset (previously: catalogue rows persisted, but `mitre_last_sync` got wiped → status lied).
|
||||||
- **Spec drift §10 #4**: amended "14 tactics" → "≥ 14 tactics (v19 ships 15)" to reflect MITRE v8+ reality.
|
- **Spec drift §10 #4**: amended "14 tactics" → "≥ 14 tactics (v19 ships 15)" to reflect MITRE v8+ reality.
|
||||||
|
|
||||||
|
### Fixed (post-M4 code-review pass)
|
||||||
|
- **SSRF allowlist on `/mitre/sync`**: host must be in `MITRE_ALLOWED_HOSTS` (defaults to `raw.githubusercontent.com`, comma-separated env override). Closes the "admin holding `mitre.sync` can pivot the api container at cloud metadata (`169.254.169.254`) or internal mirrors" vector. New `MitreSourceForbidden` exception → 400 with `source_forbidden` error code.
|
||||||
|
- **Concurrent sync race**: `seed_mitre()` now acquires `pg_advisory_xact_lock(hashtext('mitre.seed'))` at the top of the transaction so two `/mitre/sync` calls serialise cleanly across the `DELETE` + re-`INSERT` of `mitre_technique_tactics`.
|
||||||
|
- **Typed sync contract end-to-end**: Pydantic `SyncResultOut` on the backend (`app/api/mitre.py`) mirrored by a `MitreSyncResult` TS interface (`frontend/src/lib/mitre.ts`). The MitrePage mutation no longer uses an `as Record<string, unknown>` escape hatch.
|
||||||
|
- **N+1 in dotted sub-technique fallback**: pre-built `{external_id → id}` dict at function entry; was firing one extra SELECT per orphan (currently 0 with MITRE, but a latent footgun for partial bundles).
|
||||||
|
- **`SETTING_VERSION` cleared explicitly when source != default**: previously kept the stale pinned version after a custom-URL re-sync; now `_upsert_setting(..., None)` so `/mitre/status` doesn't lie.
|
||||||
|
- **Internal error scrub on `/mitre/sync`**: 500 responses no longer leak URLError / DB driver text via `str(e)` — stack lands in JSON logs only.
|
||||||
|
- **E2E pinned to exact MITRE v19 counts** (15/222/475/0 orphans) for parser-regression detection; previously `>=` thresholds could mask "revoked tactics silently included".
|
||||||
|
- **E2E uses `crypto.randomUUID()`** instead of `Math.random()` for unique test emails.
|
||||||
|
- **Test coverage for security guards**: `file://` rejection, disallowed HTTPS host, custom-URL-without-sha refusal, dotted-id fallback, version-clearing semantics — 5 new pytest covering paths the spec-review demanded but no test enforced.
|
||||||
|
|
||||||
### Decisions (intentional)
|
### Decisions (intentional)
|
||||||
- **Bundle "embarqué" interpreted as seed-time download + named-volume cache**, not "binary baked into the Docker image". Keeps the image ~150 MB, makes version bumps a constant edit, plays nicely with `make seed-mitre` re-runs. Air-gapped operators copy the file into the volume + pass `--source /data/mitre/<file>`.
|
- **Bundle "embarqué" interpreted as seed-time download + named-volume cache**, not "binary baked into the Docker image". Keeps the image ~150 MB, makes version bumps a constant edit, plays nicely with `make seed-mitre` re-runs. Air-gapped operators copy the file into the volume + pass `--source /data/mitre/<file>`.
|
||||||
- **Read endpoints unauthenticated-perm-wise but auth-required**: MITRE data is public reference material — no `mitre.read` perm. Status endpoint is similarly open (under `@require_auth`) to keep `/mitre/status` simple for the UI badge.
|
- **Read endpoints unauthenticated-perm-wise but auth-required**: MITRE data is public reference material — no `mitre.read` perm. Status endpoint is similarly open (under `@require_auth`) to keep `/mitre/status` simple for the UI badge.
|
||||||
@@ -34,7 +45,7 @@ All notable changes to this project will be documented here. Format: [Keep a Cha
|
|||||||
|
|
||||||
### Validated end-to-end (M4 DoD)
|
### Validated end-to-end (M4 DoD)
|
||||||
- `make clean && make up && make migrate && make seed-mitre` → 15 tactics / 222 techniques / 475 sub-techniques / 254 links / 0 orphans / ~1.1 s.
|
- `make clean && make up && make migrate && make seed-mitre` → 15 tactics / 222 techniques / 475 sub-techniques / 254 links / 0 orphans / ~1.1 s.
|
||||||
- `make test-api` → **53 pytest pass** (1 health + 8 schema + 15 auth + 15 RBAC + 14 MITRE) in ~5 s.
|
- `make test-api` → **58 pytest pass** (1 health + 8 schema + 15 auth + 15 RBAC + 19 MITRE) in ~5 s.
|
||||||
- `make e2e` → **34 Playwright pass** (8 M0 + 4 M1 + 8 M2 + 8 M3 + 6 M4) in ~18 s.
|
- `make e2e` → **34 Playwright pass** (8 M0 + 4 M1 + 8 M2 + 8 M3 + 6 M4) in ~18 s.
|
||||||
- Spec-reviewer PASS after fixes applied.
|
- Spec-reviewer PASS after fixes applied.
|
||||||
|
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
|||||||
import logging
|
import logging
|
||||||
|
|
||||||
from flask import Blueprint, jsonify, request
|
from flask import Blueprint, jsonify, request
|
||||||
|
from pydantic import BaseModel
|
||||||
from sqlalchemy import func, or_, select
|
from sqlalchemy import func, or_, select
|
||||||
|
|
||||||
from app.core.auth_decorators import require_auth, require_perm
|
from app.core.auth_decorators import require_auth, require_perm
|
||||||
@@ -16,6 +17,21 @@ from app.db.session import session_scope
|
|||||||
from app.models.mitre import MitreSubtechnique, MitreTactic, MitreTechnique
|
from app.models.mitre import MitreSubtechnique, MitreTactic, MitreTechnique
|
||||||
from app.services import mitre_seed as mitre_seed_svc
|
from app.services import mitre_seed as mitre_seed_svc
|
||||||
|
|
||||||
|
|
||||||
|
class SyncResultOut(BaseModel):
|
||||||
|
"""Response schema for `POST /mitre/sync`. Mirrors `SeedResult.as_dict()`."""
|
||||||
|
|
||||||
|
tactics_upserted: int
|
||||||
|
techniques_upserted: int
|
||||||
|
subtechniques_upserted: int
|
||||||
|
subtechniques_skipped_orphan: int
|
||||||
|
technique_tactic_links: int
|
||||||
|
version: str | None
|
||||||
|
source: str
|
||||||
|
started_at: str
|
||||||
|
finished_at: str
|
||||||
|
duration_ms: int
|
||||||
|
|
||||||
bp = Blueprint("mitre", __name__, url_prefix="/mitre")
|
bp = Blueprint("mitre", __name__, url_prefix="/mitre")
|
||||||
log = logging.getLogger("metamorph.api.mitre")
|
log = logging.getLogger("metamorph.api.mitre")
|
||||||
|
|
||||||
@@ -248,7 +264,8 @@ def sync():
|
|||||||
|
|
||||||
Custom `source` URLs MUST be paired with either `expected_sha256` (integrity
|
Custom `source` URLs MUST be paired with either `expected_sha256` (integrity
|
||||||
guarantee) or `allow_unverified: true` (explicit opt-out) — the seed service
|
guarantee) or `allow_unverified: true` (explicit opt-out) — the seed service
|
||||||
will raise otherwise.
|
will raise otherwise. The host is allowlisted (defaults to
|
||||||
|
raw.githubusercontent.com, overridable via the MITRE_ALLOWED_HOSTS env).
|
||||||
"""
|
"""
|
||||||
payload = request.get_json(silent=True) or {}
|
payload = request.get_json(silent=True) or {}
|
||||||
source = payload.get("source") # optional URL override
|
source = payload.get("source") # optional URL override
|
||||||
@@ -261,12 +278,19 @@ def sync():
|
|||||||
or (mitre_seed_svc.MITRE_DEFAULT_SHA256 if source is None else None),
|
or (mitre_seed_svc.MITRE_DEFAULT_SHA256 if source is None else None),
|
||||||
allow_unverified=allow_unverified,
|
allow_unverified=allow_unverified,
|
||||||
)
|
)
|
||||||
|
except mitre_seed_svc.MitreSourceForbidden as e:
|
||||||
|
return jsonify({"error": "source_forbidden", "message": str(e)}), 400
|
||||||
except mitre_seed_svc.MitreChecksumMismatch as e:
|
except mitre_seed_svc.MitreChecksumMismatch as e:
|
||||||
return jsonify({"error": "checksum_mismatch", "message": str(e)}), 502
|
return jsonify({"error": "checksum_mismatch", "message": str(e)}), 502
|
||||||
except mitre_seed_svc.MitreSeedError as e:
|
except mitre_seed_svc.MitreSeedError as e:
|
||||||
return jsonify({"error": "seed_failed", "message": str(e)}), 502
|
return jsonify({"error": "seed_failed", "message": str(e)}), 502
|
||||||
except Exception as e: # noqa: BLE001
|
except Exception: # noqa: BLE001
|
||||||
|
# Do NOT leak the internal error string to the client (URLError stack,
|
||||||
|
# DB driver text). The stack lands in our JSON logs.
|
||||||
log.exception("metamorph.api.mitre.sync_failed")
|
log.exception("metamorph.api.mitre.sync_failed")
|
||||||
return jsonify({"error": "internal_error", "message": str(e)}), 500
|
return jsonify({"error": "internal_error"}), 500
|
||||||
log.warning("metamorph.api.mitre.sync_done", extra=result.as_dict())
|
# Validate via the Pydantic Out model so the response contract is
|
||||||
return jsonify(result.as_dict())
|
# explicit (single source of truth shared with the TS interface).
|
||||||
|
payload_out = SyncResultOut.model_validate(result.as_dict()).model_dump()
|
||||||
|
log.info("metamorph.api.mitre.sync_done", extra=payload_out)
|
||||||
|
return jsonify(payload_out)
|
||||||
|
|||||||
@@ -29,7 +29,7 @@ from datetime import datetime, timezone
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Iterable
|
from typing import Iterable
|
||||||
|
|
||||||
from sqlalchemy import delete, select
|
from sqlalchemy import delete, select, text as sql_text
|
||||||
|
|
||||||
from app.db.session import session_scope
|
from app.db.session import session_scope
|
||||||
from app.models.mitre import (
|
from app.models.mitre import (
|
||||||
@@ -59,6 +59,18 @@ MITRE_DEFAULT_SHA256 = "df520ea0775a57db7bff760145b02fed89290802913e056b7ed5970b
|
|||||||
MITRE_BUNDLE_CACHE_PATH = Path(os.environ.get("MITRE_CACHE_DIR", "/data/mitre"))
|
MITRE_BUNDLE_CACHE_PATH = Path(os.environ.get("MITRE_CACHE_DIR", "/data/mitre"))
|
||||||
MITRE_DOWNLOAD_TIMEOUT_SECONDS = 120
|
MITRE_DOWNLOAD_TIMEOUT_SECONDS = 120
|
||||||
|
|
||||||
|
# Hosts authorised as a source for a MITRE sync. An admin holding `mitre.sync`
|
||||||
|
# could otherwise pivot the in-container HTTP fetch to internal services
|
||||||
|
# (169.254.169.254, db, internal mirrors). Override via the `MITRE_ALLOWED_HOSTS`
|
||||||
|
# env (comma-separated) when running against a private mirror.
|
||||||
|
MITRE_ALLOWED_HOSTS: frozenset[str] = frozenset(
|
||||||
|
h.strip()
|
||||||
|
for h in os.environ.get(
|
||||||
|
"MITRE_ALLOWED_HOSTS", "raw.githubusercontent.com"
|
||||||
|
).split(",")
|
||||||
|
if h.strip()
|
||||||
|
)
|
||||||
|
|
||||||
# Settings keys used to expose the seed metadata to the operator UI/CLI.
|
# Settings keys used to expose the seed metadata to the operator UI/CLI.
|
||||||
SETTING_LAST_SYNC = "mitre_last_sync"
|
SETTING_LAST_SYNC = "mitre_last_sync"
|
||||||
SETTING_VERSION = "mitre_version"
|
SETTING_VERSION = "mitre_version"
|
||||||
@@ -76,6 +88,10 @@ class MitreChecksumMismatch(MitreSeedError):
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class MitreSourceForbidden(MitreSeedError):
|
||||||
|
"""The provided source URL points to a host outside the allowlist."""
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class ParsedBundle:
|
class ParsedBundle:
|
||||||
tactics: list[dict] = field(default_factory=list)
|
tactics: list[dict] = field(default_factory=list)
|
||||||
@@ -123,6 +139,18 @@ def _is_url(source: str) -> bool:
|
|||||||
return parsed.scheme in ("http", "https")
|
return parsed.scheme in ("http", "https")
|
||||||
|
|
||||||
|
|
||||||
|
def _ensure_host_allowed(url: str) -> None:
|
||||||
|
"""Raise MitreSourceForbidden if the URL targets a non-allowlisted host."""
|
||||||
|
parsed = urllib.parse.urlparse(url)
|
||||||
|
if parsed.scheme not in ("http", "https"):
|
||||||
|
raise MitreSourceForbidden(f"unsupported URL scheme: {parsed.scheme!r}")
|
||||||
|
host = (parsed.hostname or "").lower()
|
||||||
|
if host not in MITRE_ALLOWED_HOSTS:
|
||||||
|
raise MitreSourceForbidden(
|
||||||
|
f"host {host!r} not in MITRE_ALLOWED_HOSTS={sorted(MITRE_ALLOWED_HOSTS)}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _sha256_of(path: Path) -> str:
|
def _sha256_of(path: Path) -> str:
|
||||||
h = hashlib.sha256()
|
h = hashlib.sha256()
|
||||||
with path.open("rb") as f:
|
with path.open("rb") as f:
|
||||||
@@ -132,6 +160,7 @@ def _sha256_of(path: Path) -> str:
|
|||||||
|
|
||||||
|
|
||||||
def _download(url: str, dest: Path, *, expected_sha256: str | None = None) -> Path:
|
def _download(url: str, dest: Path, *, expected_sha256: str | None = None) -> Path:
|
||||||
|
_ensure_host_allowed(url)
|
||||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||||
tmp = dest.with_suffix(dest.suffix + ".part")
|
tmp = dest.with_suffix(dest.suffix + ".part")
|
||||||
log.info("metamorph.mitre.download.start", extra={"url": url, "dest": str(dest)})
|
log.info("metamorph.mitre.download.start", extra={"url": url, "dest": str(dest)})
|
||||||
@@ -331,8 +360,18 @@ def _upsert_subtechniques(
|
|||||||
subtechniques: Iterable[dict],
|
subtechniques: Iterable[dict],
|
||||||
stix_to_tech_id: dict,
|
stix_to_tech_id: dict,
|
||||||
) -> tuple[int, int]:
|
) -> tuple[int, int]:
|
||||||
"""Returns (n_upserted, n_skipped_orphans)."""
|
"""Returns (n_upserted, n_skipped_orphans).
|
||||||
|
|
||||||
|
`n_upserted` is the count of rows whose state was applied (INSERT or
|
||||||
|
UPDATE) — matches Postgres upsert semantics.
|
||||||
|
"""
|
||||||
existing = {sb.external_id: sb for sb in s.scalars(select(MitreSubtechnique)).all()}
|
existing = {sb.external_id: sb for sb in s.scalars(select(MitreSubtechnique)).all()}
|
||||||
|
# Pre-index techniques by external_id so the dotted-id fallback doesn't
|
||||||
|
# issue N+1 SELECTs (was a latent footgun for partial-bundle re-syncs).
|
||||||
|
parent_by_external: dict[str, object] = {
|
||||||
|
t.external_id: t.id
|
||||||
|
for t in s.scalars(select(MitreTechnique)).all()
|
||||||
|
}
|
||||||
n_upserted = 0
|
n_upserted = 0
|
||||||
n_skipped = 0
|
n_skipped = 0
|
||||||
for sb in subtechniques:
|
for sb in subtechniques:
|
||||||
@@ -342,17 +381,7 @@ def _upsert_subtechniques(
|
|||||||
# Fall back to the dotted external_id convention (T1003.001 → T1003).
|
# Fall back to the dotted external_id convention (T1003.001 → T1003).
|
||||||
m = re.match(r"^(T\d+)\.\d+$", sb["external_id"])
|
m = re.match(r"^(T\d+)\.\d+$", sb["external_id"])
|
||||||
if m:
|
if m:
|
||||||
parent_ext = m.group(1)
|
parent_id = parent_by_external.get(m.group(1))
|
||||||
# We don't have a parent-by-external-id map here; query.
|
|
||||||
parent_row = next(
|
|
||||||
iter(
|
|
||||||
s.scalars(
|
|
||||||
select(MitreTechnique).where(MitreTechnique.external_id == parent_ext)
|
|
||||||
).all()
|
|
||||||
),
|
|
||||||
None,
|
|
||||||
)
|
|
||||||
parent_id = parent_row.id if parent_row else None
|
|
||||||
if parent_id is None:
|
if parent_id is None:
|
||||||
log.warning(
|
log.warning(
|
||||||
"metamorph.mitre.orphan_subtechnique",
|
"metamorph.mitre.orphan_subtechnique",
|
||||||
@@ -433,6 +462,13 @@ def seed_mitre(
|
|||||||
)
|
)
|
||||||
|
|
||||||
with session_scope() as s:
|
with session_scope() as s:
|
||||||
|
# Serialize concurrent /mitre/sync calls. The lock is transaction-scoped
|
||||||
|
# (released automatically at COMMIT/ROLLBACK), so a second sync arriving
|
||||||
|
# while the first is mid-DELETE+INSERT of `mitre_technique_tactics`
|
||||||
|
# blocks until the first commits. Avoids the unique-constraint race the
|
||||||
|
# code-reviewer flagged. hashtext() is stable across sessions.
|
||||||
|
s.execute(sql_text("SELECT pg_advisory_xact_lock(hashtext('mitre.seed'))"))
|
||||||
|
|
||||||
short_to_tactic_id, n_tactics = _upsert_tactics(s, parsed.tactics)
|
short_to_tactic_id, n_tactics = _upsert_tactics(s, parsed.tactics)
|
||||||
stix_to_tech_id, n_techs, n_links = _upsert_techniques(
|
stix_to_tech_id, n_techs, n_links = _upsert_techniques(
|
||||||
s, parsed.techniques, short_to_tactic_id
|
s, parsed.techniques, short_to_tactic_id
|
||||||
@@ -441,9 +477,10 @@ def seed_mitre(
|
|||||||
|
|
||||||
finished_at = datetime.now(tz=timezone.utc)
|
finished_at = datetime.now(tz=timezone.utc)
|
||||||
_upsert_setting(s, SETTING_LAST_SYNC, finished_at.isoformat())
|
_upsert_setting(s, SETTING_LAST_SYNC, finished_at.isoformat())
|
||||||
# If the URL is the pinned one, we know the version; otherwise leave None.
|
# `version` reflects the known pin only when seeded from MITRE_DEFAULT_URL;
|
||||||
|
# otherwise we explicitly clear it so /mitre/status doesn't lie about a
|
||||||
|
# stale version after a custom-URL re-sync.
|
||||||
version = MITRE_VERSION if source_label == MITRE_DEFAULT_URL else None
|
version = MITRE_VERSION if source_label == MITRE_DEFAULT_URL else None
|
||||||
if version:
|
|
||||||
_upsert_setting(s, SETTING_VERSION, version)
|
_upsert_setting(s, SETTING_VERSION, version)
|
||||||
_upsert_setting(s, SETTING_SOURCE_URL, source_label)
|
_upsert_setting(s, SETTING_SOURCE_URL, source_label)
|
||||||
|
|
||||||
|
|||||||
@@ -8,7 +8,6 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import json
|
import json
|
||||||
import secrets
|
import secrets
|
||||||
import uuid
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
@@ -245,12 +244,17 @@ def test_seed_persists_setting(app, fixture_bundle_path):
|
|||||||
assert status["version"] is None # only set when source == MITRE_DEFAULT_URL
|
assert status["version"] is None # only set when source == MITRE_DEFAULT_URL
|
||||||
|
|
||||||
|
|
||||||
def test_checksum_mismatch_aborts(tmp_path):
|
def test_checksum_mismatch_aborts(tmp_path, monkeypatch):
|
||||||
"""A wrong sha256 triggers MitreChecksumMismatch and skips DB writes."""
|
"""A wrong sha256 triggers MitreChecksumMismatch and skips DB writes.
|
||||||
|
|
||||||
|
We monkey-patch the allowlist to accept `file://` for the duration of the
|
||||||
|
test — file:// is rejected in production by `_ensure_host_allowed` (cf.
|
||||||
|
`test_seed_refuses_file_url`), but we need to drive `_download` past that
|
||||||
|
gate to exercise the sha256 path.
|
||||||
|
"""
|
||||||
path = tmp_path / "tiny.json"
|
path = tmp_path / "tiny.json"
|
||||||
path.write_text(json.dumps(MINIMAL_BUNDLE))
|
path.write_text(json.dumps(MINIMAL_BUNDLE))
|
||||||
# Force the URL path so download() is invoked. We mock by passing a file:// URL.
|
monkeypatch.setattr(mitre_svc, "_ensure_host_allowed", lambda _: None)
|
||||||
# Simpler: call _download() directly with a bogus hash.
|
|
||||||
bogus = "0" * 64
|
bogus = "0" * 64
|
||||||
with pytest.raises(mitre_svc.MitreChecksumMismatch):
|
with pytest.raises(mitre_svc.MitreChecksumMismatch):
|
||||||
mitre_svc._download(
|
mitre_svc._download(
|
||||||
@@ -387,3 +391,63 @@ def test_matrix_endpoint_requires_auth(app, fixture_bundle_path):
|
|||||||
mitre_svc.seed_mitre(source=fixture_bundle_path, expected_sha256=None)
|
mitre_svc.seed_mitre(source=fixture_bundle_path, expected_sha256=None)
|
||||||
with app.test_client() as c:
|
with app.test_client() as c:
|
||||||
assert c.get("/api/v1/mitre/matrix").status_code == 401
|
assert c.get("/api/v1/mitre/matrix").status_code == 401
|
||||||
|
|
||||||
|
|
||||||
|
# === Security guards ==========================================================
|
||||||
|
|
||||||
|
|
||||||
|
def test_seed_refuses_file_url(tmp_path):
|
||||||
|
"""file:// (or any scheme outside the allowlist) is rejected — protects
|
||||||
|
against a privileged operator pivoting the in-container fetch to local
|
||||||
|
filesystem reads via the URL path."""
|
||||||
|
path = tmp_path / "bundle.json"
|
||||||
|
path.write_text(json.dumps(MINIMAL_BUNDLE))
|
||||||
|
with pytest.raises(mitre_svc.MitreSourceForbidden):
|
||||||
|
mitre_svc._download(f"file://{path}", tmp_path / "out.json")
|
||||||
|
|
||||||
|
|
||||||
|
def test_seed_refuses_disallowed_https_host(tmp_path):
|
||||||
|
"""An HTTPS URL outside MITRE_ALLOWED_HOSTS is rejected before any I/O.
|
||||||
|
Closes the SSRF surface (cloud metadata, internal mirrors)."""
|
||||||
|
with pytest.raises(mitre_svc.MitreSourceForbidden):
|
||||||
|
mitre_svc._download("https://attacker.example/bundle.json", tmp_path / "out.json")
|
||||||
|
|
||||||
|
|
||||||
|
def test_seed_refuses_custom_url_without_sha(tmp_path):
|
||||||
|
"""End-to-end refusal: even an allowlisted custom URL needs a sha or an
|
||||||
|
explicit allow_unverified=True."""
|
||||||
|
# Use the default URL with a different sha to simulate "custom" semantics
|
||||||
|
# without actually hitting the network: pass a different MITRE_DEFAULT_URL.
|
||||||
|
# The cleanest expression is to call seed_mitre with the same URL but no sha
|
||||||
|
# — but the default URL gets the default sha auto-set; we need to bypass.
|
||||||
|
with pytest.raises(mitre_svc.MitreSeedError):
|
||||||
|
mitre_svc.seed_mitre(
|
||||||
|
source="https://raw.githubusercontent.com/some-other-path/bundle.json",
|
||||||
|
expected_sha256=None,
|
||||||
|
allow_unverified=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_dotted_id_fallback_resolves_orphan_subtechnique(app, tmp_path):
|
||||||
|
"""When the STIX `subtechnique-of` relationship is missing, the parser
|
||||||
|
must fall back to the dotted convention (T1003.001 → T1003)."""
|
||||||
|
bundle = json.loads(json.dumps(MINIMAL_BUNDLE)) # deep copy
|
||||||
|
# Strip the relationship object so the parent_stix_id lookup fails.
|
||||||
|
bundle["objects"] = [o for o in bundle["objects"] if o.get("type") != "relationship"]
|
||||||
|
bundle_path = tmp_path / "no-rel.json"
|
||||||
|
bundle_path.write_text(json.dumps(bundle))
|
||||||
|
|
||||||
|
result = mitre_svc.seed_mitre(source=bundle_path, expected_sha256=None)
|
||||||
|
# The fallback resolves T1059.001 → T1059 via the dotted-id pattern,
|
||||||
|
# so the subtechnique is still attached (no orphan).
|
||||||
|
assert result.subtechniques_upserted == 1
|
||||||
|
assert result.subtechniques_skipped_orphan == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_seed_clears_version_when_source_is_not_default(app, fixture_bundle_path):
|
||||||
|
"""A custom source must NULL `mitre_version` so /mitre/status doesn't lie
|
||||||
|
about a stale upstream pin."""
|
||||||
|
# First seed from the default URL would set version=19.0; here we seed from
|
||||||
|
# a local file path, which should write version=None.
|
||||||
|
mitre_svc.seed_mitre(source=fixture_bundle_path, expected_sha256=None)
|
||||||
|
assert mitre_svc.read_status()["version"] is None
|
||||||
|
|||||||
@@ -9,7 +9,9 @@ import { expect, test, type APIRequestContext, type Page } from '@playwright/tes
|
|||||||
* + the picker UI.
|
* + the picker UI.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
const ADMIN_EMAIL = `admin-${Math.floor(Math.random() * 1e6)}@metamorph.local`;
|
// crypto.randomUUID() guarantees uniqueness across parallel test runs; the
|
||||||
|
// Math.random() previous pattern could collide one-in-a-million in CI.
|
||||||
|
const ADMIN_EMAIL = `admin-${crypto.randomUUID().slice(0, 8)}@metamorph.local`;
|
||||||
const ADMIN_PASSWORD = 'AdminPass1234!';
|
const ADMIN_PASSWORD = 'AdminPass1234!';
|
||||||
|
|
||||||
async function resetAndMintToken(request: APIRequestContext): Promise<string> {
|
async function resetAndMintToken(request: APIRequestContext): Promise<string> {
|
||||||
@@ -55,9 +57,13 @@ test.describe('M4 — MITRE ATT&CK reference', () => {
|
|||||||
});
|
});
|
||||||
expect(sync.status(), `mitre sync failed: ${await sync.text()}`).toBe(200);
|
expect(sync.status(), `mitre sync failed: ${await sync.text()}`).toBe(200);
|
||||||
const result = await sync.json();
|
const result = await sync.json();
|
||||||
expect(result.tactics_upserted).toBeGreaterThanOrEqual(14);
|
// Pinned exactly to MITRE Enterprise v19.0 — bump alongside MITRE_VERSION
|
||||||
expect(result.techniques_upserted).toBeGreaterThanOrEqual(180);
|
// in `app/services/mitre_seed.py` when the pin changes. Exact counts catch
|
||||||
expect(result.subtechniques_upserted).toBeGreaterThanOrEqual(400);
|
// parser regressions that would silently include revoked/deprecated rows.
|
||||||
|
expect(result.tactics_upserted).toBe(15);
|
||||||
|
expect(result.techniques_upserted).toBe(222);
|
||||||
|
expect(result.subtechniques_upserted).toBe(475);
|
||||||
|
expect(result.subtechniques_skipped_orphan).toBe(0);
|
||||||
});
|
});
|
||||||
|
|
||||||
test('GET /mitre/tactics returns 14+ Enterprise tactics', async ({ request }) => {
|
test('GET /mitre/tactics returns 14+ Enterprise tactics', async ({ request }) => {
|
||||||
@@ -146,7 +152,7 @@ test.describe('M4 — MITRE ATT&CK reference', () => {
|
|||||||
test('Non-admin cannot trigger /mitre/sync', async ({ page, request }) => {
|
test('Non-admin cannot trigger /mitre/sync', async ({ page, request }) => {
|
||||||
// Invite a no-perm user via the admin.
|
// Invite a no-perm user via the admin.
|
||||||
const adminAccess = await loginAndGetAccess(request, ADMIN_EMAIL, ADMIN_PASSWORD);
|
const adminAccess = await loginAndGetAccess(request, ADMIN_EMAIL, ADMIN_PASSWORD);
|
||||||
const eveEmail = `eve-${Math.floor(Math.random() * 1e6)}@metamorph.local`;
|
const eveEmail = `eve-${crypto.randomUUID().slice(0, 8)}@metamorph.local`;
|
||||||
const inv = await request.post('/api/v1/invitations', {
|
const inv = await request.post('/api/v1/invitations', {
|
||||||
headers: { Authorization: `Bearer ${adminAccess}` },
|
headers: { Authorization: `Bearer ${adminAccess}` },
|
||||||
data: { email_hint: eveEmail },
|
data: { email_hint: eveEmail },
|
||||||
|
|||||||
@@ -51,6 +51,10 @@ export interface MitreTag {
|
|||||||
name: string;
|
name: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Query keys. `status` + `matrix` drive the M4 picker; the per-list factories
|
||||||
|
// (`tactics`/`techniques`/`subtechniques`) are unused today but the M5
|
||||||
|
// template forms will consume them for the standalone REST endpoints when
|
||||||
|
// users edit a single test's tags inline.
|
||||||
export const mitreKeys = {
|
export const mitreKeys = {
|
||||||
status: ['mitre', 'status'] as const,
|
status: ['mitre', 'status'] as const,
|
||||||
matrix: ['mitre', 'matrix'] as const,
|
matrix: ['mitre', 'matrix'] as const,
|
||||||
@@ -85,3 +89,17 @@ export interface MatrixTactic {
|
|||||||
export interface MitreMatrix {
|
export interface MitreMatrix {
|
||||||
tactics: MatrixTactic[];
|
tactics: MatrixTactic[];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Mirror of backend `SyncResultOut` (`api/mitre.py`). */
|
||||||
|
export interface MitreSyncResult {
|
||||||
|
tactics_upserted: number;
|
||||||
|
techniques_upserted: number;
|
||||||
|
subtechniques_upserted: number;
|
||||||
|
subtechniques_skipped_orphan: number;
|
||||||
|
technique_tactic_links: number;
|
||||||
|
version: string | null;
|
||||||
|
source: string;
|
||||||
|
started_at: string;
|
||||||
|
finished_at: string;
|
||||||
|
duration_ms: number;
|
||||||
|
}
|
||||||
|
|||||||
@@ -9,7 +9,12 @@ import { SectionHeader } from '@/components/ui/SectionHeader';
|
|||||||
import { Tag } from '@/components/ui/Tag';
|
import { Tag } from '@/components/ui/Tag';
|
||||||
import { ApiError, apiGet, apiPost } from '@/lib/api';
|
import { ApiError, apiGet, apiPost } from '@/lib/api';
|
||||||
import { useAuth } from '@/lib/auth';
|
import { useAuth } from '@/lib/auth';
|
||||||
import { mitreKeys, type MitreStatus, type MitreTag } from '@/lib/mitre';
|
import {
|
||||||
|
mitreKeys,
|
||||||
|
type MitreStatus,
|
||||||
|
type MitreSyncResult,
|
||||||
|
type MitreTag,
|
||||||
|
} from '@/lib/mitre';
|
||||||
|
|
||||||
export function MitrePage() {
|
export function MitrePage() {
|
||||||
const { state } = useAuth();
|
const { state } = useAuth();
|
||||||
@@ -24,14 +29,14 @@ export function MitrePage() {
|
|||||||
});
|
});
|
||||||
|
|
||||||
const sync = useMutation({
|
const sync = useMutation({
|
||||||
mutationFn: () => apiPost<Record<string, unknown>>('/mitre/sync'),
|
mutationFn: () => apiPost<MitreSyncResult>('/mitre/sync'),
|
||||||
onMutate: () => {
|
onMutate: () => {
|
||||||
setSyncResult(null);
|
setSyncResult(null);
|
||||||
setSyncError(null);
|
setSyncError(null);
|
||||||
},
|
},
|
||||||
onSuccess: async (res) => {
|
onSuccess: async (res) => {
|
||||||
const counts = `${res.tactics_upserted} tactics, ${res.techniques_upserted} techniques, ${res.subtechniques_upserted} subtechniques`;
|
const counts = `${res.tactics_upserted} tactics, ${res.techniques_upserted} techniques, ${res.subtechniques_upserted} subtechniques`;
|
||||||
setSyncResult(`Sync completed in ${(res as { duration_ms: number }).duration_ms / 1000}s — ${counts}.`);
|
setSyncResult(`Sync completed in ${(res.duration_ms / 1000).toFixed(1)}s — ${counts}.`);
|
||||||
await qc.invalidateQueries({ queryKey: ['mitre'] });
|
await qc.invalidateQueries({ queryKey: ['mitre'] });
|
||||||
},
|
},
|
||||||
onError: (e) => {
|
onError: (e) => {
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ make seed-mitre # télécharge le bundle pinné v19.0 (~50 MB, ~1 s parse)
|
|||||||
## 2. Tests automatisés
|
## 2. Tests automatisés
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
make test-api # 53 tests pytest dont 14 nouveaux MITRE (parser + 5 read endpoints + matrix + status)
|
make test-api # 58 tests pytest dont 19 MITRE (parser, idempotence, security guards, all endpoints, dotted fallback, version clearing)
|
||||||
make e2e # 34 tests Playwright dont 6 M4
|
make e2e # 34 tests Playwright dont 6 M4
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user