m5c smoke tests

m5c smoke runs runtime smoke scenarios against lowered package bundles. Use it after validation, tests, build, and lower to prove that generated resources can run against a Mach5 runtime.

Smoke tests complement package tests:

• m5c test checks source-level contracts, mappings, fixtures, and expected findings.
• m5c smoke checks a runtime scenario using lowered bundles.

Where smoke files live

Smoke scenarios are YAML files usually kept under package tests:

packages/security-analytics/
  tests/
    smoke/
      source-to-detection.yaml
      finding-to-case.yaml

Smoke files are runtime scenarios, not source-level unit tests. They should run after package lowering.

Build a smoke test

Build a smoke test in this order:

1. Start with one high-value runtime path.
2. Insert deterministic seed documents.
3. Refresh materialized views that depend on those documents.
4. Query app models to prove rows are visible.
5. Execute app actions or workflows.
6. Wait for async workflows when needed.
7. Query final state.
8. Keep the scenario small enough to debug quickly.

Smoke test document

kind: SmokeTest
name: finding-to-case
spec:
  steps:
    - type: insert_docs
      index: security-manual-detection-results
      documents:
        - finding_id: finding:SMOKE-001:evt-001
          dedupe_key: SMOKE-001:evt-001
          detection_id: SMOKE-001
          detection_title: Smoke test suspicious login
          severity: high
          confidence: 0.9
          first_seen: "2026-05-22T08:00:00Z"
          last_seen: "2026-05-22T08:00:00Z"
          status: open
          entity_kind: user
          entity_id: smoke-user@example.com
    - type: refresh_mv
      name: security_findings_mv
    - type: refresh_mv
      name: security_alerts_mv
    - type: query_model
      app: security-soc-app-poc
      model: alert
      filters:
        - field: alert_id
          op: eq
          value: finding:SMOKE-001:evt-001
      expect_rows_min: 1
      save_first_as: smoke_alert
    - type: execute_action
      app: security-soc-app-poc
      action: create_case_from_alert
      record_ref: smoke_alert
      confirmed: true
      expect_status: started
      save_workflow_run_as: case_workflow
    - type: wait_workflow
      run_ref: case_workflow
      expect_status: Completed
      timeout_secs: 30
    - type: query_model
      app: security-soc-app-poc
      model: case
      filters:
        - field: case_id
          op: eq
          value: case:finding:SMOKE-001:evt-001
      expect_rows_min: 1

Top-level fields

Steps run sequentially. A failing step fails the scenario.

Step: insert documents

Insert one or more documents into an index.

- type: insert_docs
  index: security-manual-detection-results
  documents:
    - finding_id: finding:SMOKE-001:evt-001
      status: open

Use deterministic IDs and timestamps so the scenario can be re-run.

Step: refresh materialized view

Refresh a materialized view.

- type: refresh_mv
  name: security_alerts_mv
  mode: full

Use refresh steps after inserts when later queries depend on materialized view output.

Step: query model

Query an application model and optionally save the first row for later steps.

- type: query_model
  app: security-soc-app-poc
  model: alert
  filters:
    - field: alert_id
      op: eq
      value: finding:SMOKE-001:evt-001
  expect_rows_min: 1
  save_first_as: smoke_alert

Use save_first_as when a later action should operate on the row returned by the query.

Step: execute action

Execute an app action.

- type: execute_action
  app: security-soc-app-poc
  action: create_case_from_alert
  record_ref: smoke_alert
  input:
    case_id: case:finding:SMOKE-001:evt-001
  confirmed: true
  expect_status: started
  save_workflow_run_as: case_workflow

Use either record or record_ref when the action needs a selected record.

Step: wait for workflow

Wait for an async workflow run to reach a status.

- type: wait_workflow
  run_ref: case_workflow
  expect_status: Completed
  timeout_secs: 30

Use this after action steps that start workflows.

Typical flow

m5c validate apps/security-analytics --workspace --offline
m5c test apps/security-analytics --workspace
m5c build apps/security-analytics --out dist
m5c lower apps/security-analytics --dev --out lowered
m5c smoke apps/security-analytics --artifacts-dir /tmp/m5c-smoke

Smoke output includes scenario evidence and a report path that can be attached to release notes, CI runs, or app readiness reviews.

Runtime modes

Smoke scenarios can run against a managed temporary Mach5 One runtime or an externally supplied runtime, depending on the available command flags and environment.

Use managed smoke for repeatable local checks. Use external runtime smoke when validating against a shared development or staging environment.

Scenario design

Good smoke scenarios are narrow and deterministic.

Examples:

Common mistakes

Best practices

• Keep smoke data deterministic.
• Run smoke after lowering, not before.
• Test one runtime path per scenario.
• Save and reuse queried records instead of duplicating payloads.
• Store reports as CI artifacts.
• Do not use smoke tests as the only validation gate; keep source-level tests fast and comprehensive.