markharness

0020: Lighten execution status and drop evidence management from markharness’s scope

Status

Accepted (implemented 2026-09-12; see checklist-v2-core.md). Based on the redesign grilling session (2026-09-11) for markharness-v2-design.md. Supersedes, within its own scope (execution records attached to a TestCase), the Evidence Applicability model in 0017 §5 (“Execution evidence and applicability”). §1–4 (Scenarios, shared procedures, Case revision, split/merge) are out of scope for this ADR and remain in effect. 0025 renames the type to ExecutionBinding; this ADR’s lightweight scope and retained information remain in force.

Context

0017 envisioned separating execution result, applicability, and presence of evidence, adopting evidence toward a pass only when Case UID, Case revision, target build, and required environment all match exactly. The later, now-superseded draft of markharness-v2-design.md built on this and expanded it further into a heavyweight set of types: Evidence, EvidenceSelection, an Execution Manifest, Implementation revision, and an Environment matrix.

Re-deriving the North Star (what answer someone wants in hand once they’re done using this tool) during the grilling session showed the actual required level is much lower:

This is clearly smaller than the build/environment-level matching 0017 §5 called for, and does not justify the implementation cost of the Evidence/EvidenceSelection/Manifest/Implementation revision/Environment type family (YAGNI).

Decision

1. Execution status is one mode field plus an optional reference string

The execution record attached to a TestCase carries exactly two fields:

ExecutionStatus {
  mode: automated | manual,
  reference: string (optional, free text — e.g. a path to the test code or a URL)
}

No detailed result (pass/fail/skip), execution timestamp, executor, target build, environment, or evidence artifact is stored. Where any of that is needed, reference points to where it lives (a separate tool).

The TestCase is referenced by its Case UID, not by display id (0013), so that renaming a display id does not orphan the record.

A note on naming: this record states the verification method (automated or manual) and where to look, not the fact that an execution happened. Because it carries no timestamp, run count, or result, the presence of a value must not be read as “executed against the latest Case revision.” The name “Execution status” does not imply that limit, so even if the field is renamed at implementation time, the meaning defined in this section governs.

2. “Was it run” does not require build/environment matching

The strict applicability check 0017 §5 required — Case UID, Case revision, target build, and required environment all matching — is not introduced. Change Impact / Release Coverage computations consult only whether a TestCase has an ExecutionStatus and its mode. If build- or environment-precise pass/fail tracking is genuinely needed later, that concrete requirement drives a separate ADR at that time.

3. Storing and managing evidence is out of markharness’s scope

The concepts and types for immutable evidence storage, EvidenceSelection, and an Execution Manifest are not introduced into markharness. CI-driven automatic evidence ingestion is likewise deferred until an actual request arises; the MVP records ExecutionStatus through the CLI, by a human.

4. Shape the data with a future Playwright connection in mind

Direct integration with an automated test tool (CI wiring, reporter ingestion) is out of scope for the MVP, but the shape — mode: automated with reference holding a test-file path — is chosen so it can later map cleanly onto Playwright’s inventory/annotations. No tool-specific fields (project, locator, etc.) are added at this time.

Impact

Alternatives considered and rejected