markharness

0016: Introducing a precondition/step/result model for Behavior, Condition, and ExpectedResult (supersedes 0015)

Status

Accepted (2026-09-01). 0015 has been changed to Superseded.

Context

0015’s Phase 1 added a required, ordered steps: Vec<String> to behavior.yml, shared by every Condition under that Behavior (commit 5480b85).

After shipping Phase 1 and actually building test cases against it, the following problems surfaced:

  1. Operations differ per condition. For example, an “add TODO” Behavior has one condition that enters blank text and another that enters valid text — the text being entered is itself different. A single behavior.steps cannot express this; Test Designers were forced to either bias steps toward one condition or push the per-condition difference into description’s free text.
  2. Some conditions have preconditions that no sequence of shared steps can reach. For example, a condition premised on “the target TODO has already been deleted” cannot be reproduced by the Behavior’s shared steps alone.
  3. expected_result.description (predating 0015) only supports a single sentence, so it cannot hold multiple distinct observable outcomes (e.g., “the item is added to the list,” “the input field resets,” “focus returns to the input field”) in one ExpectedResult, and it ends up mixing an implementation-detail rationale with the user-facing outcome in the same sentence.
  4. There is no way to express a verification that requires an extra action first (e.g., reloading the page before checking persistence). generate.rs::load_knowledge_snapshot currently flattens every expected/*.yml under one Condition into a single Vec<ExpectedSnapshot>, and TestCase holds only one flat steps: Vec<String> and one flat expected: Vec<String> (src/generate.rs). This makes it impossible to distinguish “a result only visible after a reload” from other results that need no reload.

These are distinct from the “duplicated procedure / copy-update drift” problem that 0015 reserved for Phase 2. Phase 2 was about the need for a shared Step registry; what actually surfaced here is that the sharing granularity at the Behavior level is too coarse for the real variation across conditions — the very premise Phase 1 locked in (“Behavior.steps is shared by every Condition”) broke against real data.

To work through this, a scratch sample (examples/bdd-sample/, deleted once this ADR is finalized) was built that deliberately ignored .markharness’s actual schema and mapped the Gherkin (Given/When/Then/Background) way of thinking directly onto the directory structure, to figure out what was actually needed, no more and no less. This ADR reflects the conclusions of that exploration.

Decision

1. Schema changes

Rename behavior.schema.json’s steps to preconditions, changing its meaning to “preconditions common to every Condition.” The actual operation sequence moves entirely to Condition.

# behavior.yml
id: add-todo
feature: todo-management
label: Add TODO
axis: [ui]
description: |
  Adds a TODO from the entered text on form submit
preconditions:
  - Open the TODO app
  - Confirm the input field is empty
# condition.yml (valid-text)
id: valid-text
behavior: add-todo
label: Valid text
description: |
  When non-empty, valid text is entered into the input field and submitted
additional_preconditions: []
steps:
  - Enter "Buy milk" into the input field
  - Click the "Add" button
# expected/001.yml (under the same valid-text directory)
id: valid-text-001
condition: valid-text
generated_by: manual
description: The valid text is added as a TODO and the input field resets
results:
  - The "Buy milk" TODO appears, unchecked, at the end of the TODO list
  - The input field resets to empty
  - Focus returns to the input field
implementation_note: |
  addTodo() pushes {id, text, completed:false} onto todos using the trimmed text,
  then calls render(). The submit handler sets input.value = "" and calls input.focus()
# expected/002.yml (example with an extra action in between)
id: valid-text-002
condition: valid-text
generated_by: manual
description: The TODO persists across a reload
additional_steps:
  - Reload the page
results:
  - The "Buy milk" TODO is still shown in the list
implementation_note: |
  addTodo() calls saveTodos(), which persists to localStorage,
  so loadTodos() restores it on reload

Field list:

entity field type required meaning
behavior preconditions (renamed from steps) Vec<String> empty array allowed (no minItems) preconditions common to every Condition
condition steps (new) Vec<String> minItems: 1 required condition-specific operation sequence (inherits the granularity rule of the old behavior.steps)
condition additional_preconditions (new) Vec<String> empty array allowed condition-specific extra preconditions (ones no sequence of steps alone can reach)
expected_result description (existing, unchanged) String required human-facing one-sentence summary; not consumed by generation
expected_result results (new) Vec<String> minItems: 1 required multiple observable outcomes; consumed by test-case generation
expected_result additional_steps (new) Vec<String> optional only for the expected_result that is first in filename order within a Condition; non-empty (at least one action) required for every subsequent one extra action(s) needed before this result can be checked
expected_result implementation_note (new) String optional implementation-detail rationale; not consumed by generation

File-splitting convention for expected/*.yml: multiple independent observations checked after the same action are written as multiple lines within one expected_result.results array, not split across files. A new file (e.g. 002.yml) is created only to express a new phase — one that requires an intervening action (additional_steps) before its results are checked. Rather than leaving this convention to review discipline alone, it is enforced mechanically: within a Condition, every expected_result after the first (in filename order) must have non-empty additional_steps (only the first expected_result may omit it, or leave it empty).

This constraint cannot be expressed by expected_result.schema.json’s JSON Schema alone — validate.rs’s validate_file validates each expected/*.yml file independently, one at a time, so JSON Schema has no way to know a given file’s ordinal position among its siblings within the same Condition directory. This constraint is therefore implemented as a validate.rs-side cross-reference check (the same category as the existing axis-tag and forked_from-target reference-integrity checks): it lists a Condition’s expected/*.yml files in filename order and errors if any file beyond the first has empty additional_steps. This turns “creating a new file with no intervening action” into a Knowledge-validation error, structurally eliminating the ambiguity a 2026-09-01 Standards/Spec review raised — whether 002 represents an independent observation, a re-run of the action, or an additional action layered on top of the retained state.

2. TestCase structure changes (generate.rs)

TestCase granularity stays “1 Condition = 1 TestCase,” as before. Its internal structure changes as follows:

The existing flat title / steps / expected fields are removed and replaced by preconditions / phases.

3. Naming policy

Gherkin terminology (Given/When/Then/Background) is not adopted; naming follows the existing schema’s convention (non-BDD terms like id/label/description/axis). preconditions / steps / additional_preconditions / additional_steps / results / implementation_note are all non-Gherkin-specific vocabulary.

4. Granularity convention

The “one element = one fact” rule 0015 set for behavior.steps carries over to every new array field, retargeted per field. Each field’s unit of “one fact” differs by the field’s nature:

As in 0015, Knowledge validation does not mechanically enforce this rule; it is left to Test Designer review discipline.

5. Relationship to the execution model (not an automated execution engine)

As execution_result.schema.json defines, markharness has no automated execution engine: a human Test Executor reads the procedure and performs it manually, recording exactly one pass/fail/skip for the whole TestCase (unchanged from before 0016). Accordingly, the phases this ADR introduces is a single, sequential procedure a human reads top to bottom, not a state machine that assumes automated execution (per-phase pass/fail recording, branching or retrying on failure). State between phases (e.g., checking persistence after a reload) carries over naturally because the same human performs the whole sequence in the same environment continuously; no explicit state-management mechanism is needed. This ADR does not introduce a teardown/cleanup concept — if a concrete need is confirmed against real data, it will be considered separately under the same criterion as 0015’s Phase 2 onward (no quantitative threshold, revisit once actual friction is observed).

“Not an automated state machine” does not mean phase boundaries carry no meaning. §1’s requirement that every expected_result after the first carry non-empty additional_steps is not there to make an automated state transition precise — it exists so a human reading the procedure can read it unambiguously. Precisely because execution is not automated, an ambiguous procedure document translates directly into a human misreading or misperforming it (an automated system would surface an ambiguous spec as a program branch; a manual procedure lets different readers silently settle on different interpretations, unnoticed). So the additional_steps requirement exists to guarantee unambiguity at authoring time (clarity as a procedure document), not to pin down an execution-time state machine — it does not contradict this section’s premise that there is no automated execution engine.

Conditions for moving to Accepted

Out of scope

Implementation notes (not decided by this ADR)