Loop briefs: require a full cargo test with the corpus before an item reaches state/needs-human #51

New Issue

fabi · 2026-09-17T19:02:11Z

fabi commented

2026-09-17 19:02:11 +00:00

Filed from the closing paragraph of sylph-pi's comment on #38, which asked for this as its own item.

What happened. 53f8345 was verified by running a named subset — its own message records "3 new disc tests; the 13 paint-order tests are green" — while the test its new rule could reach sat in a third file. That test went red and stayed red for 15 days on an agent branch. CI could not see it: CI has no disc, so the suite self-skips and still counts as passed (#16).

So the failure is not "an agent was careless"; it is that a brief lets a named subset stand as the verification, and a subset chosen by the author systematically excludes the test the author did not think of.

Proposed change to docs/agents/decoder-loop.md and docs/agents/port-loop.md: before an item may be moved to state/needs-human, the loop runs cargo test --workspace --no-fail-fast with the corpus present and reads target/sylpheed-corpus-report.txt — not the tally, which carries no information about whether the corpus was exercised — and the report goes in the hand-off.

Wording is the human's call; the brief is the artifact both loops read.

Filed from the closing paragraph of sylph-pi's comment on #38, which asked for this as its own item. **What happened.** `53f8345` was verified by running a named subset — its own message records *"3 new disc tests; the 13 paint-order tests are green"* — while the test its new rule could reach sat in a third file. That test went red and stayed red for **15 days** on an agent branch. CI could not see it: CI has no disc, so the suite self-skips and still counts as passed (#16). So the failure is not "an agent was careless"; it is that a brief lets a *named subset* stand as the verification, and a subset chosen by the author systematically excludes the test the author did not think of. **Proposed change** to `docs/agents/decoder-loop.md` and `docs/agents/port-loop.md`: before an item may be moved to `state/needs-human`, the loop runs `cargo test --workspace --no-fail-fast` with the corpus present and reads `target/sylpheed-corpus-report.txt` — not the tally, which carries no information about whether the corpus was exercised — and the report goes in the hand-off. Wording is the human's call; the brief is the artifact both loops read.

fabi added the state/proposed kind/defect labels 2026-09-17 19:02:11 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: fabi/Sylpheed#51

Loop briefs: require a full cargo test with the corpus before an item reaches state/needs-human #51

Loop briefs: require a full `cargo test` with the corpus before an item reaches `state/needs-human` #51