test: report which corpora a run actually had (#16 remedy 2) #19
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: fabi/Sylpheed#19
Reference in New Issue
Block a user
Delete Branch "fix/corpus-report"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Implements #16's remedy (2) — emit the mode unconditionally, which the issue's own
discussion settled as the only remedy that touches the report, and the report is the
defect. Remedies (1) and (3) improve the control and are deliberately left open.
What was wrong
cargo test --workspacereports the same tally whether the disc corpus was exercised orentirely absent:
mesh_consistency_discalone 1 220 sTwo mechanisms compound, and either alone would be survivable:
eprintln!("SKIP: …")and return earlyfrom a test that still passes, so a skipped suite and a fully exercised one both score
1 passed. The totals are invariant.cargo testcaptures a passing test's output, soneither log contains a
SKIP:line. The absence of one proves nothing, which makesthe obvious check useless too.
And
14 ignoredcannot help:#[ignore]is static, so that column is the literal count ofattributes in the source and cannot move at runtime.
What this adds
crates/sylpheed-formats/tests/corpus_report.rs— always runs, never fails, resolves allthree corpora exactly as the per-suite helpers do, and records what was actually available.
It writes to a file rather than relying on stdout, because a passing test's stdout is
captured and would be invisible in exactly the CI log that needs it. The workflow then
prints that file, and the test also appends to
GITHUB_STEP_SUMMARYwhen set.Run on a machine that has the corpora, it immediately shows the thing the issue is about:
SYLPHEED_DISCis not controlling anything here. That was invisible before.The branch CI takes is the one that cannot be tested here
A machine with the corpora can never reach
ABSENT, so it would have shipped unrun.resolve_renders_every_branchexercises it directly — along with "set but does notresolve", which is what a typo in the env var produces, and which is reported as a
different state from absent because the two want different fixes.
Scope, and one thing deliberately not fixed
This does not remove the hardcoded fallbacks or de-duplicate the 17
disc_root()definitions — that is remedy (3), and doing it alone would make the desktop match CI by
doing less work, which is the wrong direction. It also does not narrow the gating surface,
which is wider than #16 currently states: 46 files under
examples/, 22 undertests/,6 under
src/, and five env vars, two of which (SYLPHEED_REST_RULE,SYLPHEED_KF_TIME_SHIFT) are behaviour knobs rather than corpus locators. Worth foldinginto #16 before anyone implements (3).
Verification
cargo fmt --all -- --check— clean (this branch adds no fmt debt to a tree that justreached zero)
cargo clippy --workspacedoes not build test cfg. It fails identically on unmodified
mainon this desktopwith
only_used_in_recursionatvfs.rs:85— the rustc 1.90.0 vs runner 1.98.1divergence, i.e. #15, not this PR.
The second commit corrects the first:
4057bee's message quoted output the code did notproduce, because a cleanup edit silently no-op'd before it was committed. Not squashed —
force-push is denied on this remote, and a visible correction beats rewritten history.
Refs #16
🤖 Generated with Claude Code
`cargo test --workspace` reports the same tally whether the disc corpus was exercised or entirely absent. Measured: the disc suites RAN on a developer desktop (1936 s, mesh_consistency_disc alone 1220 s) and SKIPPED on CI (2.4 s total) -- and both reported 207 passed / 0 failed / 14 ignored across 30 suites. Two mechanisms compound: * a skip is a PASSING test. The gated suites `eprintln!("SKIP: ...")` and return early from a test that still passes, so a skipped suite and a fully exercised one both score 1 passed. The totals are invariant. * the message is invisible. `cargo test` captures a passing test's output, so NEITHER log contains a `SKIP:` line. The absence of one proves nothing, which makes the obvious check useless too. And `14 ignored` cannot help: `#[ignore]` is static, so that column is the literal count of attributes in the source and cannot move at runtime. Ask what this check would still report if the corpus were entirely absent, and the answer is 207/0/14. This is #16's remedy (2) -- the only one that touches the REPORT, which is the defect. Remedies (1) and (3) improve the control and are left open. Adds `tests/corpus_report.rs`: always runs, never fails, resolves all three corpora exactly as the per-suite helpers do, and records what was available. It writes to a FILE rather than relying on stdout, because a passing test's stdout is captured and would be invisible in exactly the CI log that needs it; the workflow then prints that file. It also appends to GITHUB_STEP_SUMMARY when set. Run here, it immediately shows the thing the issue is about -- all three corpora resolve through the HARDCODED fallback, so SYLPHEED_DISC is not controlling anything on this machine: SYLPHEED_DISC PRESENT via the HARDCODED fallback, NOT $SYLPHEED_DISC SYLPHEED_RES3D PRESENT via the HARDCODED fallback, NOT $SYLPHEED_RES3D SYLPHEED_ISO PRESENT via the HARDCODED fallback, NOT $SYLPHEED_ISO The ABSENT branch is the one CI takes and cannot be reached on a machine that has the corpora, so `resolve_renders_every_branch` exercises it directly rather than shipping it unrun -- along with "set but does not resolve", which is what a typo in the env var produces and which is deliberately reported as a DIFFERENT state from absent, since the two want different fixes. Verified: `cargo fmt --all -- --check` clean. Clippy is unchanged by this (a test target; CI's `cargo clippy --workspace` does not build test cfg) -- it fails identically on unmodified main here with `only_used_in_recursion` at vfs.rs:85, which is the rustc 1.90.0 vs the runner's 1.98.1 divergence, i.e. #15, not this. Refs #16 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>`4057bee` claimed the report prints SYLPHEED_DISC PRESENT via the HARDCODED fallback, NOT $SYLPHEED_DISC and it did not. The string was built by `"...(not ${env})".replace("${env}", env)` -- a placeholder substituted at runtime, which printed `(not SYLPHEED_DISC)` and dropped the `$`. I had cleaned this up before committing, but the edit silently no-op'd (rustfmt had already rewrapped the line, so the pattern no longer matched) and the commit went ahead with the message quoting output the code does not produce. Now a plain `format!`, which is what it should have been: shorter, no placeholder to keep in sync, and the `$` makes it read as the env var it is. The quoted output in `4057bee` is accurate as of this commit. Not squashed into it because force-push is denied on this remote, and a visible correction is worth more than a rewritten history anyway. Verified: `cargo fmt --all -- --check` clean, both tests pass, output matches the text above exactly. Refs #16 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>