Fabian Hamm 4057bee897 test: report which corpora a run actually had (#16) ...

`cargo test --workspace` reports the same tally whether the disc corpus was
exercised or entirely absent. Measured: the disc suites RAN on a developer
desktop (1936 s, mesh_consistency_disc alone 1220 s) and SKIPPED on CI
(2.4 s total) -- and both reported 207 passed / 0 failed / 14 ignored across
30 suites.

Two mechanisms compound:

  * a skip is a PASSING test. The gated suites `eprintln!("SKIP: ...")` and
    return early from a test that still passes, so a skipped suite and a
    fully exercised one both score 1 passed. The totals are invariant.
  * the message is invisible. `cargo test` captures a passing test's output,
    so NEITHER log contains a `SKIP:` line. The absence of one proves
    nothing, which makes the obvious check useless too.

And `14 ignored` cannot help: `#[ignore]` is static, so that column is the
literal count of attributes in the source and cannot move at runtime. Ask
what this check would still report if the corpus were entirely absent, and
the answer is 207/0/14.

This is #16's remedy (2) -- the only one that touches the REPORT, which is
the defect. Remedies (1) and (3) improve the control and are left open.

Adds `tests/corpus_report.rs`: always runs, never fails, resolves all three
corpora exactly as the per-suite helpers do, and records what was available.
It writes to a FILE rather than relying on stdout, because a passing test's
stdout is captured and would be invisible in exactly the CI log that needs
it; the workflow then prints that file. It also appends to
GITHUB_STEP_SUMMARY when set.

Run here, it immediately shows the thing the issue is about -- all three
corpora resolve through the HARDCODED fallback, so SYLPHEED_DISC is not
controlling anything on this machine:

  SYLPHEED_DISC   PRESENT via the HARDCODED fallback, NOT $SYLPHEED_DISC
  SYLPHEED_RES3D  PRESENT via the HARDCODED fallback, NOT $SYLPHEED_RES3D
  SYLPHEED_ISO    PRESENT via the HARDCODED fallback, NOT $SYLPHEED_ISO

The ABSENT branch is the one CI takes and cannot be reached on a machine
that has the corpora, so `resolve_renders_every_branch` exercises it
directly rather than shipping it unrun -- along with "set but does not
resolve", which is what a typo in the env var produces and which is
deliberately reported as a DIFFERENT state from absent, since the two want
different fixes.

Verified: `cargo fmt --all -- --check` clean. Clippy is unchanged by this
(a test target; CI's `cargo clippy --workspace` does not build test cfg) --
it fails identically on unmodified main here with
`only_used_in_recursion` at vfs.rs:85, which is the rustc 1.90.0 vs the
runner's 1.98.1 divergence, i.e. #15, not this.

Refs #16

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-09-09 07:39:49 +02:00

7.3 KiB

Raw Blame History

View Raw