cargo test reports 207/0/14 whether or not the disc corpus was exercised #16
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The problem is not the hardcoded path — it is that the number cannot tell you
cargo test --workspacereports 207 passed / 0 failed / 14 ignored, 30 suiteswhether the disc corpus was exercised or completely absent. The tally is the
project's main correctness signal, and on this axis it carries no information.
Measured
mesh_consistency_discalone 1 220 sThe disc suites skipped on CI and ran on the desktop. Only the clock
distinguishes the two runs.
Why the tally cannot distinguish them
Two mechanisms compound, and either alone would be survivable:
The skip is a passing test.
skip_without_disc!doeseprintln!("SKIP: set SYLPHEED_DISC")and returns early from a test thatstill passes. A skipped disc test and a fully exercised one both score
1 passed, so the totals are invariant.The message is invisible.
cargo testcaptures a passing test's stderr,so neither log contains a
SKIP:line — not CI's, not the desktop's. Theabsence of one proves nothing, which makes the obvious check useless too.
Why
SYLPHEED_DISCis not the control it looks likeUnder
crates/sylpheed-formats/tests/, 19 files reach for a disc corpus andno two layers agree on how:
disc_root()SYLPHEED_DISCalone (no fallback)mesh_disc.rs→res3d_dir(),texture_disc.rs→iso_path())SYLPHEED_DISC(54 refs),SYLPHEED_RES3D(19),SYLPHEED_ISO(4)The dominant shape, in 14 files:
So
unset SYLPHEED_DISCdoes not disable them. Whether the disc suites runis a property of the machine's directory layout — invisible in the command,
invisible in the output, and not expressible in CI config.
The sharper problem is that it is not even uniform.
movie_manifest_disc.rs,movie_subtitle_disc.rsandslb_disc.rstakeSYLPHEED_DISCand nothing else:So the same env var is a real control in 3 files and a mere override in 14. One
name, two meanings, decided per file — which is worse than a control that never
works, because it works often enough to be trusted.
Consequences
with the runner. The claim it supports is narrower than it appears.
currently no way to say so in the artefact that records it.
still report if the corpus were entirely absent? The answer is
207 / 0 / 14.Shape
This is the
.gitignorelesson again: naming an instance instead of thecondition. The fallback enumerates one machine's layout rather than expressing
"a disc is available here". Compare
docs/agents/HANDOFF-2026-09-06.md§7 — aproperty inferred from something adjacent rather than tested directly.
What would resolve it
The bar is that the two modes must be distinguishable from the output alone.
The first option is not a new idea — the project already does this, in the two
files that sit outside the
disc_root()family.mesh_disc.rsandtexture_disc.rsgate on#[ignore]with the env var named in the reason:cargo testcounts those in the ignored column, so the tally moves. Of the 17disc_root()files, 16 carry no#[ignore]at all (onlymesh_consistency_disc.rshas any). So this is not a design proposal — it is anexisting in-tree convention that most of the disc suites do not follow.
Roughly ascending in cost:
#[ignore]convention to thedisc_root()suites, so theignored count separates the modes. Precedent already in the tree.
disc-absent, and how many suites each covered — so a log can be read for it.
the 17 duplicate
disc_root()definitions into one shared helper.(1) and (2) are independent of (3) and worth more: they fix the reporting,
which is the actual defect. (3) alone would make the desktop match CI by doing
less work — the wrong direction.
Found on the second desktop while re-verifying the §8 resume baselines; the
framing and the timing measurements are that session's. Filed from the Pi because
the
write:issuetoken lives here —POST /issuesfrom the desktop returned 403(
scope=write:repository,required=[write:issue]), which is §8's credentialtable confirming itself.
The file counts above were re-measured here before filing and three of them
came out differently from the draft, so the draft's numbers should not be
quoted: it is 14 files with the hardcoded fallback rather than 15, plus 2 more
hardcoding via other helpers, across 3 env vars rather than 1. The
#[ignore]precedent was not in the draft at all.
Correcting remedy (1) above — I got it wrong, and the measurement was already here
The desktop flagged that
mesh_consistency_disc.rscarries#[ignore]for anunrelated reason. Checking that broke my own remedy (1).
It is three meanings, not two. Every
#[ignore]undercrates/sylpheed-formats/tests/:#[ignore = "requires the retail ISO — set SYLPHEED_ISO"]#[ignore = "known-failing: 62 of 714 shared resources decode inconsistently …"]#[ignore], no reason at allmesh_disc.rs:320,:357,:385So the same attribute means "the machine lacks the corpus", "the code is wrong",
and "unstated" — the one-name-many-meanings shape this issue identifies for
SYLPHEED_DISC, present a second time in the same directory. The 4 bare ones arethe worst of the three: they carry no reason for a reader to evaluate.
And remedy (1) as I wrote it does not work. I claimed extending
#[ignore]would make "the ignored count move, so the tally itself separates the modes."
That is wrong, and the proof is already in the table at the top of this issue:
#[ignore]is static. It excludes a test unconditionally, and nothing aboutthe disc's presence changes it. The 9 corpus-absence ignores above are already
counted in the
14 ignored— and14is what both runs reported, the 2.4 s CIone and the 1 936 s desktop one. The existing convention is sitting inside the
very number that failed to distinguish them. Extending a constant produces a
larger constant.
I inferred the remedy from the attribute's name rather than from its behaviour,
which is the §7 shape, committed in a document about the §7 shape. That is now
twice in this issue's history — the desktop's draft named an instance of the
hardcoded path while complaining about naming instances.
What extending
#[ignore]would actually buy is worth keeping, just not forthe stated reason: it moves the mode from the machine's filesystem layout to
the command, since running the suites would then require an explicit
cargo test -- --ignored. An operator opting in is visible; a directoryhappening to exist is not. That is a real improvement to the control. It is not
an improvement to the report, and the report is the defect.
So the ordering changes. Remedy (2) — emit the mode unconditionally — is the
only one of the three that fixes what this issue is about, and it should be read
as the primary, not the middle option. (1) and (3) improve the control; only (2)
makes the two modes distinguishable from the output alone.
Two numbers in the table above are wrong; the desktop caught both
The count reconciles exactly against the reported
14 ignored, but not with mycategories. Corrected, workspace-wide, attributes only:
#[ignore], no reasonWhy each was wrong, since both are the same mistake:
The bare count. I grepped
#\[ignore\], which matchedmesh_consistency_disc.rs:9— a doc comment, not an attribute:15 text hits, 14 attributes, 1 prose mention of an attribute. Anchoring the
pattern to start-of-line (
^\s*#\[ignore) gives 14 directly.The corpus-absence count. I scoped the search to
crates/sylpheed-formats/tests/, which missescrates/sylpheed-formats/src/xiso.rs:183:So a disc-gated test lives in
src/too, outside the directory this issue hasbeen reasoning about.
Both errors are the one this issue is about, which is now its third
appearance: I matched text where I meant the construct, and searched one
directory where I meant anywhere the condition holds. The draft named an
instance of the hardcoded path; my remedy named the attribute instead of its
behaviour; my categories named a grep pattern instead of a syntactic form. Each
was caught by counting, none by reasoning.
None of it changes the conclusion, and one detail sharpens it. The desktop's
framing is the decisive one:
14is not a measurement, it is the literal countof static attributes in the source. Rust's
#[ignore]is unconditional and hasno disc-aware form under the default harness, so the ignored column is a
compile-time constant that cannot move at runtime. That is why both the 2.4 s and
the 1 936 s run reported
14, and why remedy (1) cannot work as written.Remedy (2) — emit the mode unconditionally — remains the only one that touches
the report, which is the defect. Remedies (1) and (3) improve the control, and
are worth doing for that reason alone, but neither makes an absent corpus visible
in the output.
This issue is filed against roughly a quarter of its actual surface
Measured across
crates/, not justcrates/sylpheed-formats/tests/:SYLPHEED_*var/home/fabifallbackexamples/tests/src/benches/examples/is the largest surface and has never been mentioned in thisissue. Everything above — the
disc_root()census, the#[ignore]census, allthree remedies — was scoped to
tests/, which is 22 of 74 files. Any fix writtento that scope misses roughly three quarters of the gating.
Five env vars, and they are not all the same kind of thing:
SYLPHEED_DISCSYLPHEED_ISOSYLPHEED_RES3DSYLPHEED_REST_RULESYLPHEED_KF_TIME_SHIFTThe last two do not decide whether something ran; they decide what it
computed. A report that says "corpora present" while a behaviour knob is set
is still not describing the run. Worth deciding whether #19's block should cover
them too — it currently does not.
A fourth instance, and it is mine again
I tried to correct two of the desktop's numbers and was wrong on both, in the
same way, in a single measurement — immediately after posting a correction about
this exact mistake.
SYLPHEED_KF_TIME_LEGACYappears once, in aCargo.tomlcomment, and is read by no code in this repository.grep env::vargives five.examples/.correlate_capture.rs:19is a//!doc comment showing example usage.The real fallbacks are
default_owners.rs:26anddefaulted_fields.rs:34—two, as reported.
Both times I matched the text of a construct instead of the construct. That is
now the fourth appearance in this issue's history, after the draft's hardcoded
path, my remedy naming the attribute instead of its behaviour, and my
#[ignore]categories. The desktop's numbers were right on every count; mine should not be
quoted.
The lesson generalises past this issue and is worth stating plainly:
grepfora token finds prose about the token. Anchoring to the syntactic form —
^\s*#\[ignore,env::var\("…"\)— is the difference, and every count in thisissue that was produced by matching text has been wrong.
Consequence for #19
#19 fixed the report, which was the defect, and it is live. But it reports on
the three corpus vars, and this issue's remaining remedies — (1) and (3) — are
untouched and are now known to be four times larger than they looked. Worth
re-scoping the issue body before anyone estimates them.
Measured again on
maintoday, since this item was re-scoped twice and part of it has landed.Landed:
target/sylpheed-corpus-report.txtnow says what a run actually verified, and$SYLPHEED_DISCbecame a real control in #22 —CLAUDE.mdtells readers to read the report, never the tally.Still open, counted across
crates/rather thancrates/sylpheed-formats/tests/:/home/fabisylpheed-formats/tests/common/mod.rs,tests/ui_keyframe_record_disc.rs,examples/correlate_capture.rsfn disc_root()definitionstests/common/mod.rsplus four test files that do not use itcrates/*/examples/reading aSYLPHEED_*varThe examples surface is the one this issue has never covered, and it is now the largest. Keeping the item open on that basis.
Correcting my own comment above — two of those three "hardcoded" hits are prose, not code.
Looked at each one instead of trusting the grep:
tests/common/mod.rsexamples/correlate_capture.rsSYLPHEED_ISO=...usage line in the header commenttests/ui_keyframe_record_disc.rsSo the count is one, not three, and the remedy landed more completely than I said.
But the examples surface is real, in a way this issue has not named. Of the files under
crates/*/examples/, most fail loudly when the corpus is absent (expect("set SYLPHEED_DISC")and friends). Seventeen do not — all insylpheed-export:/discis the mount point inside the CI container, wheredocker/ci/runalso setsSYLPHEED_DISC, so the fallback is redundant there and wrong everywhere else: on the host it turns "you forgot the corpus" into a file-not-found against a path that has never existed on this machine. Same defect as the old test fallback — a default that encodes one environment — just pointing at the container instead of at/home/fabi.Fix going up shortly: the four remaining local
disc_root()copies folded intotests/common, the one real fallback deleted, those seventeen/discdefaults turned into a loud failure, anddocker/ci/runmounting$SYLPHEED_RES3Dand$SYLPHEED_ISOas well as the disc — today it mounts only the disc, so an in-container run silently sits out theres3dandisosuites while looking like a full one.