0d7199bc22
extracted", and the tool was in the build all along Two iterations ago I closed Q8 by declaring the SE audio undecodable: Static.slb has no wave boundaries, there is no XACT container anywhere, and I said the index "exists only at runtime" as though that put it out of reach. The disc half of that stands. The conclusion did not. This build of Canary carries a cvar called xma_param_probe, added by this project, whose own comment says it logs each XMA stream's parameters and head bytes so raw sound.pak entries can be matched to real decode params. It has been sitting in the startup CONFIG DUMP of every log I have read this session. Run with it, driving the main menu: a d-pad move spawns a new mono 48 kHz stream of 4 packets / 8192 bytes, and B spawns a different one of 2 packets / 4096 bytes. Searching their logged head bytes in Static.slb finds each at exactly one offset -- 0x1ec0 and 0x0ec0 -- and the two are contiguous, 0x0ec0 + 4096 = 0x1ec0. So the bank is a packed run of whole 2048-byte packets with no delimiters, which is precisely why the seek scan found nothing: there is nothing to find. A wave is (offset, packet count) and nothing else. That splits Q8's binding cleanly. Event -> WAVE is now measured: the port can have the audio. Event -> cue NAME is still a name match on the authors' identifiers, and the page says so. The same run settled something for Q10 too. Sitting on the main menu, TWO stereo 48 kHz streams were decoding simultaneously. bgm-two-stems.md said that observation was what it needed and that this container could not make it; it can, and a music bank's two waves are now measured as simultaneous rather than only inferred. METHOD gets the general lesson, because it cost two iterations: check what instrumentation the local build already has before declaring a question blocked on tooling.
212 lines
13 KiB
Markdown
212 lines
13 KiB
Markdown
# Method traps already paid for
|
||
|
||
Each line cost an iteration at least once. They are general — they are not about
|
||
Sylpheed, they are about how this kind of measurement goes wrong.
|
||
|
||
Like [`REFUTED.md`](REFUTED.md), this list had been living in the autonomous
|
||
agent's loop prompt, i.e. nowhere durable. See [`README.md`](README.md) for the
|
||
✅/🟡/❔ confidence convention itself.
|
||
|
||
## Controls
|
||
|
||
* **Every result needs a control. A control that fails kills the instrument.**
|
||
* **Run the known-positive through a new filter FIRST.** Three filters have been
|
||
killed by their own control. When one fails, **read the known-good's
|
||
disassembly** before assuming a shape.
|
||
* **A measured negative is a result** — but a negative is only as strong as the
|
||
route you ran, so **state its reach**.
|
||
* **A null result needs its cause shown to have happened.**
|
||
* **A result with NO unknowns is suspicious.**
|
||
* **Census the whole set; always run the other population as the control.**
|
||
**Zero partials is stronger than a majority.**
|
||
* **A 2×2 partition is the sharpest general tool** — both off-diagonals empty is
|
||
a law.
|
||
* **Re-derive a doc's own numbers as the control.**
|
||
|
||
## Inference
|
||
|
||
* **Never conclude from ONE sample.**
|
||
* **A law proved on one population is a hypothesis on the next.**
|
||
* **Finding one exception does not imply a family.**
|
||
* **Consistency is not proof. A suggestive coincidence is a coincidence until
|
||
measured.** **An analogy is not a measurement.**
|
||
* **Same layout ≠ same instance.** **Same record-name set ≠ same object.**
|
||
* **A marker is only proven by what it leaves out.**
|
||
* **A high-confidence SCORE is not a high-confidence MECHANISM.**
|
||
* **Knowing HOW MANY is not knowing WHICH.**
|
||
* **Round numbers matching is weak evidence — unless you read the constant.**
|
||
* **My own last-turn result is a hypothesis too.**
|
||
* **A global partition can understate a per-owner one.**
|
||
* **A residual is measured against a population — name it.**
|
||
|
||
## Searching and tooling
|
||
|
||
* **A search that returns thousands has no power; state the reach.**
|
||
* **A substring match is not a hit.** **A regex miss looks like a null result —
|
||
print one raw sample before believing a zero.**
|
||
* **A derived table can be a cross product — measure its shape first.**
|
||
* **After refuting an instrument, sweep everything that depended on it.**
|
||
* **Before measuring how wrong a tool is, read what the tool actually does.**
|
||
* **The instrument must pass its own control.**
|
||
* **Classify a bulk before mining it. The residual is the prize.**
|
||
* **Rank by similarity — the cliff is the finding.** But **read the values
|
||
before trusting the rank.**
|
||
* **Grep the nouns before designing the experiment — and believe it.**
|
||
* **Grep gives you a file list — read *every* file on it.**
|
||
* **The answer is often already in the doc that owns the subject — read it end
|
||
to end.** A 🟡 often names its own route.
|
||
* **Before re-trying a blocked idea, check whether the blocker's own doc already
|
||
tried it.**
|
||
* **Ship a regenerator with every artefact.** An artefact that moves by a pure
|
||
reorder is a tool bug.
|
||
* **Never print per-entry lines from a disc-wide sweep — aggregate.**
|
||
|
||
## Reading the data
|
||
|
||
* **Read what a loader NAMES, not where it stores.**
|
||
* **A field the disc never values still gets named by the loader.**
|
||
* **An indexed read beats a deduped-pool adjacency read.**
|
||
* **Check the whole string set, not the one matching word.**
|
||
* **A dict keyed by record name across a multi-entry pak is a lie.**
|
||
* **A set-difference over names hides reuse — join per USER.**
|
||
* **A self-index names records, not files.**
|
||
* **Case-insensitive hashing means two spellings can be one entry.**
|
||
* **An "unresolved" name may be the wrong kind, namespace or prefix — or part of
|
||
a cut asset.**
|
||
* **A garbled value may be a real string in another encoding.**
|
||
* **Two of my own counts disagreeing is a grammar clue.**
|
||
* **A game's own typo is a join key.**
|
||
* **A bias constant in the code is a join key.**
|
||
* **A prefix trap: enumerate maximal `[A-Za-z0-9_]` runs, not `startswith`.**
|
||
* **Re-deriving a format is not a finding — asking whether its values *resolve*
|
||
is.**
|
||
|
||
## Mechanics that have bitten
|
||
|
||
* **Never hand-convert a decimal VA — print `hex()`.**
|
||
* **`grep -c` counts LINES** — use `grep -o | wc -l`.
|
||
* **`Counter.most_common()` tie-breaks by insertion order — use `sorted()`.**
|
||
* **Raw grep cannot see inside compressed pak entries.**
|
||
* **Commit messages go in a file** (`git commit -F`); a literal `|` in a table
|
||
cell needs escaping; `git log --all -- <path>` can hang.
|
||
|
||
## Runtime / emulator
|
||
|
||
* **Look at the PNG** — and check its dimensions.
|
||
* **"Animating" is not "still in a mission".**
|
||
* **Dedup entity enumerations by position value.**
|
||
* **Do not diagnose timing or liveness under gdb.** `ps %cpu` is cumulative.
|
||
* **Classify screens by whole-image statistics, not named pixels** — a named
|
||
pixel is only valid while the image sits at a known place, and nothing errors
|
||
when it moves.
|
||
* **Do not poll faster than the guest updates** — it manufactures a clean curve
|
||
out of noise.
|
||
* **A probe that never performs the action will "prove" the action does not
|
||
exist.**
|
||
* **The container's Canary binary can be older than the Canary source tree, and
|
||
the failure mode is a hang, not an error.** After a merge into `sylpheed-re`
|
||
the prebuilt `xenia_canary` had no `log_ui_draws`, no `mem_watch`, no
|
||
`create_profile_if_none` — and an unknown cvar makes xenia open an SDL message
|
||
box before logging is up, which headless is an unexplained freeze. Check before
|
||
trusting a harness flag: `nm -C <binary> | grep cvars::<flag>`, and
|
||
`build-canary Release` if it is missing.
|
||
* **A capture armed *at* a screen only ever sees the steady state.** Anything
|
||
about how a screen is built or animated has to be armed *before* it exists.
|
||
Re-arming every few seconds and keeping every log tiles the approach: each F10
|
||
opens a new numbered file and closes the previous one complete.
|
||
* **Measure animation in submitted frames, not in seconds.** `VdSwap` counts are
|
||
the guest's own frames, so an emulator at 80 % of real time does not move them;
|
||
a stopwatch reading does, silently and by an unknown factor.
|
||
* **~~This game's menus drop d-pad presses shorter than ~0.3 s.~~ WITHDRAWN
|
||
2026-08-28 — the menu WRAPS, and I had not measured that.** The claim came from
|
||
reading a cursor that ended up "one item short"; once wrap-around at both ends
|
||
was measured ([`menu-navigation-semantics.md`](menu-navigation-semantics.md)),
|
||
every one of those press counts is exactly right — four presses at 0.12 s moved
|
||
four steps *through the bottom*, which lands one above where a non-wrapping
|
||
menu would put it. **No press was ever dropped.** The real lesson is the
|
||
general one: *a step count is only readable once you know the topology*, and I
|
||
invented a hardware-flakiness story rather than testing the ends of the list.
|
||
Still true and worth keeping: screenshot after every step and read the cursor,
|
||
rather than trusting arithmetic over the press count.
|
||
* **A screenshot taken right after a transition can catch a screen mid-fade.**
|
||
A grab 2.5 s after Ⓑ returned the game to the title showed the title art with
|
||
no `PRESS Ⓐ BUTTON` plate; one second later the plate was there. That very
|
||
nearly went into the corpus as "the returned title has no plate". Sample a
|
||
changing screen several times before writing down what it does *not* contain.
|
||
* **Do not identify a menu cursor by label brightness.** The obvious oracle —
|
||
"the focused label is the brightest row" — fails on this game's menus, because
|
||
the background art is brighter behind some rows than the highlight is. It
|
||
confidently named the wrong item on a frame whose ring was plainly elsewhere.
|
||
Detect the **focus ring** in the gutter left of the labels instead
|
||
(`tools/re-capture/menu_focus.py`, 254 vs <82 — no threshold tuning needed),
|
||
and look at the PNG before believing either.
|
||
* **`screenshot` samples at 0.5 Hz — it cannot time an animation.** Measured:
|
||
~2 s per grab (an `import` of the root plus an ImageMagick crop). A 0.4 s fade
|
||
falls entirely between two samples, which is why a 40-frame burst across a
|
||
screen change looked like an instant cut. For anything timed, record the
|
||
display instead: `ffmpeg -f x11grab -framerate 30 -video_size <W>x<H> -i
|
||
:98+<X>,<Y> -t <secs>`, then read per-frame statistics off the file. Take the
|
||
geometry from `xwininfo -root -tree`, the same way `bin/screenshot` does.
|
||
* **A screen's brightness curve is not its fade quad.** The incoming screen's own
|
||
elements animate in *after* the transition quad has cleared, so mean luminance
|
||
keeps rising long after the fade is over — 1.47 s against a declared 0.97 s on
|
||
one screen. Time the fade from where the frame is *pure black*, and take the
|
||
ramp itself from the keyframes.
|
||
* **A `sound.pak` entry is not a bank.** Entry windows straddle bank boundaries
|
||
(the leading-region effect), so a naive "walk the RIFFs in this entry" gives
|
||
the tail of the previous bank, then one of your own, then the *next* bank's
|
||
header. It looks like a bank with the wrong number of waves. The tell is a
|
||
`BANK id=` that is not the entry's own number, and the fix is the `seek` packet
|
||
counts — they pin each wave's join exactly, so a straddle can be realigned
|
||
rather than thrown away. 4 of 32 BGM banks look like counterexamples until this
|
||
is applied.
|
||
* **Do not time a video whose start you did not see.** An attract cycle sampled
|
||
from partway through measured "≈85 s", which was used to argue it was not the
|
||
137 s `ADV.wmv`. It was `ADV.wmv`; the sampling began 39 s in. Identify a video
|
||
by **content** — match frames against the candidates and check the matched
|
||
playhead advances monotonically — before reasoning about its duration.
|
||
* **A near-black frame has no signature.** In frame-matching, low-contrast frames
|
||
drive every correlation toward zero and the ranking becomes noise; one control
|
||
frame scored 0.000 and another tied the wrong movie at 0.98. Read a dark
|
||
frame's runner-up as *no evidence*, not as evidence for the runner-up.
|
||
* **A screen that never changes is not necessarily hung — it may be a menu.** A
|
||
90-second screenshot loop that presses nothing will report a perfectly healthy
|
||
`DIFFICULTY` menu as a standing hang, and that reading survived in this corpus
|
||
for months. Before calling a static screen a hang, *look at it*, and press
|
||
something.
|
||
* **Never run `ps -ef` in this container.** Each of the three long-lived
|
||
processes carries the whole loop prompt as its argv, so the listing is tens of
|
||
kilobytes of the prompt you already have. Use `pgrep -x <name>` or
|
||
`ps -o pid=,stat= -C <name>`.
|
||
* **A file extension is not a format.** `sound.pak` is full of `.slb` banks and
|
||
names a `.xgs` project, and neither is an XACT container: the whole 1.08 GB
|
||
archive contains zero `XGSF`, `SDBK` or `WBND` magic, and the executable has no
|
||
XACT string at all. The names came from the authoring tool and survived into the
|
||
shipped data. Check for the magic before committing to a parser for the format
|
||
the extension advertises.
|
||
* **A log-polling probe must compare a COUNT, not test for presence.** Timing an
|
||
operation by waiting for its "done" line in a log gives an instant false
|
||
positive if any earlier run left one there. This produced a confident
|
||
"39.5 fps" that was really 0.1 s of matching a stale line — and it disagreed
|
||
with the true 28.5 fps by enough to have flipped a conclusion. Snapshot
|
||
`log.count(marker)` before starting and wait for it to *increase*.
|
||
* **This container kills long emulator runs.** Twice in one session the
|
||
`xenia_canary` process vanished with **no crash line in its own log** — once
|
||
~50 s into a boot, once ~145 s in at 1.15 GB RSS — while the box sat at
|
||
~1 GB free with swap exhausted. Budget dynamic experiments to fit in ~2 minutes
|
||
of guest time, take the measurement early, and never read a disappearance as a
|
||
guest crash without checking the log tail for one.
|
||
* **When a frame-match will not resolve, sample denser before doubting the
|
||
hypothesis.** Identifying a movie from 5-second screenshots gave contrast-23
|
||
frames, a playhead that would not join up, and a runner-up winning one frame —
|
||
it looked like weak evidence for the wrong film. The same question sampled every
|
||
0.5 s (`ffmpeg x11grab` at 10 fps) answered at **0.999 with a strictly monotone
|
||
playhead**. Sparse sampling does not weaken a signal gracefully; it turns it
|
||
into noise that looks like a different answer.
|
||
* **Check what instrumentation the local build already has before declaring a
|
||
question blocked on tooling.** Two Q8 residuals were written off as needing "an
|
||
emulator whose audio path can be observed". The emulator in this container
|
||
already had `xma_param_probe` — visible in the startup CONFIG DUMP the whole
|
||
time — added by this project for exactly that purpose. Read the cvar dump, or
|
||
`nm -C <binary> | grep cvars::`, before concluding you cannot measure something.
|