Sylpheed/docs/re/METHOD.md

# Method traps already paid for

Each line cost an iteration at least once. They are general — they are not about
Sylpheed, they are about how this kind of measurement goes wrong.

Like [`REFUTED.md`](REFUTED.md), this list had been living in the autonomous
agent's loop prompt, i.e. nowhere durable. See [`README.md`](README.md) for the
✅/🟡/❔ confidence convention itself.

## Controls

* **Every result needs a control. A control that fails kills the instrument.**
* **Run the known-positive through a new filter FIRST.** Three filters have been
  killed by their own control. When one fails, **read the known-good's
  disassembly** before assuming a shape.
* **A measured negative is a result** — but a negative is only as strong as the
  route you ran, so **state its reach**.
* **A null result needs its cause shown to have happened.**
* **A result with NO unknowns is suspicious.**
* **Census the whole set; always run the other population as the control.**
  **Zero partials is stronger than a majority.**
* **A 2×2 partition is the sharpest general tool** — both off-diagonals empty is
  a law.
* **Re-derive a doc's own numbers as the control.**

## Inference

* **Never conclude from ONE sample.**
* **A law proved on one population is a hypothesis on the next.**
* **Finding one exception does not imply a family.**
* **Consistency is not proof. A suggestive coincidence is a coincidence until
  measured.** **An analogy is not a measurement.**
* **Same layout ≠ same instance.** **Same record-name set ≠ same object.**
* **A marker is only proven by what it leaves out.**
* **A high-confidence SCORE is not a high-confidence MECHANISM.**
* **Knowing HOW MANY is not knowing WHICH.**
* **Round numbers matching is weak evidence — unless you read the constant.**
* **My own last-turn result is a hypothesis too.**
* **A global partition can understate a per-owner one.**
* **A residual is measured against a population — name it.**

## Searching and tooling

* **A search that returns thousands has no power; state the reach.**
* **A substring match is not a hit.** **A regex miss looks like a null result —
  print one raw sample before believing a zero.**
* **A derived table can be a cross product — measure its shape first.**
* **After refuting an instrument, sweep everything that depended on it.**
* **Before measuring how wrong a tool is, read what the tool actually does.**
* **The instrument must pass its own control.**
* **Classify a bulk before mining it. The residual is the prize.**
* **Rank by similarity — the cliff is the finding.** But **read the values
  before trusting the rank.**
* **Grep the nouns before designing the experiment — and believe it.**
* **Grep gives you a file list — read *every* file on it.**
* **The answer is often already in the doc that owns the subject — read it end
  to end.** A 🟡 often names its own route.
* **Before re-trying a blocked idea, check whether the blocker's own doc already
  tried it.**
* **Ship a regenerator with every artefact.** An artefact that moves by a pure
  reorder is a tool bug.
* **Never print per-entry lines from a disc-wide sweep — aggregate.**

## Reading the data

* **Read what a loader NAMES, not where it stores.**
* **A field the disc never values still gets named by the loader.**
* **An indexed read beats a deduped-pool adjacency read.**
* **Check the whole string set, not the one matching word.**
* **A dict keyed by record name across a multi-entry pak is a lie.**
* **A set-difference over names hides reuse — join per USER.**
* **A self-index names records, not files.**
* **Case-insensitive hashing means two spellings can be one entry.**
* **An "unresolved" name may be the wrong kind, namespace or prefix — or part of
  a cut asset.**
* **A garbled value may be a real string in another encoding.**
* **Two of my own counts disagreeing is a grammar clue.**
* **A game's own typo is a join key.**
* **A bias constant in the code is a join key.**
* **A prefix trap: enumerate maximal `[A-Za-z0-9_]` runs, not `startswith`.**
* **Re-deriving a format is not a finding — asking whether its values *resolve*
  is.**

## Mechanics that have bitten

* **Never hand-convert a decimal VA — print `hex()`.**
* **`grep -c` counts LINES** — use `grep -o | wc -l`.
* **`Counter.most_common()` tie-breaks by insertion order — use `sorted()`.**
* **Raw grep cannot see inside compressed pak entries.**
* **Commit messages go in a file** (`git commit -F`); a literal `|` in a table
  cell needs escaping; `git log --all -- <path>` can hang.

## Runtime / emulator

* **Look at the PNG** — and check its dimensions.
* **"Animating" is not "still in a mission".**
* **Dedup entity enumerations by position value.**
* **Do not diagnose timing or liveness under gdb.** `ps %cpu` is cumulative.
* **Classify screens by whole-image statistics, not named pixels** — a named
  pixel is only valid while the image sits at a known place, and nothing errors
  when it moves.
* **Do not poll faster than the guest updates** — it manufactures a clean curve
  out of noise.
* **A probe that never performs the action will "prove" the action does not
  exist.**
* **The container's Canary binary can be older than the Canary source tree, and
  the failure mode is a hang, not an error.** After a merge into `sylpheed-re`
  the prebuilt `xenia_canary` had no `log_ui_draws`, no `mem_watch`, no
  `create_profile_if_none` — and an unknown cvar makes xenia open an SDL message
  box before logging is up, which headless is an unexplained freeze. Check before
  trusting a harness flag: `nm -C <binary> | grep cvars::<flag>`, and
  `build-canary Release` if it is missing.
* **A capture armed *at* a screen only ever sees the steady state.** Anything
  about how a screen is built or animated has to be armed *before* it exists.
  Re-arming every few seconds and keeping every log tiles the approach: each F10
  opens a new numbered file and closes the previous one complete.
* **Measure animation in submitted frames, not in seconds.** `VdSwap` counts are
  the guest's own frames, so an emulator at 80 % of real time does not move them;
  a stopwatch reading does, silently and by an unknown factor.
* **~~This game's menus drop d-pad presses shorter than ~0.3 s.~~ WITHDRAWN
  2026-08-28 — the menu WRAPS, and I had not measured that.** The claim came from
  reading a cursor that ended up "one item short"; once wrap-around at both ends
  was measured ([`menu-navigation-semantics.md`](menu-navigation-semantics.md)),
  every one of those press counts is exactly right — four presses at 0.12 s moved
  four steps *through the bottom*, which lands one above where a non-wrapping
  menu would put it. **No press was ever dropped.** The real lesson is the
  general one: *a step count is only readable once you know the topology*, and I
  invented a hardware-flakiness story rather than testing the ends of the list.
  Still true and worth keeping: screenshot after every step and read the cursor,
  rather than trusting arithmetic over the press count.
* **A screenshot taken right after a transition can catch a screen mid-fade.**
  A grab 2.5 s after Ⓑ returned the game to the title showed the title art with
  no `PRESS Ⓐ BUTTON` plate; one second later the plate was there. That very
  nearly went into the corpus as "the returned title has no plate". Sample a
  changing screen several times before writing down what it does *not* contain.
* **Do not identify a menu cursor by label brightness.** The obvious oracle —
  "the focused label is the brightest row" — fails on this game's menus, because
  the background art is brighter behind some rows than the highlight is. It
  confidently named the wrong item on a frame whose ring was plainly elsewhere.
  Detect the **focus ring** in the gutter left of the labels instead
  (`tools/re-capture/menu_focus.py`, 254 vs <82 — no threshold tuning needed),
  and look at the PNG before believing either.
* **`screenshot` samples at 0.5 Hz — it cannot time an animation.** Measured:
  ~2 s per grab (an `import` of the root plus an ImageMagick crop). A 0.4 s fade
  falls entirely between two samples, which is why a 40-frame burst across a
  screen change looked like an instant cut. For anything timed, record the
  display instead: `ffmpeg -f x11grab -framerate 30 -video_size <W>x<H> -i
  :98+<X>,<Y> -t <secs>`, then read per-frame statistics off the file. Take the
  geometry from `xwininfo -root -tree`, the same way `bin/screenshot` does.
* **A screen's brightness curve is not its fade quad.** The incoming screen's own
  elements animate in *after* the transition quad has cleared, so mean luminance
  keeps rising long after the fade is over — 1.47 s against a declared 0.97 s on
  one screen. Time the fade from where the frame is *pure black*, and take the
  ramp itself from the keyframes.
* **A `sound.pak` entry is not a bank.** Entry windows straddle bank boundaries
  (the leading-region effect), so a naive "walk the RIFFs in this entry" gives
  the tail of the previous bank, then one of your own, then the *next* bank's
  header. It looks like a bank with the wrong number of waves. The tell is a
  `BANK id=` that is not the entry's own number, and the fix is the `seek` packet
  counts — they pin each wave's join exactly, so a straddle can be realigned
  rather than thrown away. 4 of 32 BGM banks look like counterexamples until this
  is applied.
* **Do not time a video whose start you did not see.** An attract cycle sampled
  from partway through measured "≈85 s", which was used to argue it was not the
  137 s `ADV.wmv`. It was `ADV.wmv`; the sampling began 39 s in. Identify a video
  by **content** — match frames against the candidates and check the matched
  playhead advances monotonically — before reasoning about its duration.
* **A near-black frame has no signature.** In frame-matching, low-contrast frames
  drive every correlation toward zero and the ranking becomes noise; one control
  frame scored 0.000 and another tied the wrong movie at 0.98. Read a dark
  frame's runner-up as *no evidence*, not as evidence for the runner-up.
* **A screen that never changes is not necessarily hung — it may be a menu.** A
  90-second screenshot loop that presses nothing will report a perfectly healthy
  `DIFFICULTY` menu as a standing hang, and that reading survived in this corpus
  for months. Before calling a static screen a hang, *look at it*, and press
  something.
* **Never run `ps -ef` in this container.** Each of the three long-lived
  processes carries the whole loop prompt as its argv, so the listing is tens of
  kilobytes of the prompt you already have. Use `pgrep -x <name>` or
  `ps -o pid=,stat= -C <name>`.
* **A file extension is not a format.** `sound.pak` is full of `.slb` banks and
  names a `.xgs` project, and neither is an XACT container: the whole 1.08 GB
  archive contains zero `XGSF`, `SDBK` or `WBND` magic, and the executable has no
  XACT string at all. The names came from the authoring tool and survived into the
  shipped data. Check for the magic before committing to a parser for the format
  the extension advertises.
* **A log-polling probe must compare a COUNT, not test for presence.** Timing an
  operation by waiting for its "done" line in a log gives an instant false
  positive if any earlier run left one there. This produced a confident
  "39.5 fps" that was really 0.1 s of matching a stale line — and it disagreed
  with the true 28.5 fps by enough to have flipped a conclusion. Snapshot
  `log.count(marker)` before starting and wait for it to *increase*.
* **This container kills long emulator runs.** Twice in one session the
  `xenia_canary` process vanished with **no crash line in its own log** — once
  ~50 s into a boot, once ~145 s in at 1.15 GB RSS — while the box sat at
  ~1 GB free with swap exhausted. Budget dynamic experiments to fit in ~2 minutes
  of guest time, take the measurement early, and never read a disappearance as a
  guest crash without checking the log tail for one.
* **When a frame-match will not resolve, sample denser before doubting the
  hypothesis.** Identifying a movie from 5-second screenshots gave contrast-23
  frames, a playhead that would not join up, and a runner-up winning one frame —
  it looked like weak evidence for the wrong film. The same question sampled every
  0.5 s (`ffmpeg x11grab` at 10 fps) answered at **0.999 with a strictly monotone
  playhead**. Sparse sampling does not weaken a signal gracefully; it turns it
  into noise that looks like a different answer.
* **Check what instrumentation the local build already has before declaring a
  question blocked on tooling.** Two Q8 residuals were written off as needing "an
  emulator whose audio path can be observed". The emulator in this container
  already had `xma_param_probe` — visible in the startup CONFIG DUMP the whole
  time — added by this project for exactly that purpose. Read the cvar dump, or
  `nm -C <binary> | grep cvars::`, before concluding you cannot measure something.
* **"Strings a function references" is not "arguments a function passes".** A
  loose filter over the strings in a caller produced a tidy-looking screen
  vocabulary; recovering the actual register argument at each call site removed
  half of it, including two names that appeared to be corroborated by independent
  measurement. If the claim is about an argument, simulate the registers — the
  cheap proxy manufactures agreement.
* **A backward scan for `addi rN, …` misses arguments staged through another
  register.** PPC code often builds a pointer in a scratch register and moves it
  into place later (`addi r27, r11, X` … `mr r5, r27`). A backward search for the
  destination register alone found 7 of 48 sites and missed the two known-good
  ones. Simulate forward over a window instead, tracking `lis`/`addi`/`mr`.
* **Check a measurement against the WHOLE reference set, not the part you were
  reading.** Two observed BGM stream sizes were recorded as matching "no bank's
  declared waves", which spawned a wrong theory that the decoder gets a window
  rather than a whole wave. They match `BGM_103` exactly — the check had covered
  only the `BGM_0xx` rows of the census, because that is the block that had been
  on screen.
* **`instructions.function` is unpopulated for most rows in `sylpheed.db`.** A
  query keyed on it returns *nothing* rather than erroring — a `bctr` search
  scoped that way reported "no jump tables" for a function with two known ones.
  Scope by `address between <start> and <end_address>` instead, and gate any such
  query on a function whose answer you already know.
* **Before testing "state X behaves like this", check that your route actually
  reaches state X.** A test of "does Ⓐ work in phase 4 state 0" was run by
  pressing Ⓑ, on the strength of a decoded `4 → state 0` edge — but the same basic
  block also writes the *phase* field, sending it to phase 2. The test probed the
  wrong state and produced a confident refutation of a live hypothesis. When a
  transition sets more than one field, read the whole block, not the one store you
  were looking for.