Syplheed-Reborn/docs/re/METHOD.md

# Method traps already paid for

Each line cost an iteration at least once. They are general — they are not about
Sylpheed, they are about how this kind of measurement goes wrong.

Like [`REFUTED.md`](REFUTED.md), this list had been living in the autonomous
agent's loop prompt, i.e. nowhere durable. See [`README.md`](README.md) for the
✅/🟡/❔ confidence convention itself.

## Controls

* **Every result needs a control. A control that fails kills the instrument.**
* **Run the known-positive through a new filter FIRST.** Three filters have been
  killed by their own control. When one fails, **read the known-good's
  disassembly** before assuming a shape.
* **A measured negative is a result** — but a negative is only as strong as the
  route you ran, so **state its reach**.
* **A null result needs its cause shown to have happened.**
* **A result with NO unknowns is suspicious.**
* **Census the whole set; always run the other population as the control.**
  **Zero partials is stronger than a majority.**
* **A 2×2 partition is the sharpest general tool** — both off-diagonals empty is
  a law.
* **Re-derive a doc's own numbers as the control.**

## Inference

* **Never conclude from ONE sample.**
* **A law proved on one population is a hypothesis on the next.**
* **Finding one exception does not imply a family.**
* **Consistency is not proof. A suggestive coincidence is a coincidence until
  measured.** **An analogy is not a measurement.**
* **Same layout ≠ same instance.** **Same record-name set ≠ same object.**
* **A marker is only proven by what it leaves out.**
* **A high-confidence SCORE is not a high-confidence MECHANISM.**
* **Knowing HOW MANY is not knowing WHICH.**
* **Round numbers matching is weak evidence — unless you read the constant.**
* **My own last-turn result is a hypothesis too.**
* **A global partition can understate a per-owner one.**
* **A residual is measured against a population — name it.**

## Searching and tooling

* **A search that returns thousands has no power; state the reach.**
* **A substring match is not a hit.** **A regex miss looks like a null result —
  print one raw sample before believing a zero.**
* **A derived table can be a cross product — measure its shape first.**
* **After refuting an instrument, sweep everything that depended on it.**
* **Before measuring how wrong a tool is, read what the tool actually does.**
* **The instrument must pass its own control.**
* **Classify a bulk before mining it. The residual is the prize.**
* **Rank by similarity — the cliff is the finding.** But **read the values
  before trusting the rank.**
* **Grep the nouns before designing the experiment — and believe it.**
* **Grep gives you a file list — read *every* file on it.**
* **The answer is often already in the doc that owns the subject — read it end
  to end.** A 🟡 often names its own route.
* **Before re-trying a blocked idea, check whether the blocker's own doc already
  tried it.**
* **Ship a regenerator with every artefact.** An artefact that moves by a pure
  reorder is a tool bug.
* **Never print per-entry lines from a disc-wide sweep — aggregate.**

## Reading the data

* **Read what a loader NAMES, not where it stores.**
* **A field the disc never values still gets named by the loader.**
* **An indexed read beats a deduped-pool adjacency read.**
* **Check the whole string set, not the one matching word.**
* **A dict keyed by record name across a multi-entry pak is a lie.**
* **A set-difference over names hides reuse — join per USER.**
* **A self-index names records, not files.**
* **Case-insensitive hashing means two spellings can be one entry.**
* **An "unresolved" name may be the wrong kind, namespace or prefix — or part of
  a cut asset.**
* **A garbled value may be a real string in another encoding.**
* **Two of my own counts disagreeing is a grammar clue.**
* **A game's own typo is a join key.**
* **A bias constant in the code is a join key.**
* **A prefix trap: enumerate maximal `[A-Za-z0-9_]` runs, not `startswith`.**
* **Re-deriving a format is not a finding — asking whether its values *resolve*
  is.**

## Mechanics that have bitten

* **Never hand-convert a decimal VA — print `hex()`.**
* **`grep -c` counts LINES** — use `grep -o | wc -l`.
* **`Counter.most_common()` tie-breaks by insertion order — use `sorted()`.**
* **Raw grep cannot see inside compressed pak entries.**
* **Commit messages go in a file** (`git commit -F`); a literal `|` in a table
  cell needs escaping; `git log --all -- <path>` can hang.

## Runtime / emulator

* **Look at the PNG** — and check its dimensions.
* **"Animating" is not "still in a mission".**
* **Dedup entity enumerations by position value.**
* **Do not diagnose timing or liveness under gdb.** `ps %cpu` is cumulative.
* **Classify screens by whole-image statistics, not named pixels** — a named
  pixel is only valid while the image sits at a known place, and nothing errors
  when it moves.
* **Do not poll faster than the guest updates** — it manufactures a clean curve
  out of noise.
* **A probe that never performs the action will "prove" the action does not
  exist.**
* **The container's Canary binary can be older than the Canary source tree, and
  the failure mode is a hang, not an error.** After a merge into `sylpheed-re`
  the prebuilt `xenia_canary` had no `log_ui_draws`, no `mem_watch`, no
  `create_profile_if_none` — and an unknown cvar makes xenia open an SDL message
  box before logging is up, which headless is an unexplained freeze. Check before
  trusting a harness flag: `nm -C <binary> | grep cvars::<flag>`, and
  `build-canary Release` if it is missing.
* **A capture armed *at* a screen only ever sees the steady state.** Anything
  about how a screen is built or animated has to be armed *before* it exists.
  Re-arming every few seconds and keeping every log tiles the approach: each F10
  opens a new numbered file and closes the previous one complete.
* **Measure animation in submitted frames, not in seconds.** `VdSwap` counts are
  the guest's own frames, so an emulator at 80 % of real time does not move them;
  a stopwatch reading does, silently and by an unknown factor.
* **~~This game's menus drop d-pad presses shorter than ~0.3 s.~~ WITHDRAWN
  2026-08-28 — the menu WRAPS, and I had not measured that.** The claim came from
  reading a cursor that ended up "one item short"; once wrap-around at both ends
  was measured ([`menu-navigation-semantics.md`](menu-navigation-semantics.md)),
  every one of those press counts is exactly right — four presses at 0.12 s moved
  four steps *through the bottom*, which lands one above where a non-wrapping
  menu would put it. **No press was ever dropped.** The real lesson is the
  general one: *a step count is only readable once you know the topology*, and I
  invented a hardware-flakiness story rather than testing the ends of the list.
  Still true and worth keeping: screenshot after every step and read the cursor,
  rather than trusting arithmetic over the press count.
* **A screenshot taken right after a transition can catch a screen mid-fade.**
  A grab 2.5 s after Ⓑ returned the game to the title showed the title art with
  no `PRESS Ⓐ BUTTON` plate; one second later the plate was there. That very
  nearly went into the corpus as "the returned title has no plate". Sample a
  changing screen several times before writing down what it does *not* contain.
* **Do not identify a menu cursor by label brightness.** The obvious oracle —
  "the focused label is the brightest row" — fails on this game's menus, because
  the background art is brighter behind some rows than the highlight is. It
  confidently named the wrong item on a frame whose ring was plainly elsewhere.
  Detect the **focus ring** in the gutter left of the labels instead
  (`tools/re-capture/menu_focus.py`, 254 vs <82 — no threshold tuning needed),
  and look at the PNG before believing either.
* **`screenshot` samples at 0.5 Hz — it cannot time an animation.** Measured:
  ~2 s per grab (an `import` of the root plus an ImageMagick crop). A 0.4 s fade
  falls entirely between two samples, which is why a 40-frame burst across a
  screen change looked like an instant cut. For anything timed, record the
  display instead: `ffmpeg -f x11grab -framerate 30 -video_size <W>x<H> -i
  :98+<X>,<Y> -t <secs>`, then read per-frame statistics off the file. Take the
  geometry from `xwininfo -root -tree`, the same way `bin/screenshot` does.
* **A screen's brightness curve is not its fade quad.** The incoming screen's own
  elements animate in *after* the transition quad has cleared, so mean luminance
  keeps rising long after the fade is over — 1.47 s against a declared 0.97 s on
  one screen. Time the fade from where the frame is *pure black*, and take the
  ramp itself from the keyframes.
* **A `sound.pak` entry is not a bank.** Entry windows straddle bank boundaries
  (the leading-region effect), so a naive "walk the RIFFs in this entry" gives
  the tail of the previous bank, then one of your own, then the *next* bank's
  header. It looks like a bank with the wrong number of waves. The tell is a
  `BANK id=` that is not the entry's own number, and the fix is the `seek` packet
  counts — they pin each wave's join exactly, so a straddle can be realigned
  rather than thrown away. 4 of 32 BGM banks look like counterexamples until this
  is applied.
* **Do not time a video whose start you did not see.** An attract cycle sampled
  from partway through measured "≈85 s", which was used to argue it was not the
  137 s `ADV.wmv`. It was `ADV.wmv`; the sampling began 39 s in. Identify a video
  by **content** — match frames against the candidates and check the matched
  playhead advances monotonically — before reasoning about its duration.
* **A near-black frame has no signature.** In frame-matching, low-contrast frames
  drive every correlation toward zero and the ranking becomes noise; one control
  frame scored 0.000 and another tied the wrong movie at 0.98. Read a dark
  frame's runner-up as *no evidence*, not as evidence for the runner-up.
* **A screen that never changes is not necessarily hung — it may be a menu.** A
  90-second screenshot loop that presses nothing will report a perfectly healthy
  `DIFFICULTY` menu as a standing hang, and that reading survived in this corpus
  for months. Before calling a static screen a hang, *look at it*, and press
  something.
* **Never run `ps -ef` in this container.** Each of the three long-lived
  processes carries the whole loop prompt as its argv, so the listing is tens of
  kilobytes of the prompt you already have. Use `pgrep -x <name>` or
  `ps -o pid=,stat= -C <name>`.
* **A file extension is not a format.** `sound.pak` is full of `.slb` banks and
  names a `.xgs` project, and neither is an XACT container: the whole 1.08 GB
  archive contains zero `XGSF`, `SDBK` or `WBND` magic, and the executable has no
  XACT string at all. The names came from the authoring tool and survived into the
  shipped data. Check for the magic before committing to a parser for the format
  the extension advertises.
* **A log-polling probe must compare a COUNT, not test for presence.** Timing an
  operation by waiting for its "done" line in a log gives an instant false
  positive if any earlier run left one there. This produced a confident
  "39.5 fps" that was really 0.1 s of matching a stale line — and it disagreed
  with the true 28.5 fps by enough to have flipped a conclusion. Snapshot
  `log.count(marker)` before starting and wait for it to *increase*.
* **This container kills long emulator runs.** Twice in one session the
  `xenia_canary` process vanished with **no crash line in its own log** — once
  ~50 s into a boot, once ~145 s in at 1.15 GB RSS — while the box sat at
  ~1 GB free with swap exhausted. Budget dynamic experiments to fit in ~2 minutes
  of guest time, take the measurement early, and never read a disappearance as a
  guest crash without checking the log tail for one.