Sylpheed/docs/re/METHOD.md

Method traps already paid for

Each line cost an iteration at least once. They are general — they are not about Sylpheed, they are about how this kind of measurement goes wrong.

Like REFUTED.md, this list had been living in the autonomous agent's loop prompt, i.e. nowhere durable. See README.md for the ✅/🟡/❔ confidence convention itself.

Controls

• Every result needs a control. A control that fails kills the instrument.
• Run the known-positive through a new filter FIRST. Three filters have been killed by their own control. When one fails, read the known-good's disassembly before assuming a shape.
• A measured negative is a result — but a negative is only as strong as the route you ran, so state its reach.
• A null result needs its cause shown to have happened.
• A result with NO unknowns is suspicious.
• Census the whole set; always run the other population as the control. Zero partials is stronger than a majority.
• A 2×2 partition is the sharpest general tool — both off-diagonals empty is a law.
• Re-derive a doc's own numbers as the control.

Inference

• Never conclude from ONE sample.
• A law proved on one population is a hypothesis on the next.
• Finding one exception does not imply a family.
• Consistency is not proof. A suggestive coincidence is a coincidence until measured. An analogy is not a measurement.
• Same layout ≠ same instance. Same record-name set ≠ same object.
• A marker is only proven by what it leaves out.
• A high-confidence SCORE is not a high-confidence MECHANISM.
• Knowing HOW MANY is not knowing WHICH.
• Round numbers matching is weak evidence — unless you read the constant.
• My own last-turn result is a hypothesis too.
• A global partition can understate a per-owner one.
• A residual is measured against a population — name it.

Searching and tooling

• A search that returns thousands has no power; state the reach.
• A substring match is not a hit. A regex miss looks like a null result — print one raw sample before believing a zero.
• A derived table can be a cross product — measure its shape first.
• After refuting an instrument, sweep everything that depended on it.
• Before measuring how wrong a tool is, read what the tool actually does.
• The instrument must pass its own control.
• Classify a bulk before mining it. The residual is the prize.
• Rank by similarity — the cliff is the finding. But read the values before trusting the rank.
• Grep the nouns before designing the experiment — and believe it.
• Grep gives you a file list — read every file on it.
• The answer is often already in the doc that owns the subject — read it end to end. A 🟡 often names its own route.
• Before re-trying a blocked idea, check whether the blocker's own doc already tried it.
• Ship a regenerator with every artefact. An artefact that moves by a pure reorder is a tool bug.
• Never print per-entry lines from a disc-wide sweep — aggregate.

Reading the data

• Read what a loader NAMES, not where it stores.
• A field the disc never values still gets named by the loader.
• An indexed read beats a deduped-pool adjacency read.
• Check the whole string set, not the one matching word.
• A dict keyed by record name across a multi-entry pak is a lie.
• A set-difference over names hides reuse — join per USER.
• A self-index names records, not files.
• Case-insensitive hashing means two spellings can be one entry.
• An "unresolved" name may be the wrong kind, namespace or prefix — or part of a cut asset.
• A garbled value may be a real string in another encoding.
• Two of my own counts disagreeing is a grammar clue.
• A game's own typo is a join key.
• A bias constant in the code is a join key.
• A prefix trap: enumerate maximal [A-Za-z0-9_] runs, not startswith.
• Re-deriving a format is not a finding — asking whether its values resolve is.

Mechanics that have bitten

• Never hand-convert a decimal VA — print hex().
• grep -c counts LINES — use grep -o | wc -l.
• Counter.most_common() tie-breaks by insertion order — use sorted().
• Raw grep cannot see inside compressed pak entries.
• Commit messages go in a file (git commit -F); a literal | in a table cell needs escaping; git log --all -- <path> can hang.

Runtime / emulator

• Look at the PNG — and check its dimensions.
• "Animating" is not "still in a mission".
• Dedup entity enumerations by position value.
• Do not diagnose timing or liveness under gdb. ps %cpu is cumulative.
• Classify screens by whole-image statistics, not named pixels — a named pixel is only valid while the image sits at a known place, and nothing errors when it moves.
• Do not poll faster than the guest updates — it manufactures a clean curve out of noise.
• A probe that never performs the action will "prove" the action does not exist.
• The container's Canary binary can be older than the Canary source tree, and the failure mode is a hang, not an error. After a merge into sylpheed-re the prebuilt xenia_canary had no log_ui_draws, no mem_watch, no create_profile_if_none — and an unknown cvar makes xenia open an SDL message box before logging is up, which headless is an unexplained freeze. Check before trusting a harness flag: nm -C <binary> | grep cvars::<flag>, and build-canary Release if it is missing.
• A capture armed at a screen only ever sees the steady state. Anything about how a screen is built or animated has to be armed before it exists. Re-arming every few seconds and keeping every log tiles the approach: each F10 opens a new numbered file and closes the previous one complete.
• Measure animation in submitted frames, not in seconds. VdSwap counts are the guest's own frames, so an emulator at 80 % of real time does not move them; a stopwatch reading does, silently and by an unknown factor.
• ~This game's menus drop d-pad presses shorter than 0.3 s. WITHDRAWN 2026-08-28 — the menu WRAPS, and I had not measured that. The claim came from reading a cursor that ended up "one item short"; once wrap-around at both ends was measured (menu-navigation-semantics.md), every one of those press counts is exactly right — four presses at 0.12 s moved four steps through the bottom, which lands one above where a non-wrapping menu would put it. No press was ever dropped. The real lesson is the general one: a step count is only readable once you know the topology, and I invented a hardware-flakiness story rather than testing the ends of the list. Still true and worth keeping: screenshot after every step and read the cursor, rather than trusting arithmetic over the press count.
• A screenshot taken right after a transition can catch a screen mid-fade. A grab 2.5 s after Ⓑ returned the game to the title showed the title art with no PRESS Ⓐ BUTTON plate; one second later the plate was there. That very nearly went into the corpus as "the returned title has no plate". Sample a changing screen several times before writing down what it does not contain.
• Do not identify a menu cursor by label brightness. The obvious oracle — "the focused label is the brightest row" — fails on this game's menus, because the background art is brighter behind some rows than the highlight is. It confidently named the wrong item on a frame whose ring was plainly elsewhere. Detect the focus ring in the gutter left of the labels instead (tools/re-capture/menu_focus.py, 254 vs <82 — no threshold tuning needed), and look at the PNG before believing either.
• screenshot samples at 0.5 Hz — it cannot time an animation. Measured: ~2 s per grab (an import of the root plus an ImageMagick crop). A 0.4 s fade falls entirely between two samples, which is why a 40-frame burst across a screen change looked like an instant cut. For anything timed, record the display instead: ffmpeg -f x11grab -framerate 30 -video_size <W>x<H> -i :98+<X>,<Y> -t <secs>, then read per-frame statistics off the file. Take the geometry from xwininfo -root -tree, the same way bin/screenshot does.
• A screen's brightness curve is not its fade quad. The incoming screen's own elements animate in after the transition quad has cleared, so mean luminance keeps rising long after the fade is over — 1.47 s against a declared 0.97 s on one screen. Time the fade from where the frame is pure black, and take the ramp itself from the keyframes.
• A sound.pak entry is not a bank. Entry windows straddle bank boundaries (the leading-region effect), so a naive "walk the RIFFs in this entry" gives the tail of the previous bank, then one of your own, then the next bank's header. It looks like a bank with the wrong number of waves. The tell is a BANK id= that is not the entry's own number, and the fix is the seek packet counts — they pin each wave's join exactly, so a straddle can be realigned rather than thrown away. 4 of 32 BGM banks look like counterexamples until this is applied.
• Do not time a video whose start you did not see. An attract cycle sampled from partway through measured "≈85 s", which was used to argue it was not the 137 s ADV.wmv. It was ADV.wmv; the sampling began 39 s in. Identify a video by content — match frames against the candidates and check the matched playhead advances monotonically — before reasoning about its duration.
• A near-black frame has no signature. In frame-matching, low-contrast frames drive every correlation toward zero and the ranking becomes noise; one control frame scored 0.000 and another tied the wrong movie at 0.98. Read a dark frame's runner-up as no evidence, not as evidence for the runner-up.
• A screen that never changes is not necessarily hung — it may be a menu. A 90-second screenshot loop that presses nothing will report a perfectly healthy DIFFICULTY menu as a standing hang, and that reading survived in this corpus for months. Before calling a static screen a hang, look at it, and press something.
• Never run ps -ef in this container. Each of the three long-lived processes carries the whole loop prompt as its argv, so the listing is tens of kilobytes of the prompt you already have. Use pgrep -x <name> or ps -o pid=,stat= -C <name>.
• A file extension is not a format. sound.pak is full of .slb banks and names a .xgs project, and neither is an XACT container: the whole 1.08 GB archive contains zero XGSF, SDBK or WBND magic, and the executable has no XACT string at all. The names came from the authoring tool and survived into the shipped data. Check for the magic before committing to a parser for the format the extension advertises.
• A log-polling probe must compare a COUNT, not test for presence. Timing an operation by waiting for its "done" line in a log gives an instant false positive if any earlier run left one there. This produced a confident "39.5 fps" that was really 0.1 s of matching a stale line — and it disagreed with the true 28.5 fps by enough to have flipped a conclusion. Snapshot log.count(marker) before starting and wait for it to increase.
• This container kills long emulator runs. Twice in one session the xenia_canary process vanished with no crash line in its own log — once ~50 s into a boot, once ~145 s in at 1.15 GB RSS — while the box sat at ~1 GB free with swap exhausted. Budget dynamic experiments to fit in ~2 minutes of guest time, take the measurement early, and never read a disappearance as a guest crash without checking the log tail for one.