5ad6e9397c
Censusing ptc_pack's own naming vocabulary turned up a third variant of the prefix trap, and this one had been corrupting a number the corpus carried. 268 of ptc_pack's names do not start with eff_ at all. They start with EF_IDX_, as in EF_IDX_eff_d001_f. A regex anchored at eff_ chops that prefix off and merges distinct names, which is exactly where the earlier figure of 532 came from. Enumerating maximal [A-Za-z0-9_] runs gives 727. The two earlier traps were a STORED name being longer (rot_n001_break) and a BOUND name being a prefix (eff_f0002 inside eff_f0002_barnhaze); this is the third - a prefix the pattern cannot see at all, because its anchor sits in the middle of the real name. Looking each bound name up bare AND under EF_IDX_ resolves 25 of the 34 that were unlocated. The map is now 128 of 137, and the residual is 9, small enough to print: eff_e0044, eff_f0002, eff_f0002_barn, eff_h308, eff_j002_e01, eff_j002_e02, eff_m010_wep_85, eff_m011_wep_85, eff_n0071. All 17 eff_l### are among the recovered. This withdraws my own previous correction. I had recorded Base.xpr (53) as holding more bound effects than ptc_pack (46), and struck out "ptc_pack is the effect library". With the prefixed keys counted ptc_pack holds 71 - it IS the larger library, and the 46 was an undercount from the same truncating pattern. Two shared libraries remains right; which one is bigger does not. The suffix vocabulary: 106 distinct tokens over the 727 names - IDX 223 (the prefix above), _f 137, _e 119, _root 87, _col 54, _mdl 45, _break 43, _ring 38, _ALL 17, _haze 14, _thunder 10. That census counts ALL tokens rather than trailing ones, which is precisely how the EF_IDX_ PREFIX surfaced inside what I had first labelled a suffix list - the mislabel found the bug. Testing the structural candidates the way _hangar was tested, does the suffixed name have a bare parent: _ALL 17 names 17 of 17 _root 87 64 of 87 _break 30 15 of 30 _e 74 0 of 74 _f 61 0 of 61 _root is strictly terminal - 87 of 87, and it never appears mid-name. The compound shapes put it outermost: _e_root 19, _f_root 18, _break_root 13, bare _root 30. So the order is <stem>_[<faction>|<break>]_root and _root reads as a hierarchy marker rather than a variant - though 64 of 87 having a bare parent means it is not simply the parent of an existing node, and _break at 15 of 30 is likewise not a plain destroyed-twin-of-everything. _e/_f never have a bare parent, 0 of 135. That is independent asset-side confirmation of the faction law: an effect is authored per faction and there is no faction-neutral original for either side to derive from. effect-homes.txt changes 5/30 and every line pairs: five values changed (103->128, 34->9, ptc_pack 46->71 and its sort position, the residual header, 3-digit 80->105 of 110) plus 25 pure deletions, exactly the 25 recovered names. All are 3-digit, so the 4-digit line is unchanged at 23 of 27. The other sixteen artefacts are byte-identical.
Reverse-engineering knowledge base
This directory is the spec-side of the clean-room: it records what the original Project Sylpheed binary does (behaviour) and how its data is laid out, so that the Rust port can be implemented from these specs without re-deriving anything and without ever copying original code.
It exists to answer one question fast: "do we already know how X works, and how sure are we?"
The one rule that matters
Never document a claim more confidently than the evidence supports, and never paste original code here.
A wrong-but-confident note is worse than no note: someone builds on it and the bug hides for weeks. Every entry therefore carries an explicit confidence and its evidence. This mirrors the project method — measure the oracle, never infer; refute before believing.
Clean-room firewall
- ✅ Allowed: behaviour descriptions, field offsets/types, formulas, state machines,
observed input→output pairs, and references to the original by address
(
sub_821B68C0) or toxenia-rs/sylpheed.db. - ❌ Forbidden here and in
crates/: pasted decompiled C/C++ or verbatim disassembled function bodies presented as the thing to reimplement. Cite the address; describe the behaviour in your own words. Disassembly is a tool for understanding, not a source to copy.
Confidence levels
| Level | Meaning | Bar to reach it |
|---|---|---|
CONFIRMED |
Behaviour verified against ground truth. | ≥2 independent observations or one observation cross-checked against an oracle (canary framebuffer, a known-correct value, a second code path). |
PROBABLE |
Strong single-source inference. | One clean observation, or an unambiguous static read of the disassembly. |
HYPOTHESIS |
Educated guess, not yet tested. | Anything else. Must say what would confirm/refute it. |
The status markers
The table above is the confidence scale. The markers that appear in
BACKLOG.md and the structures/ pages are a separate, and until now undefined,
vocabulary. They mean:
| Marker | Meaning |
|---|---|
| ✅ | Confirmed — verified against ground truth. |
| 🟡 | Partial: true as far as it goes, or true under a stated assumption. |
| ❔ | Open question. Nobody has answered it yet. |
| 🔴 | Refuted — shown false — or blocked by something the container cannot do. |
| ❌ | A specific claim that was tried and failed. Prefer 🔴. |
| 🚧 | Work started and not finished. |
🔴 never means "we have not run it yet." That is ❔ or 🚧. Reserve 🔴's "blocked" sense for a real limit of the box — no push credentials, no hardware Vulkan (lavapipe only), or a decision only the user can make. The box can run the emulator, script input, screenshot, read guest memory, and build and test Rust, so "needs a run" is never a blocker. This paragraph exists because the marker was undefined for 98 uses and three of them were mislabelled that way.
Promotion requires new evidence, not re-reading the old evidence. A HYPOTHESIS that
"looks right again" is still a HYPOTHESIS. Only an independent check promotes it.
If evidence later contradicts an entry, demote it and record the contradiction — do
not silently edit the conclusion.
When to document
- Right after a function/structure crosses from
HYPOTHESISto at leastPROBABLE— before moving to the next code path, so the knowledge isn't lost or re-derived. - Whenever confidence changes (up or down) — append to the Evidence log, don't overwrite.
- Not while it's still a pure guess with no evidence — a one-liner in the relevant backlog/plan is enough until there's something to stand on.
What to document
- Functions/code paths →
docs/re/functions/<name>.md(one file per function or tight cluster). - Data structures / formats →
docs/re/structures/<name>.md. - Keep the index in
INDEX.md(one line each: name · confidence · one-line summary).
Use the templates: _TEMPLATE.function.md,
_TEMPLATE.structure.md.
How we find and confirm code paths (the toolchain)
Everything joins on the guest virtual address (PC) — code addresses are fixed by the XEX load, identical across our emulator and canary.
- Static (cheap, try first):
xenia-rs/sylpheed.db(DuckDB: 25 481 functions, xrefs, strings, vtables, imports). Query withxenia-rs/zq.py—zq.py grep <str>,zq.py xref <addr>,zq.py dis <lo> <hi>,zq.py fn <pc>. Entry points are usually a string (zq.py grep MSG_DEMO) or an import (movie/XMA API) xref'd back to the loader. - Dynamic (when static is ambiguous): run
xenia-rswith its probe suite —--pc-probe/--audit-pc-probe-hex(fires at block entry),--mem-watch(mid-block reads/writes of a VA),--lr-trace(call/return chains),--trace-instructions,--dump-addr(read guest memory). These already exist; prefer them over hacking canary. - Oracle (correctness ground truth): canary — the Wine cross-build
xenia-canary/build-cross/bin/Windows/Debug/xenia_canary.exe(the native Linux ELF crashes / does not run — do not use it). This is the only emulator that reaches the in-game menu; ourxenia-rsnever got past the intro video. Use canary to observe output (capture its framebuffer for texture colours), not usually to instrument code — though itsbuild-crosstoolchain does compile, so small C++ probes + rebuild are possible when needed. Run muted, one emulator process at a time, point it at the real ISO (not the symlink).
⚠️ VA-equality caveat: join code by PC (fixed), but never assume a data VA holds the same bytes across emulators — allocators differ. Compare data by content/layout.