Files

Claude d96b225e47 chore(re): stop committing game assets; captures stay local

Issue #49: the repos carry code, tooling and docs only.

Untracks 143 screenshots, 3 savegame blobs and `tools/re-capture/ob_digits.png`
(a digit-template sheet cut from game frames) -- 146 files, 76.4 MB. They stay
in the working tree and are gitignored, so the pages' relative links still
resolve where the captures exist and nothing ships.

Derived measurements (csv/tsv/txt/log/json/jsonl/npy) are our own numbers, not
game content, and stay tracked -- they are what most claims rest on.

`check-capture-citations` had its contract inverted, and it is the half worth
reading:

  * presence now comes from the WORKING TREE, not `git ls-files`. The assets are
    deliberately untracked, so asking the index would report every screenshot as
    missing and fail all 203 citations.
  * a NEW failure: a game asset that IS tracked. A screenshot that sneaks back
    in is invisible in review -- a binary shows as "Bin 0 -> 1234567 bytes" --
    and is permanent once merged, since removing it later needs a history
    rewrite. So that half has to be loud.

Verified:
  * selftest 8/8, including the new rule
  * scan: 212 present, 212 cited, 0 dangling, 0 tracked  -> exit 0
  * force-add one PNG -> "game assets TRACKED: 1", exit 1, selftest red

⚠️ This does NOT remove the blobs from history; a clone still fetches them.
That needs a filter-repo rewrite and a force-push, which is a separate,
human-run step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-09-19 21:03:46 +02:00

6.8 KiB

Raw Permalink Blame History

Reverse-engineering knowledge base

This directory is the spec-side of the clean-room: it records what the original Project Sylpheed binary does (behaviour) and how its data is laid out, so that the Rust port can be implemented from these specs without re-deriving anything and without ever copying original code.

It exists to answer one question fast: "do we already know how X works, and how sure are we?"

The one rule that matters

Never document a claim more confidently than the evidence supports, and never paste original code here.

A wrong-but-confident note is worse than no note: someone builds on it and the bug hides for weeks. Every entry therefore carries an explicit confidence and its evidence. This mirrors the project method — measure the oracle, never infer; refute before believing.

Captures are local-only (issue #49)

The repository carries code, tooling and docs. Screenshots and savegame blobs are game-derived, so since 2026-09-19 they live in docs/re/captures/ on disk and are gitignored — the pages' relative links still resolve on a machine that has them, and nothing ships.

Derived measurements (csv, tsv, txt, log, json, jsonl, npy) are our own numbers rather than game content, and stay tracked — they are what most claims here actually rest on.

tools/re/check-capture-citations enforces both halves: a cited capture must be present, and a game asset must not be tracked. ⚠️ A fresh clone has no captures, so its citations will not resolve until the captures are copied in; that is expected, and the checker is a local gate rather than a CI one.

Clean-room firewall

✅ Allowed: behaviour descriptions, field offsets/types, formulas, state machines, observed input→output pairs, and references to the original by address (sub_821B68C0) or to sylpheed.db.
❌ Forbidden here and in crates/: pasted decompiled C/C++ or verbatim disassembled function bodies presented as the thing to reimplement. Cite the address; describe the behaviour in your own words. Disassembly is a tool for understanding, not a source to copy.

Confidence levels

Level	Meaning	Bar to reach it
`CONFIRMED`	Behaviour verified against ground truth.	≥2 independent observations or one observation cross-checked against an oracle (canary framebuffer, a known-correct value, a second code path).
`PROBABLE`	Strong single-source inference.	One clean observation, or an unambiguous static read of the disassembly.
`HYPOTHESIS`	Educated guess, not yet tested.	Anything else. Must say what would confirm/refute it.

The status markers

The table above is the confidence scale. The markers that appear in BACKLOG.md and the structures/ pages are a separate, and until now undefined, vocabulary. They mean:

Marker	Meaning
✅	Confirmed — verified against ground truth.
🟡	Partial: true as far as it goes, or true under a stated assumption.
❔	Open question. Nobody has answered it yet.
🔴	Refuted — shown false — or blocked by something the container cannot do.
❌	A specific claim that was tried and failed. Prefer 🔴.
🚧	Work started and not finished.

🔴 never means "we have not run it yet." That is ❔ or 🚧. Reserve 🔴's "blocked" sense for a real limit of the box — no push credentials, no hardware Vulkan (lavapipe only), or a decision only the user can make. The box can run the emulator, script input, screenshot, read guest memory, and build and test Rust, so "needs a run" is never a blocker. This paragraph exists because the marker was undefined for 98 uses and three of them were mislabelled that way.

Promotion requires new evidence, not re-reading the old evidence. A HYPOTHESIS that "looks right again" is still a HYPOTHESIS. Only an independent check promotes it. If evidence later contradicts an entry, demote it and record the contradiction — do not silently edit the conclusion.

When to document

Right after a function/structure crosses from HYPOTHESIS to at least PROBABLE — before moving to the next code path, so the knowledge isn't lost or re-derived.
Whenever confidence changes (up or down) — append to the Evidence log, don't overwrite.
Not while it's still a pure guess with no evidence — a one-liner in the relevant backlog/plan is enough until there's something to stand on.

What to document

Functions/code paths → docs/re/functions/<name>.md (one file per function or tight cluster).
Data structures / formats → docs/re/structures/<name>.md.
Keep the index in INDEX.md (one line each: name · confidence · one-line summary).

Use the templates: _TEMPLATE.function.md, _TEMPLATE.structure.md.

How we find and confirm code paths (the toolchain)

Everything joins on the guest virtual address (PC) — code addresses are fixed by the XEX load, identical across our emulator and canary.

Static (cheap, try first): sylpheed.db at the repository root (DuckDB: 25 676 functions, xrefs, strings, vtables, imports; git-ignored — build it with sylph-xexdb dis <ISO> --db sylpheed.db --analyze sql --quiet, then re-stamp names with tools/apply_re_symbols.sql). Query with tools/zq.py — zq.py grep <str>, zq.py xref <addr>, zq.py dis <lo> <hi>, zq.py fn <pc>. Entry points are usually a string (zq.py grep MSG_DEMO) or an import (movie/XMA API) xref'd back to the loader.
Dynamic (when static is ambiguous): Xenia Canary, below. The xenia-rs probe suite this bullet used to recommend (--pc-probe, --mem-watch, --lr-trace, --dump-addr) went with that emulator when it was retired on 2026-09-16. For static .rdata reads that --dump-addr served, read the .pe directly: it is a flat VA dump, file offset = VA − 0x82000000.
Oracle (correctness ground truth): canary, in either of two builds — the Wine cross-build in xenia-canary/build-cross/bin/Windows/Debug/ (launched by run-canary-safe.sh) or the native Linux build in xenia-canary-native/ (launched by run-canary-native-safe.sh). This is the only emulator that reaches the in-game menu; our retired xenia-rs never got past the intro video. Use canary to observe output (capture its framebuffer for texture colours), not usually to instrument code — though both builds compile, so small C++ probes + rebuild are possible when needed. Run muted, one emulator process at a time, point it at the real ISO.

⚠️ VA-equality caveat: join code by PC (fixed), but never assume a data VA holds the same bytes across emulators — allocators differ. Compare data by content/layout.

6.8 KiB Raw Permalink Blame History Unescape Escape