b7028471e9
Some checks failed
Establish docs/re/ as the spec-side of the clean-room: - README: confidence tiers (HYPOTHESIS/PROBABLE/CONFIRMED), promotion requires NEW independent evidence, append-only evidence logs, demote-on-contradiction, and a clean-room firewall (describe behaviour + cite addresses; never paste decompiled code). Documents the toolchain (sylpheed.db + zq.py, xenia-rs probes, Wine-Canary oracle) and the join-by-guest-PC method with the VA-equality caveat. - Function/structure templates + a seeded INDEX (9 already-reversed formats). - First entry: texture-color-k8888 — CONFIRMED plain k_8_8_8_8 is linear UNORM (no sRGB), channel order still to be measured. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
89 lines
4.7 KiB
Markdown
89 lines
4.7 KiB
Markdown
# Reverse-engineering knowledge base
|
|
|
|
This directory is the **spec-side** of the clean-room: it records what the original
|
|
*Project Sylpheed* binary **does** (behaviour) and how its **data is laid out**, so that
|
|
the Rust port can be implemented **from these specs** without re-deriving anything and
|
|
without ever copying original code.
|
|
|
|
It exists to answer one question fast: *"do we already know how X works, and how sure are we?"*
|
|
|
|
---
|
|
|
|
## The one rule that matters
|
|
|
|
> **Never document a claim more confidently than the evidence supports, and never
|
|
> paste original code here.**
|
|
|
|
A wrong-but-confident note is worse than no note: someone builds on it and the bug hides
|
|
for weeks. Every entry therefore carries an explicit **confidence** and its **evidence**.
|
|
This mirrors the project method — *measure the oracle, never infer; refute before believing.*
|
|
|
|
### Clean-room firewall
|
|
|
|
- ✅ Allowed: behaviour descriptions, field offsets/types, formulas, state machines,
|
|
observed input→output pairs, and **references** to the original by address
|
|
(`sub_821B68C0`) or to `xenia-rs/sylpheed.db`.
|
|
- ❌ Forbidden here and in `crates/`: pasted decompiled C/C++ or verbatim disassembled
|
|
function bodies presented as the thing to reimplement. Cite the address; describe the
|
|
behaviour in your own words. Disassembly is a tool for *understanding*, not a source to copy.
|
|
|
|
---
|
|
|
|
## Confidence levels
|
|
|
|
| Level | Meaning | Bar to reach it |
|
|
|-------|---------|-----------------|
|
|
| `CONFIRMED` | Behaviour verified against ground truth. | ≥2 independent observations **or** one observation cross-checked against an oracle (canary framebuffer, a known-correct value, a second code path). |
|
|
| `PROBABLE` | Strong single-source inference. | One clean observation, or an unambiguous static read of the disassembly. |
|
|
| `HYPOTHESIS` | Educated guess, not yet tested. | Anything else. Must say what would confirm/refute it. |
|
|
|
|
**Promotion requires new evidence, not re-reading the old evidence.** A `HYPOTHESIS` that
|
|
"looks right again" is still a `HYPOTHESIS`. Only an *independent* check promotes it.
|
|
If evidence later contradicts an entry, **demote it and record the contradiction** — do
|
|
not silently edit the conclusion.
|
|
|
|
---
|
|
|
|
## When to document
|
|
|
|
- **Right after** a function/structure crosses from `HYPOTHESIS` to at least `PROBABLE` —
|
|
before moving to the next code path, so the knowledge isn't lost or re-derived.
|
|
- **Whenever confidence changes** (up or down) — append to the Evidence log, don't overwrite.
|
|
- **Not** while it's still a pure guess with no evidence — a one-liner in the relevant
|
|
backlog/plan is enough until there's something to stand on.
|
|
|
|
## What to document
|
|
|
|
- **Functions/code paths** → `docs/re/functions/<name>.md` (one file per function or tight cluster).
|
|
- **Data structures / formats** → `docs/re/structures/<name>.md`.
|
|
- Keep the index in [`INDEX.md`](INDEX.md) (one line each: name · confidence · one-line summary).
|
|
|
|
Use the templates: [`_TEMPLATE.function.md`](_TEMPLATE.function.md),
|
|
[`_TEMPLATE.structure.md`](_TEMPLATE.structure.md).
|
|
|
|
---
|
|
|
|
## How we find and confirm code paths (the toolchain)
|
|
|
|
Everything joins on the **guest virtual address (PC)** — code addresses are fixed by the
|
|
XEX load, identical across our emulator and canary.
|
|
|
|
- **Static (cheap, try first):** `xenia-rs/sylpheed.db` (DuckDB: 25 481 functions, xrefs,
|
|
strings, vtables, imports). Query with `xenia-rs/zq.py` — `zq.py grep <str>`,
|
|
`zq.py xref <addr>`, `zq.py dis <lo> <hi>`, `zq.py fn <pc>`. Entry points are usually a
|
|
**string** (`zq.py grep MSG_DEMO`) or an **import** (movie/XMA API) xref'd back to the loader.
|
|
- **Dynamic (when static is ambiguous):** run `xenia-rs` with its probe suite —
|
|
`--pc-probe` / `--audit-pc-probe-hex` (fires at block entry), `--mem-watch` (mid-block
|
|
reads/writes of a VA), `--lr-trace` (call/return chains), `--trace-instructions`,
|
|
`--dump-addr` (read guest memory). These already exist; prefer them over hacking canary.
|
|
- **Oracle (correctness ground truth):** canary — the **Wine cross-build**
|
|
`xenia-canary/build-cross/bin/Windows/Debug/xenia_canary.exe` (the native Linux ELF
|
|
**crashes / does not run** — do not use it). This is the only emulator that reaches the
|
|
in-game menu; our `xenia-rs` never got past the intro video. Use canary to *observe output*
|
|
(capture its framebuffer for texture colours), not usually to instrument code — though its
|
|
`build-cross` toolchain does compile, so small C++ probes + rebuild are possible when needed.
|
|
Run **muted, one emulator process at a time**, point it at the real ISO (not the symlink).
|
|
|
|
> ⚠️ **VA-equality caveat:** join **code** by PC (fixed), but **never** assume a data VA
|
|
> holds the same bytes across emulators — allocators differ. Compare data by content/layout.
|