Sylpheed/docs/agents/CONSOLIDATION.md

Consolidating six repositories into two

Set by the human, 2026-09-13. The end state is two repositories:

repo	holds
Sylpheed	everything — the port, the RE corpus, every tool
Xenia-Canary (our fork)	the emulator and oracle, with our instrumentation

Everything else is retired. This page is the ordered how, with a gate after each phase, and a ledger of what was checked rather than assumed.

⚠️ Most paths on this page do not resolve in this repository, on purpose. They name files in xenia-rs, xex2tractor, xenia-canary and the project root — the things being moved in. A citation checker pointed at docs/agents/ will flag every one of them until the phases below are done, and that is the correct reading: each unresolved path here is work outstanding.

📌 Two decisions from the human frame the whole plan:

"The emulator is dead and can be dropped altogether. But the DuckDB generator should still be a useful tool… We probably can move it into a standalone tool under Sylpheed."

"xex2tractor was an early attempt and is long dead too… you may want to scan and compare the tools for anything that is in xex2tractor only."

---

Where things stand

repo	server	local tree	tracked	disposition
Sylpheed	598 MB	33 GB	131 MB / 1470 files	the target
Xenia-Canary	71 MB	20 GB	—	the keeper — ⚠️ dirty
Syplheed-Reborn	103 MB	1.4 GB	100 MB / 787 files	✅ absorbed → archive
Sylpheed-Godot	0.4 MB	99 MB	0.3 MB / 39 files	✅ absorbed → archive
xenia-rs	9 MB	1.2 GB	61 MB	harvest the DB tool → archive
xex2tractor	0.3 MB	no clone	—	harvest four things → archive

Sylpheed has two root commits — feat: initialise workspace (2026-03-25, Reborn's) and scaffold the Godot port (2026-08-28). Both histories were grafted in, not copied, which is why the absorption checks below pass.

The ledger — what was verified, and how

Nothing here is inference. Each row is a command that was run on 2026-09-13.

claim	method	result
Reborn fully absorbed	all 21 remote + 8 local branch tips vs Sylpheed/main	every one an ancestor
Sylpheed-Godot absorbed	both branch tips, + the root-commit graft	every one an ancestor
agent-backups/ redundant	git bundle list-heads, 23 refs across 4 bundles	all present in Sylpheed
3 unpushed pi/* branches redundant	git patch-id --stable, per commit	all 4 patches on main
nothing lives outside git	issues / PRs / releases / wikis, all six repos	0 everywhere but Sylpheed
the DB tool is separable	git grep "use xenia_cpu::" in xenia-analysis	three symbols

🔴 What the ledger found that a naive plan would have destroyed

1. The F1 instrument exists in one place, uncommitted. xenia-canary has two dirty files — hid/file/file_input_driver.h (--pad_file_repeat) and app/xenia_main.cc — plus 4 unpushed commits on sylpheed-re and a branch, audit-handle-lifecycle-probes, contained in no remote at all. That patch is the instrument behind REPEAT_DELAY/REPEAT_INTERVAL, now shipped in the port. Its own finding page says "not yet upstreamed."
2. xenia-rs has 4 unpushed branches — iterate-4B/ui-perf, 4C/jit, 4D/parallel, 4E/native-threads. Dropping the emulator is a decision; dropping work that was never backed up is a different act.
3. The tool is not on master. xenia-rs/master last moved 2026-06-15; the DuckDB and disassembler work is on origin/iterate-4A/apu-xma-stage1 (2026-09-10, +22 commits). Extract from the default branch and you get a three-month-old tool.

---

Phase 0 — Secure the single-copy work

Before touching anything else. One git checkout in xenia-canary ends the reproducibility of a measurement that is already live in the port.

1. Commit file_input_driver.h + xenia_main.cc on sylpheed-re.
2. Push sylpheed-re (4 + 1 commits) and audit-handle-lifecycle-probes.
3. Resolve the third_party/DirectXShaderCompiler submodule drift. ✅ There is none. git submodule status shows no +: the checked-out commit matches the recorded one. The m in git status is nested submodule dirt — DXC's own external/SPIRV-Headers and SPIRV-Tools — which is upstream's business and must not be committed here.

Gate — and the first version of it was wrong. It compared each local branch to a remote by SHA, and reported seven safe branches as unpushed: a branch that is an ancestor of a pushed branch is already preserved. A check that goes red for something unactionable is the failure this repo keeps naming, so the gate tests what it means — reachability:

git for-each-ref --format='%(refname:short)' refs/heads | while read b; do
  s=$(git rev-parse "$b"); found=""
  for r in $(git for-each-ref --format='%(refname:short)' refs/remotes); do
    git merge-base --is-ancestor "$s" "$r" 2>/dev/null && { found=$r; break; }
  done
  [ -n "$found" ] || echo "🔴 $b NOT REACHABLE from any remote"
done

✅ PASSED 2026-09-13 — 8e63a9542 commits the instrument; sylpheed-re and audit-handle-lifecycle-probes pushed; all 14 local branches reachable.

Phase 1 — Make the archive honest

Push xenia-rs iterate-4B/ui-perf, 4C/jit, 4D/parallel, 4E/native-threads.

They are being retired, not rescued. But a retired branch that was never pushed is not retired — it is deleted, quietly, by someone who thought it was safe.

Gate: the same loop, run in every repo, prints nothing.

✅ PASSED 2026-09-13 — all four pushed (32 / 52 / 56 / 59 commits ahead of master). The loop is clean in all five repos except pi/clippy, pi/clippy-clean and pi/reauth3 in Sylpheed, which are unreachable by ancestry and redundant by patch-id — the ledger's reason, and Phase 7's disposal. The gate is right to flag them; they are the one case where the answer lives outside it.

Phase 2 — Harvest xex2tractor, then retire it

Four things exist only there.

take	why
XEX2 devkit + XEX1 retail master keys	xenia-xex hardcodes retail only; devkit is present but #[allow(dead_code)] and XEX1 is absent. xex2tractor tries all three in a validating loop.
extract -r	🔴 RUN, AND THE INFERENCE WAS WRONG — see the gate below. Optional, and dangerous if it is ever mistaken for the .pe.
doc/xex2_format.md (39 KB) and doc/xbox360_exports.json (938 KB, 2,913 exports across xboxkrnl / xam / xbdm)	byte-identical to untracked loose copies in the project root. Tracked by no repo.
LICENSE (MIT)	Sylpheed has none.

Leave everything else: xenia-xex is ahead on resources.rs (XDBF/XACH), pdata.rs, tls.rs, basic zero-fill compression, and the analysis layer.

Gate — ✅ RUN 2026-09-13, and it corrected this page in two directions.

extract      (xex2tractor, no -r)    ✅ IDENTICAL to the project .pe
extract -r   (xex2tractor)           ❌ differs — 1,903 bytes over 273 runs
extract      (xenia-rs, iterate-4A)  ✅ IDENTICAL — and .xex.json identical too

Right: the .pe is an xex2tractor extract. Hash-for-hash.

Wrong: it was made without -r. The project's .pe is the plain decrypted-and-decompressed image, imports unresolved — and -r's 1,903 differing bytes sit in the import thunk region from VA 0x82000600.

🔴 So the dependency this gate existed to protect does not exist. This page claimed "drop this and the file every .pe-offset citation depends on can never be rebuilt." It can: xenia-rs extract, the tool Phase 3 keeps, reproduces it byte-for-byte — along with the loose .xex.json, which turns out to have come from xenia-rs, not xex2tractor.

⚠️ And -r must never become the .pe. It writes into the image, so a .pe built with it would silently invalidate every byte-offset citation in docs/re/. If resolved-import disassembly is ever wanted, it is a separate artifact under a different name, never a replacement.

That leaves xex2tractor's unique list at three items, none urgent — the two master keys, the two doc assets, and the licence. The doc assets are now the only reason to visit that repository.

📌 Learned in passing: iterate-4A builds clean (xenia-app, one unused variable warning), so Phase 3's source is not bit-rotted — and it needs libavutil-dev and friends via ffmpeg-sys-next, dragged in by the emulator's XMA audio. The standalone tool sheds a C library the CI image did not have.

Phase 3 — Lift the DB tool out of the emulator

From origin/iterate-4A/apu-xma-stage1, never from master.

xenia-analysis          10,658   db, vtables, jumptables, func, xref, rtti,
                                 strings, xdbf, static_init, demangle, sql_views
disasm slice of cpu      3,681   disasm.rs + decoder.rs + opcode.rs  (of 21,813)
xenia-xex                2,018   header, loader, lzx, pe, pdata, tls, resources
xenia-vfs                  444   so it reads an ISO, not just a loose .xex
xenia-types                361
                       ───────
                       ≈17,200   of ~71,000

The coupling is exactly three symbols — decoder::decode, disasm::DisasmItem, disasm::format. Interpreter, JIT, scheduler, VMX, block cache, reservations: none of it is reachable from the generator.

xenia-xex declares xenia-memory and xenia-types as dependencies and has zero use of either. Probably droppable — confirm by compiling, do not assume.

Travelling with it: zq.py, tests/db_schema_golden.rs, tests/disasm_goldens.rs, examples/decode_table_check.rs.

Two corrections to make during the move, not after:

• the CLI help says "SQLite database" in two places. It is DuckDB (duckdb = { workspace = true }).
• zq.py's documented escape hatch — "the engine vtable / rdata is NOT in the DB … read it from guest memory with xenia-rs exec --dump-addr" — dies with the emulator. Replace it with Canary's dump or a direct read of the .pe at VA − 0x82000000, and say which, in the file.

Gate: regenerate sylpheed.db from the ISO; diff schema and row counts against the existing 586 MB database. db_schema_golden and disasm_goldens green.

Phase 4 — Prune the captures, and give docs/re/ the check docs/port/ has

docs/re/captures/ is 257 files / 117.6 MB, and it is not junk. Measured:

193  (81.2 MB)  cited by a docs/re page, or opened by a tool
 64  (36.5 MB)  referenced by nothing

🔴 Two are test fixtures and cargo test opens them:

crates/sylpheed-formats/tests/ui_paint_order_disc.rs  → captures/title-screen-oracle.png
crates/sylpheed-formats/tests/unit_layout_disc.rs     → captures/stage02-live-unit-definitions-deep.txt

Sixteen more tools under tools/port/ and tools/re-capture/ read from there.

Do:

1. Build tools/re/check-capture-citations, the mirror of tools/port/check-citations, which has only ever checked docs/port/*.md. It reports two classes — a capture nothing cites, and a page citing a capture that was never committed. It needs a --selftest that plants one of each, or it is not a check.
2. Fix the dangling citations first. There are already 10, including whole directories (focus-ring/, hud-runtime/, title-builds/). The corpus is already lossy in that direction and nothing was watching.
3. Then drop the 64 orphans.

⚠️ An orphan is not automatically waste. It may be evidence a page should have cited, and the page is the thing that is wrong. Fix the citations before deleting, or step 3 silently ratifies every omission in step 2.

📌 Going forward the class that matters is not captures-vs-code:

evidence — a screenshot a finding points at	belongs beside the finding, as the convention says
run output — .jsonl, .log, .csv	the same class as xenia-rs's audit-runs/, which this plan drops

The two .jsonl mission-state dumps alone are 4.3 MB, and one is an orphan.

Phase 5 — Verify the invariant, then archive

For each of Syplheed-Reborn, Sylpheed-Godot, xenia-rs, xex2tractor: re-run the containment check at that moment — not from this page's ledger, which will be days old — then archive read-only on Gitea, and only then delete the local clone.

Archive before delete, and leave them archived. The check proves the commits are reachable. It does not prove nobody will want the repository boundary back.

⚠️ The fork: history rewrite, or not

Dropping files in Phase 4 reclaims no space. The 118 MB is in history — 512 distinct blobs, 120.5 MB — and git rm leaves every byte in .git (590 MB local, 598 MB on the server). Only a rewrite reclaims it, and it has a hard blocker:

# crates/sylpheed-export/Cargo.toml:84
sylpheed-formats = { git = "…/Sylpheed.git", tag = "formats-pin-2026-09-01" }

The build depends on this repo by tag, pinned in Cargo.lock to #1cd5b8b1. A rewrite invalidates all 8 formats-pin-* tags, every commit SHA, PR #23's merge, both agents' clones, and needs a force-push through branch protection.

• Don't rewrite (recommended): 118 MB on a host with 154 GB free, where Sylpheed/target alone is 32 GB. The cost is not worth the blast radius.
• Do rewrite: then it happens here, in Phase 5 — while tags are being re-cut and agents re-cloned anyway, so the disruption is paid once instead of twice. Re-point the formats-pin dependency first.

This is a decision, not a default. Whichever is chosen, record it here.

Phase 6 — Adopt the orphan assets

Tracked by nobody today, and tools or references by the two-repo rule:

asset	→
ppc-manual/ — 397 files, including generator/generate_manual.py	Sylpheed
docs/ppc_instructions.{json,md}, docs/xbox360_exports.*, docs/xex2_format.md	Sylpheed docs/
docs/CROSS_BUILD_SETUP.md (58 KB)	Canary — it builds Canary
run-canary.sh	Sylpheed tools/
ai-agent-*.md ×4 (56 KB)	Sylpheed docs/, or discard — human's call
scratch/xbg7/extract.py — clean-room XBG7 scanner	superseded by sylpheed_formats::mesh; keep as history or drop
Sylpheed-try/…/examples/_voice_span.rs — 46-line untracked probe	keep or drop

Phase 7 — Intended drops

Stated so that nothing is dropped by omission.

drop	size	why it is safe
agent-backups/	687 MB	all 23 refs verified present in Sylpheed
stock-oracle/	34 MB	two Canary binaries, rebuildable from source
root canary_*.log / .stdout / .stderr	~15 MB	run output
vkd3d-proton.cache	15 KB	runtime cache
xenia-rs audit-runs/	57 MB, 41 files	already .gitignored; iterate-4A deleted 40 of 41
local pi/clippy, pi/clippy-clean, pi/reauth3	—	all 4 patches on main, by patch-id
Sylpheed/target/	32 GB	build output; the host is at 83 %
texcompare/ (14 MB), ship_render/ (64 KB)	—	tool output — confirm no docs/re/ page cites them first
the xenia-rs emulator	~54k LOC	the human's decision: superseded by Canary

Still open, for the human

• Retention. The .iso (7.6 GB), .pe, .xex.json and sylph_extract/ (6.2 GB) stay on disk and out of git. But docs/re/captures/ is 118 MB of game screenshots inside a public repository, about to be joined by a 938 KB export database, and all six repos are private=False. Worth choosing rather than inheriting.
• The history fork in Phase 5.
• ai-agent-*.md, scratch/, _voice_span.rs — keep or drop.