From c7a4378d375030f0828e674221254c5ccad08877 Mon Sep 17 00:00:00 2001 From: MechaCat02 Date: Sun, 13 Sep 2026 19:02:27 +0200 Subject: [PATCH 1/3] docs(agents): the plan to consolidate six repositories into two The end state the human set: Sylpheed holds the port, the corpus and every tool; the Canary fork holds the emulator and oracle. Four repos retire. Eight phases, each with a gate. What matters more than the sequence is the ledger: every absorption claim here was RUN, not reasoned. Reborn's 21 remote + 8 local branch tips -> all ancestors of Sylpheed/main Sylpheed-Godot's 2 tips + the root graft -> all ancestors agent-backups/, 23 refs across 4 bundles -> all present in Sylpheed 3 unpushed pi/* branches, by patch-id -> all 4 patches already on main issues / PRs / releases / wikis, 6 repos -> zero outside Sylpheed Three things the check found that a plan built on the obvious reading would have destroyed: 1. The F1 instrument -- Canary's --pad_file_repeat patch, the one behind the repeat rate now shipped in the port -- is UNCOMMITTED, alongside 4 unpushed commits and a branch contained in no remote. Phase 0 exists for this. 2. xenia-rs has 4 unpushed branches of JIT/multicore/native-threads work. Dropping the emulator is a decision; dropping work that was never pushed is a different act, done by accident. 3. The DuckDB tool is not on master, which last moved 2026-06-15. It is on iterate-4A/apu-xma-stage1, +22 commits, 2026-09-10. On the captures question the measurement contradicted the premise: 193 of 257 files are cited by a page or opened by a tool, and two are fixtures that cargo test opens. 64 are orphans. Phase 4 builds docs/re/ the citation check docs/port/ has had all along -- it already finds 10 dangling citations -- and fixes those BEFORE deleting orphans, so the deletion cannot ratify them. Deleting reclaims nothing anyway: the 118 MB is in history, and the build depends on this repo by tag (formats-pin-2026-09-01, pinned in Cargo.lock). That is Phase 5's explicit fork rather than a default. Co-Authored-By: Claude Opus 5 --- docs/agents/CONSOLIDATION.md | 284 +++++++++++++++++++++++++++++++++++ 1 file changed, 284 insertions(+) create mode 100644 docs/agents/CONSOLIDATION.md diff --git a/docs/agents/CONSOLIDATION.md b/docs/agents/CONSOLIDATION.md new file mode 100644 index 00000000..62a31223 --- /dev/null +++ b/docs/agents/CONSOLIDATION.md @@ -0,0 +1,284 @@ +# Consolidating six repositories into two + +**Set by the human, 2026-09-13.** The end state is **two** repositories: + +| repo | holds | +|---|---| +| **`Sylpheed`** | everything β€” the port, the RE corpus, every tool | +| **`Xenia-Canary`** (our fork) | the emulator and oracle, with our instrumentation | + +Everything else is retired. This page is the ordered **how**, with a gate after +each phase, and a ledger of what was checked rather than assumed. + +⚠️ **Most paths on this page do not resolve in this repository, on purpose.** +They name files in `xenia-rs`, `xex2tractor`, `xenia-canary` and the project +root β€” the things being moved *in*. A citation checker pointed at +`docs/agents/` will flag every one of them until the phases below are done, and +that is the correct reading: **each unresolved path here is work outstanding.** + +πŸ“Œ Two decisions from the human frame the whole plan: + +> *"The emulator is dead and can be dropped altogether. But the DuckDB generator +> should still be a useful tool… We probably can move it into a standalone tool +> under Sylpheed."* + +> *"xex2tractor was an early attempt and is long dead too… you may want to scan +> and compare the tools for anything that is in xex2tractor only."* + +--- + +## Where things stand + +| repo | server | local tree | tracked | disposition | +|---|---|---|---|---| +| **Sylpheed** | 598 MB | 33 GB | 131 MB / 1470 files | the target | +| **Xenia-Canary** | 71 MB | 20 GB | β€” | the keeper β€” ⚠️ **dirty** | +| Syplheed-Reborn | 103 MB | 1.4 GB | 100 MB / 787 files | βœ… absorbed β†’ archive | +| Sylpheed-Godot | 0.4 MB | 99 MB | 0.3 MB / 39 files | βœ… absorbed β†’ archive | +| xenia-rs | 9 MB | 1.2 GB | 61 MB | harvest the DB tool β†’ archive | +| xex2tractor | 0.3 MB | *no clone* | β€” | harvest four things β†’ archive | + +`Sylpheed` has **two root commits** β€” `feat: initialise workspace` (2026-03-25, +Reborn's) and `scaffold the Godot port` (2026-08-28). Both histories were +grafted in, not copied, which is why the absorption checks below pass. + +## The ledger β€” what was verified, and how + +Nothing here is inference. Each row is a command that was run on 2026-09-13. + +| claim | method | result | +|---|---|---| +| Reborn fully absorbed | all **21 remote + 8 local** branch tips vs `Sylpheed/main` | every one an ancestor | +| Sylpheed-Godot absorbed | both branch tips, + the root-commit graft | every one an ancestor | +| `agent-backups/` redundant | `git bundle list-heads`, 23 refs across 4 bundles | all present in Sylpheed | +| 3 unpushed `pi/*` branches redundant | `git patch-id --stable`, per commit | all 4 patches on `main` | +| nothing lives outside git | issues / PRs / releases / wikis, all six repos | **0** everywhere but Sylpheed | +| the DB tool is separable | `git grep "use xenia_cpu::"` in `xenia-analysis` | **three symbols** | + +### πŸ”΄ What the ledger found that a naive plan would have destroyed + +1. **The F1 instrument exists in one place, uncommitted.** `xenia-canary` has two + dirty files β€” `hid/file/file_input_driver.h` (`--pad_file_repeat`) and + `app/xenia_main.cc` β€” plus **4 unpushed commits** on `sylpheed-re` and a + branch, `audit-handle-lifecycle-probes`, **contained in no remote at all**. + That patch is the instrument behind `REPEAT_DELAY`/`REPEAT_INTERVAL`, now + shipped in the port. Its own finding page says *"not yet upstreamed."* +2. **xenia-rs has 4 unpushed branches** β€” `iterate-4B/ui-perf`, `4C/jit`, + `4D/parallel`, `4E/native-threads`. Dropping the emulator is a decision; + dropping work that was never backed up is a different act. +3. **The tool is not on `master`.** `xenia-rs/master` last moved **2026-06-15**; + the DuckDB and disassembler work is on `origin/iterate-4A/apu-xma-stage1` + (**2026-09-10**, +22 commits). Extract from the default branch and you get a + three-month-old tool. + +--- + +## Phase 0 β€” Secure the single-copy work + +**Before touching anything else.** One `git checkout` in `xenia-canary` ends the +reproducibility of a measurement that is already live in the port. + +1. Commit `file_input_driver.h` + `xenia_main.cc` on `sylpheed-re`. +2. Push `sylpheed-re` (4 + 1 commits) and `audit-handle-lifecycle-probes`. +3. Resolve the `third_party/DirectXShaderCompiler` submodule drift + **deliberately** β€” `HANDOFF-crash-oracle-2026-07-16.md` pins the *recorded* + pointer on purpose. + +**Gate:** `git status` clean in `xenia-canary`, and every local branch contained +in a remote: + +```bash +git for-each-ref --format='%(refname:short) %(objectname)' refs/heads | while read b s; do + git for-each-ref --format='%(objectname)' refs/remotes | grep -q "^$s$" || echo "UNPUSHED $b" +done +``` + +## Phase 1 β€” Make the archive honest + +Push xenia-rs `iterate-4B/ui-perf`, `4C/jit`, `4D/parallel`, `4E/native-threads`. + +They are being retired, not rescued. But a retired branch that was never pushed +is not retired β€” it is deleted, quietly, by someone who thought it was safe. + +**Gate:** the same loop, run in every repo, prints nothing. + +## Phase 2 β€” Harvest xex2tractor, then retire it + +Four things exist only there. + +| take | why | +|---|---| +| **XEX2 devkit + XEX1 retail master keys** | `xenia-xex` hardcodes retail only; devkit is present but `#[allow(dead_code)]` and XEX1 is absent. xex2tractor tries all three in a validating loop. | +| **`extract -r`** β€” Xenia-style thunk stubs and variable slots written into the extracted PE | πŸ”΄ **this is almost certainly how the project's `.pe` was made.** Nothing in xenia-rs writes a resolved PE. | +| **`doc/xex2_format.md`** (39 KB) and **`doc/xbox360_exports.json`** (938 KB, 2,913 exports across xboxkrnl / xam / xbdm) | byte-identical to untracked loose copies in the project root. Tracked by **no repo**. | +| **`LICENSE`** (MIT) | Sylpheed has none. | + +Leave everything else: `xenia-xex` is ahead on `resources.rs` (XDBF/XACH), +`pdata.rs`, `tls.rs`, basic zero-fill compression, and the analysis layer. + +**Gate β€” and it is the load-bearing test of this plan:** `extract -r` reproduces +the existing `.pe` **byte-for-byte**. + +> The provenance evidence: the README's example prints `9568256 bytes`; the +> project's `.pe` *is* 9,568,256 bytes with an `MZ` header, dated 18 Apr β€” three +> weeks after xex2tractor's last commit (2026-03-29). +> +> It is evidence, not proof. **If the gate fails, the inference is wrong and +> Phases 3–5 stop until it is understood** β€” because every `.pe`-offset citation +> in the corpus depends on that file being regenerable. + +## Phase 3 β€” Lift the DB tool out of the emulator + +From `origin/iterate-4A/apu-xma-stage1`, **never** from `master`. + +``` +xenia-analysis 10,658 db, vtables, jumptables, func, xref, rtti, + strings, xdbf, static_init, demangle, sql_views +disasm slice of cpu 3,681 disasm.rs + decoder.rs + opcode.rs (of 21,813) +xenia-xex 2,018 header, loader, lzx, pe, pdata, tls, resources +xenia-vfs 444 so it reads an ISO, not just a loose .xex +xenia-types 361 + ─────── + β‰ˆ17,200 of ~71,000 +``` + +The coupling is exactly three symbols β€” `decoder::decode`, `disasm::DisasmItem`, +`disasm::format`. Interpreter, JIT, scheduler, VMX, block cache, reservations: +none of it is reachable from the generator. + +`xenia-xex` declares `xenia-memory` and `xenia-types` as dependencies and has +**zero `use`** of either. Probably droppable β€” confirm by compiling, do not +assume. + +Travelling with it: `zq.py`, `tests/db_schema_golden.rs`, +`tests/disasm_goldens.rs`, `examples/decode_table_check.rs`. + +**Two corrections to make during the move, not after:** + +* the CLI help says **"SQLite database"** in two places. It is **DuckDB** + (`duckdb = { workspace = true }`). +* `zq.py`'s documented escape hatch β€” *"the engine vtable / rdata is NOT in the + DB … read it from guest memory with `xenia-rs exec --dump-addr`"* β€” **dies with + the emulator.** Replace it with Canary's dump or a direct read of the `.pe` at + `VA βˆ’ 0x82000000`, and say which, in the file. + +**Gate:** regenerate `sylpheed.db` from the ISO; diff schema and row counts +against the existing 586 MB database. `db_schema_golden` and `disasm_goldens` +green. + +## Phase 4 β€” Prune the captures, and give `docs/re/` the check `docs/port/` has + +**`docs/re/captures/` is 257 files / 117.6 MB, and it is not junk.** Measured: + +``` +193 (81.2 MB) cited by a docs/re page, or opened by a tool + 64 (36.5 MB) referenced by nothing +``` + +πŸ”΄ **Two are test fixtures and `cargo test` opens them:** + +``` +crates/sylpheed-formats/tests/ui_paint_order_disc.rs β†’ captures/title-screen-oracle.png +crates/sylpheed-formats/tests/unit_layout_disc.rs β†’ captures/stage02-live-unit-definitions-deep.txt +``` + +Sixteen more tools under `tools/port/` and `tools/re-capture/` read from there. + +**Do:** + +1. **Build `tools/re/check-capture-citations`**, the mirror of + `tools/port/check-citations`, which has only ever checked `docs/port/*.md`. + It reports two classes β€” a capture nothing cites, and a page citing a capture + that was never committed. It needs a `--selftest` that plants one of each, or + it is not a check. +2. **Fix the dangling citations first.** There are already **10**, including + whole directories (`focus-ring/`, `hud-runtime/`, `title-builds/`). The + corpus is already lossy in that direction and nothing was watching. +3. **Then drop the 64 orphans.** + +⚠️ **An orphan is not automatically waste.** It may be evidence a page *should* +have cited, and the page is the thing that is wrong. Fix the citations before +deleting, or step 3 silently ratifies every omission in step 2. + +πŸ“Œ **Going forward the class that matters is not captures-vs-code:** + +| | | +|---|---| +| **evidence** β€” a screenshot a finding points at | belongs beside the finding, as the convention says | +| **run output** β€” `.jsonl`, `.log`, `.csv` | the same class as xenia-rs's `audit-runs/`, which this plan drops | + +The two `.jsonl` mission-state dumps alone are 4.3 MB, and one is an orphan. + +## Phase 5 β€” Verify the invariant, then archive + +For each of `Syplheed-Reborn`, `Sylpheed-Godot`, `xenia-rs`, `xex2tractor`: +re-run the containment check **at that moment** β€” not from this page's ledger, +which will be days old β€” then archive read-only on Gitea, and only then delete +the local clone. + +**Archive before delete, and leave them archived.** The check proves the commits +are reachable. It does not prove nobody will want the repository boundary back. + +### ⚠️ The fork: history rewrite, or not + +Dropping files in Phase 4 **reclaims no space**. The 118 MB is in history β€” 512 +distinct blobs, 120.5 MB β€” and `git rm` leaves every byte in `.git` (590 MB +local, 598 MB on the server). Only a rewrite reclaims it, and it has a hard +blocker: + +```toml +# crates/sylpheed-export/Cargo.toml:84 +sylpheed-formats = { git = "…/Sylpheed.git", tag = "formats-pin-2026-09-01" } +``` + +**The build depends on this repo by tag**, pinned in `Cargo.lock` to `#1cd5b8b1`. +A rewrite invalidates all 8 `formats-pin-*` tags, every commit SHA, PR #23's +merge, both agents' clones, and needs a force-push through branch protection. + +* **Don't rewrite** (recommended): 118 MB on a host with 154 GB free, where + `Sylpheed/target` alone is 32 GB. The cost is not worth the blast radius. +* **Do rewrite:** then it happens **here, in Phase 5** β€” while tags are being + re-cut and agents re-cloned anyway, so the disruption is paid once instead of + twice. Re-point the `formats-pin` dependency **first**. + +This is a decision, not a default. Whichever is chosen, record it here. + +## Phase 6 β€” Adopt the orphan assets + +Tracked by nobody today, and tools or references by the two-repo rule: + +| asset | β†’ | +|---|---| +| `ppc-manual/` β€” 397 files, **including `generator/generate_manual.py`** | Sylpheed | +| `docs/ppc_instructions.{json,md}`, `docs/xbox360_exports.*`, `docs/xex2_format.md` | Sylpheed `docs/` | +| `docs/CROSS_BUILD_SETUP.md` (58 KB) | **Canary** β€” it builds Canary | +| `run-canary.sh` | Sylpheed `tools/` | +| `ai-agent-*.md` Γ—4 (56 KB) | Sylpheed `docs/`, or discard β€” human's call | +| `scratch/xbg7/extract.py` β€” clean-room XBG7 scanner | superseded by `sylpheed_formats::mesh`; keep as history or drop | +| `Sylpheed-try/…/examples/_voice_span.rs` β€” 46-line untracked probe | keep or drop | + +## Phase 7 β€” Intended drops + +Stated so that nothing is dropped by omission. + +| drop | size | why it is safe | +|---|---|---| +| `agent-backups/` | 687 MB | all 23 refs verified present in Sylpheed | +| `stock-oracle/` | 34 MB | two Canary **binaries**, rebuildable from source | +| root `canary_*.log` / `.stdout` / `.stderr` | ~15 MB | run output | +| `vkd3d-proton.cache` | 15 KB | runtime cache | +| xenia-rs `audit-runs/` | 57 MB, 41 files | already `.gitignore`d; `iterate-4A` deleted 40 of 41 | +| local `pi/clippy`, `pi/clippy-clean`, `pi/reauth3` | β€” | all 4 patches on `main`, by patch-id | +| `Sylpheed/target/` | **32 GB** | build output; the host is at 83 % | +| `texcompare/` (14 MB), `ship_render/` (64 KB) | β€” | tool output β€” **confirm no `docs/re/` page cites them first** | +| **the xenia-rs emulator** | ~54k LOC | the human's decision: superseded by Canary | + +## Still open, for the human + +* **Retention.** The `.iso` (7.6 GB), `.pe`, `.xex.json` and `sylph_extract/` + (6.2 GB) stay on disk and out of git. But `docs/re/captures/` is 118 MB of + game screenshots **inside a public repository**, about to be joined by a + 938 KB export database, and **all six repos are `private=False`**. Worth + choosing rather than inheriting. +* **The history fork** in Phase 5. +* **`ai-agent-*.md`, `scratch/`, `_voice_span.rs`** β€” keep or drop. From 524fb3684d2374b4c9994fc12b74a9903889e2e5 Mon Sep 17 00:00:00 2001 From: MechaCat02 Date: Sun, 13 Sep 2026 19:18:54 +0200 Subject: [PATCH 2/3] docs(agents): phases 0-1 done, and the Phase 2 gate corrects the page PHASE 0 -- the single-copy work is secured. Canary 8e63a9542 commits the --pad_file_repeat instrument that had been uncommitted while the numbers it produced were already shipping in the port; sylpheed-re and audit-handle-lifecycle-probes pushed; all 14 local branches reachable. Two corrections found by running it: * There is no DirectXShaderCompiler gitlink drift. `git submodule status` shows no '+'. The ' m' is NESTED submodule dirt (DXC's own SPIRV-Headers and SPIRV-Tools), which is upstream's business. * The gate itself was wrong. It compared branches to remotes BY SHA and called seven safe branches unpushed -- a branch that is an ANCESTOR of a pushed branch is already preserved. It now tests reachability, which is what it meant. A check that goes red for something unactionable is the failure this repo keeps naming, and this page shipped one. PHASE 1 -- the four retiring xenia-rs branches pushed (32/52/56/59 commits). The loop is clean everywhere except pi/clippy, pi/clippy-clean and pi/reauth3, unreachable by ancestry and redundant by patch-id: the one case where the answer lives outside the gate. PHASE 2 GATE -- run, and it corrected this page in two directions. extract (xex2tractor, no -r) IDENTICAL to the project .pe extract -r (xex2tractor) differs -- 1,903 bytes over 273 runs extract (xenia-rs, iterate-4A) IDENTICAL, and .xex.json too Right: the .pe is an xex2tractor extract. Wrong: it was made WITHOUT -r. So the dependency this gate existed to protect does not exist -- the tool Phase 3 keeps already reproduces the file. -r drops from 'must port' to optional, with a warning that it must never become the .pe, since it writes into the image and would invalidate every byte-offset citation in docs/re/. Co-Authored-By: Claude Opus 5 --- docs/agents/CONSOLIDATION.md | 76 ++++++++++++++++++++++++++++-------- 1 file changed, 59 insertions(+), 17 deletions(-) diff --git a/docs/agents/CONSOLIDATION.md b/docs/agents/CONSOLIDATION.md index 62a31223..83833cac 100644 --- a/docs/agents/CONSOLIDATION.md +++ b/docs/agents/CONSOLIDATION.md @@ -80,19 +80,31 @@ reproducibility of a measurement that is already live in the port. 1. Commit `file_input_driver.h` + `xenia_main.cc` on `sylpheed-re`. 2. Push `sylpheed-re` (4 + 1 commits) and `audit-handle-lifecycle-probes`. -3. Resolve the `third_party/DirectXShaderCompiler` submodule drift - **deliberately** β€” `HANDOFF-crash-oracle-2026-07-16.md` pins the *recorded* - pointer on purpose. +3. ~~Resolve the `third_party/DirectXShaderCompiler` submodule drift.~~ + βœ… **There is none.** `git submodule status` shows no `+`: the checked-out + commit matches the recorded one. The ` m` in `git status` is *nested* + submodule dirt β€” DXC's own `external/SPIRV-Headers` and `SPIRV-Tools` β€” which + is upstream's business and must not be committed here. -**Gate:** `git status` clean in `xenia-canary`, and every local branch contained -in a remote: +**Gate β€” and the first version of it was wrong.** It compared each local branch +to a remote **by SHA**, and reported seven safe branches as unpushed: a branch +that is an *ancestor* of a pushed branch is already preserved. A check that goes +red for something unactionable is the failure this repo keeps naming, so the +gate tests what it means β€” **reachability**: ```bash -git for-each-ref --format='%(refname:short) %(objectname)' refs/heads | while read b s; do - git for-each-ref --format='%(objectname)' refs/remotes | grep -q "^$s$" || echo "UNPUSHED $b" +git for-each-ref --format='%(refname:short)' refs/heads | while read b; do + s=$(git rev-parse "$b"); found="" + for r in $(git for-each-ref --format='%(refname:short)' refs/remotes); do + git merge-base --is-ancestor "$s" "$r" 2>/dev/null && { found=$r; break; } + done + [ -n "$found" ] || echo "πŸ”΄ $b NOT REACHABLE from any remote" done ``` +βœ… **PASSED 2026-09-13** β€” `8e63a9542` commits the instrument; `sylpheed-re` and +`audit-handle-lifecycle-probes` pushed; all **14** local branches reachable. + ## Phase 1 β€” Make the archive honest Push xenia-rs `iterate-4B/ui-perf`, `4C/jit`, `4D/parallel`, `4E/native-threads`. @@ -102,6 +114,13 @@ is not retired β€” it is deleted, quietly, by someone who thought it was safe. **Gate:** the same loop, run in every repo, prints nothing. +βœ… **PASSED 2026-09-13** β€” all four pushed (32 / 52 / 56 / 59 commits ahead of +`master`). The loop is clean in all five repos **except** `pi/clippy`, +`pi/clippy-clean` and `pi/reauth3` in Sylpheed, which are unreachable by +ancestry and redundant by *patch-id* β€” the ledger's reason, and Phase 7's +disposal. The gate is right to flag them; they are the one case where the +answer lives outside it. + ## Phase 2 β€” Harvest xex2tractor, then retire it Four things exist only there. @@ -109,23 +128,46 @@ Four things exist only there. | take | why | |---|---| | **XEX2 devkit + XEX1 retail master keys** | `xenia-xex` hardcodes retail only; devkit is present but `#[allow(dead_code)]` and XEX1 is absent. xex2tractor tries all three in a validating loop. | -| **`extract -r`** β€” Xenia-style thunk stubs and variable slots written into the extracted PE | πŸ”΄ **this is almost certainly how the project's `.pe` was made.** Nothing in xenia-rs writes a resolved PE. | +| ~~**`extract -r`**~~ | πŸ”΄ **RUN, AND THE INFERENCE WAS WRONG** β€” see the gate below. Optional, and dangerous if it is ever mistaken for the `.pe`. | | **`doc/xex2_format.md`** (39 KB) and **`doc/xbox360_exports.json`** (938 KB, 2,913 exports across xboxkrnl / xam / xbdm) | byte-identical to untracked loose copies in the project root. Tracked by **no repo**. | | **`LICENSE`** (MIT) | Sylpheed has none. | Leave everything else: `xenia-xex` is ahead on `resources.rs` (XDBF/XACH), `pdata.rs`, `tls.rs`, basic zero-fill compression, and the analysis layer. -**Gate β€” and it is the load-bearing test of this plan:** `extract -r` reproduces -the existing `.pe` **byte-for-byte**. +**Gate β€” βœ… RUN 2026-09-13, and it corrected this page in two directions.** -> The provenance evidence: the README's example prints `9568256 bytes`; the -> project's `.pe` *is* 9,568,256 bytes with an `MZ` header, dated 18 Apr β€” three -> weeks after xex2tractor's last commit (2026-03-29). -> -> It is evidence, not proof. **If the gate fails, the inference is wrong and -> Phases 3–5 stop until it is understood** β€” because every `.pe`-offset citation -> in the corpus depends on that file being regenerable. +``` +extract (xex2tractor, no -r) βœ… IDENTICAL to the project .pe +extract -r (xex2tractor) ❌ differs β€” 1,903 bytes over 273 runs +extract (xenia-rs, iterate-4A) βœ… IDENTICAL β€” and .xex.json identical too +``` + +**Right:** the `.pe` *is* an xex2tractor extract. Hash-for-hash. + +**Wrong:** it was made **without** `-r`. The project's `.pe` is the plain +decrypted-and-decompressed image, imports **unresolved** β€” and `-r`'s 1,903 +differing bytes sit in the import thunk region from `VA 0x82000600`. + +πŸ”΄ **So the dependency this gate existed to protect does not exist.** This page +claimed *"drop this and the file every `.pe`-offset citation depends on can +never be rebuilt."* It can: **`xenia-rs extract`, the tool Phase 3 keeps, +reproduces it byte-for-byte** β€” along with the loose `.xex.json`, which turns out +to have come from xenia-rs, not xex2tractor. + +⚠️ **And `-r` must never become the `.pe`.** It writes into the image, so a `.pe` +built with it would silently invalidate every byte-offset citation in +`docs/re/`. If resolved-import disassembly is ever wanted, it is a **separate +artifact under a different name**, never a replacement. + +That leaves xex2tractor's unique list at three items, none urgent β€” the two +master keys, the two doc assets, and the licence. **The doc assets are now the +only reason to visit that repository.** + +πŸ“Œ Learned in passing: `iterate-4A` **builds clean** (`xenia-app`, one unused +variable warning), so Phase 3's source is not bit-rotted β€” and it needs +`libavutil-dev` and friends via `ffmpeg-sys-next`, dragged in by the emulator's +XMA audio. The standalone tool sheds a C library the CI image did not have. ## Phase 3 β€” Lift the DB tool out of the emulator From 34d3bbc8fd9f0260986873139aea7ebd90373ea1 Mon Sep 17 00:00:00 2001 From: MechaCat02 Date: Sun, 13 Sep 2026 21:22:47 +0200 Subject: [PATCH 3/3] docs(agents): record the progress, and correct four claims this page got wrong MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Paused at six of eight phases, all committed and pushed, nothing left in a working tree. The page now carries a STATUS section the other machine can finish from: what is done, which PR each phase is in, what to run next, and what is deliberately left for the human. πŸ”΄ FOUR THINGS THIS PAGE ASSERTED WERE WRONG, and each is corrected in place rather than quietly amended: * "10 dangling citations" was ONE. Eight were DIRECTORY references, which resolve and are absent only from `git ls-tree`; one was a path ending a sentence, with the full stop pulled into the match. I reported that 10 to the human twice. * `extract -r` is not how the .pe was made and nothing depends on it. The plain extract is, and the tool Phase 3 keeps reproduces it byte-for-byte. * The Phase 0 gate compared branches by SHA and called seven safe branches unpushed -- a branch that is an ancestor of a pushed one is preserved. * There is no DirectXShaderCompiler drift; the ' m' is nested submodule dirt. The pattern is worth more than the corrections: every claim that was REASONED turned out wrong, and every one that was RUN held. Phase 2's gate refuted its own premise, Phase 3's proved two alarming row deltas were an improvement, and Phase 4's check caught a deletion I had just made. The page now says to re-run the gates rather than trust the results recorded in it. Phase 5 (archive) and Phase 7 (drops) are open and marked NOT STARTED, with the reason each needs the human: archiving is reversible and deleting a repository is not, and the drops are on the human's disk. Co-Authored-By: Claude Opus 5 --- docs/agents/CONSOLIDATION.md | 167 +++++++++++++++++++++++++++++++---- 1 file changed, 151 insertions(+), 16 deletions(-) diff --git a/docs/agents/CONSOLIDATION.md b/docs/agents/CONSOLIDATION.md index 83833cac..cb9f02af 100644 --- a/docs/agents/CONSOLIDATION.md +++ b/docs/agents/CONSOLIDATION.md @@ -73,7 +73,7 @@ Nothing here is inference. Each row is a command that was run on 2026-09-13. --- -## Phase 0 β€” Secure the single-copy work +## Phase 0 β€” Secure the single-copy work βœ… **DONE** **Before touching anything else.** One `git checkout` in `xenia-canary` ends the reproducibility of a measurement that is already live in the port. @@ -105,7 +105,7 @@ done βœ… **PASSED 2026-09-13** β€” `8e63a9542` commits the instrument; `sylpheed-re` and `audit-handle-lifecycle-probes` pushed; all **14** local branches reachable. -## Phase 1 β€” Make the archive honest +## Phase 1 β€” Make the archive honest βœ… **DONE** Push xenia-rs `iterate-4B/ui-perf`, `4C/jit`, `4D/parallel`, `4E/native-threads`. @@ -121,7 +121,7 @@ ancestry and redundant by *patch-id* β€” the ledger's reason, and Phase 7's disposal. The gate is right to flag them; they are the one case where the answer lives outside it. -## Phase 2 β€” Harvest xex2tractor, then retire it +## Phase 2 β€” Harvest xex2tractor, then retire it βœ… **DONE (PR #30)** Four things exist only there. @@ -169,7 +169,7 @@ variable warning), so Phase 3's source is not bit-rotted β€” and it needs `libavutil-dev` and friends via `ffmpeg-sys-next`, dragged in by the emulator's XMA audio. The standalone tool sheds a C library the CI image did not have. -## Phase 3 β€” Lift the DB tool out of the emulator +## Phase 3 β€” Lift the DB tool out of the emulator βœ… **DONE (PR #32)** From `origin/iterate-4A/apu-xma-stage1`, **never** from `master`. @@ -204,11 +204,15 @@ Travelling with it: `zq.py`, `tests/db_schema_golden.rs`, the emulator.** Replace it with Canary's dump or a direct read of the `.pe` at `VA βˆ’ 0x82000000`, and say which, in the file. -**Gate:** regenerate `sylpheed.db` from the ISO; diff schema and row counts -against the existing 586 MB database. `db_schema_golden` and `disasm_goldens` -green. +**Gate β€” βœ… PASSED 2026-09-13.** Schema a **superset** (22 new tables, none +lost). `instructions` 1,865,751, `pdata_entries`, `imports`, `sections` and all +three `eh_*` identical to the row; most others slightly higher. Two tables drop +~1.8 M and it is not a loss β€” the xref delta is **entirely `ind_call`** +(everything else nets **+1,512**), and the new schema records why: **6,556 sites +`truncated=true` carrying 1,801,075 candidates** against 427 untruncated +producing 4,199 rows. The old database materialised a cross product. -## Phase 4 β€” Prune the captures, and give `docs/re/` the check `docs/port/` has +## Phase 4 β€” Prune the captures, and give `docs/re/` the check `docs/port/` has βœ… **DONE (PR #33)** **`docs/re/captures/` is 257 files / 117.6 MB, and it is not junk.** Measured: @@ -228,15 +232,19 @@ Sixteen more tools under `tools/port/` and `tools/re-capture/` read from there. **Do:** -1. **Build `tools/re/check-capture-citations`**, the mirror of +1. βœ… **DONE β€” `tools/re/check-capture-citations`**, the mirror of `tools/port/check-citations`, which has only ever checked `docs/port/*.md`. It reports two classes β€” a capture nothing cites, and a page citing a capture that was never committed. It needs a `--selftest` that plants one of each, or it is not a check. -2. **Fix the dangling citations first.** There are already **10**, including - whole directories (`focus-ring/`, `hud-runtime/`, `title-builds/`). The - corpus is already lossy in that direction and nothing was watching. -3. **Then drop the 64 orphans.** +2. βœ… **DONE β€” and there was ~~**10**~~ ONE.** Eight of the ten were *directory* + references that resolve; one was sentence punctuation in the regex. The real + one, `live-submenu-unidentified.png`, had been **committed and then deleted** + on 2026-08-30 while the page kept citing it and kept making the claim it + backs. Restored, not de-linked. +3. βœ… **DONE β€” 46, not 64.** Two are opened by *filename* from + `screen_match.py`, and sixteen sit inside directories that pages cite as + directories. 30.7 MB out of the checkout. ⚠️ **An orphan is not automatically waste.** It may be evidence a page *should* have cited, and the page is the thing that is wrong. Fix the citations before @@ -251,7 +259,7 @@ deleting, or step 3 silently ratifies every omission in step 2. The two `.jsonl` mission-state dumps alone are 4.3 MB, and one is an orphan. -## Phase 5 β€” Verify the invariant, then archive +## Phase 5 β€” Verify the invariant, then archive ⬜ **NOT STARTED** For each of `Syplheed-Reborn`, `Sylpheed-Godot`, `xenia-rs`, `xex2tractor`: re-run the containment check **at that moment** β€” not from this page's ledger, @@ -285,7 +293,7 @@ merge, both agents' clones, and needs a force-push through branch protection. This is a decision, not a default. Whichever is chosen, record it here. -## Phase 6 β€” Adopt the orphan assets +## Phase 6 β€” Adopt the orphan assets βœ… **DONE (PR #34)** Tracked by nobody today, and tools or references by the two-repo rule: @@ -299,7 +307,7 @@ Tracked by nobody today, and tools or references by the two-repo rule: | `scratch/xbg7/extract.py` β€” clean-room XBG7 scanner | superseded by `sylpheed_formats::mesh`; keep as history or drop | | `Sylpheed-try/…/examples/_voice_span.rs` β€” 46-line untracked probe | keep or drop | -## Phase 7 β€” Intended drops +## Phase 7 β€” Intended drops ⬜ **NOT STARTED** Stated so that nothing is dropped by omission. @@ -315,6 +323,133 @@ Stated so that nothing is dropped by omission. | `texcompare/` (14 MB), `ship_render/` (64 KB) | β€” | tool output β€” **confirm no `docs/re/` page cites them first** | | **the xenia-rs emulator** | ~54k LOC | the human's decision: superseded by Canary | +--- + +# ▢️ STATUS 2026-09-13 β€” paused here, for the other machine to finish + +**Six of eight phases are done and pushed.** Everything below is committed; +nothing is left in a working tree. Five pull requests are open and **none has +been merged** β€” merging is the human's, and #32 contains #30 and #31. + +| phase | state | where | +|---|---|---| +| **0** secure single-copy work | βœ… done | Canary `8e63a9542`, `sylpheed-re` + `audit-handle-lifecycle-probes` pushed | +| **1** make the archive honest | βœ… done | xenia-rs `iterate-4B/4C/4D/4E` pushed (32/52/56/59 commits) | +| **2** harvest xex2tractor | βœ… done | **PR #30** `harvest/xex2tractor-assets` | +| **3** lift the DB tool | βœ… done | **PR #32** `feat/xexdb-tool` | +| **4** captures + the missing check | βœ… done | **PR #33** `fix/capture-citations` | +| **6** adopt the orphan assets | βœ… done | **PR #34** `chore/adopt-orphan-assets`, + Canary `99cc72662` | +| β€” containers and agents | βœ… done | **PR #31** `docs/containers-setup` | +| **5** archive the four repos | ⬜ **open β€” needs the human** | | +| **7** intended drops | ⬜ **open β€” partly needs the human** | | + +## πŸ”΄ Four claims on this page were wrong. Read these before trusting the rest. + +1. **"10 dangling citations" was ONE.** Eight were *directory* references, which + resolve and are simply absent from `git ls-tree`; one was a path at the end of + a sentence with the full stop pulled into the match. The figure is corrected + in Phase 4 below and in PR #33. +2. **`extract -r` is not how the `.pe` was made**, and nothing depends on it β€” + Phase 2's gate, run. +3. **The Phase 0 gate compared branches by SHA** and called seven safe branches + unpushed. It tests reachability now. +4. **There is no DXC submodule drift.** The ` m` is *nested* submodule dirt. + +## What the other machine should do next + +### Before anything: merge or decide on the open PRs + +``` +#30 harvest/xex2tractor-assets reference files + LICENSE +#31 docs/containers-setup container and agent setup +#32 feat/xexdb-tool the DuckDB tool ⚠️ CONTAINS #30 and #31 +#33 fix/capture-citations the docs/re check + prune +#34 chore/adopt-orphan-assets PPC manual + canary launcher +``` + +⚠️ **#32 is stacked.** Merge #30 and #31 first, or merge #32 and close them β€” +its `build.rs` needs #30's export table, so they cannot be independent. + +⏳ **One result outstanding:** the workspace `cargo test` count for #32. `check`, +`clippy`, `fmt` and `wasm` are all 0; the test leg was interrupted twice by +branch switches and never reported cleanly. Run it and post the number: + +```bash +docker/ci/run cargo test --workspace +``` + +### Phase 5 β€” archive, and the history fork + +**Do not trust this page's ledger when you get there.** Re-run the containment +check *at that moment* β€” days will have passed and branches may have moved: + +```bash +for repo in Syplheed-Reborn Sylpheed-Godot xenia-rs xex2tractor; do + # every ref of $repo must be an ancestor of a Sylpheed ref +done +``` + +Then **archive read-only on Gitea**. πŸ”΄ **Do not delete any repository.** Archiving +is reversible; deletion is not, and it is the human's call, separately, later. +Nothing outside git needs migrating β€” issues, PRs, releases and wikis are **zero** +in all four (verified). + +The **history-rewrite fork** is still open and still recommended AGAINST: the +118 MB is in history, and `crates/sylpheed-export/Cargo.toml` depends on this +repo **by tag** (`formats-pin-2026-09-01`, pinned in `Cargo.lock`). A rewrite +invalidates all 8 tags, every SHA, PR #23's merge and both agents' clones. + +### Phase 7 β€” the drops + +All verified, none done β€” they are deletions on the human's disk: + +``` +agent-backups/ 687 MB 23 refs verified present in Sylpheed +Sylpheed/target/ 32 GB build output; the host is at 83 % +stock-oracle/ 34 MB two Canary binaries, rebuildable +root canary_*.log 15 MB run output +xenia-rs audit-runs/ 57 MB already .gitignore'd; iterate-4A deleted 40 of 41 +pi/clippy, pi/clippy-clean, pi/reauth3 all 4 patches on main, by patch-id +texcompare/ (14 MB), ship_render/ confirm no docs/re/ page cites them first +``` + +### Still the human's, untouched + +`ai-agent-*.md` Γ—4 (56 KB), `scratch/xbg7/extract.py`, `_voice_span.rs`, and the +**retention question**: all six repos are `private=False`, and `docs/re/captures/` +is ~87 MB of game screenshots in a public repository. + +## What each finished phase actually produced + +* **Phase 2** β€” `docs/reference/{xex2-format.md, xbox360-exports.*, ppc-instructions.*}` + and an MIT `LICENSE`; this repo had none. The first two are byte-identical to + xex2tractor's, verified with `cmp`. +* **Phase 3** β€” `sylpheed-xex` + `sylpheed-ppc` + `sylpheed-xexdb`, β‰ˆ17,200 lines + of ~71,000, coupled to the emulator by **three symbols**. The gate: a + regenerated database whose schema is a superset, with `instructions`, + `pdata_entries`, `imports`, `sections` and `eh_*` identical to the row. The two + βˆ’1.8 M deltas are the `max_indirect_candidates` ceiling, recorded by the tool + as **6,556 truncated sites standing for 1,801,075 candidates** β€” checked, not + assumed: the xref delta is entirely `ind_call`. `build.rs` no longer reads a + sibling checkout, and a missing export table now **fails** instead of silently + producing a database with no import names. +* **Phase 4** β€” `tools/re/check-capture-citations`, 212 captures all cited, 46 + orphans dropped. It caught its own damage twice: two "orphans" are opened by + *filename* from `screen_match.py`, and the first prune emptied two directories + that pages cite *as directories*. +* **Phase 6** β€” `tools/ppc-manual/` (393 files) and a rewritten + `tools/run-canary.sh`; the old one pointed at a **symlink** Wine cannot resolve + and hardcoded one machine's paths. The full `CROSS_BUILD_SETUP.md` went to + Canary, where the tracked copy was a 6.8 KB stub of a 59 KB guide. + +## The method, if you only read one thing + +Every claim on this page that was *reasoned* turned out wrong, and every one that +was *run* held. The gates are the point: Phase 2's gate refuted its own premise, +Phase 3's proved the two alarming row deltas were an improvement, and Phase 4's +check caught a deletion I had just made. **Run them again rather than trusting +the results recorded here.** + ## Still open, for the human * **Retention.** The `.iso` (7.6 GB), `.pe`, `.xex.json` and `sylph_extract/`