Commit Graph

1668 Commits

Author SHA1 Message Date
sim
dedcf37867 docs(ppc-manual): quote Canary and our own decoder, not the retired xenia-rs
The generator had not been able to run correctly since the manual moved into
`tools/ppc-manual/`: it computed the repository root as `HERE.parent.parent`,
which now names `tools/`, so the XML, Canary's emitters and xenia-rs all stopped
resolving — silently, because both scrapers skipped what they could not find.
Every page's references had been pointing at paths that exist nowhere.

What each source contributed, measured on the 350 pages before this change:

  Operation (pseudocode)  251 pages: fixed boilerplate "derives from the xenia-rs
                          interpreter"; 99 carry real hand-written seeds
  C translation           337 pages: the same kind of boilerplate
  xenia-rs snapshot       336 pages: the interpreter arm, pasted in — the only
                          per-instruction semantics on unseeded pages
  links                   xenia-rs opcode/decoder/interpreter + Canary emitter

Now:

  * semantics come from **Xenia Canary**, the reference emulator, read through
    `git show` at a pinned upstream commit (`origin/canary_experimental`,
    f21ebd49e9). Not our checkout: it carries instrumentation and lacked
    upstream's `mcrf` fix, so it would have published probes and a wrong `mcrf`.
    Each page embeds the emitter (`InstrEmit_<mnem>`), and for the 128 pure
    one-line delegations also the helper that holds the semantics.
  * decode references point at `crates/sylpheed-ppc` — the decoder that
    produces `sylpheed.db` — as in-repo relative links.
  * the boilerplate now says what is true, and the C translation guide maps
    Canary's actual HIR calls, checked against `ppc_hir_builder.h` (including
    that `UpdateCR(n, v)` truncates to 32 bits).
  * `rust_scraper.py` -> `decoder_scraper.py` (interpreter half dropped);
    missing sources are now errors, not empty results.

Verified:

  consistency checks        455 XML entries, 350 families, 598 index keys
  hand-written tails        386/386 byte-identical after regeneration
  xenia-rs in generated     0
  pages with a snapshot     349/350 (was 336) — `dcbi` has no Canary emitter at all
  in-repo decoder links     910/910 resolve to a line holding the identifier
  emitter boundaries        brace counter == column-0 `}` rule on 521/521;
                            preprocessor model unit-tested (#if 0/#else/#elif)
  idempotency               re-run: 0 pages updated, 0 working-tree changes

Hand-written notes (outside the generated regions) are not rewritten here:

  * 110 links into `../../xenia-rs/...` were dead; they now point at the file in
    the archived repository (git.mc02.dev/fabi/xenia-rs @ 8401d4d). Line anchors
    were dropped because the notes predate that commit — 0 of 441 old line
    ranges match it — and a precise-looking wrong anchor is worse than none. The
    link text, which carries the author's line numbers, is unchanged.
  * 140 prose claims about xenia-rs's behaviour remain. 23 are verified to hold
    for Canary too (the 32-bit CR0 truncation, OE left unimplemented); the other
    114 need checking one by one, and some invert — e.g. `divdx` notes a correct
    64-bit CR0 update in xenia-rs where Canary's `UpdateCR` truncates. Left for
    a deliberate pass rather than a blind substitution.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:34:34 +02:00
ad30352384 Merge pull request 'test(formats): invert the primitives tripwire to pin the fix (#38, option A)' (#40) from fix/38-invert-primitives-tripwire into main
Reviewed-on: #40
2026-09-16 17:02:31 +00:00
sim
26c3d304e9 test(formats): invert the primitives tripwire to pin the fix (#38, option A)
`the_derived_order_puts_primitives_last_and_that_wipes_screens` asserted that
switching primitives on still flattened at least one screen (`wiped_on > 0`),
and said to turn `include_primitives` on by default the day it stopped.
`cfcda55` made that day arrive — a keyless primitive that would hide the screen
is now forced to paint first — and the wire had been red since 2026-08-29,
invisible to CI because the runner has no corpus and the suite self-skips.

#38 chose option A: keep the default off, and pin the fix instead. The flatness
proxy stood in for "paint order is solved", and the fix's own record says that
is only partly true — "This does **not** make `include_primitives` safe by
default" (docs/re/structures/ui-prm-primitives.md). So:

  * renamed to `no_build_is_wiped_with_primitives_on` — the old name asserted
    the bug; it survives in the doc comment for anyone searching from #38;
  * `wiped_on > 0`  ->  `assert_eq!(wiped_on, 0)`, with a message saying what a
    failure now means (`forced_backdrop` regressed) and that it must not be
    "fixed" by turning primitives off in the test;
  * `include_primitives` stays `false`; no library code changes.

It is the only end-to-end check of the rule: the four tests in
`ui_forced_backdrop_disc.rs` pin its mechanics, and none composes with
primitives on.

Verified with the corpus present, including that it can fail:

  fix                        3 passed   125 builds draw a primitive, 0 wiped
  forced_backdrop disabled   FAILED     36 of 125 wiped — the new message fired
  restored                   3 passed   125 builds, 0 wiped

The control's 36 is exactly the "36 builds ... wiped by our own sort" that
`cfcda55` and `ui-forced-backdrop.md` report — the guard independently
reproduces the fix's own number, so it measures precisely what was repaired.
The control was a one-line `return false;` in a scratch copy; the working tree
was restored and `git diff` showed only this test file before committing.

Closes #38

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 19:01:13 +02:00
cb731f2e96 Merge pull request 'feat(xexdb): re-land #35 onto main — import-thunk naming, FKs, zq.py fixes, RE symbols' (#39) from feat/xexdb-import-naming into main
Reviewed-on: #39
2026-09-16 05:09:15 +00:00
f33a2ff646 Merge pull request 'feat(port): adopt the measured held-direction repeat rate (F1)' (#27) from feat/f1-held-repeat into main
Reviewed-on: #27
2026-09-15 19:47:28 +00:00
523a2aa94b Merge pull request 'chore(tools): adopt the PPC manual and a canary launcher that works anywhere' (#34) from chore/adopt-orphan-assets into main
Reviewed-on: #34
2026-09-15 19:46:48 +00:00
cea680b24d Merge pull request 'docs(agents): the gate that cannot see a working tree, and the Phase 5/7 checks re-run' (#36) from docs/consolidation-dirty-tree-finding into main
Reviewed-on: #36
2026-09-15 19:46:37 +00:00
d3011ea8e2 Merge pull request 'docs(agents): the plan to consolidate six repositories into two' (#29) from plan/consolidation into main
Reviewed-on: #29
2026-09-15 19:46:19 +00:00
bc9f6bd49b Merge pull request 'fix(re): the capture checker counted its own selftest fixture as a citation' (#37) from fix/capture-citations-selfscan into main
Reviewed-on: #37
2026-09-15 19:46:02 +00:00
840d4b8bac Merge pull request 'docs(re): give docs/re/ the citation check docs/port/ has always had, and prune' (#33) from fix/capture-citations into main
Reviewed-on: #33
2026-09-15 19:44:48 +00:00
04e8d6244c Merge pull request 'feat(xexdb): lift the DuckDB generator and disassembler out of the retired emulator' (#32) from feat/xexdb-tool into main
Reviewed-on: #32
2026-09-15 19:15:22 +00:00
5a77229bbb Merge pull request 'docs(agents): how the containers and agents are actually set up' (#31) from docs/containers-setup into main
Reviewed-on: #31
2026-09-15 19:15:12 +00:00
9d41971646 Merge pull request 'docs(reference): adopt the four homeless reference files, and a LICENSE' (#30) from harvest/xex2tractor-assets into main
Reviewed-on: #30
2026-09-15 19:15:02 +00:00
sim
0679abb9f9 harvest: RE_SYMBOLS.md and apply_re_symbols.sql, which Phase 3 left behind
Phase 3 lifted the DuckDB tool out of `xenia-rs` but not the two files that go
with it, and this branch's own code already depends on one of them: the FK rule
in `db.rs` and `db_schema_golden.rs` exists *because* `apply_re_symbols.sql`
re-stamps reverse-engineered names onto `functions` after every regeneration.
Sylpheed referenced that file three times and contained it nowhere — on no
branch. CONSOLIDATION.md's end state is "Sylpheed holds everything", and the
`xenia-rs` clone is scheduled for deletion.

  docs/re/RE_SYMBOLS.md       byte-identical to xenia-rs/RE_SYMBOLS.md (cmp)
  tools/apply_re_symbols.sql  every statement identical; the two comment lines
                              that name paths now name this repo's paths

Verified against the real 336 MB database, which carries all 16 foreign keys,
inside a transaction that was rolled back (the database is unchanged):
every statement runs, and 208 functions carry reverse-engineered names. Run
from both the old location and this one.

Not harvested: `xenia-rs/zq_dis.py`, an untracked six-line subset of
`zq.py dis` with a hardcoded path — superseded, nothing to keep.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 19:23:07 +02:00
sim
dce085277a chore: ignore /sylpheed.db — zq.py now defaults to it
The previous commit on this branch made `tools/zq.py` look for its database at
`<repo root>/sylpheed.db`, and its refusal message tells people to put one there.
Nothing ignored that path. The database is a build artefact of several hundred
MB, so a single `git add -A` would have committed it into a public repository's
history — the class of blob CONSOLIDATION.md's history-rewrite fork exists to
argue about.

Anchored (`/sylpheed.db`, plus DuckDB's `.wal`) so it covers only the default
location, not any file of that name elsewhere. Verified by creating the files:
both ignored, `git status` silent, and a `sylpheed.db` in a subdirectory is not
swallowed by the rule.

Found while writing the Phase 7 steps: moving the 336 MB database out of
`xenia-rs` before that clone is deleted is exactly how someone would put it here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 19:20:38 +02:00
fdf498c684 docs(agents): the #32 workspace test count is in — 367 passed, and the one failure is main's
Closes the page's one explicitly outstanding result. Run in sylph-ci:local
(rustc 1.98.1, byte-identical to the runner) with the disc present and
--no-fail-fast: 45 suites, 367 passed, 1 failed, 14 ignored. The +9 against the
disc-less #35 run (377) is exactly #35's nine imports.rs unit tests.

The single failure is ui_prm_primitives_disc, red on main since #23 and
bisected there, not introduced by #32.

Records the three things that stopped the number being obtained, as rules:
--no-fail-fast (an earlier run silently covered half the workspace), a memory
cap sized to the host (the default 7 GB OOM-kills slb_leading_segment_disc on
this machine with the disc mounted), and a ~36 GB disk budget for a cold target
volume.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 08:13:02 +02:00
108378fe21 docs(agents): the formats-pin tag count is 7, and the disk figures have moved
Re-checked against the server rather than the page. `Sylpheed` has 7 tags and
all 7 are `formats-pin-*`. The eighth this page counts, `formats-pin-2026-08-29`
without a letter suffix, is in `Syplheed-Reborn` — a repo a Sylpheed rewrite
would not touch, and one Phase 5 archives read-only regardless.

The pin itself is exactly as claimed: `formats-pin-2026-09-01` resolves to
`e2630413`, and the in-container build fetches that git dependency over the
network for real, so the blocker is live rather than theoretical.

Also updates the disk figures the "don't rewrite" recommendation rests on: 90%
used and 96 GB free, not 83% and 154 GB, with `target/` at 34 GB. The
recommendation is unchanged and the argument is now stronger — reclaiming
118 MB by rewriting history while 34 GB of rebuildable output sits beside it is
the wrong lever.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 21:25:53 +02:00
d9ba2c73b1 fix(re): the capture checker counted its own selftest fixture as a citation
`check-capture-citations` exits 1 on a clean checkout of this branch, and would
have kept doing so for ever:

    🔴 cited but NOT committed: 1
        docs/re/captures/no-such-file-anywhere.png

`SEARCH` includes `tools`, so the scan reads this file -- and `selftest()`'s
planted fixture is a literal capture path. The checker cited its own test data,
to a file that will never exist by construction. A gate that is red before
anyone changes anything teaches people to ignore it, which is the opposite of
what Phase 4 built it for.

Excluding only this one file (`SELF`) keeps every other tool in scope. A real
citation belongs in a page, or in a tool that opens the capture -- never in the
checker itself.

Added a selftest case for it, because this is exactly the class of bug the rest
of this file already documents surviving: the ERE that matched nothing while
five ticks stayed green. Verified the guard can fail, rather than assuming it:

    exclusion neutralised -> "own fixtures not counted 🔴 FAILED",
                             selftest rc=2, real check rc=1 reproducing the bug
    exclusion restored    -> selftest ok (7 cases), check rc=0

After the fix, on this branch:

    captures committed          : 212
      cited by a page or a tool : 212
      cited by nothing          : 0
      🔴 cited but NOT committed: 0        rc=0

which is also the first independent confirmation of Phase 4's "212 captures,
all cited" -- the checker now agrees with the page.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 21:18:55 +02:00
d0f3885fb5 docs(agents): re-run the Phase 5 and Phase 7 checks, and record what they say
This page says not to trust its own ledger on arrival, so the checks were run
again rather than read. Nothing was archived and nothing was deleted — both are
the human's, and both phases stay open.

The substantive finding is that Phase 5's containment check is the wrong test
for half the list it names. `xenia-rs` and `xex2tractor` were never absorbed;
their disposition is harvest-then-archive. Run the check literally against
`xenia-rs` and it returns 42 of 42 refs "not present in Sylpheed" — every line
true, every line unactionable, which is the failure this page already names
once about Phase 0's first gate. For those two the invariant is that the
harvested artefacts are in Sylpheed and the repository is archived rather than
deleted; that is now checked separately, and it shows the harvest is real but
lives only on the unmerged PR branches, so Phase 5 should follow them rather
than precede them.

Containment itself holds for the two repos it does apply to: 21/21 and 2/2.

Also answers Phase 7's one open question — nothing cites `texcompare/`, and all
four `ship_render` citations name the example program, not the output
directory — and records that most of Phase 7's list does not exist on this
machine, that `has_wiki: true` is a flag rather than content (all six wikis
return 404, never initialised), and that the disk is at 90%, not the 83% the
page records.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 21:00:40 +02:00
589c403c62 docs(agents): the zq.py database path is fixed, not outstanding
This page recorded it as "not changed here, because the right replacement is a
decision". The decision was taken: PR #35 `38cc170` resolves `$SYLPH_XEXDB`,
then `<repo root>/sylpheed.db`, then refuses with both named — no third
fallback, because the database is an untracked build artefact and a default
that silently resolves to the wrong one is worse than no default.

Leaving the page saying otherwise would be the defect this section is about.

`SCHEMA.md`'s retired-repo heading is still outstanding and stays listed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 20:50:44 +02:00
5f623bf110 fix(zq): stop defaulting the database path into the repo being deleted
`zq.py` hardcoded one absolute path, and it pointed inside `xenia-rs` --
a repository CONSOLIDATION.md archives in Phase 5 and drops in Phase 7. It
resolved on exactly one machine, and on that machine it was days from becoming
a `duckdb` exception about a missing file, with nothing saying why.

Resolution order is now `$SYLPH_XEXDB`, else `<repo root>/sylpheed.db`, else a
refusal that names both. The repo-root path is taken relative to this script,
not the caller's cwd, because zq.py is run from wherever the investigation is.

No fallback beyond that, on purpose. The database is a build artefact of a few
hundred MB and is not tracked, so there is nothing to fall back *to*; a default
that silently resolves to the wrong database is worse than no default. That is
issue #16's lesson, which was about exactly this shape in the test suite.

Both failure paths say what to do and exit 1:

  $ zq.py fn 82000000            # nothing set, nothing at the repo root
  no database found.
    looked for: /…/Sylpheed/sylpheed.db
    set $SYLPH_XEXDB to an existing one, or build it with:
      sylph-xexdb dis <xex|iso> --db sylpheed.db --analyze sql

  $ SYLPH_XEXDB=/nope/missing.db zq.py fn 82000000
  $SYLPH_XEXDB is set but is not a file:
    /nope/missing.db

Also fixes the regeneration hint, which still named `xenia-rs dis` -- the
retired binary. It is `sylph-xexdb dis`.

Connecting is skipped when there is no subcommand, so `zq.py` still prints its
usage on a machine that has no database yet.

Verified: usage with no db (rc 0); missing db (rc 1); `$SYLPH_XEXDB` set to a
non-file (rc 1); and `$SYLPH_XEXDB` pointed at a real 336 MB database, where
`imp Rtl` returns its 1,280 `RtlLeaveCriticalSection` call sites. That database
turns out to have been generated from the uncommitted tree this branch ports:
it carries `import_address`, `import_role` and all 16 foreign keys -- an
independent confirmation of the schema the golden test now locks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 20:49:49 +02:00
b2f089e7d3 docs(agents): the Phase 0 gate cannot see a working tree — it missed 468 lines
Found 2026-09-14 while pulling the repos on the other machine: `xenia-rs` had
468 uncommitted lines plus an untracked 338-line `imports.rs` in the very crate
Phase 3 lifted, written 2026-09-10 and sitting there for four days. Verified
absent from `iterate-4A` and from the lifted crate in PR #32 — it existed in
exactly one place, and that place was not git.

Phase 0 is titled "Secure the single-copy work" and its gate passed anyway,
because the gate iterates `refs/heads` and a dirty working tree has no ref.
Canary's two dirty files were secured only because a human already knew about
them; nothing found them. Phase 1 then declared the archive honest while this
sat beside the branches it pushed. Phase 5 ends with "delete the local clone"
and Phase 7 drops the emulator, so a deletion was scheduled against it.

Records: the finding and the `git status --porcelain` half the gate was
missing (run across all four repos — only Canary is dirty, and both entries are
already on Phase 7's drop list); that it is secured on `harvest/import-thunk-
naming` and ported in `feat/xexdb-import-naming`; a fifth entry in the
wrong-claims list; and a `0b` row in the status table.

Also records three pieces of Phase 3 residue found while porting, the sharpest
being that `tools/zq.py` still defaults its database path to a location inside
`xenia-rs` — the repository Phase 5 archives and Phase 7 drops. Left unchanged:
the right replacement is a decision, not a guess.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 20:44:48 +02:00
13c895abff feat(xexdb): port import-thunk naming + schema-wide foreign keys
Carries `xenia-rs` `harvest/import-thunk-naming` (b4f19f1) across into the
lifted crate. That work was found UNCOMMITTED in the retired repo's working
tree on 2026-09-14 and exists nowhere else: absent from `iterate-4A`, absent
from this crate as lifted. CONSOLIDATION.md Phase 5 ends with "delete the local
clone" and Phase 7 drops the emulator, so it had a deletion scheduled against
it.

Not a cherry-pick. The lift's base is exactly the harvest's parent (8401d4d),
so each file was merged three-way -- lifted vs base vs harvest -- which is what
made the port reviewable: 10 conflicts, 9 of them pure rustfmt reflow from the
lift's formatting pass, and 1 semantic.

The semantic one is insertion ORDER. `xdbf_achievements.image_id` is now a
foreign key onto `xdbf_images(id)`, so the image rows must be inserted before
the achievement rows. The merge moved that block; the conflict was the stale
copy left at the old position.

What arrives:

* **Import-thunk recognition** (`imports.rs`, 338 lines). An XEX import is not
  a PLT jump: the linker emits a four-word thunk whose first two words are
  import RECORDS that the loader rewrites at module load. On disc they are
  still records, so a PowerPC-only decoder prints two meaningless `.long`s in
  front of an indirect branch. This maps every word of every thunk, and every
  direct branch into one, back to its `imports` row. Shape-validated rather
  than trusted: an entry is indexed only when the four words it points at
  actually have the thunk shape.

  Adds `instructions.import_address` (FK onto `imports.address`) and
  `import_role` (`'record'` | `'thunk'` | `'call'`, NULL iff import_address is
  NULL), plus an index. `tools/zq.py` gains `imp` and `impcalls`, and `dis`
  now names imports instead of printing `.long 0x01010194`.

* **Sixteen schema-wide foreign keys**, declared wherever a column is derived
  from another table. CREATE TABLE and insertion order become load-bearing.

  `functions` is deliberately NOT an FK parent and the golden test now asserts
  zero inbound FKs onto it: DuckDB implements UPDATE as delete+insert, so one
  inbound FK would make `functions.name` un-updatable and break
  `apply_re_symbols.sql`, which re-applies RE symbol names after every
  regeneration.

The database path needed no change in the binary: `DbWriter` builds its own
index inside `ingest_instructions` from `info.import_libraries`. Only the two
output paths (`enrich_section` for JSONL, `write_asm`) take it as an argument.

Verified: `cargo test -p sylpheed-xexdb` = 10 passed / 0 failed, including
`db_schema_golden` (41s, builds a real DuckDB) which locks the 16-FK set and
the no-FK-onto-functions rule. `cargo clippy -p sylpheed-xexdb --all-targets
-- -D warnings` clean; `cargo fmt --all --check` clean.

Stacked on #32 -- it ports into a crate that only exists there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 20:43:16 +02:00
MechaCat02
34d3bbc8fd docs(agents): record the progress, and correct four claims this page got wrong
Paused at six of eight phases, all committed and pushed, nothing left in a
working tree. The page now carries a STATUS section the other machine can
finish from: what is done, which PR each phase is in, what to run next, and
what is deliberately left for the human.

🔴 FOUR THINGS THIS PAGE ASSERTED WERE WRONG, and each is corrected in place
rather than quietly amended:

  * "10 dangling citations" was ONE. Eight were DIRECTORY references, which
    resolve and are absent only from `git ls-tree`; one was a path ending a
    sentence, with the full stop pulled into the match. I reported that 10 to
    the human twice.
  * `extract -r` is not how the .pe was made and nothing depends on it. The
    plain extract is, and the tool Phase 3 keeps reproduces it byte-for-byte.
  * The Phase 0 gate compared branches by SHA and called seven safe branches
    unpushed -- a branch that is an ancestor of a pushed one is preserved.
  * There is no DirectXShaderCompiler drift; the ' m' is nested submodule dirt.

The pattern is worth more than the corrections: every claim that was REASONED
turned out wrong, and every one that was RUN held. Phase 2's gate refuted its
own premise, Phase 3's proved two alarming row deltas were an improvement, and
Phase 4's check caught a deletion I had just made. The page now says to re-run
the gates rather than trust the results recorded in it.

Phase 5 (archive) and Phase 7 (drops) are open and marked NOT STARTED, with
the reason each needs the human: archiving is reversible and deleting a
repository is not, and the drops are on the human's disk.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 21:22:47 +02:00
MechaCat02
10dc260f0c chore(tools): adopt the PPC manual and a canary launcher that works anywhere
CONSOLIDATION.md Phase 6. Both lived untracked in the project root -- on one
disk, backed up by nothing.

  tools/ppc-manual/   393 files, 3.7 MB. 455 instructions, 350 family pages,
                      598 mnemonics resolvable through index.json, plus the
                      generator that produced them.
  tools/run-canary.sh the oracle launcher.

🔴 THE LAUNCHER WAS BROKEN IN TWO WAYS AND IS REWRITTEN, not copied:

  * it pointed at `xenia-rs/sylpheed.iso`, a SYMLINK. Wine cannot resolve one
    and says "path invalid", which reads as a corrupt image rather than a path
    problem -- it has cost a session before. It now points at the real file and
    warns if handed a symlink.
  * it hardcoded one machine's absolute paths, and named `xenia-rs`, which this
    consolidation retires. Now derived from the script's own location, with
    SYLPH_CANARY_BIN / SYLPH_ISO overrides and a check that each exists.

The standing constraints are in its header where someone will read them: one
emulator at a time, Canary runs MUTED, and never judge a crash or a hang from
a Bash-launched run -- a SIGKILL that looked like the binary was the editor's
process supervisor.

⚠️ The manual's GENERATOR reads the xenia-rs source tree, which is going away.
Its decoder now lives here as crates/sylpheed-ppc, so the generator must be
repointed before it is run again. Recorded in the README rather than left for
someone to discover; the manual's content is checked in and regenerates from
nothing implicitly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 21:18:00 +02:00
MechaCat02
6d4fe37110 docs(re): give docs/re/ the citation check docs/port/ has always had, and prune
CONSOLIDATION.md Phase 4. `tools/port/check-citations` has checked
`docs/port/*.md` since it was written; `docs/re/` never had one, and
`docs/re/captures/` is the largest thing in the repository -- the one place a
file can be added, never cited, and never noticed.

  captures committed          : 212
    cited by a page or a tool : 212
    cited but NOT committed   : 0

🔴 THE PREMISE I STARTED FROM WAS WRONG FOUR TIMES, AND EACH CORRECTION IS IN
THE TOOL RATHER THAN JUST IN MY HEAD:

  * "10 dangling citations" -- the real number was ONE. Eight were DIRECTORY
    references, which resolve and are simply absent from `git ls-tree`; one was
    a path at the end of a sentence with the full stop pulled into the match.
    A checker that cries wolf nine times in ten teaches you to ignore it.
  * The one real dangler, `live-submenu-unidentified.png`, was COMMITTED and
    then deleted on 2026-08-30 while the page kept citing it and kept making
    the claim it backs. Restored from 77e54e95 rather than dropping the link:
    deleting evidence while keeping the conclusion is the thing this corpus
    exists to prevent.
  * Two "orphans" are opened BY FILENAME from `screen_match.py`, which builds
    the directory separately. A path-only scan called them unreferenced and
    deleting them would have broken the tool. The check now counts any
    basename named anywhere.
  * Pruning then EMPTIED two directories that pages cite as directories, and
    the check went red on the very citations that made them evidence. It
    caught its own damage. A file inside a cited directory is cited.

The selftest covers all of it, including the failure that actually happened:
an earlier version passed five green ticks while the scan silently returned
NOTHING, because `git grep -E` is POSIX ERE and cannot compile `(?:`. A
selftest that cannot see the failure that occurred is decoration.

46 orphaned captures dropped, 30.7 MB from the checkout. ⚠️ That reclaims no
repository space -- the blobs stay in history -- and it is not meant to. It
means every capture here is now evidence some page or tool actually uses.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 21:14:52 +02:00
MechaCat02
a9291149bd test(ppc): identity_op off in the golden generator, with the reason
The shifts clippy objects to are the point. These are instruction ENCODERS:
every field is written at its ISA position, so `(((vb >> 5) & 0x3) << 0)`
sits in a column with << 11, << 16, << 21 and reads straight against the
manual's field table, and a trailing `| 0` says 'the low field is zero
here', which is information.

Reduced to what clippy wants, the encodings stop being checkable by eye
against the ISA -- and eye-checkability is all a golden GENERATOR has. There
is nothing else to verify it against.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 21:08:41 +02:00
MechaCat02
d8ce791852 docs(agents): how the containers and agents are actually set up
Written for the other machine, which asked. Everything in it was read off the
running host rather than remembered.

Also adds docker/ci/ -- the CI image recipe and a capped runner -- because the
image existed on exactly one host and its Dockerfile was in a scratch directory
under /tmp, which was swept. That is the same shape as every other thing this
consolidation has turned up: something correct that exists in one place.

  docker/ci/Dockerfile          rust 1.98.1 + the apt list copied from ci.yml
  docker/ci/Dockerfile.ffmpeg   + ffmpeg, which sylpheed-export shells out to
  docker/ci/run                 6 CPUs / 7 GB / NO SWAP, named cargo volumes

The rule the runner exists to enforce: every heavy command goes in the capped
container. CARGO_BUILD_JOBS caps codegen units, not rustc's threads, not the
linker, not the test harness -- a bare host build is unbounded and has frozen
this box repeatedly.

The document also records the two agent defects the other machine will meet:
a brief change does not reach a RESUMED session, and nothing brought an agent
back to its own red PR (fixed in #24, which is itself subject to the first).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 20:43:55 +02:00
MechaCat02
1de202d88d test(ppc): bring the disassembler's golden fixtures across
Three tests failed on the extracted tree for a reason that was not a port
defect: disasm_goldens.rs resolves its fixtures through a relative path to
xenia-cpu/tests/golden/, and I had copied the test without the data.

The fixtures (base + extended mnemonics, VMX128 registers) and the generator
test now live in sylpheed-ppc, and the shim test in sylpheed-xexdb points at
them. serde/serde_json become dev-dependencies of sylpheed-ppc, which the
fixtures need and the library does not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 20:43:48 +02:00
MechaCat02
b596f1fc71 docs(agents): how the containers and agents are actually set up
Written for the other machine, which asked. Everything in it was read off the
running host rather than remembered.

Also adds docker/ci/ -- the CI image recipe and a capped runner -- because the
image existed on exactly one host and its Dockerfile was in a scratch directory
under /tmp, which was swept. That is the same shape as every other thing this
consolidation has turned up: something correct that exists in one place.

  docker/ci/Dockerfile          rust 1.98.1 + the apt list copied from ci.yml
  docker/ci/Dockerfile.ffmpeg   + ffmpeg, which sylpheed-export shells out to
  docker/ci/run                 6 CPUs / 7 GB / NO SWAP, named cargo volumes

The rule the runner exists to enforce: every heavy command goes in the capped
container. CARGO_BUILD_JOBS caps codegen units, not rustc's threads, not the
linker, not the test harness -- a bare host build is unbounded and has frozen
this box repeatedly.

The document also records the two agent defects the other machine will meet:
a brief change does not reach a RESUMED session, and nothing brought an agent
back to its own red PR (fixed in #24, which is itself subject to the first).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 20:42:56 +02:00
MechaCat02
c9dd2cb705 fix(xexdb): clear the lint gate on the imported crates
rustfmt, then clippy -D warnings across the three new crates. Mechanical,
except three decisions that are stated rather than silently allowed:

  * lzx.rs gets file-scoped needless_range_loop/explicit_counter_loop allows.
    Index arithmetic IS the algorithm -- LZX is defined over symbol indices,
    Huffman slots and window positions, and a decompressor that is merely
    idiomatic is worth nothing if it is not bit-exact.
  * sylpheed-xexdb gets crate-scoped allows for needless_range_loop (nine
    sites index reg[r] where r is the PowerPC register number -- the index is
    the meaning), too_many_arguments and type_complexity. This code arrived
    whole from a retired repository; a refactor here would be an unreviewed
    edit dressed as a lint fix.
  * Everything else clippy asked for is FIXED, including all 14 doc-indent
    sites, the let-else, and a Prepared type alias in the binary.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 20:25:44 +02:00
MechaCat02
62b0f79590 wip: tool README, DuckDB wording, zq.py escape hatch 2026-09-13 19:42:26 +02:00
MechaCat02
8738e36374 Merge branch 'harvest/xex2tractor-assets' into feat/xexdb-tool 2026-09-13 19:31:49 +02:00
MechaCat02
86da451bfe wip: extract the xexdb tool closure 2026-09-13 19:31:49 +02:00
MechaCat02
444776ae1a docs(reference): adopt the four homeless reference files, and a LICENSE
CONSOLIDATION.md Phase 2. Each of these existed only in a repository about to
be retired, or as an untracked file on one disk -- which, for anything that
has to survive a machine, is the same as not existing.

  xex2-format.md            39 KB   the XEX2 container format
  xbox360-exports.{json,md} 1.1 MB  2,913 exports: xboxkrnl, xam, xbdm
  ppc-instructions.{json,md} 340 KB the PowerPC instruction reference
  LICENSE                            MIT -- this repo had none

The first two are byte-identical to xex2tractor's copies, verified with cmp,
and byte-identical again to the loose untracked copies in the project root:
three copies, zero of them tracked.

docs/reference/ is deliberately its own directory with a README that says
what it is NOT: nothing here is a finding, and no docs/re/ page should cite it
as evidence. Reference material filed beside measurements is how a borrowed
table starts getting read as a result.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 19:20:30 +02:00
MechaCat02
524fb3684d docs(agents): phases 0-1 done, and the Phase 2 gate corrects the page
PHASE 0 -- the single-copy work is secured. Canary 8e63a9542 commits the
--pad_file_repeat instrument that had been uncommitted while the numbers it
produced were already shipping in the port; sylpheed-re and
audit-handle-lifecycle-probes pushed; all 14 local branches reachable.

Two corrections found by running it:

  * There is no DirectXShaderCompiler gitlink drift. `git submodule status`
    shows no '+'. The ' m' is NESTED submodule dirt (DXC's own SPIRV-Headers
    and SPIRV-Tools), which is upstream's business.
  * The gate itself was wrong. It compared branches to remotes BY SHA and
    called seven safe branches unpushed -- a branch that is an ANCESTOR of a
    pushed branch is already preserved. It now tests reachability, which is
    what it meant. A check that goes red for something unactionable is the
    failure this repo keeps naming, and this page shipped one.

PHASE 1 -- the four retiring xenia-rs branches pushed (32/52/56/59 commits).
The loop is clean everywhere except pi/clippy, pi/clippy-clean and pi/reauth3,
unreachable by ancestry and redundant by patch-id: the one case where the
answer lives outside the gate.

PHASE 2 GATE -- run, and it corrected this page in two directions.

  extract      (xex2tractor, no -r)    IDENTICAL to the project .pe
  extract -r   (xex2tractor)           differs -- 1,903 bytes over 273 runs
  extract      (xenia-rs, iterate-4A)  IDENTICAL, and .xex.json too

Right: the .pe is an xex2tractor extract. Wrong: it was made WITHOUT -r. So
the dependency this gate existed to protect does not exist -- the tool Phase 3
keeps already reproduces the file. -r drops from 'must port' to optional, with
a warning that it must never become the .pe, since it writes into the image
and would invalidate every byte-offset citation in docs/re/.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 19:18:54 +02:00
MechaCat02
c7a4378d37 docs(agents): the plan to consolidate six repositories into two
The end state the human set: Sylpheed holds the port, the corpus and every
tool; the Canary fork holds the emulator and oracle. Four repos retire.

Eight phases, each with a gate. What matters more than the sequence is the
ledger: every absorption claim here was RUN, not reasoned.

  Reborn's 21 remote + 8 local branch tips  -> all ancestors of Sylpheed/main
  Sylpheed-Godot's 2 tips + the root graft  -> all ancestors
  agent-backups/, 23 refs across 4 bundles  -> all present in Sylpheed
  3 unpushed pi/* branches, by patch-id     -> all 4 patches already on main
  issues / PRs / releases / wikis, 6 repos  -> zero outside Sylpheed

Three things the check found that a plan built on the obvious reading would
have destroyed:

1. The F1 instrument -- Canary's --pad_file_repeat patch, the one behind the
   repeat rate now shipped in the port -- is UNCOMMITTED, alongside 4 unpushed
   commits and a branch contained in no remote. Phase 0 exists for this.
2. xenia-rs has 4 unpushed branches of JIT/multicore/native-threads work.
   Dropping the emulator is a decision; dropping work that was never pushed is
   a different act, done by accident.
3. The DuckDB tool is not on master, which last moved 2026-06-15. It is on
   iterate-4A/apu-xma-stage1, +22 commits, 2026-09-10.

On the captures question the measurement contradicted the premise: 193 of 257
files are cited by a page or opened by a tool, and two are fixtures that
cargo test opens. 64 are orphans. Phase 4 builds docs/re/ the citation check
docs/port/ has had all along -- it already finds 10 dangling citations -- and
fixes those BEFORE deleting orphans, so the deletion cannot ratify them.

Deleting reclaims nothing anyway: the 118 MB is in history, and the build
depends on this repo by tag (formats-pin-2026-09-01, pinned in Cargo.lock).
That is Phase 5's explicit fork rather than a default.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 19:02:27 +02:00
MechaCat02
b6e206f1bc fix(port): held_direction() polls the stick instead of reconstructing it
Reported by a human on a real controller: hold the left stick down, the
cursor moves one item and stops. The repeat never runs.

`held_direction()`'s own comment says "Polled at the DEVICE, never through
Input.is_action_pressed". It was not. The d-pad and keyboard branches polled;
the STICK branch read `_latched`, which is a reconstruction of the stick's
position from the event history.

That reconstruction is only as good as the last event seen. A stick held
still sends nothing, and one event reading below RELEASE -- a spring
settling, a deadzone-shaped value, a driver emitting a zero on focus change
-- clears it with no event afterwards to set it back. The port then believes
the stick is centred while the player is holding it, which is precisely the
symptom reported.

Now polls `Input.get_joy_axis()` against the game's own 0.61, which is what
the comment always meant. The latch stays as a fallback for INJECTED events,
so the script harness and verify-input keep testing something.

🔴 Every instrument here missed this because every instrument SUPPLIES the
input it measures: verify-input ticks repeat_due() directly, --script sends
InputEventAction which bypasses the input map, and the new --script=hold:
injects its own axis event. All three agreed with each other and none read a
device. Same shape as the 2026-09-01 report that opened gamepad.gd, one
level deeper, with the lesson already written at the top of that file.

So this adds the two things that would have caught it:

  --script=hold:down:2.0   hold one real axis deflection and log every move
  --input-probe            print what the devices report, on change

⚠️ The fix itself is NOT verified. It matches the symptom exactly and was
found by reading, but only a human holding a stick can confirm it, and the
probe exists so the answer is measured either way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 14:27:09 +02:00
MechaCat02
a88e273c40 feat(port): adopt the measured held-direction repeat rate (F1)
REPEAT_DELAY = 0.402, REPEAT_INTERVAL = 0.134, from
docs/re/f1-repeat-measured-via-driver-patch.md -- 12 and 4 frames at the
run's achieved 29.87 fps guest rate, converted to seconds because this port
does not run at the guest's rate and it is the cadence that was measured.

The mechanism has been here since 2026-09-02 and inert on purpose. The
instruction it was waiting on is now vindicated in the most awkward way: the
draft it refused to ship had 0.40 / 0.20, so the guessed delay was nearly
right and the guessed interval was off by 50 %. The half that was wrong
would have been protected by the half that was right.

The delay and the interval are NOT equally well evidenced, and the code says
so at the constants. No physical controller exists in the Decoder's
container, so the measurement fed Canary's file driver the SDL driver's own
400/100 ms constants: the 402 ms that came back is the constant that went
in, and confirms the instrument. The 133 ms interval against a fed-in 100 ms
is the new fact -- the game paces repeats to its own frame consumption. One
run; the two-run minimum is not met and the finding says so itself.

Also: the prediction that this would turn verify-input's "a held stick is
ONE step, not six" red was wrong. It stayed green, because steps() never
advances a clock and so had never called repeat_due() at all -- the rate was
about to ship into a harness with no coverage of the feature, with a green
line that would have been read as coverage.

So verify-input gains a `repeat` subject: nothing before the delay, the
first repeat on the delay, the steady interval, cadence independent of frame
rate (the code claims this in a comment, so it is now asserted), and a
direction change restarting the delay. Each asserts THESE numbers, not the
shape -- a shape-only check would have passed on 0.40 / 0.20. The control
removes the premise, a held direction, and every controllable row inverts.

The first-repeat row measures from the arming tick, not from t=0: that tick
is the frame the press is handled, which is the origin the finding measures
its 12 frames from. Measured from zero it read 0.433 vs 0.402 and the
tolerance would have had to be widened to hide a units mismatch.

Closes #2

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 13:34:14 +02:00
5aeca0a55a Merge pull request 're: land the F5/F6 title-clock corpus (docs/re, reference data, sylpheed-formats)' (#23) from auto/frame-blend-draw-path into main
Reviewed-on: #23
2026-09-13 11:16:17 +00:00
MechaCat02
66f38adaa4 fix(lint): PathBuf is unused once the merge takes #22's shared disc_root
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 16:45:05 +02:00
MechaCat02
fd39038652 Merge origin/main into auto/frame-blend-draw-path
`mesh_consistency_disc.rs` had the only conflict: this branch added a
`Sightings` type alias where #22 replaced the file's private `disc_root()`
with the shared `common::disc_root`. Both kept — they are unrelated edits
that happened to land in the same lines.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 16:44:36 +02:00
MechaCat02
ccd49ac31f fix(lint): clear the clippy gate across examples and tests
80 findings, not the 14 the first run showed -- clippy stops at the first
failing compilation unit, so `--keep-going` is what makes the list complete.

60 were machine-applicable (`cargo clippy --fix`). The rest by hand:

* five descending `sort_by` -> `sort_by_key(Reverse(..))`
* `chunks_exact(4)` on both sides of four zips, so the compared items stay
  `[u8; 4]` rather than one array against one slice
* three `type` aliases for the census maps and the captured-quad tuple
* `&PathBuf` -> `&Path` in two disc tests
* two range loops; one of them keeps `#[allow(needless_range_loop)]` with the
  reason -- the index is into a map's value, which changes each iteration
* the module doc list in `invert_capture` re-indented to markdown's rules
* `blit`'s eight arguments get `#[allow(too_many_arguments)]`, not a struct

One dead `let off = b.len();` in a `ratc` test is dropped rather than renamed.
The sibling test at :162 is the one that asserts an offset; if this one was
meant to as well, that is a test change and not a lint fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 16:42:41 +02:00
MechaCat02
c6f7a18e30 fix(lint): the hand-judged clippy sites in library code
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 16:36:43 +02:00
MechaCat02
62376dd4a1 style: rustfmt sweep — 107 files the lint gate never saw
This branch predates CI on `main`. `cargo fmt --all` only; no behaviour.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 16:34:40 +02:00
5b84b3fb5d Merge pull request 'test: make $SYLPHEED_DISC an actual control, not a decoration (#16 remedy 3)' (#22) from fix/corpus-control into main
Reviewed-on: #22
2026-09-12 14:30:22 +00:00
b7e59a6245 Merge pull request 'feat: propose-work — push, open the PR, and move the issue, in one command' (#21) from feat/propose-work into main
Reviewed-on: #21
2026-09-12 14:30:05 +00:00
5952013a91 Merge pull request 'docs(agents): a red PR is the next unit of work, and check the workspace' (#24) from fix/loop-verify-own-pr into main
Reviewed-on: #24
2026-09-12 14:27:07 +00:00
sylph-decoder
4d1d5a1f2e re: fix two stale rows -- NEW GAME's destination, and my own repeat claim
No emulator-requiring state/approved item was open this iteration (issues
#1, #3, #5 all landed and moved to state/needs-human; #25 still awaits
approval). Used the gap for corpus consistency instead of idling, per the
same "a correction that never reaches the row someone reads" failure this
project keeps naming.

1. menu-navigation-semantics.md's own Q4 table still said NEW GAME was
   "not tested" and its own prose said it was "deliberately not pressed"
   and hangs the emulator -- both refuted BY THIS SAME PAGE on 2026-08-28,
   34 lines further down ("NEW GAME -- measured ... it is not a hang. It
   opens DIFFICULTY then SELECT DATA"). The correction never propagated
   backward into the table or the status line above it, so a reader
   stopping at either would come away with the wrong (and already-refuted)
   answer. Fixed in place, struck rather than deleted, with the actual
   destination and an honest note that DIFFICULTY has no id-table name
   match (already refuted separately) while SELECT DATA plausibly matches
   GP_SELECT_STORAGE as a fresh, low-confidence guess.

2. Found the identical failure mode in my own recent work: HANDOFF.md's
   original Q1-Q10 summary table (near the top of a 6600+ line file) still
   quoted the 2026-08-30 "no auto-repeat" finding as current, three commits
   after this same session measured 12 frames delay / 4 frames interval
   through a repeat-capable driver and explained why the earlier negative
   was a driver limitation, not a game fact. A reader who only sees the
   summary table -- which is exactly what a long file trains a reader to
   rely on -- would get the withdrawn answer. Fixed with an explicit note
   pointing at the current entries rather than silently editing the number
   in place, so the correction itself stays visible.

Refutation-shaped either way: two claims ("NEW GAME untested", "no
auto-repeat") checked against this corpus's own newer evidence and found
not to survive, recorded rather than left to be rediscovered.
2026-09-12 13:24:27 +00:00
sylph-decoder
34fb1ddf1e re: F3 sting half closed -- no sting, measured with a working positive control
Continues the static lead from two iterations ago
(f3-title-sting-mechanism-found-not-value.md) with the dynamic half it
named as the next step. Booted with --xma_param_probe=true (the same
census menu-audio-cues.md used for the menu's SE cues), no pad input,
recording continuously from window-open: a glyph time series (not a
threshold trigger) and every newly-seen XMA-PARAM stream, stamped on
arrival since Xenia's own log carries no timestamps.

Positive control, and a real one: the probe caught the title's two BGM
stems starting at t=147.6s, matching f3-title-plays-bgm-102-and-103.md's
already-established finding exactly -- the instrument finds a real stream
before being asked to find nothing.

From the plate's first visible activity through 68 seconds of build-in
plus fully-settled pulsing (killed at t=220.1s), zero new XMA streams
appeared beyond the two BGM stems and three unidentified early ones (boot
splash, not this question). No SE-range stream, no second BGM, nothing --
measured, not the prior static reach limit.

Refutation attempt this iteration, recorded either way: my first read of
the fine-grained glyph series said BGM and the plate's build-in start at
"essentially the same moment". Checking the raw per-sample data instead of
a coarse table refutes that -- first non-zero glyph reading is ~0.67s after
BGM onset, and immediately noisy rather than a clean climb. Corrected in
the doc rather than left as an overclaim for someone chasing frame-accurate
sync later.

Reference data: docs/re/data/f3-sting-{glyph-timeseries,xma-param-arrivals}.tsv
-- derived numeric/log-line data, not a capture of rendered game content.
2026-09-12 12:40:34 +00:00