Sylpheed/tools/ppc-manual/generator/README.md

Manual generator

Python scripts that build the tools/ppc-manual/ tree from:

• Xenia Canary — the reference emulator — read at a pinned commit through git show, never from a working tree:
  • tools/ppc-instructions.xml — metadata for all 455 Xbox 360 PPC instructions (mnemonic, form, group, opcode, in/out fields, disasm template);
  • src/xenia/cpu/ppc/ppc_emit_*.cc — each instruction's InstrEmit_<mnem> emitter, quoted on its page as the semantic snapshot.
• crates/sylpheed-ppc — Sylpheed's own decoder, the one that produces the disassembly in sylpheed.db; referenced by opcode and decoder line.

The pin is origin/canary_experimental by default. Our own Canary checkout is a working branch: it carries instrumentation and can lag upstream by months — it lacked upstream's mcrf fix when this was written — so quoting it would publish probes and stale semantics as "Canary".

xenia-rs was a source until 2026-09-16. It is retired and archived; its interpreter is no longer quoted. Hand-written notes that still discuss its behaviour link to the archived repository.

Files

File	Purpose
generate_manual.py	Main entry point. Parses XML, builds families, renders pages, writes index.json.
xml_model.py	XML parser + expand_runtime_variants() (produces the set of Rc/OE/LK-expanded mnemonics a single XML entry covers).
bit_layout.py	Per-form bit-field tables (rendered into the Encoding section of every page and into forms/*.md).
cxx_scraper.py	CanarySource (one pinned commit) and CxxScraper: locates each InstrEmit_<mnem>, captures its full text, and follows pure one-line delegations to the helper that holds the semantics. Brace matching models #if 0 / #else.
decoder_scraper.py	Locates each PpcOpcode::<mnem> variant and decoder producer in crates/sylpheed-ppc.

Running

From the repository root:

python3 tools/ppc-manual/generator/generate_manual.py            # full generate
python3 tools/ppc-manual/generator/generate_manual.py --dry-run  # parse + consistency checks only
python3 tools/ppc-manual/generator/generate_manual.py --canary ../xenia-canary --canary-ref origin/canary_experimental
python3 tools/ppc-manual/generator/generate_manual.py --out /tmp/out   # alternate output root

--canary defaults to the xenia-canary checkout beside this repository. Fetch upstream first (git -C ../xenia-canary fetch origin) to move the pin forward; the commit used is printed, recorded in index.json under sources.canary.commit, and stamped on every snapshot. A missing checkout, ref or source file is an error — the previous generator skipped what it could not find, which is how every reference in the manual rotted unnoticed when it moved into tools/.

No third-party dependencies; Python 3.10+ standard library and git.

Idempotency

The generator is re-runnable without data loss:

1. Each page has a pair of sentinel comments:
   • <!-- GENERATED: BEGIN -->
   • <!-- GENERATED: END -->
2. On re-run, only the text between the sentinels is rewritten. Everything after END (Special Cases, Related Instructions, IBM Reference) is preserved verbatim.
3. If the END sentinel is missing, the generator assumes a reviewer has fully taken over the file and skips it entirely.

Consistency checks (enforced by --dry-run as well)

• XML entry count ≡ 455 — warns if the XML has been modified.
• family membership total ≡ XML entry count — every XML entry must land in exactly one family.
• index coverage ≡ runtime-expanded mnemonic count — the JSON index must contain a key for every runtime variant (add, add., addo, addo., bclr, bclrl, …).

Family grouping rules

Three rules applied in order (see _family_head in generate_manual.py):

1. If a mnemonic ends in 128 and the non-128 sibling exists, it joins the sibling's family. So vaddfp128 is consolidated into the vaddfp page.
2. For memory ops (group m), trailing u, x, or ux suffixes are stripped when the base exists. So lwz, lwzu, lwzx, lwzux all land on the lwz page.
3. Otherwise the mnemonic is its own family head.

All other flag variants (Rc, OE, LK) are runtime — they are NOT separate XML entries; they are listed in the page's "Assembler Mnemonics" table.

Category mapping

XML group	Category dir	Notes
i (integer)	alu/
m (memory)	memory/
b (branch)	branch/	Includes sc and traps
c (control)	control/	CR logical, SPR, sync
f (fpu)	fpu/
v (vector)	vmx/ or vmx128/	Split by form: VX128* → vmx128/

Extending the generator

• Pseudocode seeds. The PSEUDOCODE_SEEDS dict in generate_manual.py maps an XML mnemonic to a PPC-style pseudocode block. Add entries here to pre-fill the Operation section for additional mnemonics. Phase 2 reviewers can still override by writing content outside the sentinels.
• C translation seeds. Similar dict of C snippets keyed by family head.
• Field descriptions. FIELD_DESCRIPTIONS maps XML field names to IBM-style prose. Missing entries are marked "Phase 2: document this field."

Known limitations

• Extended-opcode extraction in xml_model.Instruction.extended_opcode is best-effort per form. For VMX128 variants the extracted value may not match the exact pattern used by the decoder — the page still shows it as a reference, but the sylpheed-ppc decoder line (linked on every page) is authoritative.
• cxx_scraper delimits emitters with a brace counter that ignores comments, string literals and preprocessor-dead lines. On the pinned commit it agrees with the column-0 } convention on all 521 functions that follow it; the one that does not, InstrEmit_branch, closes on a } carrying a stale // namespace ppc comment. memory/dcbi.md has no snapshot because Canary has no InstrEmit_dcbi at all.
• The generator treats mnemonics ending in x as xenia convention ("extended/XO form") and strips them for assembly display — except for the memory group, where x is the natural indexed-form suffix. If future xenia XML adds a new group where x is structural, the heuristic in xml_model.expand_runtime_variants needs updating.