# Manual generator

Python scripts that build the `tools/ppc-manual/` tree from:

- **Xenia Canary** — the reference emulator — read at a **pinned commit** through
  `git show`, never from a working tree:
  - `tools/ppc-instructions.xml` — metadata for all 455 Xbox 360 PPC instructions
    (mnemonic, form, group, opcode, in/out fields, disasm template);
  - `src/xenia/cpu/ppc/ppc_emit_*.cc` — each instruction's `InstrEmit_<mnem>`
    emitter, quoted on its page as the semantic snapshot.
- **`crates/sylpheed-ppc`** — Sylpheed's own decoder, the one that produces the
  disassembly in `sylpheed.db`; referenced by opcode and decoder line.

The pin is `origin/canary_experimental` by default. Our own Canary checkout is a
working branch: it carries instrumentation and can lag upstream by months — it
lacked upstream's `mcrf` fix when this was written — so quoting it would publish
probes and stale semantics as "Canary".

`xenia-rs` was a source until 2026-09-16. It is retired and archived; its
interpreter is no longer quoted. Hand-written notes that still discuss its
behaviour link to the archived repository.

## Files

| File | Purpose |
| --- | --- |
| `generate_manual.py` | Main entry point. Parses XML, builds families, renders pages, writes `index.json`. |
| `xml_model.py`       | XML parser + `expand_runtime_variants()` (produces the set of Rc/OE/LK-expanded mnemonics a single XML entry covers). |
| `bit_layout.py`      | Per-form bit-field tables (rendered into the Encoding section of every page and into `forms/*.md`). |
| `cxx_scraper.py`     | `CanarySource` (one pinned commit) and `CxxScraper`: locates each `InstrEmit_<mnem>`, captures its full text, and follows pure one-line delegations to the helper that holds the semantics. Brace matching models `#if 0` / `#else`. |
| `decoder_scraper.py` | Locates each `PpcOpcode::<mnem>` variant and decoder producer in `crates/sylpheed-ppc`. |

## Running

From the repository root:

```bash
python3 tools/ppc-manual/generator/generate_manual.py            # full generate
python3 tools/ppc-manual/generator/generate_manual.py --dry-run  # parse + consistency checks only
python3 tools/ppc-manual/generator/generate_manual.py --canary ../xenia-canary --canary-ref origin/canary_experimental
python3 tools/ppc-manual/generator/generate_manual.py --out /tmp/out   # alternate output root
```

`--canary` defaults to the `xenia-canary` checkout beside this repository.
Fetch upstream first (`git -C ../xenia-canary fetch origin`) to move the pin
forward; the commit used is printed, recorded in `index.json` under
`sources.canary.commit`, and stamped on every snapshot. A missing checkout, ref
or source file is an **error** — the previous generator skipped what it could
not find, which is how every reference in the manual rotted unnoticed when it
moved into `tools/`.

No third-party dependencies; Python 3.10+ standard library and `git`.

## Idempotency

The generator is re-runnable without data loss:

1. Each page has a pair of sentinel comments:
   - `<!-- GENERATED: BEGIN -->`
   - `<!-- GENERATED: END -->`
2. On re-run, only the text **between** the sentinels is rewritten.
   Everything after `END` (Special Cases, Related Instructions, IBM
   Reference) is preserved verbatim.
3. If the `END` sentinel is missing, the generator assumes a reviewer
   has fully taken over the file and skips it entirely.

## Consistency checks (enforced by `--dry-run` as well)

- **XML entry count ≡ 455** — warns if the XML has been modified.
- **family membership total ≡ XML entry count** — every XML entry
  must land in exactly one family.
- **index coverage ≡ runtime-expanded mnemonic count** — the JSON
  index must contain a key for every runtime variant (`add`, `add.`,
  `addo`, `addo.`, `bclr`, `bclrl`, …).

## Family grouping rules

Three rules applied in order (see `_family_head` in
`generate_manual.py`):

1. If a mnemonic ends in `128` and the non-128 sibling exists, it
   joins the sibling's family. So `vaddfp128` is consolidated into
   the `vaddfp` page.
2. For memory ops (group `m`), trailing `u`, `x`, or `ux` suffixes
   are stripped when the base exists. So `lwz`, `lwzu`, `lwzx`,
   `lwzux` all land on the `lwz` page.
3. Otherwise the mnemonic is its own family head.

All other flag variants (`Rc`, `OE`, `LK`) are **runtime** — they are
NOT separate XML entries; they are listed in the page's "Assembler
Mnemonics" table.

## Category mapping

| XML group | Category dir | Notes |
| --- | --- | --- |
| `i` (integer) | `alu/` | |
| `m` (memory)  | `memory/` | |
| `b` (branch)  | `branch/` | Includes `sc` and traps |
| `c` (control) | `control/` | CR logical, SPR, sync |
| `f` (fpu)     | `fpu/` | |
| `v` (vector)  | `vmx/` or `vmx128/` | Split by form: `VX128*` → `vmx128/` |

## Extending the generator

- **Pseudocode seeds.** The `PSEUDOCODE_SEEDS` dict in
  `generate_manual.py` maps an XML mnemonic to a PPC-style pseudocode
  block. Add entries here to pre-fill the Operation section for
  additional mnemonics. Phase 2 reviewers can still override by
  writing content outside the sentinels.
- **C translation seeds.** Similar dict of C snippets keyed by family
  head.
- **Field descriptions.** `FIELD_DESCRIPTIONS` maps XML field names to
  IBM-style prose. Missing entries are marked "_Phase 2: document
  this field._"

## Known limitations

- Extended-opcode extraction in `xml_model.Instruction.extended_opcode`
  is best-effort per form. For VMX128 variants the extracted value may
  not match the exact pattern used by the decoder — the page still
  shows it as a reference, but the `sylpheed-ppc` decoder line (linked
  on every page) is authoritative.
- `cxx_scraper` delimits emitters with a brace counter that ignores comments,
  string literals and preprocessor-dead lines. On the pinned commit it agrees with
  the column-0 `}` convention on all 521 functions that follow it; the one that
  does not, `InstrEmit_branch`, closes on a `}` carrying a stale
  `// namespace ppc` comment. `memory/dcbi.md` has no snapshot because Canary has
  no `InstrEmit_dcbi` at all.
- The generator treats mnemonics ending in `x` as xenia convention
  ("extended/XO form") and strips them for assembly display — except
  for the memory group, where `x` is the natural indexed-form suffix.
  If future xenia XML adds a new group where `x` is structural, the
  heuristic in `xml_model.expand_runtime_variants` needs updating.