f5315ddf59
The mounts landed but the agent could not learn of them: I documented them in CONTAINER-NOTES.md, which the decoder's prompt does not list, and then restarted the container -- so a fresh session with no memory of the exchange had a 586 MB database and a decompressed image sitting unmentioned in its filesystem. Now in the PROMPT itself, not only in a document, because the prompt is the one thing a new session is guaranteed to read. CONTAINER-NOTES.md is also added to its reading list. And the caveat that matters more than the asset. The .pe is PRIMARY -- the bytes the console executed. The database is somebody's ANALYSIS of them, produced by a disassembler that had to guess, and it is wrong in the ways disassemblers are wrong: misdecoded mnemonics where data was read as code, function boundaries short or long or merged or split, coverage missing entirely for code reached only by indirect dispatch, and names that are derived rather than symbols. So a finding resting on a database row is not established until the bytes agree: read the same address out of the .pe and check. Where they disagree the image wins, and the disagreement is itself worth recording, because it tells the next reader which parts of the database to distrust. A fast index into 9.2 MB of machine code, not a source of truth.
128 lines
6.1 KiB
Markdown
128 lines
6.1 KiB
Markdown
# Notes for an agent working inside this container
|
|
|
|
Read this before starting a dynamic-RE run. Everything here is something that
|
|
already went wrong once.
|
|
|
|
## The container fixes three old traps for you
|
|
|
|
* **The display outlives the turn.** Xvfb and openbox are children of PID 1, not
|
|
of your shell. The old "Xvfb dies on its own every few minutes" note is gone —
|
|
you no longer have to wrap a whole session in one blocking foreground call to
|
|
keep it alive.
|
|
* **The toolchain is real.** `tools/re-capture/rebuild_canary.sh` exists because
|
|
the old box had no cmake/ninja/clang and only runtime sonames, so it hand-
|
|
relinked object files. **Do not use it here.** Use `build-canary`.
|
|
* **numpy and Pillow are installed.** `entities2.py`, `flight_probe.py` and the
|
|
image oracles work. Their absence used to look like a logic bug.
|
|
|
|
## Method (the part that matters more than the tooling)
|
|
|
|
* **Measure the oracle; never infer it.** A session with zero Canary runs is a
|
|
red flag.
|
|
* **Trace upstream to where data first goes wrong**, rather than patching the
|
|
symptom you can see.
|
|
* **Try to refute before believing.** Record demotions rather than editing them
|
|
away — `docs/re/README.md` has the ✅/🟡/❔ convention, and a withdrawn result
|
|
is more useful than a quietly deleted one.
|
|
* **A probe that never performs the action will "prove" the action does not
|
|
exist.** The "targeting is automatic" conclusion came from a sweep that only
|
|
ever tapped once; target select is Ⓐ pressed *twice*.
|
|
* **Do not poll faster than the guest updates** — it manufactures a clean curve
|
|
out of noise. `rate-curve-aliased-BAD.csv` is committed as the bad example.
|
|
|
|
## Running the emulator
|
|
|
|
```bash
|
|
run-canary # correct audio/pad/display flags baked in
|
|
pad.py tap A ; pad.py dpad down # scripted input (--hid=file, no uinput)
|
|
screenshot ~/shots/now.png # cropped to the GAME surface, not the window
|
|
python3 tools/re-capture/gmem.py find hex:820af844 400
|
|
```
|
|
|
|
* **One emulator at a time.** `run-canary` enforces it with a lockfile.
|
|
* Boot is slow cold, ~25 s once the shader/code caches are warm — so a
|
|
launch-and-dump fits in a single call.
|
|
* **Screens: classify by whole-image statistics** (`screen_id.py`), not named
|
|
pixels. Named-pixel oracles are only valid while the game image sits at a
|
|
known place, and nothing errors when it moves.
|
|
|
|
## Verifying your own work
|
|
|
|
* Reborn's disc-gated tests **self-skip** without `SYLPHEED_DISC`. A green run
|
|
with it unset means almost nothing. `build-reborn test` wires it up for you.
|
|
* Prefer a headless self-verify over "it compiles": `sylpheed-cli mesh render`,
|
|
`screen render`, `save info` all produce checkable artifacts.
|
|
* A Bevy system-parameter conflict is invisible to the type checker and panics
|
|
at startup. If you touch viewer systems, *run the binary*, don't just build it.
|
|
|
|
## Reporting
|
|
|
|
State what you measured, what you assumed, and what you could not settle. If a
|
|
result is withdrawn, say so and keep the reasoning — that is the corpus's whole
|
|
convention, and the reason its numbers can be trusted.
|
|
|
|
## The static-analysis corpus — mounted, not reproducible
|
|
|
|
Two things arrive read-only from the host because **nothing in this repository
|
|
can produce them yet**:
|
|
|
|
| path | what | env |
|
|
|---|---|---|
|
|
| `/xenia-rs/sylpheed.db` | the disassembly database, 586 MB | `SYLPHEED_DB` |
|
|
| `/image/sylpheed.pe` | the decompressed image, flat VA dump | `SYLPHEED_PE` |
|
|
|
|
**The `.pe` is a flat VA dump**: file offset = `VA - 0x82000000`
|
|
(`SYLPHEED_IMAGE_BASE`). So reading `0x820A1630` is `seek(0xA1630)` — no XEX
|
|
decrypt, no LZX, and **no booted emulator**. An earlier belief that this file was
|
|
stale was tested and **refuted**; it is current.
|
|
|
|
Recovering the image by dumping `/dev/shm/xenia_memory_*` also works and
|
|
self-validates, but it needs a running emulator — a poor dependency for
|
|
something the whole static corpus rests on. Use the file.
|
|
|
|
The database is far richer than the four scripts that read it use:
|
|
|
|
```
|
|
functions 25 481 address, name, end_address, frame_size,
|
|
saved_gprs, is_leaf, pdata_validated, has_eh
|
|
classes 851 name, vtable_address, rtti_present, base_classes_json
|
|
imports 398 library, ordinal, name, address
|
|
eh_funcinfo / eh_try_blocks 2 588 / 315
|
|
function_pointer_arrays 1 526 + 8 568 entries
|
|
indirect_dispatch_candidates 1 827 297 dispatch_pc, vtable_address, method_address
|
|
```
|
|
|
|
`instructions.raw` is an **INT, not hex** — a trap this corpus has already paid
|
|
for. Query with `python3 -c 'import duckdb'`; it is not SQLite.
|
|
|
|
### ⚠️ It is derived data, and it is wrong in places
|
|
|
|
The image is **primary** — those are the bytes the console executed. The database
|
|
is **an analysis of them**, produced by a disassembler that had to guess, and it
|
|
fails the way disassemblers fail:
|
|
|
|
* **misdecoded mnemonics** — data read as code, or a decoder-table gap, produces
|
|
a plausible instruction that was never executed as one;
|
|
* **wrong function boundaries** — `end_address` short or long, neighbours merged,
|
|
one function split in two;
|
|
* **incomplete coverage** — code reached only through indirect dispatch may be
|
|
absent entirely; the 1.8 M `indirect_dispatch_candidates` are *candidates*;
|
|
* **invented names** — largely derived rather than symbols, so a name is a
|
|
hypothesis wearing a label.
|
|
|
|
**A finding resting on a database row is not established until the bytes agree.**
|
|
Read the same address out of the `.pe` and check. Where they disagree the image
|
|
wins, and the disagreement is worth recording — it tells the next reader which
|
|
parts of the database to distrust.
|
|
|
|
It is a fast index into 9.2 MB of machine code. It is not a source of truth.
|
|
|
|
### These mounts are reference material, not a deliverable
|
|
|
|
They are read-only and they come from outside the repository, which means a
|
|
fresh checkout on another machine has neither. **Reimplementing the producer —
|
|
XEX decrypt + LZX decompress, and the disassembly-to-database step — belongs in
|
|
`crates/sylpheed-formats`.** Until then, every static finding rests on an
|
|
artefact this project cannot rebuild, and that is a real gap in the corpus rather
|
|
than a convenience.
|