re: name-resolve all 40 MiscBin entries, and demonstrate the REGN<->MCOL pairing

The names live outside MiscBin: they are the MapPath / MapMesh /
CollisionMeshes field values of the per-stage StageResource object (IDXD schema
3c9ae32e, in every GP_MAIN_GAME_<lang>.pak), and each hashes with the ordinary
pak name_hash straight to a TOC entry.  40/40 resolve, no collisions -- the 11
REGN as <stem>.rgn, the 11 MCOL as <stem>.col, and the 18 remaining blobs as
CollisionSet_S01..S16 / _Tutorial / _test.bin.  The .pe string table at 651540
was the way in: MapMesh and MapPath sit adjacent there.

This upgrades the pairing claim.  The first section of mcol-collision.md could
only say REGN and MCOL had matching *distributions* of bbox and cell size, and
flagged that as not an object-to-object link.  A phase record names one .rgn and
one .col, and all 11/11 pairs share a stem and agree exactly on both.

The names also check the format work from outside it: mapmesh_box_500km.col is
the object decoded here as 8 vertices and 12 triangles spanning exactly
+-250000, and its name says that cube is 500 km across -- so one world unit is
one metre, and a wrong stride could not have produced a box that measures what
its own filename claims.  70 of the 87 phases use it: most stages' only
collision is the arena wall, and _AsteroidVolume_ names the rest.

Still open: the 18 CollisionSet_*.bin are named but not decoded (all exactly
1675148 bytes), and CMapColliderBridge in the RTTI names the runtime consumer
without following it into the code.
This commit is contained in:
Sylpheed RE agent
2026-08-26 09:21:00 +00:00
parent c0a7033295
commit fa5a51ac78
3 changed files with 206 additions and 0 deletions

Binary file not shown.

View File

@@ -486,3 +486,69 @@ any object out as a Wavefront OBJ.
belongs to — and the object-to-object pairing with `REGN` that the header
distributions only hint at — is still unknown. And the runtime consumer has not
been found, the same gap `REGN` has.
## ✅✅ All 40 `MiscBin` entries are name-resolved, and the `REGN`↔`MCOL` pairing is now demonstrated
**2026-08-26.** The names are not in `MiscBin`. They are the values of three
fields of the per-stage `StageResource` object (IDXD schema `3c9ae32e`, present
in every `dat/GP_MAIN_GAME_<lang>.pak`), and each hashes with the ordinary pak
`name_hash` straight to a TOC entry:
Phase_1 MapPath = 'S14_p1_AsteroidVolume_wp.rgn' → REGN
MapMesh = 'S14_p1_AsteroidVolume_wp.col' → MCOL
CollisionMeshes = 'CollisionSet_S14.bin'
**40 / 40 entries resolved, with no hash collisions** — the 11 `REGN`, the 11
`MCOL`, and the 18 remaining 1 675 148-byte entries, which are the
`CollisionSet_S01…S16.bin`, `CollisionSet_Tutorial.bin` and
`CollisionSet_test.bin` blobs. Nothing in the archive is unaccounted for.
The route in was the `.pe`: `MapMesh` and `MapPath` sit adjacent in the string
table at file offset 651 540, next to `CollisionMeshes` and `3DSetup.tbl`.
### The pairing, upgraded from "matching distributions" to an object-level link
The first section of this page could only say that `MCOL` and `REGN` had the
*same distribution* of bounding boxes and cell sizes, and flagged that this was
not a demonstrated object-to-object pairing. It is now: a phase record names one
`.rgn` and one `.col`, and for all **11 / 11** pairs the two share a stem and
agree exactly on bounding box and cell size.
| stem | phases | verts / tris |
|---|---|---|
| `test` | 3 | 34 / 60 |
| `mapmesh_box_500km` | 70 | 8 / 12 |
| `S01_AsteroidVolume_wp` | 3 | 111 / 218 |
| `S04_AsteroidVolume_wp` | 2 | 405 / 790 |
| `S05_AsteroidVolume_wp` | 2 | 89 / 174 |
| `S08_p1_AsteroidVolume_wp` | 1 | 134 / 260 |
| `S08_p2_AsteroidVolume_wp` | 1 | 8 / 12 |
| `S13_AsteroidVolume_wp` | 1 | 73 / 134 |
| `S14_p1_AsteroidVolume_wp` | 1 | 502 / 924 |
| `S14_p2_AsteroidVolume_wp` | 1 | 632 / 1 164 |
| `S28_p1_AsteroidVolume_wp` | 1 | 554 / 1 020 |
`_AsteroidVolume_` also says what the meshes *are*: the asteroid fields, which
is why the collision hull is a closed manifold and why most stages need none —
**70 of the 87 phases use `mapmesh_box_500km`**, the bare arena wall.
### The name confirms the decode, and gives the world unit
`mapmesh_box_500km.col` is the object this page decoded as **8 vertices and 12
triangles spanning exactly ±250 000** — a cube. Its name says that cube is
**500 km** across, and the decoded span is **500 000.0** units exactly, so
> **one world unit is one metre.**
This is a name-based inference, but the arithmetic is exact and it runs the
other way as a check on the format work: a wrong stride or index width could not
have produced a box whose measured size matches its own filename.
Reproduce with `tools/re-capture/miscbin_names.py <extract root> [--pairs]`
(recorded in [`../data/miscbin-names.txt`](../data/miscbin-names.txt)).
❔ Still open: the 18 `CollisionSet_*.bin` objects (magic `0x00000810`, and all
**exactly 1 675 148 bytes**, which is odd for per-stage data) are named but not
decoded. And the runtime consumer now has a name — the `.pe` RTTI carries
`CMapColliderBridge` and `CSingleton<CMapColliderBridge>` at offset 9 044 264 —
but has not been followed into the code.

140
tools/re-capture/miscbin_names.py Executable file
View File

@@ -0,0 +1,140 @@
#!/usr/bin/env python3
"""Resolve every `hidden/MiscBin.pak` entry to its on-disc name.
The names are not in `MiscBin` itself -- they are the values of the `MapPath`,
`MapMesh` and `CollisionMeshes` fields of the per-stage `StageResource` objects
(IDXD schema `3c9ae32e`, in every `dat/GP_MAIN_GAME_<lang>.pak`), and each one
hashes with the ordinary pak `name_hash` to a `MiscBin` TOC entry.
./miscbin_names.py <extract root> # the table
./miscbin_names.py <extract root> --pairs # check each .rgn/.col pair
`MapPath` names the `REGN` navigation mesh and `MapMesh` the `MCOL` collision
mesh of one *phase* of one stage, so the pair is what links the two formats --
an object-to-object link, not the matching-header-distributions argument that
`structures/mcol-collision.md` had to settle for before.
"""
import collections
import glob
import os
import struct
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from regn_decode import pak_entries
from unitgroup import name_hash, tag_hash
SCHEMA = 0x3C9AE32E
KEYS = ("MapPath", "MapMesh", "CollisionMeshes")
TAGS = {tag_hash(k): k for k in KEYS}
def be32(b, o):
return struct.unpack_from(">I", b, o)[0]
def parse_idxd(b):
"""Records and fields of an IDXD object; None if it is not one."""
if b[:4] != b"IDXD" or len(b) < 0x10:
return None
n = be32(b, 4)
if not 0 < n <= 200_000:
return None
recs = [(None,) + struct.unpack_from(">III", b, 0x0C)]
o = 0x18
if o + 16 * (n - 1) + 4 > len(b):
return None
for _ in range(n - 1):
recs.append(struct.unpack_from(">IIII", b, o))
o += 16
m = be32(b, o)
o += 4
if o + 12 * m + 4 > len(b):
return None
fields = [struct.unpack_from(">III", b, o + 12 * i) for i in range(m)]
return dict(schema=be32(b, 8), recs=recs, fields=fields,
pool=o + 12 * m + 4, raw=b)
def pool_str(p, off):
b, base = p["raw"], p["pool"]
if base + off >= len(b):
return None
return b[base + off:b.index(b"\0", base + off)].decode("latin-1")
def stage_phases(root):
"""(phase record name, {key: value}) for every stage-resource phase."""
out = []
for pak in sorted(glob.glob(os.path.join(root, "dat", "GP_MAIN_GAME_*.pak"))):
for _, blob in pak_entries(pak):
p = parse_idxd(blob)
if not p or p["schema"] != SCHEMA:
continue
for _, noff, fb, fe in p["recs"]:
d = {TAGS[k]: pool_str(p, vo)
for k, _, vo in p["fields"][fb:fe] if k in TAGS}
if d:
out.append((pool_str(p, noff) if noff is not None else None, d))
return out
def resolve(root):
misc = dict(pak_entries(os.path.join(root, "hidden", "MiscBin.pak")))
seen = collections.OrderedDict()
for _, d in stage_phases(root):
for k in KEYS:
if d.get(k):
seen.setdefault(d[k], k)
table = []
for nm, kind in seen.items():
h = name_hash(nm)
blob = misc.get(h)
table.append((nm, h, kind, blob[:4].decode("latin-1") if blob else None))
return misc, table
def cmd_table(root):
misc, table = resolve(root)
print(f"{'name':>30} {'name_hash':>10} {'magic':>5} {'field':>15}")
for nm, h, kind, magic in sorted(table, key=lambda r: (r[3] or "", r[0])):
print(f"{nm:>30} {h:10x} {magic or ' --':>5} {kind:>15}")
got = {h for _, h, _, m in table if m}
print(f"\nMiscBin entries {len(misc)} resolved {len(got)} "
f"unresolved {len(set(misc) - got)}")
return len(set(misc) - got) == 0
def cmd_pairs(root):
"""Each phase names a .rgn and a .col; check they describe the same volume."""
misc, _ = resolve(root)
f32 = lambda b, o: struct.unpack_from(">f", b, o)[0]
u16 = lambda b, o: struct.unpack_from(">H", b, o)[0]
pairs = collections.OrderedDict()
for _, d in stage_phases(root):
if d.get("MapPath") and d.get("MapMesh"):
pairs.setdefault((d["MapPath"], d["MapMesh"]), 0)
pairs[(d["MapPath"], d["MapMesh"])] += 1
ok = 0
print(f"{'stem':>30} {'phases':>7} {'bbox':>6} {'cell':>6} verts/tris")
for (rgn, col), n in pairs.items():
R, C = misc[name_hash(rgn)], misc[name_hash(col)]
stem = rgn.rsplit(".", 1)[0] == col.rsplit(".", 1)[0]
bb = all(f32(R, 0x10 + 4 * i) == f32(C, 0x10 + 4 * i)
and f32(R, 0x20 + 4 * i) == f32(C, 0x20 + 4 * i) for i in range(3))
cc = all(f32(R, 0x40 + 4 * i) == f32(C, 0x40 + 4 * i) for i in range(3))
ok += stem and bb and cc
print(f"{rgn.rsplit('.', 1)[0]:>30} {n:7d} {'yes' if bb else 'NO':>6} "
f"{'yes' if cc else 'NO':>6} {u16(C, 0x50)}/{u16(C, 0x52)}")
print(f"\npairs agreeing on stem, bbox and cell size: {ok}/{len(pairs)}")
return ok == len(pairs)
if __name__ == "__main__":
if len(sys.argv) < 2:
raise SystemExit(__doc__)
root = sys.argv[1]
good = cmd_pairs(root) if "--pairs" in sys.argv else cmd_table(root)
print("\nPASS" if good else "\nFAIL")
sys.exit(0 if good else 1)