re: name-resolve all 40 MiscBin entries, and demonstrate the REGN<->MCOL pairing
The names live outside MiscBin: they are the MapPath / MapMesh / CollisionMeshes field values of the per-stage StageResource object (IDXD schema 3c9ae32e, in every GP_MAIN_GAME_<lang>.pak), and each hashes with the ordinary pak name_hash straight to a TOC entry. 40/40 resolve, no collisions -- the 11 REGN as <stem>.rgn, the 11 MCOL as <stem>.col, and the 18 remaining blobs as CollisionSet_S01..S16 / _Tutorial / _test.bin. The .pe string table at 651540 was the way in: MapMesh and MapPath sit adjacent there. This upgrades the pairing claim. The first section of mcol-collision.md could only say REGN and MCOL had matching *distributions* of bbox and cell size, and flagged that as not an object-to-object link. A phase record names one .rgn and one .col, and all 11/11 pairs share a stem and agree exactly on both. The names also check the format work from outside it: mapmesh_box_500km.col is the object decoded here as 8 vertices and 12 triangles spanning exactly +-250000, and its name says that cube is 500 km across -- so one world unit is one metre, and a wrong stride could not have produced a box that measures what its own filename claims. 70 of the 87 phases use it: most stages' only collision is the arena wall, and _AsteroidVolume_ names the rest. Still open: the 18 CollisionSet_*.bin are named but not decoded (all exactly 1675148 bytes), and CMapColliderBridge in the RTTI names the runtime consumer without following it into the code.
This commit is contained in:
BIN
docs/re/data/miscbin-names.txt
Normal file
BIN
docs/re/data/miscbin-names.txt
Normal file
Binary file not shown.
@@ -486,3 +486,69 @@ any object out as a Wavefront OBJ.
|
||||
belongs to — and the object-to-object pairing with `REGN` that the header
|
||||
distributions only hint at — is still unknown. And the runtime consumer has not
|
||||
been found, the same gap `REGN` has.
|
||||
|
||||
## ✅✅ All 40 `MiscBin` entries are name-resolved, and the `REGN`↔`MCOL` pairing is now demonstrated
|
||||
|
||||
**2026-08-26.** The names are not in `MiscBin`. They are the values of three
|
||||
fields of the per-stage `StageResource` object (IDXD schema `3c9ae32e`, present
|
||||
in every `dat/GP_MAIN_GAME_<lang>.pak`), and each hashes with the ordinary pak
|
||||
`name_hash` straight to a TOC entry:
|
||||
|
||||
Phase_1 MapPath = 'S14_p1_AsteroidVolume_wp.rgn' → REGN
|
||||
MapMesh = 'S14_p1_AsteroidVolume_wp.col' → MCOL
|
||||
CollisionMeshes = 'CollisionSet_S14.bin'
|
||||
|
||||
**40 / 40 entries resolved, with no hash collisions** — the 11 `REGN`, the 11
|
||||
`MCOL`, and the 18 remaining 1 675 148-byte entries, which are the
|
||||
`CollisionSet_S01…S16.bin`, `CollisionSet_Tutorial.bin` and
|
||||
`CollisionSet_test.bin` blobs. Nothing in the archive is unaccounted for.
|
||||
|
||||
The route in was the `.pe`: `MapMesh` and `MapPath` sit adjacent in the string
|
||||
table at file offset 651 540, next to `CollisionMeshes` and `3DSetup.tbl`.
|
||||
|
||||
### The pairing, upgraded from "matching distributions" to an object-level link
|
||||
|
||||
The first section of this page could only say that `MCOL` and `REGN` had the
|
||||
*same distribution* of bounding boxes and cell sizes, and flagged that this was
|
||||
not a demonstrated object-to-object pairing. It is now: a phase record names one
|
||||
`.rgn` and one `.col`, and for all **11 / 11** pairs the two share a stem and
|
||||
agree exactly on bounding box and cell size.
|
||||
|
||||
| stem | phases | verts / tris |
|
||||
|---|---|---|
|
||||
| `test` | 3 | 34 / 60 |
|
||||
| `mapmesh_box_500km` | 70 | 8 / 12 |
|
||||
| `S01_AsteroidVolume_wp` | 3 | 111 / 218 |
|
||||
| `S04_AsteroidVolume_wp` | 2 | 405 / 790 |
|
||||
| `S05_AsteroidVolume_wp` | 2 | 89 / 174 |
|
||||
| `S08_p1_AsteroidVolume_wp` | 1 | 134 / 260 |
|
||||
| `S08_p2_AsteroidVolume_wp` | 1 | 8 / 12 |
|
||||
| `S13_AsteroidVolume_wp` | 1 | 73 / 134 |
|
||||
| `S14_p1_AsteroidVolume_wp` | 1 | 502 / 924 |
|
||||
| `S14_p2_AsteroidVolume_wp` | 1 | 632 / 1 164 |
|
||||
| `S28_p1_AsteroidVolume_wp` | 1 | 554 / 1 020 |
|
||||
|
||||
`_AsteroidVolume_` also says what the meshes *are*: the asteroid fields, which
|
||||
is why the collision hull is a closed manifold and why most stages need none —
|
||||
**70 of the 87 phases use `mapmesh_box_500km`**, the bare arena wall.
|
||||
|
||||
### The name confirms the decode, and gives the world unit
|
||||
|
||||
`mapmesh_box_500km.col` is the object this page decoded as **8 vertices and 12
|
||||
triangles spanning exactly ±250 000** — a cube. Its name says that cube is
|
||||
**500 km** across, and the decoded span is **500 000.0** units exactly, so
|
||||
|
||||
> **one world unit is one metre.**
|
||||
|
||||
This is a name-based inference, but the arithmetic is exact and it runs the
|
||||
other way as a check on the format work: a wrong stride or index width could not
|
||||
have produced a box whose measured size matches its own filename.
|
||||
|
||||
Reproduce with `tools/re-capture/miscbin_names.py <extract root> [--pairs]`
|
||||
(recorded in [`../data/miscbin-names.txt`](../data/miscbin-names.txt)).
|
||||
|
||||
❔ Still open: the 18 `CollisionSet_*.bin` objects (magic `0x00000810`, and all
|
||||
**exactly 1 675 148 bytes**, which is odd for per-stage data) are named but not
|
||||
decoded. And the runtime consumer now has a name — the `.pe` RTTI carries
|
||||
`CMapColliderBridge` and `CSingleton<CMapColliderBridge>` at offset 9 044 264 —
|
||||
but has not been followed into the code.
|
||||
|
||||
140
tools/re-capture/miscbin_names.py
Executable file
140
tools/re-capture/miscbin_names.py
Executable file
@@ -0,0 +1,140 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Resolve every `hidden/MiscBin.pak` entry to its on-disc name.
|
||||
|
||||
The names are not in `MiscBin` itself -- they are the values of the `MapPath`,
|
||||
`MapMesh` and `CollisionMeshes` fields of the per-stage `StageResource` objects
|
||||
(IDXD schema `3c9ae32e`, in every `dat/GP_MAIN_GAME_<lang>.pak`), and each one
|
||||
hashes with the ordinary pak `name_hash` to a `MiscBin` TOC entry.
|
||||
|
||||
./miscbin_names.py <extract root> # the table
|
||||
./miscbin_names.py <extract root> --pairs # check each .rgn/.col pair
|
||||
|
||||
`MapPath` names the `REGN` navigation mesh and `MapMesh` the `MCOL` collision
|
||||
mesh of one *phase* of one stage, so the pair is what links the two formats --
|
||||
an object-to-object link, not the matching-header-distributions argument that
|
||||
`structures/mcol-collision.md` had to settle for before.
|
||||
"""
|
||||
|
||||
import collections
|
||||
import glob
|
||||
import os
|
||||
import struct
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from regn_decode import pak_entries
|
||||
from unitgroup import name_hash, tag_hash
|
||||
|
||||
SCHEMA = 0x3C9AE32E
|
||||
KEYS = ("MapPath", "MapMesh", "CollisionMeshes")
|
||||
TAGS = {tag_hash(k): k for k in KEYS}
|
||||
|
||||
|
||||
def be32(b, o):
|
||||
return struct.unpack_from(">I", b, o)[0]
|
||||
|
||||
|
||||
def parse_idxd(b):
|
||||
"""Records and fields of an IDXD object; None if it is not one."""
|
||||
if b[:4] != b"IDXD" or len(b) < 0x10:
|
||||
return None
|
||||
n = be32(b, 4)
|
||||
if not 0 < n <= 200_000:
|
||||
return None
|
||||
recs = [(None,) + struct.unpack_from(">III", b, 0x0C)]
|
||||
o = 0x18
|
||||
if o + 16 * (n - 1) + 4 > len(b):
|
||||
return None
|
||||
for _ in range(n - 1):
|
||||
recs.append(struct.unpack_from(">IIII", b, o))
|
||||
o += 16
|
||||
m = be32(b, o)
|
||||
o += 4
|
||||
if o + 12 * m + 4 > len(b):
|
||||
return None
|
||||
fields = [struct.unpack_from(">III", b, o + 12 * i) for i in range(m)]
|
||||
return dict(schema=be32(b, 8), recs=recs, fields=fields,
|
||||
pool=o + 12 * m + 4, raw=b)
|
||||
|
||||
|
||||
def pool_str(p, off):
|
||||
b, base = p["raw"], p["pool"]
|
||||
if base + off >= len(b):
|
||||
return None
|
||||
return b[base + off:b.index(b"\0", base + off)].decode("latin-1")
|
||||
|
||||
|
||||
def stage_phases(root):
|
||||
"""(phase record name, {key: value}) for every stage-resource phase."""
|
||||
out = []
|
||||
for pak in sorted(glob.glob(os.path.join(root, "dat", "GP_MAIN_GAME_*.pak"))):
|
||||
for _, blob in pak_entries(pak):
|
||||
p = parse_idxd(blob)
|
||||
if not p or p["schema"] != SCHEMA:
|
||||
continue
|
||||
for _, noff, fb, fe in p["recs"]:
|
||||
d = {TAGS[k]: pool_str(p, vo)
|
||||
for k, _, vo in p["fields"][fb:fe] if k in TAGS}
|
||||
if d:
|
||||
out.append((pool_str(p, noff) if noff is not None else None, d))
|
||||
return out
|
||||
|
||||
|
||||
def resolve(root):
|
||||
misc = dict(pak_entries(os.path.join(root, "hidden", "MiscBin.pak")))
|
||||
seen = collections.OrderedDict()
|
||||
for _, d in stage_phases(root):
|
||||
for k in KEYS:
|
||||
if d.get(k):
|
||||
seen.setdefault(d[k], k)
|
||||
table = []
|
||||
for nm, kind in seen.items():
|
||||
h = name_hash(nm)
|
||||
blob = misc.get(h)
|
||||
table.append((nm, h, kind, blob[:4].decode("latin-1") if blob else None))
|
||||
return misc, table
|
||||
|
||||
|
||||
def cmd_table(root):
|
||||
misc, table = resolve(root)
|
||||
print(f"{'name':>30} {'name_hash':>10} {'magic':>5} {'field':>15}")
|
||||
for nm, h, kind, magic in sorted(table, key=lambda r: (r[3] or "", r[0])):
|
||||
print(f"{nm:>30} {h:10x} {magic or ' --':>5} {kind:>15}")
|
||||
got = {h for _, h, _, m in table if m}
|
||||
print(f"\nMiscBin entries {len(misc)} resolved {len(got)} "
|
||||
f"unresolved {len(set(misc) - got)}")
|
||||
return len(set(misc) - got) == 0
|
||||
|
||||
|
||||
def cmd_pairs(root):
|
||||
"""Each phase names a .rgn and a .col; check they describe the same volume."""
|
||||
misc, _ = resolve(root)
|
||||
f32 = lambda b, o: struct.unpack_from(">f", b, o)[0]
|
||||
u16 = lambda b, o: struct.unpack_from(">H", b, o)[0]
|
||||
pairs = collections.OrderedDict()
|
||||
for _, d in stage_phases(root):
|
||||
if d.get("MapPath") and d.get("MapMesh"):
|
||||
pairs.setdefault((d["MapPath"], d["MapMesh"]), 0)
|
||||
pairs[(d["MapPath"], d["MapMesh"])] += 1
|
||||
ok = 0
|
||||
print(f"{'stem':>30} {'phases':>7} {'bbox':>6} {'cell':>6} verts/tris")
|
||||
for (rgn, col), n in pairs.items():
|
||||
R, C = misc[name_hash(rgn)], misc[name_hash(col)]
|
||||
stem = rgn.rsplit(".", 1)[0] == col.rsplit(".", 1)[0]
|
||||
bb = all(f32(R, 0x10 + 4 * i) == f32(C, 0x10 + 4 * i)
|
||||
and f32(R, 0x20 + 4 * i) == f32(C, 0x20 + 4 * i) for i in range(3))
|
||||
cc = all(f32(R, 0x40 + 4 * i) == f32(C, 0x40 + 4 * i) for i in range(3))
|
||||
ok += stem and bb and cc
|
||||
print(f"{rgn.rsplit('.', 1)[0]:>30} {n:7d} {'yes' if bb else 'NO':>6} "
|
||||
f"{'yes' if cc else 'NO':>6} {u16(C, 0x50)}/{u16(C, 0x52)}")
|
||||
print(f"\npairs agreeing on stem, bbox and cell size: {ok}/{len(pairs)}")
|
||||
return ok == len(pairs)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2:
|
||||
raise SystemExit(__doc__)
|
||||
root = sys.argv[1]
|
||||
good = cmd_pairs(root) if "--pairs" in sys.argv else cmd_table(root)
|
||||
print("\nPASS" if good else "\nFAIL")
|
||||
sys.exit(0 if good else 1)
|
||||
Reference in New Issue
Block a user