From 5a54e447265ee9f437d406a7b5943ad112d15acf Mon Sep 17 00:00:00 2001 From: Sylpheed RE agent Date: Fri, 28 Aug 2026 08:22:36 +0000 Subject: [PATCH] =?UTF-8?q?re:=20the=20disc=20ships=20the=20unit=20datashe?= =?UTF-8?q?et's=20own=20SCHEMA=20(and=20=E6=96=87=E5=AD=97=E5=88=97=20was?= =?UTF-8?q?=20not=20a=20placeholder)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This corrects the previous commit. 文字列 is not a developer's leftover: it is one member of a six-word Shift-JIS TYPE vocabulary, and the records carrying it are a machine-readable schema for the unit datasheet. The whole non-ASCII population on the disc is 6 distinct values out of 99328 - 0 of 3496 record names and 0 of 12173 field names - and all six are type words: 文字列 string 366 uses NS_"文字列" NS_ string 12 整数 / 整数値 integer 66 / 6 浮動小数値 floating-point value 504 浮動小数値[0〜1] float in [0,1] 36 990 type-valued fields. So the reader defect noted last time is real but bounded to these six strings, and name_hash re-encodes Latin-1 byte-for-byte, so hashing was never affected. They sit in 15 records x 6 GP_MAIN_GAME_* paks = 90 instances, i.e. 15 records with ONE user. The names are exactly the unit substructure family, and six carry a literal wildcard: Turret_???, Hatch_???, Bridge_???, Thruster_???, ShieldGenerator_???, Versatile_???, and NS_*. ??? is the numeric-suffix wildcard at record AND field level - Turret_??? is the schema for Turret_000..00N, and inside it CannonFrame_??? / MuzzleFrame_??? stand for the numbered slots. Where a field's type is an enumeration the schema holds an EXAMPLE value instead of a type name: Yes for the five booleans, Vessel for Generic.Type (the 43 Craft + 71 Vessel split), Ship_ for the ID prefix convention. Maneuver is the one fully-typed record, 34 of 34. Every ResistanceTo* and every Color_* channel is declared FLOAT[0..1] - normalised by declaration, matching the sampled values in unit-datasheet-static. Generic.NozzleSpec_??? has its own type NS_"文字列" and NS_* is a record, so the nozzle spec is a nested sub-schema. Control separates schema from data cleanly: the _??? records and NS_* exist ONLY as schema, 6 of 6 instances typed, while the eight real substructure names are typed in 6 instances and untyped in the rest - Generic 6 of 3651, the others 6 of 684 each. Turret_??? carries the game's own typo NomalModel beside DamagedModel. This gives the port an authoritative field-type table: types the disc declares, rather than types inferred from sampled values. New artefact with its regenerator: tools/re-capture/datasheet_schema.py -> docs/re/data/datasheet-schema.txt. All fifteen existing artefacts byte-identical. --- docs/re/BACKLOG.md | 31 +++ docs/re/data/datasheet-schema.txt | 228 ++++++++++++++++++++ docs/re/structures/unit-datasheet-schema.md | 83 +++++++ tools/re-capture/datasheet_schema.py | 56 +++++ 4 files changed, 398 insertions(+) create mode 100644 docs/re/data/datasheet-schema.txt create mode 100644 docs/re/structures/unit-datasheet-schema.md create mode 100644 tools/re-capture/datasheet_schema.py diff --git a/docs/re/BACKLOG.md b/docs/re/BACKLOG.md index bfd699f5..d0b6e096 100644 --- a/docs/re/BACKLOG.md +++ b/docs/re/BACKLOG.md @@ -6618,3 +6618,34 @@ conventions here are worth testing precisely because they are not all real. ⚠️ Controls reproduced: `eff_n0071` declared + used 6× (= one user); `eff_e0033` declared + used 0×. The 318-cell is expected — the LOD family is overwhelmingly *equipment*, which no unit datasheet binds. + +--- + +## ✅ 2026-08-28 — the disc ships the unit datasheet's own SCHEMA + +Item (a), and it **corrects the previous entry**: `文字列` is not a developer's +placeholder. It is one member of a six-word Shift-JIS *type* vocabulary, and the +records holding it are a machine-readable schema. Written up in +[unit-datasheet-schema](structures/unit-datasheet-schema.md). + +The whole non-ASCII population is **6 distinct values of 99 328** — 0 of 3 496 +record names and 0 of 12 173 field names — all Shift-JIS type words: `文字列` +(string), `NS_"文字列"`, `整数`/`整数値` (integer), `浮動小数値` (float), +`浮動小数値[0〜1]` (float in [0,1]). **990 type-valued fields.** The reader +defect is real but bounded to these six strings; `name_hash` re-encodes Latin-1 +byte-for-byte, so hashing was never affected. + +They sit in **15 records × 6 `GP_MAIN_GAME_*` paks = one user**, named for the +substructure family, six of them carrying a literal wildcard: `Turret_???`, +`Hatch_???`, `Bridge_???`, `Thruster_???`, `ShieldGenerator_???`, +`Versatile_???`, `NS_*`. 🔑 **`???` is the numeric-suffix wildcard** at record +*and* field level. Where a field's type is an enum the schema holds an **example +value** instead: `Yes` for the five booleans, `Vessel` for `Generic.Type`, +`Ship_` for the `ID` prefix. Every `ResistanceTo*` and `Color_*` is declared +`FLOAT[0..1]`. + +⚠️ Control: the `_???` records exist ONLY as schema (6 of 6 typed); the eight +real names are typed in 6 instances and untyped in the rest (`Generic` 6 of +3 651, the others 6 of 684). + +New artefact: `tools/re-capture/datasheet_schema.py` → `docs/re/data/datasheet-schema.txt`. diff --git a/docs/re/data/datasheet-schema.txt b/docs/re/data/datasheet-schema.txt new file mode 100644 index 00000000..ae2960e4 --- /dev/null +++ b/docs/re/data/datasheet-schema.txt @@ -0,0 +1,228 @@ +the unit datasheet schema shipped in GP_MAIN_GAME_*.pak +15 schema records; identical in all six paks (x6 = one user) + +=== Bridge_??? (16 fields, 12 typed) === + DamagedModel STRING + Effect_Explosion STRING + Effect_Flare STRING + Frame STRING + HP FLOAT + ID STRING + IsDestructible example: 'Yes' + IsInvolved example: 'Yes' + IsRadarVisible example: 'Yes' + IsShielded example: 'Yes' + Name STRING + NomalModel STRING + ParentStructureID STRING + PowerRatio FLOAT + Radius FLOAT + SpreadDamage FLOAT + +=== Effect (2 fields, 2 typed) === + Shield STRING + Thruster STRING + +=== Explosion (2 fields, 2 typed) === + Delay FLOAT + DelayAdjustment FLOAT + +=== Generic (23 fields, 20 typed) === + Color_B FLOAT[0..1] + Color_G FLOAT[0..1] + Color_R FLOAT[0..1] + HP FLOAT + HQRatio FLOAT + ID example: 'Ship_' + IsDestructible example: 'Yes' + Model STRING + Name STRING + NozzleCount INT + NozzleFrame_??? STRING + NozzleSpec_??? NS_STRING + ResistanceToExplosion FLOAT[0..1] + ResistanceToOptics FLOAT[0..1] + ResistanceToShell FLOAT[0..1] + ShieldRatio FLOAT + Size_Radius FLOAT + Size_X FLOAT + Size_Y FLOAT + Size_Z FLOAT + ThrusterRatio FLOAT + Type example: 'Vessel' + UncertainName STRING + +=== Hatch_??? (19 fields, 15 typed) === + DamagedModel STRING + Effect_Explosion STRING + Effect_Flare STRING + Frame STRING + HP FLOAT + ID STRING + IsDestructible example: 'Yes' + IsInvolved example: 'Yes' + IsRadarVisible example: 'Yes' + IsShielded example: 'Yes' + LoadedCount INT + MaxAvailableCount INT + Name STRING + NomalModel STRING + ParentStructureID STRING + Radius FLOAT + SpreadDamage FLOAT + SquadronID STRING + TakeoffInterval FLOAT + +=== Maneuver (34 fields, 34 typed) === + AA_PitchMinus_Max FLOAT + AA_PitchMinus_Min FLOAT + AA_PitchPlus_Max FLOAT + AA_PitchPlus_Min FLOAT + AA_Roll_Max FLOAT + AA_Roll_Min FLOAT + AA_Yaw_Max FLOAT + AA_Yaw_Min FLOAT + AV_PitchMinus_Max FLOAT + AV_PitchMinus_Min FLOAT + AV_PitchPlus_Max FLOAT + AV_PitchPlus_Min FLOAT + AV_Roll_Max FLOAT + AV_Roll_Min FLOAT + AV_Yaw_Max FLOAT + AV_Yaw_Min FLOAT + AccPitchFactor FLOAT + Acceleration FLOAT + ArterBurner_Acc FLOAT + ArterBurner_Vc FLOAT + CruisingVelocity FLOAT + DecPitchFactor FLOAT + Deceleration FLOAT + DragFactorThreshold FLOAT + MaximumBank_Normal FLOAT + MaximumVelocity FLOAT + MinimumVelocity FLOAT + PitchDragFactor FLOAT + ReverseThrust_Acc FLOAT + ReverseThrust_Vc FLOAT + RollDragFactor FLOAT + SideThrustAcceleration FLOAT + SideThrustVelocity_Max FLOAT + YawDragFactor FLOAT + +=== Mass (2 fields, 2 typed) === + DryMass FLOAT + GrossMass FLOAT + +=== NS_* (10 fields, 8 typed) === + AttenuationAlpha example: 'Yes' + AttenuationVolume example: 'Yes' + Color_B FLOAT + Color_G FLOAT + Color_R FLOAT + Deceleration FLOAT + DeleteFadeSpeed FLOAT + Interval FLOAT + NodeCount INT_VALUE + StartingVelocity FLOAT + +=== SE (2 fields, 2 typed) === + ExplosionSE STRING + Thruster STRING + +=== Shield (4 fields, 4 typed) === + ChargeDelay FLOAT + ChargeDelay_Break FLOAT + ChargeSpeed FLOAT + MaxValue FLOAT + +=== ShieldGenerator_??? (16 fields, 12 typed) === + DamagedModel STRING + Effect_Explosion STRING + Effect_Flare STRING + Frame STRING + HP FLOAT + ID STRING + IsDestructible example: 'Yes' + IsInvolved example: 'Yes' + IsRadarVisible example: 'Yes' + IsShielded example: 'Yes' + Name STRING + NomalModel STRING + ParentStructureID STRING + PowerRatio FLOAT + Radius FLOAT + SpreadDamage FLOAT + +=== StructureCount (6 fields, 6 typed) === + BridgeCount INT + HatchCount INT + ShieldGeneratorCount INT + ThrusterCount INT + TurretCount INT + VersatileCount INT + +=== Thruster_??? (19 fields, 15 typed) === + DamagedModel STRING + Effect_Explosion STRING + Effect_Flare STRING + Frame STRING + HP FLOAT + ID STRING + IsDestructible example: 'Yes' + IsInvolved example: 'Yes' + IsRadarVisible example: 'Yes' + IsShielded example: 'Yes' + Name STRING + NomalModel STRING + NozzleCount INT + NozzleFrame_??? STRING + NozzleSpec_??? NS_STRING + ParentStructureID STRING + PowerRatio FLOAT + Radius FLOAT + SpreadDamage FLOAT + +=== Turret_??? (25 fields, 20 typed) === + AngularVelocity FLOAT + CannonFrame_??? STRING + DamagedModel STRING + Effect_Explosion STRING + Effect_Flare STRING + Frame STRING + HP FLOAT + ID STRING + IsAuto example: 'Yes' + IsDestructible example: 'Yes' + IsInvolved example: 'Yes' + IsRadarVisible example: 'Yes' + IsShielded example: 'Yes' + MuzzleFrame_??? STRING + Name STRING + NomalModel STRING + ParentStructureID STRING + PitchLimit_Depression FLOAT + PitchLimit_Elevation FLOAT + Radius FLOAT + SequencingCount INT + SequencingInterval FLOAT + SpreadDamage FLOAT + WeaponID STRING + YawLimit FLOAT + +=== Versatile_??? (15 fields, 11 typed) === + DamagedModel STRING + Effect_Explosion STRING + Effect_Flare STRING + Frame STRING + HP FLOAT + ID STRING + IsDestructible example: 'Yes' + IsInvolved example: 'Yes' + IsRadarVisible example: 'Yes' + IsShielded example: 'Yes' + Name STRING + NomalModel STRING + ParentStructureID STRING + Radius FLOAT + SpreadDamage FLOAT + diff --git a/docs/re/structures/unit-datasheet-schema.md b/docs/re/structures/unit-datasheet-schema.md new file mode 100644 index 00000000..8dc679b3 --- /dev/null +++ b/docs/re/structures/unit-datasheet-schema.md @@ -0,0 +1,83 @@ +# ✅ The unit datasheet ships its own schema + +Chasing the previous iteration's Shift-JIS value turned up something much larger +than the placeholder I called it. **Correction to that entry: `文字列` is not a +developer's leftover — it is one member of a six-word *type* vocabulary, and the +records carrying it are a machine-readable schema for the unit datasheet.** + +## The whole non-ASCII population is six words + +Across every parsed IDXD string on the disc: + +| | distinct | non-ASCII | +|---|---|---| +| values | 99 328 | **6** | +| record names | 3 496 | 0 | +| field names | 12 173 | 0 | + +All six decode cleanly as Shift-JIS, and all six are type names: + +| bytes | Shift-JIS | meaning | uses | +|---|---|---|---| +| `95B68E9A97F1` | `文字列` | string | 366 | +| `4E535F22…22` | `NS_"文字列"` | NS_ string | 12 | +| `90AE9094` | `整数` | integer | 66 | +| `90AE9094926C` | `整数値` | integer value | 6 | +| `958293AE8FAC9094926C` | `浮動小数値` | floating-point value | 504 | +| `…5B308160315D` | `浮動小数値[0〜1]` | float in [0,1] | 36 | + +990 type-valued fields in total. So the reader defect noted last time is real but +**bounded to these six strings** — nothing else on the disc is non-ASCII, and +`name_hash` re-encodes Latin-1 byte-for-byte, so hashing was never affected. + +## 15 wildcard schema records, one per substructure + +The 990 fields sit in **15 records × 6 `GP_MAIN_GAME_*` paks = 90 instances**, +i.e. **15 records with one user** under the per-pak-copy rule. Their names are +exactly the unit substructure family — and six of them carry a literal wildcard: + +`Generic`, `Maneuver`, `Effect`, `Shield`, `Mass`, `SE`, `Explosion`, +`StructureCount`, plus `Turret_???`, `Hatch_???`, `Bridge_???`, `Thruster_???`, +`ShieldGenerator_???`, `Versatile_???` and `NS_*`. + +🔑 **`???` is the numeric-suffix wildcard**, at record *and* field level: +`Turret_???` is the schema for `Turret_000…00N`, and inside it +`CannonFrame_???` / `MuzzleFrame_???` stand for the numbered frame slots. + +⚠️ The control separates schema from data cleanly. The six `_???` records and +`NS_*` exist **only** as schema — 6 of 6 instances typed. The eight real +substructure names are typed in 6 instances and untyped in the rest: +`Generic` 6 of 3 651, `Maneuver`/`Shield`/`Mass`/`SE`/`Explosion`/`Effect`/ +`StructureCount` 6 of 684 each. A schema record is one extra instance of a name +the datasheets already use. + +## Where the type is an enum, the schema holds an example + +Not every field is typed — `Generic` is 20 typed of 23, `Turret_???` 20 of 25. +The untyped slots hold a **sample value** instead, which names the enum: + +| field | schema slot | reading | +|---|---|---| +| `IsDestructible`, `IsAuto`, `IsShielded`, `IsRadarVisible`, `IsInvolved` | `Yes` | boolean | +| `Generic.Type` | `Vessel` | the Craft/Vessel enum (43 + 71) | +| `Generic.ID` | `Ship_` | the ID *prefix* convention | + +`Maneuver` is the one fully-typed record: **34 of 34**. + +Two readings worth keeping for the port: + +* every `ResistanceTo*` (`Shell`, `Explosion`, `Optics`) and every `Color_*` + channel is **`FLOAT[0..1]`** — declared normalised, matching the sampled + values (`ResistanceToShell 0.1`, `ResistanceToExplosion 0.5`) in + [[unit-datasheet-static]]; +* `Generic.NozzleSpec_???` has its own type `NS_"文字列"`, and `NS_*` is a + record — the nozzle spec is a nested sub-schema, not a plain string. + +⚠️ `Turret_???` carries the game's own typo **`NomalModel`** (beside +`DamagedModel`), which is a join key in the usual way. + +This is the authoritative field-type table for the unit datasheet: types the +disc declares, rather than types inferred from sampled values. + +Regenerate with `tools/re-capture/datasheet_schema.py` → +[`docs/re/data/datasheet-schema.txt`](../data/datasheet-schema.txt). diff --git a/tools/re-capture/datasheet_schema.py b/tools/re-capture/datasheet_schema.py new file mode 100644 index 00000000..e30a23c9 --- /dev/null +++ b/tools/re-capture/datasheet_schema.py @@ -0,0 +1,56 @@ +#!/usr/bin/env python3 +"""The unit datasheet's own schema, as shipped on the disc. + +Each GP_MAIN_GAME_*.pak carries 15 IDXD records whose field VALUES are Japanese +type names rather than data -- Shift-JIS, and the only non-ASCII strings on the +whole disc. Records whose name ends '_???' (and 'NS_*') are wildcards standing +for the numbered family, e.g. Turret_??? for Turret_000..Turret_00N. + +Where a field's type is an enumeration the schema holds an EXAMPLE value +instead of a type name ('Yes' for a boolean, 'Vessel' for Generic.Type). + +Regenerates docs/re/data/datasheet-schema.txt. +""" +import glob, os, sys +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from unit_substructures import pak_entries +import unitgroup as U + +# the six type names, held as latin-1 (how unitgroup.py hands strings back) +TYPES = { + '\x95\xb6\x8e\x9a\x97\xf1': 'STRING', + 'NS_"\x95\xb6\x8e\x9a\x97\xf1"': 'NS_STRING', + '\x90\xae\x90\x94': 'INT', + '\x90\xae\x90\x94\x92l': 'INT_VALUE', + '\x95\x82\x93\xae\x8f\xac\x90\x94\x92l': 'FLOAT', + '\x95\x82\x93\xae\x8f\xac\x90\x94\x92l[0\x81`1]': 'FLOAT[0..1]', +} + +def main(): + pak = sorted(glob.glob('/work/sylph_extract/**/GP_MAIN_GAME_D.pak', recursive=True)) + if not pak: + print('disc not mounted; nothing to do'); return + out = {} + for h, b in pak_entries(pak[0]): + if b[:4] != b'IDXD': + continue + try: + recs = U.parse(b) + except Exception: + continue + for r in recs: + if any(v in TYPES for t, f, v in r['fields']): + out[r['squadron']] = r['fields'] + print('the unit datasheet schema shipped in GP_MAIN_GAME_*.pak') + print('%d schema records; identical in all six paks (x6 = one user)' % len(out)) + print() + for name in sorted(out): + fields = out[name] + typed = sum(1 for t, f, v in fields if v in TYPES) + print('=== %s (%d fields, %d typed) ===' % (name, len(fields), typed)) + for t, f, v in sorted(fields, key=lambda x: (x[1] or '')): + print(' %-28s %s' % (f, TYPES.get(v, 'example: ' + repr(v)))) + print() + +if __name__ == '__main__': + main()