feat(formats): decode the movie manifest's full mission/phase structure

The movie manifest (tables.pak, schema 0x067025b9) is the game's authoritative mission -> phase -> cutscene table. Previously we only scraped VOICE_/SUBTITLE_/.wmv strings by prefix; now we decode the real two-array (slot-keys, then value-records) structure.

MovieEntry gains slot, kind (System/Intro/Phase/PhaseEnd/Supply), mission, phase, subtitle, and telop (the previously-ignored .prt on-screen text overlay). Valueless slots (stages with no resupply video) are recovered without decoding the binary node region, via the fully-derivable MS<NN><X> -> S<NN><X> anchors.

Backward compatible: movie/voice_token and resolve_voice_entry/voice_token/is_manifest are unchanged. Validated end-to-end on the real disc: 101 entries (104 slots - 3 gaps), typed System=6/Intro=27/Phase=32/PhaseEnd=18/Supply=18. examples/manifest_map.rs dumps the full map.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

crates/sylpheed-formats/examples/manifest_map.rs

@@ -0,0 +1,46 @@
+//! Validation: dump the parsed mission/phase cutscene map from the real manifest.
+use sylpheed_formats::movie_manifest::{self, MovieKind};
+use sylpheed_formats::pak::PakArchive;
+
+fn main() {
+    let disc = std::env::var("SYLPHEED_DISC").expect("set SYLPHEED_DISC");
+    let arc = PakArchive::open(format!("{disc}/dat/tables.pak")).unwrap();
+    let man = arc
+        .entries()
+        .iter()
+        .find_map(|e| arc.read(e).ok().filter(|b| movie_manifest::is_manifest(b)))
+        .expect("manifest");
+    let entries = movie_manifest::parse(&man);
+    let mut counts = [0usize; 5];
+    for e in &entries {
+        counts[match e.kind {
+            MovieKind::System => 0,
+            MovieKind::Intro => 1,
+            MovieKind::Phase => 2,
+            MovieKind::PhaseEnd => 3,
+            MovieKind::Supply => 4,
+        }] += 1;
+        let m = e.mission.map(|x| x.to_string()).unwrap_or_default();
+        let p = e.phase.map(|x| x.to_string()).unwrap_or_default();
+        println!(
+            "{:<22} {:<9} m={:<2} p={:<1} {:<16} {:<14} tel={:<14} sub={}",
+            e.slot,
+            format!("{:?}", e.kind),
+            m,
+            p,
+            e.movie,
+            e.voice_token.as_deref().unwrap_or("-"),
+            e.telop.as_deref().unwrap_or("-"),
+            e.subtitle.as_deref().unwrap_or("-"),
+        );
+    }
+    eprintln!(
+        "TOTAL {} entries — System={} Intro={} Phase={} PhaseEnd={} Supply={}",
+        entries.len(),
+        counts[0],
+        counts[1],
+        counts[2],
+        counts[3],
+        counts[4]
+    );
+}

crates/sylpheed-formats/src/movie_manifest.rs

@@ -1,36 +1,81 @@
-//! The movie manifest: the game's authoritative movie → subtitle → **voice**
+//! The movie manifest: the game's authoritative **mission → phase → cutscene**
-//! index (statically reversed).
+//! table (statically reversed). Beyond the movie→voice binding, it defines the
+//! whole cutscene *structure*: which video (+ subtitle, + on-screen text overlay,
+//! + voice) plays at each mission phase, and of what kind (story intro, in-mission
+//! phase cutscene, resupply, …).
 //!
-//! `dat/tables.pak` holds one `IDXD` table (a `Z1`+zlib block) whose string pool
+//! ## On-disc structure (`dat/tables.pak`, entry hash `0x5b983a08`)
-//! lists every cutscene in play order. Each movie contributes a run of ASCII
-//! strings, always led by `<movie>.wmv` and optionally followed by:
-//!   - `<pak>+…​.prt`                      — an overlay/print-art reference,
-//!   - `<pak>+SUBTITLE_<movie>.tbl`        — its caption timing track,
-//!   - `VOICE_<token>`                     — the **voice track basename**.
 //!
-//! The voice token is the payoff: it is the real bank name to look up in
+//! One `IDXD` table (`Z1`+zlib block), schema `0x067025b9`. Unlike a flat IDXD
-//! `sounds.tbl`, and it is **not** always `VOICE_<movie>`. Most story/radio
+//! object, this one is an **array of records**, and its string pool is laid out
-//! movies use `VOICE_<movie>` (in `<lang>\Movie\`), but several `hokyu_*`
+//! as **two consecutive arrays**:
-//! (resupply) movies bind to in-mission radio clips like `VOICE_D_450` (in
+//!   1. the **slot KEYS** — `LOGO1`, `ADVERTISE_MOVIE`, `STAFF_ROLL`, then one
-//! `<lang>\etc\`), and many `hokyu_*` movies + the boot logos have **no** voice
+//!      per cutscene: `MS<NN><X>`, `STAGE<NN>_PHASE0<N>`, `STAGE<NN>_PHASE_END…`,
-//! token at all — i.e. the game plays no voice-over for them. Guessing
+//!      `S<NN>_SUPPLY_<ACROPOLIS|TANKER>`, in play order;
-//! `VOICE_<movie>` therefore both misses the `VOICE_D_*` cases and invents a
+//!   2. the **value RECORDS**, in the same order — each a run led by `<movie>.wmv`
-//! non-existent track for the silent ones; this manifest is the ground truth.
+//!      and (optionally) a `<pak>+…​.prt` overlay (field `TELOP`), a
+//!      `<pak>+SUBTITLE_<movie>.tbl` (field `SUBTITLE`), and a `VOICE_<token>`
+//!      (field `VOICETRACK`). A few slot keys are **valueless** (e.g. stages 5 and
+//!      12 have no resupply video) — those slots have no record and are skipped.
 //!
-//! We read the manifest by grouping the string pool on `.wmv` (the pool is
+//! The **slot key is the semantic role** — this is how the game separates a
-//! emitted in the same order as the records), which is robust without decoding
+//! cutscene voice from in-mission radio chatter (structurally, by which slot/bank,
-//! the `IDXD` record structure.
+//! not one boolean): `MS*` = story intro (`VOICE_S*`), `STAGE*_PHASE*` = radio-box
+//! cutscene (`VOICE_RT*`), `S*_SUPPLY_*` = resupply (`VOICE_D_45x` or none).
+//! Resupply videos are **reused across stages** (`S04_SUPPLY_ACROPOLIS` →
+//! `hokyu_DS_s02A`), so the mapping is genuinely a table, not derivable from names.
+//!
+//! The voice token is likewise **not** always `VOICE_<movie>`: most movies use
+//! `VOICE_<movie>` (in `<lang>\Movie\`), but the resupply cutscenes bind to
+//! in-mission radio clips like `VOICE_D_450` (in `<lang>\etc\`), and some have no
+//! voice at all. This manifest is the ground truth.
+//!
+//! ## Alignment
+//!
+//! Slot keys and value records would be a 1:1 positional zip but for the valueless
+//! slots. We recover the gaps without decoding the binary node/index region by
+//! using the fully-derivable **`MS<NN><X>` → `S<NN><X>` anchors**: a non-`MS` slot
+//! whose record would be an `MS` target actually belongs to the upcoming `MS`
+//! slot, so the non-`MS` slot is a gap. (Decoding the node region would make this
+//! exact and also unlock the defaulted numeric fields in other IDXD tables.)
 
 use crate::slb::VoiceLang;
 
-/// One movie's manifest row.
+/// The role a cutscene slot plays in the mission flow, from its slot key.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum MovieKind {
+    /// `LOGO*`, `ADVERTISE_MOVIE`, `STAFF_ROLL` — boot/system movies.
+    System,
+    /// `MS<NN><X>` — story intro movie (camera on the speaker; stereo `VOICE_S*`).
+    Intro,
+    /// `STAGE<NN>_PHASE0<N>` — in-mission phase cutscene (radio box; `VOICE_RT*`).
+    Phase,
+    /// `STAGE<NN>_PHASE_END[_0N]` — phase-end cutscene.
+    PhaseEnd,
+    /// `S<NN>_SUPPLY_<ACROPOLIS|TANKER>` — resupply (hokyu) cutscene.
+    Supply,
+}
+
+/// One cutscene slot's manifest row: its semantic slot plus the bound assets.
 #[derive(Debug, Clone, PartialEq, Eq)]
 pub struct MovieEntry {
-    /// Movie basename without extension, e.g. `S13A`, `hokyu_DS_s13A`.
+    /// The slot key, e.g. `STAGE01_PHASE01`, `MS00A`, `S02_SUPPLY_ACROPOLIS`.
+    /// Empty when parsed from a blob without the slot-key array (fallback).
+    pub slot: String,
+    /// The slot's role in the mission flow (intro vs. phase cutscene vs. supply …).
+    pub kind: MovieKind,
+    /// Mission/stage number (`STAGE<NN>` / `MS<NN>` / `S<NN>_SUPPLY`), if any.
+    pub mission: Option<u8>,
+    /// Phase number for [`MovieKind::Phase`]/[`MovieKind::PhaseEnd`], if any.
+    pub phase: Option<u8>,
+    /// Movie basename without extension, e.g. `S13A`, `RT01A`, `hokyu_DS_s13A`.
     pub movie: String,
-    /// Voice bank basename (e.g. `VOICE_S13A`, `VOICE_D_450`), or `None` when the
+    /// Voice bank basename (`VOICE_S13A`, `VOICE_D_450`), or `None` for no voice.
-    /// game assigns no voice-over to this movie.
     pub voice_token: Option<String>,
+    /// Subtitle table basename (`SUBTITLE_S13A.tbl`), pak prefix stripped, if any.
+    pub subtitle: Option<String>,
+    /// On-screen text-overlay (`TELOP`) basename (`pwrt01.prt`), if any.
+    pub telop: Option<String>,
 }
 
 /// Does this blob look like the movie manifest? (`IDXD` whose string pool
@@ -44,25 +89,154 @@ pub fn is_manifest(bytes: &[u8]) -> bool {
         && contains(bytes, b"SUBTITLE_")
 }
 
-/// Parse the manifest's string pool into per-movie rows, in play order.
+/// The field-type keys that appear once in the value region as a schema template.
+const FIELD_KEYS: [&str; 4] = ["MOVIE", "VOICETRACK", "SUBTITLE", "TELOP"];
+
+/// Parse the manifest into per-slot cutscene rows, in play order.
+///
+/// Each returned [`MovieEntry`] carries a movie; valueless slots (e.g. stage 5's
+/// absent resupply) are omitted. When the two-array slot/record layout is present
+/// (the real manifest) every row is tagged with its authoritative slot/kind/
+/// mission/phase; otherwise (a blob without the key array) the kind is inferred
+/// from the movie name and `slot` is empty.
 pub fn parse(bytes: &[u8]) -> Vec<MovieEntry> {
-    let mut entries: Vec<MovieEntry> = Vec::new();
+    let toks = ascii_runs(bytes, 3);
-    for run in ascii_runs(bytes, 3) {
+
-        if let Some(name) = run.strip_suffix(".wmv") {
+    // Locate the slot-key array (`LOGO1` … first `<movie>.wmv`) and the value
-            entries.push(MovieEntry {
+    // region (from the first `.wmv` onward). Fall back to a keyless scrape.
+    let first_wmv = toks.iter().position(|t| t.ends_with(".wmv"));
+    let key_start = toks.iter().position(|t| t == "LOGO1");
+    let (slot_keys, value_toks): (Vec<&str>, &[String]) = match (key_start, first_wmv) {
+        (Some(k), Some(w)) if k < w => (
+            toks[k..w].iter().map(String::as_str).collect(),
+            &toks[w..],
+        ),
+        _ => (Vec::new(), toks.as_slice()),
+    };
+
+    // Build the value records (drop the schema-template field keys).
+    let mut records: Vec<MovieEntry> = Vec::new();
+    for t in value_toks.iter().filter(|t| !FIELD_KEYS.contains(&t.as_str())) {
+        if let Some(name) = t.strip_suffix(".wmv") {
+            records.push(MovieEntry {
+                slot: String::new(),
+                kind: kind_from_movie(name),
+                mission: None,
+                phase: None,
                 movie: name.to_string(),
                 voice_token: None,
+                subtitle: None,
+                telop: None,
             });
-        } else if run.starts_with("VOICE_") {
+        } else if let Some(rec) = records.last_mut() {
-            // Belongs to the movie record currently being built.
+            if let Some(v) = t.strip_prefix("VOICE_") {
-            if let Some(last) = entries.last_mut() {
+                if rec.voice_token.is_none() {
-                if last.voice_token.is_none() {
+                    rec.voice_token = Some(format!("VOICE_{v}"));
-                    last.voice_token = Some(run.clone());
                 }
+            } else if t.ends_with(".tbl") {
+                rec.subtitle.get_or_insert_with(|| after_plus(t));
+            } else if t.ends_with(".prt") {
+                rec.telop.get_or_insert_with(|| after_plus(t));
             }
         }
     }
-    entries
+
+    // Attach slot keys to records, recovering valueless-slot gaps via MS anchors.
+    if !slot_keys.is_empty() {
+        let ms_targets: std::collections::HashSet<String> = slot_keys
+            .iter()
+            .filter_map(|k| ms_target(k))
+            .collect();
+        let mut j = 0usize;
+        for slot in &slot_keys {
+            if j >= records.len() {
+                break;
+            }
+            // A gap: this slot's "record" is actually the next MS slot's movie.
+            let would_be = &records[j].movie;
+            let is_gap = match ms_target(slot) {
+                // MS slot: only binds if its record is the expected `S<NN><X>`.
+                Some(expected) => *would_be != expected,
+                // Non-MS slot: a gap iff the record belongs to an upcoming MS slot.
+                None => ms_targets.contains(would_be),
+            };
+            if is_gap {
+                continue;
+            }
+            let (kind, mission, phase) = classify_slot(slot);
+            let rec = &mut records[j];
+            rec.slot = (*slot).to_string();
+            rec.kind = kind;
+            rec.mission = mission;
+            rec.phase = phase;
+            j += 1;
+        }
+    }
+
+    records
+}
+
+/// If `slot` is an `MS<NN><X>` intro slot, the movie it must bind to (`S<NN><X>`).
+fn ms_target(slot: &str) -> Option<String> {
+    let rest = slot.strip_prefix("MS")?;
+    // `<NN><X>`: two digits then a letter suffix.
+    if rest.len() >= 3 && rest[..2].bytes().all(|b| b.is_ascii_digit()) {
+        Some(format!("S{rest}"))
+    } else {
+        None
+    }
+}
+
+/// Semantic role + mission/phase from a slot key.
+fn classify_slot(slot: &str) -> (MovieKind, Option<u8>, Option<u8>) {
+    if let Some(rest) = slot.strip_prefix("MS") {
+        if rest.len() >= 2 {
+            return (MovieKind::Intro, rest[..2].parse().ok(), None);
+        }
+    }
+    if let Some(rest) = slot.strip_prefix("STAGE") {
+        let nn: String = rest.chars().take_while(|c| c.is_ascii_digit()).collect();
+        if let Ok(m) = nn.parse::<u8>() {
+            if slot.contains("PHASE_END") {
+                // trailing `_NN` if present (`STAGE12_PHASE_END_02`)
+                let phase = slot.rsplit('_').next().and_then(|x| x.parse().ok());
+                return (MovieKind::PhaseEnd, Some(m), phase);
+            }
+            // `STAGE<NN>_PHASE0<N>`
+            let phase = rest[nn.len()..]
+                .strip_prefix("_PHASE")
+                .and_then(|p| p.parse().ok());
+            return (MovieKind::Phase, Some(m), phase);
+        }
+    }
+    if slot.contains("_SUPPLY") {
+        if let Some(rest) = slot.strip_prefix('S') {
+            let nn: String = rest.chars().take_while(|c| c.is_ascii_digit()).collect();
+            return (MovieKind::Supply, nn.parse().ok(), None);
+        }
+    }
+    (MovieKind::System, None, None)
+}
+
+/// Fallback kind from a movie basename (used when no slot-key array is present).
+fn kind_from_movie(movie: &str) -> MovieKind {
+    if movie.starts_with("hokyu_") {
+        MovieKind::Supply
+    } else if movie.starts_with("RT") {
+        MovieKind::Phase
+    } else if movie.len() >= 3
+        && movie.starts_with('S')
+        && movie[1..3].bytes().all(|b| b.is_ascii_digit())
+    {
+        MovieKind::Intro
+    } else {
+        MovieKind::System
+    }
+}
+
+/// The part of a `<pak>+<name>` reference after the `+` (or the whole string).
+fn after_plus(s: &str) -> String {
+    s.rsplit('+').next().unwrap_or(s).to_string()
 }
 
 /// The voice bank basename bound to `movie`, if the manifest lists one.
@@ -151,12 +325,82 @@ mod tests {
 
     #[test]
     fn groups_movies_and_binds_voice() {
+        // Keyless fallback path: movie + voice + subtitle + inferred kind.
         let m = parse(&synth());
         assert_eq!(m.len(), 3);
         assert_eq!(m[0].movie, "S13A");
         assert_eq!(m[0].voice_token.as_deref(), Some("VOICE_S13A"));
+        assert_eq!(m[0].kind, MovieKind::Intro);
+        assert_eq!(m[0].subtitle.as_deref(), Some("SUBTITLE_S13A.tbl"));
         assert_eq!(m[1].voice_token.as_deref(), Some("VOICE_D_450"));
+        assert_eq!(m[1].kind, MovieKind::Supply);
         assert_eq!(m[2].voice_token, None, "hokyu_DS_s13A has no voice-over");
+        assert_eq!(m[2].kind, MovieKind::Supply);
+    }
+
+    /// Build a two-array (keys-then-records) manifest like the real one.
+    fn synth_two_array(keys: &[&str], records: &[&[&str]]) -> Vec<u8> {
+        let mut b = b"IDXD".to_vec();
+        b.extend_from_slice(&[0, 0, 0, 0x05, 0, 0]);
+        for k in keys {
+            b.extend_from_slice(k.as_bytes());
+            b.push(0);
+        }
+        for rec in records {
+            for s in *rec {
+                b.extend_from_slice(s.as_bytes());
+                b.push(0);
+            }
+        }
+        b
+    }
+
+    #[test]
+    fn two_array_slots_kind_mission_phase() {
+        let blob = synth_two_array(
+            &["LOGO1", "MS01A", "STAGE01_PHASE01", "STAGE01_PHASE_END_02"],
+            &[
+                &["logo1.wmv", "MOVIE"],
+                &["S01A.wmv", "eng.pak+SUBTITLE_S01A.tbl", "VOICE_S01A"],
+                &["RT01A.wmv", "eng.pak+pwrt01.prt", "VOICE_RT01A", "VOICETRACK"],
+                &["RT01C_2.wmv", "VOICE_RT01C_2"],
+            ],
+        );
+        let m = parse(&blob);
+        assert_eq!(m.len(), 4);
+        assert_eq!((m[0].slot.as_str(), m[0].kind), ("LOGO1", MovieKind::System));
+        assert_eq!(
+            (m[1].slot.as_str(), m[1].kind, m[1].mission),
+            ("MS01A", MovieKind::Intro, Some(1))
+        );
+        assert_eq!(
+            (m[2].slot.as_str(), m[2].kind, m[2].mission, m[2].phase),
+            ("STAGE01_PHASE01", MovieKind::Phase, Some(1), Some(1))
+        );
+        assert_eq!(m[2].telop.as_deref(), Some("pwrt01.prt"));
+        assert_eq!(
+            (m[3].slot.as_str(), m[3].kind, m[3].phase),
+            ("STAGE01_PHASE_END_02", MovieKind::PhaseEnd, Some(2))
+        );
+    }
+
+    #[test]
+    fn valueless_supply_slot_is_skipped_via_ms_anchor() {
+        // S05_SUPPLY has no record; its "record" (S06A) belongs to the next MS slot.
+        let blob = synth_two_array(
+            &["LOGO1", "MS05A", "S05_SUPPLY_ACROPOLIS", "MS06A"],
+            &[
+                &["logo1.wmv"],
+                &["S05A.wmv", "VOICE_S05A"],
+                &["S06A.wmv", "VOICE_S06A"],
+            ],
+        );
+        let m = parse(&blob);
+        assert_eq!(m.len(), 3, "the valueless supply slot yields no entry");
+        assert_eq!((m[1].slot.as_str(), m[1].movie.as_str()), ("MS05A", "S05A"));
+        // S06A binds to MS06A, NOT to the intervening valueless supply slot.
+        assert_eq!((m[2].slot.as_str(), m[2].movie.as_str()), ("MS06A", "S06A"));
+        assert!(m.iter().all(|e| e.slot != "S05_SUPPLY_ACROPOLIS"));
     }
 
     #[test]