re: locate both guest hash routines; IXUD solved; two corrections

Found the routines in the disassembly DB rather than guessing from data: sub_82447DF0 IDXD tag hash (lbz+extsb, modulus 0x00FFFFDF, magic 0x2101) sub_82447E70 IXUD tag hash (lhz, 64-bit, modulus 0xFFFFFF67 then 0x00FFFFDF) Both transcribed instruction-for-instruction into Python and Rust. IXUD SOLVED. It defeated every single-modulus search because it chains TWO exact moduli -- the loop reduces mod 2^32-153 in 64-bit arithmetic and only the result is folded mod 2^24-33. A polynomial mod M1 folded through M2 is not a polynomial mod anything, which is exactly why the gcd test returned 1. Verified independently: 86/86 record keys and 108,261/108,261 field tags in GP_MAIN_GAME_E.pak, and NoRecord -> 0x1c6d9c96. CORRECTION 1: tag_hash must SIGN-EXTEND each byte (extsb). My reconstruction used unsigned bytes and matched all 1.27M disc names -- every one is ASCII -- while disagreeing on ~90% of random inputs with a byte >= 0x80 (verified: 18096/20000). The disc could never have caught this; only the disassembly did. CORRECTION 2: name_hash's reduction is EXACT, not lossy. The module doc claimed the missing conditional subtract made it something other than %. rlwinm r6,r6, 9,23,31 is just hi>>23, and with RECIP = floor(2^55/M)+1 that is Granlund- Montgomery magic division -- 0 wrong at every quotient boundary across the full 32-bit domain. Retracted. cargo test -p sylpheed-formats --lib hash: 10/10.
2026-08-25 11:28:35 +00:00
parent 63849c2fde
commit 3cdf2b24b2
7 changed files with 293 additions and 54 deletions
--- a/crates/sylpheed-formats/src/hash.rs
+++ b/crates/sylpheed-formats/src/hash.rs
@@ -20,9 +20,14 @@
 //! i.e. the low 24 bits are a modular polynomial hash and the top byte is an
 //! 8-bit additive checksum of the bytes. The reduction constant `0x8003_1493`
 //! is the reciprocal of the modulus `0x00FF_F9D7` used by the `mulhwu`/`mullw`
-//! Barrett step; there is **no** trailing conditional subtract, so the value is
-//! defined by the exact op sequence (faithfully reproduced below), not by a
-//! textbook `%`.
+//! Barrett step.
+//!
+//! **The reduction is EXACT, not lossy.** An earlier version of this note said
+//! the missing trailing conditional subtract made it something other than `%`.
+//! It does not: `rlwinm r6,r6,9,23,31` is exactly `hi >> 23`, and with
+//! `RECIP == floor(2^55/M) + 1` that is standard Granlund–Montgomery magic
+//! division. Checked at every quotient boundary (`k·M−1, k·M, k·M+1`) across the
+//! whole 32-bit domain: 0 wrong of 770. So the low 24 bits really are `A % M`.
 //!
 //! Verified against the real disc: `name_hash("files.tbl") == 0x8342_1153`
 //! and `name_hash("eng\\weapon.tbl") == 0x900C_8DCD`, both of which are present
@@ -152,43 +157,82 @@ mod tests {
 }

 /// Barrett modulus of the **record/field tag** hash — a different constant from
-/// [`MODULUS`], recovered separately (see below).
-const TAG_MODULUS: u32 = 0x00FF_FFDF; // 2^24 - 33
+/// [`MODULUS`], and the same one the IXUD hash folds down to.
+const TAG_MODULUS: u32 = 0x00FF_FFDF; // 2^24 - 33, prime
+/// The guest's divide magic for [`TAG_MODULUS`] (`floor(2^56/M) + 1`).
+const TAG_MAGIC: u32 = 0x2101;
+/// The IXUD loop modulus, applied in 64-bit arithmetic before [`TAG_MODULUS`].
+const IXUD_M1: u64 = 0xFFFF_FF67; // 2^32 - 153

-/// Hash an IDXD **record key / field tag**.
+/// Hash an IDXD **record key / field tag** — `sub_82447DF0`.
 ///
-/// This is *not* [`name_hash`]. IDXD tables key their records and name their
-/// fields with the same shape of hash — an 8-bit additive checksum in the top
-/// byte over a 24-bit modular polynomial — but with two differences:
+/// This is *not* [`name_hash`]. Same shape — an 8-bit additive checksum over a
+/// 24-bit modular polynomial — but two constants differ:
 ///
-/// * the modulus is `0x00FF_FFDF` (= 2^24 − 33, prime), not `0x00FF_F9D7`;
-/// * the bytes are **not** lowercased, so tags are case-sensitive.
+/// * modulus `0x00FF_FFDF` (2^24 − 33, prime), not `0x00FF_F9D7`;
+/// * **no lowercasing**, so tags are case-sensitive. The disc relies on this:
+///   17 name pairs differ only in case (`UNIT`/`Unit`, `TYPE`/`Type`, …) and
+///   `name_hash` collides on every one of them.
 ///
-/// Recovered empirically rather than from the executable. Every IDXD record in
-/// `GP_MAIN_GAME_E.pak` that carries an inline field name gives a known
-/// (name → tag) pair; there are **8643** such pairs, all with distinct names,
-/// and `name_hash` explains none of them. Comparing pairs of names differing in
-/// a single character yields the per-position weights `1, 0x100, 0x10000,
-/// 0x21, 0x2100, 0x210000, 0x441, …` — i.e. a base-256 polynomial in which
-/// shifting a byte out of bit 24 re-enters as `33`, which is reduction modulo
-/// `2^24 − 33`. The top byte is the plain sum of the bytes, exactly as in
-/// `name_hash` (8643/8643).
+/// A record's key is the tag of its **own** name — 190,782/190,782 records
+/// disc-wide — so records are addressable by name without reading a roster.
 ///
-/// This closes the IDXD record key: a record's key is the tag of its **name**,
-/// which each table also lists in an in-table roster record.
-///
-/// ⚠️ Implemented with exact modular arithmetic. The guest routine has **not**
-/// been located, so if it uses a Barrett step without a final fixup — as
-/// `sub_82455C78` does — there could be inputs where the two disagree. All 8643
-/// known pairs agree; nothing beyond them has been checked.
+/// The guest **sign-extends** each byte (`extsb`), and that is load-bearing:
+/// a version of this using unsigned bytes matched all 1.27M disc names, because
+/// every one is ASCII, while disagreeing on ~90% of random inputs containing a
+/// byte ≥ 0x80. Only the disassembly could catch that.
 pub fn tag_hash(name: &str) -> u32 {
-    let mut lo: u32 = 0;
-    let mut sum: u32 = 0;
-    for &byte in name.as_bytes() {
-        lo = ((lo as u64 * 256 + byte as u64) % TAG_MODULUS as u64) as u32;
-        sum = sum.wrapping_add(byte as u32);
+    tag_hash_bytes(name.as_bytes())
+}
+
+/// [`tag_hash`] over raw bytes — the form that can express a non-UTF-8 name, and
+/// the only way to exercise the `extsb` path.
+pub fn tag_hash_bytes(bytes: &[u8]) -> u32 {
+    let mut a: u32 = 0;
+    let mut b: u32 = 0;
+    for &byte in bytes {
+        let c = byte as i8 as i32 as u32; // extsb
+        a = (a << 8).wrapping_add(c);
+        b = b.wrapping_add(c);
+        // Exact magic division by TAG_MODULUS, in the guest's add-correction form.
+        let hi = ((a as u64 * TAG_MAGIC as u64) >> 32) as u32;
+        let q = hi.wrapping_add(a.wrapping_sub(hi) >> 1) >> 23;
+        a = a.wrapping_sub(q.wrapping_mul(TAG_MODULUS));
    }
-    ((sum & 0xFF) << 24) | (lo & 0x00FF_FFFF)
+    ((b << 24) & 0xFF00_0000) | (a & 0x00FF_FFFF)
+}
+
+/// Hash an **IXUD** record key / field tag — `sub_82447E70`.
+///
+/// IXUD is IDXD's wide-string sibling: identical container layout, but strings
+/// are UTF-16BE and `strsize` and every string offset are counted in **16-bit
+/// characters, not bytes** (`STR + 2·strsize == filesize`).
+///
+/// `units` is the name as big-endian UTF-16 code units.
+///
+/// It resisted every single-modulus search because it chains **two** exact
+/// moduli: the loop reduces mod `2^32 − 153` in 64-bit arithmetic, and only the
+/// final value is folded into 24 bits mod `2^24 − 33`. A polynomial mod `M1`
+/// folded through `M2` is not a polynomial mod anything, which is why a gcd over
+/// the observed pairs returns 1 and a Barrett sweep finds nothing.
+///
+/// The checksum byte sums the **full 16-bit code units**, not their low bytes —
+/// indistinguishable on this disc, where every IXUD name is ASCII, but not in
+/// general.
+pub fn ixud_hash(units: &[u16]) -> u32 {
+    let mut a: u64 = 0;
+    let mut b: u32 = 0;
+    for &ch in units {
+        a = ((a << 16) + ch as u64) % IXUD_M1;
+        b = b.wrapping_add(ch as u32);
+    }
+    ((b & 0xFF) << 24) | (a % TAG_MODULUS as u64) as u32
+}
+
+/// Convenience: [`ixud_hash`] for an ASCII/UTF-8 name.
+pub fn ixud_hash_str(name: &str) -> u32 {
+    let units: Vec<u16> = name.encode_utf16().collect();
+    ixud_hash(&units)
 }

 #[cfg(test)]
@@ -207,6 +251,22 @@ mod tag_tests {
        assert_eq!(tag_hash("Formation_ADAN_Turret07_30"), 0x30CE_86BE);
    }

+    #[test]
+    fn ixud_uses_a_different_hash_entirely() {
+        // Verified against real IXUD data: 86/86 record keys and 108,261/108,261
+        // field tags in GP_MAIN_GAME_E.pak.
+        assert_eq!(super::ixud_hash_str("NoRecord"), 0x1C6D_9C96);
+        assert_ne!(super::ixud_hash_str("NoRecord"), tag_hash("NoRecord"));
+    }
+
+    #[test]
+    fn tag_hash_sign_extends_high_bytes() {
+        // The guest uses extsb. Unsigned bytes agree on all-ASCII names but not
+        // here -- this input is the concrete counterexample.
+        let bytes = [0x4eu8, 0x3f, 0xcf, 0xa5, 0x0c, 0x86, 0x4c, 0x2b, 0x41, 0xcf];
+        assert_eq!(super::tag_hash_bytes(&bytes), 0x1AFF_849A);
+    }
+
    #[test]
    fn tags_are_case_sensitive_unlike_name_hash() {
        // name_hash lowercases first; tag_hash must not.
--- a/tools/re-capture/dialog_up.py
+++ b/tools/re-capture/dialog_up.py
@@ -0,0 +1,48 @@
+#!/usr/bin/env python3
+"""Is a modal YES/NO dialog on screen?
+
+Written because `wait_screen.sh --tap A` BLIND-taps A, and on this game's
+"Load game?" dialog the cursor starts on **NO** — so a blind tap answers NO,
+drops back to the save list, and the next tap reopens the dialog. That is a
+stable oscillation, and it burned three consecutive 300 s boots as
+"NO readyroom" while d-pad and A both worked perfectly.
+
+The game dims the whole frame behind a modal, so the dialog is detectable
+without knowing which dialog it is: sample the band where the modal sits and
+compare its brightness against the undimmed screen.
+
+Measured on the LOAD GAME save list (1279x675):
+
+    save list, no dialog     mean 59.0 / 62.4   p95 164 / 199
+    "Load game?" dialog up   mean 34.4          p95 113
+
+so the threshold sits at 45. Used ONLY to decide whether tapping A is safe:
+with a dialog up, A is the affirmative/OK button; without one, A may mean
+something destructive like "open the dialog again".
+
+⚠️ Calibrated on 1 positive and 2 negatives — thin. Widen it as runs collect
+more frames.
+
+Usage: dialog_up.py <png>     -> prints metrics, exit 0 if a dialog is up
+"""
+import sys
+
+import numpy as np
+from PIL import Image
+
+THRESHOLD = 45.0
+
+
+def metric(path):
+    im = np.asarray(Image.open(path).convert("L"), dtype=float)
+    h, w = im.shape
+    box = im[int(h * 0.33):int(h * 0.48), int(w * 0.28):int(w * 0.72)]
+    return float(box.mean()), float(np.percentile(box, 95))
+
+
+if __name__ == "__main__":
+    mean, p95 = metric(sys.argv[1])
+    up = mean < THRESHOLD
+    print("%s mean=%.1f p95=%.1f threshold=%.0f" %
+          ("dialog" if up else "no-dialog", mean, p95, THRESHOLD))
+    sys.exit(0 if up else 1)
--- a/tools/re-capture/freeze_waitobj.sh
+++ b/tools/re-capture/freeze_waitobj.sh
@@ -35,6 +35,8 @@
 #     freeze_waitobj.sh run [fly_s]    boot + healthy + watch, end to end
 #     freeze_waitobj.sh repeat [n] [gap]  N captures of a HEALTHY run, tags h1..hN
 #     freeze_waitobj.sh stable [n] [gap]  boot, then repeat
+#     freeze_waitobj.sh dist [n] [gap]    boot, N healthy captures, wait for the
+#                                         freeze, N MORE captures, compare
 #
 # `repeat`/`stable` exist because a one-sample-per-state diff cannot tell a
 # freeze transition from ordinary variation: run 1's "T74/T75 move off a
@@ -109,6 +111,18 @@ PY
 MODE="${1:-boot}"
 FLY="${2:-}"

+if [ "$MODE" = frozenrepeat ]; then
+  N="${FLY:-6}"; GAP="${3:-25}"
+  pgrep -x xenia_canary >/dev/null || { echo "NO EMULATOR"; exit 1; }
+  for i in $(seq 1 "$N"); do
+    capture "f$i"
+    [ "$i" -lt "$N" ] && sleepfor "$GAP"
+  done
+  python3 "$SD/waitobj_report.py" --stability $(seq -f 'f%g' 1 "$N")
+  python3 "$SD/waitobj_report.py" --dist "$N"
+  echo "DIST DONE"; exit 0
+fi
+
 if [ "$MODE" = repeat ]; then
  N="${FLY:-5}"; GAP="${3:-45}"
  pgrep -x xenia_canary >/dev/null || { echo "NO EMULATOR"; exit 1; }
@@ -120,6 +134,21 @@ if [ "$MODE" = repeat ]; then
  echo "STABILITY DONE"; exit 0
 fi

+if [ "$MODE" = watchonly ]; then
+  SECS="${FLY:-900}"; end=$((SECONDS + SECS))
+  while [ $SECONDS -lt $end ]; do
+    pgrep -x xenia_canary >/dev/null || { echo "EMULATOR GONE"; exit 4; }
+    if python3 "$SD/frozen.py" 5 >/dev/null 2>&1; then
+      if python3 -c "import sys;sys.path.insert(0,'$SD');import frozen;sys.exit(0 if frozen.in_flight() else 1)"; then
+        echo "FROZEN IN FLIGHT at ${SECONDS}s of this watch"; exit 0
+      fi
+      echo "frozen but NOT in flight at ${SECONDS}s"; exit 5
+    fi
+    sleepfor 12
+  done
+  echo "NO FREEZE within ${SECS}s"; exit 1
+fi
+
 if [ "$MODE" = watch ]; then
  SECS="${FLY:-500}"
  pgrep -x xenia_canary >/dev/null || { echo "NO EMULATOR"; exit 1; }
@@ -170,3 +199,13 @@ if [ "$MODE" = stable ]; then
  echo "--- sampling the HEALTHY run ($(date +%T))"
  exec "$0" repeat "${REPEAT_N:-6}" "${REPEAT_GAP:-45}"
 fi
+if [ "$MODE" = dist ]; then
+  N="${REPEAT_N:-6}"
+  echo "--- $N HEALTHY captures ($(date +%T))"
+  "$0" repeat "$N" "${REPEAT_GAP:-30}"
+  echo "--- inducing and watching for the freeze ($(date +%T))"
+  nohup python3 "$SD/heavy_read.py" 4000 2 cpu </dev/null >/tmp/heavy.log 2>&1 &
+  "$0" watchonly "${WATCH_S:-900}" || { echo "NO FREEZE -- dist half not collected"; exit 1; }
+  echo "--- $N FROZEN captures ($(date +%T))"
+  exec "$0" frozenrepeat "$N" "${REPEAT_GAP2:-25}"
+fi
--- a/tools/re-capture/launch_mission.sh
+++ b/tools/re-capture/launch_mission.sh
@@ -76,9 +76,21 @@ sleep 14                       # main menu is not input-ready before this

 step down                      # NEW GAME -> LOAD GAME
 tap A; sleep 8    # save list, slot 01 preselected
-tap A; sleep 4    # "Load game?" -- cursor starts on NO
-step up
-tap A
+# Open "Load game?" and answer YES, VERIFYING the dialog is actually up first.
+# A fixed sleep here desynchronised the whole route: if the dialog had not
+# opened yet, `step up` moved the SAVE CURSOR instead of selecting YES, and the
+# blind `--tap A` below then oscillated the dialog for 300 s. See dialog_up.py.
+opened=0
+for _ in 1 2 3; do
+  tap A; sleep 4
+  shot "lm-loaddialog.png"
+  if python3 "$SD/dialog_up.py" "$SHOTS/lm-loaddialog.png" >/dev/null 2>&1; then
+    step up       # cursor starts on NO
+    tap A; opened=1; break
+  fi
+  echo "--- load dialog not up yet, retrying"
+done
+[ $opened -eq 1 ] || { echo "LOAD DIALOG NEVER OPENED"; exit 5; }
 # NOT a fixed sleep. LOAD -> READY ROOM took longer than 28 s in both runs on
 # 2026-08-23, so the next press was eaten by the transition and the run ended up
 # in OPTIONS (once) and BRIEFINGS (once); and a freshly restored profile inserts
@@ -90,7 +102,7 @@ tap A
 # start a NEW GAME. The real fix is to wait for each screen on the route rather
 # than only for this one; only this transition has actually been measured to
 # overrun, so only this one is waited for.
-"$SD/wait_screen.sh" readyroom 300 --tap A \
+"$SD/wait_screen.sh" readyroom 300 --tap-if-dialog A \
  || { echo "NEVER REACHED READY ROOM"; exit 1; }
 # ...and the READY ROOM is DRAWN before it is usable: it comes up with a
 # "Preparing to Sortie" spinner and TAKE OFF greyed out, for tens of seconds.
--- a/tools/re-capture/unitgroup.py
+++ b/tools/re-capture/unitgroup.py
@@ -48,23 +48,54 @@ def name_hash(s):
        a = (a - (q * MODULUS)) & 0xFFFFFFFF
    return (((b << 24) & 0xFF000000) | (a & 0x00FFFFFF)) & 0xFFFFFFFF

-TAG_MODULUS = (1 << 24) - 33      # 0x00FFFFDF, prime
+TAG_MODULUS = 0x00FFFFDF          # 2^24 - 33, prime
+TAG_MAGIC = 0x2101                # floor(2^56/M)+1, the guest's divide magic
+IXUD_M1 = 0xFFFFFF67              # 2^32 - 153, the IXUD loop modulus
+

 def tag_hash(s):
-    """IDXD record key / field tag -- NOT name_hash.
+    """IDXD record key / field tag -- a transcription of `sub_82447DF0`.

-    Same shape as name_hash (8-bit byte-sum checksum over a 24-bit modular
-    polynomial) but modulo 0x00FFFFDF instead of 0x00FFF9D7, and NOT
-    lowercased, so tags are case-sensitive. Recovered empirically from the 8643
-    (name -> tag) pairs the tables themselves carry; `unitgroup.py --checktags`
-    re-verifies all of them. A record's key is the tag of its own name, which
-    each table lists in an in-table roster record.
+    NOT name_hash: modulus 0x00FFFFDF (not 0x00FFF9D7) and NOT lowercased, so
+    tags are case-sensitive. A record's key is the tag of its own name.
+
+    The guest SIGN-EXTENDS each byte (`extsb`). That is not cosmetic: an earlier
+    version of this function used unsigned bytes and agreed on every one of the
+    8643 disc names -- because all of them are ASCII -- while disagreeing on
+    ~90% of random inputs containing a byte >= 0x80. The disc could never have
+    caught it; the disassembly did.
    """
-    b = s.encode()
-    lo = 0
-    for c in b:
-        lo = (lo * 256 + c) % TAG_MODULUS
-    return ((sum(b) & 0xFF) << 24) | lo
+    a = b = 0
+    for byte in s.encode('latin-1', 'replace'):
+        c = (byte - 256) if byte > 127 else byte        # extsb
+        a = ((a << 8) & 0xFFFFFFFF)
+        a = (a + c) & 0xFFFFFFFF
+        b = (b + c) & 0xFFFFFFFF
+        hi = ((a * TAG_MAGIC) >> 32) & 0xFFFFFFFF       # mulhwu
+        q = ((hi + (((a - hi) & 0xFFFFFFFF) >> 1)) & 0xFFFFFFFF) >> 23
+        a = (a - q * TAG_MODULUS) & 0xFFFFFFFF
+    return (((b << 24) & 0xFF000000) | (a & 0x00FFFFFF)) & 0xFFFFFFFF
+
+
+def ixud_hash(units):
+    """IXUD record key / field tag -- a transcription of `sub_82447E70`.
+
+    IXUD is IDXD's wide-string sibling. `units` is the name as big-endian UTF-16
+    code units. It defeated every single-modulus search because it chains TWO
+    exact moduli: the loop reduces mod 2^32-153 in 64-bit arithmetic, and only
+    the result is folded into 24 bits mod 2^24-33. A polynomial mod M1 folded
+    through M2 is not a polynomial mod anything, which is why a gcd test over
+    the pairs returns 1.
+
+    The checksum byte sums the FULL 16-bit code units, not their low bytes --
+    indistinguishable on this disc (every IXUD name is ASCII) but not in general.
+    """
+    a = b = 0
+    for ch in units:
+        a = ((a << 16) + ch) % IXUD_M1
+        b = (b + ch) & 0xFFFFFFFF
+    return (((b & 0xFF) << 24) | (a % TAG_MODULUS)) & 0xFFFFFFFF
+

 def read_entry(pak, h):
    idx = open(pak, 'rb').read()
--- a/tools/re-capture/wait_screen.sh
+++ b/tools/re-capture/wait_screen.sh
@@ -14,6 +14,17 @@
 # STOPS at the first match, because on most screens the button that dismisses a
 # dialog also leaves the screen you are waiting for.
 #
+# ⚠️ `--tap` is BLIND and that is dangerous on a YES/NO dialog: this game starts
+# those with the cursor on **NO**, so on the LOAD GAME screen a blind A answers
+# NO, falls back to the save list, and the next tap reopens the dialog — a
+# stable oscillation that burned three consecutive 300 s boots as
+# "NO readyroom" while the pad was working perfectly.
+#
+# Prefer **`--tap-if-dialog BTN`**: same thing, but only presses while a modal
+# is actually up (dialog_up.py, which detects the dim the game draws behind a
+# modal). With a dialog up, A is the OK/affirmative button and pressing it is
+# safe; with no dialog up it does nothing, so it cannot oscillate.
+#
 # Two consecutive matches are required, so a single frame caught mid-fade does
 # not count as arrival.
 #
@@ -21,8 +32,11 @@
 set -u
 CLASS="${1:?usage: wait_screen.sh <class> [timeout_s] [--tap BTN]}"
 TIMEOUT="${2:-180}"
-TAP=""
-[ "${3:-}" = "--tap" ] && TAP="${4:-A}"
+TAP=""; TAP_ONLY_IF_DIALOG=0
+case "${3:-}" in
+  --tap)           TAP="${4:-A}" ;;
+  --tap-if-dialog) TAP="${4:-A}"; TAP_ONLY_IF_DIALOG=1 ;;
+esac
 export HOME=/sylph-home/re
 DISP="${DISPLAY:-:98}"
 SD="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
@@ -48,8 +62,12 @@ while [ $SECONDS -lt $DEADLINE ]; do
  else
    hits=0
    if [ -n "$TAP" ] && [ $seen -eq 0 ] && [ $(( SECONDS - last_tap )) -ge 6 ]; then
-      python3 "$SD/pad.py" tap "$TAP" 0.3
-      last_tap=$SECONDS
+      if [ "$TAP_ONLY_IF_DIALOG" = 1 ] && ! python3 "$SD/dialog_up.py" /tmp/ws.png >/dev/null 2>&1; then
+        :                                  # no modal up -- pressing would be blind
+      else
+        python3 "$SD/pad.py" tap "$TAP" 0.3
+        last_tap=$SECONDS
+      fi
    fi
  fi
  sleep 2
--- a/tools/re-capture/waitobj_report.py
+++ b/tools/re-capture/waitobj_report.py
@@ -121,7 +121,38 @@ def stability(tags):
    print('  a VARIES thread differing when frozen proves nothing.')


+def dist(n):
+    """Compare N healthy captures against N frozen ones, per thread.
+
+    The point of doing it this way: a thread only counts as a freeze signature
+    if the set of states it takes while FROZEN is disjoint from the set it takes
+    while HEALTHY. Two earlier "signatures" died because a single healthy sample
+    happened to differ -- a distribution cannot be fooled that way.
+    """
+    H = [per_thread('h%d' % i) for i in range(1, n + 1)]
+    F = [per_thread('f%d' % i) for i in range(1, n + 1)]
+    H = [d for d in H if d]; F = [d for d in F if d]
+    if not H or not F:
+        print('need both halves: %d healthy, %d frozen' % (len(H), len(F))); return
+    threads = sorted({t for d in H + F for t in d}, reverse=True)
+    print('=== healthy(%d) vs frozen(%d) distributions ===' % (len(H), len(F)))
+    sig = []
+    for th in threads:
+        hs = {d.get(th, '--') for d in H}
+        fs = {d.get(th, '--') for d in F}
+        mark = ''
+        if not (hs & fs):
+            mark = '  <== SIGNATURE (disjoint)'; sig.append(th)
+        print('  T%-5d healthy{%s}  frozen{%s}%s' % (
+            th, ' , '.join(sorted(hs)), ' , '.join(sorted(fs)), mark))
+    print('  --- %d thread(s) whose frozen states never occur while healthy' % len(sig))
+    if not sig:
+        print('  No signature: every frozen state is one healthy play also produces.')
+
+
 if __name__ == '__main__':
+    if sys.argv[1:2] == ['--dist']:
+        dist(int(sys.argv[2])); sys.exit(0)
    if sys.argv[1:2] == ['--stability']:
        stability(sys.argv[2:]); sys.exit(0)
    tallies = {t: report(t) for t in (sys.argv[1:] or ['healthy'])}