Files

Sylpheed RE agent ef4e975a47 re: diff the legacy IDXD reader against the field table -- 17.54% wrong, with an exact predicate

Answers the backlog's open "first step: diff the two readers across the disc and
count disagreements", statically over every IDXD object.

Of 7750 objects and 738922 named fields whose true value is numeric, legacy
get_f32 is correct 39.42%, returns None (harmless) 43.04%, and returns a WRONG
NUMBER 17.54% (129612 fields).

The wrongness has an exact predicate: single-record objects 0 of 29822 wrong
(0.00%); multi-record objects 129612 of 709100 (18.28%).  The mechanism is in
get_raw itself -- it flattens the pool to a token list, finds the FIRST occurrence
of the key, and returns the preceding token, with no notion of records.  So every
record after the first inherits record 0's value: Weight truth=1.0 legacy=0.3,
Points truth=10000 legacy=4000.

Practical rule recorded: a get_f32 number from a single-record object is safe; from
a multi-record object only the first record is.

Withdrawn in the same document: my first sweep compared against "the string before
THIS field's own key" and reported 65.90% -- that is not what get_raw does, so the
figure is not the legacy reader's error rate.

2026-08-27 04:52:05 +00:00

2.8 KiB

Raw Blame History

✅ How wrong the legacy IDXD reader is — measured disc-wide

BACKLOG.md left this open: "the legacy value-before-key string-pool reader is an approximation … every number in this corpus that came out of get_f32/get_raw is re-checkable against ground truth but not yet re-checked. First step: diff the two readers across the disc and count disagreements." Done, statically, over every IDXD object on the disc.

The answer

7 750 IDXD objects, 738 922 named fields whose true value is numeric:

legacy `get_f32` outcome	fields	share
correct	291 307	39.42 %
returns `None` — no number produced, so harmless	318 003	43.04 %
returns a WRONG NUMBER	129 612	17.54 %

✅ And the wrongness has an exact predicate

object shape	numeric fields	wrong numbers
single-record	29 822	0 — 0.00 %
multi-record	709 100	129 612 — 18.28 %

The legacy reader is exactly correct on single-record objects and unsafe on multi-record ones. Not approximately — zero disagreements in 29 822 fields.

The mechanism is in get_raw's own implementation: it flattens the string pool to a token list, finds the first occurrence of the key, and returns the token before it. It has no notion of records. So in an object with N records that each carry a Weight, every record's Weight resolves to record 0's value:

Weight   truth=1.0     legacy=0.3
Points   truth=10000   legacy=4000
Weight   truth=3.3     legacy=0.3
Points   truth=15000   legacy=4000

0.3 and 4000 are record 0's values, returned for every record after it.

What this means for the corpus

A number read with get_f32 from a single-record object is safe — the diff found no counterexample in 29 822 fields.
A number read from a multi-record object is safe only for the first record; every later record has a ~18 % chance of being record 0's value.
The None bucket (43 %) is the guard working as designed: get_f32 rejects a non-numeric neighbour rather than inventing a value, so a defaulted field reads as absent instead of wrong.

⚠️ A first pass that measured the wrong thing

My first sweep compared the truth against "the string immediately preceding this field's own key in the pool" and reported 65.90 % disagreement. That number is not the legacy reader's error rate, because it is not what get_raw does — get_raw uses the first occurrence of the key anywhere in the object, not this field's own key. The 65.90 % figure is withdrawn; 17.54 % is the measured one, and the is_definite_value guard is why it is so much lower.

The lesson is the same one this corpus keeps paying for: before measuring how wrong a tool is, read what the tool actually does.

2.8 KiB Raw Blame History