Sylpheed/docs/re/structures/ui-render-tone-curve.md

🟡 Our composite is brighter than the emulator's frame — measured, not decoded

Status: 🟡 measured, and REFUTED outside its stated reach — the single exponent holds only below render ≈ 40; see the refutation below. Closes an observation left dangling by ui-8ax-fullres-background ("the capture is ~4× darker than the render"), and puts a number on the ❔ that INDEX's texture row already carried: "exact fidelity (gamma/sRGB curve, premultiplied alpha, per-channel scale) is untested, since a hue comparison cannot see it."

⚠️ This is not a decode. A port applying it is authoring a value.

First: the geometry is right

Cross-correlating live-main-menu.png against our render over ±6 px finds the best alignment at exactly dy = 0, dx = 0, correlation 0.9466. So the composite is in the right place at the right size and only the tone differs. (The capture is 1279×675 and top-aligned; that is the screenshot tool's crop.)

🔴 The first two methods were wrong, and both failed visibly

• Three dark patches gave "capture ≈ 4× darker". Over the whole frame the best linear scale is 0.914. Three patches from one region are not a transfer curve.
• A pixel-wise fit over 854 685 pixels produced a non-monotonic transfer (render 96–127 → capture 143, brighter than render 128–159 → 132). That is the signature of edge misalignment, not of a tone curve: at a 0.947 correlation a bright render pixel routinely lands on a dark capture pixel. Mean abs error was 10–14 for every model, which is the tell that none of them fit.

Both are recorded because the second is the interesting failure — a fit whose residual is large everywhere is not a model to choose between, it is a method to throw away.

The method that works: flat patches only

16×16 patches where both images have std < 8, so local edges cannot contribute. The threshold is not arbitrary — at std < 3 there are zero patches, and the count runs 0 / 83 / 404 / 1055 / 1788 for std < 3 / 5 / 8 / 12 / 20.

screen	flat patches	gamma exponent	mean abs err	best linear	its err
main menu	404	1.491	0.28	0.276	0.34
EXTRAS	382	1.493	0.22	0.273	0.28
title	506	1.338	1.08	0.842	9.02

So capture ≈ 255·(render/255)^γ with γ ≈ 1.34 – 1.49.

🔴 REFUTED above render ≈ 40 — one exponent cannot express this curve (2026-08-29)

Raised by the port, and independently reproduced here before being adopted. Binning matched pixels by render level instead of fitting a scalar:

	render 16	23	31	39	47	64
port, all matched pixels	γ 1.26	1.18	1.10	1.03	0.93	0.85
mine, flat-neighbourhood pixels	γ 1.303	1.347	1.128	0.912	0.935	1.003

✅ The finding holds: the exponent falls monotonically and crosses 1.0. Below that the capture is darker than the render (γ > 1, which is what this page measured); above it the capture is brighter. A single exponent cannot express a curve that crosses unity, so the model above is valid only in the darks — which is exactly the reach this page already stated. ⚠️ The reach line was not a hedge; it was the finding.

🟡 Where they disagree, and it is not resolved. The crossing point is ≈44 by the port's binning and ≈35–40 by mine; the darks read 1.18–1.26 (theirs), 1.30–1.35 (mine) and 1.49 (this page's original patch fit). Three estimators on three populations — selected flat patches, all matched pixels, flat-neighbourhood pixels — and nothing here adjudicates between them. All three agree on the direction and on γ > 1 in the darks; that is the part to rely on.

⚠️ A confound in my reproduction, stated so it is not mistaken for independence it does not have: my whole-image correlation is only 0.594, because the committed live-main-menu capture and the default render differ in focus state — the port measured that 74.1 % of differing pixels fall inside the live-main-menu vs live-main-menu-options-focused signature. My bins therefore include that mismatch. It did not change the direction of the trend, but it is why my numbers are not a clean second opinion.

⚠️ Also: render 1280×720 against a 1279×675 capture requires a resample (LANCZOS here), which perturbs levels at edges — hence restricting to flat-neighbourhood pixels.

⚠️ The reach — and it is narrow

• The flat patches are almost all dark: render values ~0–60. Over that range a gamma and a linear scale are nearly indistinguishable — on the two menus the errors are 0.28 vs 0.34 and 0.22 vs 0.28, which decides nothing. Only the title separates them (1.08 vs 9.02), because its flat regions reach ~60.
• The held-out control could not test it. Running the same fit on the developer splash gives 2 918 flat patches whose render range is 0–4 — pure black. Every model scores ≈ 0.00 there. That is a control that failed to discriminate, not a control that passed.
• Nothing here constrains midtones or highlights, which is exactly where a γ = 1.4 curve does its visible work.

🔴 The confound as I stated it is REFUTED — canary applies no gamma of its own

I wrote that "canary applies its own output transform: kernel_display_gamma_type = 2 — BT.709". That is not what the cvar does. Reading the source:

void VdGetCurrentDisplayGamma_entry(lpdword_t type_ptr, lpfloat_t power_ptr) {
  // 1 - sRGB.  2 - TV (BT.709).  3 - use the power written to *power_ptr.
  // Anything else - linear.
  // Used in D3D SetGammaRamp/SetPWLGamma to adjust the ramp for the display.
  *type_ptr = cvars::kernel_display_gamma_type;
  ...

It is a getter the guest calls (xboxkrnl_video.cc, exported kStub). The cvar is a value reported to the game, which then builds its own ramp. The emulator's role is downstream and faithful:

• the guest writes its ramp to the DC_LUT registers;
• command_processor.cc reads them into gamma_ramp_256_entry_table_;
• the swap (present) path applies them — swap_apply_gamma_pipeline_layout, with apply_gamma_table.ps / apply_gamma_pwl.ps compiled in.

So there is no emulator-side BT.709 post-process to subtract. Any gamma in the captured frame is a ramp the game installed.

🟡 What that does and does not settle

✅ The stated confound is gone: the measured exponent is not an emulator artefact bolted onto the game's output.

✅ The game DOES query the display gamma — measured 2026-08-29. Booted with --log_mask=12 --log_level=3 (Kernel logging on, Cpu/Gpu off), which changes nothing about the output. VdGetCurrentDisplayGamma is called once at video init, between the command-buffer setup and VdSetDisplayMode:

d> F8000008 VdGetSystemCommandBuffer(701CF830, 701CF804)
d> F8000008 VdGetCurrentDisplayGamma(701CE1F8(00000000), 701CE1F0(0))
d> F8000008 VdSetDisplayMode(40000000)
d> F8000008 VdGetCurrentDisplayInformation(701CF110)

The control is in the same log: 359 VdRetrainEDRAM and 358 VdGetSystemCommandBuffer lines, so an absent call would have been visible. Evidence in data/gamma-call-evidence.txt.

Per the export's own comment the returned type is "used in D3D SetGammaRamp/SetPWLGamma", so the game asks the question a ramp-builder asks, at the moment one would ask it.

🟡 The ramp write: inferred from a closed chain, not directly observed

The write is a GPU register operation (XE_GPU_REG_DC_LUT_RW_INDEX in CommandProcessor::WriteRegister), invisible to kernel logging and unlogged in any case. But two facts from the source close the reasoning:

1. The swap-path gamma stage is a pure LUT — nothing else. apply_gamma_table.xesli is the whole transform:

uint3 apply_gamma_input = uint3(texel_fetch(source, pixel).rgb * 255.0 + 0.5);
apply_gamma_output.r = texel_fetch_buffer(xe_apply_gamma_ramp, input.r).b;
… .g = …(input.g).g;   … .b = …(input.b).r;

An index into a 256-entry table. No sRGB encode, no second transfer function.

2. The table defaults to IDENTITY. CommandProcessor::Initialize fills it with value = i * 0x3FF / 0xFF, and its own comment says so: "Initialize the gamma ramps to their default (linear) values — taken from what games set when starting with the sRGB (return value 1) VdGetCurrentDisplayGamma." An unwritten ramp is a no-op.

So: the only transform is a LUT; the LUT is identity unless the guest writes it; the game queries the display gamma at init (measured above); and the captured frame differs from our composite by γ ≈ 1.34–1.49, which identity cannot produce. ⇒ the guest wrote a non-identity ramp.

⚠️ This is an inference, and here is its weak joint. It assumes our composite faithfully reproduces the pre-ramp framebuffer, which it does not exactly — our renderer has its own inaccuracies. What makes it hold up is the shape: a systematic exponent near 1.4, fitted on flat patches, consistent across three screens, is not the signature of a compositor bug.

A fixed sRGB stage elsewhere in the presenter is the obvious alternative and does not fit: an sRGB encode (≈ ^0.45) brightens, and we measured darkening; an sRGB decode (^2.2) darkens far more than 1.4.

✅ Direct observation is still available and cheap, and needs the emulator only to boot: a GPU trace records gamma ramps as their own command type (TraceWriter::WriteGammaRamp), or one log line at XE_GPU_REG_DC_LUT_RW_INDEX would settle it outright. Not done.

⚠️ And the ramp a game builds depends on the display type it is told. Canary hard-codes 2 (TV/BT.709); on hardware that is the console's display setting. So the exponent is display-profile dependent by design, not a fixed property of the game.

✅ The corrected next experiment

My planned run — set kernel_display_gamma_type = 0 and re-fit — was the wrong design: it changes what the guest is told and therefore which ramp the game builds, so it could never isolate an emulator stage that does not exist. It also perturbs the capture harness, because skip_intro.sh classifies movie-vs-static on an absolute rmse threshold and a brighter frame biases it (see capture-harness-status).

The right run changes nothing about the output: boot with LOG_MASK=12 LOG_LEVEL=3 (both are needed — kernel calls log at Debug) and look for VdGetCurrentDisplayGamma and the DC_LUT writes. Same frames, no perturbation. ✅ Run, and it answered the first half — see above. ⚠️ And note it needed the emulator only to boot, not to reach a menu: video init happens in the first seconds. This had been parked behind the title-screen blocker for no reason.

🔴 The direct observation: ATTEMPTED, BLOCKED by the build tree (2026-08-30)

This page records the game's ramp write as "inferred from a closed chain, not directly observed", the chain being that the swap-path stage is a pure 256-entry LUT and that an unwritten table is identity (i * 0x3FF / 0xFF). The direct observation is a log in Canary's own write path, which /canary being read-write makes available. Attempted; blocked, and the blocker is worth recording.

The patch — command_processor.cc, inside XE_GPU_REG_DC_LUT_SEQ_COLOR, on each completed 256-entry sweep:

if (gamma_ramp_rw_index.rw_index == 255 && gamma_ramp_rw_component_ == 2) {
  auto id = [](uint32_t i) { return i * 0x3FFu / 0xFFu; };
  XELOGI("[RE-GAMMA] full 256-entry ramp written (sweep #{})", ++re_gamma_sweeps_);
  for (uint32_t i : {0u, 64u, 128u, 192u, 255u}) {
    const auto& e = gamma_ramp_256_entry_table_[i];
    XELOGI("[RE-GAMMA]   [{:3}] r={:4} g={:4} b={:4}   identity={:4}{}",
           i, uint32_t(e.color_10_red), uint32_t(e.color_10_green),
           uint32_t(e.color_10_blue), id(i),
           uint32_t(e.color_10_red) == id(i) ? "" : "   <- NOT identity");
  }
}

plus uint32_t re_gamma_sweeps_ = 0; beside gamma_ramp_rw_component_ in the header. It prints each written ramp against the identity the source documents, so a written ramp is distinguishable from an unwritten one by reading the log.

🔴 Blocked: the build tree cannot regenerate. /sylph-home/re/canary-build was configured with -S/work/xenia-canary, and that path does not exist — the tree was configured against a source location this container no longer has. ninja fails at the CMake regeneration step before compiling anything. Reconfiguring against /canary would almost certainly trigger a near-full Xenia rebuild, which is not a thing to start on the way to one log line.

✅ The patch was REVERTED and /canary verified byte-identical to its backup. Leaving instrumented source that the running binary does not contain is the source-and-binary-disagree trap this corpus has hit repeatedly — a later reader would find the logging in the tree and conclude it is live.

⚠️ So the ramp write remains inferred, not observed, exactly as this page already says. What is new is the reach: the experiment is written, and the obstacle is a build-tree path rather than anything about the game.

What a port should do with this

Treat it as authored, not transcribed. If the goal is to match the emulator — which is what every capture in this corpus is — a γ ≈ 1.4 darkening of the composite gets closer, and is best applied where it was measured (the dark background), not extrapolated to the whole range on this evidence.