[x64] Fix AVX-512 VECTOR_DENORMFLUSH merge-masking & FpRangeSign values

Copy src1 into dest before the masked vrangeps so that unmasked lanes
(normal/inf/NaN values) are preserved via merge-masking. Previously,
when the register allocator assigned different physical registers for
dest and src1, unmasked lanes would retain stale values from dest.

Tested with Intel SDE and AVX-512 cpu to verify.

Also shift unused FpRangeSign constants into bits [3:2] of the vrangeps
immediate, matching the Intel encoding. Only one used in the code was 0
so it was already correct by accident but using the others in the future
would produce incorrect results.
This commit is contained in:
Herman S.
2026-03-21 14:15:58 +09:00
parent 9621400ef6
commit bc37068f90
3 changed files with 47 additions and 5 deletions

View File

@@ -165,7 +165,8 @@ struct VECTOR_DENORMFLUSH
e.vptestnmd(denormal_mask, i.src1,
e.GetXmmConstPtr(XMMSingleDenormalMask));
e.vxorps(e.xmm1, e.xmm1, e.xmm1);
e.vrangeps(i.dest.reg() | denormal_mask, i.src1, e.xmm1,
e.vmovaps(i.dest, i.src1);
e.vrangeps(i.dest.reg() | denormal_mask, i.dest, e.xmm1,
FpRangeSelect::AbsMin | FpRangeSign::OperandA);
return;
}