Diffusion-LLM Decode Quality: An Honest Regression, Negative Evidence, and a Canvas-Geometry Reattribution

1. Contribution (Claimed / NOT claimed)

Claimed. This note reports the honest decode-quality arc of a self-hosted diffusion-style language decoder evaluated against a reference regression set, and — its central research finding — the experimental self-correction of a prior attribution. Concretely:

1. The initial partial verdict: answer-hit 7/26 (26.9%) versus reference 25/26 (96.2%), with all 26 outputs coherent and most failures (18) sharing one structural form: stuck in the thought trace, never crossing to the answer channel.
2. The load-bearing second-round correction: a controlled A/B plus an isolation matrix refuted the original attribution that "decode policy is the main cause." The binding variable is canvas geometry plus chat template, not decode policy — argmax versus multinomial is neutral in both regimes.
3. The disclosure of an apples-to-oranges harness defect that inflated the original gap.
4. A faithful multinomial denoiser (inverse-CDF sampling with an ordered pre-draw matching the reference mechanism) plus a byte-aligned chat template, taking prompt-id parity from 0/26 to 26/26.
5. The 26-prompt rerun (longer canvas, multinomial plus template): answer-hit 7/26 to 23/26, stuck 25 to 4, crossed-channel 1 to 22.
6. The honest negative results that cap the verdict at "decode-policy improved," short of reference-aligned.

NOT claimed. Not reference-aligned decode; not production-ready. Not bit-exact trace alignment with the reference (argued unreachable, see §5). Not that the multinomial decode policy improved quality (it is faithful but neutral, and slightly noisier). Not coherent-generation quality across all categories: long context is a residual weak region (1/3) and one prompt regressed (pass to miss). No performance claim — decode is unoptimized, on the order of minutes per prompt. The reference decoder was used golden-only and never treated as a first-party output.

2. Methods

The identical-geometry claim from the old report is retracted (§4): the harness passed step/length flags, but the reference's entropy-bound canvas is model-locked and its step count requires a separate flag that was never passed, so both flags were ignored. The old comparison was therefore reference long-canvas/many-steps versus a much shorter sovereign canvas with fewer steps.

3. Results

Old — partial verdict (first-party measurement). Pass 7, fail 18, reference-miss 1. Sovereign answer-hit 7/26 = 26.9% versus reference 25/26 = 96.2%. All 26 outputs coherent (no garbage); all 18 failures share one form — stuck in the thought trace, never reaching the answer channel. Template/prompt-id match 0/26. Mean text-similarity 0.382.

New — decode-policy improved (first-party measurement) (longer canvas, multinomial plus template):

Net 18 fail-to-hit conversions across the factual, math, code, instruction, translation, safety, and stress categories. New decode latency is on the order of minutes per prompt; unoptimized.

4. Ablations / negative controls

Forward is NOT the cause (cross-backend / anchor). The forward anchor holds before and after the decode changes: a fixed two-token probe yields the same argmax rows within floating-point noise of the historical values, and near-perfect cosine versus the reference. The earliest failure outputs are byte-identical to the reference opening — a second proof that forward is not implicated. (Forward parity to the reference meets a stronger reproducibility bar — cosine-stable across runs — but it is not the subject of this note.)

Decode-policy ablation (A/B, the load-bearing control). Same prompt, same canvas/steps/seed/template/RNG/renoise/self-conditioning, only the decode policy toggled, at a moderate canvas: all probed prompts hit — including prompts that failed under the old partial verdict — for both argmax and multinomial. Decode policy does not change hit/stuck at this geometry.

Isolation matrix (the geometry definer). At the original tight geometry, all probed prompts stall across legacy, argmax-plus-template, and multinomial-plus-template. Merged truth table:

This shows the binding variable is canvas geometry, not decode policy; the template alone cannot rescue a tight geometry. It refutes the round-one attribution: a tight canvas cannot hold both the thought trace and the answer.

Architecture / forward correctness are settled by the near-perfect forward cosine parity and the unchanged anchor; mixture-of-experts, attention, and self-conditioning injection were verified in forward parity. These are not re-litigated here — this note is about decode, downstream of a parity-clean forward.

5. Threats to validity / limitations

• Real regression (non-uniform dominance). One long-context summarization prompt went from old pass to new miss: multinomial plus the longer canvas pulled the denoise trajectory off-keyword. The new config is not uniformly dominant.
• Long context remains weak (1/3). Persistent failures on the hardest long-context prompt (text-similarity 0.03) and one safety prompt.
• Forward-parity is not generation quality. Near-perfect forward cosine does not imply reference-aligned decode; the gap was entirely in canvas evolution. Coherent is not correct: all 26 old outputs were coherent yet only 7 hit.
• Text-similarity is misleading here. Old stuck-in-thought text resembled the reference thought trace (high similarity); new outputs cross to a shorter real answer (low similarity but hit). Use answer-hit / crossed-channel, not text-similarity.
• Shorter canvas than the reference. The reference locks a longer canvas; this rerun used a shorter one for tractability. Fair when the canvas is at least as long as the answer; partially explains long-context weakness. An exact-geometry run is deferred.
• Bit-exact alignment is unreachable (inferred). (a) The integer/real distribution implementations are implementation-defined; (b) the reference itself runs a non-host-deterministic GPU sample-reduce (order ~1e-4 drift); (c) the forward is cosine-close, not bit-identical, so near-ties fork. The goal is policy / statistical / structural alignment, not per-token identity.
• Decode policy is faithful but neutral, and slightly noisier. Multinomial output carries light token noise; argmax output is actually cleaner. Faithful to the reference mechanism, not a quality lever.
• Performance / CI unmeasured. Decode is minutes per prompt, unoptimized; no CI; exact-geometry and per-step reference-trace runs deferred. Hence not production-ready.

6. Reproducibility (high-level)

• Evidence. Prompt set, regression and evaluation harnesses (old and revised), reference and sovereign output captures, the A/B and isolation experiment logs, template-parity records, and per-step trace dumps.
• Implementation. Sovereign implementation with custom matrix-multiply kernels and no vendor BLAS: an oracle-faithful entropy-bound decode path with a Mersenne-Twister and a decode config; the legacy decode path is retained as a baseline. A forward CLI exposes the new flags and a tokenize subcommand. The forward path is unchanged, so the anchor passes by construction.
• Controls. A/B holds everything fixed and toggles only the decode policy at a moderate canvas; the isolation matrix sweeps legacy / argmax-plus-template / multinomial-plus-template at a tight canvas; the full rerun uses oracle-template plus multinomial at the longer canvas (sovereign only; reference baseline frozen at its native geometry). Template parity is pure tokenization with no forward.
• Reference trace. The reference can be patched to append per-step JSON when a trace path is set, a strict no-op otherwise. Reference-side trace generation was deferred once the finding turned out policy-neutral; sovereign-side per-step traces were generated.