rocm-bench (HF-logits parity harness)

rocm-bench is a logits-parity harness for anyone porting or validating an inference runtime across chips and backends. It runs a model and dumps the per-token logits so they can be compared, offline, against a HuggingFace reference — the "oracle" — on identical token ids. The question it answers is narrow and load-bearing: does a non-reference runtime produce the same numbers as a validated reference, chip by chip? That is a question about trust, and it is answered with evidence rather than vibes.

How it works

The harness feeds fixed token ids through the implementation under test and records the logits at each position. Those logits are then checked, offline, against the oracle's logits for the same ids. Parity is deliberately scoped to this per-token comparison rather than an end-to-end generation, because a full generation loop compounds and hides divergence; comparing on fixed ids keeps the signal clean and attributable. Crucially, the harness does not use vendor BLAS. Pulling out the vendor maths library means a discrepancy points at the implementation you are actually validating, not at a black-box kernel underneath it.

Why it matters

Correctness across backends is easy to assert and hard to prove. rocm-bench turns that claim into a reproducible artefact: the same per-token logits it emits are what feed the HF-logits parity corpus, so validation runs accumulate into shared evidence rather than one-off checks. It sits alongside related work on apple-silicon-inference as part of a broader effort to make cross-chip inference verifiable — the same numbers, demonstrably, wherever the model runs. The harness is released under Apache-2.0.