DiffusionGemma-26B compatibility assessment: NEEDS_ARCH_SUPPORT on Apple Silicon (pre-port, external llama.cpp baseline)

1. Contribution (Claimed / NOT claimed)

Claimed. This note records a first-party compatibility assessment of the DiffusionGemma-26B-A4B-it Q4_K_M GGUF (about 16.8 GB, GGUF V3, ~5.3 bits-per-weight) against a multi-backend inference engine, evaluated on an Apple M1 Ultra (128 GB unified memory, Metal). It contributes: (a) a first-hand GGUF-header fingerprint of the architecture, parsed on-box; (b) the honest evaluation-time verdict NEEDS_ARCH_SUPPORT, with the first-hand load failures that justify it over narrower categories; (c) an external llama.cpp Metal baseline (clearly not the engine under test) as a community reality anchor; and (d) an adversarial normalization of public marketing tok/s claims against that anchor.

NOT claimed. This is not a benchmark report. The tok/s figures are a single-run external llama.cpp baseline plus public-claim normalization; they are not an engine measurement and not a multi-run benchmark. No claim is made that the engine could load, decode, or match the model at evaluation time (it could not). No iso-quality, multi-hardware, or production-throughput claim is made. A later forward-built / decode-running refinement is referenced only as a cross-link; its evidence lives in the cross-linked companion object, not here.

2. Methods

The model fingerprint, parsed first-hand from the GGUF header: a diffusion-flavored Gemma architecture; a Gemma-4-generation tokenizer with vocab 262144; non-causal attention; a fixed diffusion canvas length of 256; a mask token; self-conditioning tensors present; a top-8-of-128 MoE configuration; a dense FFN co-resident with routed experts per layer (shared dense + routed mixed FFN); 16 attention heads with a per-layer KV-head array; dual rotary position embedding (a global high-base component over a larger dimension plus a sliding-window low-base component over a smaller dimension); a sliding window with a periodic full-attention pattern; final logit softcapping; 30 blocks; an embedding dimension in the low thousands; tied embeddings. Net: a ~25B-total / ~3.8B-active block-diffusion MoE text model (Gemma-4 generation). The instruction-tuned text weights carry no vision tower and are handled as a text model.

3. Results (measured numbers + conditions)

Engine under test (in-scope, first-hand): hard load failure. Attempting to run the model against a shard directory containing the GGUF exits non-zero at tokenizer load: the engine never reads the GGUF; it requires an HF shard directory (a tokenizer JSON plus a safetensors index). No fallback was triggered; the failure is hard at tokenizer load. Correctness/parity is n/a (the model never loaded). Verdict at evaluation time: NEEDS_ARCH_SUPPORT — the engine could neither load nor decode.

External llama.cpp baseline (NOT the engine under test). Recent upstream builds fail with an unknown-architecture error; the architecture exists only in unmerged open PRs. The self-built PR binary loaded and generated coherent text on M1 Ultra Metal. Fixed config (context 4096, full GPU offload, 256 output tokens), default sampler, single-stream batch=1, one 256-token canvas block per prompt:

Single-stream wall throughput is ~14–22 tok/s (q4, M1 Ultra Metal). The "in-step parallel" column is the CLI's own canvas × steps figure — the source of advertising-style tok/s, not a usable wall rate. Correctness here is manual coherence judgment of llama.cpp output, not a formal parity test. TTFT is higher than autoregressive: a 256-token canvas must be built before iterative denoising, weights load in tens of seconds (peak RSS ~20 GB), then each step takes on the order of 0.77–0.97 s. The Metal path reports on-device sampling as unsupported and falls back to host sampling — a per-denoise-step host synchronization, the structural reason single-stream Metal lags.

4. Ablations / negative controls (architecture traps; cross-backend; public-claim normalization)

The architecture traps (why a flag cannot fix this). Multiple categories are hit simultaneously, each grounded in source or first-hand evidence: (1) GGUF schema — no production reader; one backend's test harness discards array/string key-value metadata, so the architecture string and the dual-RoPE base array are unreadable; other backends read only safetensors. (2) model architecture — no handler for this diffusion arch; a substring-matching Gemma handler mis-claims it but is a metadata-only stub that throws a not-yet-wired error; the MoE runtime explicitly rejects non-matching families. (3) decode loop (dominant) — all backends are strictly autoregressive (causal mask + KV-append + one token per forward); a tree-wide search for diffusion/canvas/mask/denoise concepts returns no hits. (4) non-causal attention — the model needs a rectangular mask; all backends hardcode causal. (5) self-conditioning — the self-conditioning tensors have no load/forward wiring (load-bearing per step, not optional). (6) MoE routing — partial: a top-8-of-128 router exists and is reusable, but the shared-dense + routed mixed FFN and the expert feed-forward length are missing. (7) dual RoPE / sliding-window pattern — a single scalar theta cannot express dual theta/dim or a per-layer pattern. (8) tokenizer/template — no Gemma-4 codec; one backend's chat template hardcodes a different model family; the mask token carries no semantics. (9) logit softcap — small: a softcap helper exists but is unwired. The gap spans roughly seven blocking items — a multi-component architecture port, not a kernel patch.

Why NEEDS_ARCH_SUPPORT and not the alternatives. Not SUPPORTED_NOW (load failed first-hand). Not LOADS_BUT_DECODE_UNSUPPORTED (load itself fails — no GGUF reader). Not BACKEND_KERNEL_GAP (the gap is reader + handler + schema + non-causal attention + denoise driver + self-conditioning + tokenizer across roughly seven blocking items). Not NOT_WORTH_INTEGRATING_YET (the model is real, there is genuine Apple-Silicon demand per a public llama.cpp issue, and it is the first non-autoregressive text-diffusion paradigm evaluated here; router / softcap / arch-agnostic dequant components are reusable).

Public-claim normalization (adversarial). Three checks: a datacenter-GPU "1000+ tok/s" claim → apples_to_oranges (high confidence): real and official, but not comparable to single-stream Metal (FP8 tensor-core vs no Metal FP8 path; FP8 vs q4; block-diffusion parallel caliber of ~15–20 tok/forward vs autoregressive 1; a window-definition trap of generation-only vs output-including-TTFT in the same serving run; non-iso-quality). A consumer-flagship "700+ tok/s" claim → apples_to_oranges (high confidence): official, with independent reproductions in the ~540–730 tok/s range on real hardware, but still NVFP4 tensor-core + parallel-denoise caliber. A previous-generation consumer-card tok/s claim → refuted (high confidence): no official or third-party source carries that figure; the only official consumer-card number is the current flagship — the previous-generation claim appears to be a misremembered flagship number. Iso-quality honesty: the model author's own disclosure puts DiffusionGemma below autoregressive Gemma 4 on several benchmarks (e.g., MMLU-Pro 77.6 vs 82.6; a math-competition score 69.1 vs 88.3; a competitive-programming rating 1429 vs 1718) — peak tok/s is not free. Cleanest same-harness number: a same-stack diffusion-vs-autoregressive comparison is roughly 1.9×, far below the 4–5× marketing ratio. Honest anchor: a realistic future Apple-Silicon single-stream q4 target is the community band ~40–60 tok/s (with host-sync optimization), not 700/1100; this report's measured 14–22 wall sits in the un-optimized band.

5. Threats to validity / limitations

• External baseline is not parity. The 14–22 tok/s and coherence are llama.cpp's, judged by manual read, not a formal cosine/argmax parity test. Prompt token counts are approximate (no independent Gemma-4 tokenizer at baseline time; flagged approximate in the results log).
• Single run, single host, single backend. One M1 Ultra, one PR build, one pass per prompt. No multi-run statistics, no warmup separation beyond what is noted.
• CUDA / ROCm are not measured first-hand. Those backends are on remote hosts; their status is static source analysis plus repro scripts, never passed off as measured.
• Forward parity is not generation quality. This note records load-time failure. The later cross-linked companion work reaches forward cosine parity to llama.cpp, but forward parity does not imply coherent-generation quality; the iso-quality disclosure above shows DiffusionGemma already scores below autoregressive Gemma 4 on the cited benchmarks.
• Performance gaps remain open. The per-step host synchronization (Metal host sampling), the missing Gemma-4 encoder/tokenizer, self-conditioning wiring, and full quantized residency are all open at the time of this assessment.
• TTFT honesty. Diffusion TTFT is structurally worse than autoregressive (the canvas must be built first); any future tok/s must be reported with its caliber window (generation-only vs output-including-TTFT) to avoid the generation-only-vs-TTFT-inclusive trap.

6. Reproducibility

• Host / engine: Apple M1 Ultra, 128 GB, Metal; a multi-backend text inference engine (Apple-Silicon backend). Date: 2026-06-15.
• Engine load attempt (fails as recorded): running the engine against a shard directory containing the Q4_K_M GGUF exits non-zero with a missing-tokenizer error, because the engine requires an HF shard directory rather than a GGUF.
• llama.cpp upstream (fails, records true cause): the diffusion CLI on a recent upstream build reports an unknown-architecture error for the model.
• llama.cpp PR build (succeeds, external baseline): clone the open architecture-support PR branch, configure a Release build with Metal enabled, build the diffusion CLI target, then run it against the Q4_K_M GGUF with context 4096, full GPU offload, and 256 output tokens.
• Wrapper scripts / data: a backend-auto-detecting llama.cpp baseline runner (JSONL sink, peak RSS via the system time utility), an engine bench runner, a results log, and the GGUF header parser with a raw log retained. CUDA/ROCm: scripts prepared, not run first-hand.
• Later refinement (cross-link, not duplicated here): the verdict was subsequently refined to forward-built + decode-running with measured cosine parity to llama.cpp; see the cross-linked companion object.