Porting Mamba (selective-scan SSM) into a sovereign sequence-mixing engine

1. Contribution

Claimed. This report documents a correctness-validation result: a hand-written, independently implemented selective-scan state-space version of a 130M-class Mamba model reproduces a reference HuggingFace transformers f16 prefill-logit forward pass at cosine similarity 0.999997, with argmax matching on 8/8 checks across two prompts. The implementation covers the selective scan with input-dependent step, input, and output projections, plus causal 1D convolution, softplus, and SiLU gating — with no attention, KV cache, or rotary position embedding. A kill-selectivity negative control that removes the input-dependent selection collapses parity to cosine -0.73, argmax 0/8, demonstrating that the selective mechanism — not incidental numerical proximity — drives the match. A linkage check confirms the build carries no dependency on the reference SSM GPU kernel library and no vendor BLAS. This is the engine's first non-attention paradigm, so the engineering altitude is paradigm coverage and faithful reproduction, not a novel algorithm.

NOT claimed. This is a logit-parity smoke test, not a benchmark. It does not establish generation quality (parity of next-token logits is not text quality), throughput or latency competitiveness (the scan runs sequentially and is not compared to a fused GPU kernel), correctness at quantized precision (only f16 was tested), or coverage of larger Mamba variants, Mamba-2/SSD, or hybrid state-space plus attention models (e.g. Jamba). The sample is small (two prompts) and single-run.

2. Methods

3. Results

4. Ablations / Negative controls

5. Threats to Validity / Limitations

• Sample size. Two prompts across a handful of token positions (8 argmax checks total) is a correctness smoke test, not a benchmark-scale evaluation. No claim about distributional behavior follows.
• Single run, no variance. Parity is reported as point values; no repetition, seed sweep, or variance is recorded.
• Precision ceiling. Only f16 was validated. Quantized correctness (8-bit / 4-bit) is untested and explicitly out of scope.
• Parity is not quality. Cosine 0.999997 on logits demonstrates faithful numerical reproduction of the reference forward pass; it says nothing about generation quality, long-context behavior, or downstream task performance.
• Baseline weakness. The oracle is the reference library's own sequential fallback (it falls back to a sequential scan when the fused GPU kernel is absent), so this validates against reference semantics but not against fused-kernel numerics. Throughput is not compared against any fused kernel.
• Provenance gaps. Exact build identifiers, library versions, and exact prompt text are not recorded, weakening exact reproducibility.
• Scope. A single 130M-class Mamba model only; no larger Mamba, Mamba-2/SSD, or hybrid state-space plus attention coverage. An experimental evidence level is retained as appropriate for a single-configuration parity result.

6. Reproducibility / Provenance

• Implementation. Hand-written selective scan with causal 1D convolution, softplus for the step parameter, and SiLU gating; no attention, KV cache, or rotary position embedding. The recurrence follows the standard published selective state-space formulation.
• Oracle caveat. Validate the oracle first: the reference must be exercising its sequential-scan fallback (no fused SSM GPU kernel present) for its semantics to match the on-box sequential scan.
• Sovereignty check. A dynamic-linkage inspection shows the GPU runtime and system libraries only — no vendor BLAS or deep-learning primitives library, and no dependency on the reference SSM kernel package.
• Build. Compiled with the AMD HIP toolchain targeting an RDNA2-class architecture. Exact build identifiers and run commands are not recorded.
• Versions. Oracle and engine stack are recent versions; precise version fingerprints are omitted.