A DiT self-attention flash kernel on RDNA2: 9.4x op, 4.67x end-to-end
Ran Tao (Octoryn Research)
Abstract
A hand-written tiled online-softmax flash-attention kernel for a video diffusion-transformer (DiT) self-attention operation on an RDNA2-class consumer GPU speeds that op 9.4x and the full standalone text-to-video pipeline from 1130s to 242s (4.67x), at cosine 1.0 parity versus the engine's own pre-flash attention path, with no vendor BLAS/attention library linked. Engineering result: the same arithmetic on a faster route, validated by numerical parity. Single-configuration, single-run; not a vendor-kernel comparison and not a generation-quality claim.
Contribution
Claimed. A hand-written flash-attention kernel (tiled, online-softmax, no full attention matrix materialized) for a video diffusion-transformer (DiT) self-attention operation on a single RDNA2-class consumer GPU (RX 6900 XT class, 16 GB) reduces the self-attention op runtime by 9.4x and the wall time of the full standalone text-to-video pipeline from 1130s to 242s (4.67x), while producing cosine 1.0 parity against the pre-flash attention path and retaining a sovereign dynamic-linkage profile (no vendor BLAS/attention library linked). This is an engineering/optimization result: a faster route to the same arithmetic, validated by numerical parity against the engine's own prior path.
NOT claimed. This note does not claim any novel attention algorithm — flash/online-softmax attention is established prior art; the contribution is a sovereign hand-port to this GPU architecture. It does not claim competitiveness with vendor attention kernels in absolute terms — no vendor-kernel A/B is recorded. It does not claim generation quality: parity is measured against the engine's prior attention path, not against a reference renderer or human judgment. It does not claim the speedups generalize to other attention shapes, resolutions, sequence lengths, or batch sizes — only the tested DiT self-attention configuration is covered.
Methods
| Field | Value |
|---|---|
| Hardware | RX 6900 XT class GPU (RDNA2, 16 GB) |
| Stack / versions | ROCm / HIP (specific version not recorded in source notes) |
| Oracle / baseline | The engine's own pre-flash self-attention path (same model weights, same pipeline) |
| Baseline version | Same standalone pipeline pre-kernel-swap |
| Procedure | Replace the DiT self-attention op with the tiled online-softmax flash kernel; measure the op runtime and the full text-to-video wall time; compare output latents/frames against the pre-flash path |
| Precision | Not recorded in source notes |
| Metric + gate | Cosine similarity versus pre-flash path (reported 1.0); speedup ratios on op and end-to-end wall time |
| Stimulus / inputs | The video DiT self-attention within the standalone text-to-video pipeline (exact prompt/shape/step-count not recorded in this source note) |
Results
| Measurement | Pre-flash | Flash kernel | Ratio |
|---|---|---|---|
| Self-attention op | — | — | 9.4x faster |
| End-to-end pipeline wall | 1130s | 242s | 4.67x faster |
| Parity (cosine vs pre-flash path) | — | 1.0 | — |
| Sovereignty (dynamic linkage) | — | no vendor BLAS/attention library | — |
(Only the op ratio and the end-to-end absolute/ratio figures are recorded in the source; the per-op absolute times are not.)
Threats to validity / Limitations
- Single-configuration, single-run. Both the 9.4x op and the 4.67x end-to-end figures are point measurements at one DiT self-attention shape on one machine. No variance, confidence interval, or repeat-run spread is recorded — treat them as single-run engineering measurements, not a benchmark-scale distribution.
- Parity != quality. Cosine 1.0 is measured against the engine's own pre-flash path, so it proves the kernel computes the same math; it says nothing about whether the generated video is correct relative to a reference renderer or perceptually good.
- No vendor baseline. The 9.4x/4.67x ratios are relative to the engine's prior (slower) path, not to a tuned vendor attention kernel. This is an internal-speedup claim, not an absolute-performance or vendor-parity claim.
- Scope bound to one op/shape. Speedups are specific to the tested video DiT self-attention shape on this GPU architecture. Other resolutions, sequence lengths, batch sizes, attention variants, or GPUs are out of scope and untested here.
- Provenance gaps. Precision, ROCm/HIP version, and exact prompt/shape/step-count are not captured in this source note. A skeptical reader should treat the reproducibility level as engine-internal reproducibility rather than third-party replication.
Reproducibility / Provenance
| Field | Value |
|---|---|
| Build approach | Compiled with the vendor HIP compiler targeting the RDNA2 architecture, linking only the base HIP runtime |
| Run command | Not recorded in source notes |
| Version pins | ROCm/HIP version not recorded in source notes |
| Sovereignty check | Dynamic linkage shows base HIP runtime plus system libraries only (no vendor BLAS/attention/DNN/MKL libraries) |
| Oracle caveat | Parity baseline is the engine's own pre-flash path; it does not validate against an external reference renderer |
Claim boundary
The author's explicit scope — what this work does and does not establish — carried over from the Octoryn Research publishing model.
Proves
- A hand-written tiled online-softmax flash kernel for a video DiT self-attention op on an RDNA2-class GPU speeds that op 9.4x and the text-to-video pipeline 4.67x (1130s->242s), at cosine 1.0 parity vs the engine's own pre-flash path, with no vendor attention library linked.
Does not prove
- Competitiveness with vendor attention kernels in absolute terms; no vendor A/B was recorded.
- Speedups for other attention shapes, resolutions, sequence lengths, or batch sizes.
- Generation quality: parity is measured against the prior path, not a reference renderer or human judgment.
- Statistical robustness: 9.4x and 4.67x are single-configuration, single-run point measurements with no recorded variance.
Applies when
- The tested video DiT self-attention operation on an RDNA2-class consumer GPU, at the single tested shape and configuration.
Does not apply when
- Absolute-throughput or vendor-parity claims, other kernels, shapes, resolutions, or GPUs, or any generation-quality assertion.
Authors
- Ran Tao — Investigation, Writing
Cite this
Citation
Tao, R., Octoryn Research. (2026). A DiT self-attention flash kernel on RDNA2: 9.4x op, 4.67x end-to-end (BR-2026-0006). Octopus Research Institute.
BibTeX
@techreport{oribr20260006,
title = {A DiT self-attention flash kernel on RDNA2: 9.4x op, 4.67x end-to-end},
author = {Tao, Ran and {Octoryn Research}},
institution = {Octopus Research Institute},
year = {2026},
note = {Permanent ID BR-2026-0006. Not peer reviewed.}
}Disclosures
- Funding
- Hardware and infrastructure provided by Octoryn / Octopus Core Pty Ltd.
- Conflicts of interest
- Octoryn ships commercial inference and governance tooling; findings are reported independently.