True CUDA text-to-video with live UMT5 encode, dependency-minimal (no cuBLAS)
Ran Tao (Octoryn Research)
Abstract
A self-contained CUDA Wan2.1 text-to-video pipeline performs a live UMT5 text encode (no prompt embeddings injected from the reference), denoises with classifier-free guidance over the Wan DiT, and decodes through the 3D-VAE. Against a diffusers reference with matched seed and schedule, early-step latents and output frames reach cosine 1.0 with frame PSNR ~89.8 dB, using a dependency-minimal build that links the CUDA runtime only (no cuBLAS). Scope: a correctness/oracle-parity validation on one consumer-class NVIDIA GPU run — not a generation-quality, throughput, or benchmark-scale result.
1. Contribution
Claimed. A self-contained CUDA text-to-video pipeline (Wan2.1) that performs a live UMT5 text encode — with nothing injected from the reference (no pre-computed prompt embeddings) — then denoises with classifier-free guidance over the Wan DiT and decodes through the 3D-VAE. Against the diffusers reference with matched seed and schedule, the early-step (step 0-3) latents and the output frames reach cosine 1.0 at frame PSNR ~89.8 dB. End-to-end wall time is ~92.5 s on a consumer-class NVIDIA GPU, and the build is dependency-minimal: a linkage check shows the CUDA runtime only, no cuBLAS. The altitude is correctness validation plus an engineering port, not a novel algorithm: it mirrors an existing AMD/ROCm standalone pipeline onto CUDA and pins numeric parity against diffusers.
NOT claimed. This does not establish generation quality (cosine/PSNR vs a reference are numeric-parity metrics, not perceptual or human-preference quality). It does not claim throughput or speed competitiveness as a benchmark — the ~92.5 s wall is a single end-to-end run, not a profiled, multi-trial throughput figure. It does not claim parity at higher resolution, longer clips, other Wan variants, other quantization, or on other GPU architectures. The speed contrast with an AMD/ROCm path (~92.5 s vs ~1130 s) is across different hardware and is reported as context, not as a controlled speedup.
2. Methods
| Field | Value |
|---|---|
| Hardware | Consumer-class NVIDIA GPU, 8 GB class |
| Stack / minimal-dependency build | CUDA toolchain; linkage check shows CUDA runtime only (no cuBLAS/cuDNN/cuTLASS) |
| Oracle / baseline | diffusers reference pipeline (matched seed/schedule); version not recorded in source notes |
| Procedure | Live UMT5 encode (no injected prompt embeddings) -> CFG denoise over Wan DiT -> 3D-VAE decode, compared to diffusers reference |
| Precision | not recorded in source notes |
| Metric + gate | cosine similarity and PSNR of latents/frames vs diffusers; gate not recorded in source notes |
| Stimulus / inputs | Single text-to-video generation with matched seed and schedule; prompt not recorded in source notes |
3. Results
| Quantity | Value |
|---|---|
| Early-step (0-3) latents (cosine vs diffusers) | 1.0 |
| Output frames (cosine vs diffusers) | 1.0 |
| Output frames PSNR vs diffusers | ~89.8 dB |
| End-to-end wall time (NVIDIA) | ~92.5 s |
| Dependency footprint | CUDA runtime only, no cuBLAS |
| Context: ROCm standalone wall (different HW) | ~1130 s |
4. Threats to validity / Limitations
- Sample size. Parity is established on what the source describes as a single end-to-end generation (early-step latents + frames). This is a correctness smoke test, not a benchmark-scale evaluation across prompts, seeds, resolutions, or clip lengths.
- Single run, no variance. The ~92.5 s wall is one run; no repeated trials, variance, or profiling are recorded. It is a wall-clock observation, not a throughput benchmark.
- Parity is not quality. Cosine 1.0 and PSNR ~89.8 dB measure numeric agreement with diffusers, not perceptual quality, temporal coherence, or human preference. A pipeline can match a reference numerically and still produce poor video if the reference is poor.
- Oracle/version pins missing. The diffusers version, precision (fp16/fp32), the parity gate threshold, the prompt, the seed value, and the schedule/step count are not recorded in the source notes. Reproducibility and the strength of the cosine 1.0 / PSNR claim depend on these and cannot be fully audited here.
- Cross-hardware contrast. The ~92.5 s vs ~1130 s comparison spans an NVIDIA/CUDA target vs an AMD/ROCm target — different silicon and backends — so it is not a controlled speedup measurement.
- Scope. Bounded to a self-contained Wan2.1 text-to-video pipeline on the NVIDIA target; no claim for other Wan variants, quantizations, or other GPU architectures.
- Evidence level. The source describes parity against the diffusers oracle, but the absence of recorded version/seed/gate pins (above) means the result should be read as oracle-parity-validated rather than independently re-run from a fully pinned recipe.
5. Reproducibility / Provenance
- Build: CUDA toolchain with optimization enabled. Run command: not recorded in source notes.
- Dependency footprint (linkage check): CUDA runtime only (no cuBLAS/cuDNN/cuTLASS).
- Version pins (CUDA, diffusers, UMT5/Wan weights): not recorded in source notes.
- Oracle caveat: validate the diffusers reference (seed, schedule, precision) before trusting the cosine 1.0 / PSNR ~89.8 dB parity; these pins are not captured in the source.
- Relation: CUDA counterpart to an AMD/ROCm standalone Wan2.1 text-to-video pipeline; correctness-pinned to the same diffusers reference.
Claim boundary
The author's explicit scope — what this work does and does not establish — carried over from the Octoryn Research publishing model.
Proves
- A self-contained CUDA Wan2.1 text-to-video pipeline with live UMT5 encode matches a diffusers reference at cosine 1.0 (early-step latents and frames) and frame PSNR ~89.8 dB, matched seed/schedule, with a CUDA-runtime-only build, on a single consumer-class NVIDIA GPU run.
Does not prove
- Generation quality, perceptual fidelity, or temporal coherence beyond numeric parity with the diffusers reference.
- Throughput or speed as a profiled benchmark; the single end-to-end wall time is one un-repeated run with no variance.
- Parity at higher resolution, longer clips, other Wan variants, other quantization, or on other GPU architectures.
- A controlled speedup vs an alternative GPU/backend path, since such a contrast spans different silicon and software stacks.
Applies when
- Self-contained Wan2.1 text-to-video on a single consumer-class NVIDIA GPU, single-config parity vs a diffusers reference with matched seed and schedule.
Does not apply when
- Generation-quality or throughput-benchmark claims; multi-prompt, multi-seed, or multi-resolution evaluation; other models, quantizations, or other GPU architectures.
Authors
- Ran Tao — Investigation, Writing
Cite this
Citation
Tao, R., Octoryn Research. (2026). True CUDA text-to-video with live UMT5 encode, dependency-minimal (no cuBLAS) (TR-2026-0022). Octopus Research Institute.
BibTeX
@techreport{oritr20260022,
title = {True CUDA text-to-video with live UMT5 encode, dependency-minimal (no cuBLAS)},
author = {Tao, Ran and {Octoryn Research}},
institution = {Octopus Research Institute},
year = {2026},
note = {Permanent ID TR-2026-0022. Not peer reviewed.}
}Disclosures
- Funding
- Hardware and infrastructure provided by Octoryn / Octopus Core Pty Ltd.
- Conflicts of interest
- Octoryn ships commercial inference and governance tooling; findings are reported independently.
