Research
Research in the open
We publish model by model and GPU by GPU, with the limits of each result stated.
AUG 2026 · BENCHMARKS · LLM
Outperforming vLLM and Llama.cpp on Gemma4-12B
Emmy, an AI-driven compiler, generates CUDA kernels automatically instead of by hand. On Gemma 4 12B in FP16 its kernels reach up to 1.6× over cuBLAS on the RTX 5090, achieved via TMA transport, hybrid FP16/FP32 accumulation, and FlashAttention-2 parity, shipped as a drop-in vLLM plugin.
JUL 2026 · LLM · OPEN SOURCE
The State of Open Coding AI Models in July 2026
A data report on open-weight coding models in July 2026: what agentic developers actually run, how far the open-to-closed gap has closed, and why the best open coders have outgrown the hardware most people own.
JUL 2026 · TUTORIALS · GPU
Learning FlashAttention the Hard Way — Part 2
Part 2 takes Part 1's twisted monoid down to the metal: a generic compiler moveset walks one attention kernel from 25959µs to 206µs on an RTX 5090 — on par with FlashAttention-2 — with a measured number for every rung, the optimizations measurement refused, and the reasons FA-3/4 cannot land on consumer silicon.
JUL 2026 · TUTORIALS · ALGEBRA
Learning FlashAttention the Hard Way — Part 1
FlashAttention looks like a bespoke GPU kernel, but it is not much different from a regular parallel reduction. Safe softmax becomes an associative reduction via denominator trick, Welford's variance, and flash attention is the same concept. This part is the complete theoretical foundation, with the numerical-stability analysis and a test for when any loop is secretly associative.
MAY 2026 · TUTORIALS · GPU
Modern GPU Matmul Optimization
How to optimize a matmul kernel on a modern GPU, one optimization at a time: register tiling, vectorized loads, shared-memory staging, cp.async, TMA, warp specialization, split-K, and tensor cores. Each is introduced, then demonstrated by toggling it on a real kernel and watching the generated code diff and the latency change on an RTX 5090.
MAY 2026 · TUTORIALS · GPU
A Principled ML Compiler Stack in 5,000 Lines of Python — Part 3
Part 3 of the from-scratch ML compiler walkthrough: replacing the hand-coded heuristics from part 2 with an SP-MCTS search loop over Tile-IR rewrite parameters. The same six-IR pipeline, the same sixteen rules — just a tree walk on top picking the parameters that bench fastest.
MAY 2026 · TUTORIALS · GPU
A Principled ML Compiler Stack in 5,000 Lines of Python — Part 2
Part 2 of the from-scratch ML compiler walkthrough: how Loop IR is lowered to a GPU schedule. An overview of the Tile IR through three running examples (pointwise, reduction, matmul) where each picks up the rules they need from a stack of small rewrites.
APR 2026 · TUTORIALS · GPU
A Principled ML Compiler Stack in 5,000 Lines of Python — Part 1
ML compilers look like black boxes. They're not. I built one from scratch (tracing, fusion, scheduling, CUDA codegen) in 5,000 lines of Python. I walk a transformer's RMSNorm layer from PyTorch through decomposition, fusion, and tile-level scheduling, ending with the emitted CUDA kernel.
APR 2026 · BENCHMARKS · INFRASTRUCTURE & DEVOPS
GPU VM Performance: Do vCPU Pinning and NUMA Topology Really Matter?
We benchmarked six GPU VM setups to find out when vCPU pinning and guest NUMA topology improve performance, when they make no measurable difference, and when they hurt multi-GPU workloads.
APR 2026 · TUTORIALS · GPU
Surfacing a 60% performance bug in cuBLAS
While benchmarking an FP32 SGEMM kernel on the RTX 5090, I found cuBLAS dispatching a tiny kernel for huge batched workloads — stuck at ~40% FMA utilization across the entire size range. The same library binary correctly escalates to 73% on the RTX PRO 6000 and 82% on the H200.
APR 2026 · TUTORIALS · GPU VIRTUALIZATION
GPU Virtualization with VFIO, NVIDIA AI Enterprise, and AMD SR-IOV
A deep dive into the three GPU virtualization strategies we use at CloudRift: VFIO passthrough, NVIDIA MIG with AI Enterprise vGPU drivers, and AMD SR-IOV with ROCm. Covers the host-side mechanics, driver lifecycle, domain XML configuration, and the trade-offs of each approach.
MAR 2026 · BENCHMARKS · LLM
Optimizing Qwen3 Coder for RTX 5090 and PRO 6000
I got Qwen3 Coder from 277 tok/s to 1,207 tok/s on a PRO 6000, and from 556 to 1,157 tok/s on an RTX 5090. Here's exactly how, with reproducible recipes.
JAN 2026 · BENCHMARKS · LLM
Blackwell Dominates. Benchmarking LLM Inference on NVIDIA B200, H200, H100, and RTX PRO 6000
We benchmarked NVIDIA B200, H200, H100, and RTX PRO 6000 for long-context LLM inference using 8K input + 8K output (16K total). B200 delivers up to 4.9× the throughput of RTX PRO 6000 and is now the cost efficiency leader across all models.
NOV 2025 · BENCHMARKS · LLM
RTX PRO 6000 vs H100, H200, and L40S: LLM Inference
RTX PRO 6000 beats H100 SXM on single-GPU LLM inference at 28% lower cost per token. H100 and H200 NVLink pull 3-4x ahead on 8-way tensor parallel.
OCT 2025 · BENCHMARKS · LLM
RTX 4090 vs 5090 vs PRO 6000: LLM Inference Benchmark
RTX 4090, 5090, and PRO 6000 benchmarked on vLLM with three quantized models (24GB, 48GB, 96GB). PRO 6000 hits 3.7x the 4090 throughput on single GPU.
SEP 2025 · BENCHMARKS · LLM
Benchmarking LLM Inference on RTX 4090, RTX 5090, and RTX PRO 6000
We ran a series of benchmarks across multiple GPU cloud servers to evaluate their performance for LLM workloads, specifically serving LLaMA and Qwen models and on RTX 4090, RTX 5090, and RTX PRO 6000 GPUs.
AUG 2025 · TUTORIALS · GPU PASSTHROUGH
Host Setup for Qemu KVM GPU Passthrough with VFIO on Linux
GPU passthrough shouldn’t feel like sorcery. If you’ve ever lost a weekend to half-working configs, random resets, or a guest that only boots when the moon is right, this guide is for you.
AUG 2025 · AI TOOLS & WORKFLOWS · LLM
How to Give Your RTX GPU Nearly Infinite Memory for LLM Inference
Network-Attached KV Cache for Long-Context, Multi-Turn Workloads. Let's be honest — we can't afford an H100. Learn how to extend your RTX GPU's effective memory using innovative KV cache offloading techniques.
AUG 2025 · INFRASTRUCTURE & DEVOPS · GPU
Bug Bounty: NVidia Reset Bug
Hey everyone — we're building a next-gen GPU cloud for AI developers at CloudRift, and we've run into a frustrating issue that's proven nearly impossible to debug. We're turning to the community for help.