Press release
RiftStack's Emmy Compiler Generates CUDA Kernels That Beat Hand-Tuned Libraries on NVIDIA RTX 5090
August 1, 2026
RiftStack, the team behind the CloudRift GPU cloud platform, today published benchmark results for Emmy, its AI-driven ML compiler. On Gemma 4 12B, Emmy-generated CUDA kernels reached up to 1.6x the throughput of NVIDIA's cuBLASLt library on the RTX 5090, with a 1.30x geometric-mean speedup over PyTorch eager across all 277 deployed kernel shapes.
Production LLM serving still depends on hand-written GPU kernels. A large model needs hundreds of kernel shapes, each tuned per GPU, and existing AI-assisted approaches take weeks per model and card with a human in the loop. Emmy generates kernels automatically from a high-level description of the model and tunes them for specific shapes and hardware, covering the same optimizations found in hand-tuned libraries, including TMA transport and automatically derived GEMM and FlashAttention schedules.
“We built Emmy because kernel engineering is the bottleneck of the whole inference stack. When the compiler finds a schedule that beats a hand-tuned library, that win applies to every similar kernel automatically. That is the difference between optimizing one model and optimizing every model we serve.”
Shipped as a drop-in vLLM plugin, Emmy now beats stock vLLM on time to first token at every long-context point measured: 16 percent faster single-stream, 11 percent faster on RAG-style long-input serving, and 10 percent faster under a saturated 64-request load.
The team will demonstrate its inference stack at AI Infra Summit in Santa Clara, September 15 to 17, including a live demo session, "From Rack to API: An Enterprise Engine for High-Performance Inference," presented by CEO Dmitry Trifonov on Wednesday, September 16.
The full benchmark methodology and results are available at https://riftstack.ai/research/optimizing-gemma-4-12b-rtx.
About RiftStack
RiftStack builds the operating system for sovereign AI deployments: a control plane that turns datacenters into AI cloud offerings, with built-in LLM inference, GPU virtualization, and the Emmy ML compiler. RiftStack is SOC 2 certified and an NVIDIA Inception member.
Media contact: Heiko Polinski, heiko@cloudrift.ai.
All press releases