An inference optimization lab
We make models run faster and cheaper through compiler and serving research.
Today that means getting more out of consumer-class and commodity GPUs, not only top-end datacenter cards.
In the press
AUG 2026 · BENCHMARKS · LLM
Outperforming vLLM and Llama.cpp on Gemma4-12B
Emmy, an AI-driven compiler, generates CUDA kernels automatically instead of by hand. On Gemma 4 12B in FP16 its kernels reach up to 1.6× over cuBLAS on the RTX 5090, achieved via TMA transport, hybrid FP16/FP32 accumulation, and FlashAttention-2 parity, shipped as a drop-in vLLM plugin.
APR 2026 · BENCHMARKS · INFRASTRUCTURE & DEVOPS
GPU VM Performance: Do vCPU Pinning and NUMA Topology Really Matter?
We benchmarked six GPU VM setups to find out when vCPU pinning and guest NUMA topology improve performance, when they make no measurable difference, and when they hurt multi-GPU workloads.
JAN 2026 · BENCHMARKS · LLM
Blackwell Dominates. Benchmarking LLM Inference on NVIDIA B200, H200, H100, and RTX PRO 6000
We benchmarked NVIDIA B200, H200, H100, and RTX PRO 6000 for long-context LLM inference using 8K input + 8K output (16K total). B200 delivers up to 4.9× the throughput of RTX PRO 6000 and is now the cost efficiency leader across all models.
NOV 2025 · BENCHMARKS · LLM
RTX PRO 6000 vs H100, H200, and L40S: LLM Inference
RTX PRO 6000 beats H100 SXM on single-GPU LLM inference at 28% lower cost per token. H100 and H200 NVLink pull 3-4x ahead on 8-way tensor parallel.
AUG 2025 · AI TOOLS & WORKFLOWS · LLM
How to Give Your RTX GPU Nearly Infinite Memory for LLM Inference
Network-Attached KV Cache for Long-Context, Multi-Turn Workloads. Let's be honest — we can't afford an H100. Learn how to extend your RTX GPU's effective memory using innovative KV cache offloading techniques.
Inference performance
Generated kernels that beat cuBLAS
Emmy tunes them for the exact GPU in front of it, and ships as a drop-in vLLM plugin. Here is where that lands on Gemma 4 12B in FP16 on the RTX 5090.
Up to 1.6× GEMM
Emmy-generated kernels against cuBLAS on the RTX 5090, via TMA transport and hybrid FP16/FP32 accumulation.
1.30× geomean
Speedup over PyTorch eager across all 277 deployed Gemma 4 12B kernel shapes. The slowest shapes still trail it.
−16% TTFT
Against stock vLLM, single-stream long context. The per-kernel wins only partly carry to raw throughput.