Research · Inference
Inference
LLM inference benchmarks, model by model and GPU by GPU — throughput, latency, and memory, with the limits of each result stated.
MAR 2026 · BENCHMARKS · LLM
Optimizing Qwen3 Coder for RTX 5090 and PRO 6000
I got Qwen3 Coder from 277 tok/s to 1,207 tok/s on a PRO 6000, and from 556 to 1,157 tok/s on an RTX 5090. Here's exactly how, with reproducible recipes.
JAN 2026 · BENCHMARKS · LLM
Blackwell Dominates. Benchmarking LLM Inference on NVIDIA B200, H200, H100, and RTX PRO 6000
We benchmarked NVIDIA B200, H200, H100, and RTX PRO 6000 for long-context LLM inference using 8K input + 8K output (16K total). B200 delivers up to 4.9× the throughput of RTX PRO 6000 and is now the cost efficiency leader across all models.
NOV 2025 · BENCHMARKS · LLM
RTX PRO 6000 vs H100, H200, and L40S: LLM Inference
RTX PRO 6000 beats H100 SXM on single-GPU LLM inference at 28% lower cost per token. H100 and H200 NVLink pull 3-4x ahead on 8-way tensor parallel.
OCT 2025 · BENCHMARKS · LLM
RTX 4090 vs 5090 vs PRO 6000: LLM Inference Benchmark
RTX 4090, 5090, and PRO 6000 benchmarked on vLLM with three quantized models (24GB, 48GB, 96GB). PRO 6000 hits 3.7x the 4090 throughput on single GPU.
SEP 2025 · BENCHMARKS · LLM
Benchmarking LLM Inference on RTX 4090, RTX 5090, and RTX PRO 6000
We ran a series of benchmarks across multiple GPU cloud servers to evaluate their performance for LLM workloads, specifically serving LLaMA and Qwen models and on RTX 4090, RTX 5090, and RTX PRO 6000 GPUs.
AUG 2025 · AI TOOLS & WORKFLOWS · LLM
How to Give Your RTX GPU Nearly Infinite Memory for LLM Inference
Network-Attached KV Cache for Long-Context, Multi-Turn Workloads. Let's be honest — we can't afford an H100. Learn how to extend your RTX GPU's effective memory using innovative KV cache offloading techniques.