Contact us Download containers
Contact us Download containers

Download optimized containers for your model / GPU / workload

Every container runs next to vLLM in your own cloud account. Same engine, same API, same weights. The serving configuration is what changes.

Browse the container library See how a container gets measured

qwen-black Agents

Qwen/Qwen3.6-27B-FP8 · FP8

-87%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Download
qwen-black Chat

Qwen/Qwen3.6-27B-FP8 · FP8

-85%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Download
qwen-black Agents

Qwen/Qwen3.8-27B-FP8 · FP8

-90%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Download
qwen-black Chat

Qwen/Qwen3.8-27B-FP8 · FP8

-85%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Download
qwen-black Chat

Qwen/Qwen3-100B · FP8

-39%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 2xH100Stack vLLM
premium Get notified
qwen-black Coding

Qwen/Qwen3-Coder-next · FP8

-26%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 4xH100Stack vLLM
premium Get notified
qwen-black Coding

Qwen/Qwen3-Coder-32B · FP8

-13%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU A100Stack vLLM
free Get notified
qwen-black Chat

Qwen/Qwen3-32B · FP8

-12%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU A100Stack vLLM
free Get notified
qwen-black Coding

Qwen3.5-32B · FP8

Bench in progress
Container and numbers on publish day.
GPU H100Stack
free Get notified
llama-black Chat

Llama-3.3-8B · INT4

Bench in progress
Container and numbers on publish day.
GPU L40SStack
free Get notified
mistral-black Chat

Mixtral-9x22B · FP16

Bench in progress
Container and numbers on publish day.
GPU 2xA100Stack
free Get notified
phi-black Agents

Phi-4-Reasoning · FP16

Bench in progress
Container and numbers on publish day.
GPU L40SStack
free Get notified
llama-black Chat

Llama-3.3-70B-Instruct · FP16

Bench in progress
Container and numbers on publish day.
GPU H100Stack
free Get notified
mistral-black Agents

Mistral-Large-3 · FP8

Bench in progress
Container and numbers on publish day.
GPU 2xH100Stack
free Get notified
cohere-black Agents

Command R2-Plus · FP8

Bench in progress
Container and numbers on publish day.
GPU 2xH100Stack
free Get notified
gemma-black Chat

Gemma-3-27B · FP16

Bench in progress
Container and numbers on publish day.
GPU A100Stack
free Get notified
deepseek-black Coding

DeepSeek-Coder-V3 · FP8

Bench in progress
Container and numbers on publish day.
GPU H100Stack
free Get notified
qwen-black Chat

Qwen3.5-14B · FP8

Bench in progress
Container and numbers on publish day.
GPU L40SStack
free Get notified
kimi-black Coding

Kimi-K2.7-Code · FP8

Bench in progress
Container and numbers on publish day.
GPU 2xH100Stack
free Get notified
minimax-black Agents

MiniMax-M3 · FP8

Bench in progress
Container and numbers on publish day.
GPU 2xH100Stack
free Get notified
gpt-oss-black Chat

GPT-OSS-120B · FP8

Bench in progress
Container and numbers on publish day.
GPU H100Stack
free Get notified
glm-black Coding

GLM-5.2-Air · INT4

Bench in progress
Container and numbers on publish day.
GPU A100Stack
free Get notified
deepseek-black Agents

DeepSeek-R1-Distill-14B · FP16

Bench in progress
Container and numbers on publish day.
GPU A100Stack
free Get notified
cohere-black Chat

Aya-Vision-35B · FP16

Bench in progress
Container and numbers on publish day.
GPU A100Stack
free Get notified
phi-black Chat

Phi-4-Vision · FP8

Bench in progress
Container and numbers on publish day.
GPU A100Stack
free Get notified
gemma-black Chat

Gemma-3-12B · INT4

Bench in progress
Container and numbers on publish day.
GPU L40SStack
free Get notified
deepseek-black Agents

DeepSeek-R2-Distill-32B · FP16

Bench in progress
Container and numbers on publish day.
GPU A100Stack
free Get notified
qwen-black Agents

Qwen3.5-72B-Instruct · FP8

Bench in progress
Container and numbers on publish day.
GPU H100Stack
free Get notified
glm-black Agents

GLM-5.2 · FP8

Bench in progress
Container and numbers on publish day.
GPU H100Stack
free Get notified
deepseek-black Agents

DeepSeek-V3.2 · FP8

Bench in progress
Container and numbers on publish day.
GPU 2xH100Stack
free Get notified
kimi-black Chat

Kimi-K2.6 · FP8

Bench in progress
Container and numbers on publish day.
GPU H100Stack
free Get notified
minimax-black Chat

MiniMax-Text-01 · FP16

Bench in progress
Container and numbers on publish day.
GPU 2xA100Stack
free Get notified
gpt-oss-black Coding

GPT-OSS-20B · INT4

Bench in progress
Container and numbers on publish day.
GPU L40SStack
free Get notified
cohere-black Chat

Command R2 · FP16

Bench in progress
Container and numbers on publish day.
GPU A100Stack
free Get notified
mistral-black Coding

Codestral-2-32B · FP16

Bench in progress
Container and numbers on publish day.
GPU A100Stack
free Get notified
qwen-black Agents

Qwen4-235B-Preview · FP8

Bench in progress
Container and numbers on publish day.
GPU 4xB200Stack
free Get notified
llama-black Agents

Llama-4-Scout · FP8

Bench in progress
Container and numbers on publish day.
GPU 2xB200Stack
free Get notified
qwen-black Chat

Qwen3.6-27B · FP8

Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack
free Get notified
glm-black Chat

GLM-5.2 · INT4

Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack
free Get notified
glm-black Agents

GLM-5.2 · INT4

Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack
free Get notified
deepseek-black Agents

deepseek-ai/DeepSeek-V4-Flash · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
deepseek-black Agents

deepseek-ai/DeepSeek-V4-Flash · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
deepseek-black Chat

deepseek-ai/DeepSeek-V4-Flash · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
deepseek-black Chat

deepseek-ai/DeepSeek-V4-Flash · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
deepseek-black Coding

deepseek-ai/DeepSeek-V4-Flash · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
deepseek-black Coding

deepseek-ai/DeepSeek-V4-Flash · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
gemma-black Agents

google/gemma-4-12B-it · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
gemma-black Agents

google/gemma-4-12B-it · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
gemma-black Chat

google/gemma-4-12B-it · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
gemma-black Chat

google/gemma-4-12B-it · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
gemma-black Coding

google/gemma-4-12B-it · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
gemma-black Coding

google/gemma-4-12B-it · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
gemma-black Agents

google/gemma-4-31B-it · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
gemma-black Agents

google/gemma-4-31B-it · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
gemma-black Chat

google/gemma-4-31B-it · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
gemma-black Chat

google/gemma-4-31B-it · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
gemma-black Coding

google/gemma-4-31B-it · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
gemma-black Coding

google/gemma-4-31B-it · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
glm-black Agents

zai-org/GLM-4.7-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
glm-black Agents

zai-org/GLM-4.7-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
glm-black Chat

zai-org/GLM-4.7-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
glm-black Chat

zai-org/GLM-4.7-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
glm-black Coding

zai-org/GLM-4.7-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
glm-black Coding

zai-org/GLM-4.7-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
llama-black Agents

meta-llama/Llama-3.1-8B-Instruct · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
llama-black Agents

meta-llama/Llama-3.1-8B-Instruct · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
llama-black Chat

meta-llama/Llama-3.1-8B-Instruct · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
llama-black Chat

meta-llama/Llama-3.1-8B-Instruct · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
llama-black Coding

meta-llama/Llama-3.1-8B-Instruct · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
llama-black Coding

meta-llama/Llama-3.1-8B-Instruct · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
llama-black Agents

nvidia/Llama-3.3-70B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
llama-black Agents

nvidia/Llama-3.3-70B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
llama-black Chat

nvidia/Llama-3.3-70B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
llama-black Chat

nvidia/Llama-3.3-70B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
llama-black Coding

nvidia/Llama-3.3-70B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
llama-black Coding

nvidia/Llama-3.3-70B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
minimax-black Agents

MiniMaxAI/MiniMax-M2.7 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
minimax-black Agents

MiniMaxAI/MiniMax-M2.7 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
minimax-black Chat

MiniMaxAI/MiniMax-M2.7 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
minimax-black Chat

MiniMaxAI/MiniMax-M2.7 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
minimax-black Coding

MiniMaxAI/MiniMax-M2.7 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
minimax-black Coding

MiniMaxAI/MiniMax-M2.7 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
mistral-black Agents

mistralai/Ministral-3-8B-Instruct-2512 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
mistral-black Agents

mistralai/Ministral-3-8B-Instruct-2512 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
mistral-black Chat

mistralai/Ministral-3-8B-Instruct-2512 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
mistral-black Chat

mistralai/Ministral-3-8B-Instruct-2512 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
mistral-black Coding

mistralai/Ministral-3-8B-Instruct-2512 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
mistral-black Coding

mistralai/Ministral-3-8B-Instruct-2512 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
mistral-black Agents

mistralai/Mistral-Medium-3.5-128B · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
mistral-black Agents

mistralai/Mistral-Medium-3.5-128B · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
mistral-black Chat

mistralai/Mistral-Medium-3.5-128B · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
mistral-black Chat

mistralai/Mistral-Medium-3.5-128B · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
mistral-black Coding

mistralai/Mistral-Medium-3.5-128B · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
mistral-black Coding

mistralai/Mistral-Medium-3.5-128B · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
nemotron-black Agents

nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
nemotron-black Agents

nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
nemotron-black Chat

nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
nemotron-black Chat

nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
nemotron-black Coding

nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
nemotron-black Coding

nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
nemotron-black Agents

nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
nemotron-black Agents

nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
nemotron-black Chat

nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
nemotron-black Chat

nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
nemotron-black Coding

nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
nemotron-black Coding

nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
phi-black Agents

microsoft/Phi-4-reasoning · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
phi-black Agents

microsoft/Phi-4-reasoning · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
phi-black Chat

microsoft/Phi-4-reasoning · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
phi-black Chat

microsoft/Phi-4-reasoning · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
phi-black Coding

microsoft/Phi-4-reasoning · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
phi-black Coding

microsoft/Phi-4-reasoning · FP16

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
qwen-black Chat

Qwen/Qwen2.5-0.5B-Instruct · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU L4Stack vLLM
free Get notified
qwen-black Agents

Qwen/Qwen3-32B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
qwen-black Agents

Qwen/Qwen3-32B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
qwen-black Chat

Qwen/Qwen3-32B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
qwen-black Chat

Qwen/Qwen3-32B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
qwen-black Agents

Qwen/Qwen3.5-27B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
qwen-black Agents

Qwen/Qwen3.5-27B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
qwen-black Chat

Qwen/Qwen3.5-27B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
qwen-black Chat

Qwen/Qwen3.5-27B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
qwen-black Coding

Qwen/Qwen3.5-27B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
qwen-black Coding

Qwen/Qwen3.5-27B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
qwen-black Agents

Qwen/Qwen3.6-27B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
qwen-black Coding

Qwen/Qwen3.6-27B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
qwen-black Coding

Qwen/Qwen3.6-27B-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
qwen-black Agents

Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
qwen-black Agents

Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
qwen-black Chat

Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
qwen-black Chat

Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
qwen-black Coding

Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
qwen-black Coding

Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
qwen-black Agents

Qwen/Qwen3-Next-80B-A3B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
qwen-black Agents

Qwen/Qwen3-Next-80B-A3B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
qwen-black Chat

Qwen/Qwen3-Next-80B-A3B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
qwen-black Chat

Qwen/Qwen3-Next-80B-A3B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified
qwen-black Coding

Qwen/Qwen3-Next-80B-A3B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xA100Stack vLLM
free Get notified
qwen-black Coding

Qwen/Qwen3-Next-80B-A3B-Instruct-FP8 · FP8

0%$/M tokens
vs Stock vLLM · same latency target
Bench in progress
Container and numbers on publish day.
GPU 8xH100Stack vLLM
free Get notified

How a number gets on a card

Every reduction on this page is measured against stock vLLM on the GPU named on the card, at a fixed latency target, with the same weights and the same request mix. Nothing is extrapolated from one GPU class to another.

A container appears in the library before its benchmark is finished. Until the measurement is signed off, the card shows the shape of the result and not the number.

Unit of analysis: model × precision × GPU class. A container measured on A100 does not carry its number to H100.

 

What we hold fixed

1
Same weights, same precision, no distillation and no pruning.
Same weights, same precision, no distillation and no pruning.
2
Same engine. Stock vLLM is the baseline, not a tuned competitor.
Same engine. Stock vLLM is the baseline, not a tuned competitor.
3
Same latency target. Throughput gained by breaking your SLO does not count.
Same latency target. Throughput gained by breaking your SLO does not count.
4
Same GPU class and count as the card states.
Same GPU class and count as the card states.

Your workload is not in the library yet

We measure your production baseline on your own GPUs first.
You keep the measurement whether or not you go further.

Talk to an engineer Request a model