Hardware Benchmarks - gpt-oss-120b

OpenAI gpt-oss-120b reasoning model, native MXFP4 (~65GB). Fits a single H100 80GB.

Results

Engine

Accelerators

Comparing 6 setups — 11 config differences

hardware	vllm-gpt-oss-120b-a10g	vllm-gpt-oss-120b-h200-bigcode	vllm-gpt-oss-120b-b200-bigcode	vllm-gpt-oss-120b-h100-bigcode	vllm-gpt-oss-120b-4xa40-bigcode	vllm-gpt-oss-120b-2xl40s-bigcode
engine	`vllm`	`vllm`	`vllm`	`vllm`	`vllm`	`vllm`
accelerators	`A10G:4`	`H200-SXM:1`	`B200:1`	`H100-SXM:1`	`A40:4`	`L40S:2`

parameter	vllm-gpt-oss-120b-a10g	vllm-gpt-oss-120b-h200-bigcode	vllm-gpt-oss-120b-b200-bigcode	vllm-gpt-oss-120b-h100-bigcode	vllm-gpt-oss-120b-4xa40-bigcode	vllm-gpt-oss-120b-2xl40s-bigcode
`NCCL_CUMEM_ENABLE`	`—`	`—`	`—`	`—`	`0`	`0`
`NCCL_DEBUG`	`—`	`—`	`—`	`—`	`INFO`	`INFO`
`NCCL_P2P_DISABLE`	`—`	`—`	`—`	`—`	`1`	`1`
`VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8`	`—`	`—`	`1`	`—`	`—`	`—`
`disable-custom-all-reduce`	`—`	`—`	`—`	`—`	`true`	`true`
`enable_auto_tool_choice`	`—`	`true`	`true`	`true`	`true`	`true`
`enable-chunked-prefill`	`true`	`—`	`—`	`—`	`—`	`—`
`gpu-memory-utilization`	`0.9`	`—`	`—`	`—`	`—`	`—`
`max-model-len`	`—`	`—`	`—`	`—`	`32768`	`32768`
`tensor-parallel-size`	`4`	`1`	`1`	`1`	`4`	`2`
`tool_call_parser`	`—`	`openai`	`openai`	`openai`	`openai`	`openai`

vllm-gpt-oss-120b-a10g
vllm-gpt-oss-120b-h200-bigcode
vllm-gpt-oss-120b-b200-bigcode
vllm-gpt-oss-120b-h100-bigcode
vllm-gpt-oss-120b-4xa40-bigcode
vllm-gpt-oss-120b-2xl40s-bigcode

Leaderboard

#	chart	setup	hardware	config	tok/s	$/1M completion
1		vllm-gpt-oss-120b-b200-bigcode	vllm · B200:1	TP=1+3 Parallelism `tensor-parallel-size` `1` Number of tensor-parallel groups. Environment `VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8` `1` Other `enable_auto_tool_choice` `true` `tool_call_parser` `openai`	18,255	$0.1046
2		vllm-gpt-oss-120b-h200-bigcode	vllm · H200-SXM:1	TP=1+2 Parallelism `tensor-parallel-size` `1` Number of tensor-parallel groups. Other `enable_auto_tool_choice` `true` `tool_call_parser` `openai`	14,426.4	$0.0996
3		vllm-gpt-oss-120b-h100-bigcode	vllm · H100-SXM:1	TP=1+2 Parallelism `tensor-parallel-size` `1` Number of tensor-parallel groups. Other `enable_auto_tool_choice` `true` `tool_call_parser` `openai`	6,852.2	$0.1425
4		vllm-gpt-oss-120b-4xa40-bigcode	vllm · A40:4	TP=4+7 Distributed `disable-custom-all-reduce` `true` Disable custom all-reduce kernel and fall back to NCCL. Model `max-model-len` `32768` Context length (prompt + output). Supports k/m suffixes or auto to fit GPU memory. Parallelism `tensor-parallel-size` `4` Number of tensor-parallel groups. Environment `NCCL_CUMEM_ENABLE` `0` `NCCL_DEBUG` `INFO` `NCCL_P2P_DISABLE` `1` Other `enable_auto_tool_choice` `true` `tool_call_parser` `openai`	3,669.7	$0.1566
5		vllm-gpt-oss-120b-2xl40s-bigcode	vllm · L40S:2	TP=2+7 Distributed `disable-custom-all-reduce` `true` Disable custom all-reduce kernel and fall back to NCCL. Model `max-model-len` `32768` Context length (prompt + output). Supports k/m suffixes or auto to fit GPU memory. Parallelism `tensor-parallel-size` `2` Number of tensor-parallel groups. Environment `NCCL_CUMEM_ENABLE` `0` `NCCL_DEBUG` `INFO` `NCCL_P2P_DISABLE` `1` Other `enable_auto_tool_choice` `true` `tool_call_parser` `openai`	3,595.8	$0.1839
6		vllm-gpt-oss-120b-a10g	vllm · A10G:4	TP=4 · chunked-prefill=true+1 KV cache `gpu-memory-utilization` `0.9` Fraction of GPU memory for the model executor (0–1). Per-instance limit. Parallelism `tensor-parallel-size` `4` Number of tensor-parallel groups. Scheduler `enable-chunked-prefill` `true` Chunk long prefills based on remaining batched-token budget.	758.2	$2.4265