Hardware Benchmarks - Ling-3.0-flash

inclusionAI Ling-3.0-flash — 124B total / 5.1B active MoE, hybrid Mamba+MLA attention, 256K context. Requires a vendor vLLM fork (inclusionAI/vllm, branch ling_3_0).

Results

Engine

Accelerators

Comparing 3 setups — 3 config differences

hardware	vllm-ling-3p0-flash-8xh200 (spot)	vllm-ling-3p0-flash-8xh200 (on-demand)	vllm-ling-3p0-flash-2xb200-runpod
engine	`vllm`	`vllm`	`vllm`
accelerators	`H200:8`	`H200:8`	`B200:2`

parameter	vllm-ling-3p0-flash-8xh200 (spot)	vllm-ling-3p0-flash-8xh200 (on-demand)	vllm-ling-3p0-flash-2xb200-runpod
`max-num-seqs`	`—`	`—`	`384`
`moe-backend`	`—`	`—`	`triton`
`tensor-parallel-size`	`4`	`4`	`2`

vllm-ling-3p0-flash-8xh200 (spot)
vllm-ling-3p0-flash-8xh200 (on-demand)
vllm-ling-3p0-flash-2xb200-runpod

Leaderboard

#	chart	setup	hardware	config	tok/s	$/1M completion
1		vllm-ling-3p0-flash-8xh200 (spot)	vllm · H200:8	TP=4+8 KV cache `enable-prefix-caching` `true` Enable prefix caching for shared prompt prefixes. `gpu-memory-utilization` `0.85` Fraction of GPU memory for the model executor (0–1). Per-instance limit. Model `trust-remote-code` `true` Trust remote code when downloading model and tokenizer. Parallelism `tensor-parallel-size` `4` Number of tensor-parallel groups. Other `enable_auto_tool_choice` `true` `mamba_cache_mode` `align` `reasoning_parser` `ling3` `speculative_config` `{"method": "mtp", "num_speculative_tokens": 3}` `tool_call_parser` `ling3`	10,950.6	$0.8570
2		vllm-ling-3p0-flash-8xh200 (on-demand)	vllm · H200:8	TP=4+8 KV cache `enable-prefix-caching` `true` Enable prefix caching for shared prompt prefixes. `gpu-memory-utilization` `0.85` Fraction of GPU memory for the model executor (0–1). Per-instance limit. Model `trust-remote-code` `true` Trust remote code when downloading model and tokenizer. Parallelism `tensor-parallel-size` `4` Number of tensor-parallel groups. Other `enable_auto_tool_choice` `true` `mamba_cache_mode` `align` `reasoning_parser` `ling3` `speculative_config` `{"method": "mtp", "num_speculative_tokens": 3}` `tool_call_parser` `ling3`	10,950.6	$2.0202
3		vllm-ling-3p0-flash-2xb200-runpod	vllm · B200:2	TP=2 · max_seqs=384+9 Kernels `moe-backend` `triton` MoE expert computation kernel backend. KV cache `enable-prefix-caching` `true` Enable prefix caching for shared prompt prefixes. `gpu-memory-utilization` `0.85` Fraction of GPU memory for the model executor (0–1). Per-instance limit. Model `trust-remote-code` `true` Trust remote code when downloading model and tokenizer. Parallelism `tensor-parallel-size` `2` Number of tensor-parallel groups. Scheduler `max-num-seqs` `384` Max sequences processed in a single iteration. Other `enable_auto_tool_choice` `true` `mamba_cache_mode` `align` `reasoning_parser` `ling3` `speculative_config` `{"method": "mtp", "num_speculative_tokens": 3}` `tool_call_parser` `ling3`	5,352.2	$0.6684