TinyLlama 1.1B Chat — lightweight model for high-concurrency load testing on modest GPUs.
Results
- vllm
Leaderboard
| # | chart | setup | hardware | config | tok/s | $/1M completion |
|---|---|---|---|---|---|---|
| 1 | vllm | vllm · A10G:1 | defaultsDefaults only | 11,235.3 | $0.0310 |
TinyLlama 1.1B Chat — lightweight model for high-concurrency load testing on modest GPUs.
| # | chart | setup | hardware | config | tok/s | $/1M completion |
|---|---|---|---|---|---|---|
| 1 | vllm | vllm · A10G:1 | defaultsDefaults only | 11,235.3 | $0.0310 |