Evaluates elementary arithmetic, fractions, decimals, percentages, money, and simple word problems using plain-number answers.
Scorer: NumericDiff
loading chart
| # | score | model | cost price | ttft | total tok |
|---|---|---|---|---|---|
| 1 | 100% | qwen/qwen3.5-9b | $0.00005 $0.10 / $0.15 per 1M | 1.43s | 328.5 |
| 2 | 100% | gpt-5.6-sol | $0.00029 $5.00 / $30.00 per 1M | 0.41s | 30.2 |
| 3 | 100% | gemini-3.6-flash | $0.00106 $0.75 / $3.75 per 1M | 1.71s | 158.4 |