by @querulous-deer
Tests whether free LLMs can accurately count occurrences of a specified letter within long, uncommon English words.
Example: How many times does a appear in palaeodictyopterous? → 2
Words are sampled from the NLTK corpus (length > 8 characters). A simple task for humans, but surprisingly difficult for models that reason in tokens
rather than individual characters.
Best in Class
Overpriced
Your Selection
Click on the model to make selection
| Config | Total Cost | Quality | Username | Position | |
|---|---|---|---|---|---|
| z-ai/glm-4.5-air:free | $0.0000 | 100.0% | @querulous-deer | #1 | |
| arcee-ai/trinity-mini:free | $0.0000 | 100.0% | @querulous-deer | #1 | |
| openai/gpt-oss-120b:free | $0.0000 | 98.0% | @querulous-deer | #3 | |
| openai/gpt-oss-20b:free | $0.0000 | 96.0% | @querulous-deer | #4 | |
| nvidia/nemotron-3-nano-30b-a3b... | $0.0000 | 92.0% | @querulous-deer | #5 | |
| deepseek/deepseek-r1-0528:free | $0.0000 | 78.0% | @querulous-deer | #6 | |
| qwen/qwen3-coder:free | $0.0000 | 62.0% | @querulous-deer | #7 | |
| meta-llama/llama-3.3-70b-instr... | $0.0000 | 58.0% | @querulous-deer | #8 | |
| nvidia/nemotron-nano-9b-v2:fre... | $0.0000 | 56.0% | @querulous-deer | #9 | |
| google/gemma-3-27b-it:free | $0.0000 | 56.0% | @querulous-deer | #9 | |
| mistralai/mistral-small-3.1-24... | $0.0000 | 52.0% | @querulous-deer | #11 | |
| meta-llama/llama-3.1-405b-inst... | $0.0000 | 52.0% | @querulous-deer | #11 | |
| nousresearch/hermes-3-llama-3.... | $0.0000 | 44.0% | @querulous-deer | #13 | |
| google/gemma-3-4b-it:free | $0.0000 | 44.0% | @querulous-deer | #13 | |
| qwen/qwen3-next-80b-a3b-instru... | $0.0000 | 40.0% | @querulous-deer | #15 | |
| google/gemma-3-12b-it:free | $0.0000 | 40.0% | @querulous-deer | #15 | |
| qwen/qwen-2.5-vl-7b-instruct:f... | $0.0000 | 38.0% | @querulous-deer | #17 | |
| google/gemma-3n-e4b-it:free | $0.0000 | 36.0% | @querulous-deer | #18 | |
| moonshotai/kimi-k2:free | $0.0000 | 26.0% | @querulous-deer | #19 | |
| xiaomi/mimo-v2-flash:free | $0.0000 | 24.0% | @querulous-deer | #20 | |
| nvidia/nemotron-nano-12b-v2-vl... | $0.0000 | 24.0% | @querulous-deer | #20 | |
| meta-llama/llama-3.2-3b-instru... | $0.0000 | 4.0% | @querulous-deer | #22 |
| Total Cost | Quality | |
|---|---|---|
| z-ai/glm-4.5-air:free | $0.0000 | 100.0% |
| arcee-ai/trinity-mini:free | $0.0000 | 100.0% |
| openai/gpt-oss-120b:free | $0.0000 | 98.0% |
| openai/gpt-oss-20b:free | $0.0000 | 96.0% |
| nvidia/nemotron-3-nano-30b-a3b... | $0.0000 | 92.0% |
| deepseek/deepseek-r1-0528:free | $0.0000 | 78.0% |
| qwen/qwen3-coder:free | $0.0000 | 62.0% |
| meta-llama/llama-3.3-70b-instr... | $0.0000 | 58.0% |
| nvidia/nemotron-nano-9b-v2:fre... | $0.0000 | 56.0% |
| google/gemma-3-27b-it:free | $0.0000 | 56.0% |
| mistralai/mistral-small-3.1-24... | $0.0000 | 52.0% |
| meta-llama/llama-3.1-405b-inst... | $0.0000 | 52.0% |
| nousresearch/hermes-3-llama-3.... | $0.0000 | 44.0% |
| google/gemma-3-4b-it:free | $0.0000 | 44.0% |
| qwen/qwen3-next-80b-a3b-instru... | $0.0000 | 40.0% |
| google/gemma-3-12b-it:free | $0.0000 | 40.0% |
| qwen/qwen-2.5-vl-7b-instruct:f... | $0.0000 | 38.0% |
| google/gemma-3n-e4b-it:free | $0.0000 | 36.0% |
| moonshotai/kimi-k2:free | $0.0000 | 26.0% |
| xiaomi/mimo-v2-flash:free | $0.0000 | 24.0% |
| nvidia/nemotron-nano-12b-v2-vl... | $0.0000 | 24.0% |
| meta-llama/llama-3.2-3b-instru... | $0.0000 | 4.0% |
Loading prompt execution data...