by @querulous-deer
Tests whether free LLMs can accurately detect and mask Personally Identifiable Information in text, limited to four entity types: first name, last
name, email address, and IPv4 address.
Example:
Input: "Contact john.doe@example.com or reach out to John Doe at 192.168.1.1"
Output: "Contact [EMAIL] or reach out to [FIRSTNAME] [LASTNAME] at [IPV4]"
A practical privacy-focused benchmark — models must identify sensitive entities precisely without over-masking surrounding context.
Best in Class
Overpriced
Your Selection
Click on the model to make selection
| Config | Total Cost | Quality | Username | Position | |
|---|---|---|---|---|---|
| deepseek/deepseek-r1-0528:free | $0.0000 | 82.0% | @querulous-deer | #1 | |
| z-ai/glm-4.5-air:free | $0.0000 | 72.0% | @querulous-deer | #2 | |
| openai/gpt-oss-120b:free | $0.0000 | 72.0% | @querulous-deer | #2 | |
| mistralai/mistral-small-3.1-24... | $0.0000 | 66.0% | @querulous-deer | #4 | |
| nvidia/nemotron-3-nano-30b-a3b... | $0.0000 | 64.0% | @querulous-deer | #5 | |
| qwen/qwen3-coder:free | $0.0000 | 62.0% | @querulous-deer | #6 | |
| openai/gpt-oss-20b:free | $0.0000 | 62.0% | @querulous-deer | #6 | |
| moonshotai/kimi-k2:free | $0.0000 | 62.0% | @querulous-deer | #6 | |
| google/gemma-3-27b-it:free | $0.0000 | 58.0% | @querulous-deer | #9 | |
| meta-llama/llama-3.3-70b-instr... | $0.0000 | 56.0% | @querulous-deer | #10 | |
| arcee-ai/trinity-mini:free | $0.0000 | 54.0% | @querulous-deer | #11 | |
| nvidia/nemotron-nano-9b-v2:fre... | $0.0000 | 52.0% | @querulous-deer | #12 | |
| nvidia/nemotron-nano-12b-v2-vl... | $0.0000 | 52.0% | @querulous-deer | #12 | |
| nousresearch/hermes-3-llama-3.... | $0.0000 | 52.0% | @querulous-deer | #12 | |
| xiaomi/mimo-v2-flash:free | $0.0000 | 48.0% | @querulous-deer | #15 | |
| meta-llama/llama-3.1-405b-inst... | $0.0000 | 44.0% | @querulous-deer | #16 | |
| qwen/qwen3-next-80b-a3b-instru... | $0.0000 | 32.0% | @querulous-deer | #17 | |
| google/gemma-3n-e4b-it:free | $0.0000 | 28.0% | @querulous-deer | #18 | |
| google/gemma-3-12b-it:free | $0.0000 | 24.0% | @querulous-deer | #19 | |
| google/gemma-3-4b-it:free | $0.0000 | 12.0% | @querulous-deer | #20 | |
| qwen/qwen-2.5-vl-7b-instruct:f... | $0.0000 | 8.0% | @querulous-deer | #21 | |
| meta-llama/llama-3.2-3b-instru... | $0.0000 | 8.0% | @querulous-deer | #21 |
| Total Cost | Quality | |
|---|---|---|
| deepseek/deepseek-r1-0528:free | $0.0000 | 82.0% |
| z-ai/glm-4.5-air:free | $0.0000 | 72.0% |
| openai/gpt-oss-120b:free | $0.0000 | 72.0% |
| mistralai/mistral-small-3.1-24... | $0.0000 | 66.0% |
| nvidia/nemotron-3-nano-30b-a3b... | $0.0000 | 64.0% |
| qwen/qwen3-coder:free | $0.0000 | 62.0% |
| openai/gpt-oss-20b:free | $0.0000 | 62.0% |
| moonshotai/kimi-k2:free | $0.0000 | 62.0% |
| google/gemma-3-27b-it:free | $0.0000 | 58.0% |
| meta-llama/llama-3.3-70b-instr... | $0.0000 | 56.0% |
| arcee-ai/trinity-mini:free | $0.0000 | 54.0% |
| nvidia/nemotron-nano-9b-v2:fre... | $0.0000 | 52.0% |
| nvidia/nemotron-nano-12b-v2-vl... | $0.0000 | 52.0% |
| nousresearch/hermes-3-llama-3.... | $0.0000 | 52.0% |
| xiaomi/mimo-v2-flash:free | $0.0000 | 48.0% |
| meta-llama/llama-3.1-405b-inst... | $0.0000 | 44.0% |
| qwen/qwen3-next-80b-a3b-instru... | $0.0000 | 32.0% |
| google/gemma-3n-e4b-it:free | $0.0000 | 28.0% |
| google/gemma-3-12b-it:free | $0.0000 | 24.0% |
| google/gemma-3-4b-it:free | $0.0000 | 12.0% |
| qwen/qwen-2.5-vl-7b-instruct:f... | $0.0000 | 8.0% |
| meta-llama/llama-3.2-3b-instru... | $0.0000 | 8.0% |
Loading prompt execution data...