Skip to main content

The scenario

You’ve picked the right model for your email routing task. Emails get classified correctly and customers are happy. Then you notice something in your logs: you spend more on input tokens than output tokens. The question worth asking is: does your prompt need to be that long?
This guide uses POST /v1/traces/cost to compare dollar cost for two prompts with the same model and output size.

The hypothesis

Verbose system prompts feel safer, but every word ships on every request. For straightforward classification, a shorter prompt might cost less with the same accuracy. Price both versions before you rewrite production traffic.

Test configuration

Keep the model fixed (gpt-oss-20b) and vary only the system prompt. Count tokens for each prompt style, then call the Pricing API with the same completion_tokens estimate.
Prompt experiment setup showing side-by-side comparison

Prompts tested

Concise prompt (~225 input tokens):
Verbose prompt (~443 input tokens):

Price each prompt

Compare cost_breakdown.total from each response. At scale, multiply by your daily request volume.

What to track

  • Cost per request from the API
  • Latency in your stack (shorter prompts often decode faster)
  • Accuracy on a fixed email set

Results

Both prompts hit 100% accuracy on the test emails in the original experiment. The concise prompt used roughly half the input tokens and responded faster. Extra formatting and guidelines did not improve routing quality.
Prompt experiment results comparison

Winner for this test

The concise prompt saves about 23% per million requests and responds 8% faster with the same accuracy. There is no quality tradeoff on this workload.

Example in action

Both prompts route this sales inquiry correctly:
Subject: Enterprise plan pricing
From: cto@fastgrowth.io
Hi there,A 200-person company is looking to upgrade from its current solution. Could you send information about:
  • Enterprise plan features
  • Volume discounts
  • Implementation timeline
  • API rate limits
A demo is also requested for next week.Thanks,
Mike Chen, CTO
Should route to: Sales team
Individual prompt result comparison

The takeaway

Shorter prompts cut input tokens on every call. In this case, the concise version is 23% cheaper and 8% faster with 100% accuracy.
Overall impact of prompt optimization
Run POST /v1/traces/cost with your real token counts before and after a prompt edit to see the dollar delta.
Pull live rates first with the quickstart or price search.