Skip to main content

​
What are variants?

Variants are your execution configurations. It’s a combination of model, system prompt, and parameters, such as temperature and max tokens

​
When do you use variants?

Use variants when you want to test how different model and router configurations perform on the same benchmark. Attach multiple variants to a benchmark to compare outcomes.

​
How do variants work?

You attach variants to benchmarks. The benchmark provides the request data, and the variant provides the execution logic. This separation lets you test multiple configurations on the same prompts.