Concepts

How it works

Start by creating A/B tests—datasets containing prompts. You can test different variants (different models and configurations) and compare them side-by-side.

Track performance with built-in metrics like quality, latency, and cost. Need human judgment? Add manual evaluations for team review.

Delegate an A/B test winning variant, by marking it as production variant. The production variant is the configuration you want to use for generating responses in your application—typically the best-performing variant from your A/B test.

Use routers to dynamically route queries to the right A/B test based on request properties. The router uses the production variant of the selected A/B test to generate responses.