Create benchmark
Set up a benchmark to organize your prompts and compare variants.
You need a Narev account to create a benchmark. You can create one by registering here.
WIth an account, you can create a benchmark in four steps.
Create the benchmark
Benchmarks and all of their management options are located in in the Narev Platform -> Benchmarks. Once you create a benchmark, you will be greeted with the todo list.
Define the data source
There are multiple options of adding data source to the benchmark, from one-off manual entry, to a programmatic entry through API endpoint.
Each of the data sources is described on a separate page:
Configure variants
In order to run a benchmark, you need to configure at least one variant. You can select variants with a tick box marked as 1.
2 allows you to create a variant from scratch and 3 gives you recommendation based on the currently selected variants.
Configure Quality Evaluation
Each benchmark requires a way to evaluate quality.
There are multiple ways to evaluate quality, described in detail in the Configuring Quality Metrics
With this you are ready to run the first benchmark
Still have questions? Ask on Discord
On This Page