Configuring Routers

Configure your router to intelligently route incoming requests to different applications based on message content, metadata, and other criteria. All changes are automatically saved.

Basic Information

Set the router's name and description to help identify its purpose within your workspace.

Routing Rules

Routing rules determine where incoming requests are sent based on filter conditions. Rules are evaluated in order from top to bottom, and the first matching rule determines the routing destination.

Adding a Rule

1. Click Add Rule to create a new routing rule
2. Configure the filter conditions for the rule
3. Select the target application to route matching requests to

Rule Structure

Each routing rule consists of:

• Conditions: One or more filter conditions that must be satisfied
• Target Application: The application that will receive matching requests

Filter Conditions

Build complex filter conditions using the following fields:

Message Content

Filter based on the actual content of the message. Useful for routing requests that contain specific keywords, phrases, or patterns.

-- Example: Route messages containing "customer support"
messages.content contains "customer support"

Message Length

Filter based on the character length of the message content. This can be useful for routing short queries differently from longer ones.

-- Example: Route messages longer than 500 characters
messages.content_length > 500

Metadata

Filter based on custom metadata key-value pairs attached to requests. Metadata provides flexible filtering based on any custom attributes you send with your requests.

To use metadata filtering:

1. Select metadata from the field dropdown
2. Enter the metadata key name (e.g., app_name, user_tier, environment)
3. Choose an operator and enter the value to match

-- Example: Route requests with metadata.environment = "production"
metadata.environment = "production"

Common metadata use cases:

• Route by user tier (free, premium, enterprise)
• Route by application name or environment
• Route by geographic region or data center
• Route by feature flags or experiment groups

Combining Conditions

Use AND and OR combinators to create complex routing logic:

• AND: All conditions must be true for the rule to match
• OR: At least one condition must be true for the rule to match

You can also create nested groups of conditions for more advanced filtering.

Example: Multiple conditions

-- Route premium users' long messages to a specialized application
metadata.user_tier = "premium" AND messages.content_length > 1000

Example: OR conditions

-- Route support or help requests to customer service
messages.content contains "support" OR messages.content contains "help"

SQL Preview

As you build filter conditions, a SQL preview shows the equivalent SQL query. This helps you understand how your conditions will be evaluated.

Target Applications

Each routing rule must specify a target application. When a request matches the rule's conditions, it will be forwarded to the selected application.

Select from applications available in your workspace that are compatible with routing:

• A/B tests
• Other routers (for nested routing)
• Individual models or endpoints

Fallback Rule

The fallback rule (also called the "else" rule) catches all requests that don't match any of your routing rules. This ensures that no request goes unhandled.

Creating a Fallback Rule

1. Click Create Fallback Rule in the Fallback Rule section
2. Select the default application to handle unmatched requests

Without a fallback rule, requests that don't match any routing rules will be rejected with an error.

Fallback Best Practices

• Always configure a fallback: Prevents unexpected rejections
• Use a general-purpose application: Choose a versatile model or endpoint
• Monitor fallback traffic: High fallback traffic may indicate your rules need refinement

Rule Evaluation Order

Understanding how rules are evaluated is critical for correct routing:

1. Top to bottom: Rules are evaluated in the order they appear
2. First match wins: The first rule whose conditions match is used
3. Fallback last: If no rules match, the fallback rule is used
4. No fallback = rejection: Without a fallback, unmatched requests fail

Reordering Rules

Drag rules using the grip handle to reorder them. More specific rules should typically appear before more general rules.

Example order:

1. Route premium users with long queries → Specialized model
2. Route all premium users → Premium model
3. Route messages about pricing → Sales model
4. Fallback → General-purpose model

In this example, a premium user with a long query matches rule 1 and never reaches rule 2, ensuring the most specific routing takes precedence.

Examples

Route by User Tier

-- Rule 1: Premium users
metadata.user_tier = "premium" → Premium Application
 
-- Rule 2: Free users
metadata.user_tier = "free" → Free Application
 
-- Fallback
→ Default Application

Route by Message Topic

-- Rule 1: Technical support
messages.content contains "error" OR messages.content contains "bug"
→ Technical Support Application
 
-- Rule 2: Sales inquiries
messages.content contains "pricing" OR messages.content contains "purchase"
→ Sales Application
 
-- Fallback
→ General Chat Application

Route by Environment and Content

-- Rule 1: Production long messages
metadata.environment = "production" AND messages.content_length > 500
→ High-Capacity Model
 
-- Rule 2: All production
metadata.environment = "production"
→ Production Model
 
-- Rule 3: Development
metadata.environment = "development"
→ Development Model
 
-- Fallback
→ Default Model

Best Practices

1. Test thoroughly: Verify rules match your intended traffic patterns
2. Start simple: Begin with basic rules and add complexity as needed
3. Order matters: Place more specific rules before general ones
4. Always use fallback: Ensure all traffic has a destination
5. Monitor performance: Review routing analytics to optimize rules
6. Document your logic: Use clear rule names and maintain documentation
7. Use metadata strategically: Design a consistent metadata schema across your applications

Automatic Saving

All changes to routing rules, conditions, and target applications are automatically saved. There's no need to manually save your configuration.