Expand description
VEX Router - Intelligent LLM Routing for VEX Protocol
A drop-in replacement for vex-llm providers that intelligently routes queries to the most appropriate LLM based on complexity, cost, or quality.
§Quick Start
use vex_router::{Router, RoutingStrategy};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let router = Router::builder()
.strategy(RoutingStrategy::Auto)
.build();
let response = router.ask("What is 2+2?").await?;
println!("{}", response);
Ok(())
}§For VEX Integration
When integrated with VEX, this implements the LlmProvider trait
for drop-in replacement with vex-llm providers.
Re-exports§
pub use classifier::QueryClassifier;pub use classifier::QueryComplexity;pub use config::Config;pub use config::ModelCapability;pub use config::ModelConfig;pub use config::RoutingStrategy;pub use models::Model;pub use models::ModelPool;pub use router::Router;pub use router::RouterBuilder;pub use router::RouterConfig;pub use router::RouterError;pub use router::RoutingDecision;pub use router::RoutingStrategy as RouterStrategy;pub use compress::CompressedPrompt;pub use compress::CompressionLevel;pub use compress::PromptCompressor;pub use guardrails::GuardrailResult;pub use guardrails::Guardrails;pub use guardrails::Violation;pub use guardrails::ViolationCategory;pub use observability::Observability;pub use observability::ObservabilitySummary;pub use observability::SavingsReport;pub use cache::SemanticCache;
Modules§
- cache
- Semantic Caching - Cache responses using vector embeddings
- classifier
- Query Classifier - Simple complexity analysis
- compress
- Prompt Compression - Reduce token count while preserving meaning
- config
- Configuration module for SmartRouter
- guardrails
- Guardrails - Content filtering, PII detection, and safety
- models
- Models module - Model pool and backend integrations
- observability
- Observability - Metrics, tracing, and cost tracking
- router
- Router - Core routing logic for VEX
Structs§
- LlmRequest
- A request to an LLM
- LlmResponse
- Response from an LLM
Enums§
- LlmError
- Errors from LLM providers
Traits§
- LlmProvider
- Trait for LLM providers