LLM Token Efficiency Gateway — Demo

Try the gateway locally and inspect route, cache, cost, and savings metrics.

Input

Tenant ID

Session ID: loading

Workload profile

Allow response cache

Answer

Submit a request to see the model response here.

Metrics

Route-

Cache hit-

Compression-

Cascade-

Consensus-

Tools-

External tools-

Reliability-

Verifier confidence-

Actual cost-

Baseline cost-

Savings-

Cache read saving-

Session memory-

Provider cached tokens-

Latency-

Cache Miss Reasons