What is LLM router?
An LLM router automatically directs each request to an appropriate model or backend based on availability, cost, capability, or a routing rule.
Routing is the decision layer inside a gateway. At minimum it maps a requested model to a backend that can serve it. More advanced routers pick a model dynamically (by task, by cost ceiling, or by which backend is currently healthy) and fall back to an equivalent model when the first choice fails.
Good routing is invisible: your request succeeds even when an individual backend is degraded, and you don't have to write retry-and-fallback logic yourself. Routing also enables aliases: stable names that resolve to a concrete model, such as the newest model of a line.
How Kyma does it
On chat completions and messages, Kyma works through a multi-layer chain: retry, then an equivalent model, then a parallel race across quality-tier peers, so a single upstream outage usually does not surface as an error.