LLM & API glossary

An LLM gateway is a single API endpoint that routes requests to many language models behind one key, one bill, and one interface.

An LLM router automatically directs each request to an appropriate model or backend based on availability, cost, capability, or a routing rule.

Prompt caching reuses a model's computation for repeated prompt prefixes, billing the cached input at a fraction of the normal rate.

A model alias is a stable name — like 'best', 'fast', or 'code' — that resolves to the current best concrete model, so your code never hardcodes a model id.

LLM failover automatically retries a failed request on a backup so a single model or backend outage doesn't surface to your users as an error.

Tool calling (also called function calling) lets a model request that your code run a named function with arguments, then continue with the result — the foundation of AI agents.

Structured outputs constrain a model to return valid JSON that matches a schema you provide, so downstream code can parse it reliably.

Pay-per-token pricing bills by the number of input and output tokens a request uses, with no subscription or seat fee — you pay only for what you consume.