Vercel AI Gateway puts one endpoint and one credential in front of many model providers, with fallbacks, bring-your-own-key and spend reporting, at provider list price with no markup on tokens. We route the monogram.io assistant and every tenant model call in our internal tools app through it, and its spend report is our only usage ledger. Its weak spots are a spend report that lags by minutes and budgets that act as soft caps.

What is Vercel AI Gateway?

AI Gateway is a managed proxy for model calls. You address a model as a slug such as anthropic/claude-haiku-4.5, and the Gateway handles the provider credentials, retries, fallbacks and logging. It works with the AI SDK, with the OpenAI and Anthropic SDKs by changing their base URL, and over plain HTTP.

Our use covers more than chat. The monogram.io assistant sends its chat model, its openai/text-embedding-3-small embeddings at 512 dimensions and its cohere/rerank-v4-fast reranking through the Gateway. Our internal tools app sends every model call through it, with one named exception covered below.

Does AI Gateway mark up token prices?

No. Vercel's documentation says the Gateway charges the provider's list price with no platform markup on tokens, on free and paid tiers, and that the same applies to bring-your-own-key requests. Some optional capabilities carry their own charges, and purchased credits expire a year after purchase, so read the pricing page for the features you plan to turn on.

How do fallbacks work when a provider goes down?

The Gateway retries a failed request across the model's other eligible providers, and you can add model-level fallbacks that move to a different model entirely. Provider retries are automatic. Model fallbacks are set per request with providerOptions.gateway.models, or stored on a virtual model, and every routing attempt appears in the Gateway's logs.

The monogram.io assistant uses model-level fallbacks across providers. Its primary model is Claude Haiku 4.5, and providerOptions.gateway.models lists openai/gpt-5-mini and then google/gemini-2.5-flash, so an outage at one provider is less likely to take the assistant down with it. The fallback list lives in a shared constants package, so the agent and anything else that needs it read the same order. One detail cost us time: eve needs the dotted slug, claude-haiku-4.5, to resolve the context-window metadata it uses for compaction.

How do you track AI spend per user or per customer?

Stamp every request with an identifier and read the Gateway's spend report back filtered on it. Vercel's Custom Reporting groups spend by model, user, tag, provider or credential type. Its reporting endpoint is available on Pro and Enterprise plans only, and Vercel lists it among the optional capabilities that can carry their own charges.

Our internal tools app is multi-tenant, and one field drives its usage reporting. Every call that has a tenant sets the Gateway's user field to the tenant's id, and the Usage tab reads getSpendReport filtered on it. There is no usage table. Counting tokens ourselves would create a second ledger that could disagree with the one the bill is written from, so the Gateway's report is the ledger.

How does bring-your-own-key work, and is it worth it?

BYOK lets a request run on your provider credentials while still passing through the Gateway's routing and reporting. Vercel charges no BYOK fee, but BYOK is available on the paid tier only and your team needs purchased AI Gateway credits to use it. When a request on your credentials fails, the Gateway falls back to system credentials, and that fallback usage is billed against your credit balance. Spend through your own credentials is not counted in Gateway budgets, so a budget cannot cap it. The case for BYOK is control. It fits a team with provider credits or negotiated rates, or a customer who wants calls billed to their own account.

Each workspace in our internal tools app picks one of 3 credential modes. It can run on Monogram's key, on its own provider keys, or on its own AI Gateway key. A tenant's provider key rides on the request inside providerOptions.gateway.byok, never on a shared client, and the Gateway prefers it. We moved Gemini's image and multimodal calls from Google's SDK onto the Gateway, which brought them under the same BYOK handling and attribution.

Does AI Gateway only work with the AI SDK, or only on Vercel?

Neither. Vercel's docs say an AI Gateway API key authenticates from any environment, and applications deployed on Vercel can use OIDC tokens instead of a stored key.

We call it from both SDKs inside one app. The OpenAI SDK serves our image tools and several text tools through the Gateway's endpoint, while the AI SDK's createGateway provider handles the rest, both on the same key. The monogram.io assistant authenticates to the Gateway with Vercel OIDC rather than a provider API key.

AI Gateway vs OpenRouter: which should you use?

Both put one API in front of many providers, and both vendors' own pages say they pass through provider prices with no markup on inference. OpenRouter's FAQ says it charges a fee when you purchase credits, and that BYOK on pay-as-you-go includes $25,000 a month of list-price usage with no fee, then 5% above that allowance. Vercel's BYOK docs say AI Gateway adds no BYOK fee but requires the paid tier and purchased credits. Vercel's own comparison page says AI Gateway authenticates Vercel deployments with OIDC and attributes spend to projects and API keys.

monogram.io and our internal tools app both deploy on Vercel and our per-tenant reporting runs on the Gateway's user field, so the Gateway is the one we use by default. We have used OpenRouter as well. The fees above are quoted from each vendor's own pages as of October 2026.

How much latency does AI Gateway add?

Vercel publishes latency, throughput and uptime per provider endpoint, measured through the Gateway so the figures include routing overhead, which makes them the place to check before routing latency-sensitive traffic.

When not to use AI Gateway

The Gateway is our default, and it has limits we design around.

The spend report lags. In one live test a call was invisible at 5, 15, 30 and 60 seconds and appeared a few minutes later. Our Usage tab misses the newest calls and says so on the page.
Budgets are soft caps. Vercel's comparison page says budgets are checked at request start, so a request that crosses the limit can finish, and BYOK spend is outside budgets entirely.
A tenant's own Gateway key hides its usage from you. A call authenticated that way is reported in the tenant's Vercel dashboard and nowhere in ours.
Reports take one grouping at a time. A by-model and a by-credential breakdown are 2 requests.
Not every model is there. The narration voice for our public What's new episodes calls ElevenLabs directly on Monogram's key, because as of October 2026 the Gateway serves no ElevenLabs model.

FAQ

Does Vercel store my prompts?

Vercel's FAQ says the Gateway does not retain prompt or response content and does not train on it. It logs request metadata such as model, tokens, cost and routing attempts. What the upstream provider retains is a separate question, and the Gateway offers zero data retention routing for that.

Can AI Gateway handle embeddings and reranking?

Yes. We run OpenAI embeddings and Cohere reranking through it, alongside chat.

Do I need the AI SDK to use AI Gateway?

No. We call it from the OpenAI SDK as well.

Is there a free tier?

Yes, a monthly included credit on a subset of models. Buying credits moves a team to the paid tier.

Can Monogram set this up for us?

Yes. See our agentic AI services, the M8 case study, our Vercel page, or contact us.

Further reading: AI Gateway docs, AI Gateway FAQ, model fallbacks, BYOK, custom reporting, OIDC.