Routing

How routing works

When you send a request to EUrouter, we don't just forward it to a single provider. Instead, we select the best available provider for your specific request based on health, price, and performance — then automatically retry with alternatives if something goes wrong.

This happens transparently. You specify a model, and EUrouter handles the rest.

Provider selection

EUrouter selects providers in this order:

1. Find all providers that host the requested model
2. Filter out disabled or unhealthy providers
3. Apply your routing preferences (if specified)
4. Score remaining candidates by health and price
5. Select the highest-scoring provider

Default scoring

By default, providers are scored using:

score = healthScore / (price)²

This balances reliability with a strong preference for cost-effective providers.

Routing preferences

You can control routing by adding a provider object to your request:

{
  "model": "mistral-large-3",
  "messages": [{ "role": "user", "content": "Hello!" }],
  "provider": {
    "order": ["scaleway", "azure"],
    "allow_fallbacks": true
  }
}

Available parameters

A parameter set to null counts as not set, and so does "provider": null.

OpenRouter compatibility

The provider object accepts the fields of OpenRouter's provider object, so you can switch without changing your requests:

Any other unknown field in provider also fails with a 400 that names the field.

Provider order

Use order to specify which providers to try first:

{
  "provider": {
    "order": ["scaleway", "mistral", "azure"]
  }
}

Providers in the list are tried in order. If all fail and allow_fallbacks is true, other providers are tried as fallbacks.

With allow_fallbacks set to false, only the providers in order can serve the request. If none of them can, the request fails instead of going to a provider you didn't list.

Exclusive providers

Use only to restrict to specific providers:

{
  "provider": {
    "only": ["scaleway"]
  }
}

No other providers will be used, even as fallbacks.

Excluding providers

Use ignore to exclude specific providers:

{
  "provider": {
    "ignore": ["azure"]
  }
}

Model variants

Append a variant suffix to the model name for quick routing shortcuts:

{
  "model": "mistral-large-3:floor",
  "messages": [{ "role": "user", "content": "Hello!" }]
}

Variants are a shorthand — you can achieve the same with explicit provider parameters.

Data residency

By default, inference traffic is eligible only for providers configured with EU-hosted endpoints. Provider ownership, retention, and any Broadcast destinations are separate controls. Use these request parameters when your policy is more specific:

{
  "provider": {
    "data_residency": "de",
    "eu_owned": true,
    "max_retention_days": 0
  }
}

Retention is never loosened by a request. When a routing rule and the request both set zdr or max_retention_days, the strictest value applies.

Quantization

Providers can serve the same model at different precisions. A quantized endpoint stores the model's weights in fewer bits, such as fp8 or fp4. This is usually faster and cheaper, and can lower quality for some prompts.

You can see each endpoint's quantization on its model page and in the API:

• GET /api/v1/models/{author}/{slug}/endpoints returns quantization for every endpoint.
• GET /api/v1/models returns quantization for every provider in providers.

The values are OpenRouter's: int4, int8, fp4, mxfp4, nvfp4, fp6, fp8, mxfp8, fp16, bf16 and fp32. A provider that doesn't declare its precision shows unknown. This includes closed models, whose labs don't publish it.

By default, routing doesn't take quantization into account. Providers are scored by health and price, so a cheaper quantized endpoint can be picked over a full-precision one.

To use only certain precisions, list them in provider.quantizations, in the request or in a routing rule:

{
  "model": "mistral-large-3",
  "messages": [{ "role": "user", "content": "Hello!" }],
  "provider": {
    "quantizations": ["fp8", "bf16"]
  }
}

• fp4 also matches mxfp4 and nvfp4, and fp8 also matches mxfp8. Every other value matches only itself.
• An endpoint that shows unknown is used only when you list unknown.
• When no endpoint of the model matches, the request fails with a 400: "No providers available for model '…' with given preferences".
• An empty list or null applies no filter.

Automatic fallbacks

If a provider fails, EUrouter automatically tries the next available provider. This happens for:

• Server errors (5xx)
• Rate limits (429)
• Timeouts (408)
• Network failures

Non-retryable errors (authentication failures, bad requests) fail immediately without fallback.

Disabling fallbacks

Set allow_fallbacks to false to disable automatic retries:

{
  "provider": {
    "order": ["scaleway"],
    "allow_fallbacks": false
  }
}

If Scaleway fails, the request fails. No other providers are tried, and you are never billed by a provider outside order.

Streaming requests

For streaming responses, fallback can only happen before the stream starts. Once tokens begin streaming, the connection is committed to that provider.

Model fallbacks

To fall back to another model, list your models in order in models:

{
  "models": ["mistral-large-3", "llama-3.1-8b"],
  "messages": [{ "role": "user", "content": "Hello!" }]
}

EUrouter tries mistral-large-3 first, with the provider fallbacks above. It moves on to llama-3.1-8b when the first model can't serve the request (unknown, retired, missing a feature the request uses, or too small a context window) or when all its providers fail with a server error, rate limit, timeout or network failure. Other errors, such as a bad request, fail immediately. When every model fails, the last error is returned.

• models works on chat completions, completions, responses and messages.
• When models has entries, it replaces model.
• Your provider settings apply to every model in the list. allow_fallbacks: false stops fallback between providers, not between models: the next model is still tried, on the providers you allowed.
• Streaming requests can only move to the next model before the stream starts.
• You are billed for the model that answered, and x-model-used tells you which one it was.

Health monitoring

EUrouter continuously monitors provider health:

• Success rate — Recent request success/failure ratio
• Latency — Response times
• Error patterns — Recurring issues

Unhealthy providers enter a cooldown period:

During cooldown, providers are deprioritized but not completely excluded.

Response headers

Every response includes routing information:

Use these headers to understand routing decisions and debug issues.