Routing

How routing works

When you send a request to EUrouter, we don't just forward it to a single provider. Instead, we select the best available provider for your specific request based on health, price, and performance — then automatically retry with alternatives if something goes wrong.

This happens transparently. You specify a model, and EUrouter handles the rest.

Provider selection

EUrouter selects providers in this order:

1. Find all providers that host the requested model
2. Filter out disabled or unhealthy providers
3. Apply your routing preferences (if specified)
4. Score remaining candidates by health and price
5. Select the highest-scoring provider

Default scoring

By default, providers are scored using:

score = healthScore / (price)²

This balances reliability with a strong preference for cost-effective providers.

Routing preferences

You can control routing by adding a provider object to your request:

{
  "model": "gpt-5.1",
  "messages": [{ "role": "user", "content": "Hello!" }],
  "provider": {
    "order": ["scaleway", "azure"],
    "allow_fallbacks": true
  }
}

Available parameters

Provider order

Use order to specify which providers to try first:

{
  "provider": {
    "order": ["scaleway", "mistral", "azure"]
  }
}

Providers in the list are tried in order. If all fail and allow_fallbacks is true, other providers are tried as fallbacks.

Exclusive providers

Use only to restrict to specific providers:

{
  "provider": {
    "only": ["scaleway"]
  }
}

No other providers will be used, even as fallbacks.

Excluding providers

Use ignore to exclude specific providers:

{
  "provider": {
    "ignore": ["azure"]
  }
}

Model variants

Append a variant suffix to the model name for quick routing shortcuts:

{
  "model": "gpt-5.1:floor",
  "messages": [{ "role": "user", "content": "Hello!" }]
}

Variants are a shorthand — you can achieve the same with explicit provider parameters.

Data residency

EUrouter routes exclusively through EU-based infrastructure by default. For additional control:

{
  "provider": {
    "data_residency": "de",
    "eu_owned": true,
    "max_retention_days": 0
  }
}

Automatic fallbacks

If a provider fails, EUrouter automatically tries the next available provider. This happens for:

• Server errors (5xx)
• Rate limits (429)
• Timeouts (408)
• Network failures

Non-retryable errors (authentication failures, bad requests) fail immediately without fallback.

Disabling fallbacks

Set allow_fallbacks to false to disable automatic retries:

{
  "provider": {
    "order": ["scaleway"],
    "allow_fallbacks": false
  }
}

If Scaleway fails, the request fails — no other providers are tried.

Streaming requests

For streaming responses, fallback can only happen before the stream starts. Once tokens begin streaming, the connection is committed to that provider.

Health monitoring

EUrouter continuously monitors provider health:

• Success rate — Recent request success/failure ratio
• Latency — Response times
• Error patterns — Recurring issues

Unhealthy providers enter a cooldown period:

During cooldown, providers are deprioritized but not completely excluded.

Response headers

Every response includes routing information:

Use these headers to understand routing decisions and debug issues.