What is Haystack?
Haystack is an open-source NLP framework built by deepset, a German AI company. It's designed for production-grade search, question answering, and retrieval-augmented generation systems. You build pipelines from composable components: document stores, retrievers, generators, prompt builders, and more.
Haystack is popular in enterprise settings where reliability and control matter. Its component architecture makes pipelines easy to test, swap parts in and out, and deploy in production. The framework includes built-in support for OpenAI-compatible endpoints through its OpenAIChatGenerator component, which means pointing it at EUrouter is a one-line change to the api_base_url parameter.
What's notable here is that both Haystack and EUrouter are European. deepset is based in Berlin, and EUrouter runs on EU infrastructure. If you want a fully European AI stack from framework to inference, this combination gets you there.
Quick to integrate
A few lines of code is all it takes. Swap your base URL and you are routed through EUrouter.
1from haystack.components.generators.chat import OpenAIChatGenerator
2from haystack.utils import Secret
3
4generator = OpenAIChatGenerator(
5 model="gpt-4o",
6 api_key=Secret.from_token("eur-..."),
7 api_base_url="https://api.eurouter.ai/v1",
8)Get started in minutes
Follow these steps to connect your application to EUrouter.
Install Haystack
Install Haystack with the OpenAI integration.
1pip install haystack-aiConfigure the generator
Set api_base_url to EUrouter in the OpenAIChatGenerator.
1from haystack.components.generators.chat import OpenAIChatGenerator
2from haystack.utils import Secret
3
4generator = OpenAIChatGenerator(
5 model="gpt-4o",
6 api_key=Secret.from_token("eur-..."),
7 api_base_url="https://api.eurouter.ai/v1",
8)Build your pipeline
Create Haystack pipelines as usual. All LLM calls route through EU infrastructure.
1from haystack import Pipeline
2from haystack.components.builders import ChatPromptBuilder
3from haystack.dataclasses import ChatMessage
4
5pipe = Pipeline()
6pipe.add_component("prompt", ChatPromptBuilder())
7pipe.add_component("llm", generator)
8pipe.connect("prompt", "llm")
9
10result = pipe.run({
11 "prompt": {
12 "template": [ChatMessage.from_user("Summarize: {{text}}")],
13 "text": "The GDPR is a regulation...",
14 }
15})Why use Haystack with EUrouter
European AI stack
Haystack is built by deepset in Berlin. EUrouter runs on EU infrastructure. Together, they give you a fully European AI stack: framework, inference, and data processing all within EU jurisdiction. For organizations that care about European digital sovereignty, this is a straightforward choice.
- Framework (deepset, Germany) and inference (EUrouter, EU) are both European
- No data processing touches US servers at any point in the pipeline
- Aligned with European AI Act principles on transparency and data governance
- Both projects are backed by active European open-source communities
Production NLP pipelines
Haystack's component architecture is built for production workloads. When you add EUrouter to the mix, you get automatic failover between model providers (if OpenAI is slow, your pipeline switches to Mistral), smart routing for cost optimization, and spend limits per pipeline.
- Automatic failover keeps your pipelines running when a provider has issues
- Smart routing optimizes each LLM call for cost, latency, or both
- Real-time monitoring shows latency and cost for every LLM call in your pipeline
- Use separate API keys per pipeline to track and limit spending per use case
Compliant document processing
If you're building a Haystack pipeline that processes contracts, HR records, medical documents, or legal filings, data residency is not optional. EUrouter keeps all the LLM calls for summarization, entity extraction, and question answering within Europe.
- Document summarization calls go through EU servers
- Named entity extraction (names, dates, amounts) stays GDPR-compliant
- Question answering on sensitive documents doesn't leave European jurisdiction
- Full audit trail for every LLM call, useful for compliance reporting and DPO reviews