The inference layer plugs into your existing stack

GPT model family — chat, reasoning, vision and embeddings.

Claude models with tool use and long-context reasoning.

Gemini models — multimodal, long context and embeddings.

Llama open-weight models via multiple hosted providers.

Mistral and Mixtral models for chat, code and embeddings.

DeepSeek chat and reasoning models at economy pricing.

Grok models for chat and reasoning workloads.

Qwen open-weight models for chat, code and vision.

Command models plus rerank and embedding endpoints.

Bedrock-hosted models through your unified API key.

Search-grounded Sonar models with citations.

Kimi long-context chat and reasoning models.

Point any official OpenAI SDK at our base URL — no code changes.

First-class provider for streaming UI and tool calling.

Chat model integration for chains, agents and tools.

LLM and embedding backends for RAG pipelines.

Generator components for Haystack pipelines.

Connector for Microsoft Semantic Kernel apps.

Trace requests, prompts and costs into Langfuse.

Request logging and analytics passthrough.

Latency, error and spend metrics in your dashboards.

Trigger inference from thousands of connected apps.

Model nodes for self-hosted workflow automation.

Inference modules for Make scenarios.