The unified inference layer
One inference layer between your app and every model provider
RootInference is not a wrapper around one vendor's SDK. It is a routing layer that connects, routes, fails over and optimizes every request across 500+ models from 60+ providers — through one OpenAI-compatible API.
| Model | Provider | Policy | Latency | Credits | Status |
|---|---|---|---|---|---|
| anthropic/claude-sonnet-4-5 | provider-virginia | quality ≥ 8 | 412 ms | 0.8 cr | 200 OK |
| openai/gpt-5-mini | provider-oregon | cheapest | 238 ms | 0.2 cr | 200 OK |
| meta/llama-4-70b | provider-frankfurt | fastest | 301 ms | 0.3 cr | 200 OK |
| mistral/large-2 | provider-berlin | fastest | 887 ms | 0.5 cr | 200 OK |
| deepseek/v4 | provider-singapore | cheapest | 356 ms | 0.2 cr | 200 OK |
| google/gemini-3-pro | provider-iowa | quality ≥ 8 | 428 ms | 0.7 cr | 200 OK |
| qwen/qwen-3-coder | provider-tokyo | cheapest | 264 ms | 0.3 cr | 200 OK |
Architecture
Connect. Route. Failover. Optimize.
Eight layers, one request path. Every layer feeds telemetry back into routing, so the gateway gets faster and cheaper as you use it.
Connect
Connect: one compatible API
A single OpenAI-compatible endpoint fronts 500+ models. Point your existing OpenAI SDK at our base URL — no new client library, no per-provider auth, no rewrites.
Catalog
Unified model catalog
Every model is normalised into one schema: chat, streaming, tool calling, structured outputs and vision behave consistently whether the model is from OpenAI, Anthropic, Google, Meta or an open-weight host.
Routing
Route: policy engine
Routing policies pick the best model and provider per request by price, latency and quality. Set defaults per API key, override per request, and A/B new models on live traffic.
Telemetry
Provider telemetry
Every provider in the pool is health-checked continuously. Live latency, error-rate and throughput scores feed routing decisions — real measurements, not published status pages.
Failover
Failover: reliability engine
When a provider errors, throttles or degrades, circuit breakers pull it from rotation and traffic reroutes in milliseconds. Fallback chains retry on alternate providers before your client sees a failure.
Responses
Response normalisation
Streaming events, tool-call payloads and usage data come back in one consistent shape regardless of which provider served the request — so failover is invisible to your code.
Metering
Metering & credit ledger
Every request is metered to the token and priced in credits — 1 credit ≈ $0.01 of inference. Credits are valid 12 months and consumed FIFO by expiry, with the full ledger exportable.
Analytics
Optimize: analytics
Unified dashboards attribute latency, volume and spend per key, team and feature across all providers. Budget alerts and routing recommendations turn usage data into savings.
Scope
What the layer manages for you
Everything between your request and the model's response — unified, measured and under your control.