API Documentation

Enterprise-grade routing, reliability, and deep observability for your AI stack. The complete reference for integrating with MLRouter programmatically.

1. Authentication

MLRouter authenticates API requests primarily via API Keys passed in the Authorization: Bearer header. Valid key formats include:

mlr-: The primary API Key format.
mas-: Legacy internal keys for standard API access.
sk-mlr-: Legacy public OpenRouter-compatible keys.
mlr-mgmt-: Management keys for administrative operations.

You can create and manage your API Keys in Settings -> API Keys.

import OpenAI from "openai";
const client = new OpenAI({
  baseURL: "https://www.mlrouter.com/v1",
  apiKey: "mlr-your-api-key",
});

javascript

2. Core Concepts

API Keys

Scoped API keys with their own spend caps, routing strategy, and guardrails, configured on the Routing page. Rotate or revoke them without touching your provider credentials.

Routing Strategies

Route each request by cost, latency, or capability - with automatic fallback to a healthy provider when one degrades.

Guardrails

Enforce allowed/blocked models, per-request token ceilings, and spend limits to keep AI usage predictable and secure.

TokenDrift

Continuously benchmark models so you know when quality drifts - and can re-route to a better option before it hits production.

Presets

Reusable configuration templates defining system prompts, default parameters, and allowed model subsets for consistent application behavior.

Fallback Chains

Ordered sequences of alternative models and providers to seamlessly transition to if a primary request fails or times out.

Data Policies

Ensure compliance by attaching policies (like zero-data-retention or HIPAA) to provider keys, dynamically filtering out non-compliant routes.

Classifiers

Real-time evaluators that analyze prompts and responses for specific intents, PII, or toxicity, seamlessly triggering alerts or blocks.

Memory & Sessions

Automatic persistent conversation tracking (mcp-persist) providing stateful context across multi-turn chat sessions.

3. Chat Completions

POST/v1/chat/completions

OpenAI-compatible chat completions. Accepts standard request bodies, handles streaming, and intelligently routes requests to the best available provider based on your active routing strategy.

models(string[])

An array of model IDs to use as an explicit, request-scoped fallback chain.

allow_fallbacks(boolean)

Defaults to true. If false, returns an error immediately on primary model failure instead of trying fallbacks.

stream(boolean)

Set to true to receive partial message deltas via Server-Sent Events (SSE).

Idempotency-Key(HTTP header)

Makes retries safe for 24 hours. Reusing a key with a different JSON body returns 422; a completed retry replays the stored response without dispatch or billing.

X-MLR-Max-Cost(HTTP header (USD))

Per-request worst-case cost ceiling with up to four decimal places. The request is refused before provider dispatch when its estimate is higher.

mlrouter.max_cost(string | number)

Body equivalent of X-MLR-Max-Cost. mlrouter.max_cost_credits is also accepted as an exact integer-credit ceiling.

x_mas_routing_strategy(string)

Overrides the virtual key's routing strategy for this single request (e.g., 'cost', 'latency'). Can also be passed as the X-MAS-Routing-Strategy HTTP header.

from openai import OpenAI

client = OpenAI(

base_url="https://www.mlrouter.com/v1",

api_key="mlr-your-api-key",

)

response = client.chat.completions.create(

model="gpt-4o-mini",

messages=[{"role": "user", "content": "Hello!"}]

)

print(response.choices[0].message.content)

4. Models

GET/v1/models

Lists every model available to your workspace, including context windows, pricing details, and live provider health scores.

GET/v1/models/:id/benchmarks

Retrieves TokenDrift benchmark scores and performance history for a specific model.

GET/v1/models/sync-status

Returns the synchronization status of the upstream model catalog.

5. Embeddings & Audio

POST/v1/embeddings

Generates vector embeddings for the provided input text, compatible with the OpenAI /v1/embeddings spec.

GET/v1/audio/models

Lists available audio generation and transcription models.

POST/v1/audio/speech

Generates spoken audio from text (TTS).

POST/v1/audio/transcriptions

Transcribes audio into text (STT).

6. API Keys Management

GET/v1/api-keys

Lists all API keys in the workspace.

POST/v1/api-keys

Creates a new API key. Returns the unmasked key exactly once.

DELETE/v1/api-keys/:id

Revokes an API key.

Routing & Governance (PATCH)

PATCH/v1/api-keys/:id/strategy

Updates the routing strategy associated with the key.

PATCH/v1/api-keys/:id/spend-cap

Modifies the spend limit constraints and reset period for the key.

PATCH/v1/api-keys/:id/rate-limit

Updates the requests-per-minute limit for the key.

PATCH/v1/api-keys/:id/guardrail

Attaches or removes a guardrail policy for the key.

PATCH/v1/api-keys/:id/compression

Toggles opt-in LLMLingua-2 prompt compression for the key. Body: { prompt_compression: boolean }.

PATCH/v1/api-keys/:id/cost-safety

Configures the integer-credit request ceiling, exact response cache TTL, semantic-cache interface, and provider prompt-cache passthrough.

Key Sub-resources (GET)

GET/v1/api-keys/:id/spend

Retrieves current spend accumulation against the key's cap.

Routing and governance for a key are configured on the Routing page. Key creation and revocation are handled separately in Settings -> API Keys.

7. Presets

GET/v1/presets

Lists all configuration presets available in the workspace.

POST/v1/presets

Creates a new preset containing default parameters and system prompts.

GET/v1/presets/:id

Retrieves details of a specific preset.

PATCH/v1/presets/:id

Creates an immutable revision; pass publish=true to publish it immediately.

GET/v1/presets/:id/revisions

Lists immutable revision history, authors, and change messages.

GET/v1/presets/:id/diff?from=&to=

Returns the field-level diff between two revisions.

POST/v1/presets/:id/publish

Moves the published pointer to revision_id.

POST/v1/presets/:id/rollback

Copies an older revision into a new revision and publishes it.

PUT/v1/presets/:id/split

Sets sticky A/B weights in integer basis points.

GET/v1/presets/:id/analytics

Reports request, cost, latency, and error metrics per revision.

DELETE/v1/presets/:id

Deletes a preset.

Shadow traffic

GET/v1/shadow-configs

Lists this workspace's shadow evaluations and budget state.

POST/v1/shadow-configs

Creates a sampled, budget-capped candidate evaluation.

PATCH/v1/shadow-configs/:id

Pauses, resumes, or changes sampling and budget settings.

GET/v1/shadow-configs/:id/report

Returns per-arm wins, cost and latency deltas, and disagreements.

POST/v1/shadow-configs/:id/promote

Promotes a confirmed candidate to the linked preset or workspace default.

8. Guardrails & Classifiers

Guardrails

GET/v1/guardrails

Lists all configured guardrails.

POST/v1/guardrails

Creates a new guardrail policy.

GET/v1/guardrails/:id

Retrieves a specific guardrail policy.

PATCH/v1/guardrails/:id

Updates a guardrail policy.

DELETE/v1/guardrails/:id

Removes a guardrail policy.

Classifiers

GET/v1/classifiers

Lists all prompt/response classifiers.

POST/v1/classifiers

Creates a new classifier.

GET/v1/classifiers/:id

Retrieves a specific classifier.

PATCH/v1/classifiers/:id

Updates a classifier.

DELETE/v1/classifiers/:id

Removes a classifier.

9. TokenDrift

Endpoints proxied to the TokenDrift service for quality and safety analysis.

POST/compare

Compares tokenizer efficiency and cost across multiple models for a given prompt.

text(string)

The prompt text to tokenize and compare.

models(string[])

Model IDs to compare token counts and estimated cost across.

POST/migration-safety

Analyzes tokenizer/vocabulary risk when migrating from one model to another using a set of representative prompts.

source_model(string)

The model currently in use.

target_model(string)

The model being considered for migration.

sample_prompts(string[])

Representative prompts (one per line) used to estimate migration risk.

POST/drift-check

Checks whether a model's tokenizer output has drifted from an expected reference token count for a given text.

model(string)

The model to check.

reference_text(string)

The reference text to re-tokenize and compare.

reference_token_count(number)

The previously-recorded token count for reference_text, used to detect drift.

10. Requests & Observability

GET/v1/requests

Queries request logs with support for date-range and filtering query parameters.

GET/v1/analytics/cache

Retrieves overall semantic cache hit rates and metrics.

GET/v1/analytics/cache-breakdown

Provides a detailed breakdown of cache performance by model.

GET/v1/analytics/cache-savings

Calculates estimated USD savings generated by cache hits.

GET/v1/analytics/tokens

Returns token usage metrics aggregated over time.

GET/v1/analytics/heatmap

Generates a latency/error heatmap for visual analysis.

11. BYOK Provider Keys

Bring your own key (BYOK) routes your workspace's requests with a credential from your provider account. Open Settings → Provider Keys, choose the provider, paste the key, and select Verify and connect. MLRouter makes a minimal authenticated models-list request and shows the result before the key can serve traffic.

Security model

• AES-256-GCM encryption at rest; plaintext is never returned by any API.
• Only the provider and last four characters are displayed after save.
• Keys are provider-bound and sent only to validated first-party hosts; redirects are refused.
• Shadow, eval, and background use is off until you explicitly opt in.

Billing

Your provider bills model tokens directly. MLRouter charges only the configured per-request platform fee, shown as a separate line. If you enable platform fallback and your key is rejected, that request uses normal platform billing and is markedplatform_fallback.

Troubleshooting verification

A 401 or 403 means the provider rejected the credential: confirm it is active, belongs to the selected provider, and can list models. A reachability error means MLRouter could not contact the provider's validated API. Verification attempts are rate limited. If a verified key is later revoked, MLRouter marks it unverified, creates a notification, and falls back only when you opted in.

API

GET/v1/provider-keys

Lists masked metadata, state, verification history, opt-ins, and data policies.

POST/v1/provider-keys

Verifies, encrypts, and connects a key. Requires recent identity reverification.

POST/v1/provider-keys/:id/verify

Runs a rate-limited live verification using the stored credential.

POST/v1/provider-keys/:id/rotate

Verifies a replacement, then atomically switches traffic to it.

PATCH/v1/provider-keys/:id

Updates enabled state, fallback/background opt-ins, or data policies.

DELETE/v1/provider-keys/:id

Permanently removes the encrypted key and policy row.

12. Notifications

GET/v1/notifications

Fetches all workspace notifications (e.g., budget alerts, billing events).

PATCH/v1/notifications/:id/read

Marks a specific notification as read.

PATCH/v1/notifications/read-all

Marks all pending notifications in the workspace as read.

13. Memory & Sessions

GET/v1/memory/sessions

Retrieves a paginated list of all active chat sessions in the persistent memory store.

GET/v1/memory/sessions/:id

Fetches full conversational history and state for a specific session ID.

14. Billing

GET/v1/billing/balance

Current workspace spend balance and limit.

GET/v1/billing/usage

Historical spend and usage charts.

GET/v1/billing/plan

Current active subscription plan details.

GET/v1/billing/credits

Available prepaid credit balance.

GET/v1/billing/transactions

History of purchases and credit adjustments.

GET/v1/billing/packages

Available prepaid credit packages with fee breakdowns.

POST/v1/billing/checkout

Generates a Stripe checkout session for purchases.

POST/v1/billing/portal

Generates a Stripe billing portal link for subscription management.

POST/v1/billing/topup

Initiates a manual credit top-up.

POST/v1/billing/subscribe

Subscribes the workspace to a premium tier.

POST/v1/billing/promo

Applies a promotional code to the workspace balance.

15. Public Status

GET/public/status

Unauthenticated endpoint providing global provider health status and uptime metrics.

16. Response Headers

Every response from /v1/chat/completions carries X-MLR-* headers explaining how the request was routed.

X-MLR-Resolved-ModelThe concrete model the request was routed to.

X-MLR-Routing-StrategyWhich strategy selected the provider (cost, latency, …).

X-MLR-Fallback-ProviderSet when the primary provider was unavailable and a fallback served the request.

X-MLR-Credential-ModeCredential that served the request: platform, byok, or platform_fallback.

X-MLR-Estimated-Cost-USDEstimated cost of the request in USD.

X-MLR-Health-ScoreHealth score (0–100) of the provider that served the request.

X-MLR-Trace-IDCorrelation id for finding this request in observability.