Smart routing

celon/auto — difficulty-based automatic routing

Request with model: "celon/auto" and a rule-based classifier (no LLM call, <5ms on CPU) computes a difficulty score in the 0–1 range. The score is a weighted sum of the features below.

Score → tier mapping (with the default balanced dial): score ≥ 0.62 → Tier 1, score ≥ 0.30 → Tier 2, and anything below that → Tier 3. The computed score is reported as classifier_score in the response metadata.

Intent dials

The same classifier, but every workload wants a different trade-off. Pick one of four presets with celon.dial (per request) or the key’s routing policy (routing.dial). The applied dial is reported in the dial response metadata and the x-celon-dial header.

Purpose routers — intent as a model name

Instead of picking a dial yourself, you can name a purpose router id as the model string. celon/code for coding workloads, celon/quality when quality is non-negotiable (always frontier), celon/nitro when speed is everything (ranked by measured throughput), and celon/floor when cost comes first (ranked by real-time lowest price) — all of them are virtual model ids, so any OpenAI SDK applies them through a single model string.

Savings-target routing

Specify celon.anchor (a quality ceiling) together with celon.savings_target (a target savings rate, in the 0–0.8 range) and the router automatically tunes how aggressively it routes downward so the realized savings converge on the target. For example, 0.4 targets “40% cheaper than always using the anchor alone.”

{
  "model": "celon/auto",
  "messages": [{ "role": "user", "content": "Summarize this quarter's metrics" }],
  "celon": {
    "anchor": "anthropic/claude-fable-5",
    "savings_target": 0.4
  }
}

The feedback loop is grounded in the usage ledger — it aggregates the realized savings rate (against the anchor baseline) from the key’s actual billing records, then routes downward more aggressively when it falls short of the target and pulls back more conservatively when it overshoots. Because the basis is the realized value left in the ledger rather than an estimate, cache hits and fallback costs are reflected exactly as they happened.

The four-stage constraint-aware pipeline

A celon/auto routing decision is not a single difficulty score — it runs through a four-stage pipeline that applies the request’s constraints in order. Each stage’s verdict is traced in the pipeline array of the response metadata.

The three tiers

The models in each tier are listed in fallback order — if the first model fails, it automatically switches to the next. Prices are per 1M tokens (input / output).

Dynamic price tracking

The table above is a snapshot. Actual billing rates sync periodically from the LiteLLM and OpenRouter price feeds (every 6 hours by default), and every change is recorded in the model_pricing history table. Read the current rates with GET /admin/pricing, force an immediate re-sync with POST /admin/pricing/refresh, or pin a specific model’s rate by hand with PUT /admin/pricing/override.

Naming a model explicitly

Catalog ids use the vendor/model form — for example anthropic/claude-fable-5, openai/gpt-5.2, deepseek/deepseek-v3.2. List them all with GET /v1/models. Name a model and the request bypasses the classifier and passes straight through (classifier_score: null); an id that isn’t in the catalog is rejected with 404 model_not_found. The celon.tier hint is ignored for explicit-model requests.

Per-request celon options

The celon object in the request body lets you control routing and caching behavior per request. Every field is optional.

{
  "model": "celon/auto",
  "messages": [{ "role": "user", "content": "Design the core API" }],
  "celon": {
    "dial": "strict-quality",
    "anchor": "anthropic/claude-fable-5",
    "savings_target": 0.3,
    "session_id": "conv_8f2a91",
    "context_compression": true,
    "no_cache": true,
    "fallbacks": ["meta-llama/llama-4-maverick", "deepseek/deepseek-v3.2"]
  }
}

How the fallback chain behaves

• Retry conditions — if an upstream returns 429 or 5xx, or a network/timeout error occurs, it automatically switches to the next candidate. Any other 4xx fails immediately.
• Switch order — it first tries the same model’s alternate providers (altProviders, e.g. DeepInfra → Novita), then moves on to the next candidate model.
• Tracing — the attempted path is recorded in the attempts array of the response metadata and the x-celon-attempts header.
• When all fail — it responds with 502 upstream_unavailable.
• Budget downgrade — when a key is over budget and its policy is downgrade, celon/auto requests are forced to Tier 3. See Governance.

Response metadata

Every response carries the routing outcome in the celon body field and the x-celon-* headers.

{
  "id": "chatcmpl-…",
  "object": "chat.completion",
  "model": "meta-llama/llama-4-maverick",
  "choices": [ … ],
  "usage": { "prompt_tokens": 21, "completion_tokens": 96, "total_tokens": 117 },
  "celon": {
    "model": "meta-llama/llama-4-maverick",
    "provider": "deepinfra",
    "tier": 2,
    "classifier_score": 0.41,
    "cache_hit": null,
    "cost_usd": 0.000122,
    "saved_usd": 0.003886,
    "latency_ms": 842,
    "pii_masked": 0,
    "attempts": [],
    "dial": "balanced",
    "sticky": true,
    "pipeline": ["modality:pass", "context:fit", "sticky:hit", "difficulty:0.41→tier2"],
    "guardrails": { "mode": "mask", "findings": 0, "blocked": false }
  }
}

With streaming the cost isn’t known up front, so only x-celon-model · x-celon-tier · x-celon-dial · x-celon-sticky · x-celon-attempts go out as headers at the start; the stream’s final data chunk (choices: []) carries the full usage and celon metadata, followed by data: [DONE].