Tiered Escalation

Tiered Escalation

v1.9

Tiered Escalation sits behind your Knowledge Base RAG pipeline and automatically routes harder questions to more powerful — and more expensive — AI processing when the fast path isn’t confident enough to answer well. Most questions never leave Tier 1; only the ones that need it escalate.

What It Does

Every incoming question is answered first by the standard RAG pipeline. When that pipeline’s confidence is low, or the question looks like it needs deeper reasoning (a multi-step “how do I…” request, or an action the bot can’t resolve from a single document), the system escalates the question to a synchronous or asynchronous LLM tier instead of returning a weak or generic answer.

Escalation is fully automatic — visitors never see a “tier” or make a choice. From their side, the bot either answers immediately or replies a little later with a deeper answer.

How Tiers Work

Tier	Name	Latency	What Happens
Tier 1	RAG (default)	Fast — typically <1s	Existing knowledge base retrieval + generation. No change in behavior from before v1.9.
Tier 2	Sync LLM Escalation	3–8 seconds	A stronger model call, made synchronously while the visitor waits, for questions RAG alone can’t answer confidently.
Tier 3	Async Investigation	2–5 minutes	A background investigation job that can take more time and more context to produce a thorough answer, delivered to the conversation once ready.

Note

Tier 1 is unchanged — it’s the same RAG pipeline documented in Knowledge Base & RAG. Tiers 2 and 3 are new, additive capabilities that only activate when Tier 1 isn’t enough.

Tier 2 — Sync LLM Escalation

Used when the question needs more reasoning than RAG retrieval can provide, but an answer is still expected within the same conversation turn. The system calls a more capable model synchronously, with the retrieved knowledge base context, and returns the answer directly in the chat — the visitor experiences a slightly longer reply, not a delay or “we’ll get back to you” message.

Tier 3 — Async Investigation

Used for questions that genuinely need deeper investigation — multi-step reasoning, cross-referencing several documents, or the kind of question a human agent would need a few minutes to look into properly. The question is enqueued as a background job; the visitor is told an answer is being prepared, and if the investigation completes successfully, the answer is delivered into the conversation automatically.

Caution

In rare cases where every model in the cascade is unavailable, an investigation can fail to complete. No automated fallback message is currently sent in that case — if a Tier 3 answer doesn’t arrive within a few minutes, check the conversation manually.

Routing Logic

Routing is automatic and based on two signals:

1. Knowledge base confidence — the RAG pipeline’s retrieval/answer confidence score. High confidence stays on Tier 1. Lower confidence escalates to Tier 2 or Tier 3 depending on how low.
2. Question type — certain question shapes (multi-step requests, action requests the bot can’t perform on its own, explicit “investigate” style questions) are routed toward escalation even when confidence alone might not trigger it.

Escalation only ever happens if it’s enabled for the organization and within budget — see Admin Configuration below. If escalation is disabled, at max tier 1, or out of budget, the bot falls back to its best Tier 1 answer rather than failing to respond.

Admin Configuration

Escalation is configured per organization in Settings → AI Settings. See AI Settings & Response Language — Tiered Escalation for the full settings reference. In summary:

• Enable/Disable — escalation is off by default; an org admin must turn it on.
• Max Tier — caps how far a question can escalate (1 = RAG only, 2 = allow sync LLM, 3 = allow full async investigation).
• Budget Limits — separate monthly call budgets for Tier 2 and Tier 3, so escalation cost stays predictable.

Model Chain

1. Escalation requests are sent to a primary model.

2. If the primary model fails or is unavailable, the request automatically fails over to the next model in a configured cascade — the visitor never sees the failure.

3. If a model repeatedly fails, a circuit breaker temporarily stops sending requests to it and routes exclusively to the next model in the chain, giving the failing provider time to recover before it’s tried again.

This cascade means a single provider outage doesn’t take escalation offline — it degrades to the next available model automatically.

Budget Management

Tier 2 and Tier 3 each have their own monthly call budget, set per organization (defaults: 500 Tier 2 calls, 50 Tier 3 calls per month).

• Budgets reset automatically at the start of each calendar month (UTC).
• If a tier’s budget is exhausted before the month resets, further questions are handled at the next tier down (or at Tier 1, if no lower escalation tier has budget) instead of being blocked.
• Budget usage does not roll over between months.

Pricing Impact

Tier 1 (RAG) usage is unaffected by escalation — it’s included the same way it always has been. Tier 2 and Tier 3 calls consume budget credits against your plan, since they use more expensive model calls than standard RAG. Monitor usage in Settings → AI Settings and adjust Max Tier or budget limits if escalation volume is higher than expected for your plan.