Subhendu Das

Most Claude-powered applications use a single model for everything. A "what's 2+2?" gets the same Sonnet treatment as "review this 500-line PR for security vulnerabilities." That's like taking a taxi to the mailbox.

Telechat's smart model router fixes this. It analyzes each incoming message and routes it to the cheapest Claude model that can handle it — without sacrificing quality on complex queries.

The cost problem

Claude's pricing varies dramatically by model:

Model Input (per 1M tokens) Output (per 1M tokens) Relative cost
Haiku $0.25 $1.25 1x
Sonnet $3.00 $15.00 12x
Opus $15.00 $75.00 60x

A typical conversational message (100 input tokens, 200 output tokens) costs:

  • Haiku: $0.000275
  • Sonnet: $0.0033
  • Opus: $0.0165

That's a 60x cost spread. If 70% of your messages are simple enough for Haiku and only 5% need Opus, you're massively overpaying by defaulting to Sonnet.

The routing algorithm

Step 1: Message classification

The router categorizes each message into complexity tiers:

  • Tier 1 (Haiku): Factual questions, simple math, translations, format conversions, quick lookups. Pattern: short messages, no code blocks, no "review/analyze/design" keywords.
  • Tier 2 (Sonnet): Code explanations, summaries, content generation, moderate analysis. Pattern: medium-length messages, single code blocks, "explain/summarize/write" keywords.
  • Tier 3 (Opus): Architecture reviews, complex multi-step reasoning, large code reviews, security analysis. Pattern: long messages, multiple code blocks, "review/architect/design/security" keywords.

Step 2: Context-aware adjustment

The router considers conversation context:

  • If the last 3 messages were all Tier 1, and this message is ambiguous, keep it at Tier 1
  • If the conversation involves code, bump ambiguous messages to Tier 2
  • If the user has explicitly set a model preference, respect it

Step 3: Confidence scoring

Each classification gets a confidence score. If below a configurable threshold (default: 0.7), the router defaults up one tier.

if confidence < threshold:
    tier = min(tier + 1, MAX_TIER)

Enter fullscreen mode Exit fullscreen mode

Conservative by default. You'd rather overspend by $0.003 than give a bad answer.

The math

Based on 30 days of moderate use (~50 messages/day):

Without routing (all Sonnet) With routing
~1,500 messages/month Same
Cost: $4.95/month Cost: $1.89/month
Haiku: 68% of messages
Sonnet: 27% of messages
Opus: 5% of messages

Savings: 62%

For heavier usage (200 messages/day): $19.80 → $7.56.

Budget caps: The safety net 

class BudgetTracker:
    def check_budget(self, user_id, estimated_cost):
        daily_spent = self.get_daily_spend(user_id)
        monthly_spent = self.get_monthly_spend(user_id)

        if daily_spent + estimated_cost > user.daily_limit:
            return BudgetStatus.BLOCKED
        if monthly_spent + estimated_cost > user.monthly_limit:
            return BudgetStatus.BLOCKED
        if daily_spent / user.daily_limit > 0.8:
            return BudgetStatus.WARNING

        return BudgetStatus.OK

Enter fullscreen mode Exit fullscreen mode

Cost is estimated before the API call and reconciled after with actual token counts.

Configuration 

{
  "routing": {
    "enabled": true,
    "default_model": "sonnet",
    "confidence_threshold": 0.7,
    "tier_overrides": {
      "code_review": "opus",
      "translation": "haiku"
    }
  },
  "budget": {
    "daily_limit_usd": 5.00,
    "monthly_limit_usd": 50.00,
    "warning_threshold": 0.8
  }
}

Enter fullscreen mode Exit fullscreen mode

Force a specific model per-message: /opus Review this deployment script for security issues.

Tradeoffs

Routing adds latency. The classification pass takes 50-100ms. For Haiku queries (~300ms), that's noticeable. For Opus queries (2-5s), it's invisible.

Misroutes happen. About 3-5% of messages get routed to a model that's arguably too weak. The confidence threshold helps, but it's not perfect.

Try it 

npm install -g telechatai
telechat init

Enter fullscreen mode Exit fullscreen mode

telechat.fyi | Open source, free forever.