How MonoRouter Handles Rate Limits Without Dropping Requests

The biggest concern people have about routing through subscriptions instead of the API is rate limits. Fair concern — subscriptions weren't designed for programmatic access. Here's how MonoRouter handles it.

The Rate Limit Landscape

Claude subscriptions have soft and hard rate limits. Soft limits slow you down. Hard limits stop you entirely. The exact numbers vary by tier and change over time, but the pattern is consistent: you get a burst allowance, a sustained rate, and a cooldown period.

MonoRouter tracks all of this per session. Every response from Claude includes rate limit headers, and MonoRouter reads them to maintain an accurate picture of each session's remaining capacity.

Session capacity tracking (internal):
┌────────────────────────────────────────┐
│ Session A (Max):                       │
│   Remaining: 142/200 requests          │
│   Reset: 47 minutes                    │
│   Status: HEALTHY                      │
│                                        │
│ Session B (Pro):                       │
│   Remaining: 12/60 requests            │
│   Reset: 23 minutes                    │
│   Status: HEALTHY (approaching limit)  │
│                                        │
│ Session C (Pro):                       │
│   Remaining: 0/60 requests             │
│   Reset: 8 minutes                     │
│   Status: THROTTLED                    │
└────────────────────────────────────────┘

Request Queuing

When all sessions in your pool are at capacity, MonoRouter doesn't fail the request. It queues it. Your SDK sees a slightly longer response time, but it gets a response. No 429 errors, no retry logic in your application code.

The queue is bounded — we won't hold requests indefinitely. The default timeout is 60 seconds, but you can adjust this per-request with the X-Queue-Timeout header:

import anthropic

client = anthropic.Anthropic(
    base_url="https://api.monorouter.dev/v1",
    default_headers={
        # Wait up to 2 minutes for a session to become available
        "X-Queue-Timeout": "120"
    }
)

# For CI pipelines where waiting is fine:
response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=4096,
    messages=[{"role": "user", "content": diff_content}]
)

If no session becomes available within the timeout, MonoRouter returns a 503 Service Unavailable with a clear error message:

{
  "type": "error",
  "error": {
    "type": "overloaded_error",
    "message": "All sessions at capacity. 3 requests queued ahead. Retry after 45s."
  }
}

This is different from the Anthropic API's 429 Too Many Requests — it tells you the queue depth and estimated wait time so your application can make an informed decision about whether to wait or bail.

Automatic Retry with Backoff

When a session hits a rate limit, MonoRouter marks it as throttled and stops sending new requests to it. It continues sending to other healthy sessions in the pool. When the cooldown expires, the session is automatically added back.

If you only have one session, MonoRouter waits for the cooldown and retries automatically with exponential backoff. Your code doesn't need to implement any retry logic.

Here's what your SDK sees in different scenarios:

# Scenario 1: Multiple sessions, one throttled
# Your code sees: normal response, maybe 200ms slower
response = client.messages.create(...)  # works fine

# Scenario 2: All sessions throttled
# Your code sees: delayed response (queued until a session recovers)
response = client.messages.create(...)  # works, but takes 30-90s

# Scenario 3: All sessions throttled + queue timeout exceeded
# Your code sees: 503 error
try:
    response = client.messages.create(...)
except anthropic.APIStatusError as e:
    if e.status_code == 503:
        print(f"Pool exhausted: {e.message}")
        # Back off and retry, or fall back to direct API

Preemptive Distribution

The smartest thing MonoRouter does is avoid rate limits in the first place. By tracking each session's remaining capacity, it can preemptively route requests away from sessions that are approaching their limits.

Without preemptive distribution:
  Session A: ████████████████████░ 95% → THROTTLED
  Session B: ████████░░░░░░░░░░░░ 40%
  Session C: ██████░░░░░░░░░░░░░░ 30%

With preemptive distribution:
  Session A: █████████████░░░░░░░ 65%
  Session B: ████████████░░░░░░░░ 60%
  Session C: ███████████░░░░░░░░░ 55%

This keeps all sessions in the "comfortable" zone rather than slamming one until it throttles. The threshold is configurable — by default, MonoRouter starts diverting traffic when a session reaches 80% of its estimated capacity.

Client-Side Configuration

For most use cases, the defaults work fine. But if you need to tune behavior for specific workloads, MonoRouter accepts several configuration headers:

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  baseURL: "https://api.monorouter.dev/v1",
  defaultHeaders: {
    "X-Queue-Timeout": "120",       // Max seconds to wait in queue
    "X-Retry-On-Throttle": "true",  // Auto-retry throttled requests (default: true)
    "X-Prefer-Session-Tier": "max", // Prefer Max sessions over Pro when available
  },
  timeout: 300_000, // 5 min — generous for queued + streamed responses
});

For Claude Code specifically, set the timeout in your shell profile:

# ~/.zshrc or ~/.bashrc
export ANTHROPIC_API_KEY="mr_live_7f3a..."
export ANTHROPIC_BASE_URL="https://api.monorouter.dev/v1"

Claude Code handles retries internally, so the default MonoRouter settings work well without additional configuration.

What You See as a Developer

From your SDK's perspective, MonoRouter is just a slightly slower API. Requests take a bit longer during high load, but they succeed. You don't need to handle rate limit errors, implement retry logic, or think about session management. Just send requests and get responses.

The dashboard gives you visibility into what's happening behind the scenes: which sessions are healthy, which are throttled, how the queue depth is trending, and whether you need to add more sessions to your pool.