Session Pooling: How MonoRouter Handles Multiple Users

When you connect multiple Claude accounts to MonoRouter, it doesn't just pick one at random. There's a session pooling system that decides which account handles each request, and getting this right matters more than you'd think.

The Problem

Claude subscriptions have implicit rate limits. Send too many requests through a single session and you'll hit throttling. But spread requests across multiple sessions and each one stays well within its limits.

The naive approach — round-robin — works until it doesn't. Some requests take 30 seconds (long Opus reasoning chains), others take 2 seconds (quick Sonnet completions). Round-robin treats them the same, which means some sessions get overloaded while others sit idle.

Here's what round-robin looks like with mixed workloads:

Round-robin with 3 sessions (bad):
┌──────────────────────────────────────────┐
│ Session A: [===Opus 28s===][===Opus 35s==>    ← overloaded
│ Session B: [=2s=][=3s=][=2s=][=1s=]          ← idle most of the time
│ Session C: [===Opus 22s===][=3s=][===Opus==>  ← overloaded
│                                                │
│ Result: A and C hit rate limits.               │
│         B barely used. Requests start failing. │
└──────────────────────────────────────────┘

Least-Connections Routing

MonoRouter uses a least-connections strategy. When a new request comes in, it goes to the session with the fewest active (in-flight) requests. This naturally balances load because long-running requests keep their session "busy" for longer, pushing new requests to less-loaded sessions.

Least-connections with 3 sessions (good):
┌──────────────────────────────────────────┐
│ Session A: [===Opus 28s===][=3s=][=2s=]     ← gets short jobs while
│ Session B: [=2s=][===Opus 30s===][=2s=]     ← others handle Opus
│ Session C: [=3s=][=1s=][===Opus 25s===]     ← evenly distributed
│                                              │
│ Result: All sessions stay in the comfort     │
│         zone. No rate limit hits.            │
└──────────────────────────────────────────┘

The selection algorithm runs on every incoming request:

for each session in pool:
    if session.status != HEALTHY: skip
    score = session.activeRequests
    if score < bestScore:
        bestScore = score
        bestSession = session

route request → bestSession

It's the same approach that load balancers like Nginx and HAProxy use, adapted for AI conversation sessions instead of HTTP backends. The difference is granularity — HTTP load balancers track TCP connections, MonoRouter tracks individual Claude API requests within a session.

Session Health Checks

Not every session is always healthy. A browser tab might crash, a session might expire, or an account might hit a temporary rate limit. MonoRouter tracks each session's state as one of:

State	Meaning	Receives requests?
HEALTHY	Active and responsive	Yes
THROTTLED	Rate-limited, cooling down	No (auto-recovers)
DISCONNECTED	Session expired or crashed	No (needs reconnection)
DRAINING	Finishing active requests before removal	No new requests

MonoRouter monitors each session continuously. When a session becomes unhealthy, it's removed from the routing pool immediately — in-flight requests complete, but no new requests are sent. When the session recovers (cooldown expires, reconnection succeeds), it's added back to the pool automatically.

Pool state over time:
┌──────────────────────────────────────────────────┐
│ t=0s   [A:HEALTHY] [B:HEALTHY] [C:HEALTHY]      │
│ t=45s  [A:HEALTHY] [B:THROTTLED] [C:HEALTHY]    │
│         ↑ B hit rate limit, removed from routing │
│ t=90s  [A:HEALTHY] [B:HEALTHY] [C:HEALTHY]      │
│         ↑ B cooldown expired, back in pool       │
└──────────────────────────────────────────────────┘

Sticky Sessions

Some workloads benefit from session affinity — sending related requests to the same session. MonoRouter supports this through an optional X-Session-Affinity header:

import anthropic

client = anthropic.Anthropic(
    base_url="https://api.monorouter.dev/v1",
    default_headers={
        "X-Session-Affinity": "pipeline-run-42"
    }
)

# All requests with the same affinity key route
# to the same session (if healthy)
step1 = client.messages.create(...)
step2 = client.messages.create(...)  # same session as step1

In TypeScript:

const client = new Anthropic({
  baseURL: "https://api.monorouter.dev/v1",
  defaultHeaders: {
    "X-Session-Affinity": `build-${buildId}`,
  },
});

Affinity is best-effort: if the pinned session becomes unhealthy, MonoRouter falls back to least-connections rather than failing the request. For most developer tooling use cases (Claude Code, Cursor, one-shot scripts), you don't need affinity at all — each request is independent.

What This Means in Practice

For a team of four developers, each with a Claude Max subscription connected to MonoRouter, the pool has four sessions. Here's a real scenario:

9:00 AM — Two developers start coding with Claude Code
           Requests: ~20/min across the pool
           Distribution: A=5, B=6, C=4, D=5
           Status: All sessions comfortable

9:15 AM — CI pipeline triggers, sends 30 requests in 2 minutes
           Distribution: A=12, B=11, C=13, D=14
           Status: Approaching limits but holding

9:20 AM — Session C hits a soft throttle
           Pool adjusts: A=16, B=15, D=19
           Session C cools down for 45 seconds

9:21 AM — C recovers, pool rebalances
           Distribution: A=12, B=11, C=8, D=13
           Status: All healthy, CI pipeline complete

No single account gets hammered, and the team gets 4x the effective throughput of a single subscription. The pool handles bursts gracefully by distributing load, and recovers from throttling automatically without any developer intervention.