Discovery & routing
How the coordinator picks which provider serves a request.The coordinator keeps a pool of connected provider Macs. It routes each buyer request to one provider that serves the requested model, is healthy, and has a free seat at that moment.
Seats
Each provider advertises how many requests it can serve at once (its seats). It reportsslots_total and slots_free in every heartbeat. The coordinator also counts its own in-flight dispatches, so a Mac’s late “busy” report cannot hide seats the coordinator knows are free, and a Mac is never sent more concurrent requests than it advertises.
Seats on a Mac today serve independent single-stream requests. Continuous batching exists in the provider runtime but is default-off and not part of production serving.
Eligibility
A provider is a candidate only if all of these hold:- Model. It advertises the requested model, and the model is admitted for paid routing. Models offered by a provider but not admitted for settlement stay out of paid routing.
- Signed catalog match. Its reported model hash matches the signed catalog entry for that model.
- Ready with a free seat. Its state is ready or busy, it has at least one free seat, and no safety hold is active. A thermal-throttling report sets a safety hold.
- Settlement readiness. Its receipt signing key is registered and not mid-rotation, so the request can settle.
- Throughput floor. Its estimated throughput is at or above the production floor (currently 1.0 tokens/sec).
- Context. The request fits the provider’s context capacity.
- Version floor. It runs at least the minimum provider version for that model.
- Circuit breaker. It is not degraded. Two faults inside 120 seconds mark a provider degraded and take it out of routing.
429 provisional_quota_exceeded.
Ordering
Production routing uses thefast objective. Among eligible providers:
- Highest throughput × tier weight first. Trusted and pinned providers have weight 1.0; provisional providers have 0.3.
- Fewest free seats breaks ties.
Sticky conversations
SendX-MacProvider-Conversation: <tag> to express a preference for the provider that served earlier turns. Within the sticky TTL (1,800 seconds, reported live in /v1/models under tier1_disclosure.sticky_affinity), the sticky provider wins only when it is eligible and also ranks first under the ordering above (an exact tie counts). In practice, the preference often does not win, and the request routes to the top-ranked provider instead. Tags are scoped to your account, so the same tag from two accounts routes independently. DELETE /v1/sticky clears your sticky routing state. See Sticky conversations.
Only a sticky hit bills cached prompt tokens at the lower cache-hit rate (see Metering & billing).
Stickiness is a preference, not a pin. Buyers cannot pin a request to a specific provider; see Provider attribution.
Automatic prefix-cache key
Authenticated chat completions also carry an automatic prefix-cache key, derived by HMAC from the prompt prefix. It lets the serving Mac reuse its KV cache across tool turns and shows up inusage.prompt_tokens_details.cached_tokens. It does not affect routing, and those tokens are billed at the full prompt rate. Without a sticky tag, requests that repeat a prefix may land on different Macs.
Response headers
X-Provider-Id: the provider that served this request. Log it with the request id when reporting problems.
Errors when no provider is available
A model can appear on the rate card with no provider serving it. Requests for it return
404 model_not_found until a Mac loads that model. GET /v1/models lists only models with at least one routable provider right now.
What gets excluded
- Providers whose last frame is older than the heartbeat window (90 s).
- Providers with an open circuit breaker.
- Providers with no free seat, or with a safety hold (for example thermal throttling).
- Providers whose reported model hash does not match the signed catalog.
- Providers below the throughput floor.