Rate limits - SynapsAI Cloud Docs

Limits protect the platform. They can vary by account — contact support to raise them. Resource ceilings (how many models you can launch) are on Quotas.

Defaults

Scope	Default
Inference per model	100 / second
Inference global	200 / second (all dynamic `/v1` POSTs and custom-container proxy)
Inference read	120 / minute (list/get models, stored objects, vector-store list/get)
Inference write	20 / minute (vector-store create/modify/delete and file writes)
Inference moderate	60 / minute (search, file content)
Agent run	20 / minute
Agent tool	60 / minute

Each account can typically launch one model at a time. Each loaded worker usually handles on the order of 6–8 concurrent requests; autoscaling adds workers when load crosses your scale-up threshold.

HTTP 429

{"error":{"type":"RateLimitExceeded","message":"Rate limit exceeded","code":429}}

Retry with exponential backoff and jitter.

import time
import random

for attempt in range(5):
    try:
        response = client.chat.completions.create(...)
        break
    except Exception as e:
        if "429" in str(e) and attempt < 4:
            time.sleep((2 ** attempt) + random.random())
        else:
            raise

Autoscaling · API overview · Troubleshooting