Rate limits

SynapsAI Cloud applies rate limits to protect platform stability. Limits may vary by account tier — contact support to discuss higher limits.

Model deployment rate limits

By default, each account can deploy one model at a time. Contact support if you need additional concurrent deployments.

Model inference rate limits

Each deployed model supports up to 100 requests per second (RPS) by default. Contact support for higher RPS requirements.

Model instance rate limits

A deployed model can run on multiple instances. Each instance typically handles 6–8 concurrent requests, depending on model type and size. The autoscaler adds instances when per-instance load exceeds your scale up threshold.

HTTP 429 responses

When rate limits are exceeded, the API returns HTTP 429 Too Many Requests. Implement exponential backoff with jitter in production clients:

import time
import random

for attempt in range(5):
    try:
        response = client.chat.completions.create(...)
        break
    except Exception as e:
        if "429" in str(e) and attempt < 4:
            time.sleep((2 ** attempt) + random.random())
        else:
            raise

Autoscaling — scale instances to absorb higher concurrency
API overview — error codes and authentication
Troubleshooting — diagnose failed requests

Data

Troubleshooting

Getting started

Guides

Examples

Manage

Platform

Resources

Support

Legal

Model deployment rate limits

Model inference rate limits

Model instance rate limits

HTTP 429 responses

​Model deployment rate limits

​Model inference rate limits

​Model instance rate limits

​HTTP 429 responses

​Related

Model deployment rate limits

Model inference rate limits

Model instance rate limits

HTTP 429 responses

Related