Autoscaling - SynapsAI Cloud Docs

Set autoscaling in the launch wizard or later on the model Configuration tab. The dashboard talks about workers. Older docs said instances — same idea: one loaded replica of the model.

Parameters

Setting	Meaning
Minimum workers	Floor. `0` allows scale-to-zero (Sleeping). `≥ 1` keeps capacity Ready
Maximum workers	Cap on replicas
Scale-up threshold	Average concurrent / pending requests per worker before another worker starts
Cooldown	Minimum time between scale events
Instance timeout	Idle time before the last worker stops, only when minimum is 0. Forced to 0 on per-token

Managed defaults are often min 0, max 2, timeout 300s, scale-up 50, cooldown 30s. Custom images often use a smaller max and a longer cooldown.

Behavior

Load is measured per worker.
Above the scale-up threshold (and after cooldown), a new worker starts, up to maximum.
When load falls, extra workers can stop.
If minimum is 0 and the last worker is idle past instance timeout, the model Sleeps. The next API call loads it again.

Tuning

Goal	Change
Absorb spikes with less delay	Lower scale-up threshold, or minimum workers ≥ 1
Spend less when idle	Minimum 0, shorter instance timeout
Stop flapping	Longer cooldown

Related: Optimizing costs, Core concepts.