Skip to main content
SynapsAI Cloud automatically adjusts compute resources based on the demand of your model. This ensures responsive inference performance while minimizing costs. Autoscaling settings can be configured in two places:
  • In the model’s settings under Autoscaling Settings
  • During model deployment under Autoscaling Settings
Related: Deploy a model, Optimizing costs.

​
Autoscaling parameters

Each model maintains its own autoscaling configuration:

​
Scale up threshold

Average number of concurrent requests per model instance required to trigger a scale-up event. When the per-instance load exceeds this threshold, a new instance is launched.

​
Scale down threshold

Average number of concurrent requests per model instance below which the system considers scaling down. When the load drops below this threshold, an unused instance may be terminated.

​
Cooldown period

Minimum amount of time between scaling events. Prevents rapid oscillation between scaling up and down and ensures stable workloads.

​
Worker timeout

Optional timeout for terminating idle instances on Super fast and Cold start deployments after the last request. This timeout only applies to the final instance. Always ready deployments keep at least one instance running regardless of worker timeout.

​
How autoscaling works

  1. The autoscaler continuously monitors average concurrent requests.
  2. If load exceeds scale up threshold, a new instance is activated (unless cooldown is active).
  3. If load drops below scale down threshold, an idle instance is marked for shutdown.
  4. The autoscaler enforces the cooldown period to avoid unnecessary scaling.
This mechanism ensures efficient resource usage while keeping your model responsive under fluctuating traffic.

​
Tuning guidance

See Core concepts for how readiness levels interact with scaling.