LoRA adapters

SynapsAI Cloud launches complete models: a Hugging Face repo or a model artifact that holds the full weights. A LoRA adapter on its own (the small adapter_model.safetensors file and adapter_config.json) is not a complete model, so it cannot be launched directly. To use an adapter, merge it into the base model it was trained on. The result is a normal model directory with Safetensors weights. You then upload it as an artifact or push it to the Hub, and launch it like any other model.

Merging bakes the adapter into the weights. The merged model runs at the same speed and cost as the base model, with no adapter overhead at inference time. Each merged adapter is its own model, so it is its own artifact and its own deployment.

Before you start

The exact base model. An adapter only works with the model it was trained on. Open the adapter’s adapter_config.json and read base_model_name_or_path. Use that repo and revision.
An adapter in PEFT format. The directory contains adapter_config.json and adapter_model.safetensors (or adapter_model.bin). This is what peft and most fine-tuning tools (TRL, Axolotl, Unsloth, LLaMA-Factory) produce. The adapter can be local or on the Hub.
Enough memory. Merging loads the full base model. Plan for roughly the size of the model in 16-bit weights (about 2 GB per billion parameters) in CPU RAM or on a GPU. A GPU is faster, but CPU works.
Disk space for the base model, the adapter, and the merged output.
A Hugging Face token if the base model is gated or private.

pip install --upgrade torch transformers peft safetensors accelerate huggingface_hub

Merge with PEFT

This example merges an adapter into a text generation model. It loads the base model in 16-bit, applies the adapter, folds it into the weights, and saves the result.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "your-org/base-model"        # base_model_name_or_path from adapter_config.json
adapter_path = "./my-adapter"          # local directory or Hub repo id
output_dir = "./merged"

base = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
    device_map="cpu",                  # use "auto" to merge on GPU
)

model = PeftModel.from_pretrained(base, adapter_path)
model = model.merge_and_unload()       # folds the LoRA weights into the base weights

model.save_pretrained(
    output_dir,
    safe_serialization=True,
    max_shard_size="5GB",
)

tokenizer = AutoTokenizer.from_pretrained(base_id)
tokenizer.save_pretrained(output_dir)

merge_and_unload() returns a plain Transformers model with no PEFT wrappers, so the saved directory has the same layout as the base model.

Tips

Match the precision

Load the base in the precision you want to serve, usually torch.bfloat16 or torch.float16. Merged weights are stored in that precision. If you want a smaller or cheaper deployment, merge in 16-bit first, then pick a quantization in the launch wizard.

Adapters trained with QLoRA

Do not merge into a 4-bit or 8-bit base. Rounding errors from the quantized weights end up in the merged model. Load the base in full 16-bit precision (as in the example above), apply the adapter, and merge there. QLoRA adapters merge fine this way.

Tokenizer and added tokens

If you added special tokens during fine-tuning, the adapter directory contains its own tokenizer files. Load the tokenizer from the adapter directory instead of the base, and resize the embeddings before applying the adapter:

tokenizer = AutoTokenizer.from_pretrained(adapter_path)
base.resize_token_embeddings(len(tokenizer))
model = PeftModel.from_pretrained(base, adapter_path)

Adapters that also trained embed_tokens or lm_head (listed under modules_to_save in adapter_config.json) are handled by the same steps.

Other model types

Use the Auto class that matches the base model’s task and architecture, for example AutoModelForSequenceClassification for classification or AutoModelForSpeechSeq2Seq for speech recognition. For vision-language models and other models with a processor, also save it with AutoProcessor.from_pretrained(base_id).save_pretrained(output_dir). For diffusion models, merge the adapter into the pipeline component it targets (usually the UNet or transformer) and save the full pipeline.

Hub repo instead of local files

adapter_path can be a Hub repo id such as your-org/my-adapter. Run huggingface-cli login first if it is private.

Check the merged model

Before you upload, confirm the output is complete and behaves like your fine-tuned model. 1. Check the files

ls ./merged

You should see config.json, one or more .safetensors files (plus model.safetensors.index.json when sharded), generation_config.json, and the tokenizer files. There should be no adapter_config.json and no adapter_model.safetensors. 2. Run a quick generation

from transformers import pipeline

pipe = pipeline("text-generation", model="./merged", torch_dtype="auto", device_map="auto")
print(pipe("Hello", max_new_tokens=30)[0]["generated_text"])

Compare the output with what you saw from the adapter during fine-tuning. If the output looks like the plain base model, see Troubleshooting below.

Upload and launch

You have two options. Both end in the same launch wizard.

Option 1: Upload as a model artifact

Use this for private weights or when you do not want to publish to the Hub. Create the artifact with the text-generation pipeline (or the pipeline that matches your model) and upload the merged directory:

from synapsai import SynapsAI

client = SynapsAI(timeout=3600.0)

created = client.model_artifacts.create(
    display_name="My merged model",
    pipeline="text-generation",
)

artifact = client.model_artifacts.upload(
    "./merged",
    artifact_id=created.artifact.id,
)
print(artifact.id, artifact.status, artifact.size_gb)

Or with the CLI, once the artifact exists:

synapsai upload-model ./merged --artifact-id <artifact-id>

When the status is ready, open the launch wizard and select the artifact as the model source. See Model artifacts for the full upload API.

Option 2: Push to Hugging Face

from huggingface_hub import HfApi

api = HfApi()
api.create_repo("your-org/my-merged-model", private=True, exist_ok=True)
api.upload_folder(
    folder_path="./merged",
    repo_id="your-org/my-merged-model",
    repo_type="model",
)

Add a pipeline_tag to the repo’s README.md (for example pipeline_tag: text-generation), then launch it from the wizard. Private repos need a Hugging Face token in the wizard. See Hugging Face setup for the full repository checklist.

Call the merged model

After launch, call it like any other deployment. The model field is your deployment id:

curl https://api.synapsai.cloud/v1/chat/completions \
  -H "Authorization: Bearer $SYNAPSAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<deployment-id>",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

See Call the API for the SDK and OpenAI-compatible client examples.

Troubleshooting

Problem	Cause and fix
Output looks like the base model	The adapter was not applied. Check that `PeftModel.from_pretrained` pointed at the right directory and that you called `merge_and_unload()` before saving.
`size mismatch` or `shape` error when loading the adapter	Wrong base model or revision, or the tokenizer was resized during training. Use `base_model_name_or_path` from `adapter_config.json`, and see the tokenizer tip above.
`Target modules ... not found in the base model`	The adapter was trained on a different architecture than the base you loaded.
Out of memory while merging	Merge on CPU with `device_map="cpu"`, or use a machine with more RAM. Loading in 16-bit instead of 32-bit halves memory.
Quality dropped after merging a QLoRA adapter	The adapter was merged into a quantized base. Reload the base in 16-bit and merge again.
Artifact deploy fails on weight or config checks	Confirm `config.json` and the `.safetensors` files are at the top of the uploaded directory, and that the pipeline matches the model. See Model artifacts.
Gated base model will not download	Run `huggingface-cli login` with a token that has access to the base model.
Missing chat template	Save the tokenizer into the output directory. The chat template lives in `tokenizer_config.json`.

Next steps

Model artifacts

Upload the merged weights and select them at launch.

Launch a model

Set workers, precision, and pricing for the deployment.

Convert to Safetensors

Convert .bin or .pt weights if your adapter or base uses them.

Optimizing costs

Quantize the merged model and tune context length.

Getting started

Models

Agents

Knowledge

Use the API

Examples

Integrations

Account

Resources

Support

Legal

Before you start

Merge with PEFT

Tips

Check the merged model

Upload and launch

Option 1: Upload as a model artifact

Option 2: Push to Hugging Face

Call the merged model

Troubleshooting

Next steps

Model artifacts

Launch a model

Convert to Safetensors

Optimizing costs

​Before you start

​Merge with PEFT

​Tips

​Check the merged model

​Upload and launch

​Option 1: Upload as a model artifact

​Option 2: Push to Hugging Face

​Call the merged model

​Troubleshooting

​Next steps

Model artifacts

Launch a model

Convert to Safetensors

Optimizing costs

Before you start

Merge with PEFT

Tips

Check the merged model

Upload and launch

Option 1: Upload as a model artifact

Option 2: Push to Hugging Face

Call the merged model

Troubleshooting

Next steps