> ## Documentation Index
> Fetch the complete documentation index at: https://docs.synapsai.cloud/llms.txt
> Use this file to discover all available pages before exploring further.

# LoRA adapters

> Turn a base model and a LoRA adapter into one complete model you can upload as an artifact and launch on SynapsAI Cloud

SynapsAI Cloud launches complete models: a Hugging Face repo or a [model artifact](https://docs.synapsai.cloud/guides/model-artifacts.md) that holds the full weights. A **LoRA adapter** on its own (the small `adapter_model.safetensors` file and `adapter_config.json`) is not a complete model, so it cannot be launched directly.

To use an adapter, **merge** it into the base model it was trained on. The result is a normal model directory with Safetensors weights. You then upload it as an artifact or push it to the Hub, and launch it like any other model.

> [!NOTE]
>
> Merging bakes the adapter into the weights. The merged model runs at the same speed and cost as the base model, with no adapter overhead at inference time. Each merged adapter is its own model, so it is its own artifact and its own deployment.

## Before you start

* **The exact base model.** An adapter only works with the model it was trained on. Open the adapter's `adapter_config.json` and read `base_model_name_or_path`. Use that repo and revision.
* **An adapter in PEFT format.** The directory contains `adapter_config.json` and `adapter_model.safetensors` (or `adapter_model.bin`). This is what `peft` and most fine-tuning tools (TRL, Axolotl, Unsloth, LLaMA-Factory) produce. The adapter can be local or on the Hub.
* **Enough memory.** Merging loads the full base model. Plan for roughly the size of the model in 16-bit weights (about 2 GB per billion parameters) in CPU RAM or on a GPU. A GPU is faster, but CPU works.
* **Disk space** for the base model, the adapter, and the merged output.
* A Hugging Face token if the base model is gated or private.

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
pip install --upgrade torch transformers peft safetensors accelerate huggingface_hub
```

## Merge with PEFT

This example merges an adapter into a text generation model. It loads the base model in 16-bit, applies the adapter, folds it into the weights, and saves the result.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "your-org/base-model"        # base_model_name_or_path from adapter_config.json
adapter_path = "./my-adapter"          # local directory or Hub repo id
output_dir = "./merged"

base = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
    device_map="cpu",                  # use "auto" to merge on GPU
)

model = PeftModel.from_pretrained(base, adapter_path)
model = model.merge_and_unload()       # folds the LoRA weights into the base weights

model.save_pretrained(
    output_dir,
    safe_serialization=True,
    max_shard_size="5GB",
)

tokenizer = AutoTokenizer.from_pretrained(base_id)
tokenizer.save_pretrained(output_dir)
```

`merge_and_unload()` returns a plain Transformers model with no PEFT wrappers, so the saved directory has the same layout as the base model.

### Tips

> [accordion] Match the precision
>
> Load the base in the precision you want to serve, usually `torch.bfloat16` or `torch.float16`. Merged weights are stored in that precision. If you want a smaller or cheaper deployment, merge in 16-bit first, then pick a quantization in the [launch wizard](https://docs.synapsai.cloud/guides/deploy-model.md#settings).

> [accordion] Adapters trained with QLoRA
>
> Do not merge into a 4-bit or 8-bit base. Rounding errors from the quantized weights end up in the merged model. Load the base in full 16-bit precision (as in the example above), apply the adapter, and merge there. QLoRA adapters merge fine this way.

> [accordion] Tokenizer and added tokens
>
> If you added special tokens during fine-tuning, the adapter directory contains its own tokenizer files. Load the tokenizer from the adapter directory instead of the base, and resize the embeddings before applying the adapter:
>
> ```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
> tokenizer = AutoTokenizer.from_pretrained(adapter_path)
> base.resize_token_embeddings(len(tokenizer))
> model = PeftModel.from_pretrained(base, adapter_path)
> ```
>
> Adapters that also trained `embed_tokens` or `lm_head` (listed under `modules_to_save` in `adapter_config.json`) are handled by the same steps.

> [accordion] Other model types
>
> Use the Auto class that matches the base model's task and architecture, for example `AutoModelForSequenceClassification` for classification or `AutoModelForSpeechSeq2Seq` for speech recognition. For vision-language models and other models with a processor, also save it with `AutoProcessor.from_pretrained(base_id).save_pretrained(output_dir)`. For diffusion models, merge the adapter into the pipeline component it targets (usually the UNet or transformer) and save the full pipeline.

> [accordion] Hub repo instead of local files
>
> `adapter_path` can be a Hub repo id such as `your-org/my-adapter`. Run `huggingface-cli login` first if it is private.

## Check the merged model

Before you upload, confirm the output is complete and behaves like your fine-tuned model.

**1. Check the files**

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
ls ./merged
```

You should see `config.json`, one or more `.safetensors` files (plus `model.safetensors.index.json` when sharded), `generation_config.json`, and the tokenizer files. There should be no `adapter_config.json` and no `adapter_model.safetensors`.

**2. Run a quick generation**

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from transformers import pipeline

pipe = pipeline("text-generation", model="./merged", torch_dtype="auto", device_map="auto")
print(pipe("Hello", max_new_tokens=30)[0]["generated_text"])
```

Compare the output with what you saw from the adapter during fine-tuning. If the output looks like the plain base model, see [Troubleshooting](#troubleshooting) below.

## Upload and launch

You have two options. Both end in the same [launch wizard](https://docs.synapsai.cloud/guides/deploy-model.md).

### Option 1: Upload as a model artifact

Use this for private weights or when you do not want to publish to the Hub. Create the artifact with the `text-generation` pipeline (or the pipeline that matches your model) and upload the merged directory:

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from synapsai import SynapsAI

client = SynapsAI(timeout=3600.0)

created = client.model_artifacts.create(
    display_name="My merged model",
    pipeline="text-generation",
)

artifact = client.model_artifacts.upload(
    "./merged",
    artifact_id=created.artifact.id,
)
print(artifact.id, artifact.status, artifact.size_gb)
```

Or with the CLI, once the artifact exists:

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
synapsai upload-model ./merged --artifact-id <artifact-id>
```

When the status is **ready**, open the [launch wizard](https://platform.synapsai.cloud/launch-model) and select the artifact as the model source. See [Model artifacts](https://docs.synapsai.cloud/guides/model-artifacts.md) for the full upload API.

### Option 2: Push to Hugging Face

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from huggingface_hub import HfApi

api = HfApi()
api.create_repo("your-org/my-merged-model", private=True, exist_ok=True)
api.upload_folder(
    folder_path="./merged",
    repo_id="your-org/my-merged-model",
    repo_type="model",
)
```

Add a `pipeline_tag` to the repo's `README.md` (for example `pipeline_tag: text-generation`), then launch it from the wizard. Private repos need a Hugging Face token in the wizard. See [Hugging Face setup](https://docs.synapsai.cloud/guides/hf-model-setup.md) for the full repository checklist.

## Call the merged model

After launch, call it like any other deployment. The `model` field is your deployment id:

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl https://api.synapsai.cloud/v1/chat/completions \
  -H "Authorization: Bearer $SYNAPSAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<deployment-id>",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

See [Call the API](https://docs.synapsai.cloud/guides/inference-quickstart.md) for the SDK and OpenAI-compatible client examples.

## Troubleshooting

| Problem | Cause and fix |
| - | - |
| Output looks like the base model | The adapter was not applied. Check that `PeftModel.from_pretrained` pointed at the right directory and that you called `merge_and_unload()` before saving. |
| `size mismatch` or `shape` error when loading the adapter | Wrong base model or revision, or the tokenizer was resized during training. Use `base_model_name_or_path` from `adapter_config.json`, and see the tokenizer tip above. |
| `Target modules ... not found in the base model` | The adapter was trained on a different architecture than the base you loaded. |
| Out of memory while merging | Merge on CPU with `device_map="cpu"`, or use a machine with more RAM. Loading in 16-bit instead of 32-bit halves memory. |
| Quality dropped after merging a QLoRA adapter | The adapter was merged into a quantized base. Reload the base in 16-bit and merge again. |
| Artifact deploy fails on weight or config checks | Confirm `config.json` and the `.safetensors` files are at the top of the uploaded directory, and that the pipeline matches the model. See [Model artifacts](https://docs.synapsai.cloud/guides/model-artifacts.md). |
| Gated base model will not download | Run `huggingface-cli login` with a token that has access to the base model. |
| Missing chat template | Save the tokenizer into the output directory. The chat template lives in `tokenizer_config.json`. |

## Next steps

> [card] [Model artifacts](https://docs.synapsai.cloud/guides/model-artifacts.md)
>
> Upload the merged weights and select them at launch.

> [card] [Launch a model](https://docs.synapsai.cloud/guides/deploy-model.md)
>
> Set workers, precision, and pricing for the deployment.

> [card] [Convert to Safetensors](https://docs.synapsai.cloud/guides/convert-safetensors.md)
>
> Convert `.bin` or `.pt` weights if your adapter or base uses them.

> [card] [Optimizing costs](https://docs.synapsai.cloud/guides/optimizing-costs.md)
>
> Quantize the merged model and tune context length.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.