What is LoRA?
LoRA (Low-Rank Adaptation) is a method for fine-tuning large models efficiently. Instead of updating all of a model’s weights, LoRA freezes the base model and trains small pairs of low-rank matrices that are added to selected layers. Only those small matrices are learned, which cuts the memory and compute cost of fine-tuning by a large factor while keeping most of the quality of full fine-tuning.
The trained result is a LoRA adapter: a compact set of weights, typically a few hundred megabytes, that represents the difference between the base model and the fine-tuned behavior. The base model itself is unchanged.
Why does LoRA matter for serving?
The economics of LoRA come from the fact that an adapter is small and the base model is large. A serving engine can load one base model once and apply many different adapters on top of it, a pattern called multi-LoRA serving. Because the expensive base weights are shared, serving fifty customer fine-tunes of the same base model can cost close to serving one, rather than fifty times as much.
This is what makes it practical to offer many fine-tuned models on shared hardware:
- One base model loaded on a set of GPUs.
- Many adapters registered against it, each selected per request by name.
- The base weights loaded once; each adapter adding only its small delta.
Serving frameworks such as NVIDIA Dynamo build on this: a single serving graph loads one base model and serves all of its adapters, while requests are routed to the right adapter by the model name they ask for.
Resources
To learn more about LoRA and parameter-efficient fine-tuning, you can explore the following resources:
