NAISS
SUPR
NAISS Projects
SUPR
LoRAFleet: Compression and Execution Co-Design for Efficient Adapter Serving
Dnr:

NAISS 2026/4-1611

Type:

NAISS Small

Principal Investigator:

Shenghui Li

Affiliation:

Uppsala universitet

Start Date:

2026-09-14

End Date:

2027-04-01

Primary Classification:

10210: Artificial Intelligence

Webpage:

Allocation

Abstract

Large foundation models are increasingly adapted to diverse tasks and users through Low-Rank Adaptation (LoRA), allowing many specialized adapters to share a common base model. While parameter-efficient adaptation substantially reduces training and storage costs, serving large collections of adapters introduces new computational challenges. In particular, adapters may contain substantial rank redundancy, while heterogeneous adapter ranks can lead to inefficient batching, GPU utilization, and kernel execution. This project develops **LoRAFleet**, a framework for compression–execution co-design of large-scale LoRA serving. Rather than treating adapter compression and inference optimization independently, LoRAFleet jointly considers model quality and GPU execution efficiency when determining how much capacity should be assigned to individual adapters and model layers. The project will investigate function-aware adapter compression, heterogeneous rank allocation, hardware-friendly rank structures, and execution strategies that improve batching and kernel reuse across multiple adapters. The requested GPU resources will be used to develop and evaluate LoRAFleet on modern NVIDIA accelerators. Experiments will characterize the relationship between adapter rank, rank heterogeneity, model quality, memory consumption, inference latency, throughput, sequence length, batch size, and adapter concurrency. We will compare uniform-rank LoRA, compression methods based only on model quality, and hardware-aware compression and execution strategies. The expected outcome is an efficient open research prototype for serving large collections of LoRA adapters together with a systematic study of how adapter compression should interact with GPU execution. The project aims to improve the scalability of parameter-efficient model serving for emerging workloads involving dynamically selected, composed, or frequently updated adapters.