I built a serverless hosting platform for LoRA adapters with vLLM
A developer has created a serverless hosting platform for LoRA adapters using vLLM, aiming to simplify the deployment of fine-tuned models. This platform addresses the challenge of hosting models without continuous local GPU operation or the expense of a dedicated GPU server. Users can host LoRA adapters, or utilize base models like qwen3-4b directly, with a 32k context window. The developer is seeking feedback on the architecture and potential use cases.
Unlike traditional methods requiring dedicated GPUs, this new platform offers a serverless solution for hosting LoRA adapters, simplifying deployment for fine-tuned models.
时间与来源
时间显示为 UTC
显示时区：UTC
本地时区尚不可用，暂时显示 UTC。
发布当时偏移：UTC+02026年9月13日 01:50 UTC
收录当时偏移：UTC+02026年9月13日 15:01 UTC
- 发布
- 2026年9月13日 01:50
- 收录
- 2026年9月13日 15:01
- 来源类型
- 开发者社区
- 档位
- 社区
- 信源状态
- 正常
档位是按信源手工设定的编辑判断，不是逐条打分。
It’s always bothered me that after fine-tuning a model for a project, there isn’t a particularly easy way to host it without either running it locally and keeping a GPU on 24/7 or paying for an entire GPU server.
There are managed options for LoRA serving on top of vLLM (AWS), but you generally still end up paying for an entire instance.
I started wondering: if 99%+ of the model weights are identical between the base model and something like a rank 8–32 LoRA/QLoRA adapter, why does each adapter need its own server?
vLLM already has a lot of the pieces needed to make this work efficiently, including batching, memory management, and fused LoRA kernels.
So I built Lorivo around the idea of having one GPU server per base model that can be shared between many LoRA adapters.
- Lorivo identifies the base model and sends it to a GPU server running that model
I also built a CLI to make uploading/deploying an adapter as simple as possible, along with a web interface for manual uploads. It's as easy as lorivo login api-key and lorivo deploy ./my-lora
Right now I’m hosting the traffic myself on my own GPU, so the available inference model is limited to Qwen 3.5 4B which will always be 100% free
I also have around $1,000 in AWS credits that I’d like to use to add some additional models. I'm not sure which ones would be most useful to the community, though.
For Qwen 3.5 4B, you can also use the base model without an adapter. Just set the model to qwen3-4b instead of qwen3-4b:my-adapter. I'm currently allowing a 32k context window.
I’m also intentionally not saving chats or inference requests. The only usage data collected is token count and timestamp.
Would love to get feedback from people here, especially on the architecture, which models would be useful to host, and whether this solves an actual problem you've run into.