For teams running vLLM or SGLang in production: what did you have to build around it?

I’ve been working on an open-source project for operating self-hosted inference, and I’m trying to sanity-check the idea before making the repo public. Getting a model running with vllm serve is usually the easy part. Things get messier once you have real traffic, multiple replicas, streaming requests, limited GPU capacity, updates, rollbacks, autoscaling, and a growing pile of scripts around all of it. What we’ve built so far lets you: connect an existing vLLM/SGLang server or deploy a new one; put a stable OpenAI-compatible endpoint in front of it; continue deployment operations if the CLI d