Aquifer: Traffic Smoothing for Bursty GPU Workloads

GPU inference doesn’t always scale as quickly as the traffic hitting it. A sudden burst of agent or API requests can fill queues, increase inference latency, trigger timeouts and retries, and put even more pressure on already-expensive GPU capacity. Aquifer is an open-source traffic-smoothing runtime built for this mismatch. It absorbs bursts into a durable queue and releases requests at a controlled pace. The inference backend can dynamically tell Aquifer to slow down as pressure increases, then gradually speed traffic back up as capacity becomes available. For self-hosted inference, model se