Architecting Cloud-Native Kafka: from Tiered Storage towards a Diskless Future

Storage disaggregation changes Kafka economics by shifting costs from infrastructure provisioning to cloud API usage, making inefficient consumer access patterns a potentially major source of operational expense. When storage costs shift from shared infrastructure to per-request API charges, platform teams need client-level visibility to attribute expenses; without it, a single replay job can produce major bill spikes with little visibility into their origin. Kafka's legacy rebalancing protocol made dynamic consumer scaling operationally disruptive because scale events triggered group-wide pro