Data Science Wire

186 database records. 187 Kafka events. Where would you look first?

Reddit r/apachekafka1mo4 min read

I’ve been experimenting with turning distributed-systems failure modes into fictional incident investigations. In this one, a lab registers 186 samples. Registration succeeds in the database, then publishes label-print commands to Kafka. Later the team discovers 187 print events. Eventually they reconstruct the sequence: DB commit succeeds → Kafka publish succeeds → ACK is lost → application retries → duplicate print command → duplicate physical label → subsequent labels shift by one. I used producer idempotence + business-level deduplication on label ID as part of the remediation, with a scan

Read the full story at Reddit r/apachekafka

More in Data Engineering