Data Science Wire

desflow: realistic Kafka event streams from a simulation

Reddit r/apachekafka2d4 min read

desflow is an open source Python package that publishes Kafka events from a discrete-event simulation of a process, built on SimPy. I shared an early version here under its old name, dynamic-des. This post shows what it does now. If you only need messages with the right shape, Faker or a custom script is enough. When the stream has to behave like a real system, with bursts, queues that build up and slow periods, a simulation of that system produces it. desflow wraps SimPy and adds: Three ways to write a model: SimPy-style process code, a builder API that modernizes SimPy code, and a YAML bluep

Read the full story at Reddit r/apachekafka →

More in Data Engineering