Data Science Wire

Migrating Kafka + Snowflake to an Iceberg lakehouse (tech talk, Aug 12)

Reddit r/apachekafka1mo4 min read

Karel Sague spent the last year migrating a production data platform from Snowflake to Apache Iceberg, streaming Kafka data in through Kafka Connect. He's giving a talk on Aug 12 (1:30pm PT / 4:30pm ET) walking through what he actually learned, what worked, and what he'd do differently next time. He'll cover the CloudEvents-based schema he built to keep ingestion consistent across tables, GitOps-based self-serve provisioning, and the metrics and SLOs that mattered for keeping pipelines observable in production. He also gets into why he avoided an all-or-nothing migration, how he prioritized wh

Read the full story at Reddit r/apachekafka

More in Data Engineering