Data Science Wire

Semi-Structured Data in Apache Iceberg: Meet the Variant Type

Apache Iceberg BlogAug 124 min read

Semi-structured data, such as JSON-like documents whose fields differ from row to row, has always been a poor fit for table formats built around fixed schemas. Iceberg v3 adds the Variant type for exactly this data: a single column can hold values of arbitrary, evolving shape, stored as a compact binary value that engines read and write consistently. This is the first post in a series on Variant in Apache Iceberg: what Variant is, why it exists, and how it fits into an Iceberg table. Parquet already defines the Variant type and its binary encoding, so this post focuses on what Iceberg adds on

Read the full story at Apache Iceberg Blog

More in Data Engineering