Apache Iceberg
More info
- Creator
- Netflix (Ryan Blue, Dan Weeks)
- Released
- Owner
- Apache Software Foundation
- Type
- Table format
- License
- Apache 2.0
- Website
- iceberg.apache.org
- Source
- github.com/apache/iceberg
- Wikipedia
- en.wikipedia.org
Open table format for data lakes — adds ACID, schema evolution, hidden partitioning and time travel on top of Parquet/ORC in object storage.
Built at Netflix (2017), donated to the ASF. Two-layer metadata: manifest files list data files, a snapshot points at manifests, and a "current snapshot pointer" is atomically updated in a catalog (Glue, Nessie, Hive Metastore). Guarantees serializable isolation.
Winning the format war against Delta Lake in open ecosystems — Snowflake, BigQuery, Trino, Spark, Athena and Databricks (2024) all support reads and usually writes. Dremio and Tabular are driving commercialisation.