Apache Iceberg: Bringing Database-Grade Capabilities to the Data Lake

Apache Iceberg is an open table format that adds database-grade capabilities—ACID transactions, time travel, schema evolution, and partition evolution—on top of object storage-based data lakes. It uses a three-layer metadata architecture (Catalog, Metadata Files, Data Files) where every write creates a new snapshot, enabling features like point-in-time queries and safe schema changes without breaking downstream consumers. Unlike Hive, Iceberg tracks columns by field ID rather than name, allowing transparent renames and structural changes without rewriting data.