Building Apache Iceberg Lakehouse Storage with S3 Table Buckets

A data platform team chose AWS S3 Table Buckets over standard S3 for their Apache Iceberg lakehouse, gaining 10x faster metadata queries and 50%+ improvement in query planning time. They implemented a three-zone medallion architecture (Raw, Clean, Curated) fully managed with Terraform, using a clear naming convention ({zone}_{domain}) for Glue databases. The infrastructure was split into two separate Terraform modules—one for KMS keys and one for S3 Table Buckets—because the KMS key serves five downstream services beyond the lake storage.