Apache Parquet
More info
- Creator
- Twitter + Cloudera
- Released
- Owner
- Apache Software Foundation
- Type
- Columnar file format
- License
- Apache 2.0
- Website
- parquet.apache.org
- Source
- github.com/apache/parquet-format
- Wikipedia
- en.wikipedia.org
Columnar file format for analytics — compressed, splittable, self-describing with embedded statistics per column chunk. The de facto standard for data lakes.
Twitter + Cloudera 2013, inspired by Google's Dremel paper. Hybrid row-group/columnar layout: the file is split into row groups → columns → pages. Supports nested data (struct, array, map) via Dremel encoding. Page stats (min/max/null count) let query engines skip irrelevant pages without reading them.
Read by Spark, Trino, DuckDB, Polars, ClickHouse, Athena, BigQuery, Snowflake, pandas. Iceberg and Delta Lake build their table formats on top of Parquet files.