IT lexicon › Database › Apache Spark

Apache Spark

Database På svenska → Updated: 2026-07-30

Unified compute engine for big data — Matei Zaharia (UC Berkeley AMPLab), 2009, donated to ASF in 2013. Much faster than MapReduce thanks to in-memory caching and DAG execution.

Four parts: Spark SQL (DataFrames + Catalyst optimizer), Structured Streaming (micro-batch streams), MLlib (machine learning), GraphX (graphs). Written in Scala, first-class APIs in Python (PySpark), R, Java.

Commercialised by Databricks. Competitors: Trino (interactive SQL), Flink (stream-first), Snowflake (managed SQL warehouse), DuckDB (single-node). Spark still dominates "industrial big data".

← Back to the lexicon