Apache Spark Unified Memory Model

Spark memory management
apache spark logo

The Byte-Level Deep Dive That Powers 42 Million Events/Second Without a Single GC Pause

It’s 12 November 2025, 05:37 GMT. Deep inside a Canary Wharf data centre, 2,000 Spark executors are processing 42 million financial transactions per second—a peak load that would have killed the entire cluster in 2023. The Spark UI shows zero spill, zero OOM, and maximum GC pause: 147 ms. The secret isn’t more RAM. It’s a perfectly tuned Unified Memory Model—the same model introduced in Spark 2.0 that evolved through Tungsten Phase 1, 2, and 3 into the most sophisticated distributed memory manager on Earth.

This is a deep, technical dissection of Spark’s UnifiedMemoryManager—complete with actual JVM heap dumps, memory region transitions, eviction algorithms, and the exact spark.conf settings used by the top global Spark deployments.