Cloudflare reported reclaiming more than 100 terabytes of RAM globally by optimizing a core algorithm used in its internal load-balancing infrastructure, according to a blog post from the company.
The optimization focused on pingora-ketama, an open-source library within Pingora Backend Router (PBR) that handles consistent hashing—a method for distributing tasks across multiple servers without requiring major changes when servers are added or removed. A performance team member named Ivan identified that PBR was consuming significantly more memory than expected, particularly in structures associated with consistent hashing.
Consistent hashing works by converting server identifiers and cache keys into hash values, then plotting them on a number line. Tasks are assigned to the nearest server on that line, creating ranges of responsibility for each server. However, because hashes are essentially random numbers, these ranges naturally become unbalanced, meaning some servers handle disproportionately larger workloads than others.
To address this imbalance, consistent hashing systems add multiple hash points per server rather than just one. Cloudflare’s optimization involved adjusting the number and distribution of these hash points. According to the blog post, NGINX and Pingora both use a default of 160 hash points per server. For a 100-server setup, using 160 points per server reduces the coefficient of variation—a measure of distribution evenness—from approximately 99% to about 8%.
The ketama algorithm, named after the library where it was first implemented, allows Cloudflare to scale workload allocation proportionally to server capacity. This proved particularly valuable since some of Cloudflare’s servers have different storage capacities and should therefore receive tasks accordingly.
This optimization represents one of a series of improvements at Cloudflare’s scale. The company operates thousands of servers globally with petabytes of RAM and millions of CPU cores. The DNS team separately reclaimed approximately 100TB of RAM the previous month through similar efficiency improvements. According to the post, at Cloudflare’s scale, even 1% improvements across services yield significant aggregate savings in resource consumption.
Key facts
- Cloudflare reclaimed over 100TB of RAM by optimizing its consistent hashing algorithm in Pingora Backend Router
- The improvement involved adjusting how multiple hash points are distributed across servers to balance workloads more evenly
- The optimization reduced the coefficient of variation (distribution unevenness measure) from roughly 99% to 8% in a 100-server scenario
- Cloudflare operates thousands of servers worldwide with petabytes of RAM and millions of CPU cores
- The DNS team had separately recovered approximately 100TB of RAM the previous month through similar efficiency efforts