Cloudflare cut 100 terabytes of RAM from its load balancer using better math
Cloudflare's own engineers found their internal load balancer was over-provisioning memory for a decade-old hashing default, and clawed back more than 100 terabytes fleet-wide by fixing it.
A load balancer had quietly over-provisioned memory for years. Fixing the math freed over 100 terabytes of RAM.
Every request that hits Cloudflare's network gets routed to a server somewhere in its fleet, and the routing decision has to happen in microseconds, millions of times a second. That system, Pingora Backend Router, uses a technique called consistent hashing to pick a server without recalculating everything when one drops out. Cloudflare engineers found that technique was quietly burning far more memory than it needed to, in some processes around 6GB just for the hashing structures, and on September 18 the company published how it clawed that memory back.
The team, Kevin Guthrie, Mariia Iurchenko, Zaidoon Abd Al Hadi, and Ivan Babrou, described two changes in the post. First, they shrank the core data structure behind each hash ring entry, called a Point, from 8 bytes down to 6, a 25 percent cut per entry just from resizing one field from a 32-bit integer to a 16-bit one. That alone adds up fast across a fleet that holds millions of these points in memory at once.
The second change was bigger. The team cut the number of virtual points assigned to each server, the extra hash-ring entries used to smooth out traffic, by about 90 percent: a server carrying a weight of 625 dropped from 100,000 points down sharply. They ran the numbers before cutting anything and found the extra points were only buying about a 0.7 percent improvement in how evenly requests spread across the fleet. To avoid a spike in cache misses during the switch, they migrated using a stable per-request choice between the old hash ring and the new one rather than cutting over all at once.
Combined, the two changes freed more than 100 terabytes of RAM across Cloudflare's fleet, with no new hardware bought to make up the difference. That figure is Cloudflare's own internal measurement, not an outside audit, though the mechanism behind it, the struct size, the point-count math, is public and checks out against Cloudflare's numbers. Some coverage has folded this in with a separate 100TB of RAM Cloudflare reclaimed the previous month through unrelated DNS-cache work: the two are different fixes to different systems, and this piece is only about the hash-ring change.
Why a build studio cares
Cloudflare Pages is where every push to this site's main branch actually ends up, and Workers, R2, and Durable Objects back several of our other tools, so a fleet-wide efficiency change here is not background reading, it is the platform our own builds sit on getting measurably leaner. The part worth stealing for our own work is not the RAM total, it is the method: someone asked why a server got 100,000 hash points when a tenth of that would do the same job, ran the math, and found the answer was "nobody checked lately," not "there is a good reason." Every system we ship has a few defaults like that, chosen once for a reason that stopped applying years ago, and this is a reminder to go find ours before someone else's audit does.
Next step: read Cloudflare's own writeup of the fix, credited to Kevin Guthrie, Mariia Iurchenko, Zaidoon Abd Al Hadi, and Ivan Babrou, for the full math behind both changes. If you want a second look at your own infrastructure's unexamined defaults, the kind nobody has re-checked since they were first set, write to us at hello@gattyworks.com.