Tiny 11-Byte Payload Triggers Major Memory Bloat in OpenSSL Servers
A newly disclosed vulnerability called HollowByte allows unauthenticated attackers to cripple OpenSSL servers using a payload of just 11 bytes. The flaw exists because vulnerable versions of OpenSSL allocate memory based on a declared message size in the TLS handshake header before actually receiving or validating the payload data. An attacker exploits this by opening connections that claim a large message is coming, causing the server to reserve significant memory for data that never arrives.
Okta's Red Team, which discovered and reported the vulnerability, explains that the attack becomes especially devastating due to how the GNU C Library handles memory. Even when connections drop and OpenSSL frees the allocated buffers, glibc retains those memory chunks for potential reuse rather than returning them to the operating system. By flooding servers with waves of connections using randomized claimed sizes, attackers fragment the heap and cause the server's Resident Set Size to climb continuously, leaving it permanently bloated even after the attack stops.
The OpenSSL team has already silently patched the vulnerability and backported fixes to older releases, though no formal identifier has been assigned. Because OpenSSL serves as the foundation for secure internet communication and is embedded in widely used software like NGINX and Apache, organizations are strongly urged to upgrade to a patched version immediately. The only way to reclaim memory on an already compromised server is to restart the affected process entirely.