Moving Data, Not Storing It: The Architecture Lesson That Transformed My API Gateway

When building high-performance API gateways in Node.js, buffering request data into the V8 heap creates a critical bottleneck: garbage collection pauses that stall the entire event loop and crash under concurrent load. The author discovered that moving data through OS-level memory via streams—rather than storing it in the JavaScript heap—dramatically improves CPU usage and throughput. This architectural insight underscores why memory management and understanding OS-level data flow are fundamental to building scalable backend infrastructure.