I found a slow, insidious memory leak in a Node.js API gateway caused by lingering event listeners; I fixed it by scoping emitters per request, enforcing cleanup in finally blocks, and adding leak‑aware tests and runtime safeguards—memory usage flattened and OOM restarts stopped.
The Incident
The gateway handled TLS termination, auth, and request fan‑out for many microservices. Over weeks its resident set size climbed in a staircase pattern until Kubernetes began OOM‑killing pods under load. The failure was gradual—light traffic ran for days, peak traffic crashed in hours—so it escaped casual monitoring.
Investigation
Heap snapshots and allocation profiles showed growing counts of small objects—closures, request metadata, and event listeners—rather than one giant allocation. Tracing revealed an internal event bus where request‑scoped listeners were attached but not always removed: an early‑exit authentication path returned before the cleanup function ran, leaving listeners that held references to request state. The GC saw those objects as live and never reclaimed them.
