Today we encountered an intriguing production resource leak that deserved to be shared. The Observation: In our production server, the CPU and Memory usage continued to increase while the new request came in. In the development server where no requests were served, the resource usage is stable. The server code was written in Go. Two suspicions were drawn from the observation right away. The problematic code resides in the hot path, ie.