An incorrectly configured modsecurity setup can cause this to happen. Some vendors that install modsecurity with their control panels do so incorrectly, and configure it in a manner that will lead to memory leaks with advanced security rules. The correct solution is to setup modsecurity correctly. There are two ways to do this:
Solution 1) Install ASL.
This will setup modsecurity correctly, and automatically.
Solution 2) Manually configure and setup modsecurity yourself.
Documentation for setting up modsecurity is available on the Atomic ModSecurity Rules page.