Video of the week

This is a must-watch video about one of us trying to reach the stars :-)

Well done #HRejterzy

McAfee & Kernel Pool Monitor (by P. Yosifovich)

Recently I spent some time figuring out the root cause for one of our servers getting stuck - memory usage would increase up to 98% after about five days of working. Resource Monitor (Windows native tool) wasn't showing anything interesting regarding the issue so I had to come up with a different approach. Here's an example and please, look at how a large amount of non-paged memory is in use:

non paged

As we know, non-paged memory cannot be stored on a disk and the bad news was that it kept growing. I was searching the Internet in order to find a useful tool, similar to Windows Resource Monitor (but better). I found Kernel Pool Monitor by Pavel Yosifovich. That was a real game-changer that helped me found which application was causing the issue. I found many sys files residing in the memory where pages were tagged in the following way:

C:\>findstr /S /M MFeS *.sys
FINDSTR: Cannot open pagefile.sys
Windows\System32\drivers\mfeavfk.sys

C:\>findstr /S /M NCPc *.sys
FINDSTR: Cannot open pagefile.sys
Program Files\McAfee\Endpoint Security\Threat Prevention\AMCore\mfencbdc.sys
Windows\System32\drivers\mfencbdc.sys

C:\>findstr /S /M MFE0 *.sys
FINDSTR: Cannot open pagefile.sys
Windows\ELAMBKUP\mfeelamk.sys
Windows\System32\drivers\mfeaack.sys
Windows\System32\drivers\mfeavfk.sys
/*... */

C:\>findstr /S /M NCm1 *.sys
FINDSTR: Cannot open pagefile.sys
Program Files\McAfee\Endpoint Security\Threat Prevention\AMCore\mfencbdc.sys
/* ... */

 

 I had done a few screenshots before as I was expecting that McAfee might have had something to do with this.

kernel pool memory

A friend of mine who is responsible for the enterprise antivirus solution used McAfee's Profiler and found a few processes he thought it would be safe to exclude from scanning. That was it. After he did it, everything went back to normal. McAfee is in place and all the systems work just fine. The problem is gone (plenty of allocations made by McAfee, however, not enough 'Frees' (de-allocations) so the Diff value was enormous).