topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Saturday October 5, 2024, 12:36 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: Log CPU and memory usage  (Read 20774 times)

rgdot

  • Supporting Member
  • Joined in 2009
  • **
  • Posts: 2,193
    • View Profile
    • Donate to Member
Log CPU and memory usage
« on: June 18, 2013, 09:08 PM »
Here's a basic question for everybody.

What should I use if I want to log (to a text file) the processes that use the most cpu and ram? Say every 5 minutes a text file is appended with top 5 process names with as much info about the usage as possible.

This is for both XP and 7 computers, something that can be set and forgotten. Then days later I open the log file to check.

Thanks


worstje

  • Honorary Member
  • Joined in 2009
  • **
  • Posts: 588
  • The Gent with the White Hat
    • View Profile
    • Donate to Member
Re: Log CPU and memory usage
« Reply #1 on: June 18, 2013, 10:57 PM »
I'm going out on a limb and ask you why this is important to have. The amount of Virtual Memory allocated by a process doesn't represent the amount of 'physically' takes up in your RAM, so while high numbers may show you there is a bad memory leak or greedy memory usage in a program, it won't mean anything with regards to 'improving' your computers performance. It's like profiling a cars performance by blindly gauging its speed and fuel left in the tank, not keeping in mind whether you are inside a city, stopped at a traffic light, or whether you may have a caravan hooked up.

Most likely, you have a specific issue, causing bad performance in a certain situation and you want to know what causes it. In general, the Sysinternals programs have a lot of diagnostic tools that can help you with this kind of issue. Download the whole suite, and unzip them somewhere of your liking (mine go into C:\Programs\Sysinternals); they're a pretty useful bunch all around. The majority you'll (almost) never use, some you'll end up using daily.

Autoruns won't help your situation too much, but it basically msconfig on steroids. Most places where programs can hook into your computer and slow it down is listed inside of it, and you can either disable or delete items to try and improve your computers performance.

Process Explorer is a replacement for Task Manager. Just get it. Seriously. GET IT. It's the powertool of process management, showing you so much ******* shit about all of your proceses that 99% of the time you'll find a very strong clue, if not direct cause, to what is causing your computer to slow down. If you keep it running in the tray, you can pop it up at any time and see the processes recent behaviour with regards to CPU, Memory and I/O. It's very easy to see which processes keep hammering your computer every few seconds, for example. The first thing I do after I reinstall Windows is to replace Task Manager with this program; it is really THAT useful.

Process Monitor is basically a logger such as the one you are looking for. However, it is also the most cumbersome to use. Why? Because it logs ****** everything. A program accessing a single value inside the Registry tends to cause multiple log entries, for example. It tracks all processes. Having it running for five seconds can easily give you 10,000 lines of diagnostic information. On a good note, it has a very powerful filtering system, so after you are done capturing it (or before you start), you can just hide everything you don't want logged. This tends to not focus on memory or cpu, but rather on actions performed. If a process stalls, you can easily spot it because the timestamps will show a gap, and Process Monitor more than likely shows what the program was doing right before or after that. All around though, this thing is like a needle in a haystack, and requies a fair bit of effort, especially if you lack experience with it.

RamMap is a kind of Process Explorer, focusing only on the memory usage at a specific point in time. Much of the simple information is stuff Process Explorer also offers, but the views in this are far better for comparing the behaviour in which its memory is used.

VMMap basically dissects a running process, showing you exactly what 'health' it was in at a certain point in time. It's basically RamMap, except it is focused on one process as opposed to being focused on your entire machines memory situation. It isn't very useful for an end-user, although it can sometimes give clues towards certain behaviour. Basically, it shows you how much memory it uses and what kind of purpose and traits it holds. For experienced users, it can be useful for spotting malware or securty holes in software.

In general, the last two or even three one are not something you want or need to use. Sometimes really extreme numbers can totally tell you what is wrong: a driver reserving tons of memory and preventing applications from using it, programs polling the registry every second, thousands of ever-increasing handles being consumed by processes: stuff like that which is otherwise hard to diagnose can become really visible with these tools.
« Last Edit: June 18, 2013, 11:00 PM by worstje, Reason: Needed to self-censor. Twice. Sorry about that. »

rgdot

  • Supporting Member
  • Joined in 2009
  • **
  • Posts: 2,193
    • View Profile
    • Donate to Member
Re: Log CPU and memory usage
« Reply #2 on: June 19, 2013, 09:07 AM »
Thanks for the details. This is not for my computer(s) and I am more interested in the general state of the system - at regular intervals - without worrying about specifics like memory leaks. That is why I wanted to have a look at it in a snapshots sort of way (checking process names and some details 'later'), without necessarily running something like Process Monitor. Personally haven't ever used RamMap or VMMap, I will have a look.

worstje

  • Honorary Member
  • Joined in 2009
  • **
  • Posts: 588
  • The Gent with the White Hat
    • View Profile
    • Donate to Member
Re: Log CPU and memory usage
« Reply #3 on: June 19, 2013, 10:03 AM »
The problem is that there is no such thing as the general state of a system. Tons of things affect performance. Process Explorer shows the biggest troublemakers in its graphs: CPU, Memory, I/O and Network usage. If those are consistently in action while a computer is idle, that is probably the biggest common denominators you can check for for your usecase.

But I still feel uneasy with your snapshot mindset; I just don't know what it will get you. You won't find enough information to tell you what is wrong, unless the problem hangs around until you come to check in on it, in which case you didn't need the snapshot to begin with.

I've had a use for VMMap and RamMap a couple of times while virus & malware hunting in the last several years. They're tools I rarely use, but they're pretty damn useful once you know what you're looking for. :-)

Anyhow, I'll assume you know what you want best. As a computer problem solver, I just can't help but shake off the feeling that you'll end up with (a lack of) information that will make it relatively useless to you to use.

Ath

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 3,626
    • View Profile
    • Donate to Member
Re: Log CPU and memory usage
« Reply #4 on: June 19, 2013, 10:09 AM »
Shouldn't generic but very capable tools like Nagios be able to track changing states like memory or I/O usage and log/alert about it?

Shades

  • Member
  • Joined in 2006
  • **
  • Posts: 2,930
    • View Profile
    • Donate to Member
Re: Log CPU and memory usage
« Reply #5 on: June 19, 2013, 10:32 AM »
PowerShell can do this. PowerShell comes with Windows Vista and Higher and can be downloaded from Microsoft for free for XP.

Creating some scripts is quite easy, especially when using a free IDE such as the one from Idera. (this one actually comes with example scripts that hardly need any modifying to do what you want. Run these scripts at the intervals you prefer and you should be done.


Curt

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 7,566
    • View Profile
    • Donate to Member
Re: Log CPU and memory usage
« Reply #6 on: June 19, 2013, 12:10 PM »
... tools like Nagios be able ...

http://www.nagios.com/products/nagiosxi/

old fashioned cool phone number:

2013-06-19_190723.gif
Nagios XI will monitor up to seven (7) hosts/nodes with unlimited services free of charge! Just select the free license in the XI administration interface. Perfect for SOHO and personal use. Support services are not included when using a free license.

rgdot

  • Supporting Member
  • Joined in 2009
  • **
  • Posts: 2,193
    • View Profile
    • Donate to Member
Re: Log CPU and memory usage
« Reply #7 on: June 20, 2013, 08:32 AM »
@worstje, There is a probably a good chance I won't get much useful information but it may also give me some idea too, especially about idle activity which according to various antivirus and antimalware software is not that kind of problem.

Thanks Ath, Shades, Curt. Will check the tools in the coming days.

Shades

  • Member
  • Joined in 2006
  • **
  • Posts: 2,930
    • View Profile
    • Donate to Member
Re: Log CPU and memory usage
« Reply #8 on: June 24, 2013, 02:20 PM »
As a PowerShell beginner and with the help of a PowerShell IDE, I was able to conjure something rudimentary up.

The Mission
Find 5 running processes sorted on highest CPU load and their use of RAM (working set, not private memory):

The result
Get-Process | Sort -Property CPU -Descending | ? {$_.mainwindowtitle.length -ne 0} | Select Name, @{Label="Application"; Expression={$_.mainwindowtitle}}, @{Label="State"; Expression={"Running"}}, CPU, @{Label="Working Set"; Expression={"{0,12:n0} KB" -f ($_.WS/1kb)}} | Select -First 5
or send this data to file:
Get-Process | Sort -Property CPU -Descending | ? {$_.mainwindowtitle.length -ne 0} | Select Name, @{Label="Application"; Expression={$_.mainwindowtitle}}, @{Label="State"; Expression={"Running"}}, CPU, @{Label="Working Set"; Expression={"{0,12:n0} KB" -f ($_.WS/1kb)}} | Select -First 5 | Export-Csv -Path G:\Temp\log1.csv -NoTypeInformation


Both lines above show only a limited set of data (the requested data). If you want the standard output use the following line:
The screen layout looks like:
Handles:   NPM:   PM:   WS:   VM:   CPU:   ID:    ProcessName:
Get-Process | Where { $_.cpu -gt 0 } | Sort -Property CPU -Descending | Select -First 5
or send this data to file  (this file will show a boatload of information...be warned!):
Get-Process | Where { $_.cpu -gt 0 } | Sort -Property CPU -Descending | Select -First 5 | ConvertTo-Csv | Out-File -FilePath G:\Temp\log2.csv


Still-to-do
Filenames should contain the date-time when they are generated. Right now I'm overwriting files. Appending data could be a solution, but I did not find such an option yet in my travels.


Notes
Save the one you like into a text file with the extension .ps1 (for example: FileNameYouDesire.ps1) and use the Windows task scheduler to run this file at the desired interval. To my knowledge you can start a script in administrator mode with the Windows 7 Task scheduler (but I'll deny having said that in court ;) ).

You will get different results, depending the level of user you login with. The adminstrator account will show you everything, when using this on a limited user account you'll see the information from processes etc. associated with the current user only. That is the way how PowerShell works.

You see me use two different ways to write the data to file. The method 'Export-Csv' is not UTF8 friendly. Reading the file into LibreOffice at first showed garbage, but all data became visible after changing the characterset from 'UTF8' to 'Western-Europe (ASCII/US)'. The other method ConvertTo-Csv did not require this change when being read into LibreOffice.    

As the commands are basic, these will work on each version of PowerShell. I would upgrade the PowerShell on XP to the highest version available though. My computer at home is not (or will be) connected to the internet, so I didn't see a reason yet to upgrade to Win7 SP1. IIRC Win7 comes with PowerShell version 2, which is upgraded to version 3 after service pack 1. XP on the other hand cannot upgrade to more than version 2.

Another idea one could entertain is to write this data into the appropiate EventLogViewer from Windows itself and use that software to get an overview. Ok, I might fill my next Saturday afternoon with this puzzle.

4wd

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 5,644
    • View Profile
    • Donate to Member
Re: Log CPU and memory usage
« Reply #9 on: June 25, 2013, 12:40 AM »
Still-to-do
Filenames should contain the date-time when they are generated. Right now I'm overwriting files. Appending data could be a solution, but I did not find such an option yet in my travels.

$time = Get-Date -format yyyyMMdd-HHmm

Then add $time into your output filename, possibly:

Out-File -FilePath G:\Temp\$time.csv

I think I need to play around with PowerShell :)

Edit: Seems to work.
« Last Edit: June 25, 2013, 01:34 AM by 4wd »

Shades

  • Member
  • Joined in 2006
  • **
  • Posts: 2,930
    • View Profile
    • Donate to Member
Re: Log CPU and memory usage
« Reply #10 on: June 25, 2013, 04:51 PM »
The CSV output format was chosen, as one can check these easily in Excel/LibreOffice. However there are three 'ConvertTo-...' options more: XML, HTML and TXT.

Just letting the OP know.

rgdot

  • Supporting Member
  • Joined in 2009
  • **
  • Posts: 2,193
    • View Profile
    • Donate to Member
Re: Log CPU and memory usage
« Reply #11 on: June 29, 2013, 11:03 PM »
Belated thanks, will report back when I have used the different methods  :)