monitoring internet usage

[ak7]

hi

on my home internet connection i've got a monthly allowance of 6gb upload/download. after that my isp throttles back my connection to dial-up speed

in the last two months i've suddenly started exceeding my limit - the thing that is most confusing is that i don't feel like my internet usage habits have changed so i'm trying to figure out what is causing the most traffic.

i just downloaded a freeware program called BitMeter which looks pretty good - that will show me spikes in traffic over a period of time. but i was thinking what would be really useful is a monitoring program that shows which accessed urls are the most responsible for high traffic. it would be really cool at the end of the month (or some other timeframe) to see a list of urls ordered by the number of megs of traffic associated with them. then i could see which websites cost me most of my usage allowance.

any suggestions gratefully received

[PhilB66]

SmartSniff -  a freeware TCP/IP Packet Sniffer.

[40hz]

Before you start doing heavy duty monitoring it might be helpful to eliminate three "common" sources of unexpected internet activity.

1. Check to see if you have any torrents or other P2P software running in the background. If other people use your machine there's a chance somebody might have installed something you're not aware of.

2. This is more of  a long shot, but you may be infected with a bot. There's a free utility that can detect many. It's called called RUBotted. Tech info and download available here:

http://www.trendsecu...urity_tools/rubotted

3. Run full scans from multiple antispyware apps. What one might miss another might catch.

Once you've eliminated the usual suspects you can go ahead with monitoring. Good luck!

Hope this was of help.

[ak7]

great suggestions thanks - SmartSniff looks interesting - first look it seems technical but i can probably get my head around it. any other program suggests along the lines of my ideal description gratefully received too.

torrents shouldn't be an issue - i have only one program that supports torrents (free download manager) and torrent support is currently switched off. having said that, it was used last month but not extensively.

i've downloaded RUBotted on your recommendation and now have that running. theoretically my system should be pretty clean though - i run spybot and ad-aware regularly along with some other anti-spyware services (scanned my computer with trendmicro's homecall and microsoft's one care the other day).

[4wd]

You may want to try Bandwidth Daemon.

You can define filters to specify what traffic to monitor, it looks like you can do it based on IP so you could set monitoring rules for your most used websites to see how much traffic they're dumping to you.

As a bonus, the trial period is 30 days - just enough time to find out where your bandwidth is going

Also, you may want to check out 3proxy, files are hosted on Sourceforge if being a Russian site discourages you.

It can do logging of data received from destination IPs outputting to a format defined by you, (check it's documentation on Sourceforge), including Squid compatible, so you can use log analysers.  Or output to a CSV and load it into a spreadsheet to work out.

It should be a simple matter of installing it and then setting your browser to use the local proxy, (at least that's what I remember when I used a local proxy).

Of the two methods above, the local proxy would probably be the most useful at determining where your data has gone.

PS. You're not a YouTube junkie are you ? 

[ak7]

i haven't tried any of your suggestions re software to monitor specific IPs yet, but i've been running this nice little program i mentioned before called BitMeter http://codebox.no-ip...oller?page=bitmeter2 which monitors overall traffic, graphs it and gives reports based on hourly, weekly and monthly breakdowns.

looking at today's statistics for my daily morning email check, news read, blog read routine it would appear that that takes up about 1.2gb a month by itself. combine that with watching some youtube (not a junkie  ) and online tv programs, some music downloading (the higher bitrate stuff adds up fast) and the great freeware programs i can't live without  - i guess 6gb per month is understandable.

i'm getting the feeling looking at that 1.2gb statistic for my morning routine that i'm not going find anything too sinister in my hunt for bandwidth bandits (did i just make up some new jargon?!). my feeling is that the web experience is just way more bandwidth hungry these days. i'm working on ad-blocking flash adverts from yahoo mail etc with this in mind.

the only other culprit that crossed my mind was the fasterfox addon... i had the prefetch function turned off - but i did install that add-on about two months ago when the increased usage started. any thoughts on whether that is a coincidence?

all comments and opinions gratefully received. and i'd be interested to know how much traffic other people use in a month too.

[pgl]

i haven't tried any of your suggestions re software to monitor specific IPs yet, but i've been running this nice little program i mentioned before called BitMeter http://codebox.no-ip...oller?page=bitmeter2 which monitors overall traffic, graphs it and gives reports based on hourly, weekly and monthly breakdowns.

-ak7 (June 18, 2008, 06:24 PM)

Very nice! I was looking for something just like this -- well, essentially I was looking for this -- for months, a while back, and had three bandwidth meters installed at one point while I tried to figure out which was actually usable I tried Dumeter, Bandwidth Monitor Pro - which I still still can't get rid of (it installs a service! and fucks with the registry!), and uhm, one other one that wa a bit shit. I wish I'd known about BitMeter before!

cheers,

 - pgl

[edit: fix broken taggery]

[f0dder]

I think I'll keep bitmeter2 installed on my workstation for a while, just for the kicks. There's no p2p'ing going on here, so basically just regular heavy-duty browsing, and the occasional large download (game update patches, video clips, ...)

[Rocker452]

Here's one that I use it's freeware and will break down stats by day, month,or year and by program. You can check it out here: http://www.netlimiter.com/

[f0dder]

Hm, for some reason I've always found NetLimiter a bit clunky - but it does seem to be able to determine internet vs. LAN traffic, which BitMeter can't (well, it uses a speed-based hack but that obviously isn't accurate).

Any idea how NetLimiter works? Driver, hooking WinSock, etc.? Since it shows app names, I'd guess WinSock hooking.