Sniffing, but can't smell

[BBigJ]

I posted on the ROKU forums that the sniffer was not working for me, and it was suggested that I try URL snooper and post here if that didn't work.  It didn't and here I am.  Any help would be appreciated.  http://www.rokulabs.com/forums/viewtopic.php?p=62481#62481

Neither snooper was able to pick up my network card, and the roku version couldn't find my soundbridge.  I had to input both by hand.  When I start sniffing, it does see packets, but does not detect URLs.  I tried positive controls such as somafm.com and club977. When I set the "protocal filter" to "show all" it picked up two urls off a website I had open, but I was not actively looking at the window at the time.

I'm using an Abit mobo with the integrated Nvidia network adaptor and Nvidia firewall (I gave permission for the sniffers to get through).  I'm behind a pair of linksys routers on a cable connection.  What other info would help?

J

[mouser]

well the fact that you said it found some urls once suggests that it's working.

let me just walk you through a test to make sure you're using it correctly, etc.

Start URL Snooper.
Go to Advancd mode, and set your network adapter manually in the General Options tab.
Now go back to Search tab, and check that your "Keyword Filter" edit box and "Also Search For" edit boxes are empty.
Set your Protocol Filter to "Show All", which should show you all urls found (not just streaming media ones).
Now click "Sniff Network" button.
Now the program is watching for network traffic.

Ok now GO TO YOUR WEB BROWSER (like firefox or internet explorer), and open up http://www.cspan.org
NOTE: do this in your normal WEB BROWSER, not using the built in "Manually scan a url" box.

If the program is working correctly, when the cspan page loads, you should see a couple hundred urls shown in the url snooper results list.

Some people don't realize that you have to visit the page in your browser while url snopper is running.
Let me know how that works with you.

[BBigJ]

I went through the steps you describe, and still nothing.  When I put cspan in the manual scan box it works fine, though.  What should I try now?

I forgot to mention I'm using XPsp2.

[mouser]

ok if its not finding anything when you search in your browser it means its not properly sniffing urls from your browser.

the manual download function just scans the web page and is not as good as scanning the network live from your regular browser.

what adapter have you selected? do you know what brand of network adapter you have?

[BBigJ]

The network adaptor is
"NVIDIA nForce MCP Networking Adaptor Driver (Microsoft's Packet Scheduler)"

It is the integrated network adapter on my Abit AN8sli motherboard.

[BBigJ]

This is the only URL it seems able to sniff.  FH is a gaming clan I'm in, and it seems to be picking up some sort of communication between the website and the gaming server.  It found this URL twice last night and once this morning.

http://fhgaming.com\g_antilag\0\g_gametype\tdm\gamename\Call of Duty 2\mapname\mp_carentan\protocol\118\scr_friendlyfire\0\scr_killcam\1\shortversion\1.3\sv_allowAnonymous\0\sv_floodProtect\1\sv_hostname\^1-F|H-^7Carentan^4 24/7\sv_maxclients\34\sv_maxPing\250\sv_maxRate\25000\sv_minPing\0\sv_privateClients\4\sv_punkbuster\1\sv_pure\1\sv_voice\0\pswrd\0\mod\1

[mouser]

here's the thing..
it doesnt make sense that it could pick up none on that test i gave you, and yet still be picking up some in some other case.

let me ask you, after you click "sniff network" and then start browsing and opening cspan.org, etc.  look at the bottom statusbar on url snooper.. does it show an increasing count of packets it has processed?

[Carol Haynes]

Is URLsnooper blocked by a firewall?

[BBigJ]

Yes, the number of packets does go up.  It clicks up by about four packets when I reload cspan and about 20 when I reload CNN.  Since I started looking at this thread this morning, (about an hour ago, but check the time stamps) it has seen 642 packets.

I don't think the firewall is the problem.  When I first ran the program the firewall asked for permission for the program to go through, and I gave it.  Also, if the firewall was blocking it, I don't think the manual scan would work.

[mouser]

it makes no sense to me that the packets are going up but its not displaying urls..
the packets going up means its sniffing them.. it's hard to imagine a scenario where that would happen and yet it wouldnt find urls..

[mouser]

Try this just to be sure:
exit program, delete the Settings.ini file in your url snooper directory, then restart and reselect network adapter, etc.
and make sure NOTHING is filled in in the "Keyword Filter" box at the top of the Search tab.

[BBigJ]

Same result...

[BBigJ]

Ok, it found a second URL:

http://www.donationcoder.com/forum/index.php?topic=6356.new

edit:
here's a third URL it found
http://www.rulesforuse.org
I have no idea where this one is from.  I'm posting these in case there is some sort of trend you might be able to pick up on.  I've had the snooper open all day, and it has received 7000 packets and sniffed 3 URLs.  I've had my browser open the entire time (with multiple tabs open) and have been surfing occasionally throughout the day.

[BBigJ]

I used the sniffer using my laptop and it worked fine.  So, I tried again on the desktop and once I killed the firewall it worked fine.  Funny that the firewall allowed it to see some traffic, but not all.  This is the Nvidia firewall that came with my motherboard.

[mouser]

Thanks for the report BBigJ, i've never heard that a firewall could block url snooper from sniffing.  What firewall is it?

[BBigJ]

It is the Nvidia Firewall that come with my motherboard.  To me, the really weird thing is that the firewall blocks some traffic but not all.  If you have any suggestions as to any tweaks I can make to my firewall settings, that would be great.  Otherwise, I'll just turn off the firewall when I need to use the snooper.

[Carol Haynes]

The nVidia forceware firewall is 'interesting' - I have an ASUS mobo that came with it and even ASUS suggest removing the firewall as it causes more problems than solutions (that was a comment from their tech support in response to a networking issue I had).

[mouser]

Thanks for that extra info Carol, I'm going to put a note in url snooper help file about this.

[Carol Haynes]

My comment wasn't specifically related to URL Snooper - just a comment about the odd behaviour or the forceware firewall. I haven't managed to find any comments from people that have used it and stuck with it.

[BBigJ]

Hmmm, this is the first 'problem' I've had with it and I'm a pretty aggressive user (home network, multiple routers, soundbridge, games, VOIP, etc.)  It was a little bit of a pain the first time I tried to set up windows file sharing, but since then everything has worked fine.  I suppose that if this was a program I planned to use frequently I would feel differently, but an overactive firewall is fine by me.

[rjam]

Thanks for the report BBigJ, i've never heard that a firewall could block url snooper from sniffing.  What firewall is it?

Well, I have AVG firewall by Grisoft.  And I'm not getting urls either.

urlsnooper sniffs pretty good.  Tallys up thousands of packets per hour, probably several packest per second.  But it doesn't smell urls.

I'll have to try to go back and follow the clean up instructions, including deleting the settings.ini file in url folder.

opening http://www.cspan.org/ finds zero urls, whether opened on separate browser, or via the url line inside urlsnooper.

I wonder if it makes a difference to have two browser windows open?  Eg one window reading these instructions on donationcoder.com/forums, and the other window with cspan or some target wite.

Or maybe I have too many command options.  I think I typed in all but one of those $_____ options.

[mouser]

the thing is, if its sniffing so many packets and you go to a site like c-span, then it's near impossible that it wouldnt be finding stuff...
unless perhaps you have something specified in the "Keyword Filter" box, which would prevent it from matching most urls.

go to advanced mode, then in the protocol drop down box select "Show All" and in the Keyword Filter box make sure it's empty, then make sure you are sniffing and open your browser to www.c-span.org and see what urls it shows.

[rjam]

I saved the settings.ini file with a different name in separate partition.
Then deleted settings.ini file from urlsnooper folder in c/programs partition

Then reopened urlsnooper,
turned off the autodetect network adapter
set network adapter to the one that works best for me
  Accton EN1207D Series PCI Fast Ethernet Adapter
toggled on "show all"
toggled off "only this computer" (packets stop if I turn this on)
opened C-span in a browser window
packets tumbling in at usual rate (fast)
but no urls found
I looked at AVG firewall
  It has urlsnooper in the "allow" list
  I deactivated the firewall for 15-20 seconds.
Packets still coming, but no urls found
Reactivated firewall
No change in packet speed.
created a command line for $url on http://www.c-span.com
closed browser
reopened with c-span
still lots of packets, but no urls.
yesterday, on youtube with video playing
  I got some pieces of urls, but none I could figure out
  and none of them were over 15-16 characters long
  ? truncated
I'd put in 1-3 alternative key word filters on youtube search yesterday.
Later I removed them.
There are none in the key word box today.
this is all with Windows Millenium OS

[mouser]

1) there are some good youtube download addone for firefox so that if your desire is to record youtube you might try one of those.

2) as for filter keywords -- these are only used to REDUCE the # of urls shown, so leave that blank always, unless you get too many urls shown.

3) im not understanding your think abour commandline $url stuff.

your comment here troubles me:
toggled off "only this computer" (packets stop if I turn this on)

are you on a network? i wonder if somehow its scanning packets from your network other computers and not yours.
how about try checking that option even if it looks like packets stop, but then click sniff and go to www.c-span.org, and hold ctrl+shift while you hit the refresh button in your browser, or just go visit some other random sites, and see if the packet counter at the bottom statusbar ever increases.

try some sites you've never been to before like: http://www.di.fm/

[rjam]

1)  I can check firefox extensions for what you suggest.
    But I try to keep my extensions down to a minimum,
          because my old pc is already heavily loaded with add-ons and utilities of many types.
    Also, I have Snagit, which can record youtube video/audio.

2)  okay, filter keywords are not my problem then

3)  Now see those command line arguments are for 3rd party applications.
  I misunderstood - thought they were for direct use on urls sniffed by urlsnooper

4)  Yep, my "only scan this computer/non-promiscuous" setting seems to work
    opposite of what's been predicted.
    I will try it again in the promiscuous mode to see if it will now work.

5  Arghh!  Am I on a network?  That word means different things to different
    people, and just like the word "love", can cause great misunderstanding.
    Where is Rebecca Saxe when I need her.  She & Nancy Kanwisher do functional MRI's
        at MIT & Harvard, that show where, how and when in the brain,
        persons process stimuli to create thoughts about what another
        person is perceiving and intending to do, way beyond what the 2nd person merely says or writes.
        Believe it or not, there is a special spot in the brain that does this.  And it can be
        used for good or ilk, as in brain washing by totally ruthless, unaccountable or even
        ignorant people operating at a primitive, tribal level.  It's more closely related to
        the limbic system of fight or flight, friend or foe, and speedier than rational,
        cerebral thought.  Therefore, it has an advantage for survival, even ahead of
        more intelligent types, if the latter don't learn to work effectively together,
        as you and others here seem to be.  (Sorry for the tangent.)
    I have a stand alone, home computer.  It is not networked within my home.
    However technically, since it does connect to the Internet via an ISP (Comcast),
    it IS on a network (at least when I've got my cable modem turned on, and
    have a website open.)  The internet is a network.

    But you have a point.  Even when I don't have any browser windows open
    to any websites, urlsnooper does receive packets, while my broadband cable
    modem is on.  So what is it sniffing then?  Well Comcast and my computer
    have ongoing chatter, to monitor whether my modem power is on, and I think it
    tests nearly continuously whether it can send & receive data.  That's what the
    LED lights are for.  At least the 5th one "Activity" whick blinks all the time,
    when the modem is on, except when the line is dead sometimes for short
    periods in this neighborhood, or when the modem is just powering up for the
    first half minute or so.

Meanwhile, I'll retry c-span, and some sites I've never been to, like www.di.fm