topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Saturday December 14, 2024, 5:44 pm
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Last post Author Topic: Sniffing, but can't smell  (Read 26341 times)

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Sniffing, but can't smell
« Reply #25 on: April 03, 2007, 11:54 PM »
The reason i told you to go to sites youve never been to is that it's possible your browser could be cacheing results and thus not actually fetching them from the internet, which url snooper requires.

Your long essay on networking is appropriate -- as you say it's hard to know with the cable modems these days if traffic from other users is passing by you and exciting url snooper :)

rjam

  • Supporting Member
  • Joined in 2007
  • **
  • default avatar
  • Posts: 17
    • View Profile
    • Donate to Member
Re: Sniffing, but can't smell
« Reply #26 on: April 04, 2007, 12:31 AM »
Ah!  I'd forgotten about the caching issue.  Will check that.

LOL!  Well, it doesn't take a cable modem for me to be aware of how much in life passes me by, without my realizing it.  And that's not even considering wireless routers.  I don't have wireless, so that shouldn't be an issue for urlsnooper tho.  Even if my neighbor townhouse has wireless, my pc doesn't receive wireless, only cable signal.

Nope, the narrow scan for "scan this computer only" does not receive any packets.
Even though a description at bottom of urlsnooper window says
  "Sniffing network traffic from WinPcap (then my network adapter name) (then)(IP 71.206.127.175.4.0.0.0"
   IP lookups says "No match found" for that IP number.
   Somtimes that line shows in the bottom of my urlsnooper window.
   Other times, same area in urlsnooper window only shows packets received & buffered, prior to stopping packets.


I've opened  http://www.di.fm/
Packets yes, urls no.

I've cleared the cache in my firefox browser just now.
packets still coming, no urls found




mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Sniffing, but can't smell
« Reply #27 on: April 04, 2007, 12:36 AM »
ok i think that ip network is a clue that something is wrong.
the thing that is confusing me so much is how it can be seeing packets but not urls.
im going to re-enable a debug mode version and see if by looking at packets we can figure out what is going wrong.

question:
if you type http://www.c-span.org in the "Manual Scan a URL" box and hit "Download" button, does it then find some resulting urls?

rjam

  • Supporting Member
  • Joined in 2007
  • **
  • default avatar
  • Posts: 17
    • View Profile
    • Donate to Member
Re: Sniffing, but can't smell
« Reply #28 on: April 04, 2007, 12:50 AM »
Holy catfish!  Yes!  urls all over the place!

But I encountered some forced variables.

1)  I was asked if I wanted to let urlsnooper connect to the internet.
     I guessed "Yes", contrary to my usual preference.

2)  I was asked if I would allow a cookie to be placed on my computer, for a 'better browsing experience'
        or words to that effect.
     Again, I guessed "Yes", contrary to my usual practice.

I think I had to do both of these, afer I entered the c-span url and clicked on the "download' button.

Well, you were puzzled before.

I'm the one who's puzzled now.

Why did this sudden change happen now?

Why didn't it happen before?

I guess I've changed so many variables, I may not be able to track down the specific one(s).

However I think it's a good problem to have.

It's like having too much money, can always give some away.l

btw, this is with "only scan this computer" turned on.

Progress!  Two nights in a row, no less!
« Last Edit: April 04, 2007, 12:56 AM by rjam »

rjam

  • Supporting Member
  • Joined in 2007
  • **
  • default avatar
  • Posts: 17
    • View Profile
    • Donate to Member
Re: Sniffing, but can't smell
« Reply #29 on: April 04, 2007, 01:07 AM »
Okay, I tried the same technique for youtube's generic site.

Got a lot of urls instantly.  Over a hundred, tho not as many as from c-span site.

Closed urlsnooper and retried c-span.
This time I said NO to all requests, eg allow cookies.
Don't remember if it asked me to allow urlsnooper to connect to Internet again.
   but if it did, I said no.
urlsnooper did not receive packets.


mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,914
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: Sniffing, but can't smell
« Reply #30 on: April 04, 2007, 01:17 AM »
the good news is that at least the world is not totally insane.

the bad news is that this method of downloading a file is not very useful, because it doesnt allow url snooper to spy on the network traffic which is what you really need to do to get most sneaky urls.

rjam

  • Supporting Member
  • Joined in 2007
  • **
  • default avatar
  • Posts: 17
    • View Profile
    • Donate to Member
Re: Sniffing, but can't smell
« Reply #31 on: April 04, 2007, 01:27 AM »
Yes, that (the bad news part) was slowly beginning to dawn on me.

I was so happy to get ANY urls, that a lot of them seemed like heaven,
even if they were not coming in a way that would be useful.

Closed out urlsnooper and browser.

This time I "allowed" urlsnooper to connect to the Internet when my firewall prompted me.
It shouldn't be promting me, because I had the firewall config file set to "allow" urlsnooper
in the application list.
I'll go back and see if/how that got changed.

Also, this time I again allowed cookies (it didn't specify from where).

And the 173 urls auto popped in from youtube.

Same process for c-span.org auto pops in 342 urls.

After 173/342 urls, the "sniff network" auto turns itself off.

Opening these two websites in my browser, rather than in the urlsnooper "download" mode,
does not get any urls.

However I inadvertently fibbed when I said no packets come in with that narrow scan feature on.

I now see that a small number of packets came in right at the start, before they stopped coming in,
when I load a browser page for c-span or youtube.
I think the number was a few dozen for one site, and maybe close to a hundred for the other.
It happend so fast, that my aging eyes didn't notice till recently.

btw, I have been meaning to thank you for the help.

Very interesting and rewarding experience, even though final solution is not yet found.

I probably should go to bed now.  2:30 is past my usual bed time.

More later, if I learn or see anything.

rjam

rjam

  • Supporting Member
  • Joined in 2007
  • **
  • default avatar
  • Posts: 17
    • View Profile
    • Donate to Member
Re: Sniffing, but can't smell
« Reply #32 on: April 04, 2007, 01:49 AM »
Rechecked my AVG-Grisoft firewall config settings.

urlsnooper is still in the "allowed" application list.

however in my profile switch tabbed section,
I have a couple areas (aka network adapters) that are "unassigned" to a profile.

profile options are to "allow"  "block"  or "stand alone computer"
if I don't assign one of those 3 options to those new areas/networks,
then I guess "unassigned" gets auto entered

my default for known existing areas/networks, is "stand alone computer"

I don't know why these new areas/network adapters are showing up there

I deleted some of the duplicates and unnassigned areas/networks there a couple days ago.

Now they're back.

Over last year or so, I sometimes get a firewall notice popup saying there's a new area.
But that happened so often for so long, that I just auto click it closed.
I don't think I knew why those new areas/networks kept showing up periodically.
I just wanted them to go away.
I can't keep assigning "stand alone computer" to an unlimited number of these.
I don't even remember for sure how to do that anymore, instead of just closing the little grayed out msg box.

rjam

  • Supporting Member
  • Joined in 2007
  • **
  • default avatar
  • Posts: 17
    • View Profile
    • Donate to Member
Re: Sniffing, but can't smell
« Reply #33 on: April 04, 2007, 08:20 PM »
I've carefully read my AVG firewall info and it's references and distinctions between

network
     "adapters"
     "interfaces" &
     "areas"

The use of those 3 names overlaps in some ways, but is separate & distinct in other ways,
depending on the tools and controls being used within the firewall configuration choices.

Bottom line, I've removed all of the extra or duplicate network "areas" for local networks/interfaces. 
I've retained two essential local "interfaces", one for TV card and one for my network card.

This deletion of extra local network interfaces made no difference in urlsnooper function.

One config tab labeled "Networks" has headings inside it as:

"All Remote Networks" &
"All Local Interfaces"

Then individual named adapters, interfaces or areas are listed within each of those headings.


Another config tab labeled "Profile switch" has headings inside it as:

"All Local Interfaces" &
"All Network Areas"

Then individual named adapters, interfaces or areas are listed within each of those headings.


= = = = =

I'd told you that my computer was a stand alone, not otherwised networked inside my home,
other than it's connection to the Internet via Comcast ISP.

Well, this AVG firewall config tab listed a

"Local Network"

under the heading of "All Network Areas"

It had an assigned profile,
plus area activation tally,
last time & date activated, &
IP address of net gateway.

I recorded that data, then deleted that local network.

As I mentioned, that did not change urlsnooper performance.


= = = = =

The other 3 network interfaces/areas that I deleted,
were under the "All Local Interfaces" heading,
and were duplicates, seemingly inactive.


= = = = = = =

Despite URLsnooper not working as prescribed,

I have been able to go back to YouTube,
and have URLsnooper get embedded urls
for the video in which I was interested.

I did that with the test you (Mouser) showed me,
by manually typing into the field at the bottom of urlsnoooper search window,
the url of the webpage offering the video,
then clicking on the "download" button.
also at bottom of urlsnooper search page.

This loaded some dozens of urls,
similar to the way c-span's page did.

I found a half dozen promising urls in that list.
And tested those out in a separate browser, not via urlsnooper.
Some of them played the same video in the same page context as YouTube does.
Some do it a bit differently.
One did it in auto full screen mode.
The quality of that video already seemed better than previouslly accessible thru YouTube.
Then when I turned off all unnecessary programs on my pc,
the quality of the video was even better, even though not perfect.
The audio has always been high.
It might be worth recording now via my Grabbit program,
or by ASF recorder.

My recording of that video by Snaggit previously,
had too low quality video, tho the sound was good.


= = = = =

Meanwhile, I still also have two items listed in my firewall config network tab
under the heading of
"All Remote Networks"

One item is "Internet"
It has the options grayed out for editing or deleting it.

The 2nd item is one which I created/added manually a couple days ago
when URLsnooper failed the network autodetect test.
That one is for my SMC EZ Card 10/100.
It has options active to edit or delete it.
Editing button has further options for IP address, range, mask or whole network.


This is a long read even for the writer.

But doing those details, helped me sort out something that works well enough to meet my basic needs
for the short term.

Thanks again for all your help.