how to extract links from a site?

[steve_rb]

I am looking for a program to extract hiden links inside a site starting with let say for example "http://www.rapidshare.de/" . Is there any program to do this or anyone has any idea or comment?

Regards
Steve

[zridling]

Check out fellow DC member Veign's donationware XSite software. I've used it to collect links from complex blog posts that I wanted to follow up on later.

[mouser]

do you just want to extract links on a single page, or on an entire site?
and you just want the links, not the entire site content?

there are some good tools for grabbing the entire content from a site (Teleport Pro, Offline Explorer).

and there are some good download managers (and probably firefox extensions) which can list all the links on a page from within your browser.

[justice]

Also, Opera has a Link panel, which gives you all the links on a page, you can then search through them, but not save them all out it seems.

If you'd use a site link checker like Xeno then you'd get all the links in a site including which ones do no longer work.

[steve_rb]

@ mouser
I want to extract links inside entire site not just one page. I also want hiden links (links you can not see directly but you can access via search field inside that site for example) to be extracted too.

@ zridling
Veign's site is down. I can't access it. Is you have Xsite program please send me via email ([email protected]).

Regards

[steve_rb]

@ justice
Do you have any download link for Xenon?

 

[justice]

http://home.snafu.de...tilman/xenulink.html
Sorry It's Xenu

[zridling]

Sent XSite your way.

[steve_rb]

@ zridling

thanks for the program but It only can extract links on the page I gave it to the program. It does not follow links. I want a program to follow all links and search all pages and extact all links in that pages. Simply all links in the entire site.

 

[steve_rb]

Xenu is great and could almost extract all links from a site but still not all hidden or redirected urls. Thanks justice for letting me know about this software. It solves my problem to some extend but I still need more power full software specially to show redirected links and follow all links untill to get to the bottom of them.

 

[Renegade]

If you feel like playing around, you can try this web robot in VB.NET or C#.

You'd be surprised just how many really cool things are available for free at the CodeProject.

(OK - They take some work, but the basics are there.)

[zridling]

Steve, as mouser mentioned, you're looking for something more powerful then. The most accurate I've found is Teleport Pro.

[Renegade]

I've found Teleport Pro to be very valuable. You'll have a hard time finding something that can automatically surf all your porn for you!

Edit: Yep - I know what you're using it for Zaine!

[Darwin]

Justice  steve_rb - I'd edit out your e-mail address above, post-haste, if I were you ( I *HATE* spam, get too much of it as it is, and don't go looking for any more of it!  )...

[ender]

I am looking for a program to extract hiden links inside a site starting with let say for example "http://www.rapidshare.de/" . Is there any program to do this or anyone has any idea or comment?

Regards
Steve

-steve_rb (April 29, 2007, 02:16 AM)

I am using either of these two programs, if I want to batch download files from rapidshare & other temp. hoster:
http://www.rapget.com/en/index.html
http://www.portablefreeware.com/?id=1111

http://translate.goo...dimonius.ru/dusd.php
http://www.portablefreeware.com/?id=1251

both still require you to manually put in the links, but after that, I can leave my computer on overnight, and the next day, all files are on my harddrive.

[Tinman57]

  Something like ScanDL that Searches HTML for links and constructs a file list?

http://scandl.com/en/news.html