Home | Blog | Software | Reviews and Features | Forum | Help | Donate | About us
topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • December 10, 2016, 08:16:15 PM
  • Proudly celebrating 10 years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: grab urls  (Read 692 times)

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,477
    • View Profile
    • Donate to Member
grab urls
« on: November 29, 2016, 05:33:29 PM »
Hello!

I have a list of URLs that I want to download from a website that has a form login.

How can I do that automatically in batch?

Is there a downloader and can the downloader use the cookie from my Firefox to authenticate?

thanks!

Target

  • Honorary Member
  • Joined in 2006
  • **
  • Posts: 1,606
    • View Profile
    • Donate to Member
Re: grab urls
« Reply #1 on: November 29, 2016, 07:01:27 PM »
probably doesnt handle the login, but downthemall?

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,477
    • View Profile
    • Donate to Member
Re: grab urls
« Reply #2 on: November 29, 2016, 07:34:13 PM »
probably doesnt handle the login, but downthemall?

Can I load a text file with the urls to that and it will use the cookie to authenticate and download?

Target

  • Honorary Member
  • Joined in 2006
  • **
  • Posts: 1,606
    • View Profile
    • Donate to Member
Re: grab urls
« Reply #3 on: November 29, 2016, 07:45:40 PM »
probably doesnt handle the login, but downthemall?

Can I load a text file with the urls to that and it will use the cookie to authenticate and download?

have you looked at the tool at all?  did you actually read what I wrote?

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,477
    • View Profile
    • Donate to Member
Re: grab urls
« Reply #4 on: November 29, 2016, 07:47:48 PM »
Well I saw the screen shots! I couldn't figure out

Target

  • Honorary Member
  • Joined in 2006
  • **
  • Posts: 1,606
    • View Profile
    • Donate to Member
Re: grab urls
« Reply #5 on: November 29, 2016, 08:26:17 PM »
DownThemAll is all you can desire from a download manager: it features an advanced accelerator that increases speed up to 4x and it allows you to pause and resume downloads at any time.

DownThemAll is fast, reliable and easy-to-use! It lets you download all the links or images contained in a webpage and much more: you can refine your downloads by fully customizable criteria to get only what you really want! Be in full control over your downloads, dedicated speed and number of parallel connections at any time. Use Metalinks or add mirrors manually to download a file from different servers at the same time.

DownThemAll is open-source and freeware. No Adware, no Spyware, no hidden costs!

feature list...

4wd

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 4,477
    • View Profile
    • Donate to Member
Re: grab urls
« Reply #6 on: November 29, 2016, 09:13:59 PM »
I have a list of URLs that I want to download from a website that has a form login.

How can I do that automatically in batch?

Is there a downloader and can the downloader use the cookie from my Firefox to authenticate?

wget

Login with wget and keep the cookie

Parameter to download files from a list:  -i list.txt

Deozaan

  • Charter Member
  • Joined in 2006
  • ***
  • Points: 1
  • Posts: 7,725
    • View Profile
    • The Blog of Deozaan
    • Read more about this member.
    • Donate to Member
Re: grab urls
« Reply #7 on: November 30, 2016, 02:27:46 AM »
This also sounds like something that could possibly be handled by JDownloader.


IainB

  • Supporting Member
  • Joined in 2008
  • **
  • Posts: 6,143
  • Slartibartfarst
    • View Profile
    • Donate to Member
Re: grab urls - try GetRight?
« Reply #8 on: November 30, 2016, 07:27:45 AM »
You could do worse than try GetRight - http://getright.com/ (from Headlight Software - http://headlightinc.com/).
I think it might have been a forerunner or led the way for meeting all/most downloading requirements, including batch downloading from a text list, passing logon credentials, etc. It works with major browsers too.
GetRight also includes the incredibly useful GetRight Browser, which enables the user to browse/view the directories of accessible download sites.

Some people (not me, you understand) might say that, with a bit of experimentation, it is surprising how much stuff one can find to download with GetRight that might not be apparently available to the casual/inexpert enquirer, by back-dooring weak/nonexistent access control and apparently locked directories ...    :o   - however, I couldn't possibly comment.
« Last Edit: November 30, 2016, 12:14:09 PM by IainB, Reason: Typo correction. »

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,477
    • View Profile
    • Donate to Member
Re: grab urls
« Reply #9 on: December 02, 2016, 05:48:45 PM »
I gave Downthemall a try, but is there a way to make the downloads delay every x seconds so that I won't abuse the server???

Or any other solution, as easy as Downthemall that integrates seamlessly with my Firefox? (I can use other browsers/software)

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,477
    • View Profile
    • Donate to Member
Re: grab urls
« Reply #10 on: December 09, 2016, 02:04:47 PM »
anyone please?

I need a download manager that will use my Firefox cookie / form authentication and that will download a list of urls I will feed to it delaying between each!

wget may work but I need to read the manual to make it work and I don't have the capacity to do tests!

Deozaan

  • Charter Member
  • Joined in 2006
  • ***
  • Points: 1
  • Posts: 7,725
    • View Profile
    • The Blog of Deozaan
    • Read more about this member.
    • Donate to Member
Re: grab urls
« Reply #11 on: December 09, 2016, 02:51:59 PM »
Did you try JDownloader or GetRight?


kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,477
    • View Profile
    • Donate to Member
Re: grab urls
« Reply #12 on: December 09, 2016, 03:05:01 PM »
I read online for JDownloader that it doesnt support delay between downloads or it needs a script or something.

I went to download GetRight but is it payware/spyware?