Main Area and Open Discussion > Living Room

grab urls

<< < (5/16) > >>

Deozaan:
Did threads just get merged or have I been into the cider too much?
-4wd (July 09, 2017, 03:35 PM)
--- End quote ---

They got merged. You can tell by some of the pixels :P, also because some of the posts have different topics than others.

4wd:
Did threads just get merged or have I been into the cider too much?
-4wd (July 09, 2017, 03:35 PM)
--- End quote ---

They got merged. You can tell by some of the pixels :P, also because some of the posts have different topics than others.
-Deozaan (July 09, 2017, 03:41 PM)
--- End quote ---

Thank heavens, I've only had one cider today, a very nice Thatchers Vintage Cider  :D

kalos:
Did threads just get merged or have I been into the cider too much?

thanks, but that stackoverflow thread is a mess-kalos (July 09, 2017, 09:14 AM)
--- End quote ---

How exactly is the first answer in the thread a "mess"?

Based on the manual page:

# Log in to the server.  This can be done only once.                   
wget --save-cookies cookies.txt \
     --keep-session-cookies \
     --post-data 'user=foo&password=bar' \
     --delete-after \
     http://server.com/auth.php

# Now grab the page or pages we care about.
wget --load-cookies cookies.txt \
     http://server.com/interesting/article.php
--- End quote ---

Substitute -i list.txt for the URL in the second command, list.txt contains a list of URLs to download.

Plus further down the answers:
I had the same problem. My solution was to do the login via Chrome and save the cookies data to a textfile. This is easily done with this Chrome extention: Chrome cookie.txt export extension.

When you get the cookies data, there is also an example on how to use them with wget. A simple copy-paste command line is provided to you.
--- End quote ---
-4wd (July 09, 2017, 03:35 PM)
--- End quote ---


Thanks but, the problem with that is that some cookies/form-logins expire. So when you go on downloading a file it displays the form to login instead.
That's why I am looking for a software that can detect that and use my login each time it is need.
Is there a way to do this?

4wd:
Thanks but, the problem with that is that some cookies/form-logins expire. So when you go on downloading a file it displays the form to login instead.
That's why I am looking for a software that can detect that and use my login each time it is need.
Is there a way to do this?-kalos (April 08, 2018, 04:13 PM)
--- End quote ---

So you're saying that the cookies expire in the couple of seconds between sending the first command followed by the second command?

kalos:
I don't feel comfortable using WGET. I need a GUI so that I can see the progress etc.

Is there a GUI download manager that will allow me to download a list of urls, from a login website, delaying 5 seconds between each attempt? Ideally, it could scan the links of the url and download any files of specific extensions in those urls?

Navigation

[0] Message Index

[#] Next page

[*] Previous page