Author Topic: grab urls (Read 34498 times)

wraith808 · « **Reply #25 on:** April 14, 2018, 03:39 PM »

I don't feel comfortable using WGET. I need a GUI so that I can see the progress etc.

Is there a GUI download manager that will allow me to download a list of urls, from a login website, delaying 5 seconds between each attempt? Ideally, it could scan the links of the url and download any files of specific extensions in those urls?

-kalos (April 14, 2018, 03:22 PM)

Downthemall might do it. I haven't used it in forever. It probably wouldn't do it as is- what you'd have to do is make a simple site with the links, and then open it in firefox. All of this is, however, speculative, as I haven't tried it.

4wd · « **Reply #26 on:** April 15, 2018, 12:08 AM »

I don't feel comfortable using WGET. I need a GUI so that I can see the progress etc.

Is there a GUI download manager that will allow me to download a list of urls, from a login website, delaying 5 seconds between each attempt? Ideally, it could scan the links of the url and download any files of specific extensions in those urls?
-kalos (April 14, 2018, 03:22 PM)

Downthemall might do it.
-wraith808 (April 14, 2018, 03:39 PM)

It was suggested, he doesn't like it or it's too complicated.

JDownloader as suggested by Deozaan could most likely handle it, (logging in and downloading), but if he baulks at DownThemAll he's going to have no hope with JDownloader.

I don't feel comfortable using WGET. I need a GUI so that I can see the progress etc.
-kalos (April 14, 2018, 03:22 PM)

It outputs it's progress to the CLI window.

kalos · « **Reply #27 on:** April 19, 2018, 03:56 PM »

It's not that I don't like DownThemAll.
First, it doesn't work with latest Firefox.
Second, it doesn't have the option to download urls with delay between them, it will crash the server and get me banned!

tomos · « **Reply #28 on:** April 19, 2018, 11:27 PM »

^ already addressed AFAICS.
And you say you can use a different browser - try PaleMoon or Firefox ESR

I gave Downthemall a try, but is there a way to make the downloads delay every x seconds so that I won't abuse the server???
-kalos (December 02, 2016, 05:48 PM)
You can configure concurrent downloads and downloads-per-server in DownThemAll - that really should be all you need to to avoid "abusing" anything
-f0dder (December 27, 2016, 10:16 AM)

4wd · « **Reply #29 on:** April 20, 2018, 04:42 AM »

^ already addressed AFAICS.
And you say you can use a different browser - try PaleMoon or Firefox ESR
I gave Downthemall a try, but is there a way to make the downloads delay every x seconds so that I won't abuse the server???
-kalos (December 02, 2016, 05:48 PM)
You can configure concurrent downloads and downloads-per-server in DownThemAll - that really should be all you need to to avoid "abusing" anything
-f0dder (December 27, 2016, 10:16 AM)

-tomos (April 19, 2018, 11:27 PM)

dTA! can't insert a delay between downloads, the next is fired off as soon as the previous finishes.

wget can insert a delay, (including a random delay).

Code: Text [Select]

@echo off
wget --save-cookies cookies.txt --keep-session-cookies --post-data 'user=foo&password=bar' --delete-after http://server.com/auth.php
wget --load-cookies cookies.txt --secure-protocol="auto" --no-http-keep-alive -w 60 --random-wait --input-file="list.txt"

kalos · « **Reply #30 on:** April 22, 2018, 12:19 PM »

I installed Jdowloader2 and this horrible software changed my Firefox search engine to horrible bing.com!
I wonder how can I revert it back and what other 'alterations' it made to my system??

4wd · « **Reply #31 on:** April 22, 2018, 08:52 PM »

I installed Jdowloader2 and ~~this horrible~~ I failed to read any of the installation requests and blindly clicked buttons so the software changed my Firefox search engine to horrible bing.com!
-kalos (April 22, 2018, 12:19 PM)

FTFY

EDIT: While it might seem I'm making fun of you above it is only indirectly because the point I am making, (and has been made in other threads on this site regarding unwanted things happening during installation of some software), is I have installed JDownloader2 half a dozen times across various machines and not once has it ever changed the search engine in any browser I use, therefore you are doing something wrong.

kalos · « **Reply #32 on:** April 23, 2018, 06:05 PM »

I am very disappointed that some people here suggested me JDownloader.

After installing that horrible software, it hijacked my Firefox, changed homepage and search engine, and I believe it did more harm as now my google searches behave strange.
As you can see in the video below, after one second of loading the google results, some strange links appear!
<a href="https:// ufile.io/k18fj"></a>

What horrendous piece of software is that and why do you really suggest it?

wraith808 · « **Reply #33 on:** April 23, 2018, 09:02 PM »

^ You posted that twice after 4wd said it doesn't do that and he installed it himself?

KynloStephen66515 · « **Reply #34 on:** April 23, 2018, 09:56 PM »

I just downloaded JDownloader and it instantly got killed by my system with a "Severe" threat level, because of: Trojan:Win32/Occamy.C

I was only doing this so I could poke through the installation and have a looksie what's going on but now I'm curious if this is a FP or otherwise? I got the download from http://jdownloader.org/download/index which redirected me to a mega.nz link in order to download "Install JDownloader.rar".

**edit**

I've googled to check if anybody else has had this show up with JDownloader and found nothing...which has me even more curious now.

wraith808 · « **Reply #35 on:** April 23, 2018, 10:31 PM »

The URL was not detected by virustotal...

https://www.virustot...7d43e36f5d/detection

KynloStephen66515 · « **Reply #36 on:** April 23, 2018, 10:35 PM »

The URL was not detected by virustotal...

https://www.virustot...7d43e36f5d/detection
-wraith808 (April 23, 2018, 10:31 PM)

That just seems to have scanned the Mega.nz website rather than the RAR that came from there.

wraith808 · « **Reply #37 on:** April 23, 2018, 11:03 PM »

It scans through the url doesn't it?

Just in case, I downloaded it, and scanned the file (Install JDownloader.rar) with Malwarebytes. Nothing.

I then uploaded that to virustotal.

https://www.virustot...211b7d581f/detection

There are a few, but I think it's a heuristic failure or something, looking at the results. It does have adware in it, apparently, which might as 4wd put it, cause the symptoms stated if installed.

I don't care enough to go further than that.

KynloStephen66515 · « **Reply #38 on:** April 23, 2018, 11:07 PM »

It scans through the url doesn't it?
-wraith808 (April 23, 2018, 11:03 PM)

I clicked the link and it just seems to be scanning Mega as a page, rather than the download - I could be wrong but that's how it looks. I kinda wanna download the file again and let it through, then upload it to be scanned after...but then again, I don't want my computer to implode...sooooooooooooooo.......I won't do it haha...it's not worth setting up a secure environment to do it...I'm really not THAT bothered, just slightly curious seeing as 4wd has recommended it above.

4wd · « **Reply #39 on:** April 24, 2018, 12:04 AM »

I just downloaded JDownloader and it instantly got killed by my system with a "Severe" threat level, because of: Trojan:Win32/Occamy.C
-Stephen66515 (April 23, 2018, 09:56 PM)

Downloaded, extracted, nothing according to MBAM and Defender:

grab urls

@Stephen: Interested to know how you got that particular result since none of the engines on VirusTotal report it as such: Results

The comments against it on VirusTotal suggest that it will ask you to install "offers" or "toolbars" ... never had it do that but maybe it is based on what it sees as the default browser. Only installed browsers I have are Vivaldi and Edge, everything else is portable.

But anyway, RevoUninstalled my existing installation, (had a RevoUninstaller log for it to really clean up), and re-installed.
NOTE: I should point out, if there is a button that says Advanced or Custom Installation, use it - that's where "extras" are normally hidden.

grab urls

grab urls

grab urls

grab urls

It required 4 separate internet connections for the installation files:

2018-04-24 15_00_23-Windows Firewall Control - Rules Panel.png

grab urls

During the installation, MBAM and Defender reported nothing, when it was finished it opened a page in the browser, (Vivaldi), saying thanks for the install - only thing bad to say about it was it stuck an icon on the Desktop after I told it not to.

Scanning of the temp files after installation with MBAM showed the following:

2018-04-24 15_10_26-Malwarebytes Anti-Malware.png

2018-04-24 15_10_26-Malwarebytes Anti-Malware.png

2018-04-24 15_09_39-Malwarebytes Anti-Malware Home (Premium) 2.2.1.1043.png

grab urls

Not a threat, just an annoyance ... besides the fact that it actually didn't install anything.

PS:

... seeing as 4wd has recommended it above.
-Stephen66515 (April 23, 2018, 11:07 PM)

Didn't recommend it, I reinterated that @Deozaan suggested it and mentioned it could most likely do what he wanted, I'll keep with my suggestion of using wget even though I do regularly use JDownloader.

Deozaan · « **Reply #40 on:** April 24, 2018, 12:50 AM »

I downloaded from your link and Windows Defender immediately flagged it with the same results you got, Stephen.

https://www.microsof...;threatid=2147726780

EDIT: Something is really fishy about the site. The download links redirect you to all sorts of places before finally ending up on Mega. And supposedly JDownloader is Open Source but I couldn't find the source code.

4wd · « **Reply #41 on:** April 24, 2018, 12:55 AM »

I downloaded from your link and Windows Defender immediately flagged it with the same results you got, Stephen.

https://www.microsof...;threatid=2147726780
-Deozaan (April 24, 2018, 12:50 AM)

That is completely weird then, Defender is up to date - it's also strange that Microsoft report something completely different via VirusTotal, (PUA:Win32/Creprote - Potentially Unwanted Application, I'm guessing), for both the RAR and EXE.

And supposedly JDownloader is Open Source but I couldn't find the source code.
-Deozaan (April 24, 2018, 12:50 AM)

It seems to be available via SVN repository, (old version at GitHub).

Something is really fishy about the site. The download links redirect you to all sorts of places before finally ending up on Mega.

It appears to be their own CDN, (there's ~3 fetch.jdcdn.org links it redirects through):

Code: Text [Select]

Whois query for jdcdn.org...
Results returned from whois.publicinterestregistry.net:
Domain Name: JDCDN.ORG
Registrant Name: Thomas Rechenmacher
Registrant Organization: AppWork GmbH

AppWork is the company that wrote JDownloader.

KynloStephen66515 · « **Reply #42 on:** April 24, 2018, 10:09 AM »

I downloaded from your link and Windows Defender immediately flagged it with the same results you got, Stephen.

https://www.microsof...;threatid=2147726780

EDIT: Something is really fishy about the site. The download links redirect you to all sorts of places before finally ending up on Mega. And supposedly JDownloader is Open Source but I couldn't find the source code.
-Deozaan (April 24, 2018, 12:50 AM)

Yeah that's what struck me as odd - The CDN redirect didn't concern me that much but then being bounced around until finally landing on mega (why use a CDN if you're just gonna host your download elsewhere?) - The download was flagged and stopped by Chrome by Windows Defender (images above).

The whole thing just strikes me as very very odd indeed.

Note: In case the developer of JDownloader pops by and reads this, I'm not suggesting that the software IS bad/malware, I'm simply pointing out that it FEELS that way (again, the report from Defender could certainly be a false positive, and it wouldn't be the first time that has happened).

Deozaan · « **Reply #43 on:** April 24, 2018, 01:20 PM »

Something is really fishy about the site. The download links redirect you to all sorts of places before finally ending up on Mega.

It appears to be their own CDN, (there's ~3 fetch.jdcdn.org links it redirects through):

Code: Text [Select]
Whois query for jdcdn.org...
Results returned from whois.publicinterestregistry.net:
Domain Name: JDCDN.ORG
Registrant Name: Thomas Rechenmacher
Registrant Organization: AppWork GmbH

AppWork is the company that wrote JDownloader.
-4wd (April 24, 2018, 12:55 AM)

I figured jdcdn was JDownloader CDN, but the final redirect before ending up on Mega was some fishy looking firsturl.de (archived link) site. It looked like one of those sites that forces you to wait 30 seconds viewing ads before allowing you to get to the real link. Except it didn't force me to wait and just redirected me to Mega after a second or so. Seems it's just a URL shortener/tracker link.

kalos · « **Reply #44 on:** April 24, 2018, 01:38 PM »

I just downloaded JDownloader and it instantly got killed by my system with a "Severe" threat level, because of: Trojan:Win32/Occamy.C

I was only doing this so I could poke through the installation and have a looksie what's going on but now I'm curious if this is a FP or otherwise? I got the download from http://jdownloader.org/download/index which redirected me to a mega.nz link in order to download "Install JDownloader.rar".

**edit**

I've googled to check if anybody else has had this show up with JDownloader and found nothing...which has me even more curious now.

[ Invalid Attachment ]
[ Invalid Attachment ]
-Stephen66515 (April 23, 2018, 09:56 PM)

Interesting, what system do you have?
To be honest, I may did a mistake and installed the toolbars the installer offers at the begining.
But isn't there a way to clean my system? I still get the fake google results.
I have tried everything RogueKiller, Zemana, MalwareBytes, AdwCleaner, JRT, with no luck.

tomos · « **Reply #45 on:** April 24, 2018, 02:51 PM »

I installed Jdowloader2 and this horrible software changed my Firefox search engine to horrible bing.com!
I wonder how can I revert it back and what other 'alterations' it made to my system??
-kalos (April 22, 2018, 12:19 PM)

Were you not change (back) the default search engine and home page yourself (?)

And can you elaborate on the 'fake google results'?

4wd · « **Reply #46 on:** April 24, 2018, 03:38 PM »

The whole thing just strikes me as very very odd indeed.
-Stephen66515 (April 24, 2018, 10:09 AM)

What I find odd is across six different systems using the same AV you're getting three different results on the same file:

Stephen: Trojan (Chrome)
Deozaan: Trojan
Wraith: Clean
Kalos: Clean (Firefox)
4wd: Clean (Vivaldi)
VirusTotal: PUA

I'm assuming that both Wraith and kalos didn't get a hit by Defender because neither said the installer was quarantined.
I'm also assuming that kalos used Firefox since he never mentions anything else.

Now I'm going to guess that Deozaan uses Chrome and Wraith, Not Chrome ?

VirusTotal, yeah well, its results change dependent on wind direction and time by the looks, only have to look at the results for the keylogger a couple of days ago, started off at 14/64, then 22/65, then 26/66, now who knows what they are. That and the overly scary (mis)classifications used by the majority of AV manufacturers based on what something looks like, not what it does ... its results are something I take with a grain of salt, (my opinion only).

kalos · « **Reply #47 on:** April 24, 2018, 03:41 PM »

I installed Jdowloader2 and this horrible software changed my Firefox search engine to horrible bing.com!
I wonder how can I revert it back and what other 'alterations' it made to my system??
-kalos (April 22, 2018, 12:19 PM)
Were you not change (back) the default search engine and home page yourself (?)

And can you elaborate on the 'fake google results'?
-tomos (April 24, 2018, 02:51 PM)

I did change back the homepage and search engine, but who knows what other damage it has caused.
About fake google results, I did post a video earlier:
<a href="https://ufile.io/k18fj"></a>

wraith808 · « **Reply #48 on:** April 24, 2018, 03:43 PM »

I installed Jdowloader2 and this horrible software changed my Firefox search engine to horrible bing.com!
I wonder how can I revert it back and what other 'alterations' it made to my system??
-kalos (April 22, 2018, 12:19 PM)
Were you not change (back) the default search engine and home page yourself (?)

And can you elaborate on the 'fake google results'?
-tomos (April 24, 2018, 02:51 PM)

I've seen it before on infected systems. They hijack any searches to do it on another platform and make it look pretty similar to google. Changing the search engine doesn't change this; it's an extension/plugin/BHO.

I installed Jdowloader2 and this horrible software changed my Firefox search engine to horrible bing.com!
I wonder how can I revert it back and what other 'alterations' it made to my system??
-kalos (April 22, 2018, 12:19 PM)
Were you not change (back) the default search engine and home page yourself (?)

And can you elaborate on the 'fake google results'?
-tomos (April 24, 2018, 02:51 PM)

I did post a video earlier:
<a href="https://ufile.io/k18fj"></a>
-kalos (April 24, 2018, 03:41 PM)

I changed that in the prior post to not be link, because archive.is shows it linking to some sketchy sites. I'll leave it, but you might want to look at using a more reputable site to host your downloads- perhaps even just attach it here?

kalos · « **Reply #49 on:** April 24, 2018, 04:21 PM »

I installed Jdowloader2 and this horrible software changed my Firefox search engine to horrible bing.com!
I wonder how can I revert it back and what other 'alterations' it made to my system??
-kalos (April 22, 2018, 12:19 PM)
Were you not change (back) the default search engine and home page yourself (?)

And can you elaborate on the 'fake google results'?
-tomos (April 24, 2018, 02:51 PM)

I've seen it before on infected systems. They hijack any searches to do it on another platform and make it look pretty similar to google. Changing the search engine doesn't change this; it's an extension/plugin/BHO.

I installed Jdowloader2 and this horrible software changed my Firefox search engine to horrible bing.com!
I wonder how can I revert it back and what other 'alterations' it made to my system??
-kalos (April 22, 2018, 12:19 PM)
Were you not change (back) the default search engine and home page yourself (?)

And can you elaborate on the 'fake google results'?
-tomos (April 24, 2018, 02:51 PM)

I did post a video earlier:
<a href="https://ufile.io/k18fj"></a>
-kalos (April 24, 2018, 03:41 PM)

I changed that in the prior post to not be link, because archive.is shows it linking to some sketchy sites. I'll leave it, but you might want to look at using a more reputable site to host your downloads- perhaps even just attach it here?
-wraith808 (April 24, 2018, 03:43 PM)

Sure, here it is.
EDIT: oh it doesn;t let me upload flv!