Welcome Guest.   Make a donation to an author on the site October 01, 2014, 09:17:55 AM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
Read the full one-year retrospective report on DonationCoder.com.
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: [1] 2 Next   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: Partial (corrupted) downloads from a server (Not a valid win32 application etc)  (Read 9784 times)
mouser
First Author
Administrator
*****
Posts: 33,422



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« on: April 11, 2011, 08:05:47 AM »

We've always had an occasional occurrence of this problem, but it seems to be getting worse lately..  Someone will try to download a program from our website (main or member server), and when they try to run it, they get an error saying its "not a valid win32 application" or "The setup files are corrupted".

Worse yet, if they try to re-download the browser gives them the cached half-finished download again so the problem recurs and they are left very confused and thinking the software is defective.

A search on the web shows that this does happen enough to everyone that the problem is well known

But my question is: Does anyone have any server experience who can help us understand how to fix this problem on the server end -- to stop this from happening so often?  What is causing downloads to fail half way through, and leave the browser thinking the download is complete? And how can we fix this on our server end?

Does anyone have any server running experience have any idea? Help!
« Last Edit: April 11, 2011, 08:11:19 AM by mouser » Logged
vlastimil
Honorary Member
**
Posts: 305



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #1 on: April 11, 2011, 08:42:25 AM »

Not that I know the answer for that question (as much as I would like to know) (maybe fiddling with http headers would help with the re-download), but it also sometimes happens to me and I usually have the download on multiple urls and have an additional .zip with the installer and I tell the people to download using one of the other urls or with different browser.
Logged
Stoic Joker
Honorary Member
**
Posts: 5,271



View Profile WWW Give some DonationCredits to this forum member
« Reply #2 on: April 11, 2011, 11:55:21 AM »

Does the server allow download resuming?
Logged
Gothi[c]
DC Server Admin
Charter Honorary Member
***
Posts: 857



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #3 on: April 11, 2011, 04:17:47 PM »

Does the server allow download resuming?
Yes. (Apache does by default)
Logged
mouser
First Author
Administrator
*****
Posts: 33,422



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #4 on: April 11, 2011, 04:24:35 PM »

The thing i find strangest is that the browser is not figuring out that the download is incomplete.. that's what's really throwing me for a loop -- i mean normally the browser knows when a connection has failed, etc.  What's particularly bizarre in this case is that the browser and server are somehow convinced that the download has fully finished successfully.
Logged
Ath
Supporting Member
**
Posts: 2,223



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #5 on: April 11, 2011, 04:36:20 PM »

Are you losing any network packets between the server and the nearest measurable connection, like a switch that went goofy or something like that (aka: [partial] hardware failure)?
Logged

mouser
First Author
Administrator
*****
Posts: 33,422



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #6 on: April 11, 2011, 04:40:43 PM »

loss of packets is quite possible -- but normally in this case doesn't the browser detect that it is having communication problems and stall, etc.  I mean thats what i see happen when we know there is a packet loss issue.  The result is slow downloads not corrupted ones.
Logged
worstje
Honorary Member
**
Posts: 555



The Gent with the White Hat

View Profile Give some DonationCredits to this forum member
« Reply #7 on: April 11, 2011, 04:57:54 PM »

Broken hardware is my bet. Several years ago, in a bit of a convoluted setup involving a big room, several bookcases and a troublesome place where the wire came out of the wall, I had a router that balanced on a bunch of heavy law books. And when cleaning, someone apparently made it fall off those books before putting it back as if nothing had happened.

The results were extra-ordinary: rather than have the network connection die or anything of the sort, stuff just started getting garbled. Websites had garbled images with random lines and color shifts and the sorts; HTML documents had random characters happen and so forth. So somehow this entire affair happened somewhere inbetween the layers of error-correction; I suspect the router had its memory or cpu get messed up, but had its networking interfaces still work peachy.

Perhaps your situation is similar in nature here.
Logged
JavaJones
Review 2.0 Designer
Charter Member
***
Posts: 2,537



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #8 on: April 11, 2011, 05:01:14 PM »

I kind of don't think such an unusual situation as worstje describes is at play. I say that mostly because I've seen similar issues many times from other servers. I personally think it's a failure of the browser's caching mechanisms or something user-side, not server-side (not certain though). Granted it doesn't happen for all downloads, which might suggest it is a server-specific thing, but when it does happen it does seem to (usually) help to clear the browser's cache and download history, so that says something to me...

- Oshyan
Logged

The New Adventures of Oshyan Greene - A life in pictures...
mouser
First Author
Administrator
*****
Posts: 33,422



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #9 on: April 11, 2011, 06:07:10 PM »

Its not limited to a specific browser brand -- it happens in all browsers.  It's random.  Its not a corrupted byte, it's a download that stops somewhere in the middle but ends up with the browser thinking it has successfully downloaded the entire file.  And yes, at that point the browser is so convinced that it has successfully downloaded the full file, that it will refuse to redownload it, and will serve up the cached copy when the user tries to redownload -- making the problem much, much worse and more confusing for the user.
Logged
jaden
Supporting Member
**
Posts: 137



View Profile WWW Give some DonationCredits to this forum member
« Reply #10 on: April 12, 2011, 12:26:20 PM »

A few months ago I ran into a problem with MediaTemple where certain downloads were corrupted.  It turned out the network guys had enabled a virus scanner that was incorrectly flagging certain files as having a virus.  They admitted the scanner had never correctly identified a payload, only given false positives, and disabled it so our downloads worked.  Perhaps something along those lines is happening here?
Logged
jaden
Supporting Member
**
Posts: 137



View Profile WWW Give some DonationCredits to this forum member
« Reply #11 on: April 12, 2011, 12:28:34 PM »

It could also be an anti-virus app on the user's system that's scanning downloads and messing with the file.  Do the failures occur with particular files, or is it completely random?
Logged
mouser
First Author
Administrator
*****
Posts: 33,422



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #12 on: April 12, 2011, 12:39:23 PM »

it's usually the bigger exe setup files i think.. but i'm not sure there is a real pattern to it other than that the files are usually several megabytes large.
Logged
JavaJones
Review 2.0 Designer
Charter Member
***
Posts: 2,537



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #13 on: April 12, 2011, 12:41:42 PM »

Having experienced this myself on the client side I can say with fair certainty it's not likely to be antivirus-related on the client end (but maybe server side AV?). The reason I say this is that I've had it happen across a range of different AV software and configuration options, including situations where the scanning of web downloads is not automatically enabled in the AV but is instead called by the browser once the download is finished (suggesting again that the browser is prematurely concluding the file is complete).

- Oshyan
Logged

The New Adventures of Oshyan Greene - A life in pictures...
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #14 on: April 12, 2011, 12:50:25 PM »

Not sure if it's in any way related, but FWIW I can't reliably grab the minecraft picture from this dcmember - displays a bit, and then stalls (FireFox 4). With wget, I'm seeing download rates at less than 10kb/s, and then massive stall. These images are relatively large, probably around a couple megabytes for some of them.

Apart from Microsoft Security Essentials, I'm not running any anti-malware or internet filtering junk - and the situation isn't any better when wget'ing from my linux server.
Logged

- carpe noctem
jaden
Supporting Member
**
Posts: 137



View Profile WWW Give some DonationCredits to this forum member
« Reply #15 on: April 12, 2011, 12:59:38 PM »

Apart from Microsoft Security Essentials, I'm not running any anti-malware or internet filtering junk - and the situation isn't any better when wget'ing from my linux server.

That pretty much rules out an issue on the client side if wget is failing.  In the MediaTemple case it only happened with very large files (more chance to conflict with a known virus pattern?) but filtering only occurred when the files were served from Apache.  Can someone scp a problem file to another server to see if the file comes across alright?
Logged
jaden
Supporting Member
**
Posts: 137



View Profile WWW Give some DonationCredits to this forum member
« Reply #16 on: April 12, 2011, 01:08:03 PM »

I just tried downloading this PNG: http://carrolld.dcmembers.com/minecraft/2011-04-07.png and I saw the behavior being described (browser thought it was done, but only the top portion of the downloaded image is visible).

In our case the files were zips and were corrupted, so I'm not sure if this could be the same issue or not.  It's possible when a server-side scanner flags a file as infected it stops the download and the browser thinks it's complete.
Logged
JavaJones
Review 2.0 Designer
Charter Member
***
Posts: 2,537



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #17 on: April 12, 2011, 01:10:05 PM »

Great find f0dder, that seems likely related. What's interesting is that the download seems to start out quickly then slow down terribly (the same behavior is common to a browser and wget with those specific PNG files). Fetching a similarly sized EXE file (FlipSuite in my test) went like lightning.

- Oshyan
Logged

The New Adventures of Oshyan Greene - A life in pictures...
40hz
Supporting Member
**
Posts: 10,675



see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #18 on: April 12, 2011, 01:26:02 PM »

Wonder if this is somehow related to the stalled picture upload problem you'll occasionally encounter during forum posting. If so, then it's definitely a server rather than a client-side issue.

I seem to recall reading somewhere that there was an issue with certain php download scripts that retrieved the requested file from a mysql database and were responsible for corrupted downloads. Had something to do with buffer settings iirc.

Can't find it now of course. Anybody else remember seeing that?
« Last Edit: April 12, 2011, 01:38:53 PM by 40hz » Logged

Don't you see? It's turtles all the way down!
jaden
Supporting Member
**
Posts: 137



View Profile WWW Give some DonationCredits to this forum member
« Reply #19 on: April 12, 2011, 01:31:37 PM »

Another data point.  The initial download of http://carrolld.dcmembers.com/minecraft/2011-04-07.png in Chrome produced a 24KB file.  In Firefox it stalled at 156KB and Opera stalled at 209KB. 
Logged
Stoic Joker
Honorary Member
**
Posts: 5,271



View Profile WWW Give some DonationCredits to this forum member
« Reply #20 on: April 12, 2011, 03:24:59 PM »

Another data point.  The initial download of http://carrolld.dcmembers.com/minecraft/2011-04-07.png in Chrome produced a 24KB file.  In Firefox it stalled at 156KB and Opera stalled at 209KB.

I may have just got lucky, but IE9 grabbed that file and all the others at the link f0dder posted just fine on the first shot.
Logged
Stoic Joker
Honorary Member
**
Posts: 5,271



View Profile WWW Give some DonationCredits to this forum member
« Reply #21 on: April 12, 2011, 03:27:48 PM »

I seem to recall reading somewhere that there was an issue with certain php download scripts that retrieved the requested file from a mysql database and were responsible for corrupted downloads. Had something to do with buffer settings iirc.

That does sound familiar, but I was thinking it was only an ASP thing (but I could be wrong). Had to do with disabling the caching if script execution results and just spitting it up the wire ... Yes??
Logged
mwb1100
Supporting Member
**
Posts: 1,315


View Profile Give some DonationCredits to this forum member
« Reply #22 on: April 12, 2011, 03:57:49 PM »

I don't know if this will help anyone narrow this down (or just add noise to the data), but I often have trouble downloading large files in Firefox or IE at work (no errors reported, just incomplete files after the download finishes). When that happens I use wget which has yet to have a problem.

That's an indication to me that the problem is the browser's fault, but what do I know?

Anyway, if someone here has these issues downloading in the browser, it might shed some light here if they find whether or not wget is more reliable.
Logged
jaden
Supporting Member
**
Posts: 137



View Profile WWW Give some DonationCredits to this forum member
« Reply #23 on: April 12, 2011, 05:00:05 PM »

I may have just got lucky, but IE9 grabbed that file and all the others at the link f0dder posted just fine on the first shot.

I tried it again and it downloaded completely in all 3 browsers, as well as wget.
Logged
Ath
Supporting Member
**
Posts: 2,223



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #24 on: April 12, 2011, 05:16:54 PM »

I tried it again and it downloaded completely in all 3 browsers, as well as wget.
I could get it to load completely in FF3 just now for the first time too

[wishful thinking mode]
Usually that happens after mouser and Gothi[c] do some config changes to the/a server, they wait for a little time to see if 'that helped', and if it did they happily report it fixed here tellme
Let's just wait for that then embarassed (I hope so, keeping my fingers crossed)
[/wishful thinking mode]
Logged

Pages: [1] 2 Next   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.043s | Server load: 0.33 ]