TB! Connection to host broken... "DELE 414", "RETR 423" over and over??

[CodeTRUCKER]

I have been trying to re-establish  TheBAT! as my primary email client, but I have been stuck on the following log error for days and nothing I have tried has helped.

6/1/2009, 01:01:17: FETCH - receiving mail messages
6/1/2009, 01:01:18: FETCH - connected to POP3 server
6/1/2009, 01:01:18: FETCH - authenticated (plain)
6/1/2009, 01:01:26: FETCH - 12730 messages in the mailbox, 12308 new
!6/1/2009, 01:02:47: FETCH - Connection to host broken (last commands sent were: "DELE 414", "RETR 423")

Other than the times and the number of messages the logs are always the same.  I realize "DELE 414", "RETR 423" means Delete message 414 and Retrieve message 432, respectively, but what I do not understand is WHY it is hanging on those messages?

The next step will be to destroy my TB! mail file and re-download all the mails from the server.  I don't mind this, except I have over a hundred folders (years of mails) and about the same number of filters.

[mouser]

i know you were trying antispam sniper, have you tried disabling that and see if it works. if it does work after that,
then do what i suggested before, and configure all your accounts to NOT delete from the server.

[CodeTRUCKER]

i know you were trying antispam sniper, have you tried disabling that and see if it works. if it does work after that,
then do what i suggested before, and configure all your accounts to NOT delete from the server.

I already disabled the AntiSpamSniper.  The error persists.  I can only guess at what is happening.

[mouser]

well you have 12000+ "new" emails on your server.. something is up.. i wonder if that alone would kill the mail program.

[CodeTRUCKER]

well you have 12000+ "new" emails on your server.. something is up.. i wonder if that alone would kill the mail program.

Sounds like something is corrupted in the "flags" in TheBAT! or on the server.  I am going to try setting about 500 mails on the server to "New" and then reset them to "Read" and see what happens.

Be right back...

[CodeTRUCKER]

 Now the TB! is locking up!  

Since this has been happening for days, I am going to just bite the bullet and download those 12,000 "new" emails pretty soon.  Something is hung up and neither the server nor the client can figure out what to do.

I will try installing a different no-frills client, like mrainey's "Courier" to determine if the issue resides on the server.

Give me a few minutes...

[Edit] - Had no trouble downloading mails with another client.  I will delete my TB! mail file and try again.  I will report back here when I have some results, but I am am still interested in any one's thoughts. 

[Innuendo]

From within The Bat! you may want to try to try Accounts/Dispatch Mail On Server/All messages and see if you can find anything odd. 12,000 new emails is odd....looks like you got mailbombed or something.

When you downloaded the messages with another client, did you actually receive 12,000 messages?

And what version of The Bat! are you using. The latest release version? A beta version, perhaps?

EDIT: And on the subject of possible database corruption, when was the last time you ran Folder/Maintenance Centre? That will check your database of emails for corruption and fix any errors it finds.

[CodeTRUCKER]

From within The Bat! you may want to try to try Accounts/Dispatch Mail On Server/All messages and see if you can find anything odd. 12,000 new emails is odd....looks like you got mailbombed or something.

I am afraid I have not been clear (that is what I get for trying to tackle "tech" issues after working for 20+ hours). 

My mail store is comprised of about 12,000 mails I have collected over the years and deemed valuable enough to keep.  I am not getting mailbombed or spammed. 

For reasons of my own, I migrated all 12K to a different area on my server.  At this point I installed Mailwasher Pro and after a while I canned it and installed AntiSpamSniper.  I had forgotten that I had migrated my mail store and went back to the server and migrated them to back to my main box.  WHAM!  TheBAT! and AntiSpamSniper began downloading the whole 12K as new mails!

At some point something went askew and no more mails would download.  The other anomaly I failed to mention earlier (sorry, Mouser) was every time I attempted to exit TheBAT! It would drive my CPU above 50% and just stay there leaving only the window "frames" on the screen until I ended the process.  Nothing else allowed termination.

When you downloaded the messages with another client, did you actually receive 12,000 messages?

I installed the latest Thunderbird.  Once installed, the mails began to come in normally.  After I was satisfied with T-Birds's performance and my server's function I closed down T-Bird and went back to the drawing board with TheBAT!

And what version of The Bat! are you using. The latest release version? A beta version, perhaps?

thebat_home_4-0-38.  I have not purchased the upgrade yet.  I am waiting to see if my needs justify the purchase.  Comments, Anyone?

EDIT: And on the subject of possible database corruption, when was the last time you ran Folder/Maintenance Centre? That will check your database of emails for corruption and fix any errors it finds.

I ran it several times during the process.  I never noticed anything amiss.

Thanks for the help!

[CodeTRUCKER]

Update:  I did a clean uninstall of TheBAT! and AntiSpamSniper and started over.  So far everything is working as it should, but the "Connection Centre" shows I have only downloaded...

Message: [ 423 / 12731 ]
Transfer: [15.52M bytes / 576.068M bytes] [843 CPS]
Time:      Elapsed: 05:22:57] [Estimated: 194:17:30]

That is only about 80 emails / 3MBs an hour.  That seems very slow to me even if I am not on a T1.  Could the AntiSpamSniper be slowing it down?

[Dormouse]

I've only just installed AntiSpamSniper and it has substantially slowed my downloads. Of course, I'm only downloading a few at a time, so doesn't really matter to me.

[CodeTRUCKER]

I've only just installed AntiSpamSniper and it has substantially slowed my downloads. Of course, I'm only downloading a few at a time, so doesn't really matter to me.

Whew!  Thanks, maybe I no longer have an issue then!  

[Edit]  -  No dice.  I just found this on my logs...

6/1/2009, 15:55:15: FETCH - authenticated (plain)
6/1/2009, 15:55:30: FETCH - 12739 messages in the mailbox, 12317 new
!6/1/2009, 15:56:48: FETCH - Connection to host broken (last commands sent were: "UIDL", "RETR 423")

Look familiar?  Will this nightmare ever end!

[CodeTRUCKER]

Yippee!  I have a new lease on life!  

My problem is my Avast! Home anti-virus is what has been slowing things down.  If I suspend Avast! TheBAT! gets the mails as fast as any of the other clients I have been experimenting with today.

Now all I have to do is figure out how to make each of the "exclamation points" play nice with each other.  

Hmmm... may this is not so easy either?  The real question is "what" caused the slowdown since it was running fine then BAM!

[Jibz]

One guess would be that your anti-virus is intercepting the mail requests and doing the actual download in the background, leaving TheBat! waiting, which makes it time out due to the large number of e-mails.

I know my AV works that way; when I press the check for new e-mail button it will show the retrieving mail window but do nothing while the AV does the actual download in the background and then all of a sudden speed through all the e-mails as the AV feeds them to TheBat! from it's local store.

If it's different in other e-mail clients, it may be because Avast installs a plug-in in TheBat! but not in the others.

[CodeTRUCKER]

One guess would be that your anti-virus is intercepting the mail requests and doing the actual download in the background, leaving TheBat! waiting, which makes it time out due to the large number of e-mails.

I know my AV works that way; when I press the check for new e-mail button it will show the retrieving mail window but do nothing while the AV does the actual download in the background and then all of a sudden speed through all the e-mails as the AV feeds them to TheBat! from it's local store.

If this is true, I am in the soup since I have so many to download.  Ever seen Timeout=10 hours

I did try to move the emails back to the other folder on the server, but somehow TheBAT! keeps finding them?  As a result, I can not hide the bulk to test this with a small store.

If it's different in other e-mail clients, it may be because Avast installs a plug-in in TheBat! but not in the others.

The only plug-in is the AntiSpamSniper.

Thank you for taking a shot.  I appreciate it!
[Edit] -

• Oddly, I just noticed that every process in TB! is being affected.
  Closing the filters window as well as any right-click "create Filter" along with marking messages as spam or good is locking up TB! functionality for 4 - 18 seconds while Avast! is running.  I wonder if Avast! is doing more than just d/l emails?

 

[mouser]

well it's nice to hear you discovered theproblem.  sounds like solution is to uninstall avast?

[f0dder]

well it's nice to hear you discovered theproblem.  sounds like solution is to uninstall avast?

Or at least disable it while doing the initial import of your mails.

Keeping a zillion mails on your server doesn't sound like the best way of doing mail 'backups' by the way, especially not on POP3.

[Steven Avery]

Hi Folks,

I agree. I think the only reason to have server email is if you are super-IMAP-oriented (about which I know very little).  It is so simple to back this stuff up to a USB stick or drive.

We also have threads about using the free MailStore edition for long-term archiving. (Usually Outlook people do that, but I think it may be possible and helpful for TheBat! or Eudora or this or that.)

I recently ran into a similar clogged email thing on a secondary server account that has decent support.  Once it got full it did not want to download.  I think we deleted the mail in this case and started over, since it was simply yahoogroups forum mail of little importance.  However that was a separate issue.  I mention that because it is still possible that the problem partly exists on the server.  

At any rate, keeping lots of mail on the server is something you might want to reconsider. You can get everything fully downloaded and then back up to external USB and/or to the cloud.  Fully empty the server and see if TheBat! still gives you trouble with new mail.  At this time you have two variables, a stuffed server and a quirky Bat, and it would be easier to get it down to one.

Shalom,
Steven Avery

[Innuendo]

CodeTRUCKER, if a) everything was working one day and then not the next; and b) everything goes back to normal when you disable Avast then the problem is most likely that Avast got a signature update that is causing problems.

You have two choices and one is to find a replacement for Avast or another is to try to 'ride things out' and hope that a future signature update will make Avast revert to its normal behavior.

Steven, I think CodeTRUCKER was only storing his mail on the server so he could troubleshoot his problem. A person who uses The Bat! never needs to keep their mail stored on a server like other email programs as The Bat! allows one to have hundreds of thousands of emails in its database with no slowdowns at all.

As a matter of fact, a fellow on The Bat! Beta Mailing List was talking about how he has over 1 million emails & The Bat! is still quick and responsive.

[CodeTRUCKER]

well it's nice to hear you discovered the problem.  sounds like solution is to uninstall avast?

Yes it is nice and I have been investigating a replacement A-V, but as I have been reading I am "hearing" that an A-V may not be necessary.

[CodeTRUCKER]

well it's nice to hear you discovered theproblem.  sounds like solution is to uninstall avast?

Or at least disable it while doing the initial import of your mails.

My thoughts, exactly and it has been successful.  My rate of download has gone from 80/Hr to 270/Hr.

Keeping a zillion mails on your server doesn't sound like the best way of doing mail 'backups' by the way,

Hmmmm... I thought I recall getting the idea a good while ago in lurking some threads on mail store backups. As a result, I have a 2GB backup space that comes with my permanent address for which I pay ~$2.00/mo. It is very short,"xxx@xxx.xx".
Maybe I missed something?

especially not on POP3.

f0dder, Why?  

[Steven Avery]

Hi Folks,

The reasons I would be slow to have any large number of emails on a server include:

1) The server does not have easily the folder intelligence that you create on-the-fly with your mailboxes on your own puter. ie. Post-filtering.

2) Local searching will be much quicker, and super-quicker if indexed. And will normally automatically be integrated into your reply and address-book system.  Even if the mail is on a USB you can work with it very easily.  I know with Eudora you can set up a separate shortcut for the data in another mappable location like a USB.

3) You can run into trouble trouble-shooting the server downloads when the server is gagging-full.  e.g There could be some arcane difficulty hanging the server with email number 5000.  Very hard to troubleshoot.

===============

If you want to have to have email archived, it is best after you have it backed up in the format you most like. These would include.

1) .mbx or .mbox or proprietary or MailStore .dat ... whatever condensed, sensible format you like

2) After everything is filtered

3) After anything you want deleted can be out first, you may want, eg. to eliminate thousands of attachments at once if your email program keeps them external to the text.  Eudora eg. has two easily searchable folders, one for embedded pics, one for attachments. Or you may want to delete a lot of spam and junk mail.  Again, easier to do at home than on the server.

Incidentally, Eudora is also fast with a gazillion emails, although likely not as techie-fast as TheBat! With Eudora I think we consider it good practice to keep the individual folders below 30K email.

Once you are done, if you want cloud backup just go ahead -- just as you would any other large files.  

Now somebody has the server space free because it is bundled with a domain.  So I could see how you might save a few dollars over other cloud backup.  I think if you search around you can find cheapo just fine, and save the hassles above.  And especially, backup locally.

Shalom,
Steven

[CodeTRUCKER]

Hi Folks,

I agree. I think the only reason to have server email is if you are super-IMAP-oriented (about which I know very little).  It is so simple to back this stuff up to a USB stick or drive.

Yes, keeping two USB sticks might be the ticket.  My total store is only 1.5GB.  It is certainly cheap enough.

We also have threads about using the free MailStore edition for long-term archiving. (Usually Outlook people do that, but I think it may be possible and helpful for TheBat! or Eudora or this or that.)

Like I mentioned to f0dder, "that" is where the idea came from for my present backup plan!?!

At any rate, keeping lots of mail on the server is something you might want to reconsider. You can get everything fully downloaded and then back up to external USB and/or to the cloud.  Fully empty the server and see if TheBat! still gives you trouble with new mail.  At this time you have two variables, a stuffed server and a quirky Bat, and it would be easier to get it down to one.

Shalom,
Steven Avery

One variable is immensely better to identify culprits than two (or more <at the thought of this, Calvin shrieks hysterically while running from the room>).  
Seriously, the comments on this thread are provoking a re-thinking of my whole backup strategy and not just for email.
Looks like it is going to be a profitable day!

[Edit] - Mr. Avery you were reading my thoughts in your above post!  Thank you for the comments.

[f0dder]

CodeTRUCKER: the MailStore Steven is referring to is a local mail archiving application, not a remote mailbox

Personally I've considered setting up IMAP on my always-on linux server, and either using that for backups of my remote mail accounts, or handling my email hosting locally and backing up the imap accounts on the linuxbox to something else (MailStore seems like a nice thing, but I don't like that it's a proprietary format).

[CodeTRUCKER]

CodeTRUCKER: the MailStore Steven is referring to is a local mail archiving application, not a remote mailbox
...

Thank you for the clarification!  Oh yeah, when you have the time, I would still like to know why it is specifically bad to store a gazillion mails on POP servers?


In regard to the composite advice to clear my POP server and maintain local archives, can someone tell me is it ok to initiate deleting the mails as I download them from this juncture or would it be best to wait until all of them are downloaded? 

I still have 8,000 more to download.  Is it safe to change horses in the middle of this stream?

[SKA]

Dear CT

1.  <I would still like to know why it is specifically bad to store a gazillion mails on POP servers? >

Basic POP3 style is delete after download (unless your Google) . Most ISP's (other than Google) keep maybe 1-2GB for POP3 user storage. many delete old mails after 30 -60-90 days , some do so earlier.   Best to check your ISP's retention policy for POP3 mails on their server. You maybe suprised to read the "small print".  

Each time you connect via POP3, the ISP 's CPUs "work harder" even tho' your downloaded mails are flagged as such.(I am not sure of this - its what an ISP once told me)

Risk of losing mails = risk of hard disk failure - at an ISP its quite a regular affair. Ofcourse they have backup, but will you trust them to keep gazilion emails for free, year in and out  , unless you've some special deal(pay them for it) ? POP3 access is usually free /included in net access but they dont expect to keep your emails for long (unlike IMAP where they charge you for storage on a monthly basis).

2. <can someone tell me is it ok to initiate deleting the mails as I download them from this juncture or would it be best to wait until all of them are downloaded?  I still have 8,000 more to download.  Is it safe to change horses in the middle of this stream? >

Not good idea at all,
SKA