topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Thursday March 28, 2024, 7:54 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: Export TheBat -> unix MBOX format  (Read 14782 times)

f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 9,153
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Read more about this member.
    • Donate to Member
Export TheBat -> unix MBOX format
« on: December 30, 2007, 05:57 PM »
Hep,

I've been considering giving ThunderBird a go, and see whether it can handle my current mail base efficiently. To do that, I obviously need to export all my emails from The Bat!. TheBat does have built-in support for MBOX export, but I would need to click each folder of each account, Ctrl+A selecting all mails for each folder, and then exporting. Suffice to say, this is tedious.

With a bit of googling I found BackupTheBat, which unfortunately has no website dedicated to it, only mentions at a few blog postings... appearantly made by the Hungerian distributors of TheBat?. Anyway, it seems like it's supposed to do the trick, but on my machine it ends up consuming 100% CPU and doing nothing.

Do any of you guys know a better (working :)) solution, or should I try my hand at writing a TheBat plugin?
- carpe noctem

lanux128

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 6,277
    • View Profile
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #1 on: December 31, 2007, 10:21 AM »
this guy is attempting to do the same. maybe you can get in touch with him.. :)

Thunderbird Import Extension
Goal

I like to have an extension for Thunderbird which is able to import mails from The Bat! 1.x into Thunderbird. Preferably all existing information such as account information, passwords and address books will be migrated, too.

http://www.felix-sch...mport_Extension_(en)


f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 9,153
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Read more about this member.
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #2 on: December 31, 2007, 12:19 PM »
lanux: that guy is doing it in a roundabout way, though - why go through the hard steps of reverse engineering and writing an importer, when you can write an exporter instead? Even an app that sends window messages to TheBat to query the folder structure and then use the built-in mbox exporting would work, and although it would obviously be a hack... well, less work than reversing the file format.
- carpe noctem

lanux128

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 6,277
    • View Profile
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #3 on: December 31, 2007, 09:16 PM »
hmm.. true enough, that sounds like more work.. but how about programs like this, would they cut the mustard?

http://www.processtext.com/abcbat.html
http://www.processtext.com/abctbb.html


f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 9,153
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Read more about this member.
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #4 on: January 01, 2008, 07:21 AM »
hmm.. true enough, that sounds like more work.. but how about programs like this, would they cut the mustard?

http://www.processtext.com/abcbat.html
http://www.processtext.com/abctbb.html
Perhaps - but it doesn't seem that any of them exports to MBOX format. Now, the XML export coudl probably be used and then I could write an XML->MBOX converter, but... ugh, probably easier to just scan the .TBB (mail base) files and ignore the .TBI (index) files :)

I think I'll have a go at either writing a TheBat! plugin for doing this (if the necessary functionality is exported - christ the SDK docs are lousy!), or an external app that simply sends windows messages to TheBat to automate the process...

Unless you can find something better, seems like my googling skills suck :(
- carpe noctem

lanux128

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 6,277
    • View Profile
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #5 on: January 01, 2008, 07:34 AM »
with all things considered, maybe it's time you started on that TheBat! plugin. it will save you a lot of agony and earn much heart-felt appreciation from the user community. even if the docs are atrocious, i think mouser is an avid user too, perhaps you can bounce ideas off him. :)

f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 9,153
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Read more about this member.
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #6 on: January 01, 2008, 08:38 AM »
with all things considered, maybe it's time you started on that TheBat! plugin. it will save you a lot of agony and earn much heart-felt appreciation from the user community. even if the docs are atrocious, i think mouser is an avid user too, perhaps you can bounce ideas off him. :)
Or Jibz, if he's not too much missing in action (I could always hop on my bicycle and find him, I know the general area he lives in :)), he's written the Piper plugin that's on the official TheBat pages...
- carpe noctem

f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 9,153
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Read more about this member.
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #7 on: September 06, 2009, 10:08 PM »
*bump*

Suffering from yet another night of insomnia, I decided to download the last version of TheBat that my 3.x license entails me to, upgrading it from 3.99.24 (which I've been running for over two years) to 4.0.38. This was kinda the last straw for me; why the profanity have they added profanity skinning to the app? Even setting "windows default" skin doesn't turn all the profanity off. There's surely enough bugs and feature requests those profanity could have worked on instead.

I looked around the net once again for some easy exporting, but it seems there hasn't been any progress (lanux's link to the reveng link hadn't been updated anytime recently), there seemed to be no other offerings, and the plugin API hasn't been updated since October 1st 2005 - and it's pretty profanity horrible ([...]considered as self-explanatory, one paragraph is in russian, ...) - furthermore, the SDK is profanity limited in scope, and basically seems only to support antivirus, antispam and custom macros. Seeing that the ClassName for the account treeview is TXTreeView and how abysmal the search results are, I dropped the idea of making an export tool that worked within TheBat.

The .TBB and .TBI files TheBat stores your folders in looks like something that could be reverse-engineered without that much hassle, but honestly I can't be bothered. I did the manual labor of selecting a folder, hitting ctrl+a, tools->export->unix mbox, hitting in filename, enter... across 5 accounts and 42 folders. No, profanity TheBat won't automatically export all messages in a folder, you have to select them; and it certainly won't profanity export a full folder structure. Woe unto he who has more mailboxes than I do.

But now my data is finally out of the clutches of profanity TheBat's proprietary hands, and in a well-defined well-supported format. Where to go now?

I'm definitely setting up ThunderBird for my active accounts, and I will be running those in IMAP mode. TB isn't perfect, but at least it uses mbox files; while those are by no means perfect, at least they're easy to port to another mail app.

What will I be missing from TheBat?
  • MicroEd - I really dig that editor, in spite of lack of proper hyphenation.
  • Virtual Folders (but those were too limited anyway).

What won't I be missing from profanity TheBat?
  • Bugs and quirks - 'nuff said.
  • RITLabs - 'nuff said.
  • The fragility of it's message store. While I haven't had a full-scale meltdown, weird things have certainly happened, and I never felt confident that I wouldn't lose data in a crash.
  • It's clumsy and overcluttered user interface.

I guess I never used any of TheBat's "really powerful" features, otherwise waving goodbye to it might have been harder. And I'll probably get pissed at ThunderBird once my mailboxes grow big... but until then, I have a feeling of relief.

Not sure whether to import my old mailboxes to ThunderBird - I certainly need access to them, but I've been playing around with MailStore Home. It's a bit tedious importing the various MBOX files, especially since the home version limits you to three sync sources (I only need to sync each file once, but that still means I need to delete the sync entries once imported); fortunately, deleting a sync source doesn't delete the stuff imported from it. Also, the way it names entries in it's tree structure is a bit inflexible, and doesn't have rename/restructure support (not that much of an issue, but still a minor annoyance). Not a perfect app, but it'll probably suffice for now - and it's pretty fast. I might even be able to import all my old mails back from the PMMail 2000 days, which would be nice.

Anyway, it was nice to let out some steam. Now I'm going to RAR up the old TheBat data folders, stuff it in the backup folder on my fileserver, and uninstall the application. RIP, The Bat! - 12th January 2005 - 7th September 2009. Then, off to install & configure ThunderBird.
- carpe noctem

Jibz

  • Developer
  • Joined in 2005
  • ***
  • Posts: 1,187
    • View Profile
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #8 on: September 07, 2009, 02:19 AM »
Apparently something called Aid4Mail should be able to convert it, but paying $25 just to switch seems a bit crazy.

What you could also try is to use the /EXPORT command line parameter. The Bat! stores it's mail in a folder structure that (mostly) reflects the actual structure you see, so a perl script that iterates over the folders and emits a /EXPORT call for each to unix mbox format might work.

f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 9,153
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Read more about this member.
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #9 on: September 07, 2009, 02:24 AM »
Jibz: /EXPORT might've be something to look into - is it documented? I didn't even bother to look in the help file, given that it generally sucks :)
- carpe noctem

Jibz

  • Developer
  • Joined in 2005
  • ***
  • Posts: 1,187
    • View Profile
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #10 on: September 07, 2009, 09:29 AM »
It's documented well enough to make it work at least .. I tried it earlier and was a bit surprised I needed to install a plugin to be able to import the Unix mail box files into Thunderbird.

mwang

  • Supporting Member
  • Joined in 2007
  • **
  • default avatar
  • Posts: 205
    • View Profile
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #11 on: September 08, 2009, 08:07 PM »
I remember setting up a keyboard macro to export one folder at a time, assigned it to a hotkey (say, F12), and sit there for a few minutes clicking F12. It's way back in 2002.

Does Mailstore Home support unicode now? Tried it a few times but it never indexed Chinese mail properly.

Innuendo

  • Charter Member
  • Joined in 2005
  • ***
  • default avatar
  • Posts: 2,266
    • View Profile
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #12 on: September 09, 2009, 12:14 AM »
I remember when I moved from PMMail 2000 to The Bat! I used Mailbag Assistant to get the emails moved over. If I remember correctly, PMMail just uses a Unix email format so they should move into Thunderbird no problem at all.

I don't think I could move away from The Bat! even if I wanted to...and I don't. My email archive comprises 30,000 email messages going back eons. Nearly every email program I tried choked on such a large number of messages...except The Bat!.

I've never run into any problems with its database, but I'm pretty strict on myself running the Maintenance Centre's functions on a regular basis. Come to think of it, I've never seen another email program with functionality to keep its messages free from errors.

mwang

  • Supporting Member
  • Joined in 2007
  • **
  • default avatar
  • Posts: 205
    • View Profile
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #13 on: September 09, 2009, 01:15 AM »
30,000 doesn't sound like a huge no. (I've 100,000+), and I think any decent email client today should handle that amount of mail reasonably well. I did use and enjoy TheBat for a few years, and was an active participant on their mailing lists. But the promised unicode support never came, which was acceptable when I was in the U.S., not after I came back to Taiwan. I check out their newest version from time to time since I left in 2002, but it's still not materialized.

After TheBat, I moved to Becky for a short while, which I still use occasionally today. Then I took the plunge and built myself an IMAP4 email server on linux using spare hardware, a decision that has paid off handsomely. Nowadays I'm free to try any email client as I please, without being held up by any proprietary databases.

f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 9,153
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Read more about this member.
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #14 on: September 09, 2009, 01:42 AM »
Then I took the plunge and built myself an IMAP4 email server on linux using spare hardware, a decision that has paid off handsomely. Nowadays I'm free to try any email client as I please, without being held up by any proprietary databases.
That's one of the things I've been considering as well - IMAP + Maildir storage is pretty nice, and so much better for backup (rsnapshot :-*) than whatever proprietary format, or the non-proprietary but meh MBOX format.

There would of course be the issue of pulling mails from my mailserver to the LAN mail server (I don't want to expose the LAN server to the internet + set up the necessary DNS items etc. to make the LAN server my primary mail server, at least not to start with).
- carpe noctem

skwire

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 5,286
    • View Profile
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #15 on: September 09, 2009, 01:57 AM »
Then I took the plunge and built myself an IMAP4 email server on linux using spare hardware, a decision that has paid off handsomely. Nowadays I'm free to try any email client as I please, without being held up by any proprietary databases.
Likewise.  I've been running my own IMAP server for nearly eight years now and also love the ability to try out any email client I want.  I've been a Rebecca/Becky user for almost ten years now but I also like Claws-Mail and Mulberry.

Innuendo

  • Charter Member
  • Joined in 2005
  • ***
  • default avatar
  • Posts: 2,266
    • View Profile
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #16 on: September 09, 2009, 11:58 AM »
30,000 doesn't sound like a huge no. (I've 100,000+), and I think any decent email client today should handle that amount of mail reasonably well.

No...30,000 doesn't sound like a huge number and you would think that any decent email client in the year 2009 would handle that amount of mail reasonably well...but they don't.  >:(

Most email clients have database issues if you try to store too many emails (Outlook I'm looking at you!) or store emails in a 1-email-per-file arrangement which gets around the database issues, but really bogs the clients down when you start trying to deal with massive amounts of messages.

Your method is probably the way of the future for those of us who deal with huge email archives, but most people don't keep any emails for any length of time which is probably why email clients today are still having trouble with huge email archives. It's just not a feature that is in high demand.

But enough of the off-topic banter. It's detracting from f0dder's escape from The Bat!.

f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 9,153
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Read more about this member.
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #17 on: September 09, 2009, 05:50 PM »
Innuendo: you're not going offtopic imho, since issues like yours are pretty relevant when picking an email client.

Thunderbird uses MBOX files, which are pretty retarded (but at least it keeps index files... kinda funny when you have a crash and the index file is corrupted, until you figure out that they need rebuilding). Iirc at least it creates a new MBOX files when the previous one grows too big... but it's still not optimal.

One-file-per-mail isn't a good option on Windows, at least not without dividing into subfolder; while NTFS isn't a bad filesystem, it doesn't deal well with zillions of small files in one folder (perhaps it's more of an issue with explorer and other apps than NTFS, but the net effect is that you don't want to browse those folders, or delete from explorer :)). I'm not sure what the optimal thing to do, if I was to code a mailstore myself, would be - I'd probably lean towards a binary format split to into multiple files based on size, along with external index files... but there's issues with internal fragmentation and all sorts of other things.
- carpe noctem

Innuendo

  • Charter Member
  • Joined in 2005
  • ***
  • default avatar
  • Posts: 2,266
    • View Profile
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #18 on: September 10, 2009, 09:12 AM »
you're not going offtopic imho, since issues like yours are pretty relevant when picking an email client.

Just wanted to be sure. Some people don't take kindly to potential topic derails to their threads.

Thunderbird uses MBOX files, which are pretty retarded (but at least it keeps index files... kinda funny when you have a crash and the index file is corrupted, until you figure out that they need rebuilding). Iirc at least it creates a new MBOX files when the previous one grows too big... but it's still not optimal.

I keep wanting to like Thunderbird. Coming from The Bat!, though, Thunderbird has plain-jane looks with plain-jane functionality. I thought I'd be able to snazz it up with some extensions, but there just are not that many available.

One option that may look interesting is the latest Eudora beta. They've gone open source & are building the Eudora functionality and interface into a Thunderbird codebase.

f0dder

  • Charter Honorary Member
  • Joined in 2005
  • ***
  • Posts: 9,153
  • [Well, THAT escalated quickly!]
    • View Profile
    • f0dder's place
    • Read more about this member.
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #19 on: September 10, 2009, 11:01 AM »
I really like Thunderbird's pretty simple user interface, as opposed to TheBat's option-cluttered skinned mess :) - haven't really missed anything except for the mail-ticker so far.

Last time I used Eudora was back pre-2000 I think... didn't like it too much back then.
- carpe noctem

mwang

  • Supporting Member
  • Joined in 2007
  • **
  • default avatar
  • Posts: 205
    • View Profile
    • Donate to Member
Re: Export TheBat -> unix MBOX format
« Reply #20 on: September 10, 2009, 05:26 PM »
I also prefer Thunderbird's simpler UI. The only thing I really miss about TheBat is the "read filters", filters that sort mail after it's read. Well, I liked MicroEd as well, but it never worked with Chinese text so it's pretty much moot for me.

Another new mailer to try, if you don't mind it's non-free, is Postbox. Its development team is lead by Scott MacGregor, the creator and for the most time before the Thunderbird reorganization the lone developer of Thunderbird, before he left abruptly. It's based on the Thunderbird codebase, but somehow it's shareware. I assume they've cleared the licensing issues. I haven't tried it myself, though.