Author Topic: How do you archive your Email or MBox2CHM? (Read 30461 times)

crono · « **on:** December 11, 2007, 09:20 AM »

Hi,

it's already December and I would like to archive my old Emails at the end of the year. I got about 4.5 GB of emails from the last 10 years - which make Thunderbird really slow. I would like too keep all the mail from 2007 in the client and archive older mails in some (still accessible) way. The problem is that Thunderbird keeps all mails in the MBox format what basically means one large file per folder.

I thought about these options:
1) Create a Filter to get all older Emails in a separate folder. This Folder could be compressed and backup'ed right away. MBox is plaintext so it is some way of search able but not really good readable and 100MB+ textfiles are slow when opened in an texteditor.

2) Just like 1) but transform the the MBox into Maildir - which stores mails as individual files. This solves the search and open issue but makes it really hard to find replies and followup's

Both "solutions" lacking the ability to handle attachments

My perfect solution would be as follows:
Create one CHM-File (Help File) per Folder per Year and keep the "reply-structure". CHM provides search, Index and HTML View - which would be simply great. I haven't found a tool to convert the MBox to CHM (only an Outlook Plugin). Does anyone know such a tool or another good solution? If no tool is found, I will go to code one by myself...

lanux128 · « **Reply #1 on:** December 11, 2007, 08:59 PM »

this program - ABC Amber (30 days trial; US $19.95 per license) claims to be able convert all sort of file-types.. maybe you can see if this suits you.

• ABC Amber Thunderbird Converter

crono · « **Reply #2 on:** December 12, 2007, 02:56 AM »

Cool - nice find

- I'll give it a try

rjbull · « **Reply #3 on:** December 12, 2007, 04:09 AM »

Aid4Mail and/or MailbagAssistant by Fookes Software?

doublewitt · « **Reply #4 on:** December 12, 2007, 04:42 AM »

How about this FREE application called MAILSTORE?
It can archive all emails from Thunderbird.
click here

Carol Haynes · « **Reply #5 on:** December 12, 2007, 06:55 AM »

Just downloaded MailStore Home. It looks really useful for bringing together an entire email archive.

The great thing is it recognises duplicates (both emails and attachments) and doesn't copy multiple copies to the archive.

Note it works with lots of email apps:

urlwolf · « **Reply #6 on:** December 12, 2007, 08:05 AM »

I'm very dissatisfied with all the crimes that Opera M2 has committed on my mail. I'd love to get a good solution; even though I think M2 is the greatest interface to handle mail, it still lost my mail. The 9.5 beta has lost indexing, and I'm not sure how to get it back to usable.

Be warned: M2 on Opera 9.5 considered harmful. And barely supported (not a single bug corrected, not a single feature added in years before 9.5; now they simply destroyed it).

crono · « **Reply #7 on:** December 12, 2007, 05:19 PM »

Thanks for the suggestions.

I tested ABC Amber and it came pretty close to that what I want (this company is really into the convert-thing

). The Demo allows only to export 10 Messages to CHM. The CHM itself was "OK", but a few things are bugging me:
1) You can only choose between TXT and HTML - not keeping both
2) It loses most of the Mail-header
After all - a good product for many users but a bit to limited for my wants...

Aid4Mail could not produce CHM files. The thing which comes closest are HTML-Archives of the Mails. The result looks like kind of Outlook Web Access with one file per mail. The big big problem with this is, its only HTML - no good way searching it.

MailStore looks really good (but has some broken Dialog Windows - strange for an DotNet App) - very stylish Interface

. I downloaded the personal edition and imported some folders. The search is really fast (they are using (Firebird) Database in the background). I will try it with all my mails - on the upcoming weekend. Really nice software

Carol Haynes · « **Reply #8 on:** December 12, 2007, 06:15 PM »

MailStore looks really good (but has some broken Dialog Windows - strange for an DotNet App)
-crono (December 12, 2007, 05:19 PM)

I haven't played extensively - where did you find broken dialogs?

The only 'bugs' I noticed were that if you request to see an email in Outlook Express it is hit and miss whether the window actually appears (seems more consistent in Thunderbird - though web based graphics seem to be an issue - and OK in Outlook 2003.

The other thing is if you right click on a folder it gives a single item menu in German (in the English version) which should be "Refresh F5" I think. It is a German app so I presume they simply missed that one translation.

crono · « **Reply #9 on:** December 12, 2007, 06:40 PM »

Every form where I can choose a folder or a file are broken on my machine - maybe it's just Vista. I think its because of their "theming"-component. I just wanted to show a screenshot - but the GUI Bug (buttons are only outlined with no visible content) didn't appear on the Screenshot - I didn't know SC is a Debugger

- but ok - nevertheless a really good product. Btw. I'm using the German translation - no problem on that...

Carol Haynes · « **Reply #10 on:** December 12, 2007, 07:22 PM »

That's strange 'cos it seems to work fine for me on Windows XP.

The website just says it requires Windows 2000 or later - it isn't specific about Vista.

doublewitt · « **Reply #11 on:** December 12, 2007, 07:28 PM »

Glad to see that you like MailStore. Maybe you can contact them about these little issues... and see how they respond... and discover what kind of service they employ...

Before using MailStore, for personal use, I was using Junkanoo, which I like also (freeware). It reads your mail from the server and copies it to an archive. It does this while applying a spam filter that conveniantly captures the emails marked as spam when also let's say, they shouldn't be, and keeps them for at least one day, so you can "catch it" and change it's status to good mail for the next mailbox scan. You can scan several accounts along with gMAIL accounts and delete emails from the server and avoid unnecessary downloads. Nice little tool.

Click here

I don't have those *problems with MailStore either... must be a Vista issue...

doublewitt · « **Reply #12 on:** December 14, 2007, 10:50 PM »

I really enjoy using this software!
[url=http://www.mailstore.com/en/default.aspx[/url]MailStore[/b] also has some nice and useful administrative tools:

http://img267.imageshack.us/img267/9951/20071214233733kf3.jpg

How do you archive your Email or MBox2CHM?

I think this is the best email archiver I've ever seen and it's so easy to setup.
In a few minutes - presto - it's all done.
give it a try...

J-Mac · « **Reply #13 on:** December 14, 2007, 11:30 PM »

I've been using MailBag Assistant by Fookes Software for years. Probably the most versatile email archiver I've seen. Here's a list of features from the web site:

* Easy to use thanks to helpful wizards.
* Handles the most popular message files.
* No need to import e-mails to use them in Mailbag Assistant.
* Handles native message files even if you no longer have the original e-mail program on your computer.
* Enables you to easily share your correspondence with other Mailbag Assistant users.
* Favorites menu to quickly reopen frequently used e-mail folders.
* Message text is colored based on quote level.
* Match tool selects all messages matching your criteria.
* Filter tool shows only messages matching your criteria.
* Search tools support Boolean operators, approximate matches, and Soundex matches.
* Groups messages together meaningfully.
* Sorts grid data when column header is clicked.
* Saves e-mails into compressed message archives.
* Stores and extracts e-mail attachments and HTML messages.
* Exports e-mails as plain text, EML files, Web pages, and other formats based on customizable templates.
* Extracts information like e-mail addresses and traffic statistics.
* Provides built-in scripting language to automate tasks.
* Integrated editor for scripts and templates, with syntax highlighting.

I will say that it is fast and offers searching in just about any way you could want. You can tell it to find messages that match a certain set of parameters, extract all email addresses from those messages and then view them, print the list, store it in a file. It carries the attachments with the messages - in the archive file. I have been archiving my email messages with it in their compressed file format and also exporting all messages to EML files and zipping them up, then writing them to CD's - and now to DVD's. (Mailbag Assistant doesn't burn anything to optical drives - I use Nero for that). I need to make sure I have all email messages saved - can't afford to have a new version of my email program lose any past files, and restoring some email message stores from a regular compressed backup doesn't always work.

This is not a free program, though. $39.95, so you really have to need it, IMO.

Fookes also sells Aid4Mail, a utility that will take all your email from most any email client and convert it to almost any other format. When I decided to stop using Pocomail and switch back to Outlook the whole process took about 15 minutes for > 15,000 messages. No preparatory work is needed. Then I switched to Thunderbird 2 from Outlook and moved them all again. Best part is it will maintain the same folder structure for you, which was a lifesaver for me. This application has four versions: Standard, Professional, Forensic, and Enterprise. If you will be converting to or from Outlook, though, you need the Professional version. Apparently there are not many other programs - if any - that will convert to and from Outlook because the PST file structure is so different from other mail apps.

Jim

kronhead · « **Reply #14 on:** December 25, 2007, 09:54 AM »

I am coming in a little late here, but I have to ask why you all don't just use Thunderbird's mailbox folders, or local folders? I move messages into folders by year. One advantage is that my desktop search tool (copernic) indexes and searches these, along with active email messages. I realize I am probably wasting space - but disk is cheap, and my time is what I want to save. One tool to search everything means a lot to me.

I do wish Thunderbird had a way of deleting attachments - just large ones that REALLY waste space. I used to use an extension for this, but I am not sure it still works.

Just my 2 cents ...

Dan

twinkler · « **Reply #15 on:** December 25, 2007, 11:37 AM »

I do wish Thunderbird had a way of deleting attachments
-kronhead (December 25, 2007, 09:54 AM)

"Beginning with version 1.5, Thunderbird allows you to detach or delete attachments from messages."
http://kb.mozillazin...hments_(Thunderbird)

J-Mac · « **Reply #16 on:** December 25, 2007, 12:14 PM »

I am coming in a little late here, but I have to ask why you all don't just use Thunderbird's mailbox folders, or local folders? I move messages into folders by year. One advantage is that my desktop search tool (copernic) indexes and searches these, along with active email messages. I realize I am probably wasting space - but disk is cheap, and my time is what I want to save. One tool to search everything means a lot to me.

I do wish Thunderbird had a way of deleting attachments - just large ones that REALLY waste space. I used to use an extension for this, but I am not sure it still works.

Just my 2 cents ...

Dan
-kronhead (December 25, 2007, 09:54 AM)

I don't see how placing messages into a Thunderbird mailbox folder archives them. Unless you place them into a folder that is in a separate Profile; a Profile that you don't use to retrieve or work with your emails. One created just for storage. (Come to think of it, that's not a bad solution on the face of it!)

crono's (OP) initial post here asks how to archive the messages because Thunderbird is slowing down due to the size of his mail store. If you just move them into a separate folder I would expect T-Bird to be just as slow as if you left them in your working folders.

Another altogether different thought is to store all of your messages in a Google Group. I read about using a Google Group to keep a backup of your Gmail messages at Gina Trapani's LifeHacker - Geek to Live, but I don't see why that wouldn't also work for Thunderbird - or any other - email client.

Basically you create a Google Group, set the access to "Restricted", make yourself the only member of the Group, and then set Thunderbird as the "non-Google notification email system". Then use filters to auto-forward or bounce a copy of every message that comes into T-Bird to the Google Group. You could also forward all of your existing messages to the Group. (That might take a while, but it would make it a really complete backup; a fully active backup that grows with each forwarded message). You will have your own private Google Group that contains all of your Thunderbird email messages - a nice online backup! If you are concerned that someone could still somehow hack into the Group and see your emails, encrypt them first.

The original article on LifeHacker where I saw the Gmail backup-to-Google-Group concept is here, and the actual instructions are here.

Jim

kronhead · « **Reply #17 on:** December 25, 2007, 02:11 PM »

I don't see how placing messages into a Thunderbird mailbox folder archives them. Unless you place them into a folder that is in a separate Profile; a Profile that you don't use to retrieve or work with your emails. One created just for storage. (Come to think of it, that's not a bad solution on the face of it!)

crono's (OP) initial post here asks how to archive the messages because Thunderbird is slowing down due to the size of his mail store. If you just move them into a separate folder I would expect T-Bird to be just as slow as if you left them in your working folders.

-J-Mac (December 25, 2007, 12:14 PM)

I'll agree that moving them around in Thunderbird is not a true archive - but I guess I had not experienced, or was not aware of, Tbird slowing down as the total # of messages it manages increases. Is it true beyond startup, and perhaps shutdown? (I probably only restart Tbird once or twice a week - and I find Firefox startup much slower and more frustrating.)

I probably have something less than 10,000 messages total in Tbird in different accounts (inbox, sent, and various "archive" folders) and another 5,000 in a variety of local folders. I should add that I recently switched to IMAP for active accounts, so I can access mail from 2 different laptops. I don't really like Web access. I still have the POP accounts defined, just for storage. Now, IMAP is certainly slower (obviously).

Thanks for the response.
Dan

J-Mac · « **Reply #18 on:** December 25, 2007, 02:48 PM »

Hi Dan. I have about 15,000 messages in T-Bird, and it's not slow.

But I didn't claim it was; crono did at the top of the page. Maybe he/she has a lot more?

Jim

f0dder · « **Reply #19 on:** December 25, 2007, 04:29 PM »

Humm, I'm considering moving my mails to an archive myself, and there's a couple of points I find very important:

Either a documented storage format, or a way to output to MBOX. I don't want to be locked-in with a proprietary format. The app itself can be as proprietary and closed-source as it wants to, though.
Has to have really good searching features, as that's obviously something you'll need for an email archive.
Easy import from the major players and formats (at least MBOX (thunderbird etc.) and outlook/express.
One Big File or at least just a few of them.

I'll look into the various offerings sometime soon, I guess... I really want to get the bulk of my mails moved to archive, and then start using IMAP instead of POP3. My host supports it, but I'm not going to trust anybody else to archive my mails.

crono · « **Reply #20 on:** December 25, 2007, 07:37 PM »

Hi all,

I am coming in a little late here, but I have to ask why you all don't just use Thunderbird's mailbox folders,
-kronhead (December 25, 2007, 09:54 AM)

Because:
1) It is one large File per folder (I lost some folder because of a silly virusscanner which deleted the whole file because of one infected mail). If Thunderbird would use QMail's "Maildir"-Format, I would consider using it for archiving.
2) Cant be searched via Windows (Vista) Search
3) No really good way to keep hierarchies in message-threads (I would like to get a Mail and all follow up Mails without reimporting it to Thunderbird)

I realize I am probably wasting space - but disk is cheap, and my time is what I want to save.
-kronhead (December 25, 2007, 09:54 AM)

Space is no issue to me too

Another altogether different thought is to store all of your messages in a Google Group.
-J-Mac (December 25, 2007, 02:48 PM)

I know this article. It would be OK for new Mails, but I don't consider forwarding the the Mails to Google an option. A lot of this Mails must be encrypted before forwarding it because of some sensible client data - which destroys the searchability completely. I don't like the Idea of storing 10 Years of personal (online) history on Google-Servers (call me paranoid)...

But I didn't claim it was; crono did at the top of the page. Maybe he/she has a lot more?
-J-Mac (December 25, 2007, 02:48 PM)

I have a whole lot more the 15000 Messages in my current Thunderbird. Something around 2.3 Million

(I'm on a lot of mailinglists and subscribed to some high traffic newsgroups) - Thunderbird uses ~550 MB RAM on idle. I'm totally certain TB wasn't designed for such a load...

Either a documented storage format, or a way to output to MBOX. I don't want to be locked-in with a proprietary format. The app itself can be as proprietary and closed-source as it wants to, though.
Has to have really good searching features, as that's obviously something you'll need for an email archive.
Easy import from the major players and formats (at least MBOX (thunderbird etc.) and outlook/express.
One Big File or at least just a few of them.
-f0dder (December 25, 2007, 04:29 PM)

I'm testing MailStore (see above) and it is really nice. The search is OK, it could import from a bunch of Clients (export also). They use a standard Firebird SQL Database (fka Interbase) for storage. - So it is one big file and you can open the DB and log on to it with opensource tools like "flamerobin" and a Firebird standard user (sysdba / masterkey).

thanks for the input
:-)

f0dder · « **Reply #21 on:** December 26, 2007, 08:02 AM »

Humm, I don't really think Maildir is a good storage format for client PCs. It's nice for servers with a small-file optimized filesystem (ReiserFS comes to mind), and it's very easy to do efficient incremental backups of it. But for archive+search usage, I'd prefer bigger bulk files.

Sounds nice that MailStore uses FireBird, at least you're not going to suffer from total proprietary lockin. I wonder why they use FireBird and not SQLite though, since FB (afaik) is a full-blown client/server database, whereas SQLite was written for being embedded...

crono · « **Reply #22 on:** December 26, 2007, 08:49 AM »

Humm, I don't really think Maildir is a good storage format for client PCs. It's nice for servers with a small-file optimized filesystem (ReiserFS comes to mind), and it's very easy to do efficient incremental backups of it. But for archive+search usage, I'd prefer bigger bulk files.
-f0dder (December 26, 2007, 08:02 AM)

For searching I could imaging putting a Lucene Search on top of the MailDir Files. But this is all theoretical because I'm satisfied with Mailstore right now

Sounds nice that MailStore uses FireBird, at least you're not going to suffer from total proprietary lockin. I wonder why they use FireBird and not SQLite though, since FB (afaik) is a full-blown client/server database, whereas SQLite was written for being embedded...
-f0dder (December 26, 2007, 08:02 AM)

There is an embedded version of firebird which requires no server on the client machine. Maybe they use Firebird because of the native .Net Data provider (Mail Store is a .NET app) for Firebird. SQLite only offers C / C++ and Tcl bindings as far as I know.

tomos · « **Reply #23 on:** December 28, 2007, 03:04 PM »

installed MailStore and backedup & deleted all those newsletters I'll probably never look at again
they werent in the way but somehow it's very nice to clear them all out

good feng shui too, I'm sure

tomos · « **Reply #24 on:** December 28, 2007, 03:09 PM »

I do wish Thunderbird had a way of deleting attachments
-kronhead (December 25, 2007, 09:54 AM)

"Beginning with version 1.5, Thunderbird allows you to detach or delete attachments from messages."
http://kb.mozillazin...hments_(Thunderbird)
-twinkler (December 25, 2007, 11:37 AM)

there's also an add-on called Attachment extractor
https://addons.mozil...hunderbird/addon/556
which allows you to extract/backup/delete attachments from multiple messages

unfortunately there's doesnt seem to be an easy way of checking which messages have had their attachments already extracted/deleted (apart from opening the mails) unless you use the tagging system (see below)

saying that just realised I have older version installed so .. updates

also this may or may not be relevant anymore about it "stealing" tag on first install:
Question:

> my query is if I can stop Attachment Extracter from making it\'s own Tag - it defaults back to Tag#6 each time I open tbird.
> I have been using the 1 to 9 shortcuts for tags for a good while now & keep tagging things with the wrong tag cause 6 to 9 are now displaced.
> I dont have \"automatically extract\" selected in options & can find no other reference to tags.
>
> I tried deleting the tag & recreating it at the end of the list but that didnt help on next restart - back to #6!
>
> OS is XP, up-to-date & Ver.0.9.6 of AE

Answer:

yeah you just need to change the 'extract messages if its tag matches' preference to something other than AE AutoExtract - it will list all your tags there - you can then delete it.