Moving from MicroDot II to Maildir format

Introduction

The Amiga email and news program MicroDot II uses its own proprietary format to store messages, thus moving to an other email or news program is not as easy as it could be, requiring manual conversion of the message base.

This document describes moving the email part of the message base to the Maildir format. It is a standard format like the usual mbox, but has technical and performance advantages over the latter.
It is supported by MUAs like Mutt, MTAs like Exim, Postfix and qmail, MDAs like maildrop and procmail and IMAP4/POP3 servers like Dovecot and Courier to just name a few.

The conversion process

While the actual conversion will happen on your Unix box, you of course have to start on the Amiga side to acquire a copy of MicroDot II's message base.

Amiga

To avoid that deleted messages reappear, launch MicroDot II and select Empty TRASH and then Cleanup database from the MicroDot menu.
This will take a while, once its done quit MicroDot II.

Open a Shell and CD into the directory where MicroDot II's Data directory resides, in most cases MicroDot II's directory. Create an archive of MicroDot II's Data directory using LhA:
lha -r -z a WORK:md2.lha Data
This assumes WORK: being a disk with enough free space where you want to store the archive.

Transfer md2.lha to some working directory on your Unix box, ie. via FTP.

Unix

Extract the archive using LhA for Unix, which should be available in your Linux Distribution or BSD flavour of choice:
lha x md2.lha
cd Data

As you only need the messages themselves which are stored in the DBX_* directories, remove all other files:
rm -f *.ix *.IX Grouplist.* UIDL.history sendtmp.*

To separate emails and news, move all messages containing Newsgroups: headers to a temporary directory:
mkdir tmp
grep -rilZ '^Newsgroups:' DBX_* | xargs -r -0 -i{} mv {} tmp

Note: This of course does include news forwarded by email and mails forwarded by news!

To recover those, move all messages containing Received: headers to another temporary directory:
mkdir DBX_tmp
grep -rilZ '^Received:' tmp | xargs -r -0 -i{} mv {} DBX_tmp

You now have to manually check all files in DBX_tmp and delete any news articles containing forwarded emails while preserving any emails containing forwarded news articles.
There should only be a few files in DBX_tmp however, e.g. I had about 10 out of 31000 in total, so this step should not be much work.

Remove the news articles:
rm -rf tmp

The DBX_* directories now only contain email messages and you are ready to convert them to the Maildir format using any MDA supporting it.

Assuming you use maildrop, create a domaildrop script somewhere in your PATH with the following content:

#!/bin/sh

cat $1 | maildrop
      

You should already have setup your system to use Maildir and have working mail filter rules set up, ie. in this case in your $HOME/.mailfilter file.
It is very important that the mail filter does work, otherwise your mails will end up in the wrong folders or even get lost. If you have any rules in your filter that check for SPAM, temporarily disable them both for performance reasons and to avoid potential false positives in your old mail.
Remember that mail from old mailing lists you are not subscribed anymore or which changed some header you filter on also require rules matching them.

Now deliver your mails into your Maildir folders:
find DBX_* -type f -exec domaildrop \{\} \;

Your mails are now sorted into your Maildir folders, however the most tedious work begins now.
As all the mails were just delivered, they are marked new and unread, thus you have to start your MUA of choice and manually set any mail flags as desired and eventually move any mail that got sorted into the wrong folder to the correct one.

Once you are done with that, you have successfully moved from MicroDot II to an open standard mail storage format accessible by a wide range of MUAs and easily convertible to other standard mail storage formats without requiring any manual work.

License

This document is free documentation; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

Links

Mutt & Maildir Mini-HOWTO