Typically, an "email server" consists of two pieces of software: an SMTP server and an IMAP server. The SMTP server is what takes care of sending mail and saving mail sent to you. The IMAP server is used to access and organize the mail.
If I organize mail, I typically do it on my local machine. I download email from Gmail over IMAP using isync, and look at it in mutt. Locally, its stored in the Maildir format, essentially meaning one file per piece of mail.
Recently, I set up my own email server. As I was setting it up, I realized that my mail is stored in the Maildir format both on the server (by OpenSMTPD) and locally (by isync). Since all I use the mail server for is receiving and sending email, I figured I might be able to get away with not even running an IMAP server. I wrote a little script which just copies the new mail to my local computer over SSH, and makes sure everything is where Maildir says it should be:
#!/bin/sh
usage () {
echo "$0 host remotedir localdir"
}
[ "$#" -ne 3 ] && (usage ; exit 1)
# tar, compress and retrieve new mail
ssh $1 "cd $2/new ; tar -cf /tmp/mail.tar.gz *" || exit
scp "$1:/tmp/mail.tar.gz" "/tmp/mail.tar.gz" || exit
# put new mail where it's supposed to go
mkdir -p "$3/INBOX/cur"
tar -xf "/tmp/mail.tar.gz" -C "$3/INBOX/cur" || exit
# cleanup
ssh $1 "rm /tmp/mail.tar.gz ; mv $2/new/* $2/cur/"
rm /tmp/mail.tar.gz
Not only is this simpler than IMAP, but it also saves my VPS resources and makes it faster to get my mail. Also SSH is arguably more secure than IMAP as there isn't even an option to have the protocol send unencrypted data.