Building an Advanced Mail Server

edwardhayes

Building an Advanced Mail Server
by Joe Stump
09/25/2003
All Linux distributions that I know of come with an MTA of some sort. The most popular is Sendmail. Other popular MTAs include Exim, postfix, and Qmail. This article discusse how to build an advanced mail server which sports all of the latest mail protocols, checks all incoming mail for spam, and scans all incoming and outgoing mail for viruses.


We will use the able Qmail MTA for SMTP and POP3. We will use vpopmail for virtual domains and Courier IMAP for our IMAP server. As our backend we will be using the trusty MySQL RDBMS to store all of our user information. Since this is a three part series, we will cover Squirrel Mail in the second part and SpamAssassin, procmail, QmailScanner, and ClamAV in the third part.

You are assumed to have a working knowledge of Linux, though the steps outlined should work on most UNIX variants with little effort. It is also assumed that you understand how email, in general, works. Finally, it is assumed that you have root access to the machine you wish to turn into an advanced mail server. Debian users are in luck; since this is the distribution I personally run I will include Debian-specific shortcuts whenever possible. If you're using another distribution, you may have to modify the provided installation notes slightly.

MySQL
Installing MySQL
All of the distributions that I have worked with either supplied MySQL binaries on the CD or made them available somewhere on the web. For detailed instructions on how to set up MySQL please read MySQL's Installation instructions.

Note: Debian users can apt-get the packages mysql-client and mysql-server.

Creating the Database User
Log into your MySQL server as root and type the following commands to create the database for vpopmail.

mysql> CREATE DATBASE vpopmail;
mysql> GRANT ALL ON vpopmail.*
    -> TO vpopmail@localhost IDENTIFIED BY 'password';
You can change the names of the database, user, and password, but make sure to keep track of them for later use. You also may wish to change the security preferences for your vpopmail user, but before you do read up on the MySQL GRANT statement.

OpenSSL
This step is optional. If you wish to run your webmail via a secure connection or enable IMAP-SSL or POP3-SSL, you will need to install OpenSSL. Your distribution should come with OpenSSL packages. Be sure to install the development versions of those packages so that we can compile Courier and Qmail from source.

Note: Debian users can apt-get the package openssl.

Qmail
Qmail was written by D. J. Bernstein (DJB) as a replacement for Sendmail. Qmail differs greatly from Sendmail so before you jump headlong into installing Qmail, you should download the source and read through the documentation. Furthermore, it may be in your best interest to read Life with qmail by Dave Sill.

Installing ucspi-tcp
ucspi-tcp is a simple TCP Server/Client created by DJB for "building TCP client-server applications" and is required to run Qmail. After you have downloaded and extracted the source, change into the directory and compile the code.

bash$ make
bash$ make setup check
You can read over DJB's installation instructions on his How to install ucspi-tcp page. Also, be sure to turn off any affected ports (25,110,143) in /etc/inetd.conf and restart your inetd server.

Note: Debian users can apt-get the package ucspi-tcp-src.

Installing daemontools
daemontools "is a collection of tools for managing UNIX services." This is the preferred way to run Qmail, but is not required to run Qmail. To install it, first make the /package directory it expects.

bash$ mkdir -p /package
bash$ chmod 1755 /package
bash$ cd /package
Download the daemontools package into the /package directory and untar it. Next, run the installer:

bash$ cd admin/daemontools-0.76
bash$ package/install
You can read over DJB's installation instructions on his How to install daemontools page.

Note: Debian users can apt-get daemontools-installer. Debian users may also wish to check out qmail-pop3-sv, qmail-smtp-sv and qmail-sv as well.

Patching Qmail
Version 1.03 is the latest version of Qmail. The default installation of Qmail is very vanilla and will require some minor patching to do what we wish to do. Particularily we will want to install:

Bill Guenter's famous Qmail QUEUE patch, which will allow us to run alternate programs as Qmail's queue.

Christopher K. Davis's oversized DNS packets patch, which enables Qmail to accept oversized DNS packets. (DJB recommends running djbdns as an alternate fix to this problem.)

Bill Shupp's maildir++ patch, which will fix vpopmail and Qmail quota issues.

Eric M. Johnston's SMTP-AUTH patch, which allows users outside of your network to send email after they have authenticated.

Installing Qmail
The installation process for Qmail is very hands on and requires you to be familiar with the command prompt. Before installing, please read INSTALL, INSTALL.ids, INSTALL.ctl and INSTALL.alias. You may wish to look over the other INSTALL.* files as well.

bash$ mkdir /var/qmail
bash$ groupadd nofiles
bash$ useradd -g nofiles -d /var/qmail/alias alias
bash$ useradd -g nofiles -d /var/qmail qmaild
bash$ useradd -g nofiles -d /var/qmail qmaill
bash$ useradd -g nofiles -d /var/qmail qmailp
bash$ groupadd qmail
bash$ useradd -g qmail -d /var/qmail qmailq
bash$ useradd -g qmail -d /var/qmail qmailr
bash$ useradd -g qmail -d /var/qmail qmails
bash$ make setup check
bash$ ./config-fast example.com
bash$ (cd ~alias; touch .qmail-postmaster .qmail-mailer-daemon .qmail-root)
bash$ chmod 644 ~alias/.qmail*
Make sure to change example.com to the actual hostname of your mail server.

Note: Debian users can apt-get qmail-src, however, the package does not include the SMTP-AUTH or maildir++ patches. Alternatively, you could try Garrit Pape's Debian packages.

Once you have Qmail installed, make sure it starts up during your boot sequence. There are a several init scripts available on the web, one of which can be found here. The script says it is for Red Hat, but it should work for any distro with a little modification.

bash$ cp qmailctl.txt /etc/init.d/qmail
Before you start Qmail, you need to set up daemontools' Qmail scripts which control how Qmail responds to various requests.

bash$ mkdir -p /var/qmail/supervise/qmail-send/log
bash$ mkdir -p /var/qmail/supervise/qmail-smtpd/log
bash$ mkdir -p /var/qmail/supervise/qmail-pop3d/log
After you are done setting up the directories you will need to create all of the controlling scripts.

/var/qmail/supervise/qmail-send/run
#!/bin/sh
exec env - PATH="/var/qmail/binPATH" qmail-start ./Maildir/
/var/qmail/supervise/qmail-send/log/run
#!/bin/sh
exec /usr/local/bin/setuidgid qmaill /usr/local/bin/multilog t \
        s10000000 n30 /var/log/qmail/send
/var/qmail/supervise/qmail-smtpd/run
#!/bin/sh
QMAILDUID=`id -u qmaild`
NOFILESGID=`id -g qmaild`

exec /usr/local/bin/softlimit -m 2000000 \
/usr/local/bin/tcpserver \
  -H -l hostname.yourdomain.com \
  -v -x /etc/tcp.smtp.cdb \
  -c 20 -R -u "$QMAILDUID" -g "$NOFILESGID" 0 smtp \
  /var/qmail/bin/qmail-smtpd hostname.yourdomain.com \
          /var/lib/vpopmail/bin/vchkpw /bin/true 2>&1
/var/qmail/supervise/qmail-smtpd/log/run

#!/bin/sh
exec /usr/local/bin/setuidgid qmaill /usr/local/bin/multilog t s10000000 \
        n30 /var/log/qmail/smtpd
/var/qmail/supervise/qmail-pop3d/run
#!/bin/sh
exec /usr/local/bin/softlimit -m 3000000 \
/usr/local/bin/tcpserver \
  -H -l example.com \
  -v -x /etc/tcp.pop3.cdb -c 30 -R 0 pop3 \
/var/qmail/bin/qmail-popup example.com \
/var/lib/vpopmail/bin/vchkpw /var/qmail/bin/qmail-pop3d Maildir 2>&1
/var/qmail/supervise/qmail-pop3d/log/run
#!/bin/sh
exec /usr/local/bin/setuidgid qmaill /usr/local/bin/multilog t s10000000 \
        n30 /var/log/qmail/pop3d
After you have taken care of all of the scripts, the last steps are to chmod the scripts, make the log directories and make daemontools aware of the new service.

bash$ chmod 755 /var/qmail/supervise/qmail-send/run
bash$ chmod 755 /var/qmail/supervise/qmail-send/log/run
bash$ chmod 755 /var/qmail/supervise/qmail-smtpd/run
bash$ chmod 755 /var/qmail/supervise/qmail-smtpd/log/run
bash$ chmod 755 /var/qmail/supervise/qmail-pop3d/run
bash$ chmod 755 /var/qmail/supervise/qmail-pop3d/log/run
bash$ mkdir /var/log/qmail
bash$ mkdir /var/log/qmail/smtpd
bash$ mkdir /var/log/qmail/send
bash$ mkdir /var/log/qmail/pop3d
bash$ chown -R qmaill /var/log/qmail
bash$ ln -s /var/qmail/supervise/qmail-send /service
bash$ ln -s /var/qmail/supervise/qmail-smtpd /service
bash$ ln -s /var/qmail/supervise/qmail-pop3d /service
bash$ /etc/init.d/qmail start

edwardhayes

vpopmail
The wonderful folks over at Inter7 developed vpopmail to handle the management of virtual domains using Qmail. Unfortunately vpopmail is limited to a measly 23 million virtual hosts, of which each are limited to only 23 million users. So if you have more than 529 trillion users you may need to look elsewhere.






Creating the vpopmail User and Group
As noted in vpopmail's INSTALL file, "[the] FreeBSD folks have reserved 89 for the group and 89 for the user for vpopmail." You will probably want to ensure vpopmail is running as uid and gid 89.

bash$ groupadd -g 89 vchkpw
bash$ useradd -g vchkpw -u 89 -d /path/to/where/you/want vpopmail
I installed vpopmail into /var/lib/vpopmail, but you can put it wherever you wish. Just remember that all email messages will be stored in /var/lib/vpopmail, so choose a partition with plenty of room.

Setting up MySQL Support
The first thing you need to do is set up vpopmail for MySQL. This is done by doing some light code editing in a header file. Fire up your favorite editor. Open up the file vmysql.h and change the following lines to match the configuration you set up in the MySQL section of this article:

#define MYSQL_UPDATE_SERVER "localhost"
#define MYSQL_UPDATE_USER   "vpopmail"
#define MYSQL_UPDATE_PASSWD "password"

#define MYSQL_READ_SERVER   "localhost"
#define MYSQL_READ_USER     "vpopmail"
#define MYSQL_READ_PASSWD   "password"
Installing vpopmail
After you have finished editing vmysql.h, compile the program. Be sure to run ./configure --help before configuring the software. Below is what I used to enable MySQL support and install vpopmail.

bash$ ./configure \
      --enable-mysql=y
      --enable-auth-logging=n
      --enable-mysql-logging=n
      --enable-logging=e
      --enable-valias=y
      --enable-passwd=n
      --enable-defaultquota=15728640
      --enable-roaming-users=y
There are a few things to note in the above configuration that you may want to change to suit your needs.

I disable --enable-auth-logging because SquirrelMail logs in and out every time a page is loaded, which can make logs grow quickly on active mail servers.

The e option to --enable-logging enables logging only of errors.

--enable-defaultquota The default quota can be computed by using the equation (MB * 1024 * 1024) where MB is the number of megabytes you would like your default quota to be.

You may need to specify the location of your MySQL include and library directories with --enable-incdir and --enable-libdir.

By disabling /etc/passwd with --enable-passwd=n, only virtual users will have access to our mail server.

NFS users may wish to look at the --enable-file-locking and --enable-file-sync options. Because Qmail uses the alternative Maildir format to store messages, it works quite nicely with NFS.

After running the configure script, compile and install vpopmail with the following commands:

bash$ make
bash$ make install-strip
Note: Debian has packages for vpopmail, however I could not get them working properly with MySQL.

Creating Domains
Adding domains is extremely simple. Before you create a domain you will want to read the man pages for vadddomain. There are options for mail delivery, quotas, etc.

bash$ /path/to/vpopmail/bin/vadddomain example.com password
password is the password for postmaster@example.com, who will be the administrator for that domain. After you have created the domain your domain's user directories and .qmail files will exist in /path/to/vpopmail/domains/example.com.

Adding Users
You can use either qmailadmin to create users or vpopmail's vadduser.

bash$ /path/to/vpopmail/bin/vadduser username@example.com password
Courier IMAP
Courier IMAP is another piece of software by Inter7. It's a simple IMAP server that was created specifically to work with Maildir. Read over the INSTALL file before we get started to familiarize yourself with the installation process.

bash$ ./configure \
      --prefix=/usr/local/courier-imap \
      --with-ssl \
      --with-authvchkpw
bash$ make
bash$ make install
bash$ make install-configure
NOTE: --with-ssl is optional

Once you are done installing Courier IMAP, you need to edit a few configuration options in its configuration file. Open /usr/local/courier-imap/etc/imapd in your favorite editor and change the following options.

editor's note: some of these options could use further explanation

Change MAXDAEMONS to 40.

Raise MAXPERIP to 100. (Many users will be using the web interface, which all come from a single IP.)

Change TCPDOPTS to "-nodnslookup -noidentlookup -user=vpopmail -group=vchkpw".

Change AUTHMODULES to "authvchkpw" to use vpopmail's authentication.

Change IMAP_EMPTYTRASH=Trash:7,Sent:30 to whatever you want. This option tells Courier to clear out these folders every 7 and 30 days, respectively. You can add extra folders there as well that you wish to have the server empty periodically.

Change IMAPDSTART to YES.

After you have all of this up and running, you should be ready to fire up your IMAP server. To start Courier IMAP at boot, copy /usr/local/src/courier-imap-1.7.x/courier-imap.sysvinit to the directory appropriate for your distro (many use /etc/init.d).

bash$ cp /usr/local/src/courier-imap-1.7.x/courier-imap.sysvinit \
        /etc/init.d/courier-imap
bash$ chmod 744 /etc/init.d/courier-imap
bash$ /etc/init.d/courier-imap start
qmailadmin
qmailadmin is a CGI interface to vpopmail. It is totally optional, but will make administering your virtual domains, users, forwards, etc. much easier. Not only does it allow your postmaster to create and manage accounts, forwards, and aliases, but it allows users to log in and change passwords, set vacation messages, etc.

Note: qmailadmin requires EZMLM, DJB's mailing list manager, which you may not wish to install. You can skip this step.

Installing autoresponder
autoresponder is a prerequisite of qmailadmin and does basic auto responding. After you have extracted the source, change into the package's directory.

bash$ make
bash$ make man
bash$ make setup
Installing EZMLM
EZMLM is DJB's mailing list software which works great with Qmail. EZMLM's main website has been down for quite some time. I suggest you grab the source from DJB's EZMLM page. You will most likely wish to patch EZMLM with the famous IDX patch which adds a plethora of options. I found a working mirror where you can download ezmlm-idx-0.40.tar.gz. After you have extracted both the EZMLM source and the IDX patch, install EZMLM.

bash$ mv ezmlm-idx-0.xx/* ezmlm-0.53/
bash$ cd ezmlm-0.53
bash$ patch < idx.patch
bash$ make mysql
bash$ make clean
bash$ make && make man
bash$ make setup
Installing qmailadmin
bash$ ./configure \
      --enable-cgibindir=/path/to/your/cgi-bin
      --with-htmllibdir=/path/to/your/html/docroot
      --enable-imagedir=/path/to/your/images/directory
There are several options to look into if you plan to allow clients to administer their own domains. --enable-maxpopusers and --enable-maxmailinglists are just two of the options you may wish to use. Be sure to check out ./configure --help for a complete list of options.

bash$ make
bash$ make install-strip
After you have installed qmailadmin, you should be able to log in to to http://example.com/cgi-bin/qmailadmin to administer your domains.

Conclusion
Now that you have everything installed and have created your users, you should be able to log in. Because vpopmail uses virtual hosts your username will be username@example.com, which is important to remember. Because of Qmail's flexibility and the power of vpopmail's MySQL support you can easily create domains and hand over the ability to manage accounts to your customers, which leaves plenty of free time.

Now that your mail server is running the hard part is over, but it is not yet complete. The second part of our series will cover installing Apache+PHP and Squirrel Mail. We will also customize Squirrel Mail with a few plugins that will make managing your web mail system a lot easier. In the third and final part of our series, you will see how easy it is to integrate both virus and spam protection into your new mail server.

Joe Stump is a freelance web developer specializing in PHP and MySQL.