Own mail server based on Dovecot, Postfix, MySQL, Rspamd and Debian 9 Stretch

This how-to is based on my previous German how-to for Ubuntu 16.04 Server. Instead of using Spamassassin, Amavis, Pyzor and Razor as well es OpenDKIM, we’ll make use of Rspamd. Rspamd as a modern replacement will reduce the complexity of our setup and let us monitor its state via a web interface. As in earlier versions of my mailserver how-to, I’ll explain most important parts of the system as we install and configure them one after another. You will need basic Linux and command line knowledge to finish this guide. Going through this article will take probably about 45 minutes - depending on your speed and skills.

If you’d rather like to use a simple Docker setup, have a look at Mailcow by André Peters.

Mail server features

To get a basic idea of what this mail server will be capable of:

  • Sending and receiving e-mails for various domains of your choice
  • Adding and removing domains, accounts, aliases, redirects and TLS policies via a MySQL DB backend
  • Per-user mailbox quota
  • Global and user-specific sieve rules
  • High performance spam server detection via postscreen
  • Extended spam message detection via Rspamd (+ web interface for spam stats)
  • “Send only” cccounts e.g. for NextCloud / blog software / …
  • DKIM-signing for outbound e-mails

Please note: There is no “official” web interface for mailbox management, but some users implemented their own mailbox account management software for this setup:


Rspamd Webinterface


Software for this setup

  • Debian 9 “Stretch”
  • Postfix
  • Dovecot
  • Rspamd
  • Redis
  • MariaDB
  • Nginx (optional)

Structural conditions

These domains will be used in this guide as place-holders:

  • mysystems.tld: Primary domain for mail accounts and mail system
  • mail.mysystems.tld: FQDN (Fully qualified domain name) for mail host
  • imap.mysystems.tld: Alias for mail.mysystems.tld, for imap access
  • smtp.mysystems.tld: Alias for mail.mysystems.tld, for smtp access
  • domain2.tld: (optional) Second domain, for which e-mails shall be sent and received
  • domain3.tld: (optional) Third domain, for which e-mails shall be sent and received

All these domains must be replaced by your own throughout this how-to!

Basic requirements

  • A virtual or dedicated server with Debian 9 “Stretch” installed. (“Basic system utilities” and SSH installed during Debian setup)
  • At least one domain (mysystems.tld) and full control over the DNS zone and its records

For a high quality and yet cheap, European hosting provider I recommend Servercow.de. A small machine with two cores and 1-2 Gigs of RAM is sufficient in most cases.

Mail system components

Mailserver Schema

Dovecot

Dovecot is a widely spread MDA (Mail Delivery Agent) and an IMAP-server. It moves inbound e-mails into the users’ mailboxes and provides an IMAP interface for MUAs (Mail user agents, such as Mozilla Thunderbird) to retrieve e-mails. Furthermore, Dovecot is our SASL authentication server for Postfix.

Postfix

Postfix as a popular MTA (Mail Transfer Agent) is often combined with Dovecot. While Dovecot handles the mailbox, Postfix does all the e-mail transfer tasks from client to server or from server to server.

MariaDB (MySQL-database)

MariaDB will be the user accounts backend for Postfix and Dovecot. A dedicated database will contain information about our domains, user accounts, aliases and TLS policies. You can add modify and remove datasets during runtime and adapt the mail system to your needs without having to restart any services. As a bonus, the MySQL database is easily accessible e.g. for PHP scripts and you can develop your own management frontend if you wish to.

Rspamd

Rspamd is a modern e-mail filter system, which replaces Amavis and Spamassassin. It also is capable of signing outbound e-mails with DKIM keys. The status of Rspamd can be monitored via a simple web interface.

Nginx (optional)

Nginx is a popular HTTP web server and -proxy. It is used for comfortable and secure access to the Rpspamd webinterface.

Redis

Redis is a high performance Key-Value-Store used by Rspamd. It will store data about recently seen mail servers and spam scores.

Preparation

Tip: Wipe your server

Please consider starting from scratch! If you’ve tried other mail how-tos before, there might be leftovers, which interfere with this guide in some cases. You can avoid trouble by starting with a freshly set up Debian machine.

Login as Root

You will need permananent root access during this tutorial, so switch over to your systems root account:

su

(Enter password for root)

Update system

Get the latest software packages and linux kernel, to make sure everything works fine:

apt update && apt upgrade

you might want to do a reboot to load the latest linux kernel.

Set hostname and FQDN

Your server gets to names: A hostname and a FQDN (Fully qualified domain name).

  • Local hostname: For identifying your server in nyour local infrastructure. E.g. “mail”.
  • FQDN (Fully Qualified Domain Name): For identifying your server on the internet. E.g. “mail.mysystems.tld”

You do not need to fit your mail server’s FQDN to any domain you want to serve with it. These domains do not need to be the same or similar. Set your local hostname as follows:

hostnamectl set-hostname --static mail

The configuration file /etc/hosts contains FQDN and local hostname next to each other. It should be similar to this one:

127.0.0.1   localhost
127.0.1.1   mail.mysystems.tld  mail

::1         localhost ip6-localhost ip6-loopback
ff02::1     ip6-allnodes
ff02::2     ip6-allrouters

If you enter the “hostname” and “hostname –fqdn” commands, this should be your output:

root@mail:~# hostname
mail
root@mail:~# hostname --fqdn
mail.mysystems.tld

Furthermore, the FQDN (“mail.mysystems.tld”) is copied to /etc/mailname:

echo $(hostname -f) > /etc/mailname

(The hostname in your shell prompt will adapt after a reboot)

Tipp: Install Unbound DNS Resolver

Rspamd, Postfix / Postscreen and more services on your system heavily depend on DNS requests. I strongly recommend to install Unbound as a local DNS resolver and -cache! Some server providers rate-limit your access to their pre-defined DNS resolvers, which might cause trouble. Especially Rspamd does a lot of DNS requests depending on the mail system load. Furthermore, Spamhaus blocklists often can be used with own DNS resolvers only (we are going to use Spamhaus with postscreen).

Install unbound:

apt install unbound

Update DNSSEC Root key and reload Unbound service:

su -c "unbound-anchor -a /var/lib/unbound/root.key" - unbound
systemctl reload unbound
The following commands will only work if the “dnsutils” package is installed to your Debian system!

A dig @127.0.0.1 denic.de should result in a similar output like this:

dig @127.0.0.1 denic.de +short +dnssec
81.91.170.12
A 8 2 3600 20170814090000 20170731090000 26155 denic.de. Jo90qnkLkZ6gI4qNHj19BMguFuGof9hCPhdeSh/fSePSQ/WXlWMmfjW1 sNDJ/bcITRMyz8DQdDzmWPDIeSJ/qPyfoZ+BjUZxtaXcs0BAl4KX8q7h R05TGmAbgPhrYBoUKJkU/q8T+jWKHAJRUeWbCd8QOJsJbneGcUKxRAPe i6Rq51/OL/id6zUCtalhclah2TfLLaqku9PmKwjbGdZm11BXSr8b56LB WX/rdLIrKWNpE+jHGAUMmDsZL84Kx3Oo

If the dig-command worked, you can now set Unbound as the primary DNS resolver for your mail system:

apt install resolvconf
echo "nameserver 127.0.0.1" >> /etc/resolvconf/resolv.conf.d/head

result of nslookup denic.de | grep Server should now be:

Server:     127.0.0.1

Now you have your own DNS resolver up and running. Let’s continue with DNS configuration …

DNS zone setup

You need some DNS records set up for your mailsystem to work. Browse the management interface of your DNS provider and look for the zone files of your domains. Then add the records shown below:

Point your system’s FQDN to its IPv4 address (and - if you have one - to your IPv6 address)

mail.mysystems.tld. 86400 IN A    5.1.76.155
mail.mysystems.tld. 86400 IN AAAA 2a00:f820:417::be19:7b23

“imap.mysystems.tld” and “smtp.mysystems.tld” are aliases for “mail.mysystems.tld”. They are not absolutely necessary, but considered “good style”. Many mail clients are looking for these domain names if you set up a new mail account.

imap.mysystems.tld. 86400 IN CNAME mail.mysystems.tld.
smtp.mysystems.tld. 86400 IN CNAME mail.mysystems.tld.

Now let’s create some MX records: Other mail servers need these to figure out, which host is responsible for mails on a certain domain. We want the host on “mail.mysystems.tld” to responsible for all e-mails, which go to @mysystemd.tld mail addresses.

mysystems.tld. 86400 IN MX 0 mail.mysystems.tld.

… and we also want it to handle e-mails for “domain2” and “domain3”:

domain2.tld. 86400 IN MX 0 mail.mysystems.tld.
domain3.tld. 86400 IN MX 0 mail.mysystems.tld.

(the last three entrys have to be created in their corresponding zone files)

Reverse DNS

A reverse DNS record (also called “PTR record” matches a FQDN to an IP-address. Many popular e-mail providers check other mail server’s PTR records and deny receiving e-mails from them, if they cannot find a proper domain name to their IPs. You will need PTR records for all your server’s IP-addresses. They all must point to mail.mysystems.tld. In most cases reverse DNS entries can be set via your server hoster’s web interface or via the server support team.

SPF-Records

SPF records were invented to support the fight against spam servers. Unfortunately it turned out as a miss-conception and you cannot rely on these records anymore. SPF describes which mail servers are allowed to send e-mails for a domain and which are not. In some cases (e.g. mailing lists) the SPF concept does not work. Still some providers expect you to set a SPF record - if you don’t, you’ll get some point off your “spam credibility score”. So let’s make a compromise and provide a neutral SPF record:

mysystems.tld. 3600 IN TXT v=spf1 a:mail.mysystems.tld ?all

Old-schooled, bad mail providers will be satisfied, and yet we don’t support bad SPF practices. The SPF record is also set for your other domains “domain2” and “domain3”, but in a slightly different way:

domain2.tld. 3600 IN TXT v=spf1 include:mysystems.tld ?all

Set up TLS certificates

A modern e-mail server can’t be operated seriously without TLS certificates. We will use Let’s Encrypt certificates for this purpose, as they are free and yet accepted by all browsers, mail clients and operating systems. If you already have valid certificates, you can use them instead.

Retrieve new certificates

Use the official “certbot” command line client to get new certificates for your mail system:

apt install certbot
certbot certonly --standalone --rsa-key-size 4096 -d mail.mysystems.tld -d imap.mysystems.tld -d smtp.mysystems.tld

After having agreed to the terms of use and having provided your e-mail address you will instantly get valid certificates. They are saved at: /etc/letsencrypt/live/mail.mysystems.tld and are valid for your three mail system domain names:

  • mail.mysystems.tld
  • imap.mysystems.tld
  • smtp.mysystems.tld

Your additional domains “domain2” and “domain3” do not need to be included into this certificate.

There are some new files in /etc/letsencrypt/live/mail.mysystems.tld which you will need:

  • cert.pem: Your mailserver certificate
  • chain.pem: CA certificate
  • fullchain.pem: mailserver certificate + CA certificate
  • privkey.pem: Private key for mailserver certificate

Later in this guide only the letter two certificate files will be used.

Auto-Renewal

Let’s encrypt certificates are valid for 90 days only, so they should be renewed in fixed intervals. You can achieve that via a cronjob:

@weekly certbot renew --pre-hook "systemctl stop nginx" --post-hook "systemctl start nginx" --renew-hook "systemctl reload nginx; systemctl reload dovecot; systemctl reload postfix" --quiet

This entry is added to the crontab file via

crontab -e

(select your favorite editor, paste the line above and save your changes)

MySQL database setup

Install MariaDB as a DBMS (database management system)

apt install mariadb-server

MariaDB should be running right after its installation. You can check the service status via systemctl status mysql (not “mariadb!”). In case it is not yet running, start it: systemctl start mysql.

mysql

will get you into the MySQL shell. A few SQL commands have to be executed there to set up the database structure. Tip: Copy and paste the following SQL commands, to make sure they are formatted correctly:

Create a new database named “vmail”:

create database vmail
    CHARACTER SET 'utf8';

Create database user “vmail”, with password “vmaildbpass”. This user will be used by Postfix and Dovecot. It is granted “select” permissions on this DB.

grant select on vmail.* to 'vmail'@'localhost' identified by 'vmaildbpass';

Now move your SQL shell to the new database:

use vmail;

The following SQl commands do not need to be modified. Just copy them as they are and paste them into your SQL command line.

Domain table

CREATE TABLE `domains` (
    `id` int unsigned NOT NULL AUTO_INCREMENT,
    `domain` varchar(255) NOT NULL,
    PRIMARY KEY (`id`),
    UNIQUE KEY (`domain`)
);

The domain table contains all domains, which shall be served by the mail server.

Account table

CREATE TABLE `accounts` (
    `id` int unsigned NOT NULL AUTO_INCREMENT,
    `username` varchar(64) NOT NULL,
    `domain` varchar(255) NOT NULL,
    `password` varchar(255) NOT NULL,
    `quota` int unsigned DEFAULT '0',
    `enabled` boolean DEFAULT '0',
    `sendonly` boolean DEFAULT '0',
    PRIMARY KEY (id),
    UNIQUE KEY (`username`, `domain`),
    FOREIGN KEY (`domain`) REFERENCES `domains` (`domain`)
);

The account table contains all data regarding user mailbox accounts, such as username, domain, password, and quota. Quota is in Megabyte (MB). If the “enabled” field if set to “true” a mailbox account is active and can be used. If “sendonly” is set to “true” this account is not able to receive mails.

Alias table

CREATE TABLE `aliases` (
    `id` int unsigned NOT NULL AUTO_INCREMENT,
    `source_username` varchar(64) NOT NULL,
    `source_domain` varchar(255) NOT NULL,
    `destination_username` varchar(64) NOT NULL,
    `destination_domain` varchar(255) NOT NULL,
    `enabled` boolean DEFAULT '0',
    PRIMARY KEY (`id`),
    UNIQUE KEY (`source_username`, `source_domain`, `destination_username`, `destination_domain`),
    FOREIGN KEY (`source_domain`) REFERENCES `domains` (`domain`)
);

The alias table contains all alias definitions / redirects.

TLS Policy table

CREATE TABLE `tlspolicies` (
    `id` int unsigned NOT NULL AUTO_INCREMENT,
    `domain` varchar(255) NOT NULL,
    `policy` enum('none', 'may', 'encrypt', 'dane', 'dane-only', 'fingerprint', 'verify', 'secure') NOT NULL,
    `params` varchar(255),
    PRIMARY KEY (`id`),
    UNIQUE KEY (`domain`)
);

The TLS policy table defines policies regarding TLS-encryption to foreign mail servers (more about that later).

Now leave the SQL shell again, to get back to your Debian Bash Shell:

quit;

vmail-user and vmail-directory

All e-mails and sieve scripts are saved into a special directory /var/vmail. Only the associated “vmail”-user has access to it. Dovecot will use this user account to do its operations on the file system.

Create vmail-directory:

mkdir /var/vmail

Create vmail-user

adduser --disabled-login --disabled-password --home /var/vmail vmail

Create some vmail subdirectories:

mkdir /var/vmail/mailboxes
mkdir -p /var/vmail/sieve/global

vmail-user gets all permissions on /var/vmail:

chown -R vmail /var/vmail
chgrp -R vmail /var/vmail
chmod -R 770 /var/vmail

Install and configure Dovecot

Install the following Dovecot components:

apt install dovecot-core dovecot-imapd dovecot-lmtpd dovecot-mysql dovecot-sieve dovecot-managesieved dovecot-antispam
  • dovecot-core: Dovecot core
  • dovecot-imapd: IMAP for Dovecot
  • dovecot-lmtp: LMTP (Local Mail Transfer Protocol) for Dovecot; LMTP is used as a transmission protocol between Postfix and Dovecot.
  • dovecot-mysql: MySQL support for Dovecot
  • dovecot-sieve: Filter module
  • dovecot-managesieved: Filter management server
  • dovecot-antispam: Interface to train anti spam software based on e-mail moves in mailbox

Dovecot is started automatically after installation. Stop is as long as we don’t have a proper configuration:

systemctl stop dovecot

All the Dovecot configuration files sit in /etc/dovecot/. As we are going to create a Dovecot configuration from scratch, you can delete all the existing files:

rm -r /etc/dovecot/*
cd /etc/dovecot

We’ll just need two configuration files in our case: dovecot.conf and dovecot-sql.conf. Create them both in /etc/dovecot/ a follows:

Main configuration file dovecot.conf

###
### aktivated protocols
#############################

protocols = imap lmtp sieve



###
### TLS configuration
#######################

ssl = required
ssl_cert = </etc/letsencrypt/live/mail.mysystems.tld/fullchain.pem
ssl_key = </etc/letsencrypt/live/mail.mysystems.tld/privkey.pem
ssl_dh_parameters_length = 2048
ssl_protocols = !SSLv3
ssl_cipher_list = EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA256:EECDH:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!IDEA:!ECDSA:kEDH:CAMELLIA128-SHA:AES128-SHA
ssl_prefer_server_ciphers = yes



###
### Dovecot services
################################

service imap-login {
    inet_listener imap {
        port = 143
    }
}


service managesieve-login {
    inet_listener sieve {
        port = 4190
    }
}


service lmtp {
    unix_listener /var/spool/postfix/private/dovecot-lmtp {
        mode = 0660
        group = postfix
        user = postfix
    }

    user = vmail
}


service auth {
    ### Auth socket für Postfix
    unix_listener /var/spool/postfix/private/auth {
        mode = 0660
        user = postfix
        group = postfix
    }

    ### Auth socket für LMTP-Dienst
    unix_listener auth-userdb {
        mode = 0660
        user = vmail
        group = vmail
    }
}


###
###  Protocol settings
#############################

protocol imap {
    mail_plugins = $mail_plugins quota imap_quota antispam
    mail_max_userip_connections = 20
    imap_idle_notify_interval = 29 mins
}

protocol lmtp {
    postmaster_address = postmaster@mysystems.tld
    mail_plugins = $mail_plugins sieve
}



###
### Client authentication
#############################

disable_plaintext_auth = yes
auth_mechanisms = plain login


passdb {
    driver = sql
    args = /etc/dovecot/dovecot-sql.conf
}

userdb {
    driver = sql
    args = /etc/dovecot/dovecot-sql.conf
}


###
### Mail location
#######################

mail_uid = vmail
mail_gid = vmail
mail_privileged_group = vmail


mail_home = /var/vmail/mailboxes/%d/%n
mail_location = maildir:~/mail:LAYOUT=fs



###
### Mailbox configuration
########################################

namespace inbox {
    inbox = yes

    mailbox Spam {
        auto = subscribe
        special_use = \Junk
    }

    mailbox Trash {
        auto = subscribe
        special_use = \Trash
    }

    mailbox Drafts {
        auto = subscribe
        special_use = \Drafts
    }

    mailbox Sent {
        auto = subscribe
        special_use = \Sent
    }
}



###
### Mail plugins
############################


plugin {
    sieve_before = /var/vmail/sieve/global/spam-global.sieve
    sieve = file:/var/vmail/sieve/%d/%n/scripts;active=/var/vmail/sieve/%d/%n/active-script.sieve

    quota = maildir:User quota
    quota_exceeded_message = Benutzer %u hat das Speichervolumen überschritten. / User %u has exhausted allowed storage space.

    antispam_backend = pipe
    antispam_spam    = Spam
    antispam_trash   = Trash
    antispam_mail_sendmail = /usr/bin/rspamc
    antispam_mail_spam     = learn_spam
    antispam_mail_notspam  = learn_ham
}

Settings to adjust:

  • ssl_cert: Path to certificate file
  • ssl_key: Path to certificate key
  • postmaster_address: Fit to your primary domain

SQL configuration file dovecot-sql.conf

driver=mysql
connect = "host=127.0.0.1 dbname=vmail user=vmail password=vmaildbpass"
default_pass_scheme = SHA512-CRYPT

password_query = SELECT username AS user, domain, password FROM accounts WHERE username = '%n' AND domain = '%d' and enabled = true;
user_query = SELECT concat('*:storage=', quota, 'M') AS quota_rule FROM accounts WHERE username = '%n' AND domain = '%d' AND sendonly = false;
iterate_query = SELECT username, domain FROM accounts where sendonly = false;

Settings to adjust:

  • Database password “vmaildbpass”

Set proper permissions for dovecot-sql.conf:

chmod 440 dovecot-sql.conf

Global sieve filter to move marked spam mails

Create a new Sieve filter script spam-global.sieve in /var/vmail/sieve/global/

require "fileinto";

if header :contains "X-Spam-Flag" "YES" {
    fileinto "Spam";
}

if header :is "X-Spam" "Yes" {
    fileinto "Spam";
}

Install and configure Postfix

Install:

apt install postfix postfix-mysql

During installation of the Postfix packages you will be asked what type of configuration you want to create. Select “No configuration”. Then stop Postfix:

systemctl stop postfix

Although you selected “No configuration” there will be configuration files in /etc/postfix. Delete some of them:

cd /etc/postfix
rm -r sasl
rm master.cf main.cf.proto master.cf.proto

Then create the following new config files in /etc/postfix:

main.cf

##
## Network settings
##

mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
inet_interfaces = 127.0.0.1, ::1, 5.1.76.152, 2a00:f820:417::7647:b2c2
myhostname = mail.mysystems.tld


##
## Mail queue settings
##

maximal_queue_lifetime = 1h
bounce_queue_lifetime = 1h
maximal_backoff_time = 15m
minimal_backoff_time = 5m
queue_run_delay = 5m


##
## TLS settings
###

tls_ssl_options = NO_COMPRESSION
tls_high_cipherlist = EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA256:EECDH:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!IDEA:!ECDSA:kEDH:CAMELLIA128-SHA:AES128-SHA


### Outbound SMTP connections (Postfix as sender)

smtp_tls_security_level = dane
smtp_dns_support_level = dnssec
smtp_tls_policy_maps = mysql:/etc/postfix/sql/tls-policy.cf
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_protocols = !SSLv2, !SSLv3
smtp_tls_ciphers = high
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt


### Inbound SMTP connections

smtpd_tls_security_level = may
smtpd_tls_protocols = !SSLv2, !SSLv3
smtpd_tls_ciphers = high
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache

smtpd_tls_cert_file=/etc/letsencrypt/live/mail.mysystems.tld/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/mail.mysystems.tld/privkey.pem


##
## Local mail delivery to Dovecot via LMTP
##

virtual_transport = lmtp:unix:private/dovecot-lmtp


##
## Spam filter and DKIM signatures via Rspamd
##

smtpd_milters = inet:localhost:11332
non_smtpd_milters = inet:localhost:11332
milter_protocol = 6
milter_mail_macros =  i {mail_addr} {client_addr} {client_name} {auth_authen}
milter_default_action = accept



##
## Server Restrictions for clients, cecipients and relaying
## (concerning S2S-connections. Mailclient-connections are configured in submission-section in master.cf)
##

### Conditions in which Postfix works as a relay. (for mail user clients)
smtpd_relay_restrictions =      reject_non_fqdn_recipient
                                reject_unknown_recipient_domain
                                permit_mynetworks
                                reject_unauth_destination


### Conditions in which Postfix accepts e-mails as recipient (additional to relay conditions)
### check_recipient_access checks if an account is "sendonly"
smtpd_recipient_restrictions = check_recipient_access mysql:/etc/postfix/sql/recipient-access.cf


### Restrictions for all sending foreign servers ("SMTP clients")
smtpd_client_restrictions =     permit_mynetworks
                                check_client_access hash:/etc/postfix/without_ptr
                                reject_unknown_client_hostname


### Foreign mail servers must present a valid "HELO"
smtpd_helo_required = yes
smtpd_helo_restrictions =   permit_mynetworks
                            reject_invalid_helo_hostname
                            reject_non_fqdn_helo_hostname
                            reject_unknown_helo_hostname

# Block clients, which start sending too early
smtpd_data_restrictions = reject_unauth_pipelining


##
## Restrictions for MUAs (Mail user agents)
##

mua_relay_restrictions = reject_non_fqdn_recipient,reject_unknown_recipient_domain,permit_mynetworks,permit_sasl_authenticated,reject
mua_sender_restrictions = permit_mynetworks,reject_non_fqdn_sender,reject_sender_login_mismatch,permit_sasl_authenticated,reject
mua_client_restrictions = permit_mynetworks,permit_sasl_authenticated,reject


##
## Postscreen Filter
##

### Postscreen Whitelist / Blocklist
postscreen_access_list =        permit_mynetworks
                                cidr:/etc/postfix/postscreen_access
postscreen_blacklist_action = drop


# Drop connections if other server is sending too quickly
postscreen_greet_action = drop


### DNS blocklists
postscreen_dnsbl_threshold = 2
postscreen_dnsbl_sites =    ix.dnsbl.manitu.net*2
                            zen.spamhaus.org*2
postscreen_dnsbl_action = drop


##
## MySQL queries
##

virtual_alias_maps = mysql:/etc/postfix/sql/aliases.cf
virtual_mailbox_maps = mysql:/etc/postfix/sql/accounts.cf
virtual_mailbox_domains = mysql:/etc/postfix/sql/domains.cf
local_recipient_maps = $virtual_mailbox_maps


##
## Miscellaneous
##

### Maximum mailbox size (0=unlimited - is already limited by Dovecot quota)
mailbox_size_limit = 0

### Maximum size of inbound e-mails (50 MB)
message_size_limit = 52428800

### Do not notify system users on new e-mail
biff = no

### Users always have to provide full e-mail addresses
append_dot_mydomain = no

### Delimiter for "Address Tagging"
recipient_delimiter = +

Settings to adjust:

  • inet_interfaces: IP addresses of your server. 5.1.76.152, 2a00:f820:417::7647:b2c2 must be replaced by your own IPv4- and IPv6-address.
  • myhostname: Replace by your own hostname
  • smtpd_tls_cert_file: Path to certificate file
  • smtpd_tls_key_file: Path to certificate key

Configuration file master.cf

# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (no)    (never) (100)
# ==========================================================================
smtp      inet  n       -       y       -       1       postscreen
    -o smtpd_sasl_auth_enable=no
smtpd     pass  -       -       y       -       -       smtpd
dnsblog   unix  -       -       y       -       0       dnsblog
tlsproxy  unix  -       -       y       -       0       tlsproxy
submission inet n       -       y       -       -       smtpd
    -o syslog_name=postfix/submission
    -o smtpd_tls_security_level=encrypt
    -o smtpd_sasl_auth_enable=yes
    -o smtpd_sasl_type=dovecot
    -o smtpd_sasl_path=private/auth
    -o smtpd_sasl_security_options=noanonymous
    -o smtpd_client_restrictions=$mua_client_restrictions
    -o smtpd_sender_restrictions=$mua_sender_restrictions
    -o smtpd_relay_restrictions=$mua_relay_restrictions
    -o milter_macro_daemon_name=ORIGINATING
    -o smtpd_sender_login_maps=mysql:/etc/postfix/sql/sender-login-maps.cf
    -o smtpd_helo_required=no
    -o smtpd_helo_restrictions=
    -o cleanup_service_name=submission-header-cleanup
pickup    unix  n       -       y       60      1       pickup
cleanup   unix  n       -       y       -       0       cleanup
qmgr      unix  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       y       1000?   1       tlsmgr
rewrite   unix  -       -       y       -       -       trivial-rewrite
bounce    unix  -       -       y       -       0       bounce
defer     unix  -       -       y       -       0       bounce
trace     unix  -       -       y       -       0       bounce
verify    unix  -       -       y       -       1       verify
flush     unix  n       -       y       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       y       -       -       smtp
relay     unix  -       -       y       -       -       smtp
showq     unix  n       -       y       -       -       showq
error     unix  -       -       y       -       -       error
retry     unix  -       -       y       -       -       error
discard   unix  -       -       y       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       y       -       -       lmtp
anvil     unix  -       -       y       -       1       anvil
scache    unix  -       -       y       -       1       scache
submission-header-cleanup unix n - n    -       0       cleanup
    -o header_checks=regexp:/etc/postfix/submission_header_cleanup

Header cleanup rules

Create a new file /etc/postfix/submission_header_cleanup with this content:

### Removes headers of MUAs for privacy reasons

/^Received:/            IGNORE
/^X-Originating-IP:/    IGNORE
/^X-Mailer:/            IGNORE
/^User-Agent:/          IGNORE

SQL configuration

SQL queries for Postfix sit in the sql/ subdirectory:

mkdir /etc/postfix/sql && cd /etc/postfix/sql/

Create these files with their corresponding content:

accounts.cf

user = vmail
password = vmaildbpass
hosts = 127.0.0.1
dbname = vmail
query = select 1 as found from accounts where username = '%u' and domain = '%d' and enabled = true LIMIT 1;

aliases.cf

user = vmail
password = vmaildbpass
hosts = 127.0.0.1
dbname = vmail
query = select concat(destination_username, '@', destination_domain) as destinations from aliases where source_username = '%u' and source_domain = '%d' and enabled = true;

domains.cf

user = vmail
password = vmaildbpass
hosts = 127.0.0.1
dbname = vmail
query = SELECT domain FROM domains WHERE domain='%s'

recipient-access.cf

user = vmail
password = vmaildbpass
hosts = 127.0.0.1
dbname = vmail
query = select if(sendonly = true, 'REJECT', 'OK') AS access from accounts where username = '%u' and domain = '%d' and enabled = true LIMIT 1;

sender-login-maps.cf

user = vmail
password = vmaildbpass
hosts = 127.0.0.1
dbname = vmail
query = select concat(username, '@', domain) as 'owns' from accounts where username = '%u' AND domain = '%d' and enabled = true union select concat(destination_username, '@', destination_domain) AS 'owns' from aliases where source_username = '%u' and source_domain = '%d' and enabled = true;

tls-policy.cf

user = vmail
password = vmaildbpass
hosts = 127.0.0.1
dbname = vmail
query = SELECT policy, params FROM tlspolicies WHERE domain = '%s'

Don’t forget to modify vmaildbpass in all of the above files, in case you are using another password!

Set proper permissions for /etc/postfix/sql:

chmod -R 640 /etc/postfix/sql

More Postfix configuration files

Create two new files in /etc/postfix. You can leave them empty.

touch /etc/postfix/without_ptr
touch /etc/postfix/postscreen_access

In without_ptr you can define entries like this:

1.2.3.3 OK

This will result in a policy, which allows server 1.2.3.3 to send e-mails to this host even if it does not have a valid PTR-record. After every change, without_ptr has to be converted into a database file and Postfix must be reloaded:

postmap /etc/postfix/without_ptr
systemctl reload postfix

For the moment, just create an empty database file:

postmap /etc/postfix/without_ptr

In postscreen_access you can define exceptions for the postscreen filter. If any mail server is blocked by postscreen and you want to grant access for any reason, add an entry similar to the following:

1.2.3.3 permit

You can do the opposite, too: If you always want to block a certain server, add “reject” instead of “permit”.

Execute

newaliases

to create the alias database file /etc/aliases.db. This file is expected by Postfix by default.

Rspamd

The official Debian repositories contain an outdated version of Rspamd, so use the Rspamd-Repository for installation instead:

apt install -y lsb-release wget
wget -O- https://rspamd.com/apt-stable/gpg.key | apt-key add -
echo "deb http://rspamd.com/apt-stable/ $(lsb_release -c -s) main" > /etc/apt/sources.list.d/rspamd.list
echo "deb-src http://rspamd.com/apt-stable/ $(lsb_release -c -s) main" >> /etc/apt/sources.list.d/rspamd.list

Update package sources and install Rspamd:

apt update
apt install rspamd

Stop Rspamd service

systemctl stop rspamd

Basic configuration

Following files are now created in /etc/rspamd/local.d/:

/etc/rspamd/local.d/options.inc: Network settings and definition of the DNS resolver to use.

local_addrs = "127.0.0.0/8, ::1";

dns {
    nameserver = ["127.0.0.1:53:10"];
}

/etc/rspamd/local.d/worker-normal.inc: Settings for the normal Rspamd worker

bind_socket = "localhost:11333";
### Anzahl der zu nutzenden Worker. Standard: Anzahl der virtuellen Prozessorkerne.
# count = 1

/etc/rspamd/local.d/worker-controller.inc: Worker controller settings: Password for web interface access, e.g.:

password = "$2$qecacwgrz13owkag4gqcy5y7yeqh7yh4$y6i6gn5q3538tzsn19ojchuudoauw3rzdj1z74h5us4gd3jj5e8y";

The password hash (“$2$ …”) must be generated by

rspamadm pw

before. Enter a password you would like to set, copy the hash and paste it into the configuration file above (instead of my example hash).

/etc/rspamd/local.d/worker-proxy.inc: Worker proxy (Milter-Module for Postfix)

bind_socket = "localhost:11332";
milter = yes;
timeout = 120s;
upstream "local" {
    default = yes;
    self_scan = yes;
}

/etc/rspamd/local.d/logging.inc: Error logging

type = "file";
filename = "/var/log/rspamd/rspamd.log";
level = "error";
debug_modules = [];

Milter Headers /etc/rspamd/local.d/milter_headers.conf

use = ["x-spamd-bar", "x-spam-level", "authentication-results"];
authenticated_headers = ["authentication-results"];

DKIM signing

Rspamd not only handles spam mail filtering, but also is capable of signing e-mails via DKIM. First, a signing key must be created:

mkdir /var/lib/rspamd/dkim/
rspamadm dkim_keygen -b 2048 -s 2017 -k /var/lib/rspamd/dkim/2017.key > /var/lib/rspamd/dkim/2017.txt
chown -R _rspamd:_rspamd /var/lib/rspamd/dkim
chmod 440 /var/lib/rspamd/dkim/*

2017 is the name of the DKIM key (also called “selector”). I’m using the current year as a name. For a DKIM signing key (/var/lib/rspamd/dkim/2017.key) there is also a corresponding public key, which has to be published in the DNS. An example of a record with this key is given in /var/lib/rspamd/dkim/2017.txt

Output example DNS record with public key:

cat /var/lib/rspamd/dkim/2017.txt

Example output:

2017._domainkey IN TXT ( "v=DKIM1; k=rsa; "
	"p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA2/al5HqXUpe+HUazCr6t9lv2VOZLR369PPB4t+dgljZQvgUsIKoYzfS/w9NagS32xZYxi1dtlDWuRfTU/ahHO2MYzE0zHE4lMfwb6VkNCG+pM6bAkCwc5cFvyRygwxAPEiHAtmtU5b0i9LY25Z/ZWgyBxEWZ0Wf+hLjYHvnvMqewPsduUqKVjDOdUqeBb1VAu3WFErOAGVUYfKqFX"
	"+yfz36Alb7/OMAort8A5Vo5t5k0vxTHzkYYg5KB6tLS8jngrNucGjyNL5+k0ijPs3yT7WpTGL3U3SEa8cX8WvOO1fIpWQz4yyZJJ1Mm62+FskSc7BHjdiMHE64Id/UBDDVjxwIDAQAB"
) ;

The important part starts with v=DKIM1 and ends with AQAB (in this example). Now put this part of the output into a new DNS record. To illustrate the process, I did this step for my DNS hoster and made a screenshot:

core-networks.de Screenshot mit DKIM Record

Make sure the _domainkey-subdomain-prefix matches your selector prefix! (“2017”)

If your record is not accepted by your DNS hoster, a 2048 bit key might be too long. You can generate a shorter key by specifying -b 1024 instead of -b 2048 and running the rspamadm dkim_keygen command again.

Create /etc/rspamd/local.d/dkim_signing.conf and set “2017” as the selector:

path = "/var/lib/rspamd/dkim/$selector.key";
selector = "2017";

### Enable DKIM signing for alias sender addresses
allow_username_mismatch = true;

This configuration is also copied to /etc/rspamd/local.d/arc.conf, since the ARC module makes use of the same parameters.

cp -R /etc/rspamd/local.d/dkim_signing.conf /etc/rspamd/local.d/arc.conf

Set ADSP record

An ADSP DNS record defines, if e-mails from a certain mail server must be DKIM-signed or if signatures are optional. I recommend mandatory DKIm signatures. Other mail servers can then be sure, that an e-mail is spam, if it does not have a valid signature.

Set this record for your primary mail domain:

_adsp._domainkey.mysystems.tld.	3600	IN	TXT		dkim=all

Zonefiles for both other domains get extended by CNAME record, which references the ADSP record of the primary domain, e.g.:

_adsp._domainkey.domain2.tld.	3600	IN	CNAME		_adsp._domainkey.mysystems.tld.

Redis

Rspamd uses Redis as a data cache. Installation is simple:

apt install redis-server

/etc/rspamd/local.d/redis.conf

servers = "127.0.0.1";

Start Rspamd

systemctl start rspamd

Nginx Proxy for Rspamd web interface (optional)

So get easy and secure access to the Rspamd web interface, you can install Nginx as a HTTP proxy with TLS-termination. As an alternative, access via a SSH tunnel is sufficient in some cases (see below).

Installation:

apt install nginx
nano /etc/nginx/sites-available/mail.mysystems.tld

Config file /etc/nginx/sites-available/mail.mysystems.tld

server {
    listen 80;
    listen [::]:80;
    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    ssl_certificate /etc/letsencrypt/live/mail.mysystems.tld/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/mail.mysystems.tld/privkey.pem;

    server_name mail.mysystems.tld;

    root /var/www/default;

    if ($ssl_protocol = "") {
        return 301 https://$server_name$request_uri;
    }

    location /rspamd/ {
        proxy_pass http://localhost:11334/;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Settings to adapt:

  • ssl_certificate: Path to certificate
  • ssl_certificate_key: Path to certificate key
  • server_name

Activate site configuration, reload and start Nginx:

ln -s /etc/nginx/sites-available/mail.mysystems.tld /etc/nginx/sites-enabled/mail.mysystems.tld
nginx -t
systemctl reload nginx

You should now be able to access the Rspamd web interface via https://mail.mysystems.tld/rspamd/ . Then enter the password you chose during Rspamd configuration.

if Rspamd throws errors like "/var/lib/rspamd/ [...]: map file is unavailable for reading". Just ignore them, since this is a bug. (See: https://github.com/vstakhov/rspamd/issues/1474)

Access Rspamd web interface via SSH tunnel (alternative to Nginx)

If your local machine is a Linux or MAC computer, enter the following command to bind the webinterface to your local TCP port 8080:

ssh -L 8080:localhost:11334 benutzer@mail.mysystems.tld -N

The web interface can then be browsed via http://localhost:8080. CTRL+C cancels the connection.

Train Rspamd with existing spam mail (optional)

If you have mailboxes in Maildir-format with spam e-mails and normal e-mails, you can use them to train Rspamd on some real world examples. Copy those mailbox folders to your new server and execute commands like this:

Train e-mails in ./oldserver/var/vmail/mailboxes/*/*/mail/Spam/cur as spam:

find ./oldserver/var/vmail/mailboxes/*/*/mail/Spam/cur -type f -exec /usr/bin/rspamc learn_spam {} \;

Train e-mails as “ham”:

find ./oldserver/var/vmail/mailboxes/*/*/mail/cur -type f -exec /usr/bin/rspamc learn_ham {} \;
find ./oldserver/var/vmail/mailboxes/*/*/mail/Sent/cur -type f -exec /usr/bin/rspamc learn_ham {} \;

Create domains, accounts, aliases and TLS-policies in database

Before the mailserver can be used reasonably, at least one domain and a corresponding user account must be existent. Fire up your msql command shell one more time:

mysql

… change to the vmail database:

use vmail;

Create a new domain data record

New user accounts can only be created for already existing domains, so create a new data set for your primary domain:

insert into domains (domain) values ('mysystems.tld');

Create a new user account

Now that the corresponding domains exists, a new user account for this domain can be created. But there is one step left to do: Before the SQL statement for a new account is executed, we need a hash of the passwort to be used with this account. Create a new password hash in the normal Bash Shell(!) via:

doveadm pw -s SHA512-CRYPT

A password hash looks similar to this:

{SHA512-CRYPT}$6$wHyJsS.doo39xoCu$KI1N4l.Vd0ZWYj5BYLI8AdK7ACwAZaM18gSy7Bko0dG2Hvli4.KfAmLk2ztFVP.R4T7oqiu7clKBehCTc4GGw0

Now copy that hash and save it somewhere else - you’ll need it in a few moments. First, paste the following into your SQL Shell(!):

insert into accounts (username, domain, password, quota, enabled, sendonly) values ('user1', 'mysystems.tld', '{SHA512-CRYPT}$6$wHyJsS[...]', 2048, true, false);

The value for the password field must now be replaced with the individual hash you’ve created before. In this example, an account for “user1@mysystems.tld” is created, with a storage quota of 2 GB, and the account is able to send and receive messages.

Create new alias address

Creating an alias address for another address is pretty streight forward:

insert into aliases (source_username, source_domain, destination_username, destination_domain, enabled) values ('alias', 'mysystems.tld', 'user1', 'mysystems.tld', true);

This would result in a re-direction of e-mails from alias@mysystems.tld to user1@mysystems.tld. If you want to implement a something like a poor man’s mailing list, you can create multiple alias entries such as:

team@domain.tld => user1@domain.tld
team@domain.tld => user2@domain.tld
team@domain.tld => user3@domain.tld

Define a new TLS policy (optional)

TLS policies let you specify how strong a connection to another mailserver must be secured. There are different levels of security and verification:

  • none: Don’t use encryption
  • may: Encrypt, if supported by other server. Self-signed certificates are accepted, because there is no certificate verification.
  • encrypt: Always encrypt. Self-signed certificates are accepted, because there is no certificate verification.
  • dane: If there are valid TLSA-records in the DNS, encryption is mandatory. The certificate is then verified via DANE. If invalid TLSA records are found, fallback is “encrypt”. If no TLSA-records are found, fallback is “may”.
  • dane-only: Encrypted connections only. Certificate verification via DANE. No fallback to weaker methods.
  • verify: Encrypted connections only. Certificate must be issued by an accepted CA. Hostname given in MX record must match hostname in certificate.
  • secure: Encrypted connections only. Certificate must be issued by an accepted CA. Hostname in certificate must by domain or subdomain of e-mail domain. No DNS used.

Example:

insert into tlspolicies (domain, policy) values ('mailbox.org', 'dane-only');

The “params” field is used for additional verification details, such as “match” parameters. E.g. for GMX Mail you would define:

insert into tlspolicies (domain, policy, params) values ('gmx.de', 'secure', 'match=.gmx.net');

But why? Well, if “secure” as a mechanism is chosen, Postfix will find out the domain part of the recipient’s mail address. For GMX that could be thomas@gmx.de for example. Now Postfix would only allow connections to mail hosts, which have “gmx.de” in their hostname, such as host1.gmx.de, host2.gmx.de and so on. The problem here is, that GMX has no hosts running at “*gmx.de”. They are running at gmx.NET instead. So the “secure” policy must be extended with match=.gmx.net - otherwise connections to GMX mail servers would fail.

You can now leave the SQL shell again by entering quit;

Start all the things!

Let’s get the party started! \o/

systemctl start dovecot
systemctl start postfix

You may want to check if everything is running correctly. Enter

tail -f /var/log/syslog

or

tail -f /var/log/mail.log

to check if any errors occur.

Connect to your new mail server

  • IMAP-Server: mail.mysystems.tld | Port: 143 | STARTTLS (also use this server for domain2 und domain3!)
  • SMTP-Server: mail.mysystems.tld | Port: 587 | STARTTLS (also use this server for domain2 und domain3!)
  • (optional Managesieve: mail.mysystems.tld | Port: 4190 | STARTTLS) (also use this server for domain2 und domain3!)
  • Username: Complete(!) e-mail address
  • Password: … you should know ;-)
The first login might take a few moments.

Mailserver Logindaten für Thunderbird

Test your spam filter

Send an e-mail with the following body content to one of your new mailboxes:

XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

This e-mail should never arrive at its destination and Rspamd should show a rejected e-mail in its history log.


Did you like my how-to?

If you like this how-to, you maybe want to appreciate it by giving some dollars?

As you may know, developing, writing and testing a how-to is much work - especially for more complex setups like a mail server. If you want to say "thanks", help me pay my test servers or just keep me motivated, please consider donating a few dollars. :-)

Support my work

Thanks a lot! :-)


Q&A

“Can you set up my mailserver for me?”

Yes, I can! Please contact me via e-mail: Contact

“Am I allowed to copy / republish your how-to?”

No, please don’t to that. You can keep private copies, if you want.

“Is there a web interface for managing user accounts / domains / … ?”

Currently there is only one finished web interface by Andreas Bresch: https://github.com/Andreas-Bresch/vmailManage/

Changelog

  • (Published on 04.08.2017)

  • 2017-08-29

    • vmail MySQL user gets “select” permissions instead of “all” permissions
    • master.cf: More services run in chroot for better security. “mua” parameters to submission part and to main.cf
  • 2017-08-31

    • Added: Copy dkim_signing.conf to arc.conf to prevent error cannot load dkim key /var/lib/rspamd/arc/domain.tld.arc.key
    • Added: non_smtpd_milters = inet:localhost:11332 to DKIM-sign e-mails from mailserver itself, too.