Using Exim and Mailman Together
===============================

[This is derived from Nigel Metheringham's "HOWTO - Using Exim and
 Mailman together", which covers Mailman 2.0.x and Exim 3.  It
 has been updated to cover Mailman 2.1 and Exim 4.  The original
 document is here: http://www.exim.org/howto/mailman.html]


Mailman configuration
---------------------

There is no Mailman configuration needed other than the standard
options detailed in the Mailman install documentation.  The Exim
configuration is transparent to Mailman.  The user and group settings
for Mailman must match those in the config fragments given below.
   

Exim configuration
------------------

The Exim configuration is built so that a list created within Mailman
automatically appears to Exim without the need for defining any
additional aliases.

The drawback of this configuration is that it will work poorly on
systems supporting lists in several different mail domains.  While
Mailman handles virtual domains, it does not yet support having two
distinct lists with the same name in different virtual domains, using
the same Mailman installation.  This will eventually change.  (But see
below for a variation on this scheme that should accommodate virtual
domains better.)

The configuration file excerpts below are for use in an already
functional Exim configuration, which accepts mail for the domain in
which the list resides.  If this domain is separate from the others
handled by your Exim configuration, then you'll need to:

  * add the list domain, "my.list.domain" to local_domains

  * add a "domains=my.list.domain" option to the director
    (router) for the list

  * (optional) exclude that domain from your other directors (routers)

[Note: the instructions in this document should work with either Exim 3
or Exim 4.  In Exim 3, you must have a 'local_domains' configuration
setting; in Exim 4, you most likely have a 'local_domains' domainlist.
If you don't, you probably know what you're doing and can adjust
accordingly.  Similarly, in Exim 4 the concept of "directors" has
disappeared -- there are only routers now.  So if you're using Exim 4,
whenever this document says "director", read "router".]

Whether you are using Exim 3 or Exim 4, you will need to add some macros
to the main section of your Exim config file.  You will also need to
define one new transport.  With Exim 3, you'll need to add a new
director; with Exim 4, a new router plays the same role.

Finally, the configuration supplied here should allow co-habiting
Mailman 2.0 and 2.1 installations, with the proviso that you'll probably
want to use "mm21" in place of "mailman" -- e.g., MM21_HOME,
mm21_transport, etc.


Main configuration settings
~~~~~~~~~~~~~~~~~~~~~~~~~~~

First, you need to add some macros to the top of your Exim config file.
These just make the director (router) and transport below a bit
cleaner.  Obviously, you'll need to edit these based on how you
configured and installed Mailman.

  # Home dir for your Mailman installation -- aka Mailman's prefix
  # directory.
  MAILMAN_HOME=/usr/local/mailman
  MAILMAN_WRAP=MAILMAN_HOME/mail/mailman

  # User and group for Mailman, should match your --with-mail-gid
  # switch to Mailman's configure script.
  MAILMAN_USER=mailman
  MAILMAN_GROUP=mailman


Transport for Exim 3
~~~~~~~~~~~~~~~~~~~~

Add this to the transports section of your Exim config file,
i.e. somewhere between the first and second "end" line:

  mailman_transport:
    driver = pipe
    command = MAILMAN_WRAP \
              '${if def:local_part_suffix \
                    {${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}} \
                    {post}}' \
              $local_part
    current_directory = MAILMAN_HOME
    home_directory = MAILMAN_HOME
    user = MAILMAN_USER
    group = MAILMAN_GROUP


Director for Exim 3
~~~~~~~~~~~~~~~~~~~

If you're using Exim 3, you'll need to add the following director to
your config file (directors go between the second and third "end"
lines).  Also, don't forget that order matters -- e.g. you can make
Mailman lists take precedence over system aliases by putting this
director in front of your aliasfile director, or vice-versa.

  # Handle all addresses related to a list 'foo': the posting address.
  # Automatically detects list existence by looking
  # for lists/$local_part/config.pck under MAILMAN_HOME.
  mailman_director:
    driver = smartuser
    require_files = MAILMAN_HOME/lists/$local_part/config.pck
    suffix_optional
    suffix = -bounces : -bounces+* : \
             -confirm+* : -join : -leave : \
             -owner : -request : -admin
    transport = mailman_transport


Router for Exim 4
~~~~~~~~~~~~~~~~~

In Exim 4, there's no such thing as directors -- you need to add a new
router instead.  Also, the canonical order of the configuration file was
changed so routers come before transports, so the router for Exim 4
comes first here.  Put this router somewhere after the "begin routers"
line of your config file, and remember that order matters.

  mailman_router:
    driver = accept
    require_files = MAILMAN_HOME/lists/$local_part/config.pck
    local_part_suffix_optional
    local_part_suffix = -bounces : -bounces+* : \
                        -confirm+* : -join : -leave : \
                        -owner : -request : -admin
    transport = mailman_transport


Transports for Exim 4
~~~~~~~~~~~~~~~~~~~~~

The transport for Exim 4 is the same as for Exim 3; just copy the
transport given above to somewhere under the "begin transports" line of
your Exim config file.


Notes
-----

Exim should be configured to allow reasonable volume -- e.g. don't set
max_recipients down to a silly value -- and with normal degrees of
security -- specifically, be sure to allow relaying from 127.0.0.1, but
pretty much nothing else.  Parallel deliveries and other tweaks can also
be used if you like; experiment with your setup to see what works.
Delay warning messages should be switched off or configured to only
happen for non-list mail, unless you like receiving tons of mail when
some random host is down.


Problems
--------

  * Mailman will send as many MAIL FROM/RCPT TO as it needs. It may result
    in more than 10 or 100 messages sent in one connection, which will exceed
    the default value of Exim's smtp_accept_queue_per_connection
    This is bad because it will cause Exim to switch into queue mode and
    severely delay delivery of your list messages.
    The way to fix this is to set mailman's SMTP_MAX_SESSIONS_PER_CONNECTION
    (in ~mailman/Mailman/mm_cfg.py) to a smaller value than Exim's
    smtp_accept_queue_per_connection

  * Mailman should ignore Exim delay warning messages, even though
    Exim should never send this to list messages.  Mailman 2.1's
    general bounce detection and VERP support should greatly improve
    the bounce detector's hit rates.

  * List existence is determined by the existence of a config.pck file
    for a list.  If you delete lists by foul means, be aware of this.

  * If you are getting Exim or Mailman complaining about user ids when
    you send mail to a list, check that the MAILMAN_USER and
    MAILMAN_GROUP match those of Mailman itself (i.e. what were used
    in the configure script).  Also make sure you do not have aliases
    in the main alias file for the list.


Receiver Verification
---------------------

Exim's receiver verification feature is very useful -- it lets Exim
reject unrouteable addresses at SMTP time.  However, this is most useful
for externally-originating mail that is addressed to mail in one of your
local domains.  For Mailman list traffic, mail originates on your
server, and is addressed to random external domains that are not under
your control.  Furthermore, each message is addressed to many recipients
-- up to 500 if you use Mailman's default configuration and don't tweak
SMTP_MAX_RCPTS.

Doing receiver verification on Mailman list traffic is a recipe for
trouble.  In particular, Exim will attempt to route every recipient
addresses in outgoing Mailman list posts.  Even though this requires
nothing more than a few DNS lookups for each address, it can still
introduce significant delays.  Therefore, you should disable recipient
verification for Mailman traffic.

Under Exim 3, put this in your main configuration section:

  receiver_verify_hosts = !127.0.0.1

Under Exim 4, this is probably already taken care of for you by the
default recipient verification ACL statement (in the "RCPT TO" ACL):

  accept  domains       = +local_domains
          endpass
          message       = unknown user
          verify        = recipient

which only does recipient verification on addresses in your domain.
(That's not exactly the same as doing recipient verification only on
messages coming from non-127.0.0.1 hosts, but it should do the trick for
Mailman.)


SMTP Callback
-------------

Exim's SMTP callback feature is an even more powerful way to detect
bogus sender addresses than normal sender verification.  Unfortunately,
lots of servers send bounce messages with a bogus address in the header,
and there are plenty that send bounces with bogus envelope senders
(even though they're supposed to just use an empty envelope sender for
bounces).

In order to ensure that Mailman can disable/remove bouncing addresses,
you generally want to receive bounces for Mailman lists, even if those
bounces are themselves not bounceable.  Thus, you might want to disable
SMTP callback on bounce messages.

With Exim 4, you can accomplish this using something like the following
in your "RCPT TO" ACL:

  # Accept bounces to lists even if callbacks or other checks would fail
  warn     message	= X-WhitelistedRCPT-nohdrfromcallback: Yes
	   condition	= \
           ${if and {{match{$local_part}{(.*)-bounces\+.*}}
                     {exists {MAILMAN_HOME/lists/$1/config.pck}}} \
                {yes}{no}}
                {yes}{no}}

  accept   condition	= \
           ${if and {{match{$local_part}{(.*)-bounces\+.*}}
                     {exists {MAILMAN_HOME/lists/$1/config.pck}}} \
                {yes}{no}}
                {yes}{no}}

  # Now, check sender address with SMTP callback.
  deny   !verify = sender/callout=90s

If you also do SMTP callbacks on header addresses, you'll want something
like this in your "DATA" ACL:

  deny   !condition = $header_X-WhitelistedRCPT-nohdrfromcallback:
	 !verify = header_sender/callout=90s


Doing VERP with Exim and Mailman
--------------------------------

VERP will send one email, with a separate envelope sender (return path),
for each of your subscribers -- read the information in
~mailman/Mailman/Default.py for the options that start with VERP.  In a
nutshell, all you need to do to enable VERP with Exim is to add these
lines to ~mailman/Mailman/mm_cfg.py:

  VERP_PASSWORD_REMINDERS = 1
  VERP_PERSONALIZED_DELIVERIES = 1
  VERP_DELIVERY_INTERVAL = 1
  VERP_CONFIRMATIONS = 1

(The director (router) above is smart enough to deal with VERP bounces.)

       
Virtual Domains
---------------

One approach to handling virtual domains is to use a separate Mailman
installation for each virtual domain.  (Currently, this is the only way
to have lists with the same name in different virtual domains handled by
the same machine.)

In this case, the MAILMAN_HOME and MAILMAN_WRAP macros are useless --
you can remove them.  Change your director (router) to something like
this:

  require_files = /virtual/${domain}/mailman/lists/${lc:$local_part}/config.pck

and change your transport like this:

  command = /virtual/${domain}/mailman/mail/mailman \
            ${if def:local_part_suffix \
                 {${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}}
                 {post}} \
              $local_part
  current_directory = /virtual/${domain}/mailman
  home_directory = /virtual/${domain}/mailman


List Verification
-----------------

This is how a set of address tests for the Exim lists look on a working
system.  The list in question is quixote-users@mems-exchange.org, and
these commands were run on the mems-exchange.org mail server ("% "
indicates the Unix shell prompt):

  % exim -bt quixote-users
  quixote-users@mems-exchange.org
    router = mailman_main_router, transport = mailman_transport

  % exim -bt quixote-users-request
  quixote-users-request@mems-exchange.org
    router = mailman_router, transport = mailman_transport

  % exim -bt quixote-users-bounces
  quixote-users-bounces@mems-exchange.org
    router = mailman_router, transport = mailman_transport

  % exim -bt quixote-users-bounces+luser=example.com
  quixote-users-bounces+luser=example.com@mems-exchange.org
    router = mailman_router, transport = mailman_transport

If your "exim -bt" output looks something like this, that's a start: at
least it means Exim will pass the right messages to the right Mailman
commands.  It by no means guarantees that your Exim/Mailman installation
is functioning perfectly, though!


Document History
----------------

Originally written by Nigel Metheringham <postmaster@exim.org>.
Updated by Marc Merlin <marc_soft@merlins.org> for Mailman 2.1, Exim 4.
Overhauled/reformatted/clarified/simplified by Greg Ward <gward@python.net>.