Integrating amavisd-new in Postfix
Patrick Ben Koetter
<[1]patrick.koetter@state-of-mind.de>
Mark Martinec
<[2]Mark.Martinec+amavis@ijs.si>
License: GNU GENERAL PUBLIC LICENSE, Version 2, June 1991
+------------------------------------------------------------------------+
| Revision History |
|------------------------------------------------------------------------|
| Revision 141 | 11. Mar 2008 | PK |
|------------------------------------------------------------------------|
| Updated parameters that override Postfix defaults for the amavisdfeed |
| and the reentry smtpd servers |
|------------------------------------------------------------------------|
| Revision 139 | 11. Mar 2008 | PK |
|------------------------------------------------------------------------|
| Added corrections sent in from Chris Pepper |
|------------------------------------------------------------------------|
| Revision 122 | 15. Jun 2007 | PK |
|------------------------------------------------------------------------|
| Added Section on Advanced Configuration |
|------------------------------------------------------------------------|
| Revision 108 | 22. Apr 2007 | PK |
|------------------------------------------------------------------------|
| Initial publication |
+------------------------------------------------------------------------+
Table of Contents
[3]1. Requirements
[4]1.1. Which Postfix version is required?
[5]1.2. Catching errors during integration
[6]2. Basic Postfix and amavisd-new configuration
[7]2.1. Configuring amavisd-new for Postfix
[8]2.2. Configuring the transport from Postfix to amavisd-new
[9]2.3. Configuring a dedicated SMTP-server for message
reinjection
[10]2.4. Testing basic configuration
[11]3. Message filtering examples
[12]3.1. Filtering E-mail globally
[13]3.2. Filtering E-mail by Postfix service
[14]3.3. Filtering E-Mails per Recipient Domain
[15]3.4. Filtering E-Mails by Sender-Domain
[16]3.5. Filtering E-mail per Content
[17]4. Advanced Postfix and amavisd-new configuration
[18]4.1. Multiple cleanup service architecture
[19]4.2. Configuring two cleanup services
[20]5. Tuning
[21]5.1. Maximum Number of Concurrent Processes
[22]5.2. Additional Tips for Tuning
Abstract
This document describes how amavisd-new can be integrated into the Postfix
SMTP delivery process. It lists the necessary requirements, explains how
Postfix and amavisd-new need to be configured to basically work together
and it gives filter-examples to show how amavisd-new can be called from
Postfix.
1. Requirements
The following requirements must be met before integration can begin:
1. amavisd-new has already been installed and successfully tested.
2. Postfix has been installed, configured for basic operations and tested
successfully.
[23][Tip] Tip
Independently of the configuration examples shown in this
document, it is advisable to set strict_rfc821_envelopes = yes
in /etc/postfix/main.cf. Postfix will reject any message from
envelope-senders, whose address can't be used to send a reply
to.
This avoids accepting e-mails from erroneous envelope-senders
that can't be informed of problems, which finally would result
in deleting the message - even if Postfix claimed successful
delivery in the first.
1.1. Which Postfix version is required?
Integrating amavisd-new into the Postfix delivery process requires that
Postfix is able to delegate messages to external content filters. The
minimum version that provides content filtering is Postfix
release-20010228.
1.2. Catching errors during integration
Chances are that configuration errors during implementation cause Postfix
to bounce legitimate messages. Setting the soft_bounce parameter during
integration and reloading the Postfix configuration afterwards prevents
Postfix from bouncing legitimate mail during that time:
# postconf -e "soft_bounce = yes"
# postfix reload
As soon as soft_bounce has been activated Postfix will treat all delivery
errors as temporary errors - any client that wants to send messages to
Postfix will keep mail in the mailqueue and it will suspend delivery until
the soft_bounce parameter has been removed or set to no.
Once the integration of amavisd-new into the Postfix delivery process has
been completed successfully soft_bounce must be removed or Postfix will
not generate bounce messages for legitimate mail.
2. Basic Postfix and amavisd-new configuration
There are several moments at which Postfix can hand messages over to
amavisd-new (before it accepts a message from a client or after) and there
are different filter approaches (globally, per recipient (domain), per
network interface, etc.) that can trigger Postfix to transport a message
to amavisd-new.
The transport methods - transporting a message from Postfix to amavisd-new
and back - however always remain the same. They will be described in this
section first. The section that follows will deal with different filtering
approaches.
[24][Tip] Integration procedure
The following examples have been structured to cause minimum
trouble on an online mail system. The order of steps ensures
that filtering will be enabled at the very last moment. Several
tests will have been conducted to verify the delivery chain
works before the filter is enabled. Once enabled the complete
system should work at once.
2.1. Configuring amavisd-new for Postfix
Configuring amavisd-new to work with Postfix answers the following two
questions:
1. Which port should the amavisd-new daemon listen to for incoming
connections from Postfix?
2. Which IP-address and port should the amavisd-new SMTP client use to
(re)inject filtered messages (and notifications about message status)
into the Postfix SMTP delivery system?
2.1.1. Configuring amavisd-new for incoming connections
The $inet_socket_port parameter in /etc/amavisd.conf sets the port number
where amavisd-new will listen for incoming (E)SMTP connections. The
following example explicitly configures amavisd-new to bind to port 10024
(default setting undef):
$inet_socket_port = 10024;
2.1.2. Configuring the reinjection path
Two parameters, $forward_method and $notify_method, need to be configured
(usually identically) to reinject messages into the Postfix mail system.
The first parameter, $forward_method, specifies where amavisd-new should
transport scanned messages to, while the second parameter, $notify_method,
specifies where notifications about scanned messages should be transported
to.
By default amavisd uses 127.0.0.1 on port 10025 to contact a SMTP server
for reinjection of filtered messages. Unless a different IP address or
port should be used, no modifications must be applied and this section can
be skipped.
In case a different IP address or port should be used, the parameters
$notify_method and $forward_method need to be adjusted to reflect these
requirements. The following example edits these parameters in
/etc/amavisd.conf and uses 192.0.2.1 as IP address and port 20025:
$notify_method = 'smtp:[192.0.2.1]:20025';
$forward_method = 'smtp:[192.0.2.1]:20025';
2.2. Configuring the transport from Postfix to amavisd-new
Both amavisd-new and Postfix are able to use either SMTP- or
LMTP-communication to transport a message from Postfix to amavisd-new.
Both variants will be described in this section.
Why configure a dedicated service?
Theoretically it's possible to transport messages from Postfix to
amavisd-new using the existing smtp-, lmtp, or even the relay-service in
/etc/postfix/master.cf.
In practice transporting messages to amavisd-new requires imposing
transport limits on the transporting service. Imposing such limits on a
globally available service would impose these limits on the complete
Postfix mail system - it would slow down the system significantly and
should be avoided.
[25][Note] Note
The number of Postfix clients that may connect simultaneously
to amavisd-new instances must be limited to the maximum number
of daemon child processes amavisd-new starts.
If the Postfix transport client was allowed to open more
connections than amavisd-new can handle, amavisd-new would
start to queue incoming Postfix connections. Postfix in turn
would interpret such behaviour as "unresponsive remote MTA" and
would itself begin to queue mail that should be filtered. All
this would possibly throttle down the complete system and all
further filtering attempts would suffer.
2.2.1. Configuring a dedicated lmtp-client
The following example creates a new, dedicated lmtp-transport named
amavisfeed in /etc/postfix/master.cf. Its configuration details are
explained following the listing:
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (yes) (never) (100)
# ==========================================================================
...
amavisfeed unix - - n - 2 lmtp
-o lmtp_data_done_timeout=1200
-o lmtp_send_xforward_command=yes
-o lmtp_tls_note_starttls_offer=no
[26][Important] Important
A noteworthy quote from the Postfix documentation: "...do
not specify whitespace around the `='. In parameter
values, either avoid whitespace altogether, ...". Further
details on master.cf configuration syntax can be found in
master.cf or master(5).
Here's a quick rundown on the settings that differ from other services
defaults:
maxproc
The maximum number of concurrent Postfix amavis-service processes
has been limited to 2 (default: default_process_limit = 100). This
value reflects the default of 2 amavisd-daemon children processes
and is a good setting to start from. The value may be raised
later, when the system works stable and still can take a higher
load. It should not exceed the number of simultaneous amavisd
child processes.
lmtp_data_done_timeout
Setting lmtp_data_done_timeout to 1200 (seconds) doubles the
default time span a regular Postfix client waits after message
delivery for the server to reply DONE to claim successful
delivery. It must be larger than amavisd setting $child_timeout
(default 8*60 seconds) and should add a sufficient safety margin,
for example to cater for periods of automatic database maintenance
(e.g. bayes database on non-SQL database types) which can take a
long time in some cases.
If the server does not reply within the configured time span, the
Postfix client will quit the connection, put the message into the
deferred queue, log a delivery failure and retry later to
transport the message to amavisd-new.
[27][Note] Note
Raising this value serves a trick amavisd uses to avoid
message loss in case of power outage etc. The trick
consists in keeping the incoming connection as long
open as it takes to filter the message and take
appropriate action (reinjection, notification,
quarantine, etc.).
Only when the message (or notifications etc.) has been
reinjected amavisd will send DONE to the client and the
client will close the connection. This way Postfix will
always keep the message in its own mail queue, where it
can be reactivated after a system failure.
lmtp_send_xforward_command
Enabling lmtp_send_xforward_command configures the Postfix
lmtp-client to forward the original clients HELO name and IP
address to amavisd-new. amavisd-new in turn can use this
information for
* logging and notifications (macro %a)
* switching policy banks (MYNETS, @mynetworks_maps)
* pen pals functionality
* p0f fingerprinting
lmtp_tls_note_starttls_offer
Starting with version 2.6 amavisd-new can offer TLS to a smtp- or
lmtp-client. This option cuts down unnecessary logging by Postfix,
just in case logging TLS session offers has been enabled globally
setting lmtp_tls_note_starttls_offer in Postfix main.cf
configuration file.
2.2.2. Configuring a dedicated smtp-client
Configuring a dedicated smtp-client is almost identical to configuring a
dedicated lmtp-client. The syntax differences in detail are that the names
of parameters start with smtp_ instead of lmtp_ and that the command at
the end of the service invokes the smtp- and not lmtp-client. The same
reasons given for differing lmtp client options apply to the dedicated
smtp client configuration.
Here's an example of a dedicated smtp client given the service name
amavisfeed:
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (yes) (never) (100)
# ==========================================================================
...
amavisfeed unix - - n - 2 smtp
-o smtp_data_done_timeout=1200
-o smtp_send_xforward_command=yes
-o smtp_tls_note_starttls_offer=no
2.3. Configuring a dedicated SMTP-server for message reinjection
The second service that needs to be added to the Postfix mail system is a
dedicated SMTP-server. It will exist only to accept filtered messages and
notifications from amavisd-new to transported them closer to their final
destination.
This dedicated smtpd server will differ in many aspects from the default
smtpd daemon. The most important difference is that it configures an empty
content_filter parameter, thus overriding any global external content
filtering settings in Postfix.
[28][Note] Note
Delegating messages to an external content filter in Postfix is
done using the content_filter parameter. If the dedicated
smtpd-daemon would not override any global content_filter
settings, the reinjected message would be sent of to the
external content filter again - the mail would end in an
endless loop.
The following Postfix example uses amavisd-new default settings taken from
the $forward_method and $notify_method parameters. These settings
configure amavisd-new to forward filtered messages and notifications to
127.0.0.1 on port 10025; the Postfix smtpd daemon will be configured to
bind to that IP address and listen on the specified port for incoming
connections:
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (yes) (never) (100)
# ==========================================================================
...
127.0.0.1:10025 inet n - n - - smtpd
-o content_filter=
-o smtpd_delay_reject=no
-o smtpd_client_restrictions=permit_mynetworks,reject
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o smtpd_data_restrictions=reject_unauth_pipelining
-o smtpd_end_of_data_restrictions=
-o smtpd_restriction_classes=
-o mynetworks=127.0.0.0/8
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
-o smtpd_client_connection_count_limit=0
-o smtpd_client_connection_rate_limit=0
-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters
-o local_header_rewrite_clients=
-o smtpd_milters=
-o local_recipient_maps=
-o relay_recipient_maps=
Here's a quick rundown on the settings that differ from smtpd defaults:
content_filter
The empty content_filter overrides other, globally set
content_filter delegations.
..._maps
Empty ..._maps override any other globally set map lookups.
Procedures to enforce settings specified in such maps have already
taken place when Postfix accepted the message from the external
client. Doing them again will not produce new results but only
waste resources.
..._restrictions_...
There's no need to apply any already enforced ..._restrictions_...
another time. It would also only waste resources.
mynetworks
To avoid abuse from remote hosts, the dedicated smtpd-daemon will
only allow clients from 127.0.0.0/8 to relay messages.
local_header_rewrite_clients
By default this option would "rewrite message header addresses in
mail from these clients and update incomplete addresses with the
domain name". If such action has already been taken by Postfix
before the message went off to amavis, it should not be done a
second time when it reenters the Postfix mail system. Leaving this
option empty disables local header rewrites and saves resources.
remaining options
All remaining options either configure the dedicated smtpd-daemon
to be more failure tolerant or exist to avoid unnecessary use of
resources.
Running the postfix reload will activate the new transports (Postfix will
not yet send regular mail to amavisd). Combined with the tail command
problems can easily be detected:
# postfix reload && tail -f /var/log/maillog
If there are no problems reported, basic configuration can be tested.
2.4. Testing basic configuration
Testing basic configuration consists of three separate tests, starting at
the end of the new delivery chain and working to it's beginning. Their
goal is to answer the following questions:
1. Will amavisd-new accept connections at the specified IP address and
port?
2. Will the new dedicated smtpd-daemon accept connections at the
specified IP address and port?
3. Will a test message, injected into amavisd-new, be filtered, sent to
Postfix and delivered into a mailbox?
2.4.1. Testing amavisd's host and port
A test, using the telnet command, serves to verify that amavisd listens on
the specified IP address and port. A successful connection looks like
this:
$ telnet localhost 10024
220 [127.0.0.1] ESMTP amavisd-new service ready
EHLO localhost
250-[127.0.0.1]
250-VRFY
250-PIPELINING
250-SIZE
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250 XFORWARD NAME ADDR PROTO HELO
QUIT
221 2.0.0 [127.0.0.1] amavisd-new closing transmission channel
If the test fails, the following questions may help to debug the problem:
* Is the amavisd-new daemon running?
* Does amavisd-new write an error to the log?
* Do the IP address and port number specified in the amavisd-new
configuration match the values used during the test?
* Does a firewall intercept connections?
2.4.2. Testing the dedicated Postfix smtpd-daemon
When Postfix was reloaded, the new, dedicated smtpd-daemon
(127.0.0.1:10025) should have been activated. A successful connection
looks like this:
$ telnet 127.0.0.1 10025
220 mail.example.com ESMTP Postfix (2.3.2)
EHLO localhost
250-mail.example.com
250-PIPELINING
250-SIZE 40960000
250-ETRN
250-STARTTLS
250-AUTH PLAIN CRAM-MD5 LOGIN DIGEST-MD5
250-AUTH=PLAIN CRAM-MD5 LOGIN DIGEST-MD5
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
QUIT
221 2.0.0 Bye
If the test fails, the following questions may help to debug the problem:
* Is the Postfix master daemon running?
* Does Postfix write an error to the log?
* Do the IP address and port number specified in the new services
configuration match the values used during the test?
* Does a firewall intercept connections?
2.4.3. Testing the new transport chain
This test proves amavisd accepts e-mail as specified in [29]Section 2.1,
"Configuring amavisd-new for Postfix", filters it and finally hands it
over to Postfix' dedicated smtpd-daemon as specified in [30]Section 2.3,
"Configuring a dedicated SMTP-server for message reinjection".
The following example uses the content of test-messages/sample-nonspam.txt
from the amavisd test-messages to send an e-mail:
$ telnet localhost 10024
220 [127.0.0.1] ESMTP amavisd-new service ready
HELO localhost
250 [127.0.0.1]
MAIL FROM: <>
250 2.1.0 Sender OK
RCPT TO: <postmaster>
250 2.1.5 Recipient postmaster OK
DATA
354 End data with <CR><LF>.<CR><LF>
From: virus-tester
To: undisclosed-recipients:;
Subject: amavisd test - simple - no spam test pattern
This is a simple test message from the amavisd-new test-messages.
.
250 2.6.0 Ok, id=30897-02, from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 079474CE44
QUIT
221 2.0.0 [127.0.0.1] amavisd-new closing transmission channel
The maillog shows the delivery path. Here's an excerpt from a successful
delivery process:
Nov 1 11:28:10 mail postfix/smtpd[30986]: connect from localhost[127.0.0.1] [31]1
Nov 1 11:28:10 mail postfix/smtpd[30986]: 079474CE44: client=localhost[127.0.0.1]
Nov 1 11:28:10 mail postfix/cleanup[30980]: 079474CE44: message-id=<20061101102810.079474CE44@mail.example.com>
Nov 1 11:28:10 mail postfix/qmgr[20432]: 079474CE44: from=<>, size=822, nrcpt=1 (queue active)
Nov 1 11:28:10 mail amavis[30897]: (30897-02) Passed BAD-HEADER, <> -> <postmaster>, quarantine: badh-le5gjszxowBk, mail_id: le5gjszxowBk, Hits: -1.76, queued_as: 079474CE44, 39505 ms [32]2
Nov 1 11:28:10 mail postfix/smtpd[30986]: disconnect from localhost[127.0.0.1]
Nov 1 11:28:10 mail postfix/local[30987]: 079474CE44: to=<postmaster@example.com>, relay=local, delay=0.27, delays=0.14/0.05/0/0.08, dsn=2.0.0, status=sent (delivered to mailbox: postmaster) [33]3
Nov 1 11:28:10 mail postfix/qmgr[20432]: 079474CE44: removed
[34]1 amavisd connects with Postfix dedicated smtpd-daemon and hands over
the e-mail that had been sent during the telnet session. smtpd gives
a queue-id of 079474CE44 that can be tracked throughout the maillog.
[35]2 amavisd notices it has checked and sent an e-mail to <postmaster>.
[36]3 Postfix' local-service logs it successfully delivered an e-mail with
queue-id 079474CE44 to the mailbox of postmaster.
If the test fails, the following questions may help to debug the problem:
* Does amavisd-new log errors?
* Does running amavisd-new in debug-mode report errors?
3. Message filtering examples
Postfix can use various criteria to decide whether a message should be
sent to amavisd-new for examination. Combinations of criteria may serve to
create different configurations. The following section describes the
following configurations:
* Filtering e-mail globally
* Filtering e-mail globally by service
* Filtering e-mail per recipient domain
* Filtering e-mail per sender domain
* Filtering e-mail by content
3.1. Filtering E-mail globally
In most cases email policies require global filtering - every inbound and
every outbound e-mail must be filtered by amavisd-new - before it may be
sent closer to its final destination.
[37][Note] Why check outgoing mail traffic?
Some reasons for checking mail coming from internal networks or
from authenticated roaming users are:
* detect an internal infected PC which is sending viruses
* detect an internal zombiized PC (or an internal open relay
or proxy) which is sending or relaying spam
* let the SpamAssassin Bayes autolearning feature see a
balanced view of all mail, including useful samples of
non-spam originating from inside
* make it possible for pen pals feature to function (if
enabled)
In Postfix global settings for its services are written to main.cf. The
content_filter parameter, the parameter configuring that messages are sent
to amavisd-new, must therefore be placed in main.cf.
The content_filter parameter requires a triplet, consisting of the
transport service's name (here: amavisfeed, given in [38]Section 2.2.1,
"Configuring a dedicated lmtp-client"), the target hosts IP address and
the port where amavisd-new listens for incoming connections. Following the
values used in this documents examples the content_filter configuration
results in this:
content_filter=amavisfeed:[127.0.0.1]:10024
The new external content filter will be activated once Postfix has been
reloaded. Sending a test-mail verifies the system works.
3.2. Filtering E-mail by Postfix service
Postfix is able to filter messages per service. Such configuration
requires the content_filter not to be applied globally to all services in
main.cf (see: [39]Section 3.1, "Filtering E-mail globally"), but
selectively, per service in master.cf.
The following example presumes Postfix runs on a system offering three IP
addresses. In this example these are: 192.0.2.1 (WAN), 127.0.0.1
(localhost) and 10.0.0.254 (LAN). The goal is to filter only e-mail that
enters from the WAN interface.
This requires to create three dedicated smtpd-daemon instances, each
binding to one of the given IP addresses and deactivating the global smtp
service calling the smtpd command.
Additionally the WAN interface (here: 192.0.2.1:25) is configured to use
content_filter =amavisfeed:[127.0.0.1]:10024 - it will delegate any
message that enters the Postfix mail system at this service to the
external amavisd content filter.
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (yes) (never) (100)
# ==========================================================================
# smtp inet n - n - - smtpd
...
192.0.2.1:25 inet n - n - - smtpd
-o content_filter=amavisfeed:[127.0.0.1]:10024
-o receive_override_options=no_address_mappings
10.0.0.254:25 inet n - n - - smtpd
127.0.0.1:10025 inet n - n - - smtpd
-o content_filter=
-o smtpd_delay_reject=no
-o smtpd_client_restrictions=permit_mynetworks,reject
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o smtpd_data_restrictions=reject_unauth_pipelining
-o smtpd_end_of_data_restrictions=
-o smtpd_restriction_classes=
-o mynetworks=127.0.0.0/8
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
-o smtpd_client_connection_count_limit=0
-o smtpd_client_connection_rate_limit=0
-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters
-o local_header_rewrite_clients=
-o smtpd_milters=
-o local_recipient_maps=
-o relay_recipient_maps=
3.3. Filtering E-Mails per Recipient Domain
Postfix is able to filter e-mails per recipient domain. In order to do
this the content_filter parameter must not be set globally (see:
[40]Section 3.1, "Filtering E-mail globally"). Instead the content_filter
parameter has to be associated with one or more recipient domains listed
in a lookup table (map).
[41][Caution] Caution
This filter method is not selective! It will send any mail
with a recipient domain listed in the lookup table to amavis
even if the mail contains another recipient that should not
be examined by the amavis framework.
If fully selective rules are required all mail should be
sent to amavis and amavis' own rule sets should be
configured to decide whether a message for a given recipient
should be examined or not.
When Postfix searches the lookup table and finds the recipients domain
listed as key, it will take the action associated with that domain. The
action will send the message to a FILTER amavisfeed:[127.0.0.1]:10024.
The following map /etc/postfix/filter_recipient_domains specifies to send
messages to the FILTER amavisfeed whenever a message for any recipient at
example.com enters the Postfix mailqueues:
example.com FILTER amavisfeed:[127.0.0.1]:10024
Once the table has been created the postmap command must be used to create
an indexed map Postfix can read:
# postmap /etc/postfix/filter_recipient_domains
Once the map has been indexed, the postmap command is used to test the
map. In the following example the postmap command queries for the domain
example.com and returns the associated action:
# postmap -q "example.com" /etc/postfix/filter_recipient_domains
FILTER amavisfeed:[127.0.0.1]:10024
The tested map must be added to main.cf, before Postfix can make use of
the new filter policy. Setting the check_recipient_access parameter in the
list of smtpd_recipient_restrictions triggers evaluation of entries in the
map - check_recipient_access is triggered by the envelope-recipient(s)
given by a SMTP-client in a SMTP-session with Postfix.
The following example puts the check_recipient_access rule before
permit_mynetworks - all clients envelope-recipient(s) will be filtered:
smtpd_recipient_restrictions =
...
check_recipient_access hash:/etc/postfix/filter_recipient_domains
...
permit_mynetworks
reject_unauth_destination
...
Filtering E-Mails per Recipient Domain only from External Clients
This example puts the check_recipient_access rule after permit_mynetworks
- only messages sent from clients that are not in Postfix $mynetworks list
(external or untrusted clients) will be filtered:
smtpd_recipient_restrictions =
...
permit_mynetworks
reject_unauth_destination
check_recipient_access hash:/etc/postfix/filter_recipient_domains
...
3.4. Filtering E-Mails by Sender-Domain
In general it doesn't make sense to filter e-mails by sender-domain, as
anyone can fake a sender-domain during a SMTP-session. Filtering by
sender-domain will probably only make sense, if messages are not filtered
globally, but e-mails from ones own domain should be checked for spam or
viruses before they leave the network.
Most of the configuration steps are identical with the ones noted in
[42]Section 3.3, "Filtering E-Mails per Recipient Domain", except for the
parameter that triggers evaluation of the indexed map. In this scenario
envelope-senders should trigger map evaluation. The map, named
/etc/postfix/filter_sender_domains this time, contains the sender domain
(example.com) and associates it with the required FILTER:
example.com FILTER amavisfeed:[127.0.0.1]:10024
Once the map has been converted and tested with the postmap command (see:
[43]Section 3.3, "Filtering E-Mails per Recipient Domain") it must be
added to the list of smtpd_recipient_restrictions using the
check_sender_access parameter:
smtpd_recipient_restrictions =
...
check_sender_access hash:/etc/postfix/filter_sender_domains
...
permit_mynetworks
reject_unauth_destination
...
[44][Important] Important
The map must be listed before permit_mynetworks, because
only then it will be applied to all clients - even the
ones Postfix trusts, which are very likely the ones from
example.com.
3.5. Filtering E-mail per Content
Postfix is able - with deliberate limitations (see: BUILTIN_FILTER_README)
- to search for strings in headers, the body and MIME-headers. If a string
matches, Postfix may call appropriate action.
The following example configures Postfix to look for the string offer in
Subject:-headers and delegate the message to an external content filter if
if finds a matching string.
A map, consisting of the search string noted as regexp-expression,
associates the search pattern with a FILTER action:
/^Subject:.*offer/ FILTER amavisfeed:[127.0.0.1]:10024
[45][Note] Indexing regexp- or pcre-maps?
regexp- or pcre-maps are and must be plaintext files. They must
not and cannot be converted to an indexed map using the postmap
command. They can be tested using the postmap command using the
-q command line option.
Once the map has been created, Postfix must be configured to use it. The
following example uses the header_checks parameter (not body_checks or
mime_header_checks as they apply to other message parts) to implement the
map into the Postfix delivery process:
header_checks = regexp:/etc/postfix/filter_header
Once Postfix has been reloaded it will send every e-mail that contains the
word offer in the Subject:-header off to the external amavisd content
filter.
4. Advanced Postfix and amavisd-new configuration
In a post-queue content filtering setup, a mail message passes through
smtpd and cleanup Postfix services twice, once before a content filter,
and the second time when an approved message is reinjected from a content
filter into the Postfix mail system. This is because checks and
transformations that have been configured in main.cf are globally active
and will be loaded and run by any instance of these two services. To avoid
wasting resources, options that control runtime behavior of these services
should not be applied globally in main.cf, but selectively to separate
instances of these services in master.cf.
Checks and transformations which are performed by a smtpd Postfix service
itself, e.g. access controls, recipient validation, milters etc., can be
controlled by adding options (-o) to appropriate smtpd services. This has
been shown in the basic configuration examples (see: [46]Section 2.3,
"Configuring a dedicated SMTP-server for message reinjection").
Checks and transformations which are performed by a cleanup Postfix
service are trickier because in a normal Postfix setup there is only one
cleanup service, unlike smtpd services of which there are many. Some of
the more important cleanup settings are dynamically controllable by a
smtpd service through the use of its receive_override_options option.
[47][Tip] Transformations and checks
Any transformation should preferably only be performed once,
either before or after content filtering. When to transform
depends on the desired effect, for example whether a content
filter should see unchanged or modified mail messages. Typical
transformations are:
* rewrite addresses
* add BCC recipients
* modify mail header.
Most checks should also be performed only once, preferably only
on the first passage, when the mail enters the Postfix mail
system the first time. This way messages can be rejected early -
if needed - and will not tie up downstream resources. Checking
early also avoids bounces in case of negative check results on a
second passage after content filtering.
4.1. Multiple cleanup service architecture
To gain more control over a cleanup service than offered by
receive_override_options, two (or more) cleanup services, each with its
own set of options, must be run. A Postfix setup with more than one
cleanup service is possible either with two separate Postfix instances, or
through a specification of services and their options in master.cf file of
a single Postfix instance.
The following diagram illustrates a setup with two cleanup services in a
single Postfix instance:
.......................................
: Postfix :
----->smtpd \ :
: -pre-cleanup-\ /local---->
---->pickup / -queue- :
: -cleanup-/ | \smtp----->
: bounces/ ^ v :
: and locally | v :
: forwarded smtpd amavisfeed :
: messages 10025 | :
...........................|...........
^ |
| v
............|...............................
: | $inet_socket_port=10024 :
: | :
: $forward_method='smtp:[127.0.0.1]:10025' :
: $notify_method ='smtp:[127.0.0.1]:10025' :
: :
: amavisd-new :
............................................
Procedure 1. Message flow with two cleanup services
1. Messages enter the Postfix system at the regular smtpd or pickup
service.
2. The pre-cleanup cleanup service performs transformations and checks on
these messages.
3. The qmgr service schedules the messages to be sent to the amavisd-new
content filter.
4. amavisd-new performs various tests on the messages.
5. Messages are re-injected into the Postfix mail system, sending them to
a dedicated, local smtpd service.
6. The cleanup cleanup service performs transformations and checks that
must be done at this stage, but omits the ones that have already been
carried out in step 2.
4.2. Configuring two cleanup services
Configuring Postfix smtpd services to use two separate, dedicated cleanup
services requires the following steps:
1. Create a second cleanup instance
2. Modify the existing cleanup service
3. Configure smtpd services to use either of the two cleanup services.
4.2.1. Creating a second cleanup instance
The following example adds a cleanup daemon named pre-cleanup. It will
handle messages before a content filter.
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (yes) (never) (100)
# ==========================================================================
# smtp inet n - n - - smtpd
...
pre-cleanup unix n - n - 0 cleanup
-o virtual_alias_maps=
The above leaves canonicalization address rewriting enabled so that a
content filter will see canonicalized (external) sender mail addresses,
but it disables globally configured virtual alias transformations.
Such transformations will be done later by the second cleanup service, so
that a content filter will see original (external) recipient mail
addresses. Other options may also be used as needed.
4.2.2. Modifying the existing cleanup service
The already existing cleanup service - having the service name cleanup -
will be used to process messages that re-enter the Postfix mail system
(also for delivery notifications and forwarding as generated internally by
Postfix).
Cleanup jobs that already have been performed by the pre-cleanup service
should not be run again. The following example disables typical checks
that have been run before or are not needed for internally generated
notifications:
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (yes) (never) (100)
# ==========================================================================
# smtp inet n - n - - smtpd
...
cleanup unix n - n - 0 cleanup
-o mime_header_checks= [48]1
-o nested_header_checks= [49]2
-o body_checks= [50]3
-o header_checks= [51]4
[52]1 The specified options disable header and body checks as these would
[53]2 already be performed by a pre-cleanup service.
[54]3
[55]4
[56][Note] always_bcc
This cleanup service would also be the appropriate one for
specifying always_bcc option - doing it globally would apply to
both cleanup services and would result in two copies of each
message to be sent to the specified address.
4.2.3. Configuring smtpd services
Finally existing smtpd services on ports 25 and 587 (submission), and the
pickup service must be configured to send messages to the new pre-cleanup
service instead of a default cleanup service:
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (yes) (never) (100)
# ==========================================================================
# smtp inet n - n - - smtpd
...
pickup fifo n - n 60 1 pickup
-o cleanup_service_name=pre-cleanup
smtp inet n - n - - smtpd
-o cleanup_service_name=pre-cleanup
submission inet n - n - - smtpd
-o cleanup_service_name=pre-cleanup
5. Tuning
5.1. Maximum Number of Concurrent Processes
The most important settings to tune and optimize in Postfix and amavisd
workflow are the maximum number of concurrent processes. The maximum
number of concurrent processes on both sides must be chosen with care.
If the number is too low, hardware resources aren't used efficiently and
delivery time will be unnecessarily prolonged. Experience tells that
raising the number of processes a little, will not raise the overall
throughput in the same proportion.
As the system resources are nearing saturation with each increase of the
number of processes, an increase in throughput becomes marginal, and
eventually even negative when the number of processes exceeds its
near-optimum value. E-mail throughput will decrease, because processes
need to wait for each other. At worst e-mail delivery stalls.
Best practice is to start with a (conservative) maximum number of 2
concurrent processes. Everyday use has shown that this value may be raised
to a value between 10 and 30 concurrent Postfix client and amavisd server
processes. This also depends on the overall resources the system may
provide, how amavisd has been integrated into the Postfix delivery process
and on the anti-virus and anti-spam software being loaded and used by
amavisd-new.
Regardless of the maximum number of concurrent processes, both sides -
Postfix and amavisd - should be synchronized. To synchronize both sides
edit, the $max_servers parameter for amavisd-new (see: amavisd.conf) and
the number of processes in master.cf listed in the dedicated transports
maxproc column for Postfix.
Both values should be identical for two reasons: If amavisd-new offers
more processes than Postfix will ever use, amavisd-new wastes resources.
On the other hand, if Postfix starts more dedicated transports than
amavisd can handle simultaneously, e-mail transport will be refused and
logged as error.
[57][Note] Controlling the maximum number of concurrent processes in
main.cf
Instead of controlling the maximum number of concurrent
processes of Postfix' dedicated transport in master.cf it is
also possible to keep the default setting - in master.cf and
set the following parameter and option in main.cf:
amavisfeed_destination_concurrency_limit = 2
The name of the parameter starts with the service in master.cf
(here: amavisfeed) that should be controlled and goes on with
the suffix _destination_concurrency_limit. Here also 2 is set
as initial (conservative) value.
5.2. Additional Tips for Tuning
Further Tuning-Tips can be found in README.performance and the slides from
[58]amavisd-new, advanced configuration and management.
References
Visible links
58. http://www.ijs.si/software/amavisd/amavisd-new-magdeburg-20050519.pdf
