Version 4.21 ------------ 1. There is a new log selector called smtp_incomplete_transaction. When a mail transaction is aborted by RSET, QUIT, loss of connection, or otherwise, the incident is logged, and the message sender plus any accepted recipients are included in the log line. This can provide evidence of dictionary attacks. 2. If the option tls_remember_esmtp is set true, Exim violates the RFCs by remembering that it is in "esmtp" state after successfully negotiating a TLS session. This provides support for broken clients that fail to send a new EHLO after starting a TLS session. 3. When an ACL is read dynamically from a file (or anywhere else), the lines are now processed in the same way as lines in the Exim configuration file. In particular, continuation lines are supported, which was not the case previously. 4. The facility for constraining the result of dnslists lookups to certain IP addresses has been extended. The original facility uses a positive list, like this: deny dnslists = a.b.c=127.0.0.2,127.0.0.3 This example means "deny if the host is in the black list at the domain a.b.c and the IP address yielded by the list is 127.0.0.2 or 127.0.0.3". It is now possible to supply a negative list, like this: deny dnslists = a.b.c!=127.0.0.2,127.0.0.3 This example means "deny if the host is in the black list at the domain a.b.c and the IP address yielded by the list is not 127.0.0.2 and not 127.0.0.3". In other words, the result of the test is inverted. Note that this kind of negation is not the same as negation in a domain, host or address list (which is why the syntax is different). If you are using just a single list, the new syntax does not gain you much. The previous example is precisely equivalent to deny dnslists = a.b.c !dnslists = a.b.c=127.0.0.2,127.0.0.3 However, if you are using multiple lists, the new syntax is clearer. Consider this example: deny dnslists = sbl.spamhaus.org : \ list.dsbl.org : \ dnsbl.njabl.org!=127.0.0.3 : \ relays.ordb.org Using only positive lists, this would have to be deny dnslists = sbl.spamhaus.org : \ list.dsbl.org deny dnslists = dnsbl.njabl.org !dnslists = dnsbl.njabl.org=127.0.0.3 deny dnslists = relays.ordb.org which is less clear, and harder to maintain. 5. The command line option -ti is equivalent to -t -i (this is for Sendmail compabitility). 6. Sendmail's -Ooption=value command line option is now ignored. 7. When execve() failed while trying to run a command in a pipe transport, Exim was returning EX_UNAVAILBLE (69) from the subprocess. However, this could be confused with a return value of 69 from the command itself. This has been changed to 127, the value the shell returns if it is asked to run a non-existent command. The wording for the related log line suggests a non-existent command as the problem. 8. If received_header_text expands to an empty string, no Received: header line is added to the message. 9. eximstats 1.82 has a new option -nt that allows certain transports to be omitted. This is useful for omitting spamassassin transport (for example) because otherwise each message gets counted twice. 10. If a DNS black list returns more than one A record, all the IP addresses are now included in the value of the $dnslist_value variable, comma-space separated. 11. The idea of the "warn" ACL verb is that it adds a header or writes to the log only when "message" or "log_message" are set. However, if one of the conditions was a negated address verification or a call to a nested ACL, the messages generated by the failing underlying test were being passed through. This no longer happens. The underlying message is available in $acl_verify_ message for both "message" and "log_message" expansions, so it can be passed through if needed. 12. This isn't a new feature: it's a note of a bug in the manual. In the description of $domain in section 11.8, it says "Whenever a domain list is being scanned, $domain contains the subject domain." This statement is NOT true when the domain list in a "sender_domains" condition in an ACL is being processed. It works this way so that, in a RCPT ACL, the sender_domains list can be dependent on the recipient domain (which is what is in $domain) if required. The sender's domain is in $sender_address_domain. 13. The way that the $h_ (and $header_) expansions work has been changed by the addition of RFC 2047 decoding. There are now three forms: $rh_xxx: and $rheader_xxx: give the original content of the header line(s), with no processing at all. $bh_xxx: and $bheader_xxx: remove leading and trailing white space, and then decode base64 or quoted-printable "words" within the header text, but do not do charset translation. If decoding of what looks superficially like a "word" fails, the raw string is returned. If decoding produces a binary zero character, it is replaced by a question mark - this is what Exim does for binary zeros that are actually received in header lines. $h_xxx: and $header_xxx: attempt to translate the $bh_ string to a standard character set. If this translation fails, the $bh_ string is returned. Translation is attempted only on operating systems that have the iconv() function. This is indicated by the compile-time macro HAVE_ICONV. In a filter file, the target charset can be specified by a command of the following form: headers charset "UTF-8" This command affects all references to $h_ (or $header_) in subsequently obeyed commands. In the absence of this command, the target charset in a filter is taken from the setting of the headers_charset option in the runtime configuration. The value of this option defaults to the value of HEADERS_CHARSET in Local/Makefile. The ultimate default is "ISO-8859-1". Some of the operating systems that supply iconv() do not support very many conversions. The GNU libiconv library (available from http://www.gnu.org/software/libiconv/) can be installed on such systems to remedy this deficiency, as well as on systems that do not supply iconv() at all. After installing libiconv, you should add HAVE_ICONV=yes to your Local/Makefile. 14. There is now support for Sieve filters (RFC 3028). Initial documentation is in doc/README.SIEVE. The RFC itself is also provided in the doc directory. Note that you cannot use a Sieve filter as a system filter, only as a per-recipient filter. The reason for this is that Sieve filters always specify what is to be done with a message (redirect, save, or discard). There is no mechanism for specifying "carry on and do the normal delivery". 15. The facility for testing for specific IP addresses in the yields of DNS black list lookups has been extended. The existing facility is to look for specific addresses by settings such as dnslists = a.b.c=127.0.0.1,127.0.0.2 If the character '&' is used instead of '=', the comparison for each listed IP address is done by a bitwise 'and' instead of by an equality test. In other words, the listed addresses are used as bit masks. The comparison is true if all the bits in the mask are present in the address that is being tested. For example, dnslists = a.b.c&0.0.0.3 matches if the address is x.x.x.3, x.x.x.7, x.x.x.15, etc. If you want to test whether one bit or another bit is present (as opposed to both being present), you must use multiple values. For example: dnslists = a.b.c&0.0.0.1,0.0.0.2 matches if the final component of the address is an odd number or two times an odd number. 16. The smtp_printf() function is now available for (careful) use in local_scan() functions. Its arguments are like printf(), but it writes to the SMTP output stream. You should use this function only when there *is* an SMTP output stream, that is, when the incoming message has arrived via interactive SMTP. Two boolean global variables are available to enable you to test for this case: smtp_input is TRUE for all SMTP input smtp_batched_input is TRUE for BSMTP input Thus, the use of smtp_printf() should be restricted to cases where smtp_input is TRUE and smtp_batched_input is FALSE. If you want to test for an incoming message from another host, you can test the value of sender_host_address, which is non-NULL in this case. If an SMTP TLS connection is established, smtp_printf() uses the TLS output function, so it can be used for all forms of SMTP connection. The strings that are written by smtp_printf() from local_scan() must start with an appropriate response code: 550 if you are going to return LOCAL_SCAN_REJECT, 451 if you are going to return LOCAL_SCAN_TEMPREJECT, and 250 otherwise. Since you are writing the initial lines of a multi- line response, the code must be followed by a hyphen to indicate that the line is not the final response line. You must also ensure that the lines you write terminate with CRLF. For example: smtp_printf("550-this is some extra info\r\n"); Note that you can also create multi-line responses by including newlines in the data returned via the return_text argument. The added value of using smtp_printf() is that, for instance, you could introduce delays between each output line. The smtp_printf() function does not return any error indication, because it does not automatically flush pending output, and therefore does not test the state of the stream. (In the main code of Exim, flushing and error detection is done when Exim is ready for the next SMTP input command.) If you want to flush the output and check for an error (for example, the dropping of a TCP/IP connection), you can call smtp_fflush(), which has no arguments. It flushes the output stream, and returns a non-zero value if there is an error. 17. The smtp transport has a new option called hosts_avoid_esmtp. This is for use with broken hosts that announce ESMTP facilities (for example, PIPELINING) and then fail to implement them properly. When a host matches hosts_avoid_esmtp, Exim sends HELO rather than EHLO at the start of the SMTP session. This means that it cannot use any of the ESMTP facilities such as AUTH, PIPELINING, SIZE, and STARTTLS. 18. Following a discussion on the list, the rules by which Exim recognises line endings on incoming messages have been changed. The -dropcr and drop_cr options are now no-ops, retained only for backwards compatibility. The following line terminators are recognized: LF CRLF CR. However, special processing applies to CR: (i) The sequence CR . CR does *not* terminate an incoming SMTP message, nor a local message in the state where . is a terminator. (ii) If a bare CR is encountered in a header line, an extra space is added after the line terminator so as not to end the header. The reasoning behind this is that bare CRs in header lines are most likely either to be mistakes, or people trying to play silly games. 19. A new feature has been added to the processing of lists of hosts by the manualroute and queryprogram routers. An item in those lists can now take the form of a domain name followed by "/MX". This notation is an indirect way of specifying a list of hosts. For example, in a manualroute router you could have route_list = * a.b.c/MX or route_list = * x.y.z:p.q.r/MX:e.f.g If the hosts_randomize option is set, the order of the items in the list is randomized exactly as before. Then, when Exim is scanning the list to find an IP address for each host, items that end with "/MX" are replaced with the list of host names obtained by looking up MX records for the name. The order is determined by the preference values in the MX records, according to the usual rules. Because randomizing happens first, it does not affect the order that is defined by MX preferences. If the local host is present in the list obtained from MX records, but is not the most preferred host in that list, it and any equally or less preferred hosts are removed before the list is inserted. If the local host is the most preferred host in the MX list, what happens depends on where in the original list of hosts the "/MX" item appears. If it is not the first item (i.e. there are previous hosts), Exim just discards this item and any subsequent items in the list, just as it does for a non-MX item that turns out to be the local host. If the MX item is first in the list of hosts, and the local host is the most preferred host, what happens is controlled by the "self" option of the router. DNS failures are treated in the same way as DNS failures when looking up IP addresses: pass_on_timeout and host_find_failed are used when relevant. 20. There's a new generic transport option called rcpt_include_affixes. Its default value is FALSE. In this state, when an address that has had any affixes (prefixes or suffixes) removed from the local part is delivered by any form of SMTP or LMTP, the affixes are not included. For example, if a router that contains local_part_prefix = *- routes the address abc-xyz@some.domain to an SMTP transport, the envelope is delivered with RCPT TO: However, if rcpt_include_affixes is set TRUE, the whole local part is included in the RCPT command. This option applies to BSMTP deliveries by the appendfile and pipe transports as well as to the lmtp and smtp transports. 21. By default, the command line options -odi etc. override the setting of queue_only in the configuration file. This is now controlled by the queue_only_override option, which defaults true. If it is set false, the -od options cannot be used to override queue_only or queue_only_file. 22. When Exim is logging to syslog, it writes the log lines for its three separate logs at different syslog priorities so that they can in principle be separated on the logging hosts. Some installations do not require this separation, and in those cases, the duplication of certain log lines is a nuisance. If syslog_duplication is set false, only one copy of any particular log line is written to syslog. For lines that normally go to both the main log and the reject log, the reject log version (possibly containing message header lines) is written, at LOG_NOTICE priority. Lines that normally go to both the main and the panic log are written at the LOG_ALERT priority. 23. Formerly, the value of the AUTH= option on a MAIL command was trusted by an Exim server only if the client host had authenticated. An ACL can now be used as an alternative way of control this. The way that Exim handles an AUTH= option on a MAIL command is now as follows: . If the connection is not using extended SMTP (i.e. HELO was used), the use of AUTH= is a syntax error. . If the value of the AUTH= parameter is "<>" it is ignored. . If acl_smtp_mailauth is defined, this ACL is run. While it is running, the value of $authenticated_sender is set to the value obtained from the AUTH= parameter. If the ACL does not yield "accept", the value of $authenticated_sender is deleted. . If acl_smtp_mailauth is not defined, the value of the AUTH= parameter is ignored if the client has not authenticated. . If the AUTH= value was accepted by either of the two previous rules, and the client has authenticated, and the authenticator has a setting for the server_mail_auth_condition, the condition is checked. If it fails, the value of $authenticated_sender is deleted. . The ACL for MAIL, if defined, is run after AUTH= is accepted or ignored. It can make use of $authenticated_sender. The converse is not true: the value of $sender_address is not yet set up when the "mailauth" ACL is run. Whenever an AUTH= value is ignored, the incident is logged. Previously, logging happened only when server_mail_auth_condition testing failed. The acl_smtp_mailauth ACL may not return "drop" or "discard". If it defers, a temporary error code (451) is given for the MAIL command. 24. By default, callouts do not happen when testing with -bh. There is now a variant, -bhc, which does actually run the callout code, including consulting and updating the callout cache. 25. Exim now has support for Cyrus saslauthd authentication. This works in a similar way to Cyrus pwcheck authentication (which is now deprecated). To build Exim with saslauthd support, set CYRUS_SASLAUTHD_SOCKET in Local/Makefile. This must define the location of the socket that is used to communicate with the saslauthd daemon. For example: CYRUS_SASLAUTHD_SOCKET=/var/state/saslauthd/mux The daemon is called by a new expansion condition: ${if saslauthd{{user}{password}{service}{realm}}{yes-string}{no-string}} Up to four arguments can be supplied, but only two are required (service and realm are optional). That is why the arguments are enclosed in a set of braces. For details of the meaning of service and realm, and how to run the saslauthd daemon, consult the Cyrus documentation. 26. The way daemon_smtp_port, local_interfaces, and -oX interact has been extended, compatibily I hope. A new option called extra_local_interfaces has been added. Here is a full description of all of this, which will eventually be the basis of a new chapter in the manual. EXIM'S USE OF NETWORK INTERFACES A host that is connected to a TCP/IP network may have one or more physical hardware interfaces. Each of these interfaces may be configured as one or more "logical" interfaces, which are the entities that a program actually works with. Each of these logical interfaces is associated with an IP address. In addition, TCP/IP software supports "loopback" interfaces (127.0.0.1 in IPv4 and ::1 in IPv6), which do not use any physical hardware. Exim uses knowledge about the host's interfaces in three different ways: (1) When a listening daemon is started, Exim needs to know which interfaces and ports to listen on. (2) When Exim is routing an address, it needs to know which IP addresses are associated with local interfaces. This is required for the correct processing of MX lists by removing the local host and others with the same or higher priority values. Also, Exim needs to detect cases when an address is routed to an IP address that in fact belongs to the local host. Unless the "self" router option or the "allow_localhost" option of the smtp transport is set (as appropriate), this is treated as an error situation. (3) When Exim connects to a remote host, it may need to know which interface to use for the outgoing connection. Exim's default behaviour is likely to be appropriate is the vast majority of cases. If your host has only one interface, and you want all its IP addresses to be treated in the same way, and you are using only the standard SMTP port, you should not need to take any special action. The rest of this chapter does not apply to you. However, in more complicated situations, you may want to change the way Exim behaves, and for this reason there are a number of options that can be used to influence Exim's behaviour. The rest of this chapter describes how they operate. STARTING A LISTENING DAEMON When a listening daemon is started (by means of the -bd command line option), the interfaces and ports on which it listens are controlled by the following two options: . daemon_smtp_ports contains a list of default ports. (For backward compatibility, this option can also be specified in the singular.) . local_interfaces contains list of interface IP addresses on which to listen. The default list separator in both cases is a colon, but this can be changed as described in section 6.15. When IPv6 addresses are involved, it is usually best to change the separator to avoid having to double all the colons in IPv6 addresses. For example: local_interfaces = <; 127.0.0.1 ; \ 192.168.23.65 ; \ ::1 ; \ 3ffe:ffff:836f::fe86:a061 A port number can be specified along with each IP address in local_interfaces. Two different formats are recognized: . The port is added onto the address with a dot separator, for example, local_interfaces = <; 192.168.23.65.1234 ; \ 3ffe:ffff:836f::fe86:a061.1234 . The IP address is enclosed in square brackets, and the port is added with a colon separator, for example, local_interfaces = <; [192.168.23.65]:1234 ; \ [3ffe:ffff:836f::fe86:a061]:1234 When a port is not specified, the value of daemon_smtp_ports is used. The default setting contains just one port: daemon_smtp_ports = smtp In this list, ports can be identified either by name (defined in /etc/services) or by number. However, when ports are given with individual IP addresses in local_interfaces only numbers (not names) can be used. The addresses 0.0.0.0 and ::0 are treated specially. They are interpreted as `all IPv4 interfaces' and `all IPv6 interfaces', respectively. In each case, Exim tells the TCP/IP stack to `listen on all IPvx interfaces' instead of setting up separate listening sockets on each interface. The default value of local_interfaces is local_interfaces = 0.0.0.0 when Exim is built without IPv6 support; otherwise it is: local_interfaces = <; ::0 ; 0.0.0.0 When a message is received over TCP/IP, the interface and port that were used are set in $interface_address and $interface_port. The -oX command line option can be used to override the values of daemon_smtp_ports and/or local_interfaces for a particular daemon instance. Another way of doing this would be to use macros and the -D option. However, -oX can be used by any admin user, whereas modification of the runtime configuration by -D is allowed only when the caller is root or exim. The value of -oX is a list of items. The default colon separator can be changed in the usual way if required. If there are any items that do not contain dots or colons (that is, are not IP addresses), the value of daemon_smtp_ports is replaced by the list of those items. If there are any items that do contain dots or colons, the value of local_interfaces is replaced by those items. Thus, for example, -oX 1225 overrides daemon_smtp_ports, but leaves local_interfaces unchanged, whereas -oX 192.168.34.5.1125 overrides local_interfaces, leaving daemon_smtp_ports unchanged. (However, since local_interfaces now contains no items without ports, the value of daemon_smtp_ports is no longer relevant.) IPV6 ADDRESS SCOPES IPv6 addresses have `scopes', and a host with multiple hardware interfaces can, in principle, have the same link-local address on different interfaces. Thus, additional information is needed, over and above the IP address, to distinguish individual interfaces. A a convention of using a percent sign followed by something (often the interface name) has been adopted in some cases, leading to addresses like this: 3ffe:2101:12:1:a00:20ff:fe86:a061%eth0 To accommodate this usage, a percent sign followed by an arbitrary string is allowed at the end of an IPv6 address. By default, Exim calls "getaddrinfo()" to convert a textual IPv6 address for actual use. This function recognizes the percent convention in operating systems that support it, and it processes the address appropriately. Unfortunately, some older libraries have problems with "getaddrinfo()". If IPV6_USE_INET_PTON=yes is set in Local/Makefile (or an OS-dependent Makefile) when Exim is built, Exim uses "inet_pton()" to convert a textual IPv6 address for actual use, instead of "getaddrinfo()". (Before version 4.14, it always used this function.) Of course, this means that the additional functionality of "getaddrinfo()" - recognizing scoped addresses - is lost. EXAMPLES OF STARTING A LISTENING DAEMON The default case in an IPv6 environment is daemon_smtp_port = smtp local_interfaces = <; ::0 ; 0.0.0.0 This specifies listening on the smtp port on all IPv6 and IPv4 interfaces. Either one or two sockets may be used, depending on the characteristics of the TCP/IP stack. (This is complicated and messy; for more information, read the comments in the daemon.c source file.) To specify listening on ports 25 and 25 on all interfaces, you could use daemon_smtp_ports = 25 : 26 (leaving local_interfaces at the default setting) or, more explicitly: local_interfaces = <; ::0.25 ; ::0.26 \ 0.0.0.0.25 ; 0.0.0.0.26 To listen on the default port on all IPv4 interfaces, and on port 26 on the IPv4 loopback address only: local_interfaces = 0.0.0.0 : 127.0.0.1.26 To specify listening on the default ports two specific interfaces only: local_interfaces = 192.168.34.67.25 : 192.168.34.67.26 Note that such a setting excludes listening on the loopback interfaces. RECOGNISING LOCAL IP ADDRESSES The local_interfaces option is also used when Exim needs to determine whether or not an IP address refers to the local host. That is, the IP addresses of all the interfaces on which a daemon is listening are always treated as local. For this usage, port numbers in local_interfaces are ignored. If either of the items 0.0.0.0 or ::0 are encountered, Exim gets a complete list of available interfaces from the operating system, and extracts the relevant (that is, IPv4 or IPv6) addresses to use for checking. Some systems set up large numbers of virtual interfaces in order to provide many virtual web servers. In this situation, you may want to listen for email on only a few of the available interfaces, but nevertheless treat all interfaces as local. You can do this by setting extra_local_interfaces to a list of IP addresses, possibly including the `all' wildcard values. These addresses are recognized as local, but are not used for listening. Consider this example: local_interfaces = <; 127.0.0.1 ; ::1 ; \ 192.168.53.235 ; \ 3ffe:2101:12:1:a00:20ff:fe86:a061 extra_local_interfaces = <; ::0 ; 0.0.0.0 The daemon listens on the local interfaces and just one IPv4 and one IPv6 address, but all available interface addresses are treated as local when Exim is routing. Another related option is hosts_treat_as_local. However, this uses host names rather than IP addresses. It operates when Exim is dealing with MX records. DELIVERING TO A REMOTE HOST Delivery to a remote host is handled by the smtp transport. By default, it allows the system's TCP/IP functions to choose which interface to use (if there is more than one) when connecting to a remote host. However, the "interface" option can be set to specify which interface is used. See the description of the smtp transport in chapter 29 for more details. 27. The new option process_log_path sets the name of the file to which Exim writes its "process log" when sent SIGUSR1. This is used by the exiwhat utility script. If this option is unset, the file called exim-process.info in Exim's spool directory is used. The ability to specify the name explicitly can be useful in environments where two different Exims are running, using different spool directories. ****