<!DOCTYPE html PUBLIC "XSLT-compat"> <html lang="en-GB"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <link rel="stylesheet" type="text/css" href="../../../../common.css"> <meta name="author" content="The Exim Project. <http://www.exim.org/>"> <meta name="copyright" content="Copyright ©2010 The Exim Project. All rights reserved"> <meta name="description" content="Exim is a message transfer agent (MTA) developed at the University of Cambridge for use on Unix systems connected to the Internet."> <meta name="keywords" content="exim,smtp,mta,email"> <meta name="robots" content="noodp,noydir,index,follow"> <meta name="viewport" content="width=device-width"> <title>53. Format of spool files</title> <link rel="stylesheet" type="text/css" href="../../../../doc/chapter.css"> <link rel="canonical" href="http://www.exim.org/exim-html-current/doc/html/spec_html/ch53.html"> </head> <body> <h1 id="header"><a href="../../../..">Exim Internet Mailer</a></h1> <div id="outer"> <ul id="nav_flow" class="nav"> <li><a href="../../../../index.html">Home</a></li> <li><a href="../../../../mirrors.html">Download</a></li> <li><a href="../../../../docs.html">Documentation</a></li> <li><a href="../../../../maillist.html">Mailing Lists</a></li> <li><a href="http://wiki.exim.org/">Wiki</a></li> <li><a href="http://www.exim.org/bugzilla/">Bugs</a></li> <li><a href="../../../../credits.html">Credits</a></li> <li class="search"><form action="http://www.google.com/search" method="get"> <span class="search_field_container"><input type="search" name="q" placeholder="Search Docs" class="search_field"></span><input type="hidden" name="hl" value="en"><input type="hidden" name="ie" value="UTF-8"><input type="hidden" name="as_qdr" value="all"><input type="hidden" name="q" value="site:www.exim.org"><input type="hidden" name="q" value="inurl:exim-html-current"> </form></li> </ul> <div id="inner"><div id="content"> <a class="previous_page" href="ch52.html"><-previous</a><a class="next_page" href="ch54.html">next-></a><div id="chapter" class="chapter"> <h2 id="CHAPspool" class="">Chapter 53 - Format of spool files</h2> <p> A message on Exim’s queue consists of two files, whose names are the message id followed by -D and -H, respectively. The data portion of the message is kept in the -D file on its own. The message’s envelope, status, and headers are all kept in the -H file, whose format is described in this chapter. Each of these two files contains the final component of its own name as its first line. This is insurance against disk crashes where the directory is lost but the files themselves are recoverable. </p> <p> Some people are tempted into editing -D files in order to modify messages. You need to be extremely careful if you do this; it is not recommended and you are on your own if you do it. Here are some of the pitfalls: </p> <ul> <li> <p> You must ensure that Exim does not try to deliver the message while you are fiddling with it. The safest way is to take out a write lock on the -D file, which is what Exim itself does, using <span class="docbook_function">fcntl()</span>. If you update the file in place, the lock will be retained. If you write a new file and rename it, the lock will be lost at the instant of rename. </p> </li> <li> <p> If you change the number of lines in the file, the value of $body_linecount, which is stored in the -H file, will be incorrect. At present, this value is not used by Exim, but there is no guarantee that this will always be the case. </p> </li> <li> <p> If the message is in MIME format, you must take care not to break it. </p> </li> <li> <p> If the message is cryptographically signed, any change will invalidate the signature. </p> </li> </ul> <p> All in all, modifying -D files is fraught with danger. </p> <p> Files whose names end with -J may also be seen in the <span class="docbook_filename">input</span> directory (or its subdirectories when <span class="docbook_option">split_spool_directory</span> is set). These are journal files, used to record addresses to which the message has been delivered during the course of a delivery attempt. If there are still undelivered recipients at the end, the -H file is updated, and the -J file is deleted. If, however, there is some kind of crash (for example, a power outage) before this happens, the -J file remains in existence. When Exim next processes the message, it notices the -J file and uses it to update the -H file before starting the next delivery attempt. </p> <div class="section"> <h3 id="SECID282" class="">1. Format of the -H file</h3> <p> The second line of the -H file contains the login name for the uid of the process that called Exim to read the message, followed by the numerical uid and gid. For a locally generated message, this is normally the user who sent the message. For a message received over TCP/IP via the daemon, it is normally the Exim user. </p> <p> The third line of the file contains the address of the message’s sender as transmitted in the envelope, contained in angle brackets. The sender address is empty for bounce messages. For incoming SMTP mail, the sender address is given in the MAIL command. For locally generated mail, the sender address is created by Exim from the login name of the current user and the configured <span class="docbook_option">qualify_domain</span>. However, this can be overridden by the <span class="docbook_option">-f</span> option or a leading “From ” line if the caller is trusted, or if the supplied address is “<>” or an address that matches <span class="docbook_option">untrusted_set_senders</span>. </p> <p> The fourth line contains two numbers. The first is the time that the message was received, in the conventional Unix form – the number of seconds since the start of the epoch. The second number is a count of the number of messages warning of delayed delivery that have been sent to the sender. </p> <p> There follow a number of lines starting with a hyphen. These can appear in any order, and are omitted when not relevant: </p> <dl> <dt> <span class="docbook_option">-acl</span> <<span class="docbook_emphasis">number</span>> <<span class="docbook_emphasis">length</span>></dt> <dd> <p> This item is obsolete, and is not generated from Exim release 4.61 onwards; <span class="docbook_option">-aclc</span> and <span class="docbook_option">-aclm</span> are used instead. However, <span class="docbook_option">-acl</span> is still recognized, to provide backward compatibility. In the old format, a line of this form is present for every ACL variable that is not empty. The number identifies the variable; the <span class="docbook_option">acl_c</span><span class="docbook_emphasis">x</span> variables are numbered 0–9 and the <span class="docbook_option">acl_m</span><span class="docbook_emphasis">x</span> variables are numbered 10–19. The length is the length of the data string for the variable. The string itself starts at the beginning of the next line, and is followed by a newline character. It may contain internal newlines. </p> </dd> <dt> <span class="docbook_option">-aclc</span> <<span class="docbook_emphasis">rest-of-name</span>> <<span class="docbook_emphasis">length</span>></dt> <dd> <p> A line of this form is present for every ACL connection variable that is defined. Note that there is a space between <span class="docbook_option">-aclc</span> and the rest of the name. The length is the length of the data string for the variable. The string itself starts at the beginning of the next line, and is followed by a newline character. It may contain internal newlines. </p> </dd> <dt> <span class="docbook_option">-aclm</span> <<span class="docbook_emphasis">rest-of-name</span>> <<span class="docbook_emphasis">length</span>></dt> <dd> <p> A line of this form is present for every ACL message variable that is defined. Note that there is a space between <span class="docbook_option">-aclm</span> and the rest of the name. The length is the length of the data string for the variable. The string itself starts at the beginning of the next line, and is followed by a newline character. It may contain internal newlines. </p> </dd> <dt> <span class="docbook_option">-active_hostname</span> <<span class="docbook_emphasis">hostname</span>></dt> <dd> <p> This is present if, when the message was received over SMTP, the value of $smtp_active_hostname was different to the value of $primary_hostname. </p> </dd> <dt><span class="docbook_option">-allow_unqualified_recipient</span></dt> <dd> <p> This is present if unqualified recipient addresses are permitted in header lines (to stop such addresses from being qualified if rewriting occurs at transport time). Local messages that were input using <span class="docbook_option">-bnq</span> and remote messages from hosts that match <span class="docbook_option">recipient_unqualified_hosts</span> set this flag. </p> </dd> <dt><span class="docbook_option">-allow_unqualified_sender</span></dt> <dd> <p> This is present if unqualified sender addresses are permitted in header lines (to stop such addresses from being qualified if rewriting occurs at transport time). Local messages that were input using <span class="docbook_option">-bnq</span> and remote messages from hosts that match <span class="docbook_option">sender_unqualified_hosts</span> set this flag. </p> </dd> <dt> <span class="docbook_option">-auth_id</span> <<span class="docbook_emphasis">text</span>></dt> <dd> <p> The id information for a message received on an authenticated SMTP connection – the value of the $authenticated_id variable. </p> </dd> <dt> <span class="docbook_option">-auth_sender</span> <<span class="docbook_emphasis">address</span>></dt> <dd> <p> The address of an authenticated sender – the value of the $authenticated_sender variable. </p> </dd> <dt> <span class="docbook_option">-body_linecount</span> <<span class="docbook_emphasis">number</span>></dt> <dd> <p> This records the number of lines in the body of the message, and is always present. </p> </dd> <dt> <span class="docbook_option">-body_zerocount</span> <<span class="docbook_emphasis">number</span>></dt> <dd> <p> This records the number of binary zero bytes in the body of the message, and is present if the number is greater than zero. </p> </dd> <dt><span class="docbook_option">-deliver_firsttime</span></dt> <dd> <p> This is written when a new message is first added to the spool. When the spool file is updated after a deferral, it is omitted. </p> </dd> <dt> <span class="docbook_option">-frozen</span> <<span class="docbook_emphasis">time</span>></dt> <dd> <p> The message is frozen, and the freezing happened at <<span class="docbook_emphasis">time</span>>. </p> </dd> <dt> <span class="docbook_option">-helo_name</span> <<span class="docbook_emphasis">text</span>></dt> <dd> <p> This records the host name as specified by a remote host in a HELO or EHLO command. </p> </dd> <dt> <span class="docbook_option">-host_address</span> <<span class="docbook_emphasis">address</span>>.<<span class="docbook_emphasis">port</span>></dt> <dd> <p> This records the IP address of the host from which the message was received and the remote port number that was used. It is omitted for locally generated messages. </p> </dd> <dt> <span class="docbook_option">-host_auth</span> <<span class="docbook_emphasis">text</span>></dt> <dd> <p> If the message was received on an authenticated SMTP connection, this records the name of the authenticator – the value of the $sender_host_authenticated variable. </p> </dd> <dt><span class="docbook_option">-host_lookup_failed</span></dt> <dd> <p> This is present if an attempt to look up the sending host’s name from its IP address failed. It corresponds to the $host_lookup_failed variable. </p> </dd> <dt> <span class="docbook_option">-host_name</span> <<span class="docbook_emphasis">text</span>></dt> <dd> <p> This records the name of the remote host from which the message was received, if the host name was looked up from the IP address when the message was being received. It is not present if no reverse lookup was done. </p> </dd> <dt> <span class="docbook_option">-ident</span> <<span class="docbook_emphasis">text</span>></dt> <dd> <p> For locally submitted messages, this records the login of the originating user, unless it was a trusted user and the <span class="docbook_option">-oMt</span> option was used to specify an ident value. For messages received over TCP/IP, this records the ident string supplied by the remote host, if any. </p> </dd> <dt> <span class="docbook_option">-interface_address</span> <<span class="docbook_emphasis">address</span>>.<<span class="docbook_emphasis">port</span>></dt> <dd> <p> This records the IP address of the local interface and the port number through which a message was received from a remote host. It is omitted for locally generated messages. </p> </dd> <dt><span class="docbook_option">-local</span></dt> <dd> <p> The message is from a local sender. </p> </dd> <dt><span class="docbook_option">-localerror</span></dt> <dd> <p> The message is a locally-generated bounce message. </p> </dd> <dt> <span class="docbook_option">-local_scan</span> <<span class="docbook_emphasis">string</span>></dt> <dd> <p> This records the data string that was returned by the <span class="docbook_function">local_scan()</span> function when the message was received – the value of the $local_scan_data variable. It is omitted if no data was returned. </p> </dd> <dt><span class="docbook_option">-manual_thaw</span></dt> <dd> <p> The message was frozen but has been thawed manually, that is, by an explicit Exim command rather than via the auto-thaw process. </p> </dd> <dt><span class="docbook_option">-N</span></dt> <dd> <p> A testing delivery process was started using the <span class="docbook_option">-N</span> option to suppress any actual deliveries, but delivery was deferred. At any further delivery attempts, <span class="docbook_option">-N</span> is assumed. </p> </dd> <dt><span class="docbook_option">-received_protocol</span></dt> <dd> <p> This records the value of the $received_protocol variable, which contains the name of the protocol by which the message was received. </p> </dd> <dt><span class="docbook_option">-sender_set_untrusted</span></dt> <dd> <p> The envelope sender of this message was set by an untrusted local caller (used to ensure that the caller is displayed in queue listings). </p> </dd> <dt> <span class="docbook_option">-spam_score_int</span> <<span class="docbook_emphasis">number</span>></dt> <dd> <p> If a message was scanned by SpamAssassin, this is present. It records the value of $spam_score_int. </p> </dd> <dt><span class="docbook_option">-tls_certificate_verified</span></dt> <dd> <p> A TLS certificate was received from the client that sent this message, and the certificate was verified by the server. </p> </dd> <dt> <span class="docbook_option">-tls_cipher</span> <<span class="docbook_emphasis">cipher name</span>></dt> <dd> <p> When the message was received over an encrypted connection, this records the name of the cipher suite that was used. </p> </dd> <dt> <span class="docbook_option">-tls_peerdn</span> <<span class="docbook_emphasis">peer DN</span>></dt> <dd> <p> When the message was received over an encrypted connection, and a certificate was received from the client, this records the Distinguished Name from that certificate. </p> </dd> </dl> <p> Following the options there is a list of those addresses to which the message is not to be delivered. This set of addresses is initialized from the command line when the <span class="docbook_option">-t</span> option is used and <span class="docbook_option">extract_addresses_remove_arguments</span> is set; otherwise it starts out empty. Whenever a successful delivery is made, the address is added to this set. The addresses are kept internally as a balanced binary tree, and it is a representation of that tree which is written to the spool file. If an address is expanded via an alias or forward file, the original address is added to the tree when deliveries to all its child addresses are complete. </p> <p> If the tree is empty, there is a single line in the spool file containing just the text “XX”. Otherwise, each line consists of two letters, which are either Y or N, followed by an address. The address is the value for the node of the tree, and the letters indicate whether the node has a left branch and/or a right branch attached to it, respectively. If branches exist, they immediately follow. Here is an example of a three-node tree: </p> <div class="docbook_literallayout"><pre> YY darcy@austen.fict.example NN alice@wonderland.fict.example NN editor@thesaurus.ref.example </pre></div> <p> After the non-recipients tree, there is a list of the message’s recipients. This is a simple list, preceded by a count. It includes all the original recipients of the message, including those to whom the message has already been delivered. In the simplest case, the list contains one address per line. For example: </p> <div class="docbook_literallayout"><pre> 4 editor@thesaurus.ref.example darcy@austen.fict.example rdo@foundation alice@wonderland.fict.example </pre></div> <p> However, when a child address has been added to the top-level addresses as a result of the use of the <span class="docbook_option">one_time</span> option on a <span class="docbook_command">redirect</span> router, each line is of the following form: </p> <div class="docbook_literallayout"><pre> <<span class="docbook_emphasis">top-level address</span>> <<span class="docbook_emphasis">errors_to address</span>> <<span class="docbook_emphasis">length</span>>,<<span class="docbook_emphasis">parent number</span>>#<<span class="docbook_emphasis">flag bits</span>> </pre></div> <p> The 01 flag bit indicates the presence of the three other fields that follow the top-level address. Other bits may be used in future to support additional fields. The <<span class="docbook_emphasis">parent number</span>> is the offset in the recipients list of the original parent of the “one time” address. The first two fields are the envelope sender that is associated with this address and its length. If the length is zero, there is no special envelope sender (there are then two space characters in the line). A non-empty field can arise from a <span class="docbook_command">redirect</span> router that has an <span class="docbook_option">errors_to</span> setting. </p> <p> A blank line separates the envelope and status information from the headers which follow. A header may occupy several lines of the file, and to save effort when reading it in, each header is preceded by a number and an identifying character. The number is the number of characters in the header, including any embedded newlines and the terminating newline. The character is one of the following: </p> <table> <tr> <td><<span class="docbook_emphasis">blank</span>></td> <td>header in which Exim has no special interest</td> </tr> <tr> <td><code class="docbook_literal">B</code></td> <td> <span class="docbook_emphasis">Bcc:</span> header</td> </tr> <tr> <td><code class="docbook_literal">C</code></td> <td> <span class="docbook_emphasis">Cc:</span> header</td> </tr> <tr> <td><code class="docbook_literal">F</code></td> <td> <span class="docbook_emphasis">From:</span> header</td> </tr> <tr> <td><code class="docbook_literal">I</code></td> <td> <span class="docbook_emphasis">Message-id:</span> header</td> </tr> <tr> <td><code class="docbook_literal">P</code></td> <td> <span class="docbook_emphasis">Received:</span> header – P for “postmark”</td> </tr> <tr> <td><code class="docbook_literal">R</code></td> <td> <span class="docbook_emphasis">Reply-To:</span> header</td> </tr> <tr> <td><code class="docbook_literal">S</code></td> <td> <span class="docbook_emphasis">Sender:</span> header</td> </tr> <tr> <td><code class="docbook_literal">T</code></td> <td> <span class="docbook_emphasis">To:</span> header</td> </tr> <tr> <td><code class="docbook_literal">*</code></td> <td>replaced or deleted header</td> </tr> </table> <p> Deleted or replaced (rewritten) headers remain in the spool file for debugging purposes. They are not transmitted when the message is delivered. Here is a typical set of headers: </p> <div class="docbook_literallayout"><pre> 111P Received: by hobbit.fict.example with local (Exim 4.00) id 14y9EI-00026G-00; Fri, 11 May 2001 10:28:59 +0100 049 Message-Id: <E14y9EI-00026G-00@hobbit.fict.example> 038* X-rewrote-sender: bb@hobbit.fict.example 042* From: Bilbo Baggins <bb@hobbit.fict.example> 049F From: Bilbo Baggins <B.Baggins@hobbit.fict.example> 099* To: alice@wonderland.fict.example, rdo@foundation, darcy@austen.fict.example, editor@thesaurus.ref.example 104T To: alice@wonderland.fict.example, rdo@foundation.example, darcy@austen.fict.example, editor@thesaurus.ref.example 038 Date: Fri, 11 May 2001 10:28:59 +0100 </pre></div> <p> The asterisked headers indicate that the envelope sender, <span class="docbook_emphasis">From:</span> header, and <span class="docbook_emphasis">To:</span> header have been rewritten, the last one because routing expanded the unqualified domain <span class="docbook_emphasis">foundation</span>. </p> </div> </div> <a class="previous_page" href="ch52.html"><-previous</a><a class="next_page" href="ch54.html">next-></a> </div></div> <iframe id="branding" name="branding" src="../../../../branding/branding.html" height="0" frameborder="no" scrolling="no"></iframe><div id="footer">Website design by <a href="https://secure.grepular.com/">Mike Cardwell</a>, of <a href="http://cardwellit.com/">Cardwell IT Ltd.</a> </div> <div class="left_bar"></div> <div class="right_bar"></div> <div id="toc"> <ul class="hidden"></ul> <img src="../../../../doc/contents.png" width="16" height="155"> </div> </div> <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.4/jquery.min.js"></script><script type="text/javascript" src="../../../../common.js"></script><script type="text/javascript" src="../../../../doc/chapter.js"></script> </body> </html>
