1d0cef73dSGregory Neil Shapiro<HTML> 2d0cef73dSGregory Neil Shapiro<HEAD> 3d0cef73dSGregory Neil Shapiro<TITLE>Technical Overview</TITLE> 4d0cef73dSGregory Neil Shapiro</HEAD> 5d0cef73dSGregory Neil Shapiro<BODY> 640266059SGregory Neil Shapiro<!-- 74313cc83SGregory Neil Shapiro$Id: overview.html,v 1.22 2013-11-22 20:51:39 ca Exp $ 840266059SGregory Neil Shapiro--> 940266059SGregory Neil Shapiro 10d0cef73dSGregory Neil Shapiro<H1>Technical Overview</H1> 1140266059SGregory Neil Shapiro 12d0cef73dSGregory Neil Shapiro<H2>Contents</H2> 1340266059SGregory Neil Shapiro 14d0cef73dSGregory Neil Shapiro<UL> 15d0cef73dSGregory Neil Shapiro <LI><A HREF="#Initialization">Initialization</A> 16d0cef73dSGregory Neil Shapiro <LI><A HREF="#ControlFlow">Control Flow</A> 17d0cef73dSGregory Neil Shapiro <LI><A HREF="#Multithreading">Multithreading</A> 18d0cef73dSGregory Neil Shapiro <LI><A HREF="#ResourceManagement">Resource Management</A> 19d0cef73dSGregory Neil Shapiro <LI><A HREF="#SignalHandling">Signal Handling</A> 20d0cef73dSGregory Neil Shapiro</UL> 2140266059SGregory Neil Shapiro 22d0cef73dSGregory Neil Shapiro<H2><A NAME="Initialization">Initialization</A></H2> 2340266059SGregory Neil Shapiro 24d0cef73dSGregory Neil ShapiroIn addition to its own initialization, 25d0cef73dSGregory Neil Shapirolibmilter expects a filter to initialize several parameters 26d0cef73dSGregory Neil Shapirobefore calling <A HREF="smfi_main.html">smfi_main</A>: 27d0cef73dSGregory Neil Shapiro<UL> 28d0cef73dSGregory Neil Shapiro <LI>The callbacks the filter wishes to be called, and the types of 29d0cef73dSGregory Neil Shapiro message modification it intends to perform (required, see 30d0cef73dSGregory Neil Shapiro <A HREF="smfi_register.html">smfi_register</A>). 3140266059SGregory Neil Shapiro 32d0cef73dSGregory Neil Shapiro <LI>The socket address to be used when communicating with the MTA 33d0cef73dSGregory Neil Shapiro (required, see <A HREF="smfi_setconn.html">smfi_setconn</A>). 3440266059SGregory Neil Shapiro 35d0cef73dSGregory Neil Shapiro <LI>The number of seconds to wait for MTA connections before 36d0cef73dSGregory Neil Shapiro timing out (optional, see 37d0cef73dSGregory Neil Shapiro <A HREF="smfi_settimeout.html">smfi_settimeout</A>). 38d0cef73dSGregory Neil Shapiro</UL> 39d0cef73dSGregory Neil Shapiro<P> 40d0cef73dSGregory Neil ShapiroIf the filter fails to initialize libmilter, 41d0cef73dSGregory Neil Shapiroor if one or more of the parameters it has passed are invalid, 42d0cef73dSGregory Neil Shapiroa subsequent call to smfi_main will fail. 4340266059SGregory Neil Shapiro 44d0cef73dSGregory Neil Shapiro<H2><A NAME="ControlFlow">Control Flow</A></H2> 4540266059SGregory Neil Shapiro 46d0cef73dSGregory Neil Shapiro<P> 4740266059SGregory Neil ShapiroThe following pseudocode describes the filtering process from the 48d0cef73dSGregory Neil Shapiroperspective of a set of <CODE>N</CODE> MTA's, 49d0cef73dSGregory Neil Shapiroeach corresponding to a connection. 50d0cef73dSGregory Neil ShapiroCallbacks are shown beside the processing stages in which they are invoked; 51d0cef73dSGregory Neil Shapiroif no callbacks are defined for a particular stage, 52d0cef73dSGregory Neil Shapirothat stage may be bypassed. 53d0cef73dSGregory Neil ShapiroThough it is not shown, 54d0cef73dSGregory Neil Shapiroprocessing may be aborted at any time during a message, 55d0cef73dSGregory Neil Shapiroin which case the 56d0cef73dSGregory Neil Shapiro<A HREF="xxfi_abort.html">xxfi_abort</A> callback is invoked and control 57d0cef73dSGregory Neil Shapiroreturns to <CODE>MESSAGE</CODE>. 58d0cef73dSGregory Neil Shapiro<P> 59d0cef73dSGregory Neil Shapiro<PRE> 6040266059SGregory Neil ShapiroFor each of N connections 6140266059SGregory Neil Shapiro{ 6240266059SGregory Neil Shapiro For each filter 63*5b0945b5SGregory Neil Shapiro egotiate MTA/milter capabilities/requirements (<A HREF="xxfi_negotiate.html">xxfi_negotiate</A>) 64*5b0945b5SGregory Neil Shapiro For each filter 659bd497b8SGregory Neil Shapiro process connection (<A HREF="xxfi_connect.html">xxfi_connect</A>) 669bd497b8SGregory Neil Shapiro For each filter 679bd497b8SGregory Neil Shapiro process helo/ehlo (<A HREF="xxfi_helo.html">xxfi_helo</A>) 6840266059SGregory Neil ShapiroMESSAGE:For each message in this connection (sequentially) 6940266059SGregory Neil Shapiro { 7040266059SGregory Neil Shapiro For each filter 71d0cef73dSGregory Neil Shapiro process sender (<A HREF="xxfi_envfrom.html">xxfi_envfrom</A>) 7240266059SGregory Neil Shapiro For each recipient 7340266059SGregory Neil Shapiro { 7440266059SGregory Neil Shapiro For each filter 75d0cef73dSGregory Neil Shapiro process recipient (<A HREF="xxfi_envrcpt.html">xxfi_envrcpt</A>) 7640266059SGregory Neil Shapiro } 7740266059SGregory Neil Shapiro For each filter 7840266059SGregory Neil Shapiro { 79d0cef73dSGregory Neil Shapiro process DATA (<A HREF="xxfi_data.html">xxfi_data</A>) 8040266059SGregory Neil Shapiro For each header 81d0cef73dSGregory Neil Shapiro process header (<A HREF="xxfi_header.html">xxfi_header</A>) 82d0cef73dSGregory Neil Shapiro process end of headers (<A HREF="xxfi_eoh.html">xxfi_eoh</A>) 8340266059SGregory Neil Shapiro For each body block 84d0cef73dSGregory Neil Shapiro process this body block (<A HREF="xxfi_body.html">xxfi_body</A>) 85d0cef73dSGregory Neil Shapiro process end of message (<A HREF="xxfi_eom.html">xxfi_eom</A>) 8640266059SGregory Neil Shapiro } 8740266059SGregory Neil Shapiro } 8840266059SGregory Neil Shapiro For each filter 89d0cef73dSGregory Neil Shapiro process end of connection (<A HREF="xxfi_close.html">xxfi_close</A>) 9040266059SGregory Neil Shapiro} 91d0cef73dSGregory Neil Shapiro</PRE> 9240266059SGregory Neil Shapiro 9340266059SGregory Neil Shapiro<P>Note: Filters are contacted in order defined in config file.</P> 9440266059SGregory Neil Shapiro 9540266059SGregory Neil Shapiro<P> 9640266059SGregory Neil ShapiroTo write a filter, a vendor supplies callbacks to process relevant 97d0cef73dSGregory Neil Shapiroparts of a message transaction. 98d0cef73dSGregory Neil ShapiroThe library then controls all sequencing, threading, 99d0cef73dSGregory Neil Shapiroand protocol exchange with the MTA. 100d0cef73dSGregory Neil Shapiro<A HREF="#figure-3">Figure 3</A> outlines control flow for a filter 10140266059SGregory Neil Shapiroprocess, showing where different callbacks are invoked. 102d0cef73dSGregory Neil Shapiro</P> 103d0cef73dSGregory Neil Shapiro 104d0cef73dSGregory Neil Shapiro<DIV ALIGN="center"><A NAME="figure-3"></A> 105d0cef73dSGregory Neil Shapiro<TABLE border=1 cellspacing=0 cellpadding=2 width="70%"> 106d0cef73dSGregory Neil Shapiro<TR bgcolor="#dddddd"><TH>SMTP Commands</TH><TH>Milter Callbacks</TH></TR> 107d0cef73dSGregory Neil Shapiro<TR><TD>(open SMTP connection)</TD><TD>xxfi_connect</TD></TR> 108d0cef73dSGregory Neil Shapiro<TR><TD>HELO ...</TD><TD>xxfi_helo</TD></TR> 109d0cef73dSGregory Neil Shapiro<TR><TD>MAIL From: ...</TD><TD>xxfi_envfrom</TD></TR> 110d0cef73dSGregory Neil Shapiro<TR><TD>RCPT To: ...</TD><TD>xxfi_envrcpt</TD></TR> 111d0cef73dSGregory Neil Shapiro<TR><TD>[more RCPTs]</TD><TD>[xxfi_envrcpt]</TD></TR> 112d0cef73dSGregory Neil Shapiro<TR><TD>DATA</TD><TD>xxfi_data</TD></TR> 113d0cef73dSGregory Neil Shapiro<TR><TD>Header: ...</TD><TD>xxfi_header</TD></TR> 114d0cef73dSGregory Neil Shapiro<TR><TD>[more headers]</TD><TD>[xxfi_header]</TD></TR> 115d0cef73dSGregory Neil Shapiro<TR><TD> </TD><TD>xxfi_eoh</TD></TR> 116d0cef73dSGregory Neil Shapiro<TR><TD>body... </TD><TD>xxfi_body</TD></TR> 117d0cef73dSGregory Neil Shapiro<TR><TD>[more body...]</TD><TD>[xxfi_body]</TD></TR> 118d0cef73dSGregory Neil Shapiro<TR><TD>.</TD><TD>xxfi_eom</TD></TR> 119d0cef73dSGregory Neil Shapiro<TR><TD>QUIT</TD><TD>xxfi_close</TD></TR> 120d0cef73dSGregory Neil Shapiro<TR><TD>(close SMTP connection)</TD><TD> </TD></TR> 121d0cef73dSGregory Neil Shapiro</TABLE> 122d0cef73dSGregory Neil Shapiro<B>Figure 3: Milter callbacks related to an SMTP transaction.</B> 123d0cef73dSGregory Neil Shapiro</DIV> 124d0cef73dSGregory Neil Shapiro 125d0cef73dSGregory Neil Shapiro<P> 12640266059SGregory Neil ShapiroNote that although only a single message is shown above, multiple 127d0cef73dSGregory Neil Shapiromessages may be sent in a single connection. 128d0cef73dSGregory Neil ShapiroNote also that a message or connection may be aborted by 129d0cef73dSGregory Neil Shapiroeither the remote host or the MTA 130d0cef73dSGregory Neil Shapiroat any point during the SMTP transaction. 1316f9c8e5bSGregory Neil ShapiroIf this occurs during a message (between the MAIL command and the final "."), 132d0cef73dSGregory Neil Shapirothe filter's 133d0cef73dSGregory Neil Shapiro<A HREF="xxfi_abort.html">xxfi_abort</A> routine will be called. 134d0cef73dSGregory Neil Shapiro<A HREF="xxfi_close.html">xxfi_close</A> is called any time the 13540266059SGregory Neil Shapiroconnection closes. 13640266059SGregory Neil Shapiro 137d0cef73dSGregory Neil Shapiro<H2><A NAME="Multithreading">Multithreading</A></H2> 13840266059SGregory Neil Shapiro 139d0cef73dSGregory Neil Shapiro<P> 14040266059SGregory Neil ShapiroA single filter process may handle any number of connections 141d0cef73dSGregory Neil Shapirosimultaneously. 142d0cef73dSGregory Neil ShapiroAll filtering callbacks must therefore be reentrant, 14340266059SGregory Neil Shapiroand use some appropriate external synchronization methods to access 144d0cef73dSGregory Neil Shapiroglobal data. 145d0cef73dSGregory Neil ShapiroFurthermore, since there is not a one-to-one correspondence 146d0cef73dSGregory Neil Shapirobetween threads and connections 147d0cef73dSGregory Neil Shapiro(N connections mapped onto M threads, M <= N), 148d0cef73dSGregory Neil Shapiroconnection-specific data must be accessed 149d0cef73dSGregory Neil Shapirothrough the handles provided by the Milter library. 150d0cef73dSGregory Neil ShapiroThe programmer cannot rely on library-supplied thread-specific data blocks 151d0cef73dSGregory Neil Shapiro(e.g., <CODE>pthread_getspecific(3)</CODE>) to store connection-specific data. 152d0cef73dSGregory Neil ShapiroSee the API documentation for 153d0cef73dSGregory Neil Shapiro<A HREF="smfi_setpriv.html">smfi_setpriv</A> and 154d0cef73dSGregory Neil Shapiro<A HREF="smfi_getpriv.html">smfi_getpriv</A> for details. 15540266059SGregory Neil Shapiro 156d0cef73dSGregory Neil Shapiro<H2><A NAME="ResourceManagement">Resource Management</A></H2> 15740266059SGregory Neil Shapiro 158d0cef73dSGregory Neil ShapiroSince filters are likely to be long-lived, 159d0cef73dSGregory Neil Shapiroand to handle many connections, 160d0cef73dSGregory Neil Shapiroproper deallocation of per-connection resources is important. 161d0cef73dSGregory Neil ShapiroThe lifetime of a connection is bracketed by calls to the 162d0cef73dSGregory Neil Shapirocallbacks <A HREF="xxfi_connect.html">xxfi_connect</A> and 163d0cef73dSGregory Neil Shapiro<A HREF="xxfi_close.html">xxfi_close</A>. 164d0cef73dSGregory Neil ShapiroTherefore connection-specific 165d0cef73dSGregory Neil Shapiroresources (accessed via <A HREF="smfi_getpriv.html">smfi_getpriv</A> 166d0cef73dSGregory Neil Shapiroand <A HREF="smfi_setpriv.html">smfi_setpriv</A>) may be allocated in 167d0cef73dSGregory Neil Shapiro<A HREF="xxfi_connect.html">xxfi_connect</A>, 168d0cef73dSGregory Neil Shapiroand should be freed in 169d0cef73dSGregory Neil Shapiro<A HREF="xxfi_close.html">xxfi_close</A>. 170d0cef73dSGregory Neil ShapiroFor further information see 171d0cef73dSGregory Neil Shapirothe <A HREF="api.html#conn-msg">discussion</A> of message- versus 172d0cef73dSGregory Neil Shapiroconnection-oriented routines. 173d0cef73dSGregory Neil ShapiroIn particular, 174d0cef73dSGregory Neil Shapironote that there is only one connection-specific data pointer per connection. 175d0cef73dSGregory Neil Shapiro<P> 17640266059SGregory Neil Shapiro 177d0cef73dSGregory Neil ShapiroEach message is bracketed by calls to 178d0cef73dSGregory Neil Shapiro<A HREF="xxfi_envfrom.html">xxfi_envfrom</A> and 179d0cef73dSGregory Neil Shapiro<A HREF="xxfi_eom.html">xxfi_eom</A> (or 180d0cef73dSGregory Neil Shapiro<A HREF="xxfi_abort.html">xxfi_abort</A>), 181d0cef73dSGregory Neil Shapiroimplying that message-specific resources can be allocated 182d0cef73dSGregory Neil Shapiroand reclaimed in these routines. 183d0cef73dSGregory Neil ShapiroSince the messages in a connection are processed sequentially by each filter, 18440266059SGregory Neil Shapirothere will be only one active message associated with a given 185d0cef73dSGregory Neil Shapiroconnection and filter (and connection-private data block). 186d0cef73dSGregory Neil ShapiroThese resources must still be accessed through 187d0cef73dSGregory Neil Shapiro<A HREF="smfi_getpriv.html">smfi_getpriv</A> and 188d0cef73dSGregory Neil Shapiro<A HREF="smfi_setpriv.html">smfi_setpriv</A>, 189d0cef73dSGregory Neil Shapiroand must be reclaimed in 190d0cef73dSGregory Neil Shapiro<A HREF="xxfi_abort.html">xxfi_abort</A>. 19140266059SGregory Neil Shapiro 192d0cef73dSGregory Neil Shapiro<H2><A NAME="SignalHandling">Signal Handling</A></H2> 19340266059SGregory Neil Shapiro 194d0cef73dSGregory Neil Shapirolibmilter takes care of signal handling, 195d0cef73dSGregory Neil Shapirothe filters are not influenced directly by signals. 19640266059SGregory Neil ShapiroThere are basically two types of signal handlers: 19740266059SGregory Neil Shapiro 198d0cef73dSGregory Neil Shapiro<OL> 199d0cef73dSGregory Neil Shapiro<LI><TT>Stop</TT>: no new connections from the MTA will be accepted, 20040266059SGregory Neil Shapirobut existing connections are allowed to continue. 201d0cef73dSGregory Neil Shapiro<LI><TT>Abort</TT>: all filters will be stopped as soon as the next 20240266059SGregory Neil Shapirocommunication with the MTA happens. 203d0cef73dSGregory Neil Shapiro</OL> 20440266059SGregory Neil Shapiro 205d0cef73dSGregory Neil ShapiroFilters are not terminated asynchronously 206d0cef73dSGregory Neil Shapiro(except by signals that can't be caught). 20740266059SGregory Neil ShapiroIn the case of <TT>Abort</TT> the 208*5b0945b5SGregory Neil Shapiro<A HREF="xxfi_abort.html">xxfi_abort</A> callback is usually invoked 209*5b0945b5SGregory Neil Shapiroif there is an active transaction. 210*5b0945b5SGregory Neil ShapiroHowever, if an invoked callback takes too long to execute 211*5b0945b5SGregory Neil Shapiro(the maximum time <TT>Abort</TT> waits is currently 5s) 212*5b0945b5SGregory Neil Shapiro<!-- XREF: MI_CHK_TIME --> 213*5b0945b5SGregory Neil Shapirothen the filter is simply terminated, i.e., 214*5b0945b5SGregory Neil Shapironeither the 215*5b0945b5SGregory Neil Shapiro<A HREF="xxfi_abort.html">xxfi_abort</A> callback 216*5b0945b5SGregory Neil Shapironor the 217*5b0945b5SGregory Neil Shapiro<A HREF="xxfi_close.html">xxfi_close</A> callback 218*5b0945b5SGregory Neil Shapirois invoked. 21940266059SGregory Neil Shapiro 220d0cef73dSGregory Neil Shapiro<HR size="1"> 221d0cef73dSGregory Neil Shapiro<FONT size="-1"> 222*5b0945b5SGregory Neil ShapiroCopyright (c) 2000, 2001, 2003, 2006, 2018 Proofpoint, Inc. and its suppliers. 22340266059SGregory Neil ShapiroAll rights reserved. 224d0cef73dSGregory Neil Shapiro<BR> 22540266059SGregory Neil ShapiroBy using this file, you agree to the terms and conditions set 2265ef517c0SGregory Neil Shapiroforth in the LICENSE. 227d0cef73dSGregory Neil Shapiro</FONT> 228d0cef73dSGregory Neil Shapiro</BODY> 229d0cef73dSGregory Neil Shapiro</HTML> 230