1d0cef73dSGregory Neil Shapiro<HTML> 2d0cef73dSGregory Neil Shapiro<HEAD> 3d0cef73dSGregory Neil Shapiro<TITLE>Technical Overview</TITLE> 4d0cef73dSGregory Neil Shapiro</HEAD> 5d0cef73dSGregory Neil Shapiro<BODY> 640266059SGregory Neil Shapiro<!-- 74313cc83SGregory Neil Shapiro$Id: overview.html,v 1.22 2013-11-22 20:51:39 ca Exp $ 840266059SGregory Neil Shapiro--> 940266059SGregory Neil Shapiro 10d0cef73dSGregory Neil Shapiro<H1>Technical Overview</H1> 1140266059SGregory Neil Shapiro 12d0cef73dSGregory Neil Shapiro<H2>Contents</H2> 1340266059SGregory Neil Shapiro 14d0cef73dSGregory Neil Shapiro<UL> 15d0cef73dSGregory Neil Shapiro <LI><A HREF="#Initialization">Initialization</A> 16d0cef73dSGregory Neil Shapiro <LI><A HREF="#ControlFlow">Control Flow</A> 17d0cef73dSGregory Neil Shapiro <LI><A HREF="#Multithreading">Multithreading</A> 18d0cef73dSGregory Neil Shapiro <LI><A HREF="#ResourceManagement">Resource Management</A> 19d0cef73dSGregory Neil Shapiro <LI><A HREF="#SignalHandling">Signal Handling</A> 20d0cef73dSGregory Neil Shapiro</UL> 2140266059SGregory Neil Shapiro 22d0cef73dSGregory Neil Shapiro<H2><A NAME="Initialization">Initialization</A></H2> 2340266059SGregory Neil Shapiro 24d0cef73dSGregory Neil ShapiroIn addition to its own initialization, 25d0cef73dSGregory Neil Shapirolibmilter expects a filter to initialize several parameters 26d0cef73dSGregory Neil Shapirobefore calling <A HREF="smfi_main.html">smfi_main</A>: 27d0cef73dSGregory Neil Shapiro<UL> 28d0cef73dSGregory Neil Shapiro <LI>The callbacks the filter wishes to be called, and the types of 29d0cef73dSGregory Neil Shapiro message modification it intends to perform (required, see 30d0cef73dSGregory Neil Shapiro <A HREF="smfi_register.html">smfi_register</A>). 3140266059SGregory Neil Shapiro 32d0cef73dSGregory Neil Shapiro <LI>The socket address to be used when communicating with the MTA 33d0cef73dSGregory Neil Shapiro (required, see <A HREF="smfi_setconn.html">smfi_setconn</A>). 3440266059SGregory Neil Shapiro 35d0cef73dSGregory Neil Shapiro <LI>The number of seconds to wait for MTA connections before 36d0cef73dSGregory Neil Shapiro timing out (optional, see 37d0cef73dSGregory Neil Shapiro <A HREF="smfi_settimeout.html">smfi_settimeout</A>). 38d0cef73dSGregory Neil Shapiro</UL> 39d0cef73dSGregory Neil Shapiro<P> 40d0cef73dSGregory Neil ShapiroIf the filter fails to initialize libmilter, 41d0cef73dSGregory Neil Shapiroor if one or more of the parameters it has passed are invalid, 42d0cef73dSGregory Neil Shapiroa subsequent call to smfi_main will fail. 4340266059SGregory Neil Shapiro 44d0cef73dSGregory Neil Shapiro<H2><A NAME="ControlFlow">Control Flow</A></H2> 4540266059SGregory Neil Shapiro 46d0cef73dSGregory Neil Shapiro<P> 4740266059SGregory Neil ShapiroThe following pseudocode describes the filtering process from the 48d0cef73dSGregory Neil Shapiroperspective of a set of <CODE>N</CODE> MTA's, 49d0cef73dSGregory Neil Shapiroeach corresponding to a connection. 50d0cef73dSGregory Neil ShapiroCallbacks are shown beside the processing stages in which they are invoked; 51d0cef73dSGregory Neil Shapiroif no callbacks are defined for a particular stage, 52d0cef73dSGregory Neil Shapirothat stage may be bypassed. 53d0cef73dSGregory Neil ShapiroThough it is not shown, 54d0cef73dSGregory Neil Shapiroprocessing may be aborted at any time during a message, 55d0cef73dSGregory Neil Shapiroin which case the 56d0cef73dSGregory Neil Shapiro<A HREF="xxfi_abort.html">xxfi_abort</A> callback is invoked and control 57d0cef73dSGregory Neil Shapiroreturns to <CODE>MESSAGE</CODE>. 58d0cef73dSGregory Neil Shapiro<P> 59d0cef73dSGregory Neil Shapiro<PRE> 6040266059SGregory Neil ShapiroFor each of N connections 6140266059SGregory Neil Shapiro{ 6240266059SGregory Neil Shapiro For each filter 63*d39bd2c1SGregory Neil Shapiro negotiate MTA/milter capabilities/requirements (<A HREF="xxfi_negotiate.html">xxfi_negotiate</A>) 645b0945b5SGregory Neil Shapiro For each filter 659bd497b8SGregory Neil Shapiro process connection (<A HREF="xxfi_connect.html">xxfi_connect</A>) 669bd497b8SGregory Neil Shapiro For each filter 679bd497b8SGregory Neil Shapiro process helo/ehlo (<A HREF="xxfi_helo.html">xxfi_helo</A>) 6840266059SGregory Neil ShapiroMESSAGE:For each message in this connection (sequentially) 6940266059SGregory Neil Shapiro { 7040266059SGregory Neil Shapiro For each filter 71d0cef73dSGregory Neil Shapiro process sender (<A HREF="xxfi_envfrom.html">xxfi_envfrom</A>) 7240266059SGregory Neil Shapiro For each recipient 7340266059SGregory Neil Shapiro { 7440266059SGregory Neil Shapiro For each filter 75d0cef73dSGregory Neil Shapiro process recipient (<A HREF="xxfi_envrcpt.html">xxfi_envrcpt</A>) 7640266059SGregory Neil Shapiro } 7740266059SGregory Neil Shapiro For each filter 78d0cef73dSGregory Neil Shapiro process DATA (<A HREF="xxfi_data.html">xxfi_data</A>) 792fb4f839SGregory Neil Shapiro For each filter 802fb4f839SGregory Neil Shapiro { 8140266059SGregory Neil Shapiro For each header 82d0cef73dSGregory Neil Shapiro process header (<A HREF="xxfi_header.html">xxfi_header</A>) 83d0cef73dSGregory Neil Shapiro process end of headers (<A HREF="xxfi_eoh.html">xxfi_eoh</A>) 8440266059SGregory Neil Shapiro For each body block 85d0cef73dSGregory Neil Shapiro process this body block (<A HREF="xxfi_body.html">xxfi_body</A>) 86d0cef73dSGregory Neil Shapiro process end of message (<A HREF="xxfi_eom.html">xxfi_eom</A>) 8740266059SGregory Neil Shapiro } 8840266059SGregory Neil Shapiro } 8940266059SGregory Neil Shapiro For each filter 90d0cef73dSGregory Neil Shapiro process end of connection (<A HREF="xxfi_close.html">xxfi_close</A>) 9140266059SGregory Neil Shapiro} 92d0cef73dSGregory Neil Shapiro</PRE> 9340266059SGregory Neil Shapiro 9440266059SGregory Neil Shapiro<P>Note: Filters are contacted in order defined in config file.</P> 9540266059SGregory Neil Shapiro 9640266059SGregory Neil Shapiro<P> 9740266059SGregory Neil ShapiroTo write a filter, a vendor supplies callbacks to process relevant 98d0cef73dSGregory Neil Shapiroparts of a message transaction. 99d0cef73dSGregory Neil ShapiroThe library then controls all sequencing, threading, 100d0cef73dSGregory Neil Shapiroand protocol exchange with the MTA. 101d0cef73dSGregory Neil Shapiro<A HREF="#figure-3">Figure 3</A> outlines control flow for a filter 10240266059SGregory Neil Shapiroprocess, showing where different callbacks are invoked. 103d0cef73dSGregory Neil Shapiro</P> 104d0cef73dSGregory Neil Shapiro 105d0cef73dSGregory Neil Shapiro<DIV ALIGN="center"><A NAME="figure-3"></A> 106d0cef73dSGregory Neil Shapiro<TABLE border=1 cellspacing=0 cellpadding=2 width="70%"> 107d0cef73dSGregory Neil Shapiro<TR bgcolor="#dddddd"><TH>SMTP Commands</TH><TH>Milter Callbacks</TH></TR> 108d0cef73dSGregory Neil Shapiro<TR><TD>(open SMTP connection)</TD><TD>xxfi_connect</TD></TR> 109d0cef73dSGregory Neil Shapiro<TR><TD>HELO ...</TD><TD>xxfi_helo</TD></TR> 110d0cef73dSGregory Neil Shapiro<TR><TD>MAIL From: ...</TD><TD>xxfi_envfrom</TD></TR> 111d0cef73dSGregory Neil Shapiro<TR><TD>RCPT To: ...</TD><TD>xxfi_envrcpt</TD></TR> 112d0cef73dSGregory Neil Shapiro<TR><TD>[more RCPTs]</TD><TD>[xxfi_envrcpt]</TD></TR> 113d0cef73dSGregory Neil Shapiro<TR><TD>DATA</TD><TD>xxfi_data</TD></TR> 114d0cef73dSGregory Neil Shapiro<TR><TD>Header: ...</TD><TD>xxfi_header</TD></TR> 115d0cef73dSGregory Neil Shapiro<TR><TD>[more headers]</TD><TD>[xxfi_header]</TD></TR> 116d0cef73dSGregory Neil Shapiro<TR><TD> </TD><TD>xxfi_eoh</TD></TR> 117d0cef73dSGregory Neil Shapiro<TR><TD>body... </TD><TD>xxfi_body</TD></TR> 118d0cef73dSGregory Neil Shapiro<TR><TD>[more body...]</TD><TD>[xxfi_body]</TD></TR> 119d0cef73dSGregory Neil Shapiro<TR><TD>.</TD><TD>xxfi_eom</TD></TR> 120d0cef73dSGregory Neil Shapiro<TR><TD>QUIT</TD><TD>xxfi_close</TD></TR> 121d0cef73dSGregory Neil Shapiro<TR><TD>(close SMTP connection)</TD><TD> </TD></TR> 122d0cef73dSGregory Neil Shapiro</TABLE> 123d0cef73dSGregory Neil Shapiro<B>Figure 3: Milter callbacks related to an SMTP transaction.</B> 124d0cef73dSGregory Neil Shapiro</DIV> 125d0cef73dSGregory Neil Shapiro 126d0cef73dSGregory Neil Shapiro<P> 12740266059SGregory Neil ShapiroNote that although only a single message is shown above, multiple 128d0cef73dSGregory Neil Shapiromessages may be sent in a single connection. 129d0cef73dSGregory Neil ShapiroNote also that a message or connection may be aborted by 130d0cef73dSGregory Neil Shapiroeither the remote host or the MTA 131d0cef73dSGregory Neil Shapiroat any point during the SMTP transaction. 1326f9c8e5bSGregory Neil ShapiroIf this occurs during a message (between the MAIL command and the final "."), 133d0cef73dSGregory Neil Shapirothe filter's 134d0cef73dSGregory Neil Shapiro<A HREF="xxfi_abort.html">xxfi_abort</A> routine will be called. 135d0cef73dSGregory Neil Shapiro<A HREF="xxfi_close.html">xxfi_close</A> is called any time the 13640266059SGregory Neil Shapiroconnection closes. 13740266059SGregory Neil Shapiro 138d0cef73dSGregory Neil Shapiro<H2><A NAME="Multithreading">Multithreading</A></H2> 13940266059SGregory Neil Shapiro 140d0cef73dSGregory Neil Shapiro<P> 14140266059SGregory Neil ShapiroA single filter process may handle any number of connections 142d0cef73dSGregory Neil Shapirosimultaneously. 143d0cef73dSGregory Neil ShapiroAll filtering callbacks must therefore be reentrant, 14440266059SGregory Neil Shapiroand use some appropriate external synchronization methods to access 145d0cef73dSGregory Neil Shapiroglobal data. 146d0cef73dSGregory Neil ShapiroFurthermore, since there is not a one-to-one correspondence 147d0cef73dSGregory Neil Shapirobetween threads and connections 148d0cef73dSGregory Neil Shapiro(N connections mapped onto M threads, M <= N), 149d0cef73dSGregory Neil Shapiroconnection-specific data must be accessed 150d0cef73dSGregory Neil Shapirothrough the handles provided by the Milter library. 151d0cef73dSGregory Neil ShapiroThe programmer cannot rely on library-supplied thread-specific data blocks 152d0cef73dSGregory Neil Shapiro(e.g., <CODE>pthread_getspecific(3)</CODE>) to store connection-specific data. 153d0cef73dSGregory Neil ShapiroSee the API documentation for 154d0cef73dSGregory Neil Shapiro<A HREF="smfi_setpriv.html">smfi_setpriv</A> and 155d0cef73dSGregory Neil Shapiro<A HREF="smfi_getpriv.html">smfi_getpriv</A> for details. 15640266059SGregory Neil Shapiro 157d0cef73dSGregory Neil Shapiro<H2><A NAME="ResourceManagement">Resource Management</A></H2> 15840266059SGregory Neil Shapiro 159d0cef73dSGregory Neil ShapiroSince filters are likely to be long-lived, 160d0cef73dSGregory Neil Shapiroand to handle many connections, 161d0cef73dSGregory Neil Shapiroproper deallocation of per-connection resources is important. 162d0cef73dSGregory Neil ShapiroThe lifetime of a connection is bracketed by calls to the 163d0cef73dSGregory Neil Shapirocallbacks <A HREF="xxfi_connect.html">xxfi_connect</A> and 164d0cef73dSGregory Neil Shapiro<A HREF="xxfi_close.html">xxfi_close</A>. 165d0cef73dSGregory Neil ShapiroTherefore connection-specific 166d0cef73dSGregory Neil Shapiroresources (accessed via <A HREF="smfi_getpriv.html">smfi_getpriv</A> 167d0cef73dSGregory Neil Shapiroand <A HREF="smfi_setpriv.html">smfi_setpriv</A>) may be allocated in 168d0cef73dSGregory Neil Shapiro<A HREF="xxfi_connect.html">xxfi_connect</A>, 169d0cef73dSGregory Neil Shapiroand should be freed in 170d0cef73dSGregory Neil Shapiro<A HREF="xxfi_close.html">xxfi_close</A>. 171d0cef73dSGregory Neil ShapiroFor further information see 172d0cef73dSGregory Neil Shapirothe <A HREF="api.html#conn-msg">discussion</A> of message- versus 173d0cef73dSGregory Neil Shapiroconnection-oriented routines. 174d0cef73dSGregory Neil ShapiroIn particular, 175d0cef73dSGregory Neil Shapironote that there is only one connection-specific data pointer per connection. 176d0cef73dSGregory Neil Shapiro<P> 17740266059SGregory Neil Shapiro 178d0cef73dSGregory Neil ShapiroEach message is bracketed by calls to 179d0cef73dSGregory Neil Shapiro<A HREF="xxfi_envfrom.html">xxfi_envfrom</A> and 180d0cef73dSGregory Neil Shapiro<A HREF="xxfi_eom.html">xxfi_eom</A> (or 181d0cef73dSGregory Neil Shapiro<A HREF="xxfi_abort.html">xxfi_abort</A>), 182d0cef73dSGregory Neil Shapiroimplying that message-specific resources can be allocated 183d0cef73dSGregory Neil Shapiroand reclaimed in these routines. 184d0cef73dSGregory Neil ShapiroSince the messages in a connection are processed sequentially by each filter, 18540266059SGregory Neil Shapirothere will be only one active message associated with a given 186d0cef73dSGregory Neil Shapiroconnection and filter (and connection-private data block). 187d0cef73dSGregory Neil ShapiroThese resources must still be accessed through 188d0cef73dSGregory Neil Shapiro<A HREF="smfi_getpriv.html">smfi_getpriv</A> and 189d0cef73dSGregory Neil Shapiro<A HREF="smfi_setpriv.html">smfi_setpriv</A>, 190d0cef73dSGregory Neil Shapiroand must be reclaimed in 191d0cef73dSGregory Neil Shapiro<A HREF="xxfi_abort.html">xxfi_abort</A>. 19240266059SGregory Neil Shapiro 193d0cef73dSGregory Neil Shapiro<H2><A NAME="SignalHandling">Signal Handling</A></H2> 19440266059SGregory Neil Shapiro 195d0cef73dSGregory Neil Shapirolibmilter takes care of signal handling, 196d0cef73dSGregory Neil Shapirothe filters are not influenced directly by signals. 19740266059SGregory Neil ShapiroThere are basically two types of signal handlers: 19840266059SGregory Neil Shapiro 199d0cef73dSGregory Neil Shapiro<OL> 200d0cef73dSGregory Neil Shapiro<LI><TT>Stop</TT>: no new connections from the MTA will be accepted, 20140266059SGregory Neil Shapirobut existing connections are allowed to continue. 202d0cef73dSGregory Neil Shapiro<LI><TT>Abort</TT>: all filters will be stopped as soon as the next 20340266059SGregory Neil Shapirocommunication with the MTA happens. 204d0cef73dSGregory Neil Shapiro</OL> 20540266059SGregory Neil Shapiro 206d0cef73dSGregory Neil ShapiroFilters are not terminated asynchronously 207d0cef73dSGregory Neil Shapiro(except by signals that can't be caught). 20840266059SGregory Neil ShapiroIn the case of <TT>Abort</TT> the 2095b0945b5SGregory Neil Shapiro<A HREF="xxfi_abort.html">xxfi_abort</A> callback is usually invoked 2105b0945b5SGregory Neil Shapiroif there is an active transaction. 2115b0945b5SGregory Neil ShapiroHowever, if an invoked callback takes too long to execute 2125b0945b5SGregory Neil Shapiro(the maximum time <TT>Abort</TT> waits is currently 5s) 2135b0945b5SGregory Neil Shapiro<!-- XREF: MI_CHK_TIME --> 2145b0945b5SGregory Neil Shapirothen the filter is simply terminated, i.e., 2155b0945b5SGregory Neil Shapironeither the 2165b0945b5SGregory Neil Shapiro<A HREF="xxfi_abort.html">xxfi_abort</A> callback 2175b0945b5SGregory Neil Shapironor the 2185b0945b5SGregory Neil Shapiro<A HREF="xxfi_close.html">xxfi_close</A> callback 2195b0945b5SGregory Neil Shapirois invoked. 22040266059SGregory Neil Shapiro 221d0cef73dSGregory Neil Shapiro<HR size="1"> 222d0cef73dSGregory Neil Shapiro<FONT size="-1"> 2235b0945b5SGregory Neil ShapiroCopyright (c) 2000, 2001, 2003, 2006, 2018 Proofpoint, Inc. and its suppliers. 22440266059SGregory Neil ShapiroAll rights reserved. 225d0cef73dSGregory Neil Shapiro<BR> 22640266059SGregory Neil ShapiroBy using this file, you agree to the terms and conditions set 2275ef517c0SGregory Neil Shapiroforth in the LICENSE. 228d0cef73dSGregory Neil Shapiro</FONT> 229d0cef73dSGregory Neil Shapiro</BODY> 230d0cef73dSGregory Neil Shapiro</HTML> 231