xref: /freebsd/share/man/man4/siftr.4 (revision 7aa383846770374466b1dcb2cefd71bde9acf463)
1.\"
2.\" Copyright (c) 2010 The FreeBSD Foundation
3.\" All rights reserved.
4.\"
5.\" Portions of this software were developed at the Centre for Advanced
6.\" Internet Architectures, Swinburne University of Technology, Melbourne,
7.\" Australia by Lawrence Stewart under sponsorship from the FreeBSD
8.\" Foundation.
9.\"
10.\" Redistribution and use in source and binary forms, with or without
11.\" modification, are permitted provided that the following conditions
12.\" are met:
13.\" 1. Redistributions of source code must retain the above copyright
14.\"    notice, this list of conditions, and the following disclaimer,
15.\"    without modification, immediately at the beginning of the file.
16.\" 2. The name of the author may not be used to endorse or promote products
17.\"    derived from this software without specific prior written permission.
18.\"
19.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
20.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
21.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
22.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR
23.\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
24.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
25.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
26.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
27.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
28.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
29.\" SUCH DAMAGE.
30.\"
31.\" $FreeBSD$
32.\"
33.Dd June 23, 2010
34.Dt SIFTR 4
35.Os
36.Sh NAME
37.Nm SIFTR
38.Nd Statistical Information For TCP Research
39.Sh SYNOPSIS
40To load
41.Ns Nm
42as a module at run-time, run the following command as root:
43.Bd -literal -offset indent
44kldload siftr
45.Ed
46.Pp
47Alternatively, to load
48.Ns Nm
49as a module at boot time, add the following line into the
50.Xr loader.conf 5
51file:
52.Bd -literal -offset indent
53siftr_load="YES"
54.Ed
55.Sh DESCRIPTION
56.Nm
57.Ns ( Em S Ns tatistical
58.Em I Ns nformation
59.Em F Ns or
60.Em T Ns CP
61.Em R Ns esearch )
62is a kernel module that logs a range of statistics on active TCP connections to
63a log file.
64It provides the ability to make highly granular measurements of TCP connection
65state, aimed at system administrators, developers and researchers.
66.Ss Compile-time Configuration
67The default operation of
68.Nm
69is to capture IPv4 TCP/IP packets.
70.Nm
71can be configured to support IPv4 and IPv6 by uncommenting:
72.Bd -literal -offset indent
73CFLAGS+=-DSIFTR_IPV6
74.Ed
75.Pp
76in
77.Aq sys/modules/siftr/Makefile
78and recompiling.
79.Pp
80In the IPv4-only (default) mode, standard dotted decimal notation (e.g.
81"136.186.229.95") is used to format IPv4 addresses for logging.
82In IPv6 mode, standard dotted decimal notation is used to format IPv4 addresses,
83and standard colon-separated hex notation (see RFC 4291) is used to format IPv6
84addresses for logging. Note that SIFTR uses uncompressed notation to format IPv6
85addresses.
86For example, the address "fe80::20f:feff:fea2:531b" would be logged as
87"fe80:0:0:0:20f:feff:fea2:531b".
88.Ss Run-time Configuration
89.Nm
90utilises the
91.Xr sysctl 8
92interface to export its configuration variables to user-space.
93The following variables are available:
94.Bl -tag -offset indent
95.It Va net.inet.siftr.enabled
96controls whether the module performs its
97measurements or not.
98By default, the value is set to 0, which means the module
99will not be taking any measurements.
100Having the module loaded with
101.Va net.inet.siftr.enabled
102set to 0 will have no impact on the performance of the network stack, as the
103packet filtering hooks are only inserted when
104.Va net.inet.siftr.enabled
105is set to 1.
106.El
107.Bl -tag -offset indent
108.It Va net.inet.siftr.ppl
109controls how many inbound/outbound packets for a given TCP connection will cause
110a log message to be generated for the connection.
111By default, the value is set to 1, which means the module will log a message for
112every packet of every TCP connection.
113The value can be set to any integer in the range [1,2^32], and can be changed at
114any time, even while the module is enabled.
115.El
116.Bl -tag -offset indent
117.It Va net.inet.siftr.logfile
118controls the path to the file that the module writes its log messages to.
119By default, the file /var/log/siftr.log is used.
120The path can be changed at any time, even while the module is enabled.
121.El
122.Bl -tag -offset indent
123.It Va net.inet.siftr.genhashes
124controls whether a hash is generated for each TCP packet seen by
125.Nm .
126By default, the value is set to 0, which means no hashes are generated.
127The hashes are useful to correlate which TCP packet triggered the generation of
128a particular log message, but calculating them adds additional computational
129overhead into the fast path.
130.El
131.Ss Log Format
132A typical
133.Nm
134log file will contain 3 different types of log message.
135All messages are written in plain ASCII text.
136.Pp
137Note: The
138.Qq \e
139present in the example log messages in this section indicates a
140line continuation and is not part of the actual log message
141.Pp
142The first type of log message is written to the file when the module is
143enabled and starts collecting data from the running kernel. The text below
144shows an example module enable log. The fields are tab delimited key-value
145pairs which describe some basic information about the system.
146.Bd -literal -offset indent
147enable_time_secs=1238556193    enable_time_usecs=462104 \\
148siftrver=1.2.2    hz=1000    tcp_rtt_scale=32 \\
149sysname=FreeBSD    sysver=604000    ipmode=4
150.Ed
151.Pp
152Field descriptions are as follows:
153.Bl -tag -offset indent
154.It Va enable_time_secs
155time at which the module was enabled, in seconds since the UNIX epoch.
156.El
157.Bl -tag -offset indent
158.It Va enable_time_usecs
159time at which the module was enabled, in microseconds since enable_time_secs.
160.El
161.Bl -tag -offset indent
162.It Va siftrver
163version of
164.Nm .
165.El
166.Bl -tag -offset indent
167.It Va hz
168tick rate of the kernel in ticks per second.
169.El
170.Bl -tag -offset indent
171.It Va tcp_rtt_scale
172smoothed RTT estimate scaling factor
173.El
174.Bl -tag -offset indent
175.It Va sysname
176operating system name
177.El
178.Bl -tag -offset indent
179.It Va sysver
180operating system version
181.El
182.Bl -tag -offset indent
183.It Va ipmode
184IP mode as defined at compile time.
185An ipmode of "4" means IPv6 is not supported and IP addresses are logged in
186regular dotted quad format.
187An ipmode of "6" means IPv6 is supported, and IP addresses are logged in dotted
188quad or hex format, as described in the
189.Qq Compile-time Configuration
190subsection.
191.El
192.Pp
193The second type of log message is written to the file when a data log message
194is generated.
195The text below shows an example data log triggered by an IPv4
196TCP/IP packet.
197The data is CSV formatted.
198.Bd -literal -offset indent
199o,0xbec491a5,1238556193.463551,172.16.7.28,22,172.16.2.5,55931, \\
2001073725440,172312,6144,66560,66608,8,1,4,1448,936,1,996,255, \\
20133304,208,66608,0,208
202.Ed
203.Pp
204Field descriptions are as follows:
205.Bl -tag -offset indent
206.It Va 1
207Direction of packet that triggered the log message.
208Either
209.Qq i
210for in, or
211.Qq o
212for out.
213.El
214.Bl -tag -offset indent
215.It Va 2
216Hash of the packet that triggered the log message.
217.El
218.Bl -tag -offset indent
219.It Va 3
220Time at which the packet that triggered the log message was processed by
221the
222.Xr pfil 9
223hook function, in seconds and microseconds since the UNIX epoch.
224.El
225.Bl -tag -offset indent
226.It Va 4
227The IPv4 or IPv6 address of the local host, in dotted quad (IPv4 packet)
228or colon-separated hex (IPv6 packet) notation.
229.El
230.Bl -tag -offset indent
231.It Va 5
232The TCP port that the local host is communicating via.
233.El
234.Bl -tag -offset indent
235.It Va 6
236The IPv4 or IPv6 address of the foreign host, in dotted quad (IPv4 packet)
237or colon-separated hex (IPv6 packet) notation.
238.El
239.Bl -tag -offset indent
240.It Va 7
241The TCP port that the foreign host is communicating via.
242.El
243.Bl -tag -offset indent
244.It Va 8
245The slow start threshold for the flow, in bytes.
246.El
247.Bl -tag -offset indent
248.It Va 9
249The current congestion window for the flow, in bytes.
250.El
251.Bl -tag -offset indent
252.It Va 10
253The current bandwidth-controlled window for the flow, in bytes.
254.El
255.Bl -tag -offset indent
256.It Va 11
257The current sending window for the flow, in bytes.
258The post scaled value is reported, except during the initial handshake (first
259few packets), during which time the unscaled value is reported.
260.El
261.Bl -tag -offset indent
262.It Va 12
263The current receive window for the flow, in bytes.
264The post scaled value is always reported.
265.El
266.Bl -tag -offset indent
267.It Va 13
268The current window scaling factor for the sending window.
269.El
270.Bl -tag -offset indent
271.It Va 14
272The current window scaling factor for the receiving window.
273.El
274.Bl -tag -offset indent
275.It Va 15
276The current state of the TCP finite state machine, as defined
277in
278.Aq Pa netinet/tcp_fsm.h .
279.El
280.Bl -tag -offset indent
281.It Va 16
282The maximum segment size for the flow, in bytes.
283.El
284.Bl -tag -offset indent
285.It Va 17
286The current smoothed RTT estimate for the flow, in units of TCP_RTT_SCALE * HZ,
287where TCP_RTT_SCALE is a define found in tcp_var.h, and HZ is the kernel's tick
288timer.
289Divide by TCP_RTT_SCALE * HZ to get the RTT in secs. TCP_RTT_SCALE and HZ are
290reported in the enable log message.
291.El
292.Bl -tag -offset indent
293.It Va 18
294SACK enabled indicator. 1 if SACK enabled, 0 otherwise.
295.El
296.Bl -tag -offset indent
297.It Va 19
298The current state of the TCP flags for the flow.
299See
300.Aq Pa netinet/tcp_var.h
301for information about the various flags.
302.El
303.Bl -tag -offset indent
304.It Va 20
305The current retransmission timeout length for the flow, in units of HZ, where HZ
306is the kernel's tick timer.
307Divide by HZ to get the timeout length in seconds. HZ is reported in the
308enable log message.
309.El
310.Bl -tag -offset indent
311.It Va 21
312The current size of the socket send buffer in bytes.
313.El
314.Bl -tag -offset indent
315.It Va 22
316The current number of bytes in the socket send buffer.
317.El
318.Bl -tag -offset indent
319.It Va 23
320The current size of the socket receive buffer in bytes.
321.El
322.Bl -tag -offset indent
323.It Va 24
324The current number of bytes in the socket receive buffer.
325.El
326.Bl -tag -offset indent
327.It Va 25
328The current number of unacknowledged bytes in-flight.
329Bytes acknowledged via SACK are not excluded from this count.
330.El
331.Pp
332The third type of log message is written to the file when the module is disabled
333and ceases collecting data from the running kernel.
334The text below shows an example module disable log.
335The fields are tab delimited key-value pairs which provide statistics about
336operations since the module was most recently enabled.
337.Bd -literal -offset indent
338disable_time_secs=1238556197    disable_time_usecs=933607 \\
339num_inbound_tcp_pkts=356    num_outbound_tcp_pkts=627 \\
340total_tcp_pkts=983    num_inbound_skipped_pkts_malloc=0 \\
341num_outbound_skipped_pkts_malloc=0    num_inbound_skipped_pkts_mtx=0 \\
342num_outbound_skipped_pkts_mtx=0    num_inbound_skipped_pkts_tcb=0 \\
343num_outbound_skipped_pkts_tcb=0    num_inbound_skipped_pkts_icb=0 \\
344num_outbound_skipped_pkts_icb=0    total_skipped_tcp_pkts=0 \\
345flow_list=172.16.7.28;22-172.16.2.5;55931,
346.Ed
347.Pp
348Field descriptions are as follows:
349.Bl -tag -offset indent
350.It Va disable_time_secs
351Time at which the module was disabled, in seconds since the UNIX epoch.
352.El
353.Bl -tag -offset indent
354.It Va disable_time_usecs
355Time at which the module was disabled, in microseconds since disable_time_secs.
356.El
357.Bl -tag -offset indent
358.It Va num_inbound_tcp_pkts
359Number of TCP packets that traversed up the network stack.
360This only includes inbound TCP packets during the periods when
361.Nm
362was enabled.
363.El
364.Bl -tag -offset indent
365.It Va num_outbound_tcp_pkts
366Number of TCP packets that traversed down the network stack.
367This only includes outbound TCP packets during the periods when
368.Nm
369was enabled.
370.El
371.Bl -tag -offset indent
372.It Va total_tcp_pkts
373The summation of num_inbound_tcp_pkts and num_outbound_tcp_pkts.
374.El
375.Bl -tag -offset indent
376.It Va num_inbound_skipped_pkts_malloc
377Number of inbound packets that were not processed because of failed malloc() calls.
378.El
379.Bl -tag -offset indent
380.It Va num_outbound_skipped_pkts_malloc
381Number of outbound packets that were not processed because of failed malloc() calls.
382.El
383.Bl -tag -offset indent
384.It Va num_inbound_skipped_pkts_mtx
385Number of inbound packets that were not processed because of failure to add the
386packet to the packet processing queue.
387.El
388.Bl -tag -offset indent
389.It Va num_outbound_skipped_pkts_mtx
390Number of outbound packets that were not processed because of failure to add the
391packet to the packet processing queue.
392.El
393.Bl -tag -offset indent
394.It Va num_inbound_skipped_pkts_tcb
395Number of inbound packets that were not processed because of failure to find the
396TCP control block associated with the packet.
397.El
398.Bl -tag -offset indent
399.It Va num_outbound_skipped_pkts_tcb
400Number of outbound packets that were not processed because of failure to find
401the TCP control block associated with the packet.
402.El
403.Bl -tag -offset indent
404.It Va num_inbound_skipped_pkts_icb
405Number of inbound packets that were not processed because of failure to find the
406IP control block associated with the packet.
407.El
408.Bl -tag -offset indent
409.It Va num_outbound_skipped_pkts_icb
410Number of outbound packets that were not processed because of failure to find
411the IP control block associated with the packet.
412.El
413.Bl -tag -offset indent
414.It Va total_skipped_tcp_pkts
415The summation of all skipped packet counters.
416.El
417.Bl -tag -offset indent
418.It Va flow_list
419A CSV list of TCP flows that triggered data log messages to be generated since
420the module was loaded.
421Each flow entry in the CSV list is
422formatted as
423.Qq local_ip;local_port-foreign_ip;foreign_port .
424If there are no entries in the list (i.e. no data log messages were generated),
425the value will be blank.
426If there is at least one entry in the list, a trailing comma will always be
427present.
428.El
429.Pp
430The total number of data log messages found in the log file for a module
431enable/disable cycle should equate to total_tcp_pkts - total_skipped_tcp_pkts.
432.Sh IMPLEMENTATION NOTES
433.Nm
434hooks into the network stack using the
435.Xr pfil 9
436interface.
437In its current incarnation, it hooks into the AF_INET/AF_INET6 (IPv4/IPv6)
438.Xr pfil 9
439filtering points, which means it sees packets at the IP layer of the network
440stack.
441This means that TCP packets inbound to the stack are intercepted before
442they have been processed by the TCP layer.
443Packets outbound from the stack are intercepted after they have been processed
444by the TCP layer.
445.Pp
446The diagram below illustrates how
447.Nm
448inserts itself into the stack.
449.Bd -literal -offset indent
450----------------------------------
451           Upper Layers
452----------------------------------
453    ^                       |
454    |                       |
455    |                       |
456    |                       v
457 TCP in                  TCP out
458----------------------------------
459    ^                      |
460    |________     _________|
461            |     |
462            |     v
463           ---------
464           | SIFTR |
465           ---------
466            ^     |
467    ________|     |__________
468    |                       |
469    |                       v
470IPv{4/6} in            IPv{4/6} out
471----------------------------------
472    ^                       |
473    |                       |
474    |                       v
475Layer 2 in             Layer 2 out
476----------------------------------
477          Physical Layer
478----------------------------------
479.Ed
480.Pp
481.Nm
482uses the
483.Xr alq 9
484interface to manage writing data to disk.
485.Pp
486At first glance, you might mistakenly think that
487.Nm
488extracts information from
489individual TCP packets.
490This is not the case.
491.Nm
492uses TCP packet events (inbound and outbound) for each TCP flow originating from
493the system to trigger a dump of the state of the TCP control block for that
494flow.
495With the PPL set to 1, we are in effect sampling each TCP flow's control block
496state as frequently as flow packets enter/leave the system.
497For example, setting PPL to 2 halves the sampling rate i.e. every second flow
498packet (inbound OR outbound) causes a dump of the control block state.
499.Pp
500The distinction between interrogating individual packets vs interrogating the
501control block is important, because
502.Nm
503does not remove the need for packet capturing tools like
504.Xr tcpdump 1 .
505.Nm
506allows you to correlate and observe the cause-and-affect relationship between
507what you see on the wire (captured using a tool like
508.Xr tcpdump 1 Ns )
509and changes in the TCP control block corresponding to the flow of interest.
510It is therefore useful to use
511.Nm
512and a tool like
513.Xr tcpdump 1
514to gather the necessary data to piece together the complete picture.
515Use of either tool on its own will not be able to provide all of the necessary
516data.
517.Pp
518As a result of needing to interrogate the TCP control block, certain packets
519during the lifecycle of a connection are unable to trigger a
520.Nm
521log message.
522The initial handshake takes place without the existence of a control block and
523the final ACK is exchanged when the connection is in the TIMEWAIT state.
524.Pp
525.Nm
526was designed to minimise the delay introduced to packets traversing the network
527stack.
528This design called for a highly optimised and minimal hook function that
529extracted the minimal details necessary whilst holding the packet up, and
530passing these details to another thread for actual processing and logging.
531.Pp
532This multithreaded design does introduce some contention issues when accessing
533the data structure shared between the threads of operation.
534When the hook function tries to place details in the structure, it must first
535acquire an exclusive lock.
536Likewise, when the processing thread tries to read details from the structure,
537it must also acquire an exclusive lock to do so.
538If one thread holds the lock, the other must wait before it can obtain it.
539This does introduce some additional bounded delay into the kernel's packet
540processing code path.
541.Pp
542In some cases (e.g. low memory, connection termination), TCP packets that enter
543the
544.Nm
545.Xr pfil 9
546hook function will not trigger a log message to be generated.
547.Nm
548refers to this outcome as a
549.Qq skipped packet .
550Note that
551.Nm
552always ensures that packets are allowed to continue through the stack, even if
553they could not successfully trigger a data log message.
554.Nm
555will therefore not introduce any packet loss for TCP/IP packets traversing the
556network stack.
557.Ss Important Behaviours
558The behaviour of a log file path change whilst the module is enabled is as
559follows:
560.Bl -enum
561.It
562Attempt to open the new file path for writing.
563If this fails, the path change will fail and the existing path will continue to
564be used.
565.It
566Assuming the new path is valid and opened successfully:
567.Bl -dash
568.It
569Flush all pending log messages to the old file path.
570.It
571Close the old file path.
572.It
573Switch the active log file pointer to point at the new file path.
574.It
575Commence logging to the new file.
576.El
577.El
578.Pp
579During the time between the flush of pending log messages to the old file and
580commencing logging to the new file, new log messages will still be generated and
581buffered.
582As soon as the new file path is ready for writing, the accumulated log messages
583will be written out to the file.
584.Sh EXAMPLES
585To enable the module's operations, run the following command as root:
586sysctl net.inet.siftr.enabled=1
587.Pp
588To change the granularity of log messages such that 1 log message is
589generated for every 10 TCP packets per connection, run the following
590command as root:
591sysctl net.inet.siftr.ppl=10
592.Pp
593To change the log file location to /tmp/siftr.log, run the following
594command as root:
595sysctl net.inet.siftr.logfile=/tmp/siftr.log
596.Sh SEE ALSO
597.Xr alq 9 ,
598.Xr pfil 9
599.Xr sysctl 8 ,
600.Xr tcp 4 ,
601.Xr tcpdump 1 ,
602.Sh ACKNOWLEDGEMENTS
603Development of this software was made possible in part by grants from the
604Cisco University Research Program Fund at Community Foundation Silicon Valley,
605and the FreeBSD Foundation.
606.Sh HISTORY
607.Nm
608first appeared in
609.Fx 9.0 .
610.Pp
611.Nm
612was first released in 2007 by Lawrence Stewart and James Healy whilst working on
613the NewTCP research project at Swinburne University's Centre for Advanced
614Internet Architectures, Melbourne, Australia, which was made possible in part by
615a grant from the Cisco University Research Program Fund at Community Foundation
616Silicon Valley.
617More details are available at:
618.Pp
619http://caia.swin.edu.au/urp/newtcp/
620.Pp
621Work on
622.Nm
623v1.2.x was sponsored by the FreeBSD Foundation as part of
624the
625.Qq Enhancing the FreeBSD TCP Implementation
626project 2008-2009.
627More details are available at:
628.Pp
629http://www.freebsdfoundation.org/
630.Pp
631http://caia.swin.edu.au/freebsd/etcp09/
632.Sh AUTHORS
633.An -nosplit
634.Nm
635was written by
636.An Lawrence Stewart Aq lstewart@FreeBSD.org
637and
638.An James Healy Aq jimmy@deefa.com .
639.Pp
640This manual page was written by
641.An Lawrence Stewart Aq lstewart@FreeBSD.org .
642.Sh BUGS
643Current known limitations and any relevant workarounds are outlined below:
644.Bl -dash
645.It
646The internal queue used to pass information between the threads of operation is
647currently unbounded.
648This allows
649.Nm
650to cope with bursty network traffic, but sustained high packet-per-second
651traffic can cause exhaustion of kernel memory if the processing thread cannot
652keep up with the packet rate.
653.It
654If using
655.Nm
656on a machine that is also running other modules utilising the
657.Xr pfil 9
658framework e.g.
659.Xr dummynet 4 ,
660.Xr ipfw 8 ,
661.Xr pf 4 Ns ,
662the order in which you load the modules is important.
663You should kldload the other modules first, as this will ensure TCP packets
664undergo any necessary manipulations before
665.Nm
666.Qq sees
667and processes them.
668.It
669There is a known, harmless lock order reversal warning between the
670.Xr pfil 9
671mutex and tcbinfo TCP lock reported by
672.Xr witness 4
673when
674.Nm
675is enabled in a kernel compiled with
676.Xr witness 4
677support.
678.It
679There is no way to filter which TCP flows you wish to capture data for.
680Post processing is required to separate out data belonging to particular flows
681of interest.
682.It
683The module does not detect deletion of the log file path.
684New log messages will simply be lost if the log file being used by
685.Nm
686is deleted whilst the module is set to use the file.
687Switching to a new log file using the
688.Em net.inet.siftr.logfile
689variable will create the new file and allow log messages to begin being written
690to disk again.
691The new log file path must differ from the path to the deleted file.
692.It
693The hash table used within the code is sized to hold 65536 flows.  This is not a
694hard limit, because chaining is used to handle collisions within the hash table
695structure.
696However, we suspect (based on analogies with other hash table performance data)
697that the hash table look up performance (and therefore the module's packet
698processing performance) will degrade in an exponential manner as the number of
699unique flows handled in a module enable/disable cycle approaches and surpasses
70065536.
701.It
702There is no garbage collection performed on the flow hash table.
703The only way currently to flush it is to disable
704.Nm .
705.It
706The PPL variable applies to packets that make it into the processing thread,
707not total packets received in the hook function.
708Packets are skipped before the PPL variable is applied, which means there may be
709a slight discrepancy in the triggering of log messages.
710For example, if PPL was set to 10, and the 8th packet since the last log message
711is skipped, the 11th packet will actually trigger the log message to be
712generated.
713This is discussed in greater depth in CAIA technical report 070824A.
714.It
715At the time of writing, there was no simple way to hook into the TCP layer
716to intercept packets.
717.Nm Ap s
718use of IP layer hook points means all IP
719traffic will be processed by the
720.Nm
721.Xr pfil 9
722hook function, which introduces minor, but nonetheless unnecessary packet delay
723and processing overhead on the system for non-TCP packets as well.
724Hooking in at the IP layer is also not ideal from the data gathering point of
725view.
726Packets traversing up the stack will be intercepted and cause a log message
727generation BEFORE they have been processed by the TCP layer, which means we
728cannot observe the cause-and-affect relationship between inbound events and the
729corresponding TCP control block as precisely as could be.
730Ideally,
731.Nm
732should intercept packets after they have been processed by the TCP layer i.e.
733intercept packets coming up the stack after they have been processed by
734tcp_input(), and intercept packets coming down the stack after they have been
735processed by tcp_output().
736The current code still gives satisfactory granularity though, as inbound events
737tend to trigger outbound events, allowing the cause-and-effect to be observed
738indirectly by capturing the state on outbound events as well.
739.It
740The
741.Qq inflight bytes
742value logged by
743.Nm
744does not take into account bytes that have been
745.No SACK Ap ed
746by the receiving host.
747.It
748Packet hash generation does not currently work for IPv6 based TCP packets.
749.It
750Compressed notation is not used for IPv6 address representation.
751This consumes more bytes than is necessary in log output.
752.El
753