1.\" 2.\" Copyright (c) 2020, Gordon Bergling 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 14.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR 17.\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23.\" SUCH DAMAGE. 24.\" 25.\" $FreeBSD$ 26.\" 27.Dd September 24, 2020 28.Dt TCP_BBR 4 29.Os 30.Sh NAME 31.Nm tcp_bbr 32.Nd TCP Bottleneck Bandwidth and Round-Trip Time Algorithm 33.Sh SYNOPSIS 34To use this TCP stack you have to place the following line in your 35kernel configuration file: 36.Bd -ragged -offset indent 37.Cd "options TCPHPTS" 38.Ed 39.Pp 40To load the driver as a module at boot time, place the following line in 41.Xr loader.conf 5 : 42.Bd -literal -offset indent 43tcp_bbr_load="YES" 44.Ed 45.Pp 46To enable the TCP stack you must place the following line in the 47.Xr sysctl.conf 5 : 48.Bd -literal -offset indent 49net.inet.tcp.functions_default=bbr 50.Ed 51.Sh DESCRIPTION 52Bottleneck bandwidth and round-trip time (BBR) is a congestion control algorithm which 53seeks high throughput with a small queue by probing BW and RTT. 54It is a round-up redesign of congestion control, which is not loss-based, delay-based, 55ECN-based or AIMD-based. 56.Pp 57The core design of BBR is about creating a model graph of the network path by 58estimating the maximum BW and minimum RTT on each ACK. 59.Sh MIB Variables 60The algorithm exposes the following scopes in the 61.Va net.inet.tcp.bbr 62branch of the 63.Xr sysctl 3 64MIB: 65.Bl -tag -width ".Va exp_backoff_scale" 66.It Va cwnd 67Cwnd controls, for example "target cwnd rtt measurment" and "BBR initial window". 68.It Va measure 69Measurement controls. 70.It Va pacing 71Connection pacing controls. 72.It Va policer 73Policer controls, for example "false detection threshold" and "loss threshold". 74.It Va probertt 75Probe RTT controls. 76.It Va startup 77Startup controls. 78.It Va states 79State controls. 80.It Va timeout 81Time out controls. 82.El 83.Pp 84Besides the variables within the above scopes the following 85variables are also exposed in the 86.Va net.inet.tcp.bbr 87branch: 88.Bl -tag -width ".Va exp_backoff_scale" 89.It Va clrlost 90Clear lost counters. 91.It Va software_pacing 92Total number of software paced flows. 93.It Va hdwr_pacing 94Total number of hardware paced flows. 95.It Va enob_no_hdwr_pacing 96Total number of enobufs for non-hardware paced flows. 97.It Va enob_hdwr_pacing 98Total number of enobufs for hardware paced flows. 99.It Va rtt_tlp_thresh 100What divisor for TLP rtt/retran will be added (1=rtt, 2=1/2 rtt etc). 101.It Va reorder_fade 102Does reorder detection fade, if so how many ms (0 means never). 103.It Va reorder_thresh 104What factor for rack will be added when seeing reordering (shift right). 105.It Va bb_verbose 106Should BBR black box logging be verbose. 107.It Va sblklimit 108When do we start ignoring small sack blocks. 109.It Va resend_use_tso 110Can resends use TSO? 111.It Va data_after_close 112Do we hold off sending a RST until all pending data is ack'd. 113.It Va kill_paceout 114When we hit this many errors in a row, kill the session? 115.It Va error_paceout 116When we hit an error what is the min to pace out in usec's? 117.It Va cheat_rxt 118Do we burst 1ms between sends on retransmissions (like rack)? 119.It Va minrto 120Minimum RTO in ms. 121.El 122.Sh SEE ALSO 123.Xr cc_chd 4 , 124.Xr cc_cubic 4 , 125.Xr cc_hd 4 , 126.Xr cc_htcp 4 , 127.Xr cc_newreno 4 , 128.Xr cc_vegas 4 , 129.Xr h_ertt 4 , 130.Xr mod_cc 4 , 131.Xr tcp 4 , 132.Xr mod_cc 9 133.Rs 134.%A "Neal Cardwell" 135.%A "Yuchung Cheng" 136.%A "Stephen Gunn" 137.%A "Soheil Hassas Yeganeh" 138.%A "Van Jacobson" 139.%T "BBR: Congestion-Based Congestion Control" 140.%J "ACM Queue, Vol. 14" 141.%D "September / October 2016" 142.Re 143.Rs 144.%A "Dominik Scholz" 145.%A "Benedikt Jaeger" 146.%A "Lukas Schwaighofer" 147.%A "Daniel Raumer" 148.%A "Fabien Geyer" 149.%A "Georg Carle" 150.%T "Towards a Deeper Understanding of TCP BBR Congestion Control" 151.%J "IFIP Networking 2018" 152.%D "May 2018" 153.%U "http://www.net.in.tum.de/fileadmin/bibtex/publications/papers/IFIP-Networking-2018-TCP-BBR.pdf" 154.Re 155.Sh HISTORY 156The 157.Nm 158congestion control module first appeared in 159.Fx 13.0 . 160.Sh AUTHORS 161.An -nosplit 162The 163.Nm 164congestion control module was written by 165.An Randall Stewart Aq Mt rrs@FreeBSD.org 166and sponsored by Netflix, Inc. 167This manual page was written by 168.An Gordon Bergling Aq Mt gbe@FreeBSD.org . 169