1.\" Copyright (c) 2020, Chelsio Inc 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions are met: 6.\" 7.\" 1. Redistributions of source code must retain the above copyright notice, 8.\" this list of conditions and the following disclaimer. 9.\" 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" 3. Neither the name of the Chelsio Inc nor the names of its 15.\" contributors may be used to endorse or promote products derived from 16.\" this software without specific prior written permission. 17.\" 18.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 19.\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 20.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 21.\" ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE 22.\" LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 23.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 24.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 25.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 26.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 27.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 28.\" POSSIBILITY OF SUCH DAMAGE. 29.\" 30.\" * Other names and brands may be claimed as the property of others. 31.\" 32.Dd December 14, 2021 33.Dt KTLS 4 34.Os 35.Sh NAME 36.Nm ktls 37.Nd kernel Transport Layer Security 38.Sh SYNOPSIS 39.Cd options KERN_TLS 40.Sh DESCRIPTION 41The 42.Nm 43facility allows the kernel to perform Transport Layer Security (TLS) 44framing on TCP sockets. 45With 46.Nm , 47the initial handshake for a socket using TLS is performed in userland. 48Once the session keys are negotiated, 49they are provided to the kernel via the 50.Dv TCP_TXTLS_ENABLE 51and 52.Dv TCP_RXTLS_ENABLE 53socket options. 54Both socket options accept a 55.Vt struct tls_enable 56structure as their argument. 57The members of this structure describe the cipher suite used for the 58TLS session and provide the session keys used for the respective 59direction. 60.Pp 61.Nm 62only permits the session keys to be set once in each direction. 63As a result, 64applications must disable rekeying when using 65.Nm . 66.Ss Modes 67.Nm 68can operate in different modes. 69A given socket may use different modes for transmit and receive, 70or a socket may only offload a single direction. 71The available modes are: 72.Bl -tag -width "Dv TCP_TLS_MODE_IFNET" 73.It Dv TCP_TLS_MODE_NONE 74.Nm 75is not enabled. 76.It Dv TCP_TLS_MODE_SW 77TLS records are encrypted or decrypted in the kernel in the socket 78layer via 79.Xr crypto 9 . 80Typically the encryption or decryption is performed in software, 81but it may also be performed by co-processors. 82.It Dv TCP_TLS_MODE_IFNET 83TLS records are encrypted or decrypted by the network interface card (NIC). 84In this mode, the network stack does not work with encrypted data. 85Instead, the NIC encrypts TLS records as they are being transmitted, 86or decrypts received TLS records before providing them to the host. 87.Pp 88Network interfaces which support this feature will advertise the 89.Dv TXTLS4 90(for IPv4) 91and/or 92.Dv TXTLS6 93(for IPv6) 94capabilities as reported by 95.Xr ifconfig 8 . 96These capabilities can also be controlled by 97.Xr ifconfig 8 . 98.Pp 99If a network interface supports rate limiting 100(also known as packet pacing) for TLS offload, 101the interface will advertise the 102.Dv TXTLS_RTLMT 103capability. 104.It Dv TCP_TLS_MODE_TOE 105TLS records are encrypted by the NIC using a TCP offload engine (TOE). 106This is similar to 107.Dv TCP_TLS_MODE_IFNET 108in that the network stack does not work with encrypted data. 109However, this mode works in tandem with a TOE to handle interactions 110between TCP and TLS. 111.El 112.Ss Transmit 113Once TLS transmit is enabled by a successful set of the 114.Dv TCP_TXTLS_ENABLE 115socket option, 116all data written on the socket is stored in TLS records and encrypted. 117Most data is transmitted in application layer TLS records, 118and the kernel chooses how to partition data among TLS records. 119Individual TLS records with a fixed length and record type can be sent 120by 121.Xr sendmsg 2 122with the TLS record type set in a 123.Dv TLS_SET_RECORD_TYPE 124control message. 125The payload of this control message is a single byte holding the desired 126TLS record type. 127This can be used to send TLS records with a type other than 128application data (for example, handshake messages) or to send 129application data records with specific contents (for example, empty 130fragments). 131.Pp 132The current TLS transmit mode of a socket can be queried via the 133.Dv TCP_TXTLS_MODE 134socket option. 135A socket using TLS transmit offload can also set the 136.Dv TCP_TXTLS_MODE 137socket option to toggle between 138.Dv TCP_TLS_MODE_SW 139and 140.Dv TCP_TLS_MODE_IFNET . 141.Ss Receive 142Once TLS receive is enabled by a successful set of the 143.Dv TCP_RXTLS_ENABLE 144socket option, 145all data read from the socket is returned as decrypted TLS records. 146Each received TLS record must be read from the socket using 147.Xr recvmsg 2 . 148Each received TLS record will contain a 149.Dv TLS_GET_RECORD 150control message along with the decrypted payload. 151The control message contains a 152.Vt struct tls_get_record 153which includes fields from the TLS record header. 154If an invalid or corrupted TLS record is received, 155.Xr recvmsg 2 156will fail with one of the following errors: 157.Bl -tag -width Er 158.It Bq Er EINVAL 159The version fields in a TLS record's header did not match the version required 160by the 161.Vt struct tls_enable 162structure used to enable in-kernel TLS. 163.It Bq Er EMSGSIZE 164A TLS record's length was either too small or too large. 165.It Bq Er EMSGSIZE 166The connection was closed after sending a truncated TLS record. 167.It Bq Er EBADMSG 168The TLS record failed to match the included authentication tag. 169.El 170.Pp 171The current TLS receive mode of a socket can be queried via the 172.Dv TCP_RXTLS_MODE 173socket option. 174At present, 175the mode cannot be changed. 176.Ss Sysctl Nodes 177.Nm 178uses several sysctl nodes under the 179.Va kern.ipc.tls 180node. 181A few of them are described below: 182.Bl -tag -width ".Va kern.ipc.tls.cbc_enable" 183.It Va kern.ipc.tls.enable 184Determines if new kernel TLS sessions can be created. 185.It Va kern.ipc.tls.cbc_enable 186Determines if new kernel TLS sessions with a cipher suite using AES-CBC 187can be created. 188.It Va kern.ipc.tls.sw 189A tree of nodes containing statistics for TLS sessions using 190.Dv TCP_TLS_MODE_SW . 191.It Va kern.ipc.tls.ifnet 192A tree of nodes containing statistics for TLS sessions using 193.Dv TCP_TLS_MODE_IFNET . 194.It Va kern.ipc.tls.toe 195A tree of nodes containing statistics for TLS sessions using 196.Dv TCP_TLS_MODE_TOE . 197.It Va kern.ipc.tls.stats 198A tree of nodes containing various kernel TLS statistics. 199.El 200.Pp 201The 202.Va kern.ipc.mb_use_ext_pgs 203sysctl controls whether the kernel may use unmapped mbufs. 204They are required for TLS transmit. 205.Ss Supported Hardware 206The 207.Xr cxgbe 4 208and 209.Xr mlx5en 4 210drivers include support for the 211.Dv TCP_TLS_MODE_IFNET 212mode. 213.Pp 214The 215.Xr cxgbe 4 216driver includes support for the 217.Dv TCP_TLS_MODE_TOE 218mode. 219.Ss Supported Libraries 220OpenSSL 3.0 and later include support for 221.Nm . 222The 223.Fa security/openssl-devel 224port may also be built with support for 225.Nm 226by enabling the 227.Dv KTLS 228option. 229OpenSSL in the base system includes KTLS support when built with 230.Dv WITH_OPENSSL_KTLS . 231.Pp 232Applications using a supported library should generally work with 233.Nm 234without any changes provided they use standard interfaces such as 235.Xr SSL_read 3 236and 237.Xr SSL_write 3 . 238Additional performance may be gained by the use of 239.Xr SSL_sendfile 3 . 240.Sh IMPLEMENTATION NOTES 241.Nm 242assumes the presence of a direct map of physical memory when performing 243software encryption and decryption. 244As a result, it is only supported on architectures with a direct map. 245.Sh SEE ALSO 246.Xr cxgbe 4 , 247.Xr mlx5en 4 , 248.Xr tcp 4 , 249.Xr src.conf 5 , 250.Xr ifconfig 8 , 251.Xr sysctl 8 , 252.Xr crypto 9 253.Sh HISTORY 254Kernel TLS first appeared in 255.Fx 13.0 . 256