xref: /linux/Documentation/networking/tls-handshake.rst (revision f9bff0e31881d03badf191d3b0005839391f5f2b)
1.. SPDX-License-Identifier: GPL-2.0
2
3=======================
4In-Kernel TLS Handshake
5=======================
6
7Overview
8========
9
10Transport Layer Security (TLS) is a Upper Layer Protocol (ULP) that runs
11over TCP. TLS provides end-to-end data integrity and confidentiality in
12addition to peer authentication.
13
14The kernel's kTLS implementation handles the TLS record subprotocol, but
15does not handle the TLS handshake subprotocol which is used to establish
16a TLS session. Kernel consumers can use the API described here to
17request TLS session establishment.
18
19There are several possible ways to provide a handshake service in the
20kernel. The API described here is designed to hide the details of those
21implementations so that in-kernel TLS consumers do not need to be
22aware of how the handshake gets done.
23
24
25User handshake agent
26====================
27
28As of this writing, there is no TLS handshake implementation in the
29Linux kernel. To provide a handshake service, a handshake agent
30(typically in user space) is started in each network namespace where a
31kernel consumer might require a TLS handshake. Handshake agents listen
32for events sent from the kernel that indicate a handshake request is
33waiting.
34
35An open socket is passed to a handshake agent via a netlink operation,
36which creates a socket descriptor in the agent's file descriptor table.
37If the handshake completes successfully, the handshake agent promotes
38the socket to use the TLS ULP and sets the session information using the
39SOL_TLS socket options. The handshake agent returns the socket to the
40kernel via a second netlink operation.
41
42
43Kernel Handshake API
44====================
45
46A kernel TLS consumer initiates a client-side TLS handshake on an open
47socket by invoking one of the tls_client_hello() functions. First, it
48fills in a structure that contains the parameters of the request:
49
50.. code-block:: c
51
52  struct tls_handshake_args {
53        struct socket   *ta_sock;
54        tls_done_func_t ta_done;
55        void            *ta_data;
56        const char      *ta_peername;
57        unsigned int    ta_timeout_ms;
58        key_serial_t    ta_keyring;
59        key_serial_t    ta_my_cert;
60        key_serial_t    ta_my_privkey;
61        unsigned int    ta_num_peerids;
62        key_serial_t    ta_my_peerids[5];
63  };
64
65The @ta_sock field references an open and connected socket. The consumer
66must hold a reference on the socket to prevent it from being destroyed
67while the handshake is in progress. The consumer must also have
68instantiated a struct file in sock->file.
69
70
71@ta_done contains a callback function that is invoked when the handshake
72has completed. Further explanation of this function is in the "Handshake
73Completion" sesction below.
74
75The consumer can provide a NUL-terminated hostname in the @ta_peername
76field that is sent as part of ClientHello. If no peername is provided,
77the DNS hostname associated with the server's IP address is used instead.
78
79The consumer can fill in the @ta_timeout_ms field to force the servicing
80handshake agent to exit after a number of milliseconds. This enables the
81socket to be fully closed once both the kernel and the handshake agent
82have closed their endpoints.
83
84Authentication material such as x.509 certificates, private certificate
85keys, and pre-shared keys are provided to the handshake agent in keys
86that are instantiated by the consumer before making the handshake
87request. The consumer can provide a private keyring that is linked into
88the handshake agent's process keyring in the @ta_keyring field to prevent
89access of those keys by other subsystems.
90
91To request an x.509-authenticated TLS session, the consumer fills in
92the @ta_my_cert and @ta_my_privkey fields with the serial numbers of
93keys containing an x.509 certificate and the private key for that
94certificate. Then, it invokes this function:
95
96.. code-block:: c
97
98  ret = tls_client_hello_x509(args, gfp_flags);
99
100The function returns zero when the handshake request is under way. A
101zero return guarantees the callback function @ta_done will be invoked
102for this socket. The function returns a negative errno if the handshake
103could not be started. A negative errno guarantees the callback function
104@ta_done will not be invoked on this socket.
105
106
107To initiate a client-side TLS handshake with a pre-shared key, use:
108
109.. code-block:: c
110
111  ret = tls_client_hello_psk(args, gfp_flags);
112
113However, in this case, the consumer fills in the @ta_my_peerids array
114with serial numbers of keys containing the peer identities it wishes
115to offer, and the @ta_num_peerids field with the number of array
116entries it has filled in. The other fields are filled in as above.
117
118
119To initiate an anonymous client-side TLS handshake use:
120
121.. code-block:: c
122
123  ret = tls_client_hello_anon(args, gfp_flags);
124
125The handshake agent presents no peer identity information to the remote
126during this type of handshake. Only server authentication (ie the client
127verifies the server's identity) is performed during the handshake. Thus
128the established session uses encryption only.
129
130
131Consumers that are in-kernel servers use:
132
133.. code-block:: c
134
135  ret = tls_server_hello_x509(args, gfp_flags);
136
137or
138
139.. code-block:: c
140
141  ret = tls_server_hello_psk(args, gfp_flags);
142
143The argument structure is filled in as above.
144
145
146If the consumer needs to cancel the handshake request, say, due to a ^C
147or other exigent event, the consumer can invoke:
148
149.. code-block:: c
150
151  bool tls_handshake_cancel(sock);
152
153This function returns true if the handshake request associated with
154@sock has been canceled. The consumer's handshake completion callback
155will not be invoked. If this function returns false, then the consumer's
156completion callback has already been invoked.
157
158
159Handshake Completion
160====================
161
162When the handshake agent has completed processing, it notifies the
163kernel that the socket may be used by the consumer again. At this point,
164the consumer's handshake completion callback, provided in the @ta_done
165field in the tls_handshake_args structure, is invoked.
166
167The synopsis of this function is:
168
169.. code-block:: c
170
171  typedef void	(*tls_done_func_t)(void *data, int status,
172                                   key_serial_t peerid);
173
174The consumer provides a cookie in the @ta_data field of the
175tls_handshake_args structure that is returned in the @data parameter of
176this callback. The consumer uses the cookie to match the callback to the
177thread waiting for the handshake to complete.
178
179The success status of the handshake is returned via the @status
180parameter:
181
182+------------+----------------------------------------------+
183|  status    |  meaning                                     |
184+============+==============================================+
185|  0         |  TLS session established successfully        |
186+------------+----------------------------------------------+
187|  -EACCESS  |  Remote peer rejected the handshake or       |
188|            |  authentication failed                       |
189+------------+----------------------------------------------+
190|  -ENOMEM   |  Temporary resource allocation failure       |
191+------------+----------------------------------------------+
192|  -EINVAL   |  Consumer provided an invalid argument       |
193+------------+----------------------------------------------+
194|  -ENOKEY   |  Missing authentication material             |
195+------------+----------------------------------------------+
196|  -EIO      |  An unexpected fault occurred                |
197+------------+----------------------------------------------+
198
199The @peerid parameter contains the serial number of a key containing the
200remote peer's identity or the value TLS_NO_PEERID if the session is not
201authenticated.
202
203A best practice is to close and destroy the socket immediately if the
204handshake failed.
205
206
207Other considerations
208--------------------
209
210While a handshake is under way, the kernel consumer must alter the
211socket's sk_data_ready callback function to ignore all incoming data.
212Once the handshake completion callback function has been invoked, normal
213receive operation can be resumed.
214
215Once a TLS session is established, the consumer must provide a buffer
216for and then examine the control message (CMSG) that is part of every
217subsequent sock_recvmsg(). Each control message indicates whether the
218received message data is TLS record data or session metadata.
219
220See tls.rst for details on how a kTLS consumer recognizes incoming
221(decrypted) application data, alerts, and handshake packets once the
222socket has been promoted to use the TLS ULP.
223