1.\" Copyright (c) 2020, Chelsio Inc 2.\" 3.\" Redistribution and use in source and binary forms, with or without 4.\" modification, are permitted provided that the following conditions are met: 5.\" 6.\" 1. Redistributions of source code must retain the above copyright notice, 7.\" this list of conditions and the following disclaimer. 8.\" 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" 3. Neither the name of the Chelsio Inc nor the names of its 14.\" contributors may be used to endorse or promote products derived from 15.\" this software without specific prior written permission. 16.\" 17.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 18.\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 19.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 20.\" ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE 21.\" LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 22.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 23.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 24.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 25.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 26.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 27.\" POSSIBILITY OF SUCH DAMAGE. 28.\" 29.\" * Other names and brands may be claimed as the property of others. 30.\" 31.\" $FreeBSD$ 32.\" 33.Dd April 20, 2020 34.Dt CRYPTO_REQUEST 9 35.Os 36.Sh NAME 37.Nm crypto_request 38.Nd symmetric cryptographic operations 39.Sh SYNOPSIS 40.In opencrypto/cryptodev.h 41.Ft int 42.Fn crypto_dispatch "struct cryptop *crp" 43.Ft void 44.Fn crypto_freereq "struct cryptop *crp" 45.Ft "struct cryptop *" 46.Fn crypto_getreq "crypto_session_t cses" "int how" 47.Sh DESCRIPTION 48Each symmetric cryptographic operation in the kernel is described by 49an instance of 50.Vt struct cryptop 51and is associated with an active session. 52.Pp 53New requests are allocated by 54.Fn crypto_getreq . 55.Fa cses 56is a reference to an active session. 57.Fa how 58is passed to 59.Xr malloc 9 60and should be set to either 61.Dv M_NOWAIT 62or 63.Dv M_WAITOK . 64The caller should then set fields in the returned structure to describe 65request-specific parameters. 66Unused fields should be left as-is. 67.Pp 68.Fn crypto_dispatch 69passes a crypto request to the driver attached to the request's session. 70If there are errors in the request's fields, this function may return 71an error to the caller. 72If errors are encountered while servicing the request, they will instead 73be reported to the request's callback function 74.Pq Fa crp_callback 75via 76.Fa crp_etype . 77.Pp 78Note that a request's callback function may be invoked before 79.Fn crypto_dispatch 80returns. 81.Pp 82Once a request has signaled completion by invoking its callback function, 83it should be feed via 84.Fn crypto_freereq . 85.Pp 86Cryptographic operations include several fields to describe the request. 87.Ss Buffer Types 88Requests are associated with a single data buffer that is modified in place. 89The type of the data buffer and the buffer itself are described by the 90following fields: 91.Bl -tag -width crp_buf_type 92.It Fa crp_buf_type 93The type of the data buffer. 94The following types are supported: 95.Bl -tag -width CRYPTO_BUF_CONTIG 96.It Dv CRYPTO_BUF_CONTIG 97An array of bytes mapped into the kernel's address space. 98.It Dv CRYPTO_BUF_UIO 99A scatter/gather list of kernel buffers as described in 100.Xr uio 9 . 101.It Dv CRYPTO_BUF_MBUF 102A network memory buffer as described in 103.Xr mbuf 9 . 104.El 105.It Fa crp_buf 106A pointer to the start of a 107.Dv CRYPTO_BUF_CONTIG 108data buffer. 109.It Fa crp_ilen 110The length of a 111.Dv CRYPTO_BUF_CONTIG 112data buffer 113.It Fa crp_mbuf 114A pointer to a 115.Vt struct mbuf 116for 117.Dv CRYPTO_BUF_MBUF . 118.It Fa crp_uio 119A pointer to a 120.Vt struct uio 121for 122.Dv CRYPTO_BUF_UIO . 123.It Fa crp_olen 124Used with compression and decompression requests to describe the updated 125length of the payload region in the data buffer. 126.Pp 127If a compression request increases the size of the payload, 128then the data buffer is unmodified, the request completes successfully, 129and 130.Fa crp_olen 131is set to the size the compressed data would have used. 132Callers can compare this to the payload region length to determine if 133the compressed data was discarded. 134.El 135.Ss Request Regions 136Each request describes one or more regions in the data buffer using. 137Each region is described by an offset relative to the start of the 138data buffer and a length. 139The length of some regions is the same for all requests belonging to 140a session. 141Those lengths are set in the session parameters of the associated 142session. 143All requests must define a payload region. 144Other regions are only required for specific session modes. 145The following regions are defined: 146.Bl -column "Payload" "crp_payload_start" "crp_payload_length" 147.It Sy Region Ta Sy Start Ta Sy Length Ta Sy Description 148.It AAD Ta Fa crp_aad_start Ta Fa crp_aad_length Ta 149Additional Authenticated Data 150.It IV Ta Fa crp_iv_start Ta Fa csp_ivlen Ta 151Embedded IV or nonce 152.It Payload Ta Fa crp_payload_start Ta Fa crp_payload_length Ta 153Data to encrypt, decrypt, compress, or decompress 154.It Digest Ta Fa crp_digest_start Ta Fa csp_auth_mlen Ta 155Authentication digest, hash, or tag 156.El 157.Pp 158Requests are permitted to operate on only a subset of the data buffer. 159For example, 160requests from IPsec operate on network packets that include headers not 161used as either additional authentication data (AAD) or payload data. 162.Ss Request Operations 163All requests must specify the type of operation to perform in 164.Fa crp_op . 165Available operations depend on the session's mode. 166.Pp 167Compression requests support the following operations: 168.Bl -tag -width CRYPTO_OP_DECOMPRESS 169.It Dv CRYPTO_OP_COMPRESS 170Compress the data in the payload region of the data buffer. 171.It Dv CRYPTO_OP_DECOMPRESS 172Decompress the data in the payload region of the data buffer. 173.El 174.Pp 175Cipher requests support the following operations: 176.Bl -tag -width CRYPTO_OP_DECRYPT 177.It Dv CRYPTO_OP_ENCRYPT 178Encrypt the data in the payload region of the data buffer. 179.It Dv CRYPTO_OP_DECRYPT 180Decrypt the data in the payload region of the data buffer. 181.El 182.Pp 183Digest requests support the following operations: 184.Bl -tag -width CRYPTO_OP_COMPUTE_DIGEST 185.It Dv CRYPTO_OP_COMPUTE_DIGEST 186Calculate a digest over the payload region of the data buffer 187and store the result in the digest region. 188.It Dv CRYPTO_OP_VERIFY_DIGEST 189Calculate a digest over the payload region of the data buffer. 190Compare the calculated digest to the existing digest from the digest region. 191If the digests match, 192complete the request successfully. 193If the digests do not match, 194fail the request with 195.Er EBADMSG . 196.El 197.Pp 198AEAD and Encrypt-then-Authenticate requests support the following 199operations: 200.Bl -tag -width CRYPTO_OP 201.It Dv CRYPTO_OP_ENCRYPT | Dv CRYPTO_OP_COMPUTE_DIGEST 202Encrypt the data in the payload region of the data buffer. 203Calculate a digest over the AAD and payload regions and store the 204result in the data buffer. 205.It Dv CRYPTO_OP_DECRYPT | Dv CRYPTO_OP_VERIFY_DIGEST 206Calculate a digest over the AAD and payload regions of the data buffer. 207Compare the calculated digest to the existing digest from the digest region. 208If the digests match, 209decrypt the payload region. 210If the digests do not match, 211fail the request with 212.Er EBADMSG . 213.El 214.Ss Request IV and/or Nonce 215Some cryptographic operations require an IV or nonce as an input. 216An IV may be stored either in the IV region of the data buffer or in 217.Fa crp_iv . 218By default, 219the IV is assumed to be stored in the IV region. 220If the IV is stored in 221.Fa crp_iv , 222.Dv CRYPTO_F_IV_SEPARATE 223should be set in 224.Fa crp_flags 225and 226.Fa crp_digest_start 227should be left as zero. 228.Pp 229Requests that store part, but not all, of the IV in the data buffer should 230store the partial IV in the data buffer and pass the full IV separately in 231.Fa crp_iv . 232.Ss Request and Callback Scheduling 233The crypto framework provides multiple methods of scheduling the dispatch 234of requests to drivers along with the processing of driver callbacks. 235Requests use flags in 236.Fa crp_flags 237to select the desired scheduling methods. 238.Pp 239.Fn crypto_dispatch 240can pass the request to the session's driver via three different methods: 241.Bl -enum 242.It 243The request is queued to a taskqueue backed by a pool of worker threads. 244By default the pool is sized to provide one thread for each CPU. 245Worker threads dequeue requests and pass them to the driver 246asynchronously. 247.It 248The request is passed to the driver synchronously in the context of the 249thread invoking 250.Fn crypto_dispatch . 251.It 252The request is queued to a queue of pending requests. 253A single worker thread dequeues requests and passes them to the driver 254asynchronously. 255.El 256.Pp 257To select the first method (taskqueue backed by multiple threads), 258requests should set 259.Dv CRYPTO_F_ASYNC . 260To always use the third method (queue to single worker thread), 261requests should set 262.Dv CRYPTO_F_BATCH . 263If both flags are set, 264.Dv CRYPTO_F_ASYNC 265takes precedence. 266If neither flag is set, 267.Fn crypto_dispatch 268will first attempt the second method (invoke driver synchronously). 269If the driver is blocked, 270the request will be queued using the third method. 271One caveat is that the first method is only used for requests using software 272drivers which use host CPUs to process requests. 273Requests whose session is associated with a hardware driver will ignore 274.Dv CRYPTO_F_ASYNC 275and only use 276.Dv CRYPTO_F_BATCH 277to determine how requests should be scheduled. 278.Pp 279In addition to bypassing synchronous dispatch in 280.Fn crypto_dispatch , 281.Dv CRYPTO_F_BATCH 282requests additional changes aimed at optimizing batches of requests to 283the same driver. 284When the worker thread processes a request with 285.Dv CRYPTO_F_BATCH , 286it will search the pending request queue for any other requests for the same 287driver, 288including requests from different sessions. 289If any other requests are present, 290.Dv CRYPTO_HINT_MORE 291is passed to the driver's process method. 292Drivers may use this to batch completion interrupts. 293.Pp 294Callback function scheduling is simpler than request scheduling. 295Callbacks can either be invoked synchronously from 296.Fn crypto_done , 297or they can be queued to a pool of worker threads. 298This pool of worker threads is also sized to provide one worker thread 299for each CPU by default. 300Note that a callback function invoked synchronously from 301.Fn crypto_done 302must follow the same restrictions placed on threaded interrupt handlers. 303.Pp 304By default, 305callbacks are invoked asynchronously by a worker thread. 306If 307.Dv CRYPTO_F_CBIMM 308is set, 309the callback is always invoked synchronously from 310.Fn crypto_done . 311If 312.Dv CRYPTO_F_CBIFSYNC 313is set, 314the callback is invoked synchronously if the request was processed by a 315software driver or asynchronously if the request was processed by a 316hardware driver. 317.Pp 318If a request was scheduled to the taskqueue via 319.Dv CRYPTO_F_ASYNC , 320callbacks are always invoked asynchronously ignoring 321.Dv CRYPTO_F_CBIMM 322and 323.Dv CRYPTO_F_CBIFSYNC . 324In this case, 325.Dv CRYPTO_F_ASYNC_KEEPORDER 326may be set to ensure that callbacks for requests on a given session are 327invoked in the same order that requests were queued to the session via 328.Fn crypto_dispatch . 329This flag is used by IPsec to ensure that decrypted network packets are 330passed up the network stack in roughly the same order they were received. 331.Pp 332.Ss Other Request Fields 333In addition to the fields and flags enumerated above, 334.Vt struct cryptop 335includes the following: 336.Bl -tag -width crp_payload_length 337.It Fa crp_session 338A reference to the active session. 339This is set when the request is created by 340.Fn crypto_getreq 341and should not be modified. 342Drivers can use this to fetch driver-specific session state or 343session parameters. 344.It Fa crp_etype 345Error status. 346Either zero on success, or an error if a request fails. 347Set by drivers prior to completing a request via 348.Fn crypto_done . 349.It Fa crp_flags 350A bitmask of flags. 351The following flags are available in addition to flags discussed previously: 352.Bl -tag -width CRYPTO_F_DONE 353.It Dv CRYPTO_F_DONE 354Set by 355.Fa crypto_done 356before calling 357.Fa crp_callback . 358This flag is not very useful and will likely be removed in the future. 359It can only be safely checked from the callback routine at which point 360it is always set. 361.El 362.It Fa crp_cipher_key 363Pointer to a request-specific encryption key. 364If this value is not set, 365the request uses the session encryption key. 366.It Fa crp_auth_key 367Pointer to a request-specific authentication key. 368If this value is not set, 369the request uses the session authentication key. 370.It Fa crp_opaque 371An opaque pointer. 372This pointer permits users of the cryptographic framework to store 373information about a request to be used in the callback. 374.It Fa crp_callback 375Callback function. 376This must point to a callback function of type 377.Vt void (*)(struct cryptop *) . 378The callback function should inspect 379.Fa crp_etype 380to determine the status of the completed operation. 381It should also arrange for the request to be freed via 382.Fn crypto_freereq . 383.El 384.Sh RETURN VALUES 385.Fn crypto_dispatch 386returns an error if the request contained invalid fields, 387or zero if the request was valid. 388.Fn crypto_getreq 389returns a pointer to a new request structure on success, 390or 391.Dv NULL 392on failure. 393.Dv NULL 394can only be returned if 395.Dv M_NOWAIT 396was passed in 397.Fa how . 398.Sh SEE ALSO 399.Xr ipsec 4 , 400.Xr crypto 7 , 401.Xr crypto 9 , 402.Xr crypto_session 9 , 403.Xr mbuf 9 404.Xr uio 9 405.Sh BUGS 406Not all drivers properly handle mixing session and per-request keys 407within a single session. 408Consumers should either use a single key for a session specified in 409the session parameters or always use per-request keys. 410