1.\" Copyright (c) 2020, Chelsio Inc 2.\" 3.\" Redistribution and use in source and binary forms, with or without 4.\" modification, are permitted provided that the following conditions are met: 5.\" 6.\" 1. Redistributions of source code must retain the above copyright notice, 7.\" this list of conditions and the following disclaimer. 8.\" 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" 3. Neither the name of the Chelsio Inc nor the names of its 14.\" contributors may be used to endorse or promote products derived from 15.\" this software without specific prior written permission. 16.\" 17.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 18.\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 19.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 20.\" ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE 21.\" LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 22.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 23.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 24.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 25.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 26.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 27.\" POSSIBILITY OF SUCH DAMAGE. 28.\" 29.\" * Other names and brands may be claimed as the property of others. 30.\" 31.\" $FreeBSD$ 32.\" 33.Dd August 12, 2020 34.Dt CRYPTO_REQUEST 9 35.Os 36.Sh NAME 37.Nm crypto_request 38.Nd symmetric cryptographic operations 39.Sh SYNOPSIS 40.In opencrypto/cryptodev.h 41.Ft int 42.Fn crypto_dispatch "struct cryptop *crp" 43.Ft void 44.Fn crypto_destroyreq "struct cryptop *crp" 45.Ft void 46.Fn crypto_freereq "struct cryptop *crp" 47.Ft "struct cryptop *" 48.Fn crypto_getreq "crypto_session_t cses" "int how" 49.Ft void 50.Fn crypto_initreq "crypto_session_t cses" "int how" 51.Ft void 52.Fn crypto_use_buf "struct cryptop *crp" "void *buf" "int len" 53.Ft void 54.Fn crypto_use_mbuf "struct cryptop *crp" "struct mbuf *m" 55.Ft void 56.Fn crypto_use_uio "struct cryptop *crp" "struct uio *uio" 57.Ft void 58.Fn crypto_use_vmpage "struct cryptop *crp" "vm_page_t *pages" "int len" "int offset" 59.Ft void 60.Fn crypto_use_output_buf "struct cryptop *crp" "void *buf" "int len" 61.Ft void 62.Fn crypto_use_output_mbuf "struct cryptop *crp" "struct mbuf *m" 63.Ft void 64.Fn crypto_use_output_uio "struct cryptop *crp" "struct uio *uio" 65.Ft void 66.Fn crypto_use_output_vmpage "struct cryptop *crp" "vm_page_t *pages" "int len" "int offset" 67.Sh DESCRIPTION 68Each symmetric cryptographic operation in the kernel is described by 69an instance of 70.Vt struct cryptop 71and is associated with an active session. 72.Pp 73Requests can either be allocated dynamically or use caller-supplied 74storage. 75Dynamically allocated requests should be allocated by 76.Fn crypto_getreq 77and freed by 78.Fn crypto_freereq 79once the request has completed. 80Requests using caller-supplied storage should be initialized by 81.Fn crypto_initreq 82at the start of each operation and destroyed by 83.Fn crypto_destroyreq 84once the request has completed. 85.Pp 86For both 87.Fn crypto_getreq 88and 89.Fn crypto_initreq , 90.Fa cses 91is a reference to an active session. 92For 93.Fn crypto_getreq , 94.Fa how 95is passed to 96.Xr malloc 9 97and should be set to either 98.Dv M_NOWAIT 99or 100.Dv M_WAITOK . 101.Pp 102Once a request has been initialized, 103the caller should set fields in the structure to describe 104request-specific parameters. 105Unused fields should be left as-is. 106.Pp 107.Fn crypto_dispatch 108passes a crypto request to the driver attached to the request's session. 109If there are errors in the request's fields, this function may return 110an error to the caller. 111If errors are encountered while servicing the request, they will instead 112be reported to the request's callback function 113.Pq Fa crp_callback 114via 115.Fa crp_etype . 116.Pp 117Note that a request's callback function may be invoked before 118.Fn crypto_dispatch 119returns. 120.Pp 121Once a request has signaled completion by invoking its callback function, 122it should be freed via 123.Fn crypto_destroyreq 124or 125.Fn crypto_freereq . 126.Pp 127Cryptographic operations include several fields to describe the request. 128.Ss Request Buffers 129Requests can either specify a single data buffer that is modified in place 130.Po 131.Fa crp_buf 132.Pc 133or separate input 134.Po 135.Fa crp_buf 136.Pc 137and output 138.Po 139.Fa crp_obuf 140.Pc 141buffers. 142Note that separate input and output buffers are not supported for compression 143mode requests. 144.Pp 145All requests must have a valid 146.Fa crp_buf 147initialized by one of the following functions: 148.Bl -tag -width "Fn crypto_use_vmpage" 149.It Fn crypto_use_buf 150Uses an array of 151.Fa len 152bytes pointed to by 153.Fa buf 154as the data buffer. 155.It Fn crypto_use_mbuf 156Uses the network memory buffer 157.Fa m 158as the data buffer. 159.It Fn crypto_use_uio 160Uses the scatter/gather list 161.Fa uio 162as the data buffer. 163.It Fn crypto_use_vmpage 164Uses the array of 165.Vt vm_page_t 166structures as the data buffer. 167.El 168.Pp 169One of the following functions should be used to initialize 170.Fa crp_obuf 171for requests that use separate input and output buffers: 172.Bl -tag -width "Fn crypto_use_output_vmpage" 173.It Fn crypto_use_output_buf 174Uses an array of 175.Fa len 176bytes pointed to by 177.Fa buf 178as the output buffer. 179.It Fn crypto_use_output_mbuf 180Uses the network memory buffer 181.Fa m 182as the output buffer. 183.It Fn crypto_use_output_uio 184Uses the scatter/gather list 185.Fa uio 186as the output buffer. 187.It Fn crypto_use_output_vmpage 188Uses the array of 189.Vt vm_page_t 190structures as the output buffer. 191.El 192.Ss Request Regions 193Each request describes one or more regions in the data buffers. 194Each region is described by an offset relative to the start of a 195data buffer and a length. 196The length of some regions is the same for all requests belonging to 197a session. 198Those lengths are set in the session parameters of the associated 199session. 200All requests must define a payload region. 201Other regions are only required for specific session modes. 202.Pp 203For requests with separate input and output data buffers, 204the AAD, IV, and payload regions are always defined as regions in the 205input buffer, 206and a separate payload output region is defined to hold the output of 207encryption or decryption in the output buffer. 208The digest region describes a region in the input data buffer for 209requests that verify an existing digest. 210For requests that compute a digest, 211the digest region describes a region in the output data buffer. 212Note that the only data written to the output buffer is the encryption 213or decryption result and any computed digest. 214AAD and IV regions are not copied from the input buffer into the output 215buffer but are only used as inputs. 216.Pp 217The following regions are defined: 218.Bl -column "Payload Output" "Input/Output" 219.It Sy Region Ta Sy Buffer Ta Sy Description 220.It AAD Ta Input Ta 221Embedded Additional Authenticated Data 222.It IV Ta Input Ta 223Embedded IV or nonce 224.It Payload Ta Input Ta 225Data to encrypt, decrypt, compress, or decompress 226.It Payload Output Ta Output Ta 227Encrypted or decrypted data 228.It Digest Ta Input/Output Ta 229Authentication digest, hash, or tag 230.El 231.Bl -column "Payload Output" ".Fa crp_payload_output_start" 232.It Sy Region Ta Sy Start Ta Sy Length 233.It AAD Ta Fa crp_aad_start Ta Fa crp_aad_length 234.It IV Ta Fa crp_iv_start Ta Fa csp_ivlen 235.It Payload Ta Fa crp_payload_start Ta Fa crp_payload_length 236.It Payload Output Ta Fa crp_payload_output_start Ta Fa crp_payload_length 237.It Digest Ta Fa crp_digest_start Ta Fa csp_auth_mlen 238.El 239.Pp 240Requests are permitted to operate on only a subset of the data buffer. 241For example, 242requests from IPsec operate on network packets that include headers not 243used as either additional authentication data (AAD) or payload data. 244.Ss Request Operations 245All requests must specify the type of operation to perform in 246.Fa crp_op . 247Available operations depend on the session's mode. 248.Pp 249Compression requests support the following operations: 250.Bl -tag -width CRYPTO_OP_DECOMPRESS 251.It Dv CRYPTO_OP_COMPRESS 252Compress the data in the payload region of the data buffer. 253.It Dv CRYPTO_OP_DECOMPRESS 254Decompress the data in the payload region of the data buffer. 255.El 256.Pp 257Cipher requests support the following operations: 258.Bl -tag -width CRYPTO_OP_DECRYPT 259.It Dv CRYPTO_OP_ENCRYPT 260Encrypt the data in the payload region of the data buffer. 261.It Dv CRYPTO_OP_DECRYPT 262Decrypt the data in the payload region of the data buffer. 263.El 264.Pp 265Digest requests support the following operations: 266.Bl -tag -width CRYPTO_OP_COMPUTE_DIGEST 267.It Dv CRYPTO_OP_COMPUTE_DIGEST 268Calculate a digest over the payload region of the data buffer 269and store the result in the digest region. 270.It Dv CRYPTO_OP_VERIFY_DIGEST 271Calculate a digest over the payload region of the data buffer. 272Compare the calculated digest to the existing digest from the digest region. 273If the digests match, 274complete the request successfully. 275If the digests do not match, 276fail the request with 277.Er EBADMSG . 278.El 279.Pp 280AEAD and Encrypt-then-Authenticate requests support the following 281operations: 282.Bl -tag -width CRYPTO_OP 283.It Dv CRYPTO_OP_ENCRYPT | Dv CRYPTO_OP_COMPUTE_DIGEST 284Encrypt the data in the payload region of the data buffer. 285Calculate a digest over the AAD and payload regions and store the 286result in the data buffer. 287.It Dv CRYPTO_OP_DECRYPT | Dv CRYPTO_OP_VERIFY_DIGEST 288Calculate a digest over the AAD and payload regions of the data buffer. 289Compare the calculated digest to the existing digest from the digest region. 290If the digests match, 291decrypt the payload region. 292If the digests do not match, 293fail the request with 294.Er EBADMSG . 295.El 296.Ss Request AAD 297AEAD and Encrypt-then-Authenticate requests may optionally include 298Additional Authenticated Data. 299AAD may either be supplied in the AAD region of the input buffer or 300as a single buffer pointed to by 301.Fa crp_aad . 302In either case, 303.Fa crp_aad_length 304always indicates the amount of AAD in bytes. 305.Ss Request IV and/or Nonce 306Some cryptographic operations require an IV or nonce as an input. 307An IV may be stored either in the IV region of the data buffer or in 308.Fa crp_iv . 309By default, 310the IV is assumed to be stored in the IV region. 311If the IV is stored in 312.Fa crp_iv , 313.Dv CRYPTO_F_IV_SEPARATE 314should be set in 315.Fa crp_flags 316and 317.Fa crp_iv_start 318should be left as zero. 319.Pp 320Requests that store part, but not all, of the IV in the data buffer should 321store the partial IV in the data buffer and pass the full IV separately in 322.Fa crp_iv . 323.Ss Request and Callback Scheduling 324The crypto framework provides multiple methods of scheduling the dispatch 325of requests to drivers along with the processing of driver callbacks. 326Requests use flags in 327.Fa crp_flags 328to select the desired scheduling methods. 329.Pp 330.Fn crypto_dispatch 331can pass the request to the session's driver via three different methods: 332.Bl -enum 333.It 334The request is queued to a taskqueue backed by a pool of worker threads. 335By default the pool is sized to provide one thread for each CPU. 336Worker threads dequeue requests and pass them to the driver 337asynchronously. 338.It 339The request is passed to the driver synchronously in the context of the 340thread invoking 341.Fn crypto_dispatch . 342.It 343The request is queued to a queue of pending requests. 344A single worker thread dequeues requests and passes them to the driver 345asynchronously. 346.El 347.Pp 348To select the first method (taskqueue backed by multiple threads), 349requests should set 350.Dv CRYPTO_F_ASYNC . 351To always use the third method (queue to single worker thread), 352requests should set 353.Dv CRYPTO_F_BATCH . 354If both flags are set, 355.Dv CRYPTO_F_ASYNC 356takes precedence. 357If neither flag is set, 358.Fn crypto_dispatch 359will first attempt the second method (invoke driver synchronously). 360If the driver is blocked, 361the request will be queued using the third method. 362One caveat is that the first method is only used for requests using software 363drivers which use host CPUs to process requests. 364Requests whose session is associated with a hardware driver will ignore 365.Dv CRYPTO_F_ASYNC 366and only use 367.Dv CRYPTO_F_BATCH 368to determine how requests should be scheduled. 369.Pp 370In addition to bypassing synchronous dispatch in 371.Fn crypto_dispatch , 372.Dv CRYPTO_F_BATCH 373requests additional changes aimed at optimizing batches of requests to 374the same driver. 375When the worker thread processes a request with 376.Dv CRYPTO_F_BATCH , 377it will search the pending request queue for any other requests for the same 378driver, 379including requests from different sessions. 380If any other requests are present, 381.Dv CRYPTO_HINT_MORE 382is passed to the driver's process method. 383Drivers may use this to batch completion interrupts. 384.Pp 385Callback function scheduling is simpler than request scheduling. 386Callbacks can either be invoked synchronously from 387.Fn crypto_done , 388or they can be queued to a pool of worker threads. 389This pool of worker threads is also sized to provide one worker thread 390for each CPU by default. 391Note that a callback function invoked synchronously from 392.Fn crypto_done 393must follow the same restrictions placed on threaded interrupt handlers. 394.Pp 395By default, 396callbacks are invoked asynchronously by a worker thread. 397If 398.Dv CRYPTO_F_CBIMM 399is set, 400the callback is always invoked synchronously from 401.Fn crypto_done . 402If 403.Dv CRYPTO_F_CBIFSYNC 404is set, 405the callback is invoked synchronously if the request was processed by a 406software driver or asynchronously if the request was processed by a 407hardware driver. 408.Pp 409If a request was scheduled to the taskqueue via 410.Dv CRYPTO_F_ASYNC , 411callbacks are always invoked asynchronously ignoring 412.Dv CRYPTO_F_CBIMM 413and 414.Dv CRYPTO_F_CBIFSYNC . 415In this case, 416.Dv CRYPTO_F_ASYNC_KEEPORDER 417may be set to ensure that callbacks for requests on a given session are 418invoked in the same order that requests were queued to the session via 419.Fn crypto_dispatch . 420This flag is used by IPsec to ensure that decrypted network packets are 421passed up the network stack in roughly the same order they were received. 422.Pp 423.Ss Other Request Fields 424In addition to the fields and flags enumerated above, 425.Vt struct cryptop 426includes the following: 427.Bl -tag -width crp_payload_length 428.It Fa crp_session 429A reference to the active session. 430This is set when the request is created by 431.Fn crypto_getreq 432and should not be modified. 433Drivers can use this to fetch driver-specific session state or 434session parameters. 435.It Fa crp_etype 436Error status. 437Either zero on success, or an error if a request fails. 438Set by drivers prior to completing a request via 439.Fn crypto_done . 440.It Fa crp_flags 441A bitmask of flags. 442The following flags are available in addition to flags discussed previously: 443.Bl -tag -width CRYPTO_F_DONE 444.It Dv CRYPTO_F_DONE 445Set by 446.Fa crypto_done 447before calling 448.Fa crp_callback . 449This flag is not very useful and will likely be removed in the future. 450It can only be safely checked from the callback routine at which point 451it is always set. 452.El 453.It Fa crp_cipher_key 454Pointer to a request-specific encryption key. 455If this value is not set, 456the request uses the session encryption key. 457.It Fa crp_auth_key 458Pointer to a request-specific authentication key. 459If this value is not set, 460the request uses the session authentication key. 461.It Fa crp_opaque 462An opaque pointer. 463This pointer permits users of the cryptographic framework to store 464information about a request to be used in the callback. 465.It Fa crp_callback 466Callback function. 467This must point to a callback function of type 468.Vt void (*)(struct cryptop *) . 469The callback function should inspect 470.Fa crp_etype 471to determine the status of the completed operation. 472It should also arrange for the request to be freed via 473.Fn crypto_freereq . 474.It Fa crp_olen 475Used with compression and decompression requests to describe the updated 476length of the payload region in the data buffer. 477.Pp 478If a compression request increases the size of the payload, 479then the data buffer is unmodified, the request completes successfully, 480and 481.Fa crp_olen 482is set to the size the compressed data would have used. 483Callers can compare this to the payload region length to determine if 484the compressed data was discarded. 485.El 486.Sh RETURN VALUES 487.Fn crypto_dispatch 488returns an error if the request contained invalid fields, 489or zero if the request was valid. 490.Fn crypto_getreq 491returns a pointer to a new request structure on success, 492or 493.Dv NULL 494on failure. 495.Dv NULL 496can only be returned if 497.Dv M_NOWAIT 498was passed in 499.Fa how . 500.Sh SEE ALSO 501.Xr ipsec 4 , 502.Xr crypto 7 , 503.Xr crypto 9 , 504.Xr crypto_session 9 , 505.Xr mbuf 9 506.Xr uio 9 507.Sh BUGS 508Not all drivers properly handle mixing session and per-request keys 509within a single session. 510Consumers should either use a single key for a session specified in 511the session parameters or always use per-request keys. 512