1.\" Copyright (c) 2020, Chelsio Inc 2.\" 3.\" Redistribution and use in source and binary forms, with or without 4.\" modification, are permitted provided that the following conditions are met: 5.\" 6.\" 1. Redistributions of source code must retain the above copyright notice, 7.\" this list of conditions and the following disclaimer. 8.\" 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" 3. Neither the name of the Chelsio Inc nor the names of its 14.\" contributors may be used to endorse or promote products derived from 15.\" this software without specific prior written permission. 16.\" 17.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 18.\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 19.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 20.\" ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE 21.\" LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 22.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 23.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 24.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 25.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 26.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 27.\" POSSIBILITY OF SUCH DAMAGE. 28.\" 29.\" * Other names and brands may be claimed as the property of others. 30.\" 31.\" $FreeBSD$ 32.\" 33.Dd February 8, 2021 34.Dt CRYPTO_REQUEST 9 35.Os 36.Sh NAME 37.Nm crypto_request 38.Nd symmetric cryptographic operations 39.Sh SYNOPSIS 40.In opencrypto/cryptodev.h 41.Ft int 42.Fn crypto_dispatch "struct cryptop *crp" 43.Ft int 44.Fn crypto_dispatch_async "struct cryptop *crp" "int flags" 45.Ft void 46.Fn crypto_dispatch_batch "struct cryptopq *crpq" "int flags" 47.Ft void 48.Fn crypto_destroyreq "struct cryptop *crp" 49.Ft void 50.Fn crypto_freereq "struct cryptop *crp" 51.Ft "struct cryptop *" 52.Fn crypto_getreq "crypto_session_t cses" "int how" 53.Ft void 54.Fn crypto_initreq "crypto_session_t cses" "int how" 55.Ft void 56.Fn crypto_use_buf "struct cryptop *crp" "void *buf" "int len" 57.Ft void 58.Fn crypto_use_mbuf "struct cryptop *crp" "struct mbuf *m" 59.Ft void 60.Fn crypto_use_uio "struct cryptop *crp" "struct uio *uio" 61.Ft void 62.Fn crypto_use_vmpage "struct cryptop *crp" "vm_page_t *pages" "int len" "int offset" 63.Ft void 64.Fn crypto_use_output_buf "struct cryptop *crp" "void *buf" "int len" 65.Ft void 66.Fn crypto_use_output_mbuf "struct cryptop *crp" "struct mbuf *m" 67.Ft void 68.Fn crypto_use_output_uio "struct cryptop *crp" "struct uio *uio" 69.Ft void 70.Fn crypto_use_output_vmpage "struct cryptop *crp" "vm_page_t *pages" "int len" "int offset" 71.Sh DESCRIPTION 72Each symmetric cryptographic operation in the kernel is described by 73an instance of 74.Vt struct cryptop 75and is associated with an active session. 76.Pp 77Requests can either be allocated dynamically or use caller-supplied 78storage. 79Dynamically allocated requests should be allocated by 80.Fn crypto_getreq 81and freed by 82.Fn crypto_freereq 83once the request has completed. 84Requests using caller-supplied storage should be initialized by 85.Fn crypto_initreq 86at the start of each operation and destroyed by 87.Fn crypto_destroyreq 88once the request has completed. 89.Pp 90For both 91.Fn crypto_getreq 92and 93.Fn crypto_initreq , 94.Fa cses 95is a reference to an active session. 96For 97.Fn crypto_getreq , 98.Fa how 99is passed to 100.Xr malloc 9 101and should be set to either 102.Dv M_NOWAIT 103or 104.Dv M_WAITOK . 105.Pp 106Once a request has been initialized, 107the caller should set fields in the structure to describe 108request-specific parameters. 109Unused fields should be left as-is. 110.Pp 111The 112.Fn crypto_dispatch , 113.Fn crypto_dispatch_async , 114and 115.Fn crypto_dispatch_batch 116functions pass one or more crypto requests to the driver attached to the 117request's session. 118If there are errors in the request's fields, these functions may return an 119error to the caller. 120If errors are encountered while servicing the request, they will instead 121be reported to the request's callback function 122.Pq Fa crp_callback 123via 124.Fa crp_etype . 125.Pp 126Note that a request's callback function may be invoked before 127.Fn crypto_dispatch 128returns. 129.Pp 130Once a request has signaled completion by invoking its callback function, 131it should be freed via 132.Fn crypto_destroyreq 133or 134.Fn crypto_freereq . 135.Pp 136Cryptographic operations include several fields to describe the request. 137.Ss Request Buffers 138Requests can either specify a single data buffer that is modified in place 139.Po 140.Fa crp_buf 141.Pc 142or separate input 143.Po 144.Fa crp_buf 145.Pc 146and output 147.Po 148.Fa crp_obuf 149.Pc 150buffers. 151Note that separate input and output buffers are not supported for compression 152mode requests. 153.Pp 154All requests must have a valid 155.Fa crp_buf 156initialized by one of the following functions: 157.Bl -tag -width "Fn crypto_use_vmpage" 158.It Fn crypto_use_buf 159Uses an array of 160.Fa len 161bytes pointed to by 162.Fa buf 163as the data buffer. 164.It Fn crypto_use_mbuf 165Uses the network memory buffer 166.Fa m 167as the data buffer. 168.It Fn crypto_use_uio 169Uses the scatter/gather list 170.Fa uio 171as the data buffer. 172.It Fn crypto_use_vmpage 173Uses the array of 174.Vt vm_page_t 175structures as the data buffer. 176.El 177.Pp 178One of the following functions should be used to initialize 179.Fa crp_obuf 180for requests that use separate input and output buffers: 181.Bl -tag -width "Fn crypto_use_output_vmpage" 182.It Fn crypto_use_output_buf 183Uses an array of 184.Fa len 185bytes pointed to by 186.Fa buf 187as the output buffer. 188.It Fn crypto_use_output_mbuf 189Uses the network memory buffer 190.Fa m 191as the output buffer. 192.It Fn crypto_use_output_uio 193Uses the scatter/gather list 194.Fa uio 195as the output buffer. 196.It Fn crypto_use_output_vmpage 197Uses the array of 198.Vt vm_page_t 199structures as the output buffer. 200.El 201.Ss Request Regions 202Each request describes one or more regions in the data buffers. 203Each region is described by an offset relative to the start of a 204data buffer and a length. 205The length of some regions is the same for all requests belonging to 206a session. 207Those lengths are set in the session parameters of the associated 208session. 209All requests must define a payload region. 210Other regions are only required for specific session modes. 211.Pp 212For requests with separate input and output data buffers, 213the AAD, IV, and payload regions are always defined as regions in the 214input buffer, 215and a separate payload output region is defined to hold the output of 216encryption or decryption in the output buffer. 217The digest region describes a region in the input data buffer for 218requests that verify an existing digest. 219For requests that compute a digest, 220the digest region describes a region in the output data buffer. 221Note that the only data written to the output buffer is the encryption 222or decryption result and any computed digest. 223AAD and IV regions are not copied from the input buffer into the output 224buffer but are only used as inputs. 225.Pp 226The following regions are defined: 227.Bl -column "Payload Output" "Input/Output" 228.It Sy Region Ta Sy Buffer Ta Sy Description 229.It AAD Ta Input Ta 230Embedded Additional Authenticated Data 231.It IV Ta Input Ta 232Embedded IV or nonce 233.It Payload Ta Input Ta 234Data to encrypt, decrypt, compress, or decompress 235.It Payload Output Ta Output Ta 236Encrypted or decrypted data 237.It Digest Ta Input/Output Ta 238Authentication digest, hash, or tag 239.El 240.Bl -column "Payload Output" ".Fa crp_payload_output_start" 241.It Sy Region Ta Sy Start Ta Sy Length 242.It AAD Ta Fa crp_aad_start Ta Fa crp_aad_length 243.It IV Ta Fa crp_iv_start Ta Fa csp_ivlen 244.It Payload Ta Fa crp_payload_start Ta Fa crp_payload_length 245.It Payload Output Ta Fa crp_payload_output_start Ta Fa crp_payload_length 246.It Digest Ta Fa crp_digest_start Ta Fa csp_auth_mlen 247.El 248.Pp 249Requests are permitted to operate on only a subset of the data buffer. 250For example, 251requests from IPsec operate on network packets that include headers not 252used as either additional authentication data (AAD) or payload data. 253.Ss Request Operations 254All requests must specify the type of operation to perform in 255.Fa crp_op . 256Available operations depend on the session's mode. 257.Pp 258Compression requests support the following operations: 259.Bl -tag -width CRYPTO_OP_DECOMPRESS 260.It Dv CRYPTO_OP_COMPRESS 261Compress the data in the payload region of the data buffer. 262.It Dv CRYPTO_OP_DECOMPRESS 263Decompress the data in the payload region of the data buffer. 264.El 265.Pp 266Cipher requests support the following operations: 267.Bl -tag -width CRYPTO_OP_DECRYPT 268.It Dv CRYPTO_OP_ENCRYPT 269Encrypt the data in the payload region of the data buffer. 270.It Dv CRYPTO_OP_DECRYPT 271Decrypt the data in the payload region of the data buffer. 272.El 273.Pp 274Digest requests support the following operations: 275.Bl -tag -width CRYPTO_OP_COMPUTE_DIGEST 276.It Dv CRYPTO_OP_COMPUTE_DIGEST 277Calculate a digest over the payload region of the data buffer 278and store the result in the digest region. 279.It Dv CRYPTO_OP_VERIFY_DIGEST 280Calculate a digest over the payload region of the data buffer. 281Compare the calculated digest to the existing digest from the digest region. 282If the digests match, 283complete the request successfully. 284If the digests do not match, 285fail the request with 286.Er EBADMSG . 287.El 288.Pp 289AEAD and Encrypt-then-Authenticate requests support the following 290operations: 291.Bl -tag -width CRYPTO_OP 292.It Dv CRYPTO_OP_ENCRYPT | Dv CRYPTO_OP_COMPUTE_DIGEST 293Encrypt the data in the payload region of the data buffer. 294Calculate a digest over the AAD and payload regions and store the 295result in the data buffer. 296.It Dv CRYPTO_OP_DECRYPT | Dv CRYPTO_OP_VERIFY_DIGEST 297Calculate a digest over the AAD and payload regions of the data buffer. 298Compare the calculated digest to the existing digest from the digest region. 299If the digests match, 300decrypt the payload region. 301If the digests do not match, 302fail the request with 303.Er EBADMSG . 304.El 305.Ss Request AAD 306AEAD and Encrypt-then-Authenticate requests may optionally include 307Additional Authenticated Data. 308AAD may either be supplied in the AAD region of the input buffer or 309as a single buffer pointed to by 310.Fa crp_aad . 311In either case, 312.Fa crp_aad_length 313always indicates the amount of AAD in bytes. 314.Ss Request ESN 315IPsec requests may optionally include Extended Sequence Numbers (ESN). 316ESN may either be supplied in 317.Fa crp_esn 318or as part of the AAD pointed to by 319.Fa crp_aad . 320.Pp 321If the ESN is stored in 322.Fa crp_esn , 323.Dv CSP_F_ESN 324should be set in 325.Fa csp_flags . 326This use case is dedicated for encrypt and authenticate mode, since the 327high-order 32 bits of the sequence number are appended after the Next Header 328(RFC 4303). 329.Pp 330AEAD modes supply the ESN in a separate AAD buffer (see e.g. RFC 4106, Chapter 5 331AAD Construction). 332.Ss Request IV and/or Nonce 333Some cryptographic operations require an IV or nonce as an input. 334An IV may be stored either in the IV region of the data buffer or in 335.Fa crp_iv . 336By default, 337the IV is assumed to be stored in the IV region. 338If the IV is stored in 339.Fa crp_iv , 340.Dv CRYPTO_F_IV_SEPARATE 341should be set in 342.Fa crp_flags 343and 344.Fa crp_iv_start 345should be left as zero. 346.Pp 347Requests that store part, but not all, of the IV in the data buffer should 348store the partial IV in the data buffer and pass the full IV separately in 349.Fa crp_iv . 350.Ss Request and Callback Scheduling 351The crypto framework provides multiple methods of scheduling the dispatch 352of requests to drivers along with the processing of driver callbacks. 353The 354.Fn crypto_dispatch , 355.Fn crypto_dispatch_async , 356and 357.Fn crypto_dispatch_batch 358functions can be used to request different dispatch scheduling policies. 359.Pp 360.Fn crypto_dispatch 361synchronously passes the request to the driver. 362The driver itself may process the request synchronously or asynchronously 363depending on whether the driver is implemented by software or hardware. 364.Pp 365.Fn crypto_dispatch_async 366dispatches the request asynchronously. 367If the driver is inherently synchronous, the request is queued to a taskqueue 368backed by a pool of worker threads. 369This can increase througput by allowing requests from a single producer to be 370processed in parallel. 371By default the pool is sized to provide one thread for each CPU. 372Worker threads dequeue requests and pass them to the driver asynchronously. 373.Fn crypto_dispatch_async 374additionally takes a 375.Va flags 376parameter. 377The 378.Dv CRYPTO_ASYNC_ORDERED 379flag indicates that completion callbacks for requests must be called in the 380same order as requests were dispatched. 381If the driver is asynchronous, the behavior of 382.Fn crypto_dispatch_async 383is identical to that of 384.Fn crypto_dispatch . 385.Pp 386.Fn crypto_dispatch_batch 387allows the caller to collect a batch of requests and submit them to the driver 388at the same time. 389This allows hardware drivers to optimize the scheduling of request processing 390and batch completion interrupts. 391A batch is submitted to the driver by invoking the driver's process method on 392each request, specifying 393.Dv CRYPTO_HINT_MORE 394with each request except for the last. 395The 396.Fa flags 397parameter to 398.Fn crypto_dispatch_batch 399is currently ignored. 400.Pp 401Callback function scheduling is simpler than request scheduling. 402Callbacks can either be invoked synchronously from 403.Fn crypto_done , 404or they can be queued to a pool of worker threads. 405This pool of worker threads is also sized to provide one worker thread 406for each CPU by default. 407Note that a callback function invoked synchronously from 408.Fn crypto_done 409must follow the same restrictions placed on threaded interrupt handlers. 410.Pp 411By default, 412callbacks are invoked asynchronously by a worker thread. 413If 414.Dv CRYPTO_F_CBIMM 415is set, 416the callback is always invoked synchronously from 417.Fn crypto_done . 418If 419.Dv CRYPTO_F_CBIFSYNC 420is set, 421the callback is invoked synchronously if the request was processed by a 422software driver or asynchronously if the request was processed by a 423hardware driver. 424.Pp 425If a request was scheduled to the taskqueue via 426.Dv CRYPTO_F_ASYNC , 427callbacks are always invoked asynchronously ignoring 428.Dv CRYPTO_F_CBIMM 429and 430.Dv CRYPTO_F_CBIFSYNC . 431In this case, 432.Dv CRYPTO_F_ASYNC_KEEPORDER 433may be set to ensure that callbacks for requests on a given session are 434invoked in the same order that requests were queued to the session via 435.Fn crypto_dispatch . 436This flag is used by IPsec to ensure that decrypted network packets are 437passed up the network stack in roughly the same order they were received. 438.Pp 439.Ss Other Request Fields 440In addition to the fields and flags enumerated above, 441.Vt struct cryptop 442includes the following: 443.Bl -tag -width crp_payload_length 444.It Fa crp_session 445A reference to the active session. 446This is set when the request is created by 447.Fn crypto_getreq 448and should not be modified. 449Drivers can use this to fetch driver-specific session state or 450session parameters. 451.It Fa crp_etype 452Error status. 453Either zero on success, or an error if a request fails. 454Set by drivers prior to completing a request via 455.Fn crypto_done . 456.It Fa crp_flags 457A bitmask of flags. 458The following flags are available in addition to flags discussed previously: 459.Bl -tag -width CRYPTO_F_DONE 460.It Dv CRYPTO_F_DONE 461Set by 462.Fa crypto_done 463before calling 464.Fa crp_callback . 465This flag is not very useful and will likely be removed in the future. 466It can only be safely checked from the callback routine at which point 467it is always set. 468.El 469.It Fa crp_cipher_key 470Pointer to a request-specific encryption key. 471If this value is not set, 472the request uses the session encryption key. 473.It Fa crp_auth_key 474Pointer to a request-specific authentication key. 475If this value is not set, 476the request uses the session authentication key. 477.It Fa crp_opaque 478An opaque pointer. 479This pointer permits users of the cryptographic framework to store 480information about a request to be used in the callback. 481.It Fa crp_callback 482Callback function. 483This must point to a callback function of type 484.Vt void (*)(struct cryptop *) . 485The callback function should inspect 486.Fa crp_etype 487to determine the status of the completed operation. 488It should also arrange for the request to be freed via 489.Fn crypto_freereq . 490.It Fa crp_olen 491Used with compression and decompression requests to describe the updated 492length of the payload region in the data buffer. 493.Pp 494If a compression request increases the size of the payload, 495then the data buffer is unmodified, the request completes successfully, 496and 497.Fa crp_olen 498is set to the size the compressed data would have used. 499Callers can compare this to the payload region length to determine if 500the compressed data was discarded. 501.El 502.Sh RETURN VALUES 503.Fn crypto_dispatch 504returns an error if the request contained invalid fields, 505or zero if the request was valid. 506.Fn crypto_getreq 507returns a pointer to a new request structure on success, 508or 509.Dv NULL 510on failure. 511.Dv NULL 512can only be returned if 513.Dv M_NOWAIT 514was passed in 515.Fa how . 516.Sh SEE ALSO 517.Xr ipsec 4 , 518.Xr crypto 7 , 519.Xr crypto 9 , 520.Xr crypto_session 9 , 521.Xr mbuf 9 522.Xr uio 9 523.Sh BUGS 524Not all drivers properly handle mixing session and per-request keys 525within a single session. 526Consumers should either use a single key for a session specified in 527the session parameters or always use per-request keys. 528