1.. SPDX-License-Identifier: GPL-2.0 2 3====================== 4RxRPC Network Protocol 5====================== 6 7The RxRPC protocol driver provides a reliable two-phase transport on top of UDP 8that can be used to perform RxRPC remote operations. This is done over sockets 9of AF_RXRPC family, using sendmsg() and recvmsg() with control data to send and 10receive data, aborts and errors. 11 12Contents of this document: 13 14 (#) Overview. 15 16 (#) RxRPC protocol summary. 17 18 (#) AF_RXRPC driver model. 19 20 (#) Control messages. 21 22 (#) Socket options. 23 24 (#) Security. 25 26 (#) Example client usage. 27 28 (#) Example server usage. 29 30 (#) AF_RXRPC kernel interface. 31 32 (#) Configurable parameters. 33 34 35Overview 36======== 37 38RxRPC is a two-layer protocol. There is a session layer which provides 39reliable virtual connections using UDP over IPv4 (or IPv6) as the transport 40layer, but implements a real network protocol; and there's the presentation 41layer which renders structured data to binary blobs and back again using XDR 42(as does SunRPC):: 43 44 +-------------+ 45 | Application | 46 +-------------+ 47 | XDR | Presentation 48 +-------------+ 49 | RxRPC | Session 50 +-------------+ 51 | UDP | Transport 52 +-------------+ 53 54 55AF_RXRPC provides: 56 57 (1) Part of an RxRPC facility for both kernel and userspace applications by 58 making the session part of it a Linux network protocol (AF_RXRPC). 59 60 (2) A two-phase protocol. The client transmits a blob (the request) and then 61 receives a blob (the reply), and the server receives the request and then 62 transmits the reply. 63 64 (3) Retention of the reusable bits of the transport system set up for one call 65 to speed up subsequent calls. 66 67 (4) A secure protocol, using the Linux kernel's key retention facility to 68 manage security on the client end. The server end must of necessity be 69 more active in security negotiations. 70 71AF_RXRPC does not provide XDR marshalling/presentation facilities. That is 72left to the application. AF_RXRPC only deals in blobs. Even the operation ID 73is just the first four bytes of the request blob, and as such is beyond the 74kernel's interest. 75 76 77Sockets of AF_RXRPC family are: 78 79 (1) created as type SOCK_DGRAM; 80 81 (2) provided with a protocol of the type of underlying transport they're going 82 to use - currently only PF_INET is supported. 83 84 85The Andrew File System (AFS) is an example of an application that uses this and 86that has both kernel (filesystem) and userspace (utility) components. 87 88 89RxRPC Protocol Summary 90====================== 91 92An overview of the RxRPC protocol: 93 94 (#) RxRPC sits on top of another networking protocol (UDP is the only option 95 currently), and uses this to provide network transport. UDP ports, for 96 example, provide transport endpoints. 97 98 (#) RxRPC supports multiple virtual "connections" from any given transport 99 endpoint, thus allowing the endpoints to be shared, even to the same 100 remote endpoint. 101 102 (#) Each connection goes to a particular "service". A connection may not go 103 to multiple services. A service may be considered the RxRPC equivalent of 104 a port number. AF_RXRPC permits multiple services to share an endpoint. 105 106 (#) Client-originating packets are marked, thus a transport endpoint can be 107 shared between client and server connections (connections have a 108 direction). 109 110 (#) Up to a billion connections may be supported concurrently between one 111 local transport endpoint and one service on one remote endpoint. An RxRPC 112 connection is described by seven numbers:: 113 114 Local address } 115 Local port } Transport (UDP) address 116 Remote address } 117 Remote port } 118 Direction 119 Connection ID 120 Service ID 121 122 (#) Each RxRPC operation is a "call". A connection may make up to four 123 billion calls, but only up to four calls may be in progress on a 124 connection at any one time. 125 126 (#) Calls are two-phase and asymmetric: the client sends its request data, 127 which the service receives; then the service transmits the reply data 128 which the client receives. 129 130 (#) The data blobs are of indefinite size, the end of a phase is marked with a 131 flag in the packet. The number of packets of data making up one blob may 132 not exceed 4 billion, however, as this would cause the sequence number to 133 wrap. 134 135 (#) The first four bytes of the request data are the service operation ID. 136 137 (#) Security is negotiated on a per-connection basis. The connection is 138 initiated by the first data packet on it arriving. If security is 139 requested, the server then issues a "challenge" and then the client 140 replies with a "response". If the response is successful, the security is 141 set for the lifetime of that connection, and all subsequent calls made 142 upon it use that same security. In the event that the server lets a 143 connection lapse before the client, the security will be renegotiated if 144 the client uses the connection again. 145 146 (#) Calls use ACK packets to handle reliability. Data packets are also 147 explicitly sequenced per call. 148 149 (#) There are two types of positive acknowledgment: hard-ACKs and soft-ACKs. 150 A hard-ACK indicates to the far side that all the data received to a point 151 has been received and processed; a soft-ACK indicates that the data has 152 been received but may yet be discarded and re-requested. The sender may 153 not discard any transmittable packets until they've been hard-ACK'd. 154 155 (#) Reception of a reply data packet implicitly hard-ACK's all the data 156 packets that make up the request. 157 158 (#) An call is complete when the request has been sent, the reply has been 159 received and the final hard-ACK on the last packet of the reply has 160 reached the server. 161 162 (#) An call may be aborted by either end at any time up to its completion. 163 164 165AF_RXRPC Driver Model 166===================== 167 168About the AF_RXRPC driver: 169 170 (#) The AF_RXRPC protocol transparently uses internal sockets of the transport 171 protocol to represent transport endpoints. 172 173 (#) AF_RXRPC sockets map onto RxRPC connection bundles. Actual RxRPC 174 connections are handled transparently. One client socket may be used to 175 make multiple simultaneous calls to the same service. One server socket 176 may handle calls from many clients. 177 178 (#) Additional parallel client connections will be initiated to support extra 179 concurrent calls, up to a tunable limit. 180 181 (#) Each connection is retained for a certain amount of time [tunable] after 182 the last call currently using it has completed in case a new call is made 183 that could reuse it. 184 185 (#) Each internal UDP socket is retained [tunable] for a certain amount of 186 time [tunable] after the last connection using it discarded, in case a new 187 connection is made that could use it. 188 189 (#) A client-side connection is only shared between calls if they have 190 the same key struct describing their security (and assuming the calls 191 would otherwise share the connection). Non-secured calls would also be 192 able to share connections with each other. 193 194 (#) A server-side connection is shared if the client says it is. 195 196 (#) ACK'ing is handled by the protocol driver automatically, including ping 197 replying. 198 199 (#) SO_KEEPALIVE automatically pings the other side to keep the connection 200 alive [TODO]. 201 202 (#) If an ICMP error is received, all calls affected by that error will be 203 aborted with an appropriate network error passed through recvmsg(). 204 205 206Interaction with the user of the RxRPC socket: 207 208 (#) A socket is made into a server socket by binding an address with a 209 non-zero service ID. 210 211 (#) In the client, sending a request is achieved with one or more sendmsgs, 212 followed by the reply being received with one or more recvmsgs. 213 214 (#) The first sendmsg for a request to be sent from a client contains a tag to 215 be used in all other sendmsgs or recvmsgs associated with that call. The 216 tag is carried in the control data. 217 218 (#) connect() is used to supply a default destination address for a client 219 socket. This may be overridden by supplying an alternate address to the 220 first sendmsg() of a call (struct msghdr::msg_name). 221 222 (#) If connect() is called on an unbound client, a random local port will 223 bound before the operation takes place. 224 225 (#) A server socket may also be used to make client calls. To do this, the 226 first sendmsg() of the call must specify the target address. The server's 227 transport endpoint is used to send the packets. 228 229 (#) Once the application has received the last message associated with a call, 230 the tag is guaranteed not to be seen again, and so it can be used to pin 231 client resources. A new call can then be initiated with the same tag 232 without fear of interference. 233 234 (#) In the server, a request is received with one or more recvmsgs, then the 235 the reply is transmitted with one or more sendmsgs, and then the final ACK 236 is received with a last recvmsg. 237 238 (#) When sending data for a call, sendmsg is given MSG_MORE if there's more 239 data to come on that call. 240 241 (#) When receiving data for a call, recvmsg flags MSG_MORE if there's more 242 data to come for that call. 243 244 (#) When receiving data or messages for a call, MSG_EOR is flagged by recvmsg 245 to indicate the terminal message for that call. 246 247 (#) A call may be aborted by adding an abort control message to the control 248 data. Issuing an abort terminates the kernel's use of that call's tag. 249 Any messages waiting in the receive queue for that call will be discarded. 250 251 (#) Aborts, busy notifications and challenge packets are delivered by recvmsg, 252 and control data messages will be set to indicate the context. Receiving 253 an abort or a busy message terminates the kernel's use of that call's tag. 254 255 (#) The control data part of the msghdr struct is used for a number of things: 256 257 (#) The tag of the intended or affected call. 258 259 (#) Sending or receiving errors, aborts and busy notifications. 260 261 (#) Notifications of incoming calls. 262 263 (#) Sending debug requests and receiving debug replies [TODO]. 264 265 (#) When the kernel has received and set up an incoming call, it sends a 266 message to server application to let it know there's a new call awaiting 267 its acceptance [recvmsg reports a special control message]. The server 268 application then uses sendmsg to assign a tag to the new call. Once that 269 is done, the first part of the request data will be delivered by recvmsg. 270 271 (#) The server application has to provide the server socket with a keyring of 272 secret keys corresponding to the security types it permits. When a secure 273 connection is being set up, the kernel looks up the appropriate secret key 274 in the keyring and then sends a challenge packet to the client and 275 receives a response packet. The kernel then checks the authorisation of 276 the packet and either aborts the connection or sets up the security. 277 278 (#) The name of the key a client will use to secure its communications is 279 nominated by a socket option. 280 281 282Notes on sendmsg: 283 284 (#) MSG_WAITALL can be set to tell sendmsg to ignore signals if the peer is 285 making progress at accepting packets within a reasonable time such that we 286 manage to queue up all the data for transmission. This requires the 287 client to accept at least one packet per 2*RTT time period. 288 289 If this isn't set, sendmsg() will return immediately, either returning 290 EINTR/ERESTARTSYS if nothing was consumed or returning the amount of data 291 consumed. 292 293 294Notes on recvmsg: 295 296 (#) If there's a sequence of data messages belonging to a particular call on 297 the receive queue, then recvmsg will keep working through them until: 298 299 (a) it meets the end of that call's received data, 300 301 (b) it meets a non-data message, 302 303 (c) it meets a message belonging to a different call, or 304 305 (d) it fills the user buffer. 306 307 If recvmsg is called in blocking mode, it will keep sleeping, awaiting the 308 reception of further data, until one of the above four conditions is met. 309 310 (2) MSG_PEEK operates similarly, but will return immediately if it has put any 311 data in the buffer rather than sleeping until it can fill the buffer. 312 313 (3) If a data message is only partially consumed in filling a user buffer, 314 then the remainder of that message will be left on the front of the queue 315 for the next taker. MSG_TRUNC will never be flagged. 316 317 (4) If there is more data to be had on a call (it hasn't copied the last byte 318 of the last data message in that phase yet), then MSG_MORE will be 319 flagged. 320 321 322Control Messages 323================ 324 325AF_RXRPC makes use of control messages in sendmsg() and recvmsg() to multiplex 326calls, to invoke certain actions and to report certain conditions. These are: 327 328 ======================= === =========== =============================== 329 MESSAGE ID SRT DATA MEANING 330 ======================= === =========== =============================== 331 RXRPC_USER_CALL_ID sr- User ID App's call specifier 332 RXRPC_ABORT srt Abort code Abort code to issue/received 333 RXRPC_ACK -rt n/a Final ACK received 334 RXRPC_NET_ERROR -rt error num Network error on call 335 RXRPC_BUSY -rt n/a Call rejected (server busy) 336 RXRPC_LOCAL_ERROR -rt error num Local error encountered 337 RXRPC_NEW_CALL -r- n/a New call received 338 RXRPC_ACCEPT s-- n/a Accept new call 339 RXRPC_EXCLUSIVE_CALL s-- n/a Make an exclusive client call 340 RXRPC_UPGRADE_SERVICE s-- n/a Client call can be upgraded 341 RXRPC_TX_LENGTH s-- data len Total length of Tx data 342 ======================= === =========== =============================== 343 344 (SRT = usable in Sendmsg / delivered by Recvmsg / Terminal message) 345 346 (#) RXRPC_USER_CALL_ID 347 348 This is used to indicate the application's call ID. It's an unsigned long 349 that the app specifies in the client by attaching it to the first data 350 message or in the server by passing it in association with an RXRPC_ACCEPT 351 message. recvmsg() passes it in conjunction with all messages except 352 those of the RXRPC_NEW_CALL message. 353 354 (#) RXRPC_ABORT 355 356 This is can be used by an application to abort a call by passing it to 357 sendmsg, or it can be delivered by recvmsg to indicate a remote abort was 358 received. Either way, it must be associated with an RXRPC_USER_CALL_ID to 359 specify the call affected. If an abort is being sent, then error EBADSLT 360 will be returned if there is no call with that user ID. 361 362 (#) RXRPC_ACK 363 364 This is delivered to a server application to indicate that the final ACK 365 of a call was received from the client. It will be associated with an 366 RXRPC_USER_CALL_ID to indicate the call that's now complete. 367 368 (#) RXRPC_NET_ERROR 369 370 This is delivered to an application to indicate that an ICMP error message 371 was encountered in the process of trying to talk to the peer. An 372 errno-class integer value will be included in the control message data 373 indicating the problem, and an RXRPC_USER_CALL_ID will indicate the call 374 affected. 375 376 (#) RXRPC_BUSY 377 378 This is delivered to a client application to indicate that a call was 379 rejected by the server due to the server being busy. It will be 380 associated with an RXRPC_USER_CALL_ID to indicate the rejected call. 381 382 (#) RXRPC_LOCAL_ERROR 383 384 This is delivered to an application to indicate that a local error was 385 encountered and that a call has been aborted because of it. An 386 errno-class integer value will be included in the control message data 387 indicating the problem, and an RXRPC_USER_CALL_ID will indicate the call 388 affected. 389 390 (#) RXRPC_NEW_CALL 391 392 This is delivered to indicate to a server application that a new call has 393 arrived and is awaiting acceptance. No user ID is associated with this, 394 as a user ID must subsequently be assigned by doing an RXRPC_ACCEPT. 395 396 (#) RXRPC_ACCEPT 397 398 This is used by a server application to attempt to accept a call and 399 assign it a user ID. It should be associated with an RXRPC_USER_CALL_ID 400 to indicate the user ID to be assigned. If there is no call to be 401 accepted (it may have timed out, been aborted, etc.), then sendmsg will 402 return error ENODATA. If the user ID is already in use by another call, 403 then error EBADSLT will be returned. 404 405 (#) RXRPC_EXCLUSIVE_CALL 406 407 This is used to indicate that a client call should be made on a one-off 408 connection. The connection is discarded once the call has terminated. 409 410 (#) RXRPC_UPGRADE_SERVICE 411 412 This is used to make a client call to probe if the specified service ID 413 may be upgraded by the server. The caller must check msg_name returned to 414 recvmsg() for the service ID actually in use. The operation probed must 415 be one that takes the same arguments in both services. 416 417 Once this has been used to establish the upgrade capability (or lack 418 thereof) of the server, the service ID returned should be used for all 419 future communication to that server and RXRPC_UPGRADE_SERVICE should no 420 longer be set. 421 422 (#) RXRPC_TX_LENGTH 423 424 This is used to inform the kernel of the total amount of data that is 425 going to be transmitted by a call (whether in a client request or a 426 service response). If given, it allows the kernel to encrypt from the 427 userspace buffer directly to the packet buffers, rather than copying into 428 the buffer and then encrypting in place. This may only be given with the 429 first sendmsg() providing data for a call. EMSGSIZE will be generated if 430 the amount of data actually given is different. 431 432 This takes a parameter of __s64 type that indicates how much will be 433 transmitted. This may not be less than zero. 434 435The symbol RXRPC__SUPPORTED is defined as one more than the highest control 436message type supported. At run time this can be queried by means of the 437RXRPC_SUPPORTED_CMSG socket option (see below). 438 439 440Socket Options 441============== 442 443AF_RXRPC sockets support a few socket options at the SOL_RXRPC level: 444 445 (#) RXRPC_SECURITY_KEY 446 447 This is used to specify the description of the key to be used. The key is 448 extracted from the calling process's keyrings with request_key() and 449 should be of "rxrpc" type. 450 451 The optval pointer points to the description string, and optlen indicates 452 how long the string is, without the NUL terminator. 453 454 (#) RXRPC_SECURITY_KEYRING 455 456 Similar to above but specifies a keyring of server secret keys to use (key 457 type "keyring"). See the "Security" section. 458 459 (#) RXRPC_EXCLUSIVE_CONNECTION 460 461 This is used to request that new connections should be used for each call 462 made subsequently on this socket. optval should be NULL and optlen 0. 463 464 (#) RXRPC_MIN_SECURITY_LEVEL 465 466 This is used to specify the minimum security level required for calls on 467 this socket. optval must point to an int containing one of the following 468 values: 469 470 (a) RXRPC_SECURITY_PLAIN 471 472 Encrypted checksum only. 473 474 (b) RXRPC_SECURITY_AUTH 475 476 Encrypted checksum plus packet padded and first eight bytes of packet 477 encrypted - which includes the actual packet length. 478 479 (c) RXRPC_SECURITY_ENCRYPT 480 481 Encrypted checksum plus entire packet padded and encrypted, including 482 actual packet length. 483 484 (#) RXRPC_UPGRADEABLE_SERVICE 485 486 This is used to indicate that a service socket with two bindings may 487 upgrade one bound service to the other if requested by the client. optval 488 must point to an array of two unsigned short ints. The first is the 489 service ID to upgrade from and the second the service ID to upgrade to. 490 491 (#) RXRPC_SUPPORTED_CMSG 492 493 This is a read-only option that writes an int into the buffer indicating 494 the highest control message type supported. 495 496 497Security 498======== 499 500Currently, only the kerberos 4 equivalent protocol has been implemented 501(security index 2 - rxkad). This requires the rxkad module to be loaded and, 502on the client, tickets of the appropriate type to be obtained from the AFS 503kaserver or the kerberos server and installed as "rxrpc" type keys. This is 504normally done using the klog program. An example simple klog program can be 505found at: 506 507 http://people.redhat.com/~dhowells/rxrpc/klog.c 508 509The payload provided to add_key() on the client should be of the following 510form:: 511 512 struct rxrpc_key_sec2_v1 { 513 uint16_t security_index; /* 2 */ 514 uint16_t ticket_length; /* length of ticket[] */ 515 uint32_t expiry; /* time at which expires */ 516 uint8_t kvno; /* key version number */ 517 uint8_t __pad[3]; 518 uint8_t session_key[8]; /* DES session key */ 519 uint8_t ticket[0]; /* the encrypted ticket */ 520 }; 521 522Where the ticket blob is just appended to the above structure. 523 524 525For the server, keys of type "rxrpc_s" must be made available to the server. 526They have a description of "<serviceID>:<securityIndex>" (eg: "52:2" for an 527rxkad key for the AFS VL service). When such a key is created, it should be 528given the server's secret key as the instantiation data (see the example 529below). 530 531 add_key("rxrpc_s", "52:2", secret_key, 8, keyring); 532 533A keyring is passed to the server socket by naming it in a sockopt. The server 534socket then looks the server secret keys up in this keyring when secure 535incoming connections are made. This can be seen in an example program that can 536be found at: 537 538 http://people.redhat.com/~dhowells/rxrpc/listen.c 539 540 541Example Client Usage 542==================== 543 544A client would issue an operation by: 545 546 (1) An RxRPC socket is set up by:: 547 548 client = socket(AF_RXRPC, SOCK_DGRAM, PF_INET); 549 550 Where the third parameter indicates the protocol family of the transport 551 socket used - usually IPv4 but it can also be IPv6 [TODO]. 552 553 (2) A local address can optionally be bound:: 554 555 struct sockaddr_rxrpc srx = { 556 .srx_family = AF_RXRPC, 557 .srx_service = 0, /* we're a client */ 558 .transport_type = SOCK_DGRAM, /* type of transport socket */ 559 .transport.sin_family = AF_INET, 560 .transport.sin_port = htons(7000), /* AFS callback */ 561 .transport.sin_address = 0, /* all local interfaces */ 562 }; 563 bind(client, &srx, sizeof(srx)); 564 565 This specifies the local UDP port to be used. If not given, a random 566 non-privileged port will be used. A UDP port may be shared between 567 several unrelated RxRPC sockets. Security is handled on a basis of 568 per-RxRPC virtual connection. 569 570 (3) The security is set:: 571 572 const char *key = "AFS:cambridge.redhat.com"; 573 setsockopt(client, SOL_RXRPC, RXRPC_SECURITY_KEY, key, strlen(key)); 574 575 This issues a request_key() to get the key representing the security 576 context. The minimum security level can be set:: 577 578 unsigned int sec = RXRPC_SECURITY_ENCRYPT; 579 setsockopt(client, SOL_RXRPC, RXRPC_MIN_SECURITY_LEVEL, 580 &sec, sizeof(sec)); 581 582 (4) The server to be contacted can then be specified (alternatively this can 583 be done through sendmsg):: 584 585 struct sockaddr_rxrpc srx = { 586 .srx_family = AF_RXRPC, 587 .srx_service = VL_SERVICE_ID, 588 .transport_type = SOCK_DGRAM, /* type of transport socket */ 589 .transport.sin_family = AF_INET, 590 .transport.sin_port = htons(7005), /* AFS volume manager */ 591 .transport.sin_address = ..., 592 }; 593 connect(client, &srx, sizeof(srx)); 594 595 (5) The request data should then be posted to the server socket using a series 596 of sendmsg() calls, each with the following control message attached: 597 598 ================== =================================== 599 RXRPC_USER_CALL_ID specifies the user ID for this call 600 ================== =================================== 601 602 MSG_MORE should be set in msghdr::msg_flags on all but the last part of 603 the request. Multiple requests may be made simultaneously. 604 605 An RXRPC_TX_LENGTH control message can also be specified on the first 606 sendmsg() call. 607 608 If a call is intended to go to a destination other than the default 609 specified through connect(), then msghdr::msg_name should be set on the 610 first request message of that call. 611 612 (6) The reply data will then be posted to the server socket for recvmsg() to 613 pick up. MSG_MORE will be flagged by recvmsg() if there's more reply data 614 for a particular call to be read. MSG_EOR will be set on the terminal 615 read for a call. 616 617 All data will be delivered with the following control message attached: 618 619 RXRPC_USER_CALL_ID - specifies the user ID for this call 620 621 If an abort or error occurred, this will be returned in the control data 622 buffer instead, and MSG_EOR will be flagged to indicate the end of that 623 call. 624 625A client may ask for a service ID it knows and ask that this be upgraded to a 626better service if one is available by supplying RXRPC_UPGRADE_SERVICE on the 627first sendmsg() of a call. The client should then check srx_service in the 628msg_name filled in by recvmsg() when collecting the result. srx_service will 629hold the same value as given to sendmsg() if the upgrade request was ignored by 630the service - otherwise it will be altered to indicate the service ID the 631server upgraded to. Note that the upgraded service ID is chosen by the server. 632The caller has to wait until it sees the service ID in the reply before sending 633any more calls (further calls to the same destination will be blocked until the 634probe is concluded). 635 636 637Example Server Usage 638==================== 639 640A server would be set up to accept operations in the following manner: 641 642 (1) An RxRPC socket is created by:: 643 644 server = socket(AF_RXRPC, SOCK_DGRAM, PF_INET); 645 646 Where the third parameter indicates the address type of the transport 647 socket used - usually IPv4. 648 649 (2) Security is set up if desired by giving the socket a keyring with server 650 secret keys in it:: 651 652 keyring = add_key("keyring", "AFSkeys", NULL, 0, 653 KEY_SPEC_PROCESS_KEYRING); 654 655 const char secret_key[8] = { 656 0xa7, 0x83, 0x8a, 0xcb, 0xc7, 0x83, 0xec, 0x94 }; 657 add_key("rxrpc_s", "52:2", secret_key, 8, keyring); 658 659 setsockopt(server, SOL_RXRPC, RXRPC_SECURITY_KEYRING, "AFSkeys", 7); 660 661 The keyring can be manipulated after it has been given to the socket. This 662 permits the server to add more keys, replace keys, etc. while it is live. 663 664 (3) A local address must then be bound:: 665 666 struct sockaddr_rxrpc srx = { 667 .srx_family = AF_RXRPC, 668 .srx_service = VL_SERVICE_ID, /* RxRPC service ID */ 669 .transport_type = SOCK_DGRAM, /* type of transport socket */ 670 .transport.sin_family = AF_INET, 671 .transport.sin_port = htons(7000), /* AFS callback */ 672 .transport.sin_address = 0, /* all local interfaces */ 673 }; 674 bind(server, &srx, sizeof(srx)); 675 676 More than one service ID may be bound to a socket, provided the transport 677 parameters are the same. The limit is currently two. To do this, bind() 678 should be called twice. 679 680 (4) If service upgrading is required, first two service IDs must have been 681 bound and then the following option must be set:: 682 683 unsigned short service_ids[2] = { from_ID, to_ID }; 684 setsockopt(server, SOL_RXRPC, RXRPC_UPGRADEABLE_SERVICE, 685 service_ids, sizeof(service_ids)); 686 687 This will automatically upgrade connections on service from_ID to service 688 to_ID if they request it. This will be reflected in msg_name obtained 689 through recvmsg() when the request data is delivered to userspace. 690 691 (5) The server is then set to listen out for incoming calls:: 692 693 listen(server, 100); 694 695 (6) The kernel notifies the server of pending incoming connections by sending 696 it a message for each. This is received with recvmsg() on the server 697 socket. It has no data, and has a single dataless control message 698 attached:: 699 700 RXRPC_NEW_CALL 701 702 The address that can be passed back by recvmsg() at this point should be 703 ignored since the call for which the message was posted may have gone by 704 the time it is accepted - in which case the first call still on the queue 705 will be accepted. 706 707 (7) The server then accepts the new call by issuing a sendmsg() with two 708 pieces of control data and no actual data: 709 710 ================== ============================== 711 RXRPC_ACCEPT indicate connection acceptance 712 RXRPC_USER_CALL_ID specify user ID for this call 713 ================== ============================== 714 715 (8) The first request data packet will then be posted to the server socket for 716 recvmsg() to pick up. At that point, the RxRPC address for the call can 717 be read from the address fields in the msghdr struct. 718 719 Subsequent request data will be posted to the server socket for recvmsg() 720 to collect as it arrives. All but the last piece of the request data will 721 be delivered with MSG_MORE flagged. 722 723 All data will be delivered with the following control message attached: 724 725 726 ================== =================================== 727 RXRPC_USER_CALL_ID specifies the user ID for this call 728 ================== =================================== 729 730 (9) The reply data should then be posted to the server socket using a series 731 of sendmsg() calls, each with the following control messages attached: 732 733 ================== =================================== 734 RXRPC_USER_CALL_ID specifies the user ID for this call 735 ================== =================================== 736 737 MSG_MORE should be set in msghdr::msg_flags on all but the last message 738 for a particular call. 739 740(10) The final ACK from the client will be posted for retrieval by recvmsg() 741 when it is received. It will take the form of a dataless message with two 742 control messages attached: 743 744 ================== =================================== 745 RXRPC_USER_CALL_ID specifies the user ID for this call 746 RXRPC_ACK indicates final ACK (no data) 747 ================== =================================== 748 749 MSG_EOR will be flagged to indicate that this is the final message for 750 this call. 751 752(11) Up to the point the final packet of reply data is sent, the call can be 753 aborted by calling sendmsg() with a dataless message with the following 754 control messages attached: 755 756 ================== =================================== 757 RXRPC_USER_CALL_ID specifies the user ID for this call 758 RXRPC_ABORT indicates abort code (4 byte data) 759 ================== =================================== 760 761 Any packets waiting in the socket's receive queue will be discarded if 762 this is issued. 763 764Note that all the communications for a particular service take place through 765the one server socket, using control messages on sendmsg() and recvmsg() to 766determine the call affected. 767 768 769AF_RXRPC Kernel Interface 770========================= 771 772The AF_RXRPC module also provides an interface for use by in-kernel utilities 773such as the AFS filesystem. This permits such a utility to: 774 775 (1) Use different keys directly on individual client calls on one socket 776 rather than having to open a whole slew of sockets, one for each key it 777 might want to use. 778 779 (2) Avoid having RxRPC call request_key() at the point of issue of a call or 780 opening of a socket. Instead the utility is responsible for requesting a 781 key at the appropriate point. AFS, for instance, would do this during VFS 782 operations such as open() or unlink(). The key is then handed through 783 when the call is initiated. 784 785 (3) Request the use of something other than GFP_KERNEL to allocate memory. 786 787 (4) Avoid the overhead of using the recvmsg() call. RxRPC messages can be 788 intercepted before they get put into the socket Rx queue and the socket 789 buffers manipulated directly. 790 791To use the RxRPC facility, a kernel utility must still open an AF_RXRPC socket, 792bind an address as appropriate and listen if it's to be a server socket, but 793then it passes this to the kernel interface functions. 794 795The kernel interface functions are as follows: 796 797 (#) Begin a new client call:: 798 799 struct rxrpc_call * 800 rxrpc_kernel_begin_call(struct socket *sock, 801 struct sockaddr_rxrpc *srx, 802 struct key *key, 803 unsigned long user_call_ID, 804 s64 tx_total_len, 805 gfp_t gfp, 806 rxrpc_notify_rx_t notify_rx, 807 bool upgrade, 808 bool intr, 809 unsigned int debug_id); 810 811 This allocates the infrastructure to make a new RxRPC call and assigns 812 call and connection numbers. The call will be made on the UDP port that 813 the socket is bound to. The call will go to the destination address of a 814 connected client socket unless an alternative is supplied (srx is 815 non-NULL). 816 817 If a key is supplied then this will be used to secure the call instead of 818 the key bound to the socket with the RXRPC_SECURITY_KEY sockopt. Calls 819 secured in this way will still share connections if at all possible. 820 821 The user_call_ID is equivalent to that supplied to sendmsg() in the 822 control data buffer. It is entirely feasible to use this to point to a 823 kernel data structure. 824 825 tx_total_len is the amount of data the caller is intending to transmit 826 with this call (or -1 if unknown at this point). Setting the data size 827 allows the kernel to encrypt directly to the packet buffers, thereby 828 saving a copy. The value may not be less than -1. 829 830 notify_rx is a pointer to a function to be called when events such as 831 incoming data packets or remote aborts happen. 832 833 upgrade should be set to true if a client operation should request that 834 the server upgrade the service to a better one. The resultant service ID 835 is returned by rxrpc_kernel_recv_data(). 836 837 intr should be set to true if the call should be interruptible. If this 838 is not set, this function may not return until a channel has been 839 allocated; if it is set, the function may return -ERESTARTSYS. 840 841 debug_id is the call debugging ID to be used for tracing. This can be 842 obtained by atomically incrementing rxrpc_debug_id. 843 844 If this function is successful, an opaque reference to the RxRPC call is 845 returned. The caller now holds a reference on this and it must be 846 properly ended. 847 848 (#) Shut down a client call:: 849 850 void rxrpc_kernel_shutdown_call(struct socket *sock, 851 struct rxrpc_call *call); 852 853 This is used to shut down a previously begun call. The user_call_ID is 854 expunged from AF_RXRPC's knowledge and will not be seen again in 855 association with the specified call. 856 857 (#) Release the ref on a client call:: 858 859 void rxrpc_kernel_put_call(struct socket *sock, 860 struct rxrpc_call *call); 861 862 This is used to release the caller's ref on an rxrpc call. 863 864 (#) Send data through a call:: 865 866 typedef void (*rxrpc_notify_end_tx_t)(struct sock *sk, 867 unsigned long user_call_ID, 868 struct sk_buff *skb); 869 870 int rxrpc_kernel_send_data(struct socket *sock, 871 struct rxrpc_call *call, 872 struct msghdr *msg, 873 size_t len, 874 rxrpc_notify_end_tx_t notify_end_rx); 875 876 This is used to supply either the request part of a client call or the 877 reply part of a server call. msg.msg_iovlen and msg.msg_iov specify the 878 data buffers to be used. msg_iov may not be NULL and must point 879 exclusively to in-kernel virtual addresses. msg.msg_flags may be given 880 MSG_MORE if there will be subsequent data sends for this call. 881 882 The msg must not specify a destination address, control data or any flags 883 other than MSG_MORE. len is the total amount of data to transmit. 884 885 notify_end_rx can be NULL or it can be used to specify a function to be 886 called when the call changes state to end the Tx phase. This function is 887 called with a spinlock held to prevent the last DATA packet from being 888 transmitted until the function returns. 889 890 (#) Receive data from a call:: 891 892 int rxrpc_kernel_recv_data(struct socket *sock, 893 struct rxrpc_call *call, 894 void *buf, 895 size_t size, 896 size_t *_offset, 897 bool want_more, 898 u32 *_abort, 899 u16 *_service) 900 901 This is used to receive data from either the reply part of a client call 902 or the request part of a service call. buf and size specify how much 903 data is desired and where to store it. *_offset is added on to buf and 904 subtracted from size internally; the amount copied into the buffer is 905 added to *_offset before returning. 906 907 want_more should be true if further data will be required after this is 908 satisfied and false if this is the last item of the receive phase. 909 910 There are three normal returns: 0 if the buffer was filled and want_more 911 was true; 1 if the buffer was filled, the last DATA packet has been 912 emptied and want_more was false; and -EAGAIN if the function needs to be 913 called again. 914 915 If the last DATA packet is processed but the buffer contains less than 916 the amount requested, EBADMSG is returned. If want_more wasn't set, but 917 more data was available, EMSGSIZE is returned. 918 919 If a remote ABORT is detected, the abort code received will be stored in 920 ``*_abort`` and ECONNABORTED will be returned. 921 922 The service ID that the call ended up with is returned into *_service. 923 This can be used to see if a call got a service upgrade. 924 925 (#) Abort a call?? 926 927 :: 928 929 void rxrpc_kernel_abort_call(struct socket *sock, 930 struct rxrpc_call *call, 931 u32 abort_code); 932 933 This is used to abort a call if it's still in an abortable state. The 934 abort code specified will be placed in the ABORT message sent. 935 936 (#) Intercept received RxRPC messages:: 937 938 typedef void (*rxrpc_interceptor_t)(struct sock *sk, 939 unsigned long user_call_ID, 940 struct sk_buff *skb); 941 942 void 943 rxrpc_kernel_intercept_rx_messages(struct socket *sock, 944 rxrpc_interceptor_t interceptor); 945 946 This installs an interceptor function on the specified AF_RXRPC socket. 947 All messages that would otherwise wind up in the socket's Rx queue are 948 then diverted to this function. Note that care must be taken to process 949 the messages in the right order to maintain DATA message sequentiality. 950 951 The interceptor function itself is provided with the address of the socket 952 and handling the incoming message, the ID assigned by the kernel utility 953 to the call and the socket buffer containing the message. 954 955 The skb->mark field indicates the type of message: 956 957 =============================== ======================================= 958 Mark Meaning 959 =============================== ======================================= 960 RXRPC_SKB_MARK_DATA Data message 961 RXRPC_SKB_MARK_FINAL_ACK Final ACK received for an incoming call 962 RXRPC_SKB_MARK_BUSY Client call rejected as server busy 963 RXRPC_SKB_MARK_REMOTE_ABORT Call aborted by peer 964 RXRPC_SKB_MARK_NET_ERROR Network error detected 965 RXRPC_SKB_MARK_LOCAL_ERROR Local error encountered 966 RXRPC_SKB_MARK_NEW_CALL New incoming call awaiting acceptance 967 =============================== ======================================= 968 969 The remote abort message can be probed with rxrpc_kernel_get_abort_code(). 970 The two error messages can be probed with rxrpc_kernel_get_error_number(). 971 A new call can be accepted with rxrpc_kernel_accept_call(). 972 973 Data messages can have their contents extracted with the usual bunch of 974 socket buffer manipulation functions. A data message can be determined to 975 be the last one in a sequence with rxrpc_kernel_is_data_last(). When a 976 data message has been used up, rxrpc_kernel_data_consumed() should be 977 called on it. 978 979 Messages should be handled to rxrpc_kernel_free_skb() to dispose of. It 980 is possible to get extra refs on all types of message for later freeing, 981 but this may pin the state of a call until the message is finally freed. 982 983 (#) Accept an incoming call:: 984 985 struct rxrpc_call * 986 rxrpc_kernel_accept_call(struct socket *sock, 987 unsigned long user_call_ID); 988 989 This is used to accept an incoming call and to assign it a call ID. This 990 function is similar to rxrpc_kernel_begin_call() and calls accepted must 991 be ended in the same way. 992 993 If this function is successful, an opaque reference to the RxRPC call is 994 returned. The caller now holds a reference on this and it must be 995 properly ended. 996 997 (#) Reject an incoming call:: 998 999 int rxrpc_kernel_reject_call(struct socket *sock); 1000 1001 This is used to reject the first incoming call on the socket's queue with 1002 a BUSY message. -ENODATA is returned if there were no incoming calls. 1003 Other errors may be returned if the call had been aborted (-ECONNABORTED) 1004 or had timed out (-ETIME). 1005 1006 (#) Allocate a null key for doing anonymous security:: 1007 1008 struct key *rxrpc_get_null_key(const char *keyname); 1009 1010 This is used to allocate a null RxRPC key that can be used to indicate 1011 anonymous security for a particular domain. 1012 1013 (#) Get the peer address of a call:: 1014 1015 void rxrpc_kernel_get_peer(struct socket *sock, struct rxrpc_call *call, 1016 struct sockaddr_rxrpc *_srx); 1017 1018 This is used to find the remote peer address of a call. 1019 1020 (#) Set the total transmit data size on a call:: 1021 1022 void rxrpc_kernel_set_tx_length(struct socket *sock, 1023 struct rxrpc_call *call, 1024 s64 tx_total_len); 1025 1026 This sets the amount of data that the caller is intending to transmit on a 1027 call. It's intended to be used for setting the reply size as the request 1028 size should be set when the call is begun. tx_total_len may not be less 1029 than zero. 1030 1031 (#) Get call RTT:: 1032 1033 u64 rxrpc_kernel_get_rtt(struct socket *sock, struct rxrpc_call *call); 1034 1035 Get the RTT time to the peer in use by a call. The value returned is in 1036 nanoseconds. 1037 1038 (#) Check call still alive:: 1039 1040 bool rxrpc_kernel_check_life(struct socket *sock, 1041 struct rxrpc_call *call, 1042 u32 *_life); 1043 void rxrpc_kernel_probe_life(struct socket *sock, 1044 struct rxrpc_call *call); 1045 1046 The first function passes back in ``*_life`` a number that is updated when 1047 ACKs are received from the peer (notably including PING RESPONSE ACKs 1048 which we can elicit by sending PING ACKs to see if the call still exists 1049 on the server). The caller should compare the numbers of two calls to see 1050 if the call is still alive after waiting for a suitable interval. It also 1051 returns true as long as the call hasn't yet reached the completed state. 1052 1053 This allows the caller to work out if the server is still contactable and 1054 if the call is still alive on the server while waiting for the server to 1055 process a client operation. 1056 1057 The second function causes a ping ACK to be transmitted to try to provoke 1058 the peer into responding, which would then cause the value returned by the 1059 first function to change. Note that this must be called in TASK_RUNNING 1060 state. 1061 1062 (#) Apply the RXRPC_MIN_SECURITY_LEVEL sockopt to a socket from within in the 1063 kernel:: 1064 1065 int rxrpc_sock_set_min_security_level(struct sock *sk, 1066 unsigned int val); 1067 1068 This specifies the minimum security level required for calls on this 1069 socket. 1070 1071 1072Configurable Parameters 1073======================= 1074 1075The RxRPC protocol driver has a number of configurable parameters that can be 1076adjusted through sysctls in /proc/net/rxrpc/: 1077 1078 (#) req_ack_delay 1079 1080 The amount of time in milliseconds after receiving a packet with the 1081 request-ack flag set before we honour the flag and actually send the 1082 requested ack. 1083 1084 Usually the other side won't stop sending packets until the advertised 1085 reception window is full (to a maximum of 255 packets), so delaying the 1086 ACK permits several packets to be ACK'd in one go. 1087 1088 (#) soft_ack_delay 1089 1090 The amount of time in milliseconds after receiving a new packet before we 1091 generate a soft-ACK to tell the sender that it doesn't need to resend. 1092 1093 (#) idle_ack_delay 1094 1095 The amount of time in milliseconds after all the packets currently in the 1096 received queue have been consumed before we generate a hard-ACK to tell 1097 the sender it can free its buffers, assuming no other reason occurs that 1098 we would send an ACK. 1099 1100 (#) resend_timeout 1101 1102 The amount of time in milliseconds after transmitting a packet before we 1103 transmit it again, assuming no ACK is received from the receiver telling 1104 us they got it. 1105 1106 (#) max_call_lifetime 1107 1108 The maximum amount of time in seconds that a call may be in progress 1109 before we preemptively kill it. 1110 1111 (#) dead_call_expiry 1112 1113 The amount of time in seconds before we remove a dead call from the call 1114 list. Dead calls are kept around for a little while for the purpose of 1115 repeating ACK and ABORT packets. 1116 1117 (#) connection_expiry 1118 1119 The amount of time in seconds after a connection was last used before we 1120 remove it from the connection list. While a connection is in existence, 1121 it serves as a placeholder for negotiated security; when it is deleted, 1122 the security must be renegotiated. 1123 1124 (#) transport_expiry 1125 1126 The amount of time in seconds after a transport was last used before we 1127 remove it from the transport list. While a transport is in existence, it 1128 serves to anchor the peer data and keeps the connection ID counter. 1129 1130 (#) rxrpc_rx_window_size 1131 1132 The size of the receive window in packets. This is the maximum number of 1133 unconsumed received packets we're willing to hold in memory for any 1134 particular call. 1135 1136 (#) rxrpc_rx_mtu 1137 1138 The maximum packet MTU size that we're willing to receive in bytes. This 1139 indicates to the peer whether we're willing to accept jumbo packets. 1140 1141 (#) rxrpc_rx_jumbo_max 1142 1143 The maximum number of packets that we're willing to accept in a jumbo 1144 packet. Non-terminal packets in a jumbo packet must contain a four byte 1145 header plus exactly 1412 bytes of data. The terminal packet must contain 1146 a four byte header plus any amount of data. In any event, a jumbo packet 1147 may not exceed rxrpc_rx_mtu in size. 1148 1149 1150API Function Reference 1151====================== 1152 1153.. kernel-doc:: net/rxrpc/af_rxrpc.c 1154.. kernel-doc:: net/rxrpc/call_object.c 1155.. kernel-doc:: net/rxrpc/key.c 1156.. kernel-doc:: net/rxrpc/oob.c 1157.. kernel-doc:: net/rxrpc/peer_object.c 1158.. kernel-doc:: net/rxrpc/recvmsg.c 1159.. kernel-doc:: net/rxrpc/rxgk.c 1160.. kernel-doc:: net/rxrpc/rxkad.c 1161.. kernel-doc:: net/rxrpc/sendmsg.c 1162.. kernel-doc:: net/rxrpc/server_key.c 1163