1*e7be843bSPierre ProncheryQUIC Server API Design 2*e7be843bSPierre Pronchery====================== 3*e7be843bSPierre Pronchery 4*e7be843bSPierre ProncheryRequirements 5*e7be843bSPierre Pronchery------------ 6*e7be843bSPierre Pronchery 7*e7be843bSPierre ProncheryEssential: 8*e7be843bSPierre Pronchery 9*e7be843bSPierre Pronchery- Support server mode. 10*e7be843bSPierre Pronchery- Support multiple clients on one UDP socket/BIO. 11*e7be843bSPierre Pronchery- Accept incoming clients and get a `SSL *` object which works just like a 12*e7be843bSPierre Pronchery client QCSO. 13*e7be843bSPierre Pronchery 14*e7be843bSPierre ProncheryMaybe: 15*e7be843bSPierre Pronchery 16*e7be843bSPierre Pronchery- Binding multiple BIOs (UDP sockets) to one listener? 17*e7be843bSPierre Pronchery 18*e7be843bSPierre ProncheryImplied: 19*e7be843bSPierre Pronchery 20*e7be843bSPierre Pronchery- Support connection migration and preferred addressing in the future. 21*e7be843bSPierre Pronchery- Maximise reuse of existing APIs. 22*e7be843bSPierre Pronchery 23*e7be843bSPierre ProncheryApproach 24*e7be843bSPierre Pronchery-------- 25*e7be843bSPierre Pronchery 26*e7be843bSPierre ProncheryTwo design sketches are shown below, their relative merits are evaluated, and 27*e7be843bSPierre Proncheryone is chosen. 28*e7be843bSPierre Pronchery 29*e7be843bSPierre ProncherySketch A: `SSL_LISTENER` 30*e7be843bSPierre Pronchery------------------------ 31*e7be843bSPierre Pronchery 32*e7be843bSPierre ProncheryA new type `SSL_LISTENER` is introduced to the public API. This represents an 33*e7be843bSPierre Proncheryentity which listens for incoming connections. It is **not** intended to be QUIC 34*e7be843bSPierre Proncheryspecific and provides an abstract interface which could be adapted to other 35*e7be843bSPierre Proncherycases in the future. For example, it could be used for TLS over TCP to make 36*e7be843bSPierre Proncheryaccepting incoming TCP connections easy. 37*e7be843bSPierre Pronchery 38*e7be843bSPierre ProncheryInternally, an `SSL_LISTENER` has a common header in much the same way that 39*e7be843bSPierre Pronchery`SSL` objects now do. For example, if called using a QUIC `SSL_CTX`, the 40*e7be843bSPierre Proncheryinternal structure might be a `QUIC_LISTENER`, etc. If called using a TLS 41*e7be843bSPierre Pronchery`SSL_CTX`, this might be a `TCP_LISTENER` doing something very ordinary. We can 42*e7be843bSPierre Proncherysupport DTLS in the future in the same way as QUIC, etc. 43*e7be843bSPierre Pronchery 44*e7be843bSPierre ProncheryInternally, an `SSL_LISTENER` stores any common state that needs to be held 45*e7be843bSPierre Proncherycollectively for all connections created by that listener. 46*e7be843bSPierre Pronchery 47*e7be843bSPierre Pronchery`SSL_LISTENER` objects are refcounted and referenced by any connection which 48*e7be843bSPierre Proncherydescends from them. Thus, an application may free the `SSL_LISTENER`, but 49*e7be843bSPierre Proncheryit will be retained until it is no longer depended on by any connection. 50*e7be843bSPierre Pronchery 51*e7be843bSPierre ProncheryThe operation of an `SSL_LISTENER` is simple: call 52*e7be843bSPierre Pronchery`SSL_LISTENER_accept_connection`. For QUIC, this returns a new QCSO, which is 53*e7be843bSPierre Proncheryconsidered a child of the listener. 54*e7be843bSPierre Pronchery 55*e7be843bSPierre Pronchery```c 56*e7be843bSPierre ProncherySSL_LISTENER *SSL_LISTENER_new(SSL_CTX *ctx, BIO *net_rbio, BIO *net_wbio); 57*e7be843bSPierre Proncheryint SSL_LISTENER_up_ref(SSL_LISTENER *l); 58*e7be843bSPierre Proncheryint SSL_LISTENER_free(SSL_LISTENER *l); 59*e7be843bSPierre Pronchery 60*e7be843bSPierre ProncheryBIO *SSL_LISTENER_get0_net_rbio(SSL_LISTENER *l); 61*e7be843bSPierre ProncheryBIO *SSL_LISTENER_get0_net_wbio(SSL_LISTENER *l); 62*e7be843bSPierre Proncheryint SSL_LISTENER_set1_net_rbio(SSL_LISTENER *l, BIO *net_rbio); 63*e7be843bSPierre Proncheryint SSL_LISTENER_set1_net_wbio(SSL_LISTENER *l, BIO *net_wbio); 64*e7be843bSPierre Pronchery 65*e7be843bSPierre ProncherySSL *SSL_LISTENER_accept_connection(SSL_LISTENER *l, uint64_t flags); 66*e7be843bSPierre Proncherysize_t SSL_LISTENER_get_accept_queue_len(SSL_LISTENER *l); 67*e7be843bSPierre Pronchery``` 68*e7be843bSPierre Pronchery 69*e7be843bSPierre Pronchery### Discussion 70*e7be843bSPierre Pronchery 71*e7be843bSPierre ProncheryHowever, there are a number of disadvantages to introducing a new object type. 72*e7be843bSPierre ProncheryThe concept of a listener shares many of the same concepts inherent to 73*e7be843bSPierre Proncherysupporting non-blocking network I/O which our SSL API for QUIC connections has 74*e7be843bSPierre Proncherybeen carefully designed to support: 75*e7be843bSPierre Pronchery 76*e7be843bSPierre Pronchery- Poll descriptors (`SSL_get_[rw]poll_descriptor`); 77*e7be843bSPierre Pronchery- `SSL_net_(read|write)_desired`; 78*e7be843bSPierre Pronchery- `SSL_handle_events`; 79*e7be843bSPierre Pronchery- `SSL_get_event_timeout`; 80*e7be843bSPierre Pronchery- BIO configuration (`SSL_set_[rw]bio`); 81*e7be843bSPierre Pronchery- Refcounting (`SSL_up_ref`, `SSL_free`). 82*e7be843bSPierre Pronchery 83*e7be843bSPierre ProncheryWe would need to duplicate all of these APIs. Moreover application developers 84*e7be843bSPierre Proncherywould need to use both and the distinction between the two would become 85*e7be843bSPierre Proncheryconfusing. 86*e7be843bSPierre Pronchery 87*e7be843bSPierre ProncherySketch B: New SSL Object Type: Listener 88*e7be843bSPierre Pronchery--------------------------------------- 89*e7be843bSPierre Pronchery 90*e7be843bSPierre ProncheryInstead of the above, the definition of a new SSL object subtype — a Listener — 91*e7be843bSPierre Proncheryis proposed. A QLSO (QUIC Listener SSL Object) is defined in much the same way 92*e7be843bSPierre Proncherythat QCSOs and QSSOs are currently defined, using the common SSL structure 93*e7be843bSPierre Proncheryheader. 94*e7be843bSPierre Pronchery 95*e7be843bSPierre ProncheryMany existing APIs around the handling of non-blocking I/O can then be supported 96*e7be843bSPierre Proncheryand work in exactly the same way. 97*e7be843bSPierre Pronchery 98*e7be843bSPierre Pronchery```c 99*e7be843bSPierre Pronchery/* 100*e7be843bSPierre Pronchery * SSL_new_listener 101*e7be843bSPierre Pronchery * ---------------- 102*e7be843bSPierre Pronchery * 103*e7be843bSPierre Pronchery * Creates a QLSO, assuming a QUIC SSL_CTX. 104*e7be843bSPierre Pronchery * 105*e7be843bSPierre Pronchery * TCP and DTLS can be supported in the future; not supported for non-QUIC 106*e7be843bSPierre Pronchery * SSL_CTXs for now. 107*e7be843bSPierre Pronchery */ 108*e7be843bSPierre ProncherySSL *SSL_new_listener(SSL_CTX *ctx, uint64_t flags); 109*e7be843bSPierre Pronchery 110*e7be843bSPierre Pronchery/* 111*e7be843bSPierre Pronchery * A QLSO supports the following existing APIs: 112*e7be843bSPierre Pronchery * 113*e7be843bSPierre Pronchery * - Poll descriptors (`SSL_get_[rw]poll_descriptor`) 114*e7be843bSPierre Pronchery * - `SSL_net_(read|write)_desired` 115*e7be843bSPierre Pronchery * - `SSL_handle_events` 116*e7be843bSPierre Pronchery * - `SSL_get_event_timeout` 117*e7be843bSPierre Pronchery * - BIO configuration (`SSL_set_[rw]bio`) 118*e7be843bSPierre Pronchery * - Refcounting (`SSL_up_ref`, `SSL_free`) 119*e7be843bSPierre Pronchery */ 120*e7be843bSPierre Pronchery 121*e7be843bSPierre Pronchery/* 122*e7be843bSPierre Pronchery * New API for QLSOs: SSL_accept_connection 123*e7be843bSPierre Pronchery * ---------------------------------------- 124*e7be843bSPierre Pronchery * 125*e7be843bSPierre Pronchery * These APIs are exactly the same as SSL_accept_stream, but made a separate 126*e7be843bSPierre Pronchery * function to avoid confusion. Supported on QLSOs only. 127*e7be843bSPierre Pronchery */ 128*e7be843bSPierre ProncherySSL *SSL_accept_connection(SSL *ssl, uint64_t flags); 129*e7be843bSPierre Proncherysize_t SSL_get_accept_connection_queue_len(SSL *ssl); 130*e7be843bSPierre Pronchery 131*e7be843bSPierre Pronchery/* 132*e7be843bSPierre Pronchery * New API for QLSOs: SSL_listen 133*e7be843bSPierre Pronchery * ----------------------------- 134*e7be843bSPierre Pronchery * 135*e7be843bSPierre Pronchery * This is not usually needed as SSL_accept_connection will implicitly trigger 136*e7be843bSPierre Pronchery * this if it has not been called yet. It is only needed if a server wishes to 137*e7be843bSPierre Pronchery * ensure it has started to accept incoming connections but does not wish to 138*e7be843bSPierre Pronchery * actually call SSL_accept_connection yet. In this regard it is similar to 139*e7be843bSPierre Pronchery * listen(2). 140*e7be843bSPierre Pronchery */ 141*e7be843bSPierre Proncheryint SSL_listen(SSL *ssl); 142*e7be843bSPierre Pronchery 143*e7be843bSPierre Pronchery/* 144*e7be843bSPierre Pronchery * New API for QCSOs: SSL_get_peer_addr 145*e7be843bSPierre Pronchery * ------------------------------------ 146*e7be843bSPierre Pronchery * 147*e7be843bSPierre Pronchery * An API to retrieve a L4 peer address. Supported on QCSOs only. 148*e7be843bSPierre Pronchery */ 149*e7be843bSPierre Proncheryint SSL_get_peer_addr(SSL *ssl, BIO_ADDR *peer_addr); 150*e7be843bSPierre Pronchery 151*e7be843bSPierre Pronchery/* SSL_get_local_addr may be introduced once it is relevant (migration etc.) */ 152*e7be843bSPierre Pronchery 153*e7be843bSPierre Pronchery/* Returns 1 for QLSOs (or future TLS/DTLS listeners) only. */ 154*e7be843bSPierre Proncheryint SSL_is_listener(SSL *ssl); 155*e7be843bSPierre Pronchery 156*e7be843bSPierre Pronchery/* Returns the QLSO (if any) from which a QCSO or QSSO is descended. */ 157*e7be843bSPierre ProncherySSL *SSL_get0_listener(SSL *ssl); 158*e7be843bSPierre Pronchery 159*e7be843bSPierre Pronchery/* SSL_is_quic returns 1 */ 160*e7be843bSPierre Pronchery``` 161*e7be843bSPierre Pronchery 162*e7be843bSPierre ProncherySelection of Design 163*e7be843bSPierre Pronchery------------------- 164*e7be843bSPierre Pronchery 165*e7be843bSPierre ProncheryWe could also consider using an `SSL_CTX` as the listener type. Since a 166*e7be843bSPierre Pronchery`SSL_CTX` is intended to span the lifespan of multiple TLS connections there is 167*e7be843bSPierre Proncherya degree of logic here. However, this does not match up naturally because an 168*e7be843bSPierre Pronchery`SSL_CTX` is not designed to hold network resources or manage network I/O. It 169*e7be843bSPierre Proncherydoes not have the facilities for polling and event handling that the `SSL` 170*e7be843bSPierre Proncheryobject API has been enhanced with. It would also be a fairly major paradigm 171*e7be843bSPierre Proncheryshift for existing server applications and make things difficult if a server 172*e7be843bSPierre Proncherywanted to use different `SSL_CTX` instances to use different e.g. TLS settings 173*e7be843bSPierre Proncherywithin the same listener. 174*e7be843bSPierre Pronchery 175*e7be843bSPierre ProncheryBased on the merits of the above, Sketch B is chosen for adoption. 176*e7be843bSPierre Pronchery 177*e7be843bSPierre ProncheryDiscussion of Relevant Issues 178*e7be843bSPierre Pronchery----------------------------- 179*e7be843bSPierre Pronchery 180*e7be843bSPierre Pronchery### Notes on Internal Refactoring to Support Multiple Connections 181*e7be843bSPierre Pronchery 182*e7be843bSPierre ProncheryCurrently a client connection is a QCSO 183*e7be843bSPierre Pronchery(`QUIC_CONNECTION`) which contains a `QUIC_CHANNEL`. The `QUIC_CONNECTION` 184*e7be843bSPierre Proncheryprovides API Personality Layer (APL) implementation and wraps the APL-agnostic 185*e7be843bSPierre Pronchery`QUIC_CHANNEL`. We currently have two APLs, the libssl API (“the” APL) and 186*e7be843bSPierre Pronchery`QUIC_TSERVER` for internal testing use. `QUIC_XSO` is the APL object 187*e7be843bSPierre Proncheryrepresenting a QUIC stream and wraps our APL-agnostic channel-owned 188*e7be843bSPierre Pronchery`QUIC_STREAM` object. There is no specific object in the `QUIC_TSERVER` APL for 189*e7be843bSPierre Proncherya stream as they are referenced in that API by stream ID only. 190*e7be843bSPierre Pronchery 191*e7be843bSPierre Pronchery`QUIC_CHANNEL` corresponds exactly to a QUIC connection. In a similar manner, 192*e7be843bSPierre Pronchery`QUIC_STREAM` corresponds exactly to a QUIC stream. In server operation, 193*e7be843bSPierre Proncherymultiple `QUIC_CHANNEL` instances will exist, one per accepted connection. 194*e7be843bSPierre Pronchery 195*e7be843bSPierre ProncheryAbove we introduce the notion of a QUIC Listener SSL Object (QLSO), which 196*e7be843bSPierre Proncheryinternally will be represented by the type `QUIC_LISTENER`. This forms a new 197*e7be843bSPierre Proncheryobject in the libssl APL. 198*e7be843bSPierre Pronchery 199*e7be843bSPierre ProncheryWith server operation, multiple connections will share a single UDP 200*e7be843bSPierre Proncherysocket/network BIO. Certain resources currently tied 1:1 into a `QUIC_CHANNEL` 201*e7be843bSPierre Proncherywill be broken out, such as the network BIO pointers and `QUIC_DEMUX` instance. 202*e7be843bSPierre ProncheryA `QUIC_CHANNEL` continues to own e.g. the QRX/QTX as these are 203*e7be843bSPierre Proncheryconnection-specific. 204*e7be843bSPierre Pronchery 205*e7be843bSPierre ProncheryBecause preserving the separation of the APL(s) and the QUIC core is desirable, 206*e7be843bSPierre Proncherythe concept of a `QUIC_PORT` is introduced. This is the internal object tracking 207*e7be843bSPierre Proncherya demux and network BIOs which a `QUIC_CHANNEL` will belong to. In other words, 208*e7be843bSPierre Proncherythe new situation is as follows: 209*e7be843bSPierre Pronchery 210*e7be843bSPierre Pronchery| Abbr. | libssl APL Object | QUIC Core Object | 211*e7be843bSPierre Pronchery|-------|-------------------|------------------| 212*e7be843bSPierre Pronchery| QLSO | `QUIC_LISTENER` | `QUIC_PORT` | 213*e7be843bSPierre Pronchery| QCSO | `QUIC_CONNECTION` | `QUIC_CHANNEL` | 214*e7be843bSPierre Pronchery| QSSO | `QUIC_XSO` | `QUIC_STREAM` | 215*e7be843bSPierre Pronchery 216*e7be843bSPierre Pronchery### Listener Flexibility 217*e7be843bSPierre Pronchery 218*e7be843bSPierre ProncheryA listener allows multiple connections to be received on a UDP socket. However, 219*e7be843bSPierre Proncherythere is no intrinsic requirement in QUIC that a socket only be used for client 220*e7be843bSPierre Proncheryuse, or only for server use. Therefore, it should be possible for an application 221*e7be843bSPierre Proncheryto create a QLSO, use it to accept connections, and also use it to initiate 222*e7be843bSPierre Proncheryoutgoing connections. 223*e7be843bSPierre Pronchery 224*e7be843bSPierre ProncheryThis can be supported in a simple manner: 225*e7be843bSPierre Pronchery 226*e7be843bSPierre Pronchery```c 227*e7be843bSPierre ProncherySSL *SSL_new_from_listener(SSL *ssl); 228*e7be843bSPierre Pronchery``` 229*e7be843bSPierre Pronchery 230*e7be843bSPierre ProncheryThis creates an unconnected (idle) QCSO from a QLSO. Use of 231*e7be843bSPierre Pronchery`SSL_set1_initial_peer_addr` is mandatory in this case as peer address 232*e7be843bSPierre Proncheryautodetection is inhibited. 233*e7be843bSPierre Pronchery 234*e7be843bSPierre ProncheryThe created QCSO otherwise works in a completely normal fashion. 235*e7be843bSPierre Pronchery 236*e7be843bSPierre Pronchery### Client-Only Listeners 237*e7be843bSPierre Pronchery 238*e7be843bSPierre ProncheryIn fact, the realisation of this hybrid capability is simply an emergent 239*e7be843bSPierre Proncheryconsequence of our internal refactor. Since a QUIC server will be split into an 240*e7be843bSPierre Proncheryinternal `QUIC_PORT` and zero or more `QUIC_CONNECTION` instances, so too will a 241*e7be843bSPierre Proncheryclient connection become simply a degenerate case where there is a `QUIC_PORT` 242*e7be843bSPierre Proncheryand only one `QUIC_CONNECTION`. So the client case basically just becomes a 243*e7be843bSPierre Proncherysimplified API where we spin up a `QUIC_PORT`/listener automatically to support 244*e7be843bSPierre Proncherya single connection. 245*e7be843bSPierre Pronchery 246*e7be843bSPierre ProncheryIt is potentially desirable for a client to be able to use the listener 247*e7be843bSPierre ProncheryAPI even if it is only initiating connections as a client and never accepting 248*e7be843bSPierre Proncheryconnections. There are several reasons why an application might want to do this: 249*e7be843bSPierre Pronchery 250*e7be843bSPierre Pronchery- it allows the application to centralise its I/O polling and networking arrangements; 251*e7be843bSPierre Pronchery- it allows the application to handle the TERMINATING phase of connections in a 252*e7be843bSPierre Pronchery better (and more RFC-compliant) way. 253*e7be843bSPierre Pronchery 254*e7be843bSPierre ProncheryFor the latter reason, consider how connection termination works. Once a QUIC 255*e7be843bSPierre ProncheryImmediate Close is initiated using a `CONNECTION_CLOSE` frame, a connection 256*e7be843bSPierre Proncheryenters the `TERMINATING` state and is expected to stay there for some period of 257*e7be843bSPierre Proncherytime until the possibility of various contingencies (such as loss of a 258*e7be843bSPierre Pronchery`CONNECTION_CLOSE` frame and its necessary retransmission) has been accounted 259*e7be843bSPierre Proncheryfor. However, many applications may not wish to hang around for potentially 260*e7be843bSPierre Proncheryseveral seconds or longer while tearing down a QUIC connection. 261*e7be843bSPierre Pronchery 262*e7be843bSPierre ProncheryAllowing clients to use a listener resolves this problem, if the lifetime of the 263*e7be843bSPierre Proncheryapplication exceeds the lifetime of most client connections. The listener can 264*e7be843bSPierre Proncherycontinue to hold the `QUIC_PORT` internally and respond to enquiries for 265*e7be843bSPierre Proncheryterminating and terminated connections, even if (as far as the application is 266*e7be843bSPierre Proncheryconcerned) no client connections currently exist and all prior client 267*e7be843bSPierre Proncheryconnections have been freed using `SSL_free`. The internal `QUIC_CHANNEL` can be 268*e7be843bSPierre Proncherykept around by the `QUIC_PORT` until it reaches the `TERMINATED` state. 269*e7be843bSPierre Pronchery 270*e7be843bSPierre ProncheryA client-only listener can be created one of two ways: 271*e7be843bSPierre Pronchery 272*e7be843bSPierre Pronchery1. by calling `SSL_new_listener` while passing this flag: 273*e7be843bSPierre Pronchery 274*e7be843bSPierre Pronchery ``` 275*e7be843bSPierre Pronchery #define SSL_LISTENER_FLAG_NO_ACCEPT (1U << 0) 276*e7be843bSPierre Pronchery ``` 277*e7be843bSPierre Pronchery 278*e7be843bSPierre Pronchery2. by using a client `SSL_METHOD`, which implies `NO_ACCEPT`. 279*e7be843bSPierre Pronchery 280*e7be843bSPierre ProncheryThis flag simply indicates that this listener is not to be used to accept 281*e7be843bSPierre Proncheryincoming connections. A client can then use `SSL_new_from_listener` as usual to 282*e7be843bSPierre Proncherycreate an outgoing QCSO. 283*e7be843bSPierre Pronchery 284*e7be843bSPierre Pronchery### CID Length Determination 285*e7be843bSPierre Pronchery 286*e7be843bSPierre ProncheryCurrently a client connection uses a zero-length local CID. Since we want to 287*e7be843bSPierre Proncherysupport multiple outgoing client connections on one socket these will need to 288*e7be843bSPierre Proncherysupport a non-zero-length local CID. Equally servers will need to support CIDs. 289*e7be843bSPierre Pronchery 290*e7be843bSPierre ProncheryBy default when a QLSO is used, we will use a real, non-zero-length CID. If the 291*e7be843bSPierre Proncheryexisting QUIC client API is used (with no QLSO) we will likely spin up a 292*e7be843bSPierre Pronchery`QUIC_PORT` internally but specially choose to use a zero-length local CID since 293*e7be843bSPierre Proncherywe know this port will never be used for more than one connection. 294*e7be843bSPierre Pronchery 295*e7be843bSPierre ProncheryIf an application creates a QLSO explicitly but knows it will not need to 296*e7be843bSPierre Proncherycreate more than one connection, it can pass the following flag when creating 297*e7be843bSPierre Proncherythe listener: 298*e7be843bSPierre Pronchery 299*e7be843bSPierre Pronchery```c 300*e7be843bSPierre Pronchery#define SSL_LISTENER_FLAG_SINGLE_CONN (1U << 1) 301*e7be843bSPierre Pronchery``` 302*e7be843bSPierre Pronchery 303*e7be843bSPierre ProncheryFailure to do so is nonfatal and simply increases packet overheads. 304*e7be843bSPierre Pronchery 305*e7be843bSPierre Pronchery### Future Compatibility: TLS over TCP Support 306*e7be843bSPierre Pronchery 307*e7be843bSPierre ProncheryAssuming we choose to use the SSL object type as the basis of a new listener 308*e7be843bSPierre Proncheryobject type, let us consider how this API could also be used to support TLS over 309*e7be843bSPierre ProncheryTCP, at least in a server role: 310*e7be843bSPierre Pronchery 311*e7be843bSPierre Pronchery```c 312*e7be843bSPierre Pronchery/* Blocking example: */ 313*e7be843bSPierre Pronchery{ 314*e7be843bSPierre Pronchery SSL *l, *conn; 315*e7be843bSPierre Pronchery BIO *tcp_listener = get_bio_sock_for_listening_tcp_socket(); 316*e7be843bSPierre Pronchery SSL_CTX *ctx = create_ssl_ctx(TLS_server_method()); 317*e7be843bSPierre Pronchery BIO_ADDR addr; 318*e7be843bSPierre Pronchery 319*e7be843bSPierre Pronchery l = SSL_new_listener(ctx); 320*e7be843bSPierre Pronchery SSL_set_bio(l, tcp_listener, tcp_listener); 321*e7be843bSPierre Pronchery 322*e7be843bSPierre Pronchery for (;;) { 323*e7be843bSPierre Pronchery /* standard SSL_CONNECTION is returned */ 324*e7be843bSPierre Pronchery conn = SSL_accept_connection(l, 0); 325*e7be843bSPierre Pronchery 326*e7be843bSPierre Pronchery spawn_thread_to_process_conn(conn, &addr); 327*e7be843bSPierre Pronchery } 328*e7be843bSPierre Pronchery} 329*e7be843bSPierre Pronchery``` 330*e7be843bSPierre Pronchery 331*e7be843bSPierre ProncheryEssentially, when `SSL_new_listener` is given a `SSL_CTX` which is using a TLS 332*e7be843bSPierre Proncherymethod, it returns some different kind of object, say a `TCP_LISTENER`, which 333*e7be843bSPierre Proncheryalso is a kind of SSL object. `SSL_accept_connection` calls `accept(2)` via some 334*e7be843bSPierre ProncheryBIO abstraction on the provided BIO, and constructs a new BIO on the resulting 335*e7be843bSPierre Proncherysocket, wrapping it via standard `SSL_new` using the same `SSL_CTX`. 336*e7be843bSPierre Pronchery 337*e7be843bSPierre ProncheryThus, this API should be compatible with any future adaptation to also support 338*e7be843bSPierre ProncheryTCP. 339*e7be843bSPierre Pronchery 340*e7be843bSPierre Pronchery### Future Compatibility: DTLS over UDP Support 341*e7be843bSPierre Pronchery 342*e7be843bSPierre ProncheryThe relevant concerns regarding DTLS are likely to be highly similar to QUIC, 343*e7be843bSPierre Proncheryso this API should also be adaptable to support DTLS in the future, providing 344*e7be843bSPierre Proncherya unified server-side API. `SSL_new_listener` will return e.g. a 345*e7be843bSPierre Pronchery`DTLS_LISTENER`. 346*e7be843bSPierre Pronchery 347*e7be843bSPierre Pronchery### Supporting Multiple Sockets 348*e7be843bSPierre Pronchery 349*e7be843bSPierre ProncheryThere are three means that we can provide to support a server which wants to 350*e7be843bSPierre Proncherylisten on multiple UDP sockets: 351*e7be843bSPierre Pronchery 352*e7be843bSPierre Pronchery1. Allow a QLSO to bind multiple BIOs; 353*e7be843bSPierre Pronchery2. Create multiple QLSOs and use some polling mechanism to service all of them; 354*e7be843bSPierre Pronchery3. Create some higher level object which can aggregate multiple QLSOs. 355*e7be843bSPierre Pronchery 356*e7be843bSPierre Pronchery(1) would require more extensive API changes and should only be done if a 357*e7be843bSPierre Proncherycompelling reason to do so arises. (3) is a more advanced design which is 358*e7be843bSPierre Proncherytheoretically possible in future and is discussed in the next section. 359*e7be843bSPierre ProncheryFor now, we will support only (2) (which we can support via our planned polling 360*e7be843bSPierre ProncheryAPI) until a compelling need arises. 361*e7be843bSPierre Pronchery 362*e7be843bSPierre Pronchery### Roles and Domains: Concepts and Future Evolution 363*e7be843bSPierre Pronchery 364*e7be843bSPierre ProncheryA client can use QUIC using a single QCSO without ever creating a QLSO. 365*e7be843bSPierre ProncheryConversely a client or server can use QUIC by creating a QLSO and then one or 366*e7be843bSPierre Proncherymore subsidiary QCSOs. This implies that, at least in terms of the publicly 367*e7be843bSPierre Proncheryvisible API, the roles of certain essential aspects of QUIC can belong to 368*e7be843bSPierre Proncherydifferent publicly visible API objects. 369*e7be843bSPierre Pronchery 370*e7be843bSPierre ProncheryThe following concepts are introduced: 371*e7be843bSPierre Pronchery 372*e7be843bSPierre Pronchery- **Port Leader:** A Port Leader is the APL object which caused the creation 373*e7be843bSPierre Pronchery of a `QUIC_PORT`, which is responsible for servicing a particular pair 374*e7be843bSPierre Pronchery of network BIOs. Currently maps to one UDP socket. 375*e7be843bSPierre Pronchery 376*e7be843bSPierre Pronchery In the simple client API, the QCSO is the port leader. 377*e7be843bSPierre Pronchery In a client using a QLSO, the QLSO is the port leader. 378*e7be843bSPierre Pronchery In a server, the QLSO is the port leader. 379*e7be843bSPierre Pronchery 380*e7be843bSPierre Pronchery- **Event Leader:** An Event Leader is the APL object which handles event 381*e7be843bSPierre Pronchery processing. Calls to SSL_handle_events() for any subsidiary object is 382*e7be843bSPierre Pronchery equivalent to a call to SSL_handle_events() on the Event Leader, which handles 383*e7be843bSPierre Pronchery state machine processing for that object and all subsidiary objects. 384*e7be843bSPierre Pronchery 385*e7be843bSPierre Pronchery The event leader is the highest-level APL object in a QUIC APL object 386*e7be843bSPierre Pronchery hierarchy: 387*e7be843bSPierre Pronchery 388*e7be843bSPierre Pronchery In the simple client API, the QCSO is the event leader. 389*e7be843bSPierre Pronchery In a client using a QLSO, the QLSO is the event leader. 390*e7be843bSPierre Pronchery In a server, the QLSO is the event leader. 391*e7be843bSPierre Pronchery 392*e7be843bSPierre Pronchery- **Domain:** A QUIC domain is a set of APL objects rooted in an Event Leader. 393*e7be843bSPierre Pronchery In other words, requests to handle events on subsidiary objects in a domain 394*e7be843bSPierre Pronchery are channeled up to the Event Leader and processed there. A QUIC domain is a 395*e7be843bSPierre Pronchery set of logically grouped QUIC objects which share a single I/O reactor and 396*e7be843bSPierre Pronchery event processing context. 397*e7be843bSPierre Pronchery 398*e7be843bSPierre Pronchery A domain could contain multiple QLSOs (i.e., multiple UDP sockets) in future. 399*e7be843bSPierre Pronchery In this case QLSOs may cease to be the most senior APL object type and there 400*e7be843bSPierre Pronchery may be some new hypothetical SSL_new_quic_domain() call. 401*e7be843bSPierre Pronchery 402*e7be843bSPierre Pronchery However, there is no plan to implement domains at this time; for the time 403*e7be843bSPierre Pronchery being, this is purely conceptual and ensures our architecture can grow to 404*e7be843bSPierre Pronchery supporting multiple sockets and listeners in a given QUIC context if need be. 405*e7be843bSPierre Pronchery 406*e7be843bSPierre Pronchery If ever implemented, a domain will be reified by a `QUIC_DOMAIN` structure as 407*e7be843bSPierre Pronchery part of the APL and a `QUIC_ENGINE` structure as part of the QUIC core. The 408*e7be843bSPierre Pronchery SSL object type will be abbreviated QDSO (QUIC Domain SSL Object). 409*e7be843bSPierre Pronchery 410*e7be843bSPierre ProncheryThe introduction of the concept of a domain demonstrates the value of the Port 411*e7be843bSPierre ProncheryLeader and Domain Leader concepts going forward: 412*e7be843bSPierre Pronchery 413*e7be843bSPierre ProncheryExample 1 — Explicitly Created Domain: 414*e7be843bSPierre Pronchery 415*e7be843bSPierre Pronchery```text 416*e7be843bSPierre ProncheryQDSO (QUIC_DOMAIN/QUIC_ENGINE) Event Leader 417*e7be843bSPierre Pronchery QLSO (QUIC_LISTENER/QUIC_PORT) Port Leader 418*e7be843bSPierre Pronchery QCSO (QUIC_CONNECTION/QUIC_CHANNEL) 419*e7be843bSPierre Pronchery QSSO (QUIC_XSO/QUIC_STREAM) 420*e7be843bSPierre Pronchery``` 421*e7be843bSPierre Pronchery 422*e7be843bSPierre ProncheryExample 2 - Explicitly Created QLSO, Implicitly Created Domain: 423*e7be843bSPierre Pronchery 424*e7be843bSPierre Pronchery```text 425*e7be843bSPierre ProncheryQLSO (QUIC_LISTENER/QUIC_PORT) Event Leader, Port Leader 426*e7be843bSPierre Pronchery [QUIC_ENGINE] 427*e7be843bSPierre Pronchery QCSO (QUIC_CONNECTION/QUIC_CHANNEL) 428*e7be843bSPierre Pronchery QSSO (QUIC_XSO/QUIC_STREAM) 429*e7be843bSPierre Pronchery``` 430*e7be843bSPierre Pronchery 431*e7be843bSPierre ProncheryHere we see that the QLSO now has the Event Leader role, as it is the most 432*e7be843bSPierre Proncherysenior object visible publicly as a constructed API object. Note that a 433*e7be843bSPierre ProncheryQUIC_ENGINE will be created internally to allow the QLSO to service the Event 434*e7be843bSPierre ProncheryLeader role, but the corresponding QUIC_DOMAIN APL object will not exist, as 435*e7be843bSPierre Proncherythis is not API-visible. 436*e7be843bSPierre Pronchery 437*e7be843bSPierre ProncheryExample 3 - Explicitly Created Single Client QCSO: 438*e7be843bSPierre Pronchery 439*e7be843bSPierre Pronchery```text 440*e7be843bSPierre ProncheryQCSO (QUIC_CONNECTION/QUIC_CHANNEL) Event Leader, Port Leader 441*e7be843bSPierre Pronchery [QUIC_ENGINE] 442*e7be843bSPierre Pronchery [QUIC_PORT] 443*e7be843bSPierre Pronchery QSSO (QUIC_XSO/QUIC_STREAM) 444*e7be843bSPierre Pronchery``` 445*e7be843bSPierre Pronchery 446*e7be843bSPierre ProncheryHere we see that the QCSO now has the Port Leader role. Note that a QUIC_PORT 447*e7be843bSPierre Proncherywill be created internally to allow the QCSO to service the Port Leader role, 448*e7be843bSPierre Proncherybut the corresponding QUIC_LISTENER APL object will not exist, as this is not 449*e7be843bSPierre ProncheryAPI-visible. Similarly a QUIC_ENGINE is spun up to handle the Event Leader role, 450*e7be843bSPierre Proncheryas above. 451*e7be843bSPierre Pronchery 452*e7be843bSPierre ProncheryIt should be emphasised that everything in this section is largely academic and 453*e7be843bSPierre Proncheryintended to demonstrate potential future directions and the adaptability of the 454*e7be843bSPierre Proncherydesign to future evolution. There is no immediate plan to implement a QDSO 455*e7be843bSPierre Proncheryobject type in the APL. However it will be viable to do so if we want to support 456*e7be843bSPierre Proncherymultiple sockets per domain someday. 457*e7be843bSPierre Pronchery 458*e7be843bSPierre Pronchery| Abbr. | libssl APL Object | QUIC Core Object | Seniority | Potential Roles | 459*e7be843bSPierre Pronchery|-------|-------------------|------------------|-------------|-----------------| 460*e7be843bSPierre Pronchery| QDSO | `QUIC_DOMAIN` | `QUIC_ENGINE` | 1 (Highest) | EL | 461*e7be843bSPierre Pronchery| QLSO | `QUIC_LISTENER` | `QUIC_PORT` | 2 | EL PL | 462*e7be843bSPierre Pronchery| QCSO | `QUIC_CONNECTION` | `QUIC_CHANNEL` | 3 | EL PL | 463*e7be843bSPierre Pronchery| QSSO | `QUIC_XSO` | `QUIC_STREAM` | 4 | | 464*e7be843bSPierre Pronchery 465*e7be843bSPierre Pronchery### Polling 466*e7be843bSPierre Pronchery 467*e7be843bSPierre ProncheryWe will support polling all QUIC object types (QLSOs, QCSOs, QSSOs) using a new 468*e7be843bSPierre Proncherypolling API. This is the subject of its own design and will be discussed in more 469*e7be843bSPierre Proncherydetail in a separate design document. 470*e7be843bSPierre Pronchery 471*e7be843bSPierre Pronchery### Impact of `SSL_METHOD` selection 472*e7be843bSPierre Pronchery 473*e7be843bSPierre ProncheryThe interaction of our various QCSO and QLSO construction APIs in the APL, 474*e7be843bSPierre Proncheryand the choice of the `SSL_METHOD` for a `SSL_CTX`, must also be considered. 475*e7be843bSPierre ProncheryThe following interactions are chosen: 476*e7be843bSPierre Pronchery 477*e7be843bSPierre Pronchery```text 478*e7be843bSPierre ProncherySSL_new(QUIC_client_method) → QCSO(Client) 479*e7be843bSPierre ProncherySSL_new(QUIC_client_thread_method) → QCSO(Client) 480*e7be843bSPierre ProncherySSL_new(QUIC_server_method) → (not supported; error) 481*e7be843bSPierre Pronchery 482*e7be843bSPierre ProncherySSL_new_listener(QUIC_client_method) → QLSO(Client) (implicit NO_ACCEPT) 483*e7be843bSPierre ProncherySSL_new_listener(QUIC_client_thread_method) → QLSO(Client) (implicit NO_ACCEPT) 484*e7be843bSPierre ProncherySSL_new_listener(QUIC_server_method) → QLSO(Server + Client) 485*e7be843bSPierre Pronchery 486*e7be843bSPierre ProncherySSL_new_from_listener(QUIC_client_method) → QCSO (Client) 487*e7be843bSPierre ProncherySSL_new_from_listener(QUIC_client_thread_method) → QCSO (Client) 488*e7be843bSPierre ProncherySSL_new_from_listener(QUIC_server_method) → QCSO (Client) 489*e7be843bSPierre Pronchery``` 490*e7be843bSPierre Pronchery 491*e7be843bSPierre ProncheryObservations: 492*e7be843bSPierre Pronchery 493*e7be843bSPierre Pronchery- It makes no sense to try and make a single QCSO proactively as a server, 494*e7be843bSPierre Pronchery so this just results in an error. 495*e7be843bSPierre Pronchery 496*e7be843bSPierre Pronchery- A server QLSO can also make outgoing client connections. 497*e7be843bSPierre Pronchery 498*e7be843bSPierre Pronchery- By contrast, if a client `SSL_METHOD` is specified when creating a listener, 499*e7be843bSPierre Pronchery it is assumed to be uninterested in accepting connections and this implies the 500*e7be843bSPierre Pronchery `NO_ACCEPT` flag. In this regard the functionality of a server is a superset 501*e7be843bSPierre Pronchery of the functionality of a client. 502*e7be843bSPierre Pronchery 503*e7be843bSPierre Pronchery- `SSL_new_from_listener` always creates a client QCSO. 504*e7be843bSPierre Pronchery 505*e7be843bSPierre Pronchery- `SINGLE_CONN` is implied automatically only when calling `SSL_new` and not 506*e7be843bSPierre Pronchery using a QLSO. 507*e7be843bSPierre Pronchery 508*e7be843bSPierre ProncheryUsage Example 509*e7be843bSPierre Pronchery------------- 510*e7be843bSPierre Pronchery 511*e7be843bSPierre ProncheryThe following is a blocking example, where `SSL_accept_connection` waits until a 512*e7be843bSPierre Proncherynew connection is available. 513*e7be843bSPierre Pronchery 514*e7be843bSPierre Pronchery```c 515*e7be843bSPierre Pronchery{ 516*e7be843bSPierre Pronchery SSL *l, *conn; 517*e7be843bSPierre Pronchery SSL_CTX *ctx = create_ssl_ctx(QUIC_server_method()); 518*e7be843bSPierre Pronchery BIO *dgram_bio = get_dgram_bio(); 519*e7be843bSPierre Pronchery 520*e7be843bSPierre Pronchery l = SSL_new_listener(ctx); 521*e7be843bSPierre Pronchery SSL_set_bio(l, dgram_bio, dgram_bio); 522*e7be843bSPierre Pronchery 523*e7be843bSPierre Pronchery for (;;) { 524*e7be843bSPierre Pronchery /* automatically starts and calls SSL_listen on first call */ 525*e7be843bSPierre Pronchery conn = SSL_accept_connection(l, 0); /* standard QCSO is returned */ 526*e7be843bSPierre Pronchery 527*e7be843bSPierre Pronchery spawn_thread_to_process_conn(conn); 528*e7be843bSPierre Pronchery } 529*e7be843bSPierre Pronchery} 530*e7be843bSPierre Pronchery``` 531*e7be843bSPierre Pronchery 532*e7be843bSPierre ProncheryIn non-blocking mode, `SSL_accept_connection` will simply act in a non-blocking 533*e7be843bSPierre Proncheryfashion and return immediately with NULL if no new incoming connection is 534*e7be843bSPierre Proncheryavailable, just like `SSL_accept_stream`. The details of how polling and waiting 535*e7be843bSPierre Proncheryfor incoming stream events, and other events, will work will be discussed in the 536*e7be843bSPierre Proncherysubsequent polling design document. 537