Lines Matching +full:a +full:- +full:side
5 RTRS (RDMA Transport) is a reliable high speed transport library
11 possibility to either write data from an sg list to the remote side
12 or to request ("read") data transfer from the remote side into a given
15 RTRS provides I/O fail-over and load-balancing capabilities by using
17 Documentation/ABI/testing/sysfs-class-rtrs-client).
26 --------
27 An established connection between a client and a server is called rtrs
28 session. A session is associated with a set of memory chunks reserved on the
29 server side for a given client for rdma transfer. A session
30 consists of multiple paths, each representing a separate physical link
36 chunks reserved for him on the server side. Their number, size and addresses
49 discussed in LPC RDMA MC 2019. When always_invalidate=Y, on the server side we
51 then pass it to the block layer. A new rkey is generated and registered for the
55 registration on each IO causes performance drop of up to 20%. A user of the
57 (always_invalidate=N), if he understands and can take the risk of a malicious
58 client being able to corrupt memory of a server it is connected to. This might
59 be a reasonable option in a scenario where all the clients and all the servers
60 are located within a secure datacenter.
64 ------------------------
66 1. Client starts establishing connections belonging to a path of a session one
69 established. They are used by the server to find a persisting session/path or
70 to create a new one when necessary. The message also contains the protocol
74 client is trying to reconnect a path, while server is still destroying the old
84 3. After all connections of a path are established client sends to server the
93 (i.e. steps 1-4 finished for all paths requested for a session)
96 messages with an immediate field) which are used to detect a crash on remote
97 side or network outage in an absence of IO.
99 7. On any RDMA related error or in the case of a heartbeat timeout, the
100 corresponding path is disconnected, all the inflight IO are failed over to a
104 *for each connection belonging to a path and for each path:
105 RTRS_MSG_CON_REQ ------------------->
106 <------------------- RTRS_MSG_CON_RSP
109 RTRS_MSG_INFO_REQ ------------------->
110 <------------------- RTRS_MSG_INFO_RSP
112 -------------------> [RTRS_HB_MSG_IMM]
113 [RTRS_HB_MSG_ACK] <-------------------
114 [RTRS_HB_MSG_IMM] <-------------------
115 -------------------> [RTRS_HB_MSG_ACK]
118 -------
122 1. When processing a write request client selects one of the memory chunks
123 on the server side and rdma writes there the user data, user header and the
129 2. When confirming a write request server sends an "empty" rdma message with
134 usr_data + usr_hdr + rtrs_msg_rdma_write -----------------> [RTRS_IO_REQ_IMM]
135 [RTRS_IO_RSP_IMM] <----------------- (id + errno)
139 1. When processing a write request client selects one of the memory chunks
140 on the server side and rdma writes there the user data, user header and the
147 2. When confirming a write request server sends an "empty" rdma message with
155 usr_data + usr_hdr + rtrs_msg_rdma_write -----------------> [RTRS_IO_REQ_IMM]
156 [RTRS_MSG_RKEY_RSP] <----------------- (RTRS_MSG_RKEY_RSP)
157 [RTRS_IO_RSP_IMM] <----------------- (id + errno)
162 1. When processing a read request client selects one of the memory chunks
163 on the server side and rdma writes there the user header and the
168 2. When confirming a read request server transfers the requested data first,
174 usr_hdr + rtrs_msg_rdma_read --------------> [RTRS_IO_REQ_IMM]
175 [RTRS_IO_RSP_IMM] <-------------- usr_data + (id + errno)
177 [RTRS_IO_RSP_IMM_W_INV] <-------------- usr_data + (INV) + (id + errno)
181 1. When processing a read request client selects one of the memory chunks
182 on the server side and rdma writes there the user header and the
189 2. When confirming a read request server transfers the requested data first,
198 usr_hdr + rtrs_msg_rdma_read --------------> [RTRS_IO_REQ_IMM]
199 [RTRS_IO_RSP_IMM] <-------------- usr_data + (id + errno)
200 [RTRS_MSG_RKEY_RSP] <----------------- (RTRS_MSG_RKEY_RSP)
202 [RTRS_IO_RSP_IMM_W_INV] <-------------- usr_data + (INV) + (id + errno)