1This document describes the multiplexing protocol used by ssh(1)'s 2ControlMaster connection-sharing. 3 4Multiplexing starts with a ssh(1) configured to act as a multiplexing 5master. This will cause ssh(1) to listen on a Unix domain socket for 6requests from clients. Clients communicate over this socket using a 7simple packetised protocol, where each message is proceeded with 8a length and message type in SSH uint32 wire format: 9 10 uint32 packet length 11 uint32 packet type 12 ... packet body 13 14Most messages from the client to the server contain a "request id" 15field. This field is returned in replies as "client request id" to 16facilitate matching of responses to requests. 17 18Many multiplexing (mux) client requests yield immediate responses from 19the mux process; requesting a forwarding, performing an alive check or 20requesting the master terminate itself fall in to this category. 21 22The most common use of multiplexing however is to maintain multiple 23concurrent sessions. These are supported via two separate modes: 24 25"Passenger" clients start by requesting a new session with a 26MUX_C_NEW_SESSION message and passing stdio file descriptors over the 27Unix domain control socket. The passenger client then waits until it is 28signaled or the mux server closes the session. This mode is so named as 29the client waits around while the mux server does all the driving. 30 31Stdio forwarding (requested using MUX_C_NEW_STDIO_FWD) is another 32example of passenger mode; the client passes the stdio file descriptors 33and passively waits for something to happen. 34 35"Proxy" clients, requested using MUX_C_PROXY, work quite differently. In 36this mode, the mux client/server connection socket will stop speaking 37the multiplexing protocol and start proxying SSH connection protocol 38messages between the client and server. The client therefore must 39speak a significant subset of the SSH protocol, but in return is able 40to access basically the full suite of connection protocol features. 41Moreover, as no file descriptor passing is required, the connection 42supporting a proxy client may itself be forwarded or relayed to another 43host if necessary. 44 451. Connection setup 46 47When a multiplexing connection is made to a ssh(1) operating as a 48ControlMaster from a client ssh(1), the first action of each is send 49a hello messages to its peer: 50 51 uint32 MUX_MSG_HELLO 52 uint32 protocol version 53 string extension name [optional] 54 string extension value [optional] 55 ... 56 57The current version of the mux protocol is 4. A client should refuse 58to connect to a master that speaks an unsupported protocol version. 59 60Following the version identifier are zero or more extensions represented 61as a name/value pair. No extensions are currently defined. 62 632. Opening a passenger mode session 64 65To open a new multiplexed session in passenger mode, a client sends the 66following request: 67 68 uint32 MUX_C_NEW_SESSION 69 uint32 request id 70 string reserved 71 bool want tty flag 72 bool want X11 forwarding flag 73 bool want agent flag 74 bool subsystem flag 75 uint32 escape char 76 string terminal type 77 string command 78 string environment string 0 [optional] 79 ... 80 81To disable the use of an escape character, "escape char" may be set 82to 0xffffffff. "terminal type" is generally set to the value of 83$TERM. zero or more environment strings may follow the command. 84 85The client then sends its standard input, output and error file 86descriptors (in that order) using Unix domain socket control messages. 87 88The contents of "reserved" are currently ignored. 89 90If successful, the server will reply with MUX_S_SESSION_OPENED 91 92 uint32 MUX_S_SESSION_OPENED 93 uint32 client request id 94 uint32 session id 95 96Otherwise it will reply with an error: MUX_S_PERMISSION_DENIED or 97MUX_S_FAILURE. 98 99Once the server has received the fds, it will respond with MUX_S_OK 100indicating that the session is up. The client now waits for the 101session to end. When it does, the server will send an exit status 102message: 103 104 uint32 MUX_S_EXIT_MESSAGE 105 uint32 session id 106 uint32 exit value 107 108The client should exit with this value to mimic the behaviour of a 109non-multiplexed ssh(1) connection. Two additional cases that the 110client must cope with are it receiving a signal itself and the 111server disconnecting without sending an exit message. 112 113A master may also send a MUX_S_TTY_ALLOC_FAIL before MUX_S_EXIT_MESSAGE 114if remote TTY allocation was unsuccessful. The client may use this to 115return its local tty to "cooked" mode. 116 117 uint32 MUX_S_TTY_ALLOC_FAIL 118 uint32 session id 119 1203. Requesting passenger-mode stdio forwarding 121 122A client may request the master to establish a stdio forwarding: 123 124 uint32 MUX_C_NEW_STDIO_FWD 125 uint32 request id 126 string reserved 127 string connect host 128 string connect port 129 130The client then sends its standard input and output file descriptors 131(in that order) using Unix domain socket control messages. 132 133The contents of "reserved" are currently ignored. 134 135A server may reply with a MUX_S_SESSION_OPENED, a MUX_S_PERMISSION_DENIED 136or a MUX_S_FAILURE. 137 1384. Health checks 139 140The client may request a health check/PID report from a server: 141 142 uint32 MUX_C_ALIVE_CHECK 143 uint32 request id 144 145The server replies with: 146 147 uint32 MUX_S_ALIVE 148 uint32 client request id 149 uint32 server pid 150 1515. Remotely terminating a master 152 153A client may request that a master terminate immediately: 154 155 uint32 MUX_C_TERMINATE 156 uint32 request id 157 158The server will reply with one of MUX_S_OK or MUX_S_PERMISSION_DENIED. 159 1606. Requesting establishment of port forwards 161 162A client may request the master to establish a port forward: 163 164 uint32 MUX_C_OPEN_FWD 165 uint32 request id 166 uint32 forwarding type 167 string listen host 168 uint32 listen port 169 string connect host 170 uint32 connect port 171 172forwarding type may be MUX_FWD_LOCAL, MUX_FWD_REMOTE, MUX_FWD_DYNAMIC. 173 174If listen port is (unsigned int) -2, then the listen host is treated as 175a unix socket path name. 176 177If connect port is (unsigned int) -2, then the connect host is treated 178as a unix socket path name. 179 180A server may reply with a MUX_S_OK, a MUX_S_REMOTE_PORT, a 181MUX_S_PERMISSION_DENIED or a MUX_S_FAILURE. 182 183For dynamically allocated listen port the server replies with 184 185 uint32 MUX_S_REMOTE_PORT 186 uint32 client request id 187 uint32 allocated remote listen port 188 1897. Requesting closure of port forwards 190 191A client may request the master to close a port forward: 192 193 uint32 MUX_C_CLOSE_FWD 194 uint32 request id 195 uint32 forwarding type 196 string listen host 197 uint32 listen port 198 string connect host 199 uint32 connect port 200 201A server may reply with a MUX_S_OK, a MUX_S_PERMISSION_DENIED or a 202MUX_S_FAILURE. 203 2048. Requesting shutdown of mux listener 205 206A client may request the master to stop accepting new multiplexing requests 207and remove its listener socket. 208 209 uint32 MUX_C_STOP_LISTENING 210 uint32 request id 211 212A server may reply with a MUX_S_OK, a MUX_S_PERMISSION_DENIED or a 213MUX_S_FAILURE. 214 2159. Requesting proxy mode 216 217A client may request that the control connection be placed in proxy 218mode: 219 220 uint32 MUX_C_PROXY 221 uint32 request id 222 223When a mux master receives this message, it will reply with a 224confirmation: 225 226 uint32 MUX_S_PROXY 227 uint32 request id 228 229And go into proxy mode. All subsequent data over the connection will 230be formatted as unencrypted, unpadded, SSH transport messages: 231 232 uint32 packet length 233 byte 0 (padding length) 234 byte packet type 235 byte[packet length - 2] ... 236 237The mux master will accept most connection messages and global requests, 238and will translate channel identifiers to ensure that the proxy client has 239globally unique channel numbers (i.e. a proxy client need not worry about 240collisions with other clients). 241 24210. Status messages 243 244The MUX_S_OK message is empty: 245 246 uint32 MUX_S_OK 247 uint32 client request id 248 249The MUX_S_PERMISSION_DENIED and MUX_S_FAILURE include a reason: 250 251 uint32 MUX_S_PERMISSION_DENIED 252 uint32 client request id 253 string reason 254 255 uint32 MUX_S_FAILURE 256 uint32 client request id 257 string reason 258 25911. Protocol numbers 260 261#define MUX_MSG_HELLO 0x00000001 262#define MUX_C_NEW_SESSION 0x10000002 263#define MUX_C_ALIVE_CHECK 0x10000004 264#define MUX_C_TERMINATE 0x10000005 265#define MUX_C_OPEN_FWD 0x10000006 266#define MUX_C_CLOSE_FWD 0x10000007 267#define MUX_C_NEW_STDIO_FWD 0x10000008 268#define MUX_C_STOP_LISTENING 0x10000009 269#define MUX_S_OK 0x80000001 270#define MUX_S_PERMISSION_DENIED 0x80000002 271#define MUX_S_FAILURE 0x80000003 272#define MUX_S_EXIT_MESSAGE 0x80000004 273#define MUX_S_ALIVE 0x80000005 274#define MUX_S_SESSION_OPENED 0x80000006 275#define MUX_S_REMOTE_PORT 0x80000007 276#define MUX_S_TTY_ALLOC_FAIL 0x80000008 277 278#define MUX_FWD_LOCAL 1 279#define MUX_FWD_REMOTE 2 280#define MUX_FWD_DYNAMIC 3 281 282XXX TODO 283XXX extended status (e.g. report open channels / forwards) 284XXX lock (maybe) 285XXX watch in/out traffic (pre/post crypto) 286XXX inject packet (what about replies) 287XXX server->client error/warning notifications 288XXX send signals via mux 289XXX ^Z support in passengers 290XXX extensions for multi-agent 291XXX extensions for multi-X11 292XXX session inspection via master 293XXX signals via mux request 294XXX list active connections via mux 295 296$OpenBSD: PROTOCOL.mux,v 1.14 2024/01/08 05:11:18 djm Exp $ 297