1This document describes the multiplexing protocol used by ssh(1)'s 2ControlMaster connection-sharing. 3 4Multiplexing starts with a ssh(1) configured to act as a multiplexing 5master. This will cause ssh(1) to listen on a Unix domain socket for 6requests from clients. Clients communicate over this socket using a 7simple packetised protocol, where each message is proceeded with 8a length and message type in SSH uint32 wire format: 9 10 uint32 packet length 11 uint32 packet type 12 ... packet body 13 14Most messages from the client to the server contain a "request id" 15field. This field is returned in replies as "client request id" to 16facilitate matching of responses to requests. 17 18Many multiplexing (mux) client requests yield immediate responses from 19the mux process; requesting a forwarding, performing an alive check or 20requesting the master terminate itself fall in to this category. 21 22The most common use of multiplexing however is to maintain multiple 23concurrent sessions. These are supported via two separate modes: 24 25"Passenger" clients start by requesting a new session with a 26MUX_C_NEW_SESSION message and passing stdio file descriptors over the 27Unix domain control socket. The passenger client then waits until it is 28signaled or the mux server closes the session. This mode is so named as 29the client waits around while the mux server does all the driving. 30 31Stdio forwarding (requested using MUX_C_NEW_STDIO_FWD) is another 32example of passenger mode; the client passes the stdio file descriptors 33and passively waits for something to happen. 34 35"Proxy" clients, requested using MUX_C_PROXY, work quite differently. In 36this mode, the mux client/server connection socket will stop speaking 37the multiplexing protocol and start proxying SSH connection protocol 38messages between the client and server. The client therefore must 39speak a significant subset of the SSH protocol, but in return is able 40to access basically the full suite of connection protocol features. 41Moreover, as no file descriptor passing is required, the connection 42supporting a proxy client may itself be forwarded or relayed to another 43host if necessary. 44 451. Connection setup 46 47When a multiplexing connection is made to a ssh(1) operating as a 48ControlMaster from a client ssh(1), the first action of each is send 49a hello messages to its peer: 50 51 uint32 MUX_MSG_HELLO 52 uint32 protocol version 53 string extension name [optional] 54 string extension value [optional] 55 ... 56 57The current version of the mux protocol is 4. A client should refuse 58to connect to a master that speaks an unsupported protocol version. 59 60Following the version identifier are zero or more extensions represented 61as a name/value pair. No extensions are currently defined. 62 632. Opening a passenger mode session 64 65To open a new multiplexed session in passenger mode, a client sends the 66following request: 67 68 uint32 MUX_C_NEW_SESSION 69 uint32 request id 70 string reserved 71 bool want tty flag 72 bool want X11 forwarding flag 73 bool want agent flag 74 bool subsystem flag 75 uint32 escape char 76 string terminal type 77 string command 78 string environment string 0 [optional] 79 ... 80 81To disable the use of an escape character, "escape char" may be set 82to 0xffffffff. "terminal type" is generally set to the value of 83$TERM. zero or more environment strings may follow the command. 84 85The client then sends its standard input, output and error file 86descriptors (in that order) using Unix domain socket control messages. 87 88The contents of "reserved" are currently ignored. 89 90If successful, the server will reply with MUX_S_SESSION_OPENED 91 92 uint32 MUX_S_SESSION_OPENED 93 uint32 client request id 94 uint32 session id 95 96Otherwise it will reply with an error: MUX_S_PERMISSION_DENIED or 97MUX_S_FAILURE. 98 99Once the server has received the fds, it will respond with MUX_S_OK 100indicating that the session is up. The client now waits for the 101session to end. When it does, the server will send an exit status 102message: 103 104 uint32 MUX_S_EXIT_MESSAGE 105 uint32 session id 106 uint32 exit value 107 108The client should exit with this value to mimic the behaviour of a 109non-multiplexed ssh(1) connection. Two additional cases that the 110client must cope with are it receiving a signal itself and the 111server disconnecting without sending an exit message. 112 113A master may also send a MUX_S_TTY_ALLOC_FAIL before MUX_S_EXIT_MESSAGE 114if remote TTY allocation was unsuccessful. The client may use this to 115return its local tty to "cooked" mode. 116 117 uint32 MUX_S_TTY_ALLOC_FAIL 118 uint32 session id 119 1203. Requesting passenger-mode stdio forwarding 121 122A client may request the master to establish a stdio forwarding: 123 124 uint32 MUX_C_NEW_STDIO_FWD 125 uint32 request id 126 string reserved 127 string connect host 128 string connect port 129 130The client then sends its standard input and output file descriptors 131(in that order) using Unix domain socket control messages. 132 133The contents of "reserved" are currently ignored. 134 135A server may reply with a MUX_S_SESSION_OPENED, a MUX_S_PERMISSION_DENIED 136or a MUX_S_FAILURE. 137 1384. Health checks 139 140The client may request a health check/PID report from a server: 141 142 uint32 MUX_C_ALIVE_CHECK 143 uint32 request id 144 145The server replies with: 146 147 uint32 MUX_S_ALIVE 148 uint32 client request id 149 uint32 server pid 150 1515. Remotely terminating a master 152 153A client may request that a master terminate immediately: 154 155 uint32 MUX_C_TERMINATE 156 uint32 request id 157 158The server will reply with one of MUX_S_OK or MUX_S_PERMISSION_DENIED. 159 1606. Requesting establishment of port forwards 161 162A client may request the master to establish a port forward: 163 164 uint32 MUX_C_OPEN_FWD 165 uint32 request id 166 uint32 forwarding type 167 string listen host 168 uint32 listen port 169 string connect host 170 uint32 connect port 171 172forwarding type may be MUX_FWD_LOCAL, MUX_FWD_REMOTE, MUX_FWD_DYNAMIC. 173 174If listen port is (unsigned int) -2, then the listen host is treated as 175a unix socket path name. 176 177If connect port is (unsigned int) -2, then the connect host is treated 178as a unix socket path name. 179 180A server may reply with a MUX_S_OK, a MUX_S_REMOTE_PORT, a 181MUX_S_PERMISSION_DENIED or a MUX_S_FAILURE. 182 183For dynamically allocated listen port the server replies with 184 185 uint32 MUX_S_REMOTE_PORT 186 uint32 client request id 187 uint32 allocated remote listen port 188 1897. Requesting closure of port forwards 190 191Note: currently unimplemented (server will always reply with MUX_S_FAILURE). 192 193A client may request the master to close a port forward: 194 195 uint32 MUX_C_CLOSE_FWD 196 uint32 request id 197 uint32 forwarding type 198 string listen host 199 uint32 listen port 200 string connect host 201 uint32 connect port 202 203A server may reply with a MUX_S_OK, a MUX_S_PERMISSION_DENIED or a 204MUX_S_FAILURE. 205 2068. Requesting shutdown of mux listener 207 208A client may request the master to stop accepting new multiplexing requests 209and remove its listener socket. 210 211 uint32 MUX_C_STOP_LISTENING 212 uint32 request id 213 214A server may reply with a MUX_S_OK, a MUX_S_PERMISSION_DENIED or a 215MUX_S_FAILURE. 216 2179. Requesting proxy mode 218 219A client may request that the control connection be placed in proxy 220mode: 221 222 uint32 MUX_C_PROXY 223 uint32 request id 224 225When a mux master receives this message, it will reply with a 226confirmation: 227 228 uint32 MUX_S_PROXY 229 uint32 request id 230 231And go into proxy mode. All subsequent data over the connection will 232be formatted as unencrypted, unpadded, SSH transport messages: 233 234 uint32 packet length 235 byte 0 (padding length) 236 byte packet type 237 byte[packet length - 2] ... 238 239The mux master will accept most connection messages and global requests, 240and will translate channel identifiers to ensure that the proxy client has 241globally unique channel numbers (i.e. a proxy client need not worry about 242collisions with other clients). 243 24410. Status messages 245 246The MUX_S_OK message is empty: 247 248 uint32 MUX_S_OK 249 uint32 client request id 250 251The MUX_S_PERMISSION_DENIED and MUX_S_FAILURE include a reason: 252 253 uint32 MUX_S_PERMISSION_DENIED 254 uint32 client request id 255 string reason 256 257 uint32 MUX_S_FAILURE 258 uint32 client request id 259 string reason 260 26111. Protocol numbers 262 263#define MUX_MSG_HELLO 0x00000001 264#define MUX_C_NEW_SESSION 0x10000002 265#define MUX_C_ALIVE_CHECK 0x10000004 266#define MUX_C_TERMINATE 0x10000005 267#define MUX_C_OPEN_FWD 0x10000006 268#define MUX_C_CLOSE_FWD 0x10000007 269#define MUX_C_NEW_STDIO_FWD 0x10000008 270#define MUX_C_STOP_LISTENING 0x10000009 271#define MUX_S_OK 0x80000001 272#define MUX_S_PERMISSION_DENIED 0x80000002 273#define MUX_S_FAILURE 0x80000003 274#define MUX_S_EXIT_MESSAGE 0x80000004 275#define MUX_S_ALIVE 0x80000005 276#define MUX_S_SESSION_OPENED 0x80000006 277#define MUX_S_REMOTE_PORT 0x80000007 278#define MUX_S_TTY_ALLOC_FAIL 0x80000008 279 280#define MUX_FWD_LOCAL 1 281#define MUX_FWD_REMOTE 2 282#define MUX_FWD_DYNAMIC 3 283 284XXX TODO 285XXX extended status (e.g. report open channels / forwards) 286XXX lock (maybe) 287XXX watch in/out traffic (pre/post crypto) 288XXX inject packet (what about replies) 289XXX server->client error/warning notifications 290XXX send signals via mux 291XXX ^Z support in passengers 292XXX extensions for multi-agent 293XXX extensions for multi-X11 294XXX session inspection via master 295XXX signals via mux request 296XXX list active connections via mux 297 298$OpenBSD: PROTOCOL.mux,v 1.13 2022/01/01 01:55:30 jsg Exp $ 299