1*745b6a3dSJack WangWhat: /sys/class/rtrs-client 2*745b6a3dSJack WangDate: Feb 2020 3*745b6a3dSJack WangKernelVersion: 5.7 4*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 5*745b6a3dSJack WangDescription: When a user of RTRS API creates a new session, a directory entry with 6*745b6a3dSJack Wang the name of that session is created under /sys/class/rtrs-client/<session-name>/ 7*745b6a3dSJack Wang 8*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/add_path 9*745b6a3dSJack WangDate: Feb 2020 10*745b6a3dSJack WangKernelVersion: 5.7 11*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 12*745b6a3dSJack WangDescription: RW, adds a new path (connection) to an existing session. Expected format is the 13*745b6a3dSJack Wang following: 14*745b6a3dSJack Wang 15*745b6a3dSJack Wang <[source addr,]destination addr> 16*745b6a3dSJack Wang *addr ::= [ ip:<ipv4|ipv6> | gid:<gid> ] 17*745b6a3dSJack Wang 18*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/max_reconnect_attempts 19*745b6a3dSJack WangDate: Feb 2020 20*745b6a3dSJack WangKernelVersion: 5.7 21*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 22*745b6a3dSJack WangDescription: Maximum number reconnect attempts the client should make before giving up 23*745b6a3dSJack Wang after connection breaks unexpectedly. 24*745b6a3dSJack Wang 25*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/mp_policy 26*745b6a3dSJack WangDate: Feb 2020 27*745b6a3dSJack WangKernelVersion: 5.7 28*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 29*745b6a3dSJack WangDescription: Multipath policy specifies which path should be selected on each IO: 30*745b6a3dSJack Wang 31*745b6a3dSJack Wang round-robin (0): 32*745b6a3dSJack Wang select path in per CPU round-robin manner. 33*745b6a3dSJack Wang 34*745b6a3dSJack Wang min-inflight (1): 35*745b6a3dSJack Wang select path with minimum inflights. 36*745b6a3dSJack Wang 37*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/ 38*745b6a3dSJack WangDate: Feb 2020 39*745b6a3dSJack WangKernelVersion: 5.7 40*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 41*745b6a3dSJack WangDescription: Each path belonging to a given session is listed here by its source and 42*745b6a3dSJack Wang destination address. When a new path is added to a session by writing to 43*745b6a3dSJack Wang the "add_path" entry, a directory <src@dst> is created. 44*745b6a3dSJack Wang 45*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/state 46*745b6a3dSJack WangDate: Feb 2020 47*745b6a3dSJack WangKernelVersion: 5.7 48*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 49*745b6a3dSJack WangDescription: RO, Contains "connected" if the session is connected to the peer and fully 50*745b6a3dSJack Wang functional. Otherwise the file contains "disconnected" 51*745b6a3dSJack Wang 52*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/reconnect 53*745b6a3dSJack WangDate: Feb 2020 54*745b6a3dSJack WangKernelVersion: 5.7 55*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 56*745b6a3dSJack WangDescription: Write "1" to the file in order to reconnect the path. 57*745b6a3dSJack Wang Operation is blocking and returns 0 if reconnect was successful. 58*745b6a3dSJack Wang 59*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/disconnect 60*745b6a3dSJack WangDate: Feb 2020 61*745b6a3dSJack WangKernelVersion: 5.7 62*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 63*745b6a3dSJack WangDescription: Write "1" to the file in order to disconnect the path. 64*745b6a3dSJack Wang Operation blocks until RTRS path is disconnected. 65*745b6a3dSJack Wang 66*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/remove_path 67*745b6a3dSJack WangDate: Feb 2020 68*745b6a3dSJack WangKernelVersion: 5.7 69*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 70*745b6a3dSJack WangDescription: Write "1" to the file in order to disconnected and remove the path 71*745b6a3dSJack Wang from the session. Operation blocks until the path is disconnected 72*745b6a3dSJack Wang and removed from the session. 73*745b6a3dSJack Wang 74*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_name 75*745b6a3dSJack WangDate: Feb 2020 76*745b6a3dSJack WangKernelVersion: 5.7 77*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 78*745b6a3dSJack WangDescription: RO, Contains the the name of HCA the connection established on. 79*745b6a3dSJack Wang 80*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_port 81*745b6a3dSJack WangDate: Feb 2020 82*745b6a3dSJack WangKernelVersion: 5.7 83*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 84*745b6a3dSJack WangDescription: RO, Contains the port number of active port traffic is going through. 85*745b6a3dSJack Wang 86*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/src_addr 87*745b6a3dSJack WangDate: Feb 2020 88*745b6a3dSJack WangKernelVersion: 5.7 89*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 90*745b6a3dSJack WangDescription: RO, Contains the source address of the path 91*745b6a3dSJack Wang 92*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/dst_addr 93*745b6a3dSJack WangDate: Feb 2020 94*745b6a3dSJack WangKernelVersion: 5.7 95*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 96*745b6a3dSJack WangDescription: RO, Contains the destination address of the path 97*745b6a3dSJack Wang 98*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reset_all 99*745b6a3dSJack WangDate: Feb 2020 100*745b6a3dSJack WangKernelVersion: 5.7 101*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 102*745b6a3dSJack WangDescription: RW, Read will return usage help, write 0 will clear all the statistics. 103*745b6a3dSJack Wang 104*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/cpu_migration 105*745b6a3dSJack WangDate: Feb 2020 106*745b6a3dSJack WangKernelVersion: 5.7 107*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 108*745b6a3dSJack WangDescription: RTRS expects that each HCA IRQ is pinned to a separate CPU. If it's 109*745b6a3dSJack Wang not the case, the processing of an I/O response could be processed on a 110*745b6a3dSJack Wang different CPU than where it was originally submitted. This file shows 111*745b6a3dSJack Wang how many interrupts where generated on a non expected CPU. 112*745b6a3dSJack Wang "from:" is the CPU on which the IRQ was expected, but not generated. 113*745b6a3dSJack Wang "to:" is the CPU on which the IRQ was generated, but not expected. 114*745b6a3dSJack Wang 115*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reconnects 116*745b6a3dSJack WangDate: Feb 2020 117*745b6a3dSJack WangKernelVersion: 5.7 118*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 119*745b6a3dSJack WangDescription: Contains 2 unsigned int values, the first one records number of successful 120*745b6a3dSJack Wang reconnects in the path lifetime, the second one records number of failed 121*745b6a3dSJack Wang reconnects in the path lifetime. 122*745b6a3dSJack Wang 123*745b6a3dSJack WangWhat: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/rdma 124*745b6a3dSJack WangDate: Feb 2020 125*745b6a3dSJack WangKernelVersion: 5.7 126*745b6a3dSJack WangContact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> 127*745b6a3dSJack WangDescription: Contains statistics regarding rdma operations and inflight operations. 128*745b6a3dSJack Wang The output consists of 6 values: 129*745b6a3dSJack Wang 130*745b6a3dSJack Wang <read-count> <read-total-size> <write-count> <write-total-size> \ 131*745b6a3dSJack Wang <inflights> <failovered> 132