1What: /sys/kernel/debug/qat_<device>_<BDF>/telemetry/control 2Date: March 2024 3KernelVersion: 6.8 4Contact: qat-linux@intel.com 5Description: (RW) Enables/disables the reporting of telemetry metrics. 6 7 Allowed values to write: 8 ======================== 9 * 0: disable telemetry 10 * 1: enable telemetry 11 * 2, 3, 4: enable telemetry and calculate minimum, maximum 12 and average for each counter over 2, 3 or 4 samples 13 14 Returned values: 15 ================ 16 * 1-4: telemetry is enabled and running 17 * 0: telemetry is disabled 18 19 Example. 20 21 Writing '3' to this file starts the collection of 22 telemetry metrics. Samples are collected every second and 23 stored in a circular buffer of size 3. These values are then 24 used to calculate the minimum, maximum and average for each 25 counter. After enabling, counters can be retrieved through 26 the ``device_data`` file:: 27 28 echo 3 > /sys/kernel/debug/qat_4xxx_0000:6b:00.0/telemetry/control 29 30 Writing '0' to this file stops the collection of telemetry 31 metrics:: 32 33 echo 0 > /sys/kernel/debug/qat_4xxx_0000:6b:00.0/telemetry/control 34 35 This attribute is only available for qat_4xxx and qat_6xxx devices. 36 37What: /sys/kernel/debug/qat_<device>_<BDF>/telemetry/device_data 38Date: March 2024 39KernelVersion: 6.8 40Contact: qat-linux@intel.com 41Description: (RO) Reports device telemetry counters. 42 Reads report metrics about performance and utilization of 43 a QAT device: 44 45 ======================= ======================================== 46 Field Description 47 ======================= ======================================== 48 sample_cnt number of acquisitions of telemetry data 49 from the device. Reads are performed 50 every 1000 ms. 51 pci_trans_cnt number of PCIe partial transactions 52 max_rd_lat maximum logged read latency [ns] (could 53 be any read operation) 54 rd_lat_acc_avg average read latency [ns] 55 max_gp_lat max get to put latency [ns] (only takes 56 samples for AE0) 57 gp_lat_acc_avg average get to put latency [ns] 58 bw_in PCIe, write bandwidth [Mbps] 59 bw_out PCIe, read bandwidth [Mbps] 60 re_acc_avg average ring empty time [ns] 61 at_page_req_lat_avg Address Translator(AT), average page 62 request latency [ns] 63 at_trans_lat_avg AT, average page translation latency [ns] 64 at_max_tlb_used AT, maximum uTLB used 65 util_cpr<N> utilization of Compression slice N [%] 66 exec_cpr<N> execution count of Compression slice N 67 util_xlt<N> utilization of Translator slice N [%] 68 exec_xlt<N> execution count of Translator slice N 69 util_dcpr<N> utilization of Decompression slice N [%] 70 exec_dcpr<N> execution count of Decompression slice N 71 util_cnv<N> utilization of Compression and verify slice N [%] 72 exec_cnv<N> execution count of Compression and verify slice N 73 util_dcprz<N> utilization of Decompression slice N [%] 74 exec_dcprz<N> execution count of Decompression slice N 75 util_pke<N> utilization of PKE N [%] 76 exec_pke<N> execution count of PKE N 77 util_ucs<N> utilization of UCS slice N [%] 78 exec_ucs<N> execution count of UCS slice N 79 util_wat<N> utilization of Wireless Authentication 80 slice N [%] 81 exec_wat<N> execution count of Wireless Authentication 82 slice N 83 util_wcp<N> utilization of Wireless Cipher slice N [%] 84 exec_wcp<N> execution count of Wireless Cipher slice N 85 util_cph<N> utilization of Cipher slice N [%] 86 exec_cph<N> execution count of Cipher slice N 87 util_ath<N> utilization of Authentication slice N [%] 88 exec_ath<N> execution count of Authentication slice N 89 cmdq_wait_cnv<N> wait time for cmdq N to get Compression and verify 90 slice ownership 91 cmdq_exec_cnv<N> Compression and verify slice execution time while 92 owned by cmdq N 93 cmdq_drain_cnv<N> time taken for cmdq N to release Compression and 94 verify slice ownership 95 cmdq_wait_dcprz<N> wait time for cmdq N to get Decompression 96 slice N ownership 97 cmdq_exec_dcprz<N> Decompression slice execution time while 98 owned by cmdq N 99 cmdq_drain_dcprz<N> time taken for cmdq N to release Decompression 100 slice ownership 101 cmdq_wait_pke<N> wait time for cmdq N to get PKE slice ownership 102 cmdq_exec_pke<N> PKE slice execution time while owned by cmdq N 103 cmdq_drain_pke<N> time taken for cmdq N to release PKE slice 104 ownership 105 cmdq_wait_ucs<N> wait time for cmdq N to get UCS slice ownership 106 cmdq_exec_ucs<N> UCS slice execution time while owned by cmdq N 107 cmdq_drain_ucs<N> time taken for cmdq N to release UCS slice 108 ownership 109 cmdq_wait_ath<N> wait time for cmdq N to get Authentication slice 110 ownership 111 cmdq_exec_ath<N> Authentication slice execution time while owned 112 by cmdq N 113 cmdq_drain_ath<N> time taken for cmdq N to release Authentication 114 slice ownership 115 ======================= ======================================== 116 117 The telemetry report file can be read with the following command:: 118 119 cat /sys/kernel/debug/qat_4xxx_0000:6b:00.0/telemetry/device_data 120 121 If ``control`` is set to 1, only the current values of the 122 counters are displayed:: 123 124 <counter_name> <current> 125 126 If ``control`` is 2, 3 or 4, counters are displayed in the 127 following format:: 128 129 <counter_name> <current> <min> <max> <avg> 130 131 If a device lacks of a specific accelerator, the corresponding 132 attribute is not reported. 133 134 This attribute is only available for qat_4xxx and qat_6xxx devices. 135 136What: /sys/kernel/debug/qat_<device>_<BDF>/telemetry/rp_<A/B/C/D>_data 137Date: March 2024 138KernelVersion: 6.8 139Contact: qat-linux@intel.com 140Description: (RW) Selects up to 4 Ring Pairs (RP) to monitor, one per file, 141 and report telemetry counters related to each. 142 143 Allowed values to write: 144 ======================== 145 * 0 to ``<num_rps - 1>``: 146 Ring pair to be monitored. The value of ``num_rps`` can be 147 retrieved through ``/sys/bus/pci/devices/<BDF>/qat/num_rps``. 148 See Documentation/ABI/testing/sysfs-driver-qat. 149 150 Reads report metrics about performance and utilization of 151 the selected RP: 152 153 ======================= ======================================== 154 Field Description 155 ======================= ======================================== 156 sample_cnt number of acquisitions of telemetry data 157 from the device. Reads are performed 158 every 1000 ms 159 rp_num RP number associated with slot <A/B/C/D> 160 service_type service associated to the RP 161 pci_trans_cnt number of PCIe partial transactions 162 gp_lat_acc_avg average get to put latency [ns] 163 bw_in PCIe, write bandwidth [Mbps] 164 bw_out PCIe, read bandwidth [Mbps] 165 at_glob_devtlb_hit Message descriptor DevTLB hit rate 166 at_glob_devtlb_miss Message descriptor DevTLB miss rate 167 tl_at_payld_devtlb_hit Payload DevTLB hit rate 168 tl_at_payld_devtlb_miss Payload DevTLB miss rate 169 ======================= ======================================== 170 171 Example. 172 173 Writing the value '32' to the file ``rp_C_data`` starts the 174 collection of telemetry metrics for ring pair 32:: 175 176 echo 32 > /sys/kernel/debug/qat_4xxx_0000:6b:00.0/telemetry/rp_C_data 177 178 Once a ring pair is selected, statistics can be read accessing 179 the file:: 180 181 cat /sys/kernel/debug/qat_4xxx_0000:6b:00.0/telemetry/rp_C_data 182 183 If ``control`` is set to 1, only the current values of the 184 counters are displayed:: 185 186 <counter_name> <current> 187 188 If ``control`` is 2, 3 or 4, counters are displayed in the 189 following format:: 190 191 <counter_name> <current> <min> <max> <avg> 192 193 194 On QAT GEN4 devices there are 64 RPs on a PF, so the allowed 195 values are 0..63. This number is absolute to the device. 196 If Virtual Functions (VF) are used, the ring pair number can 197 be derived from the Bus, Device, Function of the VF: 198 199 ============ ====== ====== ====== ====== 200 PCI BDF/VF RP0 RP1 RP2 RP3 201 ============ ====== ====== ====== ====== 202 0000:6b:0.1 RP 0 RP 1 RP 2 RP 3 203 0000:6b:0.2 RP 4 RP 5 RP 6 RP 7 204 0000:6b:0.3 RP 8 RP 9 RP 10 RP 11 205 0000:6b:0.4 RP 12 RP 13 RP 14 RP 15 206 0000:6b:0.5 RP 16 RP 17 RP 18 RP 19 207 0000:6b:0.6 RP 20 RP 21 RP 22 RP 23 208 0000:6b:0.7 RP 24 RP 25 RP 26 RP 27 209 0000:6b:1.0 RP 28 RP 29 RP 30 RP 31 210 0000:6b:1.1 RP 32 RP 33 RP 34 RP 35 211 0000:6b:1.2 RP 36 RP 37 RP 38 RP 39 212 0000:6b:1.3 RP 40 RP 41 RP 42 RP 43 213 0000:6b:1.4 RP 44 RP 45 RP 46 RP 47 214 0000:6b:1.5 RP 48 RP 49 RP 50 RP 51 215 0000:6b:1.6 RP 52 RP 53 RP 54 RP 55 216 0000:6b:1.7 RP 56 RP 57 RP 58 RP 59 217 0000:6b:2.0 RP 60 RP 61 RP 62 RP 63 218 ============ ====== ====== ====== ====== 219 220 The mapping is only valid for the BDFs of VFs on the host. 221 222 223 The service provided on a ring-pair varies depending on the 224 configuration. The configuration for a given device can be 225 queried and set using ``cfg_services``. 226 See Documentation/ABI/testing/sysfs-driver-qat for details. 227 228 The following table reports how ring pairs are mapped to VFs 229 on the PF 0000:6b:0.0 configured for `sym;asym` or `asym;sym`: 230 231 =========== ============ =========== ============ =========== 232 PCI BDF/VF RP0/service RP1/service RP2/service RP3/service 233 =========== ============ =========== ============ =========== 234 0000:6b:0.1 RP 0 asym RP 1 sym RP 2 asym RP 3 sym 235 0000:6b:0.2 RP 4 asym RP 5 sym RP 6 asym RP 7 sym 236 0000:6b:0.3 RP 8 asym RP 9 sym RP10 asym RP11 sym 237 ... ... ... ... ... 238 =========== ============ =========== ============ =========== 239 240 All VFs follow the same pattern. 241 242 243 The following table reports how ring pairs are mapped to VFs on 244 the PF 0000:6b:0.0 configured for `dc`: 245 246 =========== ============ =========== ============ =========== 247 PCI BDF/VF RP0/service RP1/service RP2/service RP3/service 248 =========== ============ =========== ============ =========== 249 0000:6b:0.1 RP 0 dc RP 1 dc RP 2 dc RP 3 dc 250 0000:6b:0.2 RP 4 dc RP 5 dc RP 6 dc RP 7 dc 251 0000:6b:0.3 RP 8 dc RP 9 dc RP10 dc RP11 dc 252 ... ... ... ... ... 253 =========== ============ =========== ============ =========== 254 255 The mapping of a RP to a service can be retrieved using 256 ``rp2srv`` from sysfs. 257 See Documentation/ABI/testing/sysfs-driver-qat for details. 258 259 This attribute is only available for qat_4xxx and qat_6xxx devices. 260