1.. SPDX-License-Identifier: GPL-2.0 2 3 4=========== 5TPH Support 6=========== 7 8:Copyright: 2024 Advanced Micro Devices, Inc. 9:Authors: - Eric van Tassell <eric.vantassell@amd.com> 10 - Wei Huang <wei.huang2@amd.com> 11 12 13Overview 14======== 15 16TPH (TLP Processing Hints) is a PCIe feature that allows endpoint devices 17to provide optimization hints for requests that target memory space. 18These hints, in a format called Steering Tags (STs), are embedded in the 19requester's TLP headers, enabling the system hardware, such as the Root 20Complex, to better manage platform resources for these requests. 21 22For example, on platforms with TPH-based direct data cache injection 23support, an endpoint device can include appropriate STs in its DMA 24traffic to specify which cache the data should be written to. This allows 25the CPU core to have a higher probability of getting data from cache, 26potentially improving performance and reducing latency in data 27processing. 28 29 30How to Use TPH 31============== 32 33TPH is presented as an optional extended capability in PCIe. The Linux 34kernel handles TPH discovery during boot, but it is up to the device 35driver to request TPH enablement if it is to be utilized. Once enabled, 36the driver uses the provided API to obtain the Steering Tag for the 37target memory and to program the ST into the device's ST table. 38 39Enable TPH support in Linux 40--------------------------- 41 42To support TPH, the kernel must be built with the CONFIG_PCIE_TPH option 43enabled. 44 45Manage TPH 46---------- 47 48To enable TPH for a device, use the following function:: 49 50 int pcie_enable_tph(struct pci_dev *pdev, int mode); 51 52This function enables TPH support for device with a specific ST mode. 53Current supported modes include: 54 55 * PCI_TPH_ST_NS_MODE - NO ST Mode 56 * PCI_TPH_ST_IV_MODE - Interrupt Vector Mode 57 * PCI_TPH_ST_DS_MODE - Device Specific Mode 58 59`pcie_enable_tph()` checks whether the requested mode is actually 60supported by the device before enabling. The device driver can figure out 61which TPH mode is supported and can be properly enabled based on the 62return value of `pcie_enable_tph()`. 63 64To disable TPH, use the following function:: 65 66 void pcie_disable_tph(struct pci_dev *pdev); 67 68Manage ST 69--------- 70 71Steering Tags are platform specific. PCIe spec does not specify where STs 72are from. Instead PCI Firmware Specification defines an ACPI _DSM method 73(see the `Revised _DSM for Cache Locality TPH Features ECN 74<https://members.pcisig.com/wg/PCI-SIG/document/15470>`_) for retrieving 75STs for a target memory of various properties. This method is what is 76supported in this implementation. 77 78To retrieve a Steering Tag for a target memory associated with a specific 79CPU, use the following function:: 80 81 int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type type, 82 unsigned int cpu_uid, u16 *tag); 83 84The `type` argument is used to specify the memory type, either volatile 85or persistent, of the target memory. The `cpu_uid` argument specifies the 86CPU where the memory is associated to. 87 88After the ST value is retrieved, the device driver can use the following 89function to write the ST into the device:: 90 91 int pcie_tph_set_st_entry(struct pci_dev *pdev, unsigned int index, 92 u16 tag); 93 94The `index` argument is the ST table entry index the ST tag will be 95written into. `pcie_tph_set_st_entry()` will figure out the proper 96location of ST table, either in the MSI-X table or in the TPH Extended 97Capability space, and write the Steering Tag into the ST entry pointed by 98the `index` argument. 99 100It is completely up to the driver to decide how to use these TPH 101functions. For example a network device driver can use the TPH APIs above 102to update the Steering Tag when interrupt affinity of a RX/TX queue has 103been changed. Here is a sample code for IRQ affinity notifier: 104 105.. code-block:: c 106 107 static void irq_affinity_notified(struct irq_affinity_notify *notify, 108 const cpumask_t *mask) 109 { 110 struct drv_irq *irq; 111 unsigned int cpu_id; 112 u16 tag; 113 114 irq = container_of(notify, struct drv_irq, affinity_notify); 115 cpumask_copy(irq->cpu_mask, mask); 116 117 /* Pick a right CPU as the target - here is just an example */ 118 cpu_id = cpumask_first(irq->cpu_mask); 119 120 if (pcie_tph_get_cpu_st(irq->pdev, TPH_MEM_TYPE_VM, cpu_id, 121 &tag)) 122 return; 123 124 if (pcie_tph_set_st_entry(irq->pdev, irq->msix_nr, tag)) 125 return; 126 } 127 128Disable TPH system-wide 129----------------------- 130 131There is a kernel command line option available to control TPH feature: 132 * "notph": TPH will be disabled for all endpoint devices. 133