1============================ 2 Core Driver Infrastructure 3============================ 4 5GPU Hardware Structure 6====================== 7 8Each ASIC is a collection of hardware blocks. We refer to them as 9"IPs" (Intellectual Property blocks). Each IP encapsulates certain 10functionality. IPs are versioned and can also be mixed and matched. 11E.g., you might have two different ASICs that both have System DMA (SDMA) 5.x IPs. 12The driver is arranged by IPs. There are driver components to handle 13the initialization and operation of each IP. There are also a bunch 14of smaller IPs that don't really need much if any driver interaction. 15Those end up getting lumped into the common stuff in the soc files. 16The soc files (e.g., vi.c, soc15.c nv.c) contain code for aspects of 17the SoC itself rather than specific IPs. E.g., things like GPU resets 18and register access functions are SoC dependent. 19 20An APU contains more than just CPU and GPU, it also contains all of 21the platform stuff (audio, usb, gpio, etc.). Also, a lot of 22components are shared between the CPU, platform, and the GPU (e.g., 23SMU, PSP, etc.). Specific components (CPU, GPU, etc.) usually have 24their interface to interact with those common components. For things 25like S0i3 there is a ton of coordination required across all the 26components, but that is probably a bit beyond the scope of this 27section. 28 29With respect to the GPU, we have the following major IPs: 30 31GMC (Graphics Memory Controller) 32 This was a dedicated IP on older pre-vega chips, but has since 33 become somewhat decentralized on vega and newer chips. They now 34 have dedicated memory hubs for specific IPs or groups of IPs. We 35 still treat it as a single component in the driver however since 36 the programming model is still pretty similar. This is how the 37 different IPs on the GPU get the memory (VRAM or system memory). 38 It also provides the support for per process GPU virtual address 39 spaces. 40 41IH (Interrupt Handler) 42 This is the interrupt controller on the GPU. All of the IPs feed 43 their interrupts into this IP and it aggregates them into a set of 44 ring buffers that the driver can parse to handle interrupts from 45 different IPs. 46 47PSP (Platform Security Processor) 48 This handles security policy for the SoC and executes trusted 49 applications, and validates and loads firmwares for other blocks. 50 51SMU (System Management Unit) 52 This is the power management microcontroller. It manages the entire 53 SoC. The driver interacts with it to control power management 54 features like clocks, voltages, power rails, etc. 55 56DCN (Display Controller Next) 57 This is the display controller. It handles the display hardware. 58 It is described in more details in :ref:`Display Core <amdgpu-display-core>`. 59 60SDMA (System DMA) 61 This is a multi-purpose DMA engine. The kernel driver uses it for 62 various things including paging and GPU page table updates. It's also 63 exposed to userspace for use by user mode drivers (OpenGL, Vulkan, 64 etc.) 65 66GC (Graphics and Compute) 67 This is the graphics and compute engine, i.e., the block that 68 encompasses the 3D pipeline and and shader blocks. This is by far the 69 largest block on the GPU. The 3D pipeline has tons of sub-blocks. In 70 addition to that, it also contains the CP microcontrollers (ME, PFP, 71 CE, MEC) and the RLC microcontroller. It's exposed to userspace for 72 user mode drivers (OpenGL, Vulkan, OpenCL, etc.) 73 74VCN (Video Core Next) 75 This is the multi-media engine. It handles video and image encode and 76 decode. It's exposed to userspace for user mode drivers (VA-API, 77 OpenMAX, etc.) 78 79Graphics and Compute Microcontrollers 80------------------------------------- 81 82CP (Command Processor) 83 The name for the hardware block that encompasses the front end of the 84 GFX/Compute pipeline. Consists mainly of a bunch of microcontrollers 85 (PFP, ME, CE, MEC). The firmware that runs on these microcontrollers 86 provides the driver interface to interact with the GFX/Compute engine. 87 88 MEC (MicroEngine Compute) 89 This is the microcontroller that controls the compute queues on the 90 GFX/compute engine. 91 92 MES (MicroEngine Scheduler) 93 This is a new engine for managing queues. This is currently unused. 94 95RLC (RunList Controller) 96 This is another microcontroller in the GFX/Compute engine. It handles 97 power management related functionality within the GFX/Compute engine. 98 The name is a vestige of old hardware where it was originally added 99 and doesn't really have much relation to what the engine does now. 100 101 102GFX, Compute, and SDMA Overall Behavior 103======================================= 104 105.. note:: For simplicity, whenever the term block is used in this section, it 106 means GFX, Compute, and SDMA. 107 108GFX, Compute and SDMA share a similar form of operation that can be abstracted 109to facilitate understanding of the behavior of these blocks. See the figure 110below illustrating the common components of these blocks: 111 112.. kernel-figure:: pipe_and_queue_abstraction.svg 113 114In the central part of this figure, you can see two hardware elements, one called 115**Pipes** and another called **Queues**; it is important to highlight that Queues 116must be associated with a Pipe and vice-versa. Every specific hardware IP may have 117a different number of Pipes and, in turn, a different number of Queues; for 118example, GFX 11 has two Pipes and two Queues per Pipe for the GFX front end. 119 120Pipe is the hardware that processes the instructions available in the Queues; 121in other words, it is a thread executing the operations inserted in the Queue. 122One crucial characteristic of Pipes is that they can only execute one Queue at 123a time; no matter if the hardware has multiple Queues in the Pipe, it only runs 124one Queue per Pipe. 125 126Pipes have the mechanics of swapping between queues at the hardware level. 127Nonetheless, they only make use of Queues that are considered mapped. Pipes can 128switch between queues based on any of the following inputs: 129 1301. Command Stream; 1312. Packet by Packet; 1323. Other hardware requests the change (e.g., MES). 133 134Queues within Pipes are defined by the Hardware Queue Descriptors (HQD). 135Associated with the HQD concept, we have the Memory Queue Descriptor (MQD), 136which is responsible for storing information about the state of each of the 137available Queues in the memory. The state of a Queue contains information such 138as the GPU virtual address of the queue itself, save areas, doorbell, etc. The 139MQD also stores the HQD registers, which are vital for activating or 140deactivating a given Queue. The scheduling firmware (e.g., MES) is responsible 141for loading HQDs from MQDs and vice versa. 142 143The Queue-switching process can also happen with the firmware requesting the 144preemption or unmapping of a Queue. The firmware waits for the HQD_ACTIVE bit 145to change to low before saving the state into the MQD. To make a different 146Queue become active, the firmware copies the MQD state into the HQD registers 147and loads any additional state. Finally, it sets the HQD_ACTIVE bit to high to 148indicate that the queue is active. The Pipe will then execute work from active 149Queues. 150 151Driver Structure 152================ 153 154In general, the driver has a list of all of the IPs on a particular 155SoC and for things like init/fini/suspend/resume, more or less just 156walks the list and handles each IP. 157 158Some useful constructs: 159 160KIQ (Kernel Interface Queue) 161 This is a control queue used by the kernel driver to manage other gfx 162 and compute queues on the GFX/compute engine. You can use it to 163 map/unmap additional queues, etc. 164 165IB (Indirect Buffer) 166 A command buffer for a particular engine. Rather than writing 167 commands directly to the queue, you can write the commands into a 168 piece of memory and then put a pointer to the memory into the queue. 169 The hardware will then follow the pointer and execute the commands in 170 the memory, then returning to the rest of the commands in the ring. 171 172.. _amdgpu_memory_domains: 173 174Memory Domains 175============== 176 177.. kernel-doc:: include/uapi/drm/amdgpu_drm.h 178 :doc: memory domains 179 180Buffer Objects 181============== 182 183.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_object.c 184 :doc: amdgpu_object 185 186.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_object.c 187 :internal: 188 189PRIME Buffer Sharing 190==================== 191 192.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c 193 :doc: PRIME Buffer Sharing 194 195.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c 196 :internal: 197 198MMU Notifier 199============ 200 201.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c 202 :doc: MMU Notifier 203 204.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c 205 :internal: 206 207AMDGPU Virtual Memory 208===================== 209 210.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c 211 :doc: GPUVM 212 213.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c 214 :internal: 215 216Interrupt Handling 217================== 218 219.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c 220 :doc: Interrupt Handling 221 222.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c 223 :internal: 224 225IP Blocks 226========= 227 228.. kernel-doc:: drivers/gpu/drm/amd/include/amd_shared.h 229 :doc: IP Blocks 230 231.. kernel-doc:: drivers/gpu/drm/amd/include/amd_shared.h 232 :identifiers: amd_ip_block_type amd_ip_funcs DC_DEBUG_MASK 233
