1.. SPDX-License-Identifier: GPL-2.0 2 3:orphan: 4 5============== 6MSM Preemption 7============== 8 9Preemption allows Adreno GPUs to switch to a higher priority ring when work is 10pushed to it, reducing latency for high priority submissions. 11 12When preemption is enabled 4 rings are initialized, corresponding to different 13priority levels. Having multiple rings is purely a software concept as the GPU 14only has registers to keep track of one graphics ring. 15The kernel is able to switch which ring is currently being processed by 16requesting preemption. When certain conditions are met, depending on the 17priority level, the GPU will save its current state in a series of buffers, 18then restores state from a similar set of buffers specified by the kernel. It 19then resumes execution and fires an IRQ to let the kernel know the context 20switch has completed. 21 22This mechanism can be used by the kernel to switch between rings. Whenever a 23submission occurs the kernel finds the highest priority ring which isn't empty 24and preempts to it if said ring is not the one being currently executed. This is 25also done whenever a submission completes to make sure execution resumes on a 26lower priority ring when a higher priority ring is done. 27 28Preemption levels 29----------------- 30 31Preemption can only occur at certain boundaries. The exact conditions can be 32configured by changing the preemption level, this allows to compromise between 33latency (ie. the time that passes between when the kernel requests preemption 34and when the SQE begins saving state) and overhead (the amount of state that 35needs to be saved). 36 37The GPU offers 3 levels: 38 39Level 0 40 Preemption only occurs at the submission level. This requires the least amount 41 of state to be saved as the execution of userspace submitted IBs is never 42 interrupted, however it offers very little benefit compared to not enabling 43 preemption of any kind. 44 45Level 1 46 Preemption occurs at either bin level, if using GMEM rendering, or draw level 47 in the sysmem rendering case. 48 49Level 2 50 Preemption occurs at draw level. 51 52Level 1 is the mode that is used by the msm driver. 53 54Additionally the GPU allows to specify a `skip_save_restore` option. This 55disables the saving and restoring of all registers except those relating to the 56operation of the SQE itself, reducing overhead. Saving and restoring is only 57skipped when using GMEM with Level 1 preemption. When enabling this userspace is 58expected to set the state that isn't preserved whenever preemption occurs which 59is done by specifying preamble and postambles. Those are IBs that are executed 60before and after preemption. 61 62Preemption buffers 63------------------ 64 65A series of buffers are necessary to store the state of rings while they are not 66being executed. There are different kinds of preemption records and most of 67those require one buffer per ring. This is because preemption never occurs 68between submissions on the same ring, which always run in sequence when the ring 69is active. This means that only one context per ring is effectively active. 70 71SMMU_INFO 72 This buffer contains info about the current SMMU configuration such as the 73 ttbr0 register. The SQE firmware isn't actually able to save this record. 74 As a result SMMU info must be saved manually from the CP to a buffer and the 75 SMMU record updated with info from said buffer before triggering 76 preemption. 77 78NON_SECURE 79 This is the main preemption record where most state is saved. It is mostly 80 opaque to the kernel except for the first few words that must be initialized 81 by the kernel. 82 83SECURE 84 This saves state related to the GPU's secure mode. 85 86NON_PRIV 87 The intended purpose of this record is unknown. The SQE firmware actually 88 ignores it and therefore msm doesn't handle it. 89 90COUNTER 91 This record is used to save and restore performance counters. 92 93Handling the permissions of those buffers is critical for security. All but the 94NON_PRIV records need to be inaccessible from userspace, so they must be mapped 95in the kernel address space with the MSM_BO_MAP_PRIV flag. 96For example, making the NON_SECURE record accessible from userspace would allow 97any process to manipulate a saved ring's RPTR which can be used to skip the 98execution of some packets in a ring and execute user commands with higher 99privileges. 100