gpu/tests/buddy: add dirty tracker performance KUnit test

Add gpu_test_buddy_dirty_tracker_performance to demonstrate the key
advantage of the decoupled dirty-tracker design over the old dual-tree
/

gpu/tests/buddy: add dirty tracker performance KUnit test

Add gpu_test_buddy_dirty_tracker_performance to demonstrate the key
advantage of the decoupled dirty-tracker design over the old dual-tree
/ force_merge approach.

The test runs two scenarios on a 4 GiB pool after alternating
clear/dirty fragmentation at 4 KiB granularity:

1. Contiguous 2 GiB alloc: smaller than the 4 GiB span so it routes
through the contiguous allocator rather than the exact-range fast
path. The old design requires __force_merge() to rebuild the
block; the new design coalesces during free() so the alloc is
O(log N).

old (force_merge) - 45 ms
dirty tracker design - 11 ms

2. Repeated 256 KiB alloc throughput: the old design pays
__force_merge() on every alloc; the new design does not.

old (force_merge) - 86 ms
dirty tracker design - 20 ms

v2:
- Force a contiguous 256 KiB allocation in the repeated-alloc loop so
the baseline actually exercises __force_merge(). (sashiko)

v3:
- Make the first test allocate 2 GiB instead of the full 4 GiB pool,
so it really uses the contiguous path; update the numbers. (sashiko)

Assisted-by: Claude:claude-opus-4-8
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260825172432.80355-2-Arunpravin.PaneerSelvam@amd.com

show more ...

gpu/buddy: replace dual-tree/force_merge with decoupled dirty tracker

The current buddy allocator maintains separate clear_tree[] and
dirty_tree[] rbtrees per order, preventing coalescing between cl

gpu/buddy: replace dual-tree/force_merge with decoupled dirty tracker

The current buddy allocator maintains separate clear_tree[] and
dirty_tree[] rbtrees per order, preventing coalescing between cleared
and dirty buddies. Under mixed workloads, this creates a merge barrier:
adjacent buddies frequently end up split across trees, forcing reliance
on __force_merge() during allocation.

__force_merge() performs an O(N x max_order) scan under the VRAM manager
lock, leading to allocation stalls and failures for large contiguous
requests even when sufficient total free memory is available.

Solution

Replace the dual-tree design with:
- A single free_tree[order] rbtree for dirty and mixed free blocks
(fully cleared free blocks float outside this tree)
- A lightweight out-of-band dirty tracker (gpu_dirty_tracker)

Fully cleared free blocks are tracked outside the buddy trees using an
augmented interval rbtree, enabling O(log E) lookup of the largest
cleared extents.

Buddy coalescing is now unconditional in __gpu_buddy_free(), regardless
of clear/dirty state. This removes the merge barrier and eliminates the
need for __force_merge().

Benefits

- Correct high-order allocations after mixed clear/dirty workloads
- Elimination of O(N x max_order) merge cost from the allocation path
- O(log E) cleared-extent lookup replacing O(N) scans
- Predictable allocation latency under fragmentation
- Reduced complexity with a single tree per order

Test:
dEQP-VK.memory.allocation.basic.size_8KiB.reverse.count_4000

Below data is from /sys/kernel/debug/dri/1/amdgpu_vram_mm:

Base (dual-tree), before VKCTS test:
order- 6 free: 6 MiB, blocks: 26
order- 5 free: 1 MiB, blocks: 15
order- 4 free: 960 KiB, blocks: 15
order- 3 free: 5 MiB, blocks: 171
order- 2 free: 2 MiB, blocks: 176
order- 1 free: 1 MiB, blocks: 165
order- 0 free: 16 KiB, blocks: 4

Base (dual-tree), after VKCTS test:
order- 6 free: 768 KiB, blocks: 3
order- 5 free: 499 MiB, blocks: 3999
order- 4 free: 250 MiB, blocks: 4001
order- 3 free: 129 MiB, blocks: 4157
order- 2 free: 65 MiB, blocks: 4161
order- 1 free: 63 MiB, blocks: 8138
order- 0 free: 20 KiB, blocks: 5

Dirty tracker, before VKCTS test:
order- 6 free: 4 MiB, blocks: 19
order- 5 free: 2 MiB, blocks: 18
order- 4 free: 704 KiB, blocks: 11
order- 3 free: 5 MiB, blocks: 168
order- 2 free: 2 MiB, blocks: 174
order- 1 free: 1 MiB, blocks: 167
order- 0 free: 32 KiB, blocks: 8

Dirty tracker, after VKCTS test:
order- 6 free: 4 MiB, blocks: 19
order- 5 free: 2 MiB, blocks: 18
order- 4 free: 704 KiB, blocks: 11
order- 3 free: 5 MiB, blocks: 168
order- 2 free: 2 MiB, blocks: 174
order- 1 free: 1 MiB, blocks: 167
order- 0 free: 28 KiB, blocks: 7

v2:
- Code-style cleanup and minor refactoring
- Renamed locals for clarity

v3:
- Keep cleared blocks inside free_tree[] instead of floating them.
- Add subtree_has_dirty rbtree augment for O(log N) dirty-first walk.

v4:
- Fixed checkpatch warnings.
- Optimized gpu_buddy_reset_clear() to a single post-order walk that
flips block headers and recomputes the rbtree augment in one pass.
- Propagate subtree_max_size top-down in insert_extent() so ancestors
are not left with stale values on no-rotation inserts. (sashiko)
- Drop the whole extent in gpu_dirty_tracker_mark_dirty() when the
inside-split allocation fails, avoiding a stale clear claim. (sashiko)
- Make gpu_dirty_tracker_find() alignment-aware and fall back to the
dirty tree on steered failure to avoid spurious -ENOSPC. (sashiko)

v5:
- Track dirty extents instead of cleared ones: steer dirty allocs onto
tracked dirty windows and pick clear allocs via a free-tree augment,
avoiding clear-memory wastage by keeping cleared free blocks untouched
during dirty allocation.

v6:
- Make __alloc_range_bias() return the highest/right-most address by
default, establishing top-down as the intended placement for
range-biased allocations.
- Honour GPU_BUDDY_CLEAR_ALLOCATION in __alloc_range_bias() by steering
the descent towards clear subtrees for non-top-down clear
requests. (sashiko)
- Skip dirty-tracker steering for offset-aligned requests so they keep
their min_block_size alignment. (sashiko)
- sashiko reported that the __GFP_NOFAIL dirty-extent allocations on
the free path could deadlock during memory reclaim, since that is a
GFP_KERNEL allocation on the free path; move to a per-tracker
mempool so extent nodes are guaranteed without __GFP_NOFAIL.
(sashiko)
- Derive each free block's clear/dirty class from the blocks already
in hand on split, free, alloc, trim and init instead of querying the
dirty tracker, removing the tracker lookups from the hot paths.

v7:
- Preserve mixed-block clear state in __gpu_buddy_free() when a mixed
split child is re-merged after an undone split. (sashiko)
- Prefer a fully-clear block over a mixed one of the same order via a
single ordered clear-state max augment on free_tree[].

v8:
- Coalesce contiguous dirty blocks in __gpu_buddy_free_list() into one
dirty extent update instead of one mark_dirty() per block. (Matthew)

v9:
- Reset has_clear on allocation so a mixed block taken whole and later
freed fully dirty is not re-tracked as mixed. (sashiko)

v10:
- Use a plain slab allocation for dirty extents; skip and log once on
failure. (Matthew)
- Assert a non-zero size in the dirty-tracker range helpers. (Matthew)
- Drop the cached clear_avail member; derive it on demand. (Matthew)
- Collapse the two dirty branches of gpu_buddy_reset_clear(). (Matthew)
- Move the gpu_block_state enum above the gpu_buddy_block kernel-doc
so the doc directly precedes its struct. (Matthew)
- Mark the gpu_dirty_tracker struct private. (Matthew)
- Preserve a block's clear state on non-clear allocation instead of
force-dirtying it. (Matthew)
- Drop the redundant header clear in gpu_buddy_block_trim(). (Matthew)

v11:
- Assert the manager lock is held in gpu_buddy_clear_avail(). (Matthew)

Assisted-by: Claude:claude-opus-4-8
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260825172432.80355-1-Arunpravin.PaneerSelvam@amd.com

show more ...

drm/tests/gpu_buddy: fix interleaving in buffer clearance test

The resume clearance test skipped every other allocation, expecting an
interleaved clear/dirty layout. But the buddy allocator hands ou

drm/tests/gpu_buddy: fix interleaving in buffer clearance test

The resume clearance test skipped every other allocation, expecting an
interleaved clear/dirty layout. But the buddy allocator hands out blocks
contiguously, so this just allocated half the pages in one chunk and never
exercised gpu_buddy_reset_clear()'s force-merge of opposite-state buddies.
Allocate all pages into two lists instead and free one cleared, one dirty,
to build a truly interleaved pattern.

v2: Use for loops instead of do-while for the allocation loops (Jani Nikula)

Fixes: e3335ccbf4da ("drm/tests/gpu_buddy: add a new test case for buffer clearance during resume")
Reported-by: Sashiko-bot <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260721114236.507578-1-Arunpravin.PaneerSelvam@amd.com?part=1
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: Christian König <christian.koenig@amd.com>
Assisted-by: GitHub_Copilot:claude-opus-4.8
Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260803065656.2960810-1-Arunpravin.PaneerSelvam@amd.com

show more ...

gpu/tests/gpu_buddy: Add KUnit test for gpu_buddy_allocated_addr_to_block

Add a new KUnit test gpu_test_buddy_addr_to_block() that validates the
gpu_buddy_allocated_addr_to_block() helper which trac

gpu/tests/gpu_buddy: Add KUnit test for gpu_buddy_allocated_addr_to_block

Add a new KUnit test gpu_test_buddy_addr_to_block() that validates the
gpu_buddy_allocated_addr_to_block() helper which traces a address back
to its allocated buddy block.

The test covers:
- Exact address matching returns the correct allocated block
- An unallocated address inside the manager should return NULL
- An address outside the manager should return -ENXIO

v4(MattA):
- Add test for unaligned address
v3(Sashiko):
- remove unused target_addr variable
v2(Sashiko):
- Drop the mutex and lockdep annotation; standalone KUnit tests do
not register a driver lock.

Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Link: https://patch.msgid.link/20260806053624.3215216-6-tejas.upadhyay@intel.com

show more ...

gpu/tests: Add a dedicated .kunitconfig

Add a dedicated .kunitconfig for running the GPU buddy allocator tests.

Signed-off-by: Krzysztof Niemiec <krzysztof.niemiec@intel.com>
Reviewed-by: Sebastian

gpu/tests: Add a dedicated .kunitconfig

Add a dedicated .kunitconfig for running the GPU buddy allocator tests.

Signed-off-by: Krzysztof Niemiec <krzysztof.niemiec@intel.com>
Reviewed-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Link: https://patch.msgid.link/20260729091816.27860-4-krzysztof.niemiec@intel.com
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>

show more ...

drm/tests/gpu_buddy: add a new test case for buffer clearance during resume

Add a new unit test case for buffer clearance issue during
resume.

Using a non-power-of-two mm size, allocate alternating

drm/tests/gpu_buddy: add a new test case for buffer clearance during resume

Add a new unit test case for buffer clearance issue during
resume.

Using a non-power-of-two mm size, allocate alternating blocks of
4KiB in an even sequence and free them as cleared. All alternate
blocks should be marked as dirty and the split blocks should be
merged back to their original size when the blocks clear reset
function is called.

Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260721114236.507578-1-Arunpravin.PaneerSelvam@amd.com

show more ...