1===================== 2DRM Memory Management 3===================== 4 5Modern Linux systems require large amount of graphics memory to store 6frame buffers, textures, vertices and other graphics-related data. Given 7the very dynamic nature of many of that data, managing graphics memory 8efficiently is thus crucial for the graphics stack and plays a central 9role in the DRM infrastructure. 10 11The DRM core includes two memory managers, namely Translation Table Manager 12(TTM) and Graphics Execution Manager (GEM). TTM was the first DRM memory 13manager to be developed and tried to be a one-size-fits-them all 14solution. It provides a single userspace API to accommodate the need of 15all hardware, supporting both Unified Memory Architecture (UMA) devices 16and devices with dedicated video RAM (i.e. most discrete video cards). 17This resulted in a large, complex piece of code that turned out to be 18hard to use for driver development. 19 20GEM started as an Intel-sponsored project in reaction to TTM's 21complexity. Its design philosophy is completely different: instead of 22providing a solution to every graphics memory-related problems, GEM 23identified common code between drivers and created a support library to 24share it. GEM has simpler initialization and execution requirements than 25TTM, but has no video RAM management capabilities and is thus limited to 26UMA devices. 27 28.. contents:: 29 30The Translation Table Manager (TTM) 31=================================== 32 33.. kernel-doc:: drivers/gpu/drm/ttm/ttm_module.c 34 :doc: TTM 35 36.. kernel-doc:: include/drm/ttm/ttm_caching.h 37 :internal: 38 39TTM device object reference 40--------------------------- 41 42.. kernel-doc:: include/drm/ttm/ttm_device.h 43 :internal: 44 45.. kernel-doc:: drivers/gpu/drm/ttm/ttm_device.c 46 :export: 47 48TTM resource placement reference 49-------------------------------- 50 51.. kernel-doc:: include/drm/ttm/ttm_placement.h 52 :internal: 53 54TTM resource object reference 55----------------------------- 56 57.. kernel-doc:: include/drm/ttm/ttm_resource.h 58 :internal: 59 60.. kernel-doc:: drivers/gpu/drm/ttm/ttm_resource.c 61 :export: 62 63TTM TT object reference 64----------------------- 65 66.. kernel-doc:: include/drm/ttm/ttm_tt.h 67 :internal: 68 69.. kernel-doc:: drivers/gpu/drm/ttm/ttm_tt.c 70 :export: 71 72TTM page pool reference 73----------------------- 74 75.. kernel-doc:: include/drm/ttm/ttm_pool.h 76 :internal: 77 78.. kernel-doc:: drivers/gpu/drm/ttm/ttm_pool.c 79 :export: 80 81The Graphics Execution Manager (GEM) 82==================================== 83 84The GEM design approach has resulted in a memory manager that doesn't 85provide full coverage of all (or even all common) use cases in its 86userspace or kernel API. GEM exposes a set of standard memory-related 87operations to userspace and a set of helper functions to drivers, and 88let drivers implement hardware-specific operations with their own 89private API. 90 91The GEM userspace API is described in the `GEM - the Graphics Execution 92Manager <http://lwn.net/Articles/283798/>`__ article on LWN. While 93slightly outdated, the document provides a good overview of the GEM API 94principles. Buffer allocation and read and write operations, described 95as part of the common GEM API, are currently implemented using 96driver-specific ioctls. 97 98GEM is data-agnostic. It manages abstract buffer objects without knowing 99what individual buffers contain. APIs that require knowledge of buffer 100contents or purpose, such as buffer allocation or synchronization 101primitives, are thus outside of the scope of GEM and must be implemented 102using driver-specific ioctls. 103 104On a fundamental level, GEM involves several operations: 105 106- Memory allocation and freeing 107- Command execution 108- Aperture management at command execution time 109 110Buffer object allocation is relatively straightforward and largely 111provided by Linux's shmem layer, which provides memory to back each 112object. 113 114Device-specific operations, such as command execution, pinning, buffer 115read & write, mapping, and domain ownership transfers are left to 116driver-specific ioctls. 117 118GEM Initialization 119------------------ 120 121Drivers that use GEM must set the DRIVER_GEM bit in the struct 122:c:type:`struct drm_driver <drm_driver>` driver_features 123field. The DRM core will then automatically initialize the GEM core 124before calling the load operation. Behind the scene, this will create a 125DRM Memory Manager object which provides an address space pool for 126object allocation. 127 128In a KMS configuration, drivers need to allocate and initialize a 129command ring buffer following core GEM initialization if required by the 130hardware. UMA devices usually have what is called a "stolen" memory 131region, which provides space for the initial framebuffer and large, 132contiguous memory regions required by the device. This space is 133typically not managed by GEM, and must be initialized separately into 134its own DRM MM object. 135 136GEM Objects Creation 137-------------------- 138 139GEM splits creation of GEM objects and allocation of the memory that 140backs them in two distinct operations. 141 142GEM objects are represented by an instance of struct :c:type:`struct 143drm_gem_object <drm_gem_object>`. Drivers usually need to 144extend GEM objects with private information and thus create a 145driver-specific GEM object structure type that embeds an instance of 146struct :c:type:`struct drm_gem_object <drm_gem_object>`. 147 148To create a GEM object, a driver allocates memory for an instance of its 149specific GEM object type and initializes the embedded struct 150:c:type:`struct drm_gem_object <drm_gem_object>` with a call 151to drm_gem_object_init(). The function takes a pointer 152to the DRM device, a pointer to the GEM object and the buffer object 153size in bytes. 154 155GEM uses shmem to allocate anonymous pageable memory. 156drm_gem_object_init() will create an shmfs file of the 157requested size and store it into the struct :c:type:`struct 158drm_gem_object <drm_gem_object>` filp field. The memory is 159used as either main storage for the object when the graphics hardware 160uses system memory directly or as a backing store otherwise. Drivers 161can call drm_gem_huge_mnt_create() to create, mount and use a huge 162shmem mountpoint instead of the default one ('shm_mnt'). For builds 163with CONFIG_TRANSPARENT_HUGEPAGE enabled, further calls to 164drm_gem_object_init() will let shmem allocate huge pages when 165possible. 166 167Drivers are responsible for the actual physical pages allocation by 168calling shmem_read_mapping_page_gfp() for each page. 169Note that they can decide to allocate pages when initializing the GEM 170object, or to delay allocation until the memory is needed (for instance 171when a page fault occurs as a result of a userspace memory access or 172when the driver needs to start a DMA transfer involving the memory). 173 174Anonymous pageable memory allocation is not always desired, for instance 175when the hardware requires physically contiguous system memory as is 176often the case in embedded devices. Drivers can create GEM objects with 177no shmfs backing (called private GEM objects) by initializing them with a call 178to drm_gem_private_object_init() instead of drm_gem_object_init(). Storage for 179private GEM objects must be managed by drivers. 180 181GEM Objects Lifetime 182-------------------- 183 184All GEM objects are reference-counted by the GEM core. References can be 185acquired and release by calling drm_gem_object_get() and drm_gem_object_put() 186respectively. 187 188When the last reference to a GEM object is released the GEM core calls 189the :c:type:`struct drm_gem_object_funcs <gem_object_funcs>` free 190operation. That operation is mandatory for GEM-enabled drivers and must 191free the GEM object and all associated resources. 192 193void (\*free) (struct drm_gem_object \*obj); Drivers are 194responsible for freeing all GEM object resources. This includes the 195resources created by the GEM core, which need to be released with 196drm_gem_object_release(). 197 198GEM Objects Naming 199------------------ 200 201Communication between userspace and the kernel refers to GEM objects 202using local handles, global names or, more recently, file descriptors. 203All of those are 32-bit integer values; the usual Linux kernel limits 204apply to the file descriptors. 205 206GEM handles are local to a DRM file. Applications get a handle to a GEM 207object through a driver-specific ioctl, and can use that handle to refer 208to the GEM object in other standard or driver-specific ioctls. Closing a 209DRM file handle frees all its GEM handles and dereferences the 210associated GEM objects. 211 212To create a handle for a GEM object drivers call drm_gem_handle_create(). The 213function takes a pointer to the DRM file and the GEM object and returns a 214locally unique handle. When the handle is no longer needed drivers delete it 215with a call to drm_gem_handle_delete(). Finally the GEM object associated with a 216handle can be retrieved by a call to drm_gem_object_lookup(). 217 218Handles don't take ownership of GEM objects, they only take a reference 219to the object that will be dropped when the handle is destroyed. To 220avoid leaking GEM objects, drivers must make sure they drop the 221reference(s) they own (such as the initial reference taken at object 222creation time) as appropriate, without any special consideration for the 223handle. For example, in the particular case of combined GEM object and 224handle creation in the implementation of the dumb_create operation, 225drivers must drop the initial reference to the GEM object before 226returning the handle. 227 228GEM names are similar in purpose to handles but are not local to DRM 229files. They can be passed between processes to reference a GEM object 230globally. Names can't be used directly to refer to objects in the DRM 231API, applications must convert handles to names and names to handles 232using the DRM_IOCTL_GEM_FLINK and DRM_IOCTL_GEM_OPEN ioctls 233respectively. The conversion is handled by the DRM core without any 234driver-specific support. 235 236GEM also supports buffer sharing with dma-buf file descriptors through 237PRIME. GEM-based drivers must use the provided helpers functions to 238implement the exporting and importing correctly. See ?. Since sharing 239file descriptors is inherently more secure than the easily guessable and 240global GEM names it is the preferred buffer sharing mechanism. Sharing 241buffers through GEM names is only supported for legacy userspace. 242Furthermore PRIME also allows cross-device buffer sharing since it is 243based on dma-bufs. 244 245GEM Objects Mapping 246------------------- 247 248Because mapping operations are fairly heavyweight GEM favours 249read/write-like access to buffers, implemented through driver-specific 250ioctls, over mapping buffers to userspace. However, when random access 251to the buffer is needed (to perform software rendering for instance), 252direct access to the object can be more efficient. 253 254The mmap system call can't be used directly to map GEM objects, as they 255don't have their own file handle. Two alternative methods currently 256co-exist to map GEM objects to userspace. The first method uses a 257driver-specific ioctl to perform the mapping operation, calling 258do_mmap() under the hood. This is often considered 259dubious, seems to be discouraged for new GEM-enabled drivers, and will 260thus not be described here. 261 262The second method uses the mmap system call on the DRM file handle. void 263\*mmap(void \*addr, size_t length, int prot, int flags, int fd, off_t 264offset); DRM identifies the GEM object to be mapped by a fake offset 265passed through the mmap offset argument. Prior to being mapped, a GEM 266object must thus be associated with a fake offset. To do so, drivers 267must call drm_gem_create_mmap_offset() on the object. 268 269Once allocated, the fake offset value must be passed to the application 270in a driver-specific way and can then be used as the mmap offset 271argument. 272 273The GEM core provides a helper method drm_gem_mmap() to 274handle object mapping. The method can be set directly as the mmap file 275operation handler. It will look up the GEM object based on the offset 276value and set the VMA operations to the :c:type:`struct drm_driver 277<drm_driver>` gem_vm_ops field. Note that drm_gem_mmap() doesn't map memory to 278userspace, but relies on the driver-provided fault handler to map pages 279individually. 280 281To use drm_gem_mmap(), drivers must fill the struct :c:type:`struct drm_driver 282<drm_driver>` gem_vm_ops field with a pointer to VM operations. 283 284The VM operations is a :c:type:`struct vm_operations_struct <vm_operations_struct>` 285made up of several fields, the more interesting ones being: 286 287.. code-block:: c 288 289 struct vm_operations_struct { 290 void (*open)(struct vm_area_struct * area); 291 void (*close)(struct vm_area_struct * area); 292 vm_fault_t (*fault)(struct vm_fault *vmf); 293 }; 294 295 296The open and close operations must update the GEM object reference 297count. Drivers can use the drm_gem_vm_open() and drm_gem_vm_close() helper 298functions directly as open and close handlers. 299 300The fault operation handler is responsible for mapping pages to 301userspace when a page fault occurs. Depending on the memory allocation 302scheme, drivers can allocate pages at fault time, or can decide to 303allocate memory for the GEM object at the time the object is created. 304 305Drivers that want to map the GEM object upfront instead of handling page 306faults can implement their own mmap file operation handler. 307 308In order to reduce page table overhead, if the internal shmem mountpoint 309"shm_mnt" is configured to use transparent huge pages (for builds with 310CONFIG_TRANSPARENT_HUGEPAGE enabled) and if the shmem backing store 311managed to allocate a huge page for a faulty address, the fault handler 312will first attempt to insert that huge page into the VMA before falling 313back to individual page insertion. mmap() user address alignment for GEM 314objects is handled by providing a custom get_unmapped_area file 315operation which forwards to the shmem backing store. For most drivers, 316which don't create a huge mountpoint by default or through a module 317parameter, transparent huge pages can be enabled by either setting the 318"transparent_hugepage_shmem" kernel parameter or the 319"/sys/kernel/mm/transparent_hugepage/shmem_enabled" sysfs knob. 320 321For platforms without MMU the GEM core provides a helper method 322drm_gem_dma_get_unmapped_area(). The mmap() routines will call this to get a 323proposed address for the mapping. 324 325To use drm_gem_dma_get_unmapped_area(), drivers must fill the struct 326:c:type:`struct file_operations <file_operations>` get_unmapped_area field with 327a pointer on drm_gem_dma_get_unmapped_area(). 328 329More detailed information about get_unmapped_area can be found in 330Documentation/admin-guide/mm/nommu-mmap.rst 331 332Memory Coherency 333---------------- 334 335When mapped to the device or used in a command buffer, backing pages for 336an object are flushed to memory and marked write combined so as to be 337coherent with the GPU. Likewise, if the CPU accesses an object after the 338GPU has finished rendering to the object, then the object must be made 339coherent with the CPU's view of memory, usually involving GPU cache 340flushing of various kinds. This core CPU<->GPU coherency management is 341provided by a device-specific ioctl, which evaluates an object's current 342domain and performs any necessary flushing or synchronization to put the 343object into the desired coherency domain (note that the object may be 344busy, i.e. an active render target; in that case, setting the domain 345blocks the client and waits for rendering to complete before performing 346any necessary flushing operations). 347 348Command Execution 349----------------- 350 351Perhaps the most important GEM function for GPU devices is providing a 352command execution interface to clients. Client programs construct 353command buffers containing references to previously allocated memory 354objects, and then submit them to GEM. At that point, GEM takes care to 355bind all the objects into the GTT, execute the buffer, and provide 356necessary synchronization between clients accessing the same buffers. 357This often involves evicting some objects from the GTT and re-binding 358others (a fairly expensive operation), and providing relocation support 359which hides fixed GTT offsets from clients. Clients must take care not 360to submit command buffers that reference more objects than can fit in 361the GTT; otherwise, GEM will reject them and no rendering will occur. 362Similarly, if several objects in the buffer require fence registers to 363be allocated for correct rendering (e.g. 2D blits on pre-965 chips), 364care must be taken not to require more fence registers than are 365available to the client. Such resource management should be abstracted 366from the client in libdrm. 367 368GEM Function Reference 369---------------------- 370 371.. kernel-doc:: include/drm/drm_gem.h 372 :internal: 373 374.. kernel-doc:: drivers/gpu/drm/drm_gem.c 375 :export: 376 377GEM DMA Helper Functions Reference 378---------------------------------- 379 380.. kernel-doc:: drivers/gpu/drm/drm_gem_dma_helper.c 381 :doc: dma helpers 382 383.. kernel-doc:: include/drm/drm_gem_dma_helper.h 384 :internal: 385 386.. kernel-doc:: drivers/gpu/drm/drm_gem_dma_helper.c 387 :export: 388 389GEM SHMEM Helper Function Reference 390----------------------------------- 391 392.. kernel-doc:: drivers/gpu/drm/drm_gem_shmem_helper.c 393 :doc: overview 394 395.. kernel-doc:: include/drm/drm_gem_shmem_helper.h 396 :internal: 397 398.. kernel-doc:: drivers/gpu/drm/drm_gem_shmem_helper.c 399 :export: 400 401GEM VRAM Helper Functions Reference 402----------------------------------- 403 404.. kernel-doc:: drivers/gpu/drm/drm_gem_vram_helper.c 405 :doc: overview 406 407.. kernel-doc:: include/drm/drm_gem_vram_helper.h 408 :internal: 409 410.. kernel-doc:: drivers/gpu/drm/drm_gem_vram_helper.c 411 :export: 412 413GEM TTM Helper Functions Reference 414----------------------------------- 415 416.. kernel-doc:: drivers/gpu/drm/drm_gem_ttm_helper.c 417 :doc: overview 418 419.. kernel-doc:: drivers/gpu/drm/drm_gem_ttm_helper.c 420 :export: 421 422VMA Offset Manager 423================== 424 425.. kernel-doc:: drivers/gpu/drm/drm_vma_manager.c 426 :doc: vma offset manager 427 428.. kernel-doc:: include/drm/drm_vma_manager.h 429 :internal: 430 431.. kernel-doc:: drivers/gpu/drm/drm_vma_manager.c 432 :export: 433 434.. _prime_buffer_sharing: 435 436PRIME Buffer Sharing 437==================== 438 439PRIME is the cross device buffer sharing framework in drm, originally 440created for the OPTIMUS range of multi-gpu platforms. To userspace PRIME 441buffers are dma-buf based file descriptors. 442 443Overview and Lifetime Rules 444--------------------------- 445 446.. kernel-doc:: drivers/gpu/drm/drm_prime.c 447 :doc: overview and lifetime rules 448 449PRIME Helper Functions 450---------------------- 451 452.. kernel-doc:: drivers/gpu/drm/drm_prime.c 453 :doc: PRIME Helpers 454 455PRIME Function References 456------------------------- 457 458.. kernel-doc:: include/drm/drm_prime.h 459 :internal: 460 461.. kernel-doc:: drivers/gpu/drm/drm_prime.c 462 :export: 463 464DRM MM Range Allocator 465====================== 466 467Overview 468-------- 469 470.. kernel-doc:: drivers/gpu/drm/drm_mm.c 471 :doc: Overview 472 473LRU Scan/Eviction Support 474------------------------- 475 476.. kernel-doc:: drivers/gpu/drm/drm_mm.c 477 :doc: lru scan roster 478 479DRM MM Range Allocator Function References 480------------------------------------------ 481 482.. kernel-doc:: include/drm/drm_mm.h 483 :internal: 484 485.. kernel-doc:: drivers/gpu/drm/drm_mm.c 486 :export: 487 488.. _drm_gpuvm: 489 490DRM GPUVM 491========= 492 493Overview 494-------- 495 496.. kernel-doc:: drivers/gpu/drm/drm_gpuvm.c 497 :doc: Overview 498 499Split and Merge 500--------------- 501 502.. kernel-doc:: drivers/gpu/drm/drm_gpuvm.c 503 :doc: Split and Merge 504 505.. _drm_gpuvm_locking: 506 507Locking 508------- 509 510.. kernel-doc:: drivers/gpu/drm/drm_gpuvm.c 511 :doc: Locking 512 513Examples 514-------- 515 516.. kernel-doc:: drivers/gpu/drm/drm_gpuvm.c 517 :doc: Examples 518 519DRM GPUVM Function References 520----------------------------- 521 522.. kernel-doc:: include/drm/drm_gpuvm.h 523 :internal: 524 525.. kernel-doc:: drivers/gpu/drm/drm_gpuvm.c 526 :export: 527 528DRM Buddy Allocator 529=================== 530 531Buddy Allocator Function References (GPU buddy) 532----------------------------------------------- 533 534.. kernel-doc:: drivers/gpu/buddy.c 535 :export: 536 537DRM Buddy Specific Logging Function References 538---------------------------------------------- 539 540.. kernel-doc:: drivers/gpu/drm/drm_buddy.c 541 :export: 542 543DRM Cache Handling and Fast WC memcpy() 544======================================= 545 546.. kernel-doc:: drivers/gpu/drm/drm_cache.c 547 :export: 548 549.. _drm_sync_objects: 550 551DRM Sync Objects 552================ 553 554.. kernel-doc:: drivers/gpu/drm/drm_syncobj.c 555 :doc: Overview 556 557.. kernel-doc:: include/drm/drm_syncobj.h 558 :internal: 559 560.. kernel-doc:: drivers/gpu/drm/drm_syncobj.c 561 :export: 562 563DRM Execution context 564===================== 565 566.. kernel-doc:: drivers/gpu/drm/drm_exec.c 567 :doc: Overview 568 569.. kernel-doc:: include/drm/drm_exec.h 570 :internal: 571 572.. kernel-doc:: drivers/gpu/drm/drm_exec.c 573 :export: 574 575GPU Scheduler 576============= 577 578Overview 579-------- 580 581.. kernel-doc:: drivers/gpu/drm/scheduler/sched_main.c 582 :doc: Overview 583 584Flow Control 585------------ 586 587.. kernel-doc:: drivers/gpu/drm/scheduler/sched_main.c 588 :doc: Flow Control 589 590Scheduler Function References 591----------------------------- 592 593.. kernel-doc:: include/drm/gpu_scheduler.h 594 :internal: 595 596.. kernel-doc:: drivers/gpu/drm/scheduler/sched_main.c 597 :export: 598 599.. kernel-doc:: drivers/gpu/drm/scheduler/sched_entity.c 600 :export: 601