1===================== 2DRM Memory Management 3===================== 4 5Modern Linux systems require large amount of graphics memory to store 6frame buffers, textures, vertices and other graphics-related data. Given 7the very dynamic nature of many of that data, managing graphics memory 8efficiently is thus crucial for the graphics stack and plays a central 9role in the DRM infrastructure. 10 11The DRM core includes two memory managers, namely Translation Table Manager 12(TTM) and Graphics Execution Manager (GEM). TTM was the first DRM memory 13manager to be developed and tried to be a one-size-fits-them all 14solution. It provides a single userspace API to accommodate the need of 15all hardware, supporting both Unified Memory Architecture (UMA) devices 16and devices with dedicated video RAM (i.e. most discrete video cards). 17This resulted in a large, complex piece of code that turned out to be 18hard to use for driver development. 19 20GEM started as an Intel-sponsored project in reaction to TTM's 21complexity. Its design philosophy is completely different: instead of 22providing a solution to every graphics memory-related problems, GEM 23identified common code between drivers and created a support library to 24share it. GEM has simpler initialization and execution requirements than 25TTM, but has no video RAM management capabilities and is thus limited to 26UMA devices. 27 28The Translation Table Manager (TTM) 29=================================== 30 31.. kernel-doc:: drivers/gpu/drm/ttm/ttm_module.c 32 :doc: TTM 33 34.. kernel-doc:: include/drm/ttm/ttm_caching.h 35 :internal: 36 37TTM device object reference 38--------------------------- 39 40.. kernel-doc:: include/drm/ttm/ttm_device.h 41 :internal: 42 43.. kernel-doc:: drivers/gpu/drm/ttm/ttm_device.c 44 :export: 45 46TTM resource placement reference 47-------------------------------- 48 49.. kernel-doc:: include/drm/ttm/ttm_placement.h 50 :internal: 51 52TTM resource object reference 53----------------------------- 54 55.. kernel-doc:: include/drm/ttm/ttm_resource.h 56 :internal: 57 58.. kernel-doc:: drivers/gpu/drm/ttm/ttm_resource.c 59 :export: 60 61TTM TT object reference 62----------------------- 63 64.. kernel-doc:: include/drm/ttm/ttm_tt.h 65 :internal: 66 67.. kernel-doc:: drivers/gpu/drm/ttm/ttm_tt.c 68 :export: 69 70TTM page pool reference 71----------------------- 72 73.. kernel-doc:: include/drm/ttm/ttm_pool.h 74 :internal: 75 76.. kernel-doc:: drivers/gpu/drm/ttm/ttm_pool.c 77 :export: 78 79The Graphics Execution Manager (GEM) 80==================================== 81 82The GEM design approach has resulted in a memory manager that doesn't 83provide full coverage of all (or even all common) use cases in its 84userspace or kernel API. GEM exposes a set of standard memory-related 85operations to userspace and a set of helper functions to drivers, and 86let drivers implement hardware-specific operations with their own 87private API. 88 89The GEM userspace API is described in the `GEM - the Graphics Execution 90Manager <http://lwn.net/Articles/283798/>`__ article on LWN. While 91slightly outdated, the document provides a good overview of the GEM API 92principles. Buffer allocation and read and write operations, described 93as part of the common GEM API, are currently implemented using 94driver-specific ioctls. 95 96GEM is data-agnostic. It manages abstract buffer objects without knowing 97what individual buffers contain. APIs that require knowledge of buffer 98contents or purpose, such as buffer allocation or synchronization 99primitives, are thus outside of the scope of GEM and must be implemented 100using driver-specific ioctls. 101 102On a fundamental level, GEM involves several operations: 103 104- Memory allocation and freeing 105- Command execution 106- Aperture management at command execution time 107 108Buffer object allocation is relatively straightforward and largely 109provided by Linux's shmem layer, which provides memory to back each 110object. 111 112Device-specific operations, such as command execution, pinning, buffer 113read & write, mapping, and domain ownership transfers are left to 114driver-specific ioctls. 115 116GEM Initialization 117------------------ 118 119Drivers that use GEM must set the DRIVER_GEM bit in the struct 120:c:type:`struct drm_driver <drm_driver>` driver_features 121field. The DRM core will then automatically initialize the GEM core 122before calling the load operation. Behind the scene, this will create a 123DRM Memory Manager object which provides an address space pool for 124object allocation. 125 126In a KMS configuration, drivers need to allocate and initialize a 127command ring buffer following core GEM initialization if required by the 128hardware. UMA devices usually have what is called a "stolen" memory 129region, which provides space for the initial framebuffer and large, 130contiguous memory regions required by the device. This space is 131typically not managed by GEM, and must be initialized separately into 132its own DRM MM object. 133 134GEM Objects Creation 135-------------------- 136 137GEM splits creation of GEM objects and allocation of the memory that 138backs them in two distinct operations. 139 140GEM objects are represented by an instance of struct :c:type:`struct 141drm_gem_object <drm_gem_object>`. Drivers usually need to 142extend GEM objects with private information and thus create a 143driver-specific GEM object structure type that embeds an instance of 144struct :c:type:`struct drm_gem_object <drm_gem_object>`. 145 146To create a GEM object, a driver allocates memory for an instance of its 147specific GEM object type and initializes the embedded struct 148:c:type:`struct drm_gem_object <drm_gem_object>` with a call 149to drm_gem_object_init(). The function takes a pointer 150to the DRM device, a pointer to the GEM object and the buffer object 151size in bytes. 152 153GEM uses shmem to allocate anonymous pageable memory. 154drm_gem_object_init() will create an shmfs file of the 155requested size and store it into the struct :c:type:`struct 156drm_gem_object <drm_gem_object>` filp field. The memory is 157used as either main storage for the object when the graphics hardware 158uses system memory directly or as a backing store otherwise. Drivers 159can call drm_gem_huge_mnt_create() to create, mount and use a huge 160shmem mountpoint instead of the default one ('shm_mnt'). For builds 161with CONFIG_TRANSPARENT_HUGEPAGE enabled, further calls to 162drm_gem_object_init() will let shmem allocate huge pages when 163possible. 164 165Drivers are responsible for the actual physical pages allocation by 166calling shmem_read_mapping_page_gfp() for each page. 167Note that they can decide to allocate pages when initializing the GEM 168object, or to delay allocation until the memory is needed (for instance 169when a page fault occurs as a result of a userspace memory access or 170when the driver needs to start a DMA transfer involving the memory). 171 172Anonymous pageable memory allocation is not always desired, for instance 173when the hardware requires physically contiguous system memory as is 174often the case in embedded devices. Drivers can create GEM objects with 175no shmfs backing (called private GEM objects) by initializing them with a call 176to drm_gem_private_object_init() instead of drm_gem_object_init(). Storage for 177private GEM objects must be managed by drivers. 178 179GEM Objects Lifetime 180-------------------- 181 182All GEM objects are reference-counted by the GEM core. References can be 183acquired and release by calling drm_gem_object_get() and drm_gem_object_put() 184respectively. 185 186When the last reference to a GEM object is released the GEM core calls 187the :c:type:`struct drm_gem_object_funcs <gem_object_funcs>` free 188operation. That operation is mandatory for GEM-enabled drivers and must 189free the GEM object and all associated resources. 190 191void (\*free) (struct drm_gem_object \*obj); Drivers are 192responsible for freeing all GEM object resources. This includes the 193resources created by the GEM core, which need to be released with 194drm_gem_object_release(). 195 196GEM Objects Naming 197------------------ 198 199Communication between userspace and the kernel refers to GEM objects 200using local handles, global names or, more recently, file descriptors. 201All of those are 32-bit integer values; the usual Linux kernel limits 202apply to the file descriptors. 203 204GEM handles are local to a DRM file. Applications get a handle to a GEM 205object through a driver-specific ioctl, and can use that handle to refer 206to the GEM object in other standard or driver-specific ioctls. Closing a 207DRM file handle frees all its GEM handles and dereferences the 208associated GEM objects. 209 210To create a handle for a GEM object drivers call drm_gem_handle_create(). The 211function takes a pointer to the DRM file and the GEM object and returns a 212locally unique handle. When the handle is no longer needed drivers delete it 213with a call to drm_gem_handle_delete(). Finally the GEM object associated with a 214handle can be retrieved by a call to drm_gem_object_lookup(). 215 216Handles don't take ownership of GEM objects, they only take a reference 217to the object that will be dropped when the handle is destroyed. To 218avoid leaking GEM objects, drivers must make sure they drop the 219reference(s) they own (such as the initial reference taken at object 220creation time) as appropriate, without any special consideration for the 221handle. For example, in the particular case of combined GEM object and 222handle creation in the implementation of the dumb_create operation, 223drivers must drop the initial reference to the GEM object before 224returning the handle. 225 226GEM names are similar in purpose to handles but are not local to DRM 227files. They can be passed between processes to reference a GEM object 228globally. Names can't be used directly to refer to objects in the DRM 229API, applications must convert handles to names and names to handles 230using the DRM_IOCTL_GEM_FLINK and DRM_IOCTL_GEM_OPEN ioctls 231respectively. The conversion is handled by the DRM core without any 232driver-specific support. 233 234GEM also supports buffer sharing with dma-buf file descriptors through 235PRIME. GEM-based drivers must use the provided helpers functions to 236implement the exporting and importing correctly. See ?. Since sharing 237file descriptors is inherently more secure than the easily guessable and 238global GEM names it is the preferred buffer sharing mechanism. Sharing 239buffers through GEM names is only supported for legacy userspace. 240Furthermore PRIME also allows cross-device buffer sharing since it is 241based on dma-bufs. 242 243GEM Objects Mapping 244------------------- 245 246Because mapping operations are fairly heavyweight GEM favours 247read/write-like access to buffers, implemented through driver-specific 248ioctls, over mapping buffers to userspace. However, when random access 249to the buffer is needed (to perform software rendering for instance), 250direct access to the object can be more efficient. 251 252The mmap system call can't be used directly to map GEM objects, as they 253don't have their own file handle. Two alternative methods currently 254co-exist to map GEM objects to userspace. The first method uses a 255driver-specific ioctl to perform the mapping operation, calling 256do_mmap() under the hood. This is often considered 257dubious, seems to be discouraged for new GEM-enabled drivers, and will 258thus not be described here. 259 260The second method uses the mmap system call on the DRM file handle. void 261\*mmap(void \*addr, size_t length, int prot, int flags, int fd, off_t 262offset); DRM identifies the GEM object to be mapped by a fake offset 263passed through the mmap offset argument. Prior to being mapped, a GEM 264object must thus be associated with a fake offset. To do so, drivers 265must call drm_gem_create_mmap_offset() on the object. 266 267Once allocated, the fake offset value must be passed to the application 268in a driver-specific way and can then be used as the mmap offset 269argument. 270 271The GEM core provides a helper method drm_gem_mmap() to 272handle object mapping. The method can be set directly as the mmap file 273operation handler. It will look up the GEM object based on the offset 274value and set the VMA operations to the :c:type:`struct drm_driver 275<drm_driver>` gem_vm_ops field. Note that drm_gem_mmap() doesn't map memory to 276userspace, but relies on the driver-provided fault handler to map pages 277individually. 278 279To use drm_gem_mmap(), drivers must fill the struct :c:type:`struct drm_driver 280<drm_driver>` gem_vm_ops field with a pointer to VM operations. 281 282The VM operations is a :c:type:`struct vm_operations_struct <vm_operations_struct>` 283made up of several fields, the more interesting ones being: 284 285.. code-block:: c 286 287 struct vm_operations_struct { 288 void (*open)(struct vm_area_struct * area); 289 void (*close)(struct vm_area_struct * area); 290 vm_fault_t (*fault)(struct vm_fault *vmf); 291 }; 292 293 294The open and close operations must update the GEM object reference 295count. Drivers can use the drm_gem_vm_open() and drm_gem_vm_close() helper 296functions directly as open and close handlers. 297 298The fault operation handler is responsible for mapping pages to 299userspace when a page fault occurs. Depending on the memory allocation 300scheme, drivers can allocate pages at fault time, or can decide to 301allocate memory for the GEM object at the time the object is created. 302 303Drivers that want to map the GEM object upfront instead of handling page 304faults can implement their own mmap file operation handler. 305 306In order to reduce page table overhead, if the internal shmem mountpoint 307"shm_mnt" is configured to use transparent huge pages (for builds with 308CONFIG_TRANSPARENT_HUGEPAGE enabled) and if the shmem backing store 309managed to allocate a huge page for a faulty address, the fault handler 310will first attempt to insert that huge page into the VMA before falling 311back to individual page insertion. mmap() user address alignment for GEM 312objects is handled by providing a custom get_unmapped_area file 313operation which forwards to the shmem backing store. For most drivers, 314which don't create a huge mountpoint by default or through a module 315parameter, transparent huge pages can be enabled by either setting the 316"transparent_hugepage_shmem" kernel parameter or the 317"/sys/kernel/mm/transparent_hugepage/shmem_enabled" sysfs knob. 318 319For platforms without MMU the GEM core provides a helper method 320drm_gem_dma_get_unmapped_area(). The mmap() routines will call this to get a 321proposed address for the mapping. 322 323To use drm_gem_dma_get_unmapped_area(), drivers must fill the struct 324:c:type:`struct file_operations <file_operations>` get_unmapped_area field with 325a pointer on drm_gem_dma_get_unmapped_area(). 326 327More detailed information about get_unmapped_area can be found in 328Documentation/admin-guide/mm/nommu-mmap.rst 329 330Memory Coherency 331---------------- 332 333When mapped to the device or used in a command buffer, backing pages for 334an object are flushed to memory and marked write combined so as to be 335coherent with the GPU. Likewise, if the CPU accesses an object after the 336GPU has finished rendering to the object, then the object must be made 337coherent with the CPU's view of memory, usually involving GPU cache 338flushing of various kinds. This core CPU<->GPU coherency management is 339provided by a device-specific ioctl, which evaluates an object's current 340domain and performs any necessary flushing or synchronization to put the 341object into the desired coherency domain (note that the object may be 342busy, i.e. an active render target; in that case, setting the domain 343blocks the client and waits for rendering to complete before performing 344any necessary flushing operations). 345 346Command Execution 347----------------- 348 349Perhaps the most important GEM function for GPU devices is providing a 350command execution interface to clients. Client programs construct 351command buffers containing references to previously allocated memory 352objects, and then submit them to GEM. At that point, GEM takes care to 353bind all the objects into the GTT, execute the buffer, and provide 354necessary synchronization between clients accessing the same buffers. 355This often involves evicting some objects from the GTT and re-binding 356others (a fairly expensive operation), and providing relocation support 357which hides fixed GTT offsets from clients. Clients must take care not 358to submit command buffers that reference more objects than can fit in 359the GTT; otherwise, GEM will reject them and no rendering will occur. 360Similarly, if several objects in the buffer require fence registers to 361be allocated for correct rendering (e.g. 2D blits on pre-965 chips), 362care must be taken not to require more fence registers than are 363available to the client. Such resource management should be abstracted 364from the client in libdrm. 365 366GEM Function Reference 367---------------------- 368 369.. kernel-doc:: include/drm/drm_gem.h 370 :internal: 371 372.. kernel-doc:: drivers/gpu/drm/drm_gem.c 373 :export: 374 375GEM DMA Helper Functions Reference 376---------------------------------- 377 378.. kernel-doc:: drivers/gpu/drm/drm_gem_dma_helper.c 379 :doc: dma helpers 380 381.. kernel-doc:: include/drm/drm_gem_dma_helper.h 382 :internal: 383 384.. kernel-doc:: drivers/gpu/drm/drm_gem_dma_helper.c 385 :export: 386 387GEM SHMEM Helper Function Reference 388----------------------------------- 389 390.. kernel-doc:: drivers/gpu/drm/drm_gem_shmem_helper.c 391 :doc: overview 392 393.. kernel-doc:: include/drm/drm_gem_shmem_helper.h 394 :internal: 395 396.. kernel-doc:: drivers/gpu/drm/drm_gem_shmem_helper.c 397 :export: 398 399GEM VRAM Helper Functions Reference 400----------------------------------- 401 402.. kernel-doc:: drivers/gpu/drm/drm_gem_vram_helper.c 403 :doc: overview 404 405.. kernel-doc:: include/drm/drm_gem_vram_helper.h 406 :internal: 407 408.. kernel-doc:: drivers/gpu/drm/drm_gem_vram_helper.c 409 :export: 410 411GEM TTM Helper Functions Reference 412----------------------------------- 413 414.. kernel-doc:: drivers/gpu/drm/drm_gem_ttm_helper.c 415 :doc: overview 416 417.. kernel-doc:: drivers/gpu/drm/drm_gem_ttm_helper.c 418 :export: 419 420VMA Offset Manager 421================== 422 423.. kernel-doc:: drivers/gpu/drm/drm_vma_manager.c 424 :doc: vma offset manager 425 426.. kernel-doc:: include/drm/drm_vma_manager.h 427 :internal: 428 429.. kernel-doc:: drivers/gpu/drm/drm_vma_manager.c 430 :export: 431 432.. _prime_buffer_sharing: 433 434PRIME Buffer Sharing 435==================== 436 437PRIME is the cross device buffer sharing framework in drm, originally 438created for the OPTIMUS range of multi-gpu platforms. To userspace PRIME 439buffers are dma-buf based file descriptors. 440 441Overview and Lifetime Rules 442--------------------------- 443 444.. kernel-doc:: drivers/gpu/drm/drm_prime.c 445 :doc: overview and lifetime rules 446 447PRIME Helper Functions 448---------------------- 449 450.. kernel-doc:: drivers/gpu/drm/drm_prime.c 451 :doc: PRIME Helpers 452 453PRIME Function References 454------------------------- 455 456.. kernel-doc:: include/drm/drm_prime.h 457 :internal: 458 459.. kernel-doc:: drivers/gpu/drm/drm_prime.c 460 :export: 461 462DRM MM Range Allocator 463====================== 464 465Overview 466-------- 467 468.. kernel-doc:: drivers/gpu/drm/drm_mm.c 469 :doc: Overview 470 471LRU Scan/Eviction Support 472------------------------- 473 474.. kernel-doc:: drivers/gpu/drm/drm_mm.c 475 :doc: lru scan roster 476 477DRM MM Range Allocator Function References 478------------------------------------------ 479 480.. kernel-doc:: include/drm/drm_mm.h 481 :internal: 482 483.. kernel-doc:: drivers/gpu/drm/drm_mm.c 484 :export: 485 486.. _drm_gpuvm: 487 488DRM GPUVM 489========= 490 491Overview 492-------- 493 494.. kernel-doc:: drivers/gpu/drm/drm_gpuvm.c 495 :doc: Overview 496 497Split and Merge 498--------------- 499 500.. kernel-doc:: drivers/gpu/drm/drm_gpuvm.c 501 :doc: Split and Merge 502 503.. _drm_gpuvm_locking: 504 505Locking 506------- 507 508.. kernel-doc:: drivers/gpu/drm/drm_gpuvm.c 509 :doc: Locking 510 511Examples 512-------- 513 514.. kernel-doc:: drivers/gpu/drm/drm_gpuvm.c 515 :doc: Examples 516 517DRM GPUVM Function References 518----------------------------- 519 520.. kernel-doc:: include/drm/drm_gpuvm.h 521 :internal: 522 523.. kernel-doc:: drivers/gpu/drm/drm_gpuvm.c 524 :export: 525 526DRM Buddy Allocator 527=================== 528 529DRM Buddy Function References 530----------------------------- 531 532.. kernel-doc:: drivers/gpu/drm/drm_buddy.c 533 :export: 534 535DRM Cache Handling and Fast WC memcpy() 536======================================= 537 538.. kernel-doc:: drivers/gpu/drm/drm_cache.c 539 :export: 540 541.. _drm_sync_objects: 542 543DRM Sync Objects 544================ 545 546.. kernel-doc:: drivers/gpu/drm/drm_syncobj.c 547 :doc: Overview 548 549.. kernel-doc:: include/drm/drm_syncobj.h 550 :internal: 551 552.. kernel-doc:: drivers/gpu/drm/drm_syncobj.c 553 :export: 554 555DRM Execution context 556===================== 557 558.. kernel-doc:: drivers/gpu/drm/drm_exec.c 559 :doc: Overview 560 561.. kernel-doc:: include/drm/drm_exec.h 562 :internal: 563 564.. kernel-doc:: drivers/gpu/drm/drm_exec.c 565 :export: 566 567GPU Scheduler 568============= 569 570Overview 571-------- 572 573.. kernel-doc:: drivers/gpu/drm/scheduler/sched_main.c 574 :doc: Overview 575 576Flow Control 577------------ 578 579.. kernel-doc:: drivers/gpu/drm/scheduler/sched_main.c 580 :doc: Flow Control 581 582Scheduler Function References 583----------------------------- 584 585.. kernel-doc:: include/drm/gpu_scheduler.h 586 :internal: 587 588.. kernel-doc:: drivers/gpu/drm/scheduler/sched_main.c 589 :export: 590 591.. kernel-doc:: drivers/gpu/drm/scheduler/sched_entity.c 592 :export: 593