1 /* SPDX-License-Identifier: MIT */ 2 /* 3 * Copyright © 2022 Intel Corporation 4 */ 5 6 #ifndef _XE_MIGRATE_DOC_H_ 7 #define _XE_MIGRATE_DOC_H_ 8 9 /** 10 * DOC: Migrate Layer 11 * 12 * The XE migrate layer is used generate jobs which can copy memory (eviction), 13 * clear memory, or program tables (binds). This layer exists in every GT, has 14 * a migrate engine, and uses a special VM for all generated jobs. 15 * 16 * Special VM details 17 * ================== 18 * 19 * The special VM is configured with a page structure where we can dynamically 20 * map BOs which need to be copied and cleared, dynamically map other VM's page 21 * table BOs for updates, and identity map the entire device's VRAM with 1 GB 22 * pages. 23 * 24 * Currently the page structure consists of 32 physical pages with 16 being 25 * reserved for BO mapping during copies and clear, 1 reserved for kernel binds, 26 * several pages are needed to setup the identity mappings (exact number based 27 * on how many bits of address space the device has), and the rest are reserved 28 * user bind operations. 29 * 30 * TODO: Diagram of layout 31 * 32 * Bind jobs 33 * ========= 34 * 35 * A bind job consist of two batches and runs either on the migrate engine 36 * (kernel binds) or the bind engine passed in (user binds). In both cases the 37 * VM of the engine is the migrate VM. 38 * 39 * The first batch is used to update the migration VM page structure to point to 40 * the bind VM page table BOs which need to be updated. A physical page is 41 * required for this. If it is a user bind, the page is allocated from pool of 42 * pages reserved user bind operations with drm_suballoc managing this pool. If 43 * it is a kernel bind, the page reserved for kernel binds is used. 44 * 45 * The first batch is only required for devices without VRAM as when the device 46 * has VRAM the bind VM page table BOs are in VRAM and the identity mapping can 47 * be used. 48 * 49 * The second batch is used to program page table updated in the bind VM. Why 50 * not just one batch? Well the TLBs need to be invalidated between these two 51 * batches and that only can be done from the ring. 52 * 53 * When the bind job complete, the page allocated is returned the pool of pages 54 * reserved for user bind operations if a user bind. No need do this for kernel 55 * binds as the reserved kernel page is serially used by each job. 56 * 57 * Copy / clear jobs 58 * ================= 59 * 60 * A copy or clear job consist of two batches and runs on the migrate engine. 61 * 62 * Like binds, the first batch is used update the migration VM page structure. 63 * In copy jobs, we need to map the source and destination of the BO into page 64 * the structure. In clear jobs, we just need to add 1 mapping of BO into the 65 * page structure. We use the 16 reserved pages in migration VM for mappings, 66 * this gives us a maximum copy size of 16 MB and maximum clear size of 32 MB. 67 * 68 * The second batch is used do either do the copy or clear. Again similar to 69 * binds, two batches are required as the TLBs need to be invalidated from the 70 * ring between the batches. 71 * 72 * More than one job will be generated if the BO is larger than maximum copy / 73 * clear size. 74 * 75 * Future work 76 * =========== 77 * 78 * Update copy and clear code to use identity mapped VRAM. 79 * 80 * Can we rework the use of the pages async binds to use all the entries in each 81 * page? 82 * 83 * Using large pages for sysmem mappings. 84 * 85 * Is it possible to identity map the sysmem? We should explore this. 86 */ 87 88 #endif 89