1.. SPDX-License-Identifier: GPL-2.0 2 3Uacce (Unified/User-space-access-intended Accelerator Framework) 4================================================================ 5 6Introduction 7------------ 8 9Uacce (Unified/User-space-access-intended Accelerator Framework) targets to 10provide Shared Virtual Addressing (SVA) between accelerators and processes. 11So accelerator can access any data structure of the main cpu. 12This differs from the data sharing between cpu and io device, which share 13only data content rather than address. 14Because of the unified address, hardware and user space of process can 15share the same virtual address in the communication. 16Uacce takes the hardware accelerator as a heterogeneous processor, while 17IOMMU share the same CPU page tables and as a result the same translation 18from va to pa. 19 20:: 21 22 __________________________ __________________________ 23 | | | | 24 | User application (CPU) | | Hardware Accelerator | 25 |__________________________| |__________________________| 26 27 | | 28 | va | va 29 V V 30 __________ __________ 31 | | | | 32 | MMU | | IOMMU | 33 |__________| |__________| 34 | | 35 | | 36 V pa V pa 37 _______________________________________ 38 | | 39 | Memory | 40 |_______________________________________| 41 42 43 44Architecture 45------------ 46 47Uacce is the kernel module, taking charge of iommu and address sharing. 48The user drivers and libraries are called WarpDrive. 49 50The uacce device, built around the IOMMU SVA API, can access multiple 51address spaces, including the one without PASID. 52 53A virtual concept, queue, is used for the communication. It provides a 54FIFO-like interface. And it maintains a unified address space between the 55application and all involved hardware. 56 57:: 58 59 ___________________ ________________ 60 | | user API | | 61 | WarpDrive library | ------------> | user driver | 62 |___________________| |________________| 63 | | 64 | | 65 | queue fd | 66 | | 67 | | 68 v | 69 ___________________ _________ | 70 | | | | | mmap memory 71 | Other framework | | uacce | | r/w interface 72 | crypto/nic/others | |_________| | 73 |___________________| | 74 | | | 75 | register | register | 76 | | | 77 | | | 78 | _________________ __________ | 79 | | | | | | 80 ------------- | Device Driver | | IOMMU | | 81 |_________________| |__________| | 82 | | 83 | V 84 | ___________________ 85 | | | 86 -------------------------- | Device(Hardware) | 87 |___________________| 88 89 90How does it work 91---------------- 92 93Uacce uses mmap and IOMMU to play the trick. 94 95Uacce creates a chrdev for every device registered to it. New queue is 96created when user application open the chrdev. The file descriptor is used 97as the user handle of the queue. 98The accelerator device present itself as an Uacce object, which exports as 99a chrdev to the user space. The user application communicates with the 100hardware by ioctl (as control path) or share memory (as data path). 101 102The control path to the hardware is via file operation, while data path is 103via mmap space of the queue fd. 104 105The queue file address space: 106 107:: 108 109 /** 110 * enum uacce_qfrt: qfrt type 111 * @UACCE_QFRT_MMIO: device mmio region 112 * @UACCE_QFRT_DUS: device user share region 113 */ 114 enum uacce_qfrt { 115 UACCE_QFRT_MMIO = 0, 116 UACCE_QFRT_DUS = 1, 117 }; 118 119All regions are optional and differ from device type to type. 120Each region can be mmapped only once, otherwise -EEXIST returns. 121 122The device mmio region is mapped to the hardware mmio space. It is generally 123used for doorbell or other notification to the hardware. It is not fast enough 124as data channel. 125 126The device user share region is used for share data buffer between user process 127and device. 128 129 130The Uacce register API 131---------------------- 132 133The register API is defined in uacce.h. 134 135:: 136 137 struct uacce_interface { 138 char name[UACCE_MAX_NAME_SIZE]; 139 unsigned int flags; 140 const struct uacce_ops *ops; 141 }; 142 143According to the IOMMU capability, uacce_interface flags can be: 144 145:: 146 147 /** 148 * UACCE Device flags: 149 * UACCE_DEV_SVA: Shared Virtual Addresses 150 * Support PASID 151 * Support device page faults (PCI PRI or SMMU Stall) 152 */ 153 #define UACCE_DEV_SVA BIT(0) 154 155 struct uacce_device *uacce_alloc(struct device *parent, 156 struct uacce_interface *interface); 157 int uacce_register(struct uacce_device *uacce); 158 void uacce_remove(struct uacce_device *uacce); 159 160uacce_register results can be: 161 162a. If uacce module is not compiled, ERR_PTR(-ENODEV) 163 164b. Succeed with the desired flags 165 166c. Succeed with the negotiated flags, for example 167 168 uacce_interface.flags = UACCE_DEV_SVA but uacce->flags = ~UACCE_DEV_SVA 169 170 So user driver need check return value as well as the negotiated uacce->flags. 171 172 173The user driver 174--------------- 175 176The queue file mmap space will need a user driver to wrap the communication 177protocol. Uacce provides some attributes in sysfs for the user driver to 178match the right accelerator accordingly. 179More details in Documentation/ABI/testing/sysfs-driver-uacce. 180