1.. SPDX-License-Identifier: GPL-2.0 2 3======================================= 4FUSE-over-io-uring design documentation 5======================================= 6 7This documentation covers basic details how the fuse 8kernel/userspace communication through io-uring is configured 9and works. For generic details about FUSE see fuse.rst. 10 11This document also covers the current interface, which is 12still in development and might change. 13 14Limitations 15=========== 16As of now not all requests types are supported through io-uring, userspace 17is required to also handle requests through /dev/fuse after io-uring setup 18is complete. Specifically notifications (initiated from the daemon side) 19and interrupts. 20 21Fuse io-uring configuration 22=========================== 23 24Fuse kernel requests are queued through the classical /dev/fuse 25read/write interface - until io-uring setup is complete. 26 27In order to set up fuse-over-io-uring fuse-server (user-space) 28needs to submit SQEs (opcode = IORING_OP_URING_CMD) to the /dev/fuse 29connection file descriptor. Initial submit is with the sub command 30FUSE_URING_REQ_REGISTER, which will just register entries to be 31available in the kernel. 32 33Once at least one entry per queue is submitted, kernel starts 34to enqueue to ring queues. 35Note, every CPU core has its own fuse-io-uring queue. 36Userspace handles the CQE/fuse-request and submits the result as 37subcommand FUSE_URING_REQ_COMMIT_AND_FETCH - kernel completes 38the requests and also marks the entry available again. If there are 39pending requests waiting the request will be immediately submitted 40to the daemon again. 41 42Initial SQE 43-----------:: 44 45 | | FUSE filesystem daemon 46 | | 47 | | >io_uring_submit() 48 | | IORING_OP_URING_CMD / 49 | | FUSE_URING_CMD_REGISTER 50 | | [wait cqe] 51 | | >io_uring_wait_cqe() or 52 | | >io_uring_submit_and_wait() 53 | | 54 | >fuse_uring_cmd() | 55 | >fuse_uring_register() | 56 57 58Sending requests with CQEs 59--------------------------:: 60 61 | | FUSE filesystem daemon 62 | | [waiting for CQEs] 63 | "rm /mnt/fuse/file" | 64 | | 65 | >sys_unlink() | 66 | >fuse_unlink() | 67 | [allocate request] | 68 | >fuse_send_one() | 69 | ... | 70 | >fuse_uring_queue_fuse_req | 71 | [queue request on fg queue] | 72 | >fuse_uring_add_req_to_ring_ent() | 73 | ... | 74 | >fuse_uring_copy_to_ring() | 75 | >io_uring_cmd_done() | 76 | >request_wait_answer() | 77 | [sleep on req->waitq] | 78 | | [receives and handles CQE] 79 | | [submit result and fetch next] 80 | | >io_uring_submit() 81 | | IORING_OP_URING_CMD/ 82 | | FUSE_URING_CMD_COMMIT_AND_FETCH 83 | >fuse_uring_cmd() | 84 | >fuse_uring_commit_fetch() | 85 | >fuse_uring_commit() | 86 | >fuse_uring_copy_from_ring() | 87 | [ copy the result to the fuse req] | 88 | >fuse_uring_req_end() | 89 | >fuse_request_end() | 90 | [wake up req->waitq] | 91 | >fuse_uring_next_fuse_req | 92 | [wait or handle next req] | 93 | | 94 | [req->waitq woken up] | 95 | <fuse_unlink() | 96 | <sys_unlink() | 97 98 99 100