1.. SPDX-License-Identifier: GPL-2.0 2 3=========================== 4SMB Direct - SMB3 over RDMA 5=========================== 6 7This document describes how to set up the Linux SMB client and server to 8use RDMA. 9 10Overview 11======== 12The Linux SMB kernel client supports SMB Direct, which is a transport 13scheme for SMB3 that uses RDMA (Remote Direct Memory Access) to provide 14high throughput and low latencies by bypassing the traditional TCP/IP 15stack. 16SMB Direct on the Linux SMB client can be tested against KSMBD - a 17kernel-space SMB server. 18 19Installation 20============= 21- Install an RDMA device. As long as the RDMA device driver is supported 22 by the kernel, it should work. This includes both software emulators (soft 23 RoCE, soft iWARP) and hardware devices (InfiniBand, RoCE, iWARP). 24 25- Install a kernel with SMB Direct support. The first kernel release to 26 support SMB Direct on both the client and server side is 5.15. Therefore, 27 a distribution compatible with kernel 5.15 or later is required. 28 29- Install cifs-utils, which provides the `mount.cifs` command to mount SMB 30 shares. 31 32- Configure the RDMA stack 33 34 Make sure that your kernel configuration has RDMA support enabled. Under 35 Device Drivers -> Infiniband support, update the kernel configuration to 36 enable Infiniband support. 37 38 Enable the appropriate IB HCA support or iWARP adapter support, 39 depending on your hardware. 40 41 If you are using InfiniBand, enable IP-over-InfiniBand support. 42 43 For soft RDMA, enable either the soft iWARP (`RDMA _SIW`) or soft RoCE 44 (`RDMA_RXE`) module. Install the `iproute2` package and use the 45 `rdma link add` command to load the module and create an 46 RDMA interface. 47 48 e.g. if your local ethernet interface is `eth0`, you can use: 49 50 .. code-block:: bash 51 52 sudo rdma link add siw0 type siw netdev eth0 53 54- Enable SMB Direct support for both the server and the client in the kernel 55 configuration. 56 57 Server Setup 58 59 .. code-block:: text 60 61 Network File Systems ---> 62 <M> SMB3 server support 63 [*] Support for SMB Direct protocol 64 65 Client Setup 66 67 .. code-block:: text 68 69 Network File Systems ---> 70 <M> SMB3 and CIFS support (advanced network filesystem) 71 [*] SMB Direct support 72 73- Build and install the kernel. SMB Direct support will be enabled in the 74 cifs.ko and ksmbd.ko modules. 75 76Setup and Usage 77================ 78 79- Set up and start a KSMBD server as described in the `KSMBD documentation 80 <https://www.kernel.org/doc/Documentation/filesystems/smb/ksmbd.rst>`_. 81 Also add the "server multi channel support = yes" parameter to ksmbd.conf. 82 83- On the client, mount the share with `rdma` mount option to use SMB Direct 84 (specify a SMB version 3.0 or higher using `vers`). 85 86 For example: 87 88 .. code-block:: bash 89 90 mount -t cifs //server/share /mnt/point -o vers=3.1.1,rdma 91 92- To verify that the mount is using SMB Direct, you can check dmesg for the 93 following log line after mounting: 94 95 .. code-block:: text 96 97 CIFS: VFS: RDMA transport established 98 99 Or, verify `rdma` mount option for the share in `/proc/mounts`: 100 101 .. code-block:: bash 102 103 cat /proc/mounts | grep cifs 104