Lines Matching +full:mode +full:- +full:flag
1 .. SPDX-License-Identifier: GPL-2.0
20 XDP programs to redirect frames to a memory buffer in a user-space
59 corresponding two rings, sets the XDP_SHARED_UMEM flag in the bind
64 single-consumer / single-producer (for performance reasons), the new
72 user-space application can place an XSK at an arbitrary place in this
83 when loading the XDP program, XDP_SKB mode is employed that uses SKBs
85 space. A fallback mode that works for any network device. On the other
99 http://vger.kernel.org/lpc_net2018_talks/lpc18_paper_af_xdp_perf-v2.pdf. Do
106 ----
109 equal-sized frames. An UMEM is associated to a netdev and a specific
117 the next socket B can do this by setting the XDP_SHARED_UMEM flag in
121 The UMEM has two single-producer/single-consumer rings that are used
123 user-space application.
126 -----
129 TX. All rings are single-producer/single-consumer, so the user-space
145 calls and mmapped to user-space using the appropriate offset to mmap()
155 user-space to kernel-space. The UMEM addrs are passed in the ring. As
162 running the application with aligned chunk mode, the kernel will mask
166 chunks mode, then the incoming addr will be left untouched.
173 kernel-space to user-space. Just like the FILL ring, UMEM indices are
176 Frames passed from the kernel to user-space are frames that has been
177 sent (TX ring) and can be used by user-space again.
244 ------------------------------------
246 When you bind to a socket, the kernel will first try to use zero-copy
247 copy. If zero-copy is not supported, it will fall back on using copy
248 mode, i.e. copying all packets out to user space. But if you would
249 like to force a certain mode, you can use the following flags. If you
250 pass the XDP_COPY flag to the bind call, the kernel will force the
251 socket into copy mode. If it cannot use copy mode, the bind call will
252 fail with an error. Conversely, the XDP_ZEROCOPY flag will force the
253 socket into zero-copy mode or fail.
255 XDP_SHARED_UMEM bind flag
256 -------------------------
258 This flag enables you to bind multiple sockets to the same UMEM. It
260 netdevs/devices. In this mode, each socket has their own RX and TX
269 we have bound to. To use this mode, create the first socket and bind
280 round-robin example of distributing packets is shown below:
282 .. code-block:: c
300 rr = (rr + 1) & (MAX_SOCKS - 1);
311 Libbpf uses this mode if you create more than one socket tied to the
347 XDP_USE_NEED_WAKEUP bind flag
348 -----------------------------
350 This option adds support for a new flag called need_wakeup that is
353 need_wakeup flag will be set if the kernel needs to be explicitly
354 woken up by a syscall to continue processing packets. If the flag is
357 If the flag is set on the FILL ring, the application needs to call
363 need_wakeup flag is set so that user space can put buffers on the
367 If the flag is set for the TX ring, it means that the application
375 .. code-block:: c
380 I.e., only use the syscall if the flag is set.
382 We recommend that you always enable this mode as it usually leads to
389 ------------------------------------------------------
398 socket. But if the XDP_SHARED_UMEM flag is used, any socket after the
401 be used. Note, that the rings are single-producer single-consumer, so
405 In libbpf, you can create Rx-only and Tx-only sockets by supplying
409 If you create a Tx-only socket, we recommend that you do not put any
415 -----------------------
430 be dealt with in separate sections for each UMEM flag.
433 --------------------------
437 is created by a privileged process and passed to a non-privileged one.
442 --------------------------------
446 mode to allow application to tune the per-socket maximum iteration for
448 Allowed range is [32, xs->tx->nentries].
451 -------------------------
456 .. code-block:: c
465 ----------------------
468 XDP_OPTIONS_ZEROCOPY which tells you if zero-copy is on or not.
470 Multi-Buffer Support
473 With multi-buffer support, programs using AF_XDP sockets can receive
475 zero-copy mode. For example, a packet can consist of two
488 To enable multi-buffer support for an AF_XDP socket, use the new bind
489 flag XDP_USE_SG. If this is not provided, all multi-buffer packets
491 needs to be in multi-buffer mode. This can be accomplished by using
494 To represent a packet consisting of multiple frames, a new flag called
498 of the packet. Why the reverse logic of end-of-packet (eop) flag found
499 in many NICs? Just to preserve compatibility with non-multi-buffer
516 * For copy mode, the maximum supported number of frames in a packet is
523 * For zero-copy mode, the limit is up to what the NIC HW
526 CONFIG_MAX_SKB_FRAGS + 1) for zero-copy mode, as it would have
528 NIC supports. Kind of defeats the purpose of zero-copy mode. How to
529 probe for this limit is explained in the "probe for multi-buffer
532 On the Rx path in copy-mode, the xsk core copies the XDP data into
533 multiple descriptors, if needed, and sets the XDP_PKT_CONTD flag as
534 detailed before. Zero-copy mode works the same, though the data is not
536 flag set to one, it means that the packet consists of multiple buffers
551 An example program each for Rx and Tx multi-buffer support can be found
555 -----
557 In order to use AF_XDP sockets two parts are needed. The user-space
559 please refer to the xdp-project at
560 https://github.com/xdp-project/bpf-examples/tree/main/AF_XDP-example.
564 .. code-block:: c
568 int index = ctx->rx_queue_index;
581 .. code-block:: c
603 __u32 entries = *ring->producer - *ring->consumer;
606 return -1;
608 // read-barrier!
610 *item = ring->desc[*ring->consumer & (RING_SIZE - 1)];
611 (*ring->consumer)++;
617 u32 free_entries = RING_SIZE - (*ring->producer - *ring->consumer);
620 return -1;
622 ring->desc[*ring->producer & (RING_SIZE - 1)] = *item;
624 // write-barrier!
626 (*ring->producer)++;
633 Usage Multi-Buffer Rx
634 ---------------------
636 Here is a simple Rx path pseudo-code example (using libxdp interfaces
639 .. code-block:: c
647 int rcvd = xsk_ring_cons__peek(&xsk->rx, opt_batch_size, &idx_rx);
649 xsk_ring_prod__reserve(&xsk->umem->fq, rcvd, &idx_fq);
652 struct xdp_desc *desc = xsk_ring_cons__rx_desc(&xsk->rx, idx_rx++);
653 char *frag = xsk_umem__get_data(xsk->umem->buffer, desc->addr);
654 bool eop = !(desc->options & XDP_PKT_CONTD);
666 *xsk_ring_prod__fill_addr(&xsk->umem->fq, idx_fq++) = desc->addr;
669 xsk_ring_prod__submit(&xsk->umem->fq, rcvd);
670 xsk_ring_cons__release(&xsk->rx, rcvd);
673 Usage Multi-Buffer Tx
674 ---------------------
676 Here is an example Tx path pseudo-code (using libxdp interfaces for
681 .. code-block:: c
688 xsk_ring_prod__reserve(&xsk->tx, batch_size, &idx);
697 tx_desc = xsk_ring_prod__tx_desc(&xsk->tx, idx + i++);
698 tx_desc->addr = addr;
701 tx_desc->len = xsk_frame_size;
702 tx_desc->options = XDP_PKT_CONTD;
704 tx_desc->len = len;
705 tx_desc->options = 0;
708 len -= tx_desc->len;
720 xsk_ring_prod__submit(&xsk->tx, i);
723 Probing for Multi-Buffer Support
724 --------------------------------
726 To discover if a driver supports multi-buffer AF_XDP in SKB or DRV
727 mode, use the XDP_FEATURES feature of netlink in linux/netdev.h to
728 query for NETDEV_XDP_ACT_RX_SG support. This is the same flag as for
729 querying for XDP multi-buffer support. If XDP supports multi-buffer in
730 a driver, then AF_XDP will also support that in SKB and DRV mode.
732 To discover if a driver supports multi-buffer AF_XDP in zero-copy
733 mode, use XDP_FEATURES and first check the NETDEV_XDP_ACT_XSK_ZEROCOPY
734 flag. If it is set, it means that at least zero-copy is supported and
738 supported by this device in zero-copy mode. These are the possible
741 1: Multi-buffer for zero-copy is not supported by this device, as max
742 one fragment supported means that multi-buffer is not possible.
744 >=2: Multi-buffer is supported in zero-copy mode for this device. The
750 Multi-Buffer Support for Zero-Copy Drivers
751 ------------------------------------------
753 Zero-copy drivers usually use the batched APIs for Rx and Tx
756 to facilitate extending a zero-copy driver with multi-buffer support.
761 https://github.com/xdp-project/bpf-examples/tree/main/AF_XDP-example
767 ethtool -N p3p2 rx-flow-hash udp4 fn
768 ethtool -N p3p2 flow-type udp4 src-port 4242 dst-port 4242 \
771 Running the rxdrop benchmark in XDP_DRV mode can then be done
774 samples/bpf/xdpsock -i p3p2 -q 16 -r -N
776 For XDP_SKB mode, use the switch "-S" instead of "-N" and all options
777 can be displayed with "-h", as usual.
805 sudo ethtool -L <interface> combined 1
812 sudo ethtool -N <interface> rx-flow-hash udp4 fn
813 sudo ethtool -N <interface> flow-type udp4 src-port 4242 dst-port \
820 in copy mode?
827 to the same queue id Y. In zero-copy mode, you should use the
840 XDP_SHARED_UMEM flag.
845 - Björn Töpel (AF_XDP core)
846 - Magnus Karlsson (AF_XDP core)
847 - Alexander Duyck
848 - Alexei Starovoitov
849 - Daniel Borkmann
850 - Jesper Dangaard Brouer
851 - John Fastabend
852 - Jonathan Corbet (LWN coverage)
853 - Michael S. Tsirkin
854 - Qi Z Zhang
855 - Willem de Bruijn