Lines Matching +full:multi +full:- +full:system
1 .. SPDX-License-Identifier: GPL-2.0
20 XDP programs to redirect frames to a memory buffer in a user-space
64 single-consumer / single-producer (for performance reasons), the new
72 user-space application can place an XSK at an arbitrary place in this
99 http://vger.kernel.org/lpc_net2018_talks/lpc18_paper_af_xdp_perf-v2.pdf. Do
106 ----
109 equal-sized frames. An UMEM is associated to a netdev and a specific
112 system call. A UMEM is bound to a netdev and queue id, via the bind()
113 system call.
121 The UMEM has two single-producer/single-consumer rings that are used
123 user-space application.
126 -----
129 TX. All rings are single-producer/single-consumer, so the user-space
144 The rings are configured and created via the _RING setsockopt system
145 calls and mmapped to user-space using the appropriate offset to mmap()
155 user-space to kernel-space. The UMEM addrs are passed in the ring. As
173 kernel-space to user-space. Just like the FILL ring, UMEM indices are
176 Frames passed from the kernel to user-space are frames that has been
177 sent (TX ring) and can be used by user-space again.
201 To start the transfer a sendmsg() system call is required. This might
228 system call.
244 ------------------------------------
246 When you bind to a socket, the kernel will first try to use zero-copy
247 copy. If zero-copy is not supported, it will fall back on using copy
253 socket into zero-copy mode or fail.
256 -------------------------
280 round-robin example of distributing packets is shown below:
282 .. code-block:: c
300 rr = (rr + 1) & (MAX_SOCKS - 1);
348 -----------------------------
375 .. code-block:: c
389 ------------------------------------------------------
401 be used. Note, that the rings are single-producer single-consumer, so
405 In libbpf, you can create Rx-only and Tx-only sockets by supplying
409 If you create a Tx-only socket, we recommend that you do not put any
415 -----------------------
433 --------------------------
437 is created by a privileged process and passed to a non-privileged one.
442 --------------------------------
446 mode to allow application to tune the per-socket maximum iteration for
448 Allowed range is [32, xs->tx->nentries].
451 -------------------------
456 .. code-block:: c
465 ----------------------
468 XDP_OPTIONS_ZEROCOPY which tells you if zero-copy is on or not.
470 Multi-Buffer Support
473 With multi-buffer support, programs using AF_XDP sockets can receive
475 zero-copy mode. For example, a packet can consist of two
488 To enable multi-buffer support for an AF_XDP socket, use the new bind
489 flag XDP_USE_SG. If this is not provided, all multi-buffer packets
491 needs to be in multi-buffer mode. This can be accomplished by using
498 of the packet. Why the reverse logic of end-of-packet (eop) flag found
499 in many NICs? Just to preserve compatibility with non-multi-buffer
519 invalid. To produce an application that will work on any system
523 * For zero-copy mode, the limit is up to what the NIC HW
526 CONFIG_MAX_SKB_FRAGS + 1) for zero-copy mode, as it would have
528 NIC supports. Kind of defeats the purpose of zero-copy mode. How to
529 probe for this limit is explained in the "probe for multi-buffer
532 On the Rx path in copy-mode, the xsk core copies the XDP data into
534 detailed before. Zero-copy mode works the same, though the data is not
551 An example program each for Rx and Tx multi-buffer support can be found
555 -----
557 In order to use AF_XDP sockets two parts are needed. The user-space
559 please refer to the xdp-project at
560 https://github.com/xdp-project/bpf-examples/tree/main/AF_XDP-example.
564 .. code-block:: c
568 int index = ctx->rx_queue_index;
581 .. code-block:: c
603 __u32 entries = *ring->producer - *ring->consumer;
606 return -1;
608 // read-barrier!
610 *item = ring->desc[*ring->consumer & (RING_SIZE - 1)];
611 (*ring->consumer)++;
617 u32 free_entries = RING_SIZE - (*ring->producer - *ring->consumer);
620 return -1;
622 ring->desc[*ring->producer & (RING_SIZE - 1)] = *item;
624 // write-barrier!
626 (*ring->producer)++;
633 Usage Multi-Buffer Rx
634 ---------------------
636 Here is a simple Rx path pseudo-code example (using libxdp interfaces
639 .. code-block:: c
647 int rcvd = xsk_ring_cons__peek(&xsk->rx, opt_batch_size, &idx_rx);
649 xsk_ring_prod__reserve(&xsk->umem->fq, rcvd, &idx_fq);
652 struct xdp_desc *desc = xsk_ring_cons__rx_desc(&xsk->rx, idx_rx++);
653 char *frag = xsk_umem__get_data(xsk->umem->buffer, desc->addr);
654 bool eop = !(desc->options & XDP_PKT_CONTD);
666 *xsk_ring_prod__fill_addr(&xsk->umem->fq, idx_fq++) = desc->addr;
669 xsk_ring_prod__submit(&xsk->umem->fq, rcvd);
670 xsk_ring_cons__release(&xsk->rx, rcvd);
673 Usage Multi-Buffer Tx
674 ---------------------
676 Here is an example Tx path pseudo-code (using libxdp interfaces for
681 .. code-block:: c
688 xsk_ring_prod__reserve(&xsk->tx, batch_size, &idx);
697 tx_desc = xsk_ring_prod__tx_desc(&xsk->tx, idx + i++);
698 tx_desc->addr = addr;
701 tx_desc->len = xsk_frame_size;
702 tx_desc->options = XDP_PKT_CONTD;
704 tx_desc->len = len;
705 tx_desc->options = 0;
708 len -= tx_desc->len;
720 xsk_ring_prod__submit(&xsk->tx, i);
723 Probing for Multi-Buffer Support
724 --------------------------------
726 To discover if a driver supports multi-buffer AF_XDP in SKB or DRV
729 querying for XDP multi-buffer support. If XDP supports multi-buffer in
732 To discover if a driver supports multi-buffer AF_XDP in zero-copy
734 flag. If it is set, it means that at least zero-copy is supported and
738 supported by this device in zero-copy mode. These are the possible
741 1: Multi-buffer for zero-copy is not supported by this device, as max
742 one fragment supported means that multi-buffer is not possible.
744 >=2: Multi-buffer is supported in zero-copy mode for this device. The
750 Multi-Buffer Support for Zero-Copy Drivers
751 ------------------------------------------
753 Zero-copy drivers usually use the batched APIs for Rx and Tx
756 to facilitate extending a zero-copy driver with multi-buffer support.
761 https://github.com/xdp-project/bpf-examples/tree/main/AF_XDP-example
767 ethtool -N p3p2 rx-flow-hash udp4 fn
768 ethtool -N p3p2 flow-type udp4 src-port 4242 dst-port 4242 \
774 samples/bpf/xdpsock -i p3p2 -q 16 -r -N
776 For XDP_SKB mode, use the switch "-S" instead of "-N" and all options
777 can be displayed with "-h", as usual.
790 allocates one RX and TX queue pair per core. So on a 8 core system,
805 sudo ethtool -L <interface> combined 1
812 sudo ethtool -N <interface> rx-flow-hash udp4 fn
813 sudo ethtool -N <interface> flow-type udp4 src-port 4242 dst-port \
827 to the same queue id Y. In zero-copy mode, you should use the
845 - Björn Töpel (AF_XDP core)
846 - Magnus Karlsson (AF_XDP core)
847 - Alexander Duyck
848 - Alexei Starovoitov
849 - Daniel Borkmann
850 - Jesper Dangaard Brouer
851 - John Fastabend
852 - Jonathan Corbet (LWN coverage)
853 - Michael S. Tsirkin
854 - Qi Z Zhang
855 - Willem de Bruijn