1*1a7551f1SToke Høiland-Jørgensen.. SPDX-License-Identifier: GPL-2.0 2*1a7551f1SToke Høiland-Jørgensen 3*1a7551f1SToke Høiland-Jørgensen=================================== 4*1a7551f1SToke Høiland-JørgensenRunning BPF programs from userspace 5*1a7551f1SToke Høiland-Jørgensen=================================== 6*1a7551f1SToke Høiland-Jørgensen 7*1a7551f1SToke Høiland-JørgensenThis document describes the ``BPF_PROG_RUN`` facility for running BPF programs 8*1a7551f1SToke Høiland-Jørgensenfrom userspace. 9*1a7551f1SToke Høiland-Jørgensen 10*1a7551f1SToke Høiland-Jørgensen.. contents:: 11*1a7551f1SToke Høiland-Jørgensen :local: 12*1a7551f1SToke Høiland-Jørgensen :depth: 2 13*1a7551f1SToke Høiland-Jørgensen 14*1a7551f1SToke Høiland-Jørgensen 15*1a7551f1SToke Høiland-JørgensenOverview 16*1a7551f1SToke Høiland-Jørgensen-------- 17*1a7551f1SToke Høiland-Jørgensen 18*1a7551f1SToke Høiland-JørgensenThe ``BPF_PROG_RUN`` command can be used through the ``bpf()`` syscall to 19*1a7551f1SToke Høiland-Jørgensenexecute a BPF program in the kernel and return the results to userspace. This 20*1a7551f1SToke Høiland-Jørgensencan be used to unit test BPF programs against user-supplied context objects, and 21*1a7551f1SToke Høiland-Jørgensenas way to explicitly execute programs in the kernel for their side effects. The 22*1a7551f1SToke Høiland-Jørgensencommand was previously named ``BPF_PROG_TEST_RUN``, and both constants continue 23*1a7551f1SToke Høiland-Jørgensento be defined in the UAPI header, aliased to the same value. 24*1a7551f1SToke Høiland-Jørgensen 25*1a7551f1SToke Høiland-JørgensenThe ``BPF_PROG_RUN`` command can be used to execute BPF programs of the 26*1a7551f1SToke Høiland-Jørgensenfollowing types: 27*1a7551f1SToke Høiland-Jørgensen 28*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_SOCKET_FILTER`` 29*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_SCHED_CLS`` 30*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_SCHED_ACT`` 31*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_XDP`` 32*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_SK_LOOKUP`` 33*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_CGROUP_SKB`` 34*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_LWT_IN`` 35*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_LWT_OUT`` 36*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_LWT_XMIT`` 37*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_LWT_SEG6LOCAL`` 38*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_FLOW_DISSECTOR`` 39*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_STRUCT_OPS`` 40*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_RAW_TRACEPOINT`` 41*1a7551f1SToke Høiland-Jørgensen- ``BPF_PROG_TYPE_SYSCALL`` 42*1a7551f1SToke Høiland-Jørgensen 43*1a7551f1SToke Høiland-JørgensenWhen using the ``BPF_PROG_RUN`` command, userspace supplies an input context 44*1a7551f1SToke Høiland-Jørgensenobject and (for program types operating on network packets) a buffer containing 45*1a7551f1SToke Høiland-Jørgensenthe packet data that the BPF program will operate on. The kernel will then 46*1a7551f1SToke Høiland-Jørgensenexecute the program and return the results to userspace. Note that programs will 47*1a7551f1SToke Høiland-Jørgensennot have any side effects while being run in this mode; in particular, packets 48*1a7551f1SToke Høiland-Jørgensenwill not actually be redirected or dropped, the program return code will just be 49*1a7551f1SToke Høiland-Jørgensenreturned to userspace. A separate mode for live execution of XDP programs is 50*1a7551f1SToke Høiland-Jørgensenprovided, documented separately below. 51*1a7551f1SToke Høiland-Jørgensen 52*1a7551f1SToke Høiland-JørgensenRunning XDP programs in "live frame mode" 53*1a7551f1SToke Høiland-Jørgensen----------------------------------------- 54*1a7551f1SToke Høiland-Jørgensen 55*1a7551f1SToke Høiland-JørgensenThe ``BPF_PROG_RUN`` command has a separate mode for running live XDP programs, 56*1a7551f1SToke Høiland-Jørgensenwhich can be used to execute XDP programs in a way where packets will actually 57*1a7551f1SToke Høiland-Jørgensenbe processed by the kernel after the execution of the XDP program as if they 58*1a7551f1SToke Høiland-Jørgensenarrived on a physical interface. This mode is activated by setting the 59*1a7551f1SToke Høiland-Jørgensen``BPF_F_TEST_XDP_LIVE_FRAMES`` flag when supplying an XDP program to 60*1a7551f1SToke Høiland-Jørgensen``BPF_PROG_RUN``. 61*1a7551f1SToke Høiland-Jørgensen 62*1a7551f1SToke Høiland-JørgensenThe live packet mode is optimised for high performance execution of the supplied 63*1a7551f1SToke Høiland-JørgensenXDP program many times (suitable for, e.g., running as a traffic generator), 64*1a7551f1SToke Høiland-Jørgensenwhich means the semantics are not quite as straight-forward as the regular test 65*1a7551f1SToke Høiland-Jørgensenrun mode. Specifically: 66*1a7551f1SToke Høiland-Jørgensen 67*1a7551f1SToke Høiland-Jørgensen- When executing an XDP program in live frame mode, the result of the execution 68*1a7551f1SToke Høiland-Jørgensen will not be returned to userspace; instead, the kernel will perform the 69*1a7551f1SToke Høiland-Jørgensen operation indicated by the program's return code (drop the packet, redirect 70*1a7551f1SToke Høiland-Jørgensen it, etc). For this reason, setting the ``data_out`` or ``ctx_out`` attributes 71*1a7551f1SToke Høiland-Jørgensen in the syscall parameters when running in this mode will be rejected. In 72*1a7551f1SToke Høiland-Jørgensen addition, not all failures will be reported back to userspace directly; 73*1a7551f1SToke Høiland-Jørgensen specifically, only fatal errors in setup or during execution (like memory 74*1a7551f1SToke Høiland-Jørgensen allocation errors) will halt execution and return an error. If an error occurs 75*1a7551f1SToke Høiland-Jørgensen in packet processing, like a failure to redirect to a given interface, 76*1a7551f1SToke Høiland-Jørgensen execution will continue with the next repetition; these errors can be detected 77*1a7551f1SToke Høiland-Jørgensen via the same trace points as for regular XDP programs. 78*1a7551f1SToke Høiland-Jørgensen 79*1a7551f1SToke Høiland-Jørgensen- Userspace can supply an ifindex as part of the context object, just like in 80*1a7551f1SToke Høiland-Jørgensen the regular (non-live) mode. The XDP program will be executed as though the 81*1a7551f1SToke Høiland-Jørgensen packet arrived on this interface; i.e., the ``ingress_ifindex`` of the context 82*1a7551f1SToke Høiland-Jørgensen object will point to that interface. Furthermore, if the XDP program returns 83*1a7551f1SToke Høiland-Jørgensen ``XDP_PASS``, the packet will be injected into the kernel networking stack as 84*1a7551f1SToke Høiland-Jørgensen though it arrived on that ifindex, and if it returns ``XDP_TX``, the packet 85*1a7551f1SToke Høiland-Jørgensen will be transmitted *out* of that same interface. Do note, though, that 86*1a7551f1SToke Høiland-Jørgensen because the program execution is not happening in driver context, an 87*1a7551f1SToke Høiland-Jørgensen ``XDP_TX`` is actually turned into the same action as an ``XDP_REDIRECT`` to 88*1a7551f1SToke Høiland-Jørgensen that same interface (i.e., it will only work if the driver has support for the 89*1a7551f1SToke Høiland-Jørgensen ``ndo_xdp_xmit`` driver op). 90*1a7551f1SToke Høiland-Jørgensen 91*1a7551f1SToke Høiland-Jørgensen- When running the program with multiple repetitions, the execution will happen 92*1a7551f1SToke Høiland-Jørgensen in batches. The batch size defaults to 64 packets (which is same as the 93*1a7551f1SToke Høiland-Jørgensen maximum NAPI receive batch size), but can be specified by userspace through 94*1a7551f1SToke Høiland-Jørgensen the ``batch_size`` parameter, up to a maximum of 256 packets. For each batch, 95*1a7551f1SToke Høiland-Jørgensen the kernel executes the XDP program repeatedly, each invocation getting a 96*1a7551f1SToke Høiland-Jørgensen separate copy of the packet data. For each repetition, if the program drops 97*1a7551f1SToke Høiland-Jørgensen the packet, the data page is immediately recycled (see below). Otherwise, the 98*1a7551f1SToke Høiland-Jørgensen packet is buffered until the end of the batch, at which point all packets 99*1a7551f1SToke Høiland-Jørgensen buffered this way during the batch are transmitted at once. 100*1a7551f1SToke Høiland-Jørgensen 101*1a7551f1SToke Høiland-Jørgensen- When setting up the test run, the kernel will initialise a pool of memory 102*1a7551f1SToke Høiland-Jørgensen pages of the same size as the batch size. Each memory page will be initialised 103*1a7551f1SToke Høiland-Jørgensen with the initial packet data supplied by userspace at ``BPF_PROG_RUN`` 104*1a7551f1SToke Høiland-Jørgensen invocation. When possible, the pages will be recycled on future program 105*1a7551f1SToke Høiland-Jørgensen invocations, to improve performance. Pages will generally be recycled a full 106*1a7551f1SToke Høiland-Jørgensen batch at a time, except when a packet is dropped (by return code or because 107*1a7551f1SToke Høiland-Jørgensen of, say, a redirection error), in which case that page will be recycled 108*1a7551f1SToke Høiland-Jørgensen immediately. If a packet ends up being passed to the regular networking stack 109*1a7551f1SToke Høiland-Jørgensen (because the XDP program returns ``XDP_PASS``, or because it ends up being 110*1a7551f1SToke Høiland-Jørgensen redirected to an interface that injects it into the stack), the page will be 111*1a7551f1SToke Høiland-Jørgensen released and a new one will be allocated when the pool is empty. 112*1a7551f1SToke Høiland-Jørgensen 113*1a7551f1SToke Høiland-Jørgensen When recycling, the page content is not rewritten; only the packet boundary 114*1a7551f1SToke Høiland-Jørgensen pointers (``data``, ``data_end`` and ``data_meta``) in the context object will 115*1a7551f1SToke Høiland-Jørgensen be reset to the original values. This means that if a program rewrites the 116*1a7551f1SToke Høiland-Jørgensen packet contents, it has to be prepared to see either the original content or 117*1a7551f1SToke Høiland-Jørgensen the modified version on subsequent invocations. 118