1.. SPDX-License-Identifier: GPL-2.0-only 2.. Copyright Red Hat 3 4============================================== 5BPF_MAP_TYPE_SOCKMAP and BPF_MAP_TYPE_SOCKHASH 6============================================== 7 8.. note:: 9 - ``BPF_MAP_TYPE_SOCKMAP`` was introduced in kernel version 4.14 10 - ``BPF_MAP_TYPE_SOCKHASH`` was introduced in kernel version 4.18 11 12``BPF_MAP_TYPE_SOCKMAP`` and ``BPF_MAP_TYPE_SOCKHASH`` maps can be used to 13redirect skbs between sockets or to apply policy at the socket level based on 14the result of a BPF (verdict) program with the help of the BPF helpers 15``bpf_sk_redirect_map()``, ``bpf_sk_redirect_hash()``, 16``bpf_msg_redirect_map()`` and ``bpf_msg_redirect_hash()``. 17 18``BPF_MAP_TYPE_SOCKMAP`` is backed by an array that uses an integer key as the 19index to look up a reference to a ``struct sock``. The map values are socket 20descriptors. Similarly, ``BPF_MAP_TYPE_SOCKHASH`` is a hash backed BPF map that 21holds references to sockets via their socket descriptors. 22 23.. note:: 24 The value type is either __u32 or __u64; the latter (__u64) is to support 25 returning socket cookies to userspace. Returning the ``struct sock *`` that 26 the map holds to user-space is neither safe nor useful. 27 28These maps may have BPF programs attached to them, specifically a parser program 29and a verdict program. The parser program determines how much data has been 30parsed and therefore how much data needs to be queued to come to a verdict. The 31verdict program is essentially the redirect program and can return a verdict 32of ``__SK_DROP``, ``__SK_PASS``, or ``__SK_REDIRECT``. 33 34When a socket is inserted into one of these maps, its socket callbacks are 35replaced and a ``struct sk_psock`` is attached to it. Additionally, this 36``sk_psock`` inherits the programs that are attached to the map. 37 38A sock object may be in multiple maps, but can only inherit a single 39parse or verdict program. If adding a sock object to a map would result 40in having multiple parser programs the update will return an EBUSY error. 41 42The supported programs to attach to these maps are: 43 44.. code-block:: c 45 46 struct sk_psock_progs { 47 struct bpf_prog *msg_parser; 48 struct bpf_prog *stream_parser; 49 struct bpf_prog *stream_verdict; 50 struct bpf_prog *skb_verdict; 51 }; 52 53.. note:: 54 Users are not allowed to attach ``stream_verdict`` and ``skb_verdict`` 55 programs to the same map. 56 57The attach types for the map programs are: 58 59- ``msg_parser`` program - ``BPF_SK_MSG_VERDICT``. 60- ``stream_parser`` program - ``BPF_SK_SKB_STREAM_PARSER``. 61- ``stream_verdict`` program - ``BPF_SK_SKB_STREAM_VERDICT``. 62- ``skb_verdict`` program - ``BPF_SK_SKB_VERDICT``. 63 64There are additional helpers available to use with the parser and verdict 65programs: ``bpf_msg_apply_bytes()`` and ``bpf_msg_cork_bytes()``. With 66``bpf_msg_apply_bytes()`` BPF programs can tell the infrastructure how many 67bytes the given verdict should apply to. The helper ``bpf_msg_cork_bytes()`` 68handles a different case where a BPF program cannot reach a verdict on a msg 69until it receives more bytes AND the program doesn't want to forward the packet 70until it is known to be good. 71 72Finally, the helpers ``bpf_msg_pull_data()`` and ``bpf_msg_push_data()`` are 73available to ``BPF_PROG_TYPE_SK_MSG`` BPF programs to pull in data and set the 74start and end pointers to given values or to add metadata to the ``struct 75sk_msg_buff *msg``. 76 77All these helpers will be described in more detail below. 78 79Usage 80===== 81Kernel BPF 82---------- 83bpf_msg_redirect_map() 84^^^^^^^^^^^^^^^^^^^^^^ 85.. code-block:: c 86 87 long bpf_msg_redirect_map(struct sk_msg_buff *msg, struct bpf_map *map, u32 key, u64 flags) 88 89This helper is used in programs implementing policies at the socket level. If 90the message ``msg`` is allowed to pass (i.e., if the verdict BPF program 91returns ``SK_PASS``), redirect it to the socket referenced by ``map`` (of type 92``BPF_MAP_TYPE_SOCKMAP``) at index ``key``. Both ingress and egress interfaces 93can be used for redirection. The ``BPF_F_INGRESS`` value in ``flags`` is used 94to select the ingress path otherwise the egress path is selected. This is the 95only flag supported for now. 96 97Returns ``SK_PASS`` on success, or ``SK_DROP`` on error. 98 99bpf_sk_redirect_map() 100^^^^^^^^^^^^^^^^^^^^^ 101.. code-block:: c 102 103 long bpf_sk_redirect_map(struct sk_buff *skb, struct bpf_map *map, u32 key u64 flags) 104 105Redirect the packet to the socket referenced by ``map`` (of type 106``BPF_MAP_TYPE_SOCKMAP``) at index ``key``. Both ingress and egress interfaces 107can be used for redirection. The ``BPF_F_INGRESS`` value in ``flags`` is used 108to select the ingress path otherwise the egress path is selected. This is the 109only flag supported for now. 110 111Returns ``SK_PASS`` on success, or ``SK_DROP`` on error. 112 113bpf_map_lookup_elem() 114^^^^^^^^^^^^^^^^^^^^^ 115.. code-block:: c 116 117 void *bpf_map_lookup_elem(struct bpf_map *map, const void *key) 118 119socket entries of type ``struct sock *`` can be retrieved using the 120``bpf_map_lookup_elem()`` helper. 121 122bpf_sock_map_update() 123^^^^^^^^^^^^^^^^^^^^^ 124.. code-block:: c 125 126 long bpf_sock_map_update(struct bpf_sock_ops *skops, struct bpf_map *map, void *key, u64 flags) 127 128Add an entry to, or update a ``map`` referencing sockets. The ``skops`` is used 129as a new value for the entry associated to ``key``. The ``flags`` argument can 130be one of the following: 131 132- ``BPF_ANY``: Create a new element or update an existing element. 133- ``BPF_NOEXIST``: Create a new element only if it did not exist. 134- ``BPF_EXIST``: Update an existing element. 135 136If the ``map`` has BPF programs (parser and verdict), those will be inherited 137by the socket being added. If the socket is already attached to BPF programs, 138this results in an error. 139 140Returns 0 on success, or a negative error in case of failure. 141 142bpf_sock_hash_update() 143^^^^^^^^^^^^^^^^^^^^^^ 144.. code-block:: c 145 146 long bpf_sock_hash_update(struct bpf_sock_ops *skops, struct bpf_map *map, void *key, u64 flags) 147 148Add an entry to, or update a sockhash ``map`` referencing sockets. The ``skops`` 149is used as a new value for the entry associated to ``key``. 150 151The ``flags`` argument can be one of the following: 152 153- ``BPF_ANY``: Create a new element or update an existing element. 154- ``BPF_NOEXIST``: Create a new element only if it did not exist. 155- ``BPF_EXIST``: Update an existing element. 156 157If the ``map`` has BPF programs (parser and verdict), those will be inherited 158by the socket being added. If the socket is already attached to BPF programs, 159this results in an error. 160 161Returns 0 on success, or a negative error in case of failure. 162 163bpf_msg_redirect_hash() 164^^^^^^^^^^^^^^^^^^^^^^^ 165.. code-block:: c 166 167 long bpf_msg_redirect_hash(struct sk_msg_buff *msg, struct bpf_map *map, void *key, u64 flags) 168 169This helper is used in programs implementing policies at the socket level. If 170the message ``msg`` is allowed to pass (i.e., if the verdict BPF program returns 171``SK_PASS``), redirect it to the socket referenced by ``map`` (of type 172``BPF_MAP_TYPE_SOCKHASH``) using hash ``key``. Both ingress and egress 173interfaces can be used for redirection. The ``BPF_F_INGRESS`` value in 174``flags`` is used to select the ingress path otherwise the egress path is 175selected. This is the only flag supported for now. 176 177Returns ``SK_PASS`` on success, or ``SK_DROP`` on error. 178 179bpf_sk_redirect_hash() 180^^^^^^^^^^^^^^^^^^^^^^ 181.. code-block:: c 182 183 long bpf_sk_redirect_hash(struct sk_buff *skb, struct bpf_map *map, void *key, u64 flags) 184 185This helper is used in programs implementing policies at the skb socket level. 186If the sk_buff ``skb`` is allowed to pass (i.e., if the verdict BPF program 187returns ``SK_PASS``), redirect it to the socket referenced by ``map`` (of type 188``BPF_MAP_TYPE_SOCKHASH``) using hash ``key``. Both ingress and egress 189interfaces can be used for redirection. The ``BPF_F_INGRESS`` value in 190``flags`` is used to select the ingress path otherwise the egress path is 191selected. This is the only flag supported for now. 192 193Returns ``SK_PASS`` on success, or ``SK_DROP`` on error. 194 195bpf_msg_apply_bytes() 196^^^^^^^^^^^^^^^^^^^^^^ 197.. code-block:: c 198 199 long bpf_msg_apply_bytes(struct sk_msg_buff *msg, u32 bytes) 200 201For socket policies, apply the verdict of the BPF program to the next (number 202of ``bytes``) of message ``msg``. For example, this helper can be used in the 203following cases: 204 205- A single ``sendmsg()`` or ``sendfile()`` system call contains multiple 206 logical messages that the BPF program is supposed to read and for which it 207 should apply a verdict. 208- A BPF program only cares to read the first ``bytes`` of a ``msg``. If the 209 message has a large payload, then setting up and calling the BPF program 210 repeatedly for all bytes, even though the verdict is already known, would 211 create unnecessary overhead. 212 213Returns 0 214 215bpf_msg_cork_bytes() 216^^^^^^^^^^^^^^^^^^^^^^ 217.. code-block:: c 218 219 long bpf_msg_cork_bytes(struct sk_msg_buff *msg, u32 bytes) 220 221For socket policies, prevent the execution of the verdict BPF program for 222message ``msg`` until the number of ``bytes`` have been accumulated. 223 224This can be used when one needs a specific number of bytes before a verdict can 225be assigned, even if the data spans multiple ``sendmsg()`` or ``sendfile()`` 226calls. 227 228Returns 0 229 230bpf_msg_pull_data() 231^^^^^^^^^^^^^^^^^^^^^^ 232.. code-block:: c 233 234 long bpf_msg_pull_data(struct sk_msg_buff *msg, u32 start, u32 end, u64 flags) 235 236For socket policies, pull in non-linear data from user space for ``msg`` and set 237pointers ``msg->data`` and ``msg->data_end`` to ``start`` and ``end`` bytes 238offsets into ``msg``, respectively. 239 240If a program of type ``BPF_PROG_TYPE_SK_MSG`` is run on a ``msg`` it can only 241parse data that the (``data``, ``data_end``) pointers have already consumed. 242For ``sendmsg()`` hooks this is likely the first scatterlist element. But for 243calls relying on the ``sendpage`` handler (e.g., ``sendfile()``) this will be 244the range (**0**, **0**) because the data is shared with user space and by 245default the objective is to avoid allowing user space to modify data while (or 246after) BPF verdict is being decided. This helper can be used to pull in data 247and to set the start and end pointers to given values. Data will be copied if 248necessary (i.e., if data was not linear and if start and end pointers do not 249point to the same chunk). 250 251A call to this helper is susceptible to change the underlying packet buffer. 252Therefore, at load time, all checks on pointers previously done by the verifier 253are invalidated and must be performed again, if the helper is used in 254combination with direct packet access. 255 256All values for ``flags`` are reserved for future usage, and must be left at 257zero. 258 259Returns 0 on success, or a negative error in case of failure. 260 261bpf_map_lookup_elem() 262^^^^^^^^^^^^^^^^^^^^^ 263 264.. code-block:: c 265 266 void *bpf_map_lookup_elem(struct bpf_map *map, const void *key) 267 268Look up a socket entry in the sockmap or sockhash map. 269 270Returns the socket entry associated to ``key``, or NULL if no entry was found. 271 272bpf_map_update_elem() 273^^^^^^^^^^^^^^^^^^^^^ 274.. code-block:: c 275 276 long bpf_map_update_elem(struct bpf_map *map, const void *key, const void *value, u64 flags) 277 278Add or update a socket entry in a sockmap or sockhash. 279 280The flags argument can be one of the following: 281 282- BPF_ANY: Create a new element or update an existing element. 283- BPF_NOEXIST: Create a new element only if it did not exist. 284- BPF_EXIST: Update an existing element. 285 286Returns 0 on success, or a negative error in case of failure. 287 288bpf_map_delete_elem() 289^^^^^^^^^^^^^^^^^^^^^^ 290.. code-block:: c 291 292 long bpf_map_delete_elem(struct bpf_map *map, const void *key) 293 294Delete a socket entry from a sockmap or a sockhash. 295 296Returns 0 on success, or a negative error in case of failure. 297 298User space 299---------- 300bpf_map_update_elem() 301^^^^^^^^^^^^^^^^^^^^^ 302.. code-block:: c 303 304 int bpf_map_update_elem(int fd, const void *key, const void *value, __u64 flags) 305 306Sockmap entries can be added or updated using the ``bpf_map_update_elem()`` 307function. The ``key`` parameter is the index value of the sockmap array. And the 308``value`` parameter is the FD value of that socket. 309 310Under the hood, the sockmap update function uses the socket FD value to 311retrieve the associated socket and its attached psock. 312 313The flags argument can be one of the following: 314 315- BPF_ANY: Create a new element or update an existing element. 316- BPF_NOEXIST: Create a new element only if it did not exist. 317- BPF_EXIST: Update an existing element. 318 319bpf_map_lookup_elem() 320^^^^^^^^^^^^^^^^^^^^^ 321.. code-block:: c 322 323 int bpf_map_lookup_elem(int fd, const void *key, void *value) 324 325Sockmap entries can be retrieved using the ``bpf_map_lookup_elem()`` function. 326 327.. note:: 328 The entry returned is a socket cookie rather than a socket itself. 329 330bpf_map_delete_elem() 331^^^^^^^^^^^^^^^^^^^^^ 332.. code-block:: c 333 334 int bpf_map_delete_elem(int fd, const void *key) 335 336Sockmap entries can be deleted using the ``bpf_map_delete_elem()`` 337function. 338 339Returns 0 on success, or negative error in case of failure. 340 341Examples 342======== 343 344Kernel BPF 345---------- 346Several examples of the use of sockmap APIs can be found in: 347 348- `tools/testing/selftests/bpf/progs/test_sockmap_kern.h`_ 349- `tools/testing/selftests/bpf/progs/sockmap_parse_prog.c`_ 350- `tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c`_ 351- `tools/testing/selftests/bpf/progs/test_sockmap_listen.c`_ 352- `tools/testing/selftests/bpf/progs/test_sockmap_update.c`_ 353 354The following code snippet shows how to declare a sockmap. 355 356.. code-block:: c 357 358 struct { 359 __uint(type, BPF_MAP_TYPE_SOCKMAP); 360 __uint(max_entries, 1); 361 __type(key, __u32); 362 __type(value, __u64); 363 } sock_map_rx SEC(".maps"); 364 365The following code snippet shows a sample parser program. 366 367.. code-block:: c 368 369 SEC("sk_skb/stream_parser") 370 int bpf_prog_parser(struct __sk_buff *skb) 371 { 372 return skb->len; 373 } 374 375The following code snippet shows a simple verdict program that interacts with a 376sockmap to redirect traffic to another socket based on the local port. 377 378.. code-block:: c 379 380 SEC("sk_skb/stream_verdict") 381 int bpf_prog_verdict(struct __sk_buff *skb) 382 { 383 __u32 lport = skb->local_port; 384 __u32 idx = 0; 385 386 if (lport == 10000) 387 return bpf_sk_redirect_map(skb, &sock_map_rx, idx, 0); 388 389 return SK_PASS; 390 } 391 392The following code snippet shows how to declare a sockhash map. 393 394.. code-block:: c 395 396 struct socket_key { 397 __u32 src_ip; 398 __u32 dst_ip; 399 __u32 src_port; 400 __u32 dst_port; 401 }; 402 403 struct { 404 __uint(type, BPF_MAP_TYPE_SOCKHASH); 405 __uint(max_entries, 1); 406 __type(key, struct socket_key); 407 __type(value, __u64); 408 } sock_hash_rx SEC(".maps"); 409 410The following code snippet shows a simple verdict program that interacts with a 411sockhash to redirect traffic to another socket based on a hash of some of the 412skb parameters. 413 414.. code-block:: c 415 416 static inline 417 void extract_socket_key(struct __sk_buff *skb, struct socket_key *key) 418 { 419 key->src_ip = skb->remote_ip4; 420 key->dst_ip = skb->local_ip4; 421 key->src_port = skb->remote_port >> 16; 422 key->dst_port = (bpf_htonl(skb->local_port)) >> 16; 423 } 424 425 SEC("sk_skb/stream_verdict") 426 int bpf_prog_verdict(struct __sk_buff *skb) 427 { 428 struct socket_key key; 429 430 extract_socket_key(skb, &key); 431 432 return bpf_sk_redirect_hash(skb, &sock_hash_rx, &key, 0); 433 } 434 435User space 436---------- 437Several examples of the use of sockmap APIs can be found in: 438 439- `tools/testing/selftests/bpf/prog_tests/sockmap_basic.c`_ 440- `tools/testing/selftests/bpf/test_sockmap.c`_ 441- `tools/testing/selftests/bpf/test_maps.c`_ 442 443The following code sample shows how to create a sockmap, attach a parser and 444verdict program, as well as add a socket entry. 445 446.. code-block:: c 447 448 int create_sample_sockmap(int sock, int parse_prog_fd, int verdict_prog_fd) 449 { 450 int index = 0; 451 int map, err; 452 453 map = bpf_map_create(BPF_MAP_TYPE_SOCKMAP, NULL, sizeof(int), sizeof(int), 1, NULL); 454 if (map < 0) { 455 fprintf(stderr, "Failed to create sockmap: %s\n", strerror(errno)); 456 return -1; 457 } 458 459 err = bpf_prog_attach(parse_prog_fd, map, BPF_SK_SKB_STREAM_PARSER, 0); 460 if (err){ 461 fprintf(stderr, "Failed to attach_parser_prog_to_map: %s\n", strerror(errno)); 462 goto out; 463 } 464 465 err = bpf_prog_attach(verdict_prog_fd, map, BPF_SK_SKB_STREAM_VERDICT, 0); 466 if (err){ 467 fprintf(stderr, "Failed to attach_verdict_prog_to_map: %s\n", strerror(errno)); 468 goto out; 469 } 470 471 err = bpf_map_update_elem(map, &index, &sock, BPF_NOEXIST); 472 if (err) { 473 fprintf(stderr, "Failed to update sockmap: %s\n", strerror(errno)); 474 goto out; 475 } 476 477 out: 478 close(map); 479 return err; 480 } 481 482References 483=========== 484 485- https://github.com/jrfastab/linux-kernel-xdp/commit/c89fd73cb9d2d7f3c716c3e00836f07b1aeb261f 486- https://lwn.net/Articles/731133/ 487- http://vger.kernel.org/lpc_net2018_talks/ktls_bpf_paper.pdf 488- https://lwn.net/Articles/748628/ 489- https://lore.kernel.org/bpf/20200218171023.844439-7-jakub@cloudflare.com/ 490 491.. _`tools/testing/selftests/bpf/progs/test_sockmap_kern.h`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/progs/test_sockmap_kern.h 492.. _`tools/testing/selftests/bpf/progs/sockmap_parse_prog.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/progs/sockmap_parse_prog.c 493.. _`tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c 494.. _`tools/testing/selftests/bpf/prog_tests/sockmap_basic.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c 495.. _`tools/testing/selftests/bpf/test_sockmap.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/test_sockmap.c 496.. _`tools/testing/selftests/bpf/test_maps.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/test_maps.c 497.. _`tools/testing/selftests/bpf/progs/test_sockmap_listen.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/progs/test_sockmap_listen.c 498.. _`tools/testing/selftests/bpf/progs/test_sockmap_update.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/progs/test_sockmap_update.c 499