1.\" $FreeBSD$ 2.\" 3.Dd September 29, 2016 4.Dt LIBRSS 3 5.Os 6.Sh NAME 7.Nm librss 8.Nd Provide Receive-side scaling awareness to userland applications 9.Sh LIBRARY 10.Lb librss 11.Sh SYNOPSIS 12.In librss.h 13.Ft struct rss_config * 14.Fn rss_config_get "void" 15.Ft void 16.Fn rss_config_free "struct rss_config *cfg" 17.Ft int 18.Fn rss_config_get_bucket_count "struct rss_config *cfg" 19.Ft int 20.Fn rss_set_bucket_rebalance_cb "rss_bucket_rebalance_cb_t *cb" "void *cbdata" 21.Ft int 22.Fn rss_sock_set_bindmulti "int fd" "int af" "int val" 23.Ft int 24.Fn rss_sock_set_rss_bucket "int fd" "int af" "int rss_bucket" 25.Ft int 26.Fn rss_sock_set_recvrss "int fd" "int af" "int val" 27.Sh DESCRIPTION 28The 29.Nm 30library and the functions it provides are used for both fetching 31the system RSS configuration and interacting with RSS aware 32sockets. 33.Pp 34Applications will typically call 35.Fn rss_config_get 36to fetch the current RSS configuration from the system and perform 37initial setup. 38This typically involves spawning worker threads, one per RSS bucket, 39and optionally binding them to the per-bucket CPU set. 40.Pp 41The 42.Vt rss_config 43struct is defined as: 44.Bd -literal 45struct rss_config { 46 int rss_ncpus; 47 int rss_nbuckets; 48 int rss_basecpu; 49 int *rss_bucket_map; 50}; 51.Ed 52.Pp 53Applications will typically use the 54.Fn rss_config_get_bucket_count 55function to fetch the number of RSS buckets, create one thread 56per RSS bucket for RSS aware work, then one RSS aware socket to receive 57UDP datagrams or TCP connections 58in each particular RSS bucket / thread. 59.Pp 60The 61.Fn rss_get_bucket_cpuset 62function sets the given cpuset up for the given 63RSS bucket and behaviour. 64Typically applications will wish to just query for 65.Vt RSS_BUCKET_TYPE_KERNEL_ALL 66unless they wish to potentially setup different 67worker threads for transmit and receive. 68.Pp 69The 70.Vt rss_bucket_type_t 71enum is defined as: 72.Bd -literal 73typedef enum { 74 RSS_BUCKET_TYPE_NONE = 0, 75 RSS_BUCKET_TYPE_KERNEL_ALL = 1, 76 RSS_BUCKET_TYPE_KERNEL_TX = 2, 77 RSS_BUCKET_TYPE_KERNEL_RX = 3, 78 RSS_BUCKET_TYPE_MAX = 3, 79} rss_bucket_type_t; 80.Ed 81.Pp 82The rebalance callback 83.Vt rss_bucket_rebalance_cb_t 84is defined as: 85.Bd -literal 86typedef void rss_bucket_rebalance_cb_t(void *arg); 87.Ed 88.Pp 89The 90.Fn rss_set_bucket_rebalance_cb 91function sets an optional callback that will be called if the kernel 92rebalances RSS buckets. 93This is intended as a future expansion to rebalance buckets rather than 94reprogram the RSS key, so typically the only work to be performed 95is to rebind worker threads to an updated cpuset. 96.Pp 97Once RSS setup is completed, 98.Fn rss_config_free 99is called to free the RSS configuration structure. 100.Pp 101To make a 102.Vt bind 103socket RSS aware, the 104.Fn rss_sock_set_bindmulti 105function is used to enable or disable per-RSS bucket 106behaviour. 107The socket filedescriptor, address family and enable flag 108.Vt val 109are passed in. 110.Pp 111If 112.Vt val 113is set to 1, the socket can be placed in an RSS bucket and will only accept 114datagrams (for UDP) or connections (for TCP) that are received for that 115RSS bucket. 116If set to 0, the socket is placed in the default PCB and will see 117datagrams/connections that are not initially consumed by a PCB aware 118socket. 119.Pp 120The 121.Fn rss_sock_set_rss_bucket 122function configures the RSS bucket which a socket belongs in. 123Note that TCP sockets created by 124.Xr accept 2 125will automatically be assigned to the RSS bucket. 126.Pp 127The 128.Fn rss_sock_set_recvrss 129function enables or disables receiving RSS related information 130as socket options in. 131.2 recvmsg 132calls. 133.Pp 134When enabled, UDP datagrams will have a message with the 135.Vt IP_RECVFLOWID 136option indicating the 32-bit receive flowid as a uint32_t, 137and the 138.Vt IP_RECVRSSBUCKETID 139option indicating the 32 bit RSS bucket id as a uint32_t. 140.Sh ERRORS 141The functions return either <0 or NULL as appropriate upon error. 142.Sh SEE ALSO 143.Xr PCBGROUP 9 144.Sh HISTORY 145The 146.Xr librss.3 147library first appeared in 148.Fx 11.0 . 149.Sh AUTHORS 150.An Adrian Chadd Aq Mt adrian@FreeBSD.org 151.Sh BUGS 152There is currently no kernel mechanism to rebalance the RSS bucket to CPU 153mapping, and so the callback mechanism is a no-op. 154