1.\" $FreeBSD$ 2.\" 3.Dd October 23, 2016 4.Dt LIBRSS 3 5.Os 6.Sh NAME 7.Nm librss 8.Nd Provide Receive-side scaling awareness to userland applications 9.Sh LIBRARY 10.Lb librss 11.Sh SYNOPSIS 12.In sys/param.h 13.In sys/cpuset.h 14.In librss.h 15.Ft struct rss_config * 16.Fn rss_config_get "void" 17.Ft void 18.Fn rss_config_free "struct rss_config *cfg" 19.Ft int 20.Fn rss_config_get_bucket_count "struct rss_config *cfg" 21.Ft int 22.Fn rss_get_bucket_cpuset "struct rss_config *rc" "rss_bucket_type_t btype" "int bucket" "cpuset_t *cs" 23.Ft int 24.Fn rss_set_bucket_rebalance_cb "rss_bucket_rebalance_cb_t *cb" "void *cbdata" 25.Ft int 26.Fn rss_sock_set_rss_bucket "int fd" "int af" "int rss_bucket" 27.Ft int 28.Fn rss_sock_set_recvrss "int fd" "int af" "int val" 29.Sh DESCRIPTION 30The 31.Nm 32library and the functions it provides are used for both fetching 33the system RSS configuration and interacting with RSS aware 34sockets. 35.Pp 36Applications will typically call 37.Fn rss_config_get 38to fetch the current RSS configuration from the system and perform 39initial setup. 40This typically involves spawning worker threads, one per RSS bucket, 41and optionally binding them to the per-bucket CPU set. 42.Pp 43The 44.Vt rss_config 45struct is defined as: 46.Bd -literal 47struct rss_config { 48 int rss_ncpus; 49 int rss_nbuckets; 50 int rss_basecpu; 51 int *rss_bucket_map; 52}; 53.Ed 54.Pp 55Applications will typically use the 56.Fn rss_config_get_bucket_count 57function to fetch the number of RSS buckets, create one thread 58per RSS bucket for RSS aware work, then one RSS aware socket to receive 59UDP datagrams or TCP connections 60in each particular RSS bucket / thread. 61.Pp 62The 63.Fn rss_get_bucket_cpuset 64function sets the given cpuset up for the given 65RSS bucket and behaviour. 66Typically applications will wish to just query for 67.Vt RSS_BUCKET_TYPE_KERNEL_ALL 68unless they wish to potentially setup different 69worker threads for transmit and receive. 70.Pp 71The 72.Vt rss_bucket_type_t 73enum is defined as: 74.Bd -literal 75typedef enum { 76 RSS_BUCKET_TYPE_NONE = 0, 77 RSS_BUCKET_TYPE_KERNEL_ALL = 1, 78 RSS_BUCKET_TYPE_KERNEL_TX = 2, 79 RSS_BUCKET_TYPE_KERNEL_RX = 3, 80 RSS_BUCKET_TYPE_MAX = 3, 81} rss_bucket_type_t; 82.Ed 83.Pp 84The rebalance callback 85.Vt rss_bucket_rebalance_cb_t 86is defined as: 87.Bd -literal 88typedef void rss_bucket_rebalance_cb_t(void *arg); 89.Ed 90.Pp 91The 92.Fn rss_set_bucket_rebalance_cb 93function sets an optional callback that will be called if the kernel 94rebalances RSS buckets. 95This is intended as a future expansion to rebalance buckets rather than 96reprogram the RSS key, so typically the only work to be performed 97is to rebind worker threads to an updated cpuset. 98.Pp 99Once RSS setup is completed, 100.Fn rss_config_free 101is called to free the RSS configuration structure. 102.Pp 103If 104.Vt val 105is set to 1, the socket can be placed in an RSS bucket and will only accept 106datagrams (for UDP) or connections (for TCP) that are received for that 107RSS bucket. 108If set to 0, the socket is placed in the default PCB and will see 109datagrams/connections that are not initially consumed by a PCB aware 110socket. 111.Pp 112The 113.Fn rss_sock_set_rss_bucket 114function configures the RSS bucket which a socket belongs in. 115Note that TCP sockets created by 116.Xr accept 2 117will automatically be assigned to the RSS bucket. 118.Pp 119The 120.Fn rss_sock_set_recvrss 121function enables or disables receiving RSS related information 122as socket options in. 123.2 recvmsg 124calls. 125.Pp 126When enabled, UDP datagrams will have a message with the 127.Vt IP_RECVFLOWID 128option indicating the 32-bit receive flowid as a uint32_t, 129and the 130.Vt IP_RECVRSSBUCKETID 131option indicating the 32 bit RSS bucket id as a uint32_t. 132.Sh ERRORS 133The functions return either <0 or NULL as appropriate upon error. 134.Sh HISTORY 135The 136.Xr librss.3 137library first appeared in 138.Fx 11.0 . 139.Sh AUTHORS 140.An Adrian Chadd Aq Mt adrian@FreeBSD.org 141.Sh BUGS 142There is currently no kernel mechanism to rebalance the RSS bucket to CPU 143mapping, and so the callback mechanism is a no-op. 144