1.\" 2.Dd October 23, 2016 3.Dt LIBRSS 3 4.Os 5.Sh NAME 6.Nm librss 7.Nd Provide Receive-side scaling awareness to userland applications 8.Sh LIBRARY 9.Lb librss 10.Sh SYNOPSIS 11.In sys/param.h 12.In sys/cpuset.h 13.In librss.h 14.Ft struct rss_config * 15.Fn rss_config_get "void" 16.Ft void 17.Fn rss_config_free "struct rss_config *cfg" 18.Ft int 19.Fn rss_config_get_bucket_count "struct rss_config *cfg" 20.Ft int 21.Fn rss_get_bucket_cpuset "struct rss_config *rc" "rss_bucket_type_t btype" "int bucket" "cpuset_t *cs" 22.Ft int 23.Fn rss_set_bucket_rebalance_cb "rss_bucket_rebalance_cb_t *cb" "void *cbdata" 24.Ft int 25.Fn rss_sock_set_recvrss "int fd" "int af" "int val" 26.Sh DESCRIPTION 27The 28.Nm 29library and the functions it provides are used for both fetching 30the system RSS configuration and interacting with RSS aware 31sockets. 32.Pp 33Applications will typically call 34.Fn rss_config_get 35to fetch the current RSS configuration from the system and perform 36initial setup. 37This typically involves spawning worker threads, one per RSS bucket, 38and optionally binding them to the per-bucket CPU set. 39.Pp 40The 41.Vt rss_config 42struct is defined as: 43.Bd -literal 44struct rss_config { 45 int rss_ncpus; 46 int rss_nbuckets; 47 int rss_basecpu; 48 int *rss_bucket_map; 49}; 50.Ed 51.Pp 52Applications will typically use the 53.Fn rss_config_get_bucket_count 54function to fetch the number of RSS buckets, create one thread 55per RSS bucket for RSS aware work, then one RSS aware socket to receive 56UDP datagrams or TCP connections 57in each particular RSS bucket / thread. 58.Pp 59The 60.Fn rss_get_bucket_cpuset 61function sets the given cpuset up for the given 62RSS bucket and behaviour. 63Typically applications will wish to just query for 64.Vt RSS_BUCKET_TYPE_KERNEL_ALL 65unless they wish to potentially setup different 66worker threads for transmit and receive. 67.Pp 68The 69.Vt rss_bucket_type_t 70enum is defined as: 71.Bd -literal 72typedef enum { 73 RSS_BUCKET_TYPE_NONE = 0, 74 RSS_BUCKET_TYPE_KERNEL_ALL = 1, 75 RSS_BUCKET_TYPE_KERNEL_TX = 2, 76 RSS_BUCKET_TYPE_KERNEL_RX = 3, 77 RSS_BUCKET_TYPE_MAX = 3, 78} rss_bucket_type_t; 79.Ed 80.Pp 81The rebalance callback 82.Vt rss_bucket_rebalance_cb_t 83is defined as: 84.Bd -literal 85typedef void rss_bucket_rebalance_cb_t(void *arg); 86.Ed 87.Pp 88The 89.Fn rss_set_bucket_rebalance_cb 90function sets an optional callback that will be called if the kernel 91rebalances RSS buckets. 92This is intended as a future expansion to rebalance buckets rather than 93reprogram the RSS key, so typically the only work to be performed 94is to rebind worker threads to an updated cpuset. 95.Pp 96Once RSS setup is completed, 97.Fn rss_config_free 98is called to free the RSS configuration structure. 99.Pp 100If 101.Vt val 102is set to 1, the socket can be placed in an RSS bucket and will only accept 103datagrams (for UDP) or connections (for TCP) that are received for that 104RSS bucket. 105If set to 0, the socket is placed in the default PCB and will see 106datagrams/connections that are not initially consumed by a PCB aware 107socket. 108.Pp 109The 110.Fn rss_sock_set_recvrss 111function enables or disables receiving RSS related information 112as socket options in. 113.2 recvmsg 114calls. 115.Pp 116When enabled, UDP datagrams will have a message with the 117.Vt IP_RECVFLOWID 118option indicating the 32-bit receive flowid as a uint32_t, 119and the 120.Vt IP_RECVRSSBUCKETID 121option indicating the 32 bit RSS bucket id as a uint32_t. 122.Sh ERRORS 123The functions return either <0 or NULL as appropriate upon error. 124.Sh HISTORY 125The 126.Xr librss.3 127library first appeared in 128.Fx 11.0 . 129.Sh AUTHORS 130.An Adrian Chadd Aq Mt adrian@FreeBSD.org 131.Sh BUGS 132There is currently no kernel mechanism to rebalance the RSS bucket to CPU 133mapping, and so the callback mechanism is a no-op. 134