1.\" Copyright (c) 2008 Christian Brueffer 2.\" Copyright (c) 2008 Jeffrey Roberson 3.\" Copyright (c) 2021 Robert N. M. Watson 4.\" All rights reserved. 5.\" 6.\" Redistribution and use in source and binary forms, with or without 7.\" modification, are permitted provided that the following conditions 8.\" are met: 9.\" 1. Redistributions of source code must retain the above copyright 10.\" notice, this list of conditions and the following disclaimer. 11.\" 2. Redistributions in binary form must reproduce the above copyright 12.\" notice, this list of conditions and the following disclaimer in the 13.\" documentation and/or other materials provided with the distribution. 14.\" 15.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 16.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 17.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 18.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 19.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 20.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 21.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 22.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 23.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 24.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 25.\" SUCH DAMAGE. 26.\" 27.Dd January 29, 2023 28.Dt CPUSET 2 29.Os 30.Sh NAME 31.Nm cpuset , 32.Nm cpuset_getid , 33.Nm cpuset_setid 34.Nd manage CPU affinity sets 35.Sh LIBRARY 36.Lb libc 37.Sh SYNOPSIS 38.In sys/param.h 39.In sys/cpuset.h 40.Ft int 41.Fn cpuset "cpusetid_t *setid" 42.Ft int 43.Fn cpuset_setid "cpuwhich_t which" "id_t id" "cpusetid_t setid" 44.Ft int 45.Fn cpuset_getid "cpulevel_t level" "cpuwhich_t which" "id_t id" "cpusetid_t *setid" 46.Sh DESCRIPTION 47The 48.Nm 49family of system calls allow applications to control sets of processors and 50memory domains and assign processes and threads to these sets. 51Processor sets contain lists of CPUs and domains that members may run on 52and exist only as long as some process is a member of the set. 53All processes in the system have an assigned set. 54The default set for all processes in the system is the set numbered 1. 55Threads belong to the same set as the process which contains them, 56however, they may further restrict their set with the anonymous 57per-thread mask to bind to a specific CPU or subset of CPUs and memory domains. 58.Pp 59Sets are referenced by a number of type 60.Ft cpuset_id_t . 61Each thread has a root set, an assigned set, and an anonymous mask. 62Only the root and assigned sets are numbered. 63The root set is the set of all CPUs and memory domains available in the system 64or in the system partition the thread is running in. 65The assigned set is a subset of the root set and is administratively 66assignable on a per-process basis. 67Many processes and threads may be members of a numbered set. 68.Pp 69The anonymous set is a further thread-specific refinement on the assigned 70set. 71It is intended that administrators will manipulate numbered sets using 72.Xr cpuset 1 73while application developers will manipulate anonymous sets using 74.Xr cpuset_setaffinity 2 and 75.Xr cpuset_setdomain 2 . 76.Pp 77To select the correct set a value of type 78.Ft cpulevel_t 79is used. 80The following values for 81.Fa level 82are supported: 83.Bl -column CPU_LEVEL_CPUSET -offset indent 84.It Dv CPU_LEVEL_ROOT Ta "Root set" 85.It Dv CPU_LEVEL_CPUSET Ta "Assigned set" 86.It Dv CPU_LEVEL_WHICH Ta "Set specified by which argument" 87.El 88.Pp 89The 90.Fa which 91argument determines how the value of 92.Fa id 93is interpreted and is of type 94.Ft cpuwhich_t . 95The 96.Fa which 97argument may have the following values: 98.Bl -column CPU_WHICH_INTRHANDLER -offset indent 99.It Dv CPU_WHICH_TID Ta "id is lwpid_t (thread id)" 100.It Dv CPU_WHICH_PID Ta "id is pid_t (process id)" 101.It Dv CPU_WHICH_TIDPID Ta "id is either a thread or process id" 102.It Dv CPU_WHICH_JAIL Ta "id is jid (jail id)" 103.It Dv CPU_WHICH_CPUSET Ta "id is a cpusetid_t (cpuset id)" 104.It Dv CPU_WHICH_IRQ Ta "id is an irq number" 105.It Dv CPU_WHICH_INTRHANDLER Ta "id is an irq number for an interrupt handler" 106.It Dv CPU_WHICH_ITHREAD Ta "id is an irq number for an ithread" 107.It Dv CPU_WHICH_DOMAIN Ta "id is a NUMA domain" 108.El 109.Pp 110An 111.Fa id 112of '-1' may be used with a 113.Fa which 114of 115.Dv CPU_WHICH_TID , 116.Dv CPU_WHICH_PID , 117.Dv CPU_WHICH_TIDPID , 118or 119.Dv CPU_WHICH_CPUSET 120to mean the current thread, process, or current thread's 121cpuset. 122All cpuset syscalls allow this usage. 123.Pp 124A 125.Fa level 126argument of 127.Dv CPU_LEVEL_WHICH 128combined with a 129.Fa which 130argument other than 131.Dv CPU_WHICH_CPUSET 132refers to the anonymous mask of the object. 133This mask does not have an id and may only be manipulated with 134.Xr cpuset_setaffinity 2 . 135.Pp 136.Fn cpuset 137creates a new set containing the same CPUs as the root set of the current 138process and stores its id in the space provided by 139.Fa setid . 140On successful completion the calling process joins the set and is the 141only member. 142Children inherit this set after a call to 143.Xr fork 2 . 144.Pp 145.Fn cpuset_setid 146attempts to set the id of the object specified by the 147.Fa which 148argument. 149Currently 150.Dv CPU_WHICH_PID 151is the only acceptable value for which as 152threads do not have an id distinct from their process and the API does 153not permit changing the id of an existing set. 154Upon successful completion all of the threads in the target process will 155be running on CPUs permitted by the set. 156.Pp 157.Fn cpuset_getid 158retrieves a set id from the object indicated by 159.Fa which 160and stores it in the space pointed to by 161.Fa setid . 162The retrieved id may be that of either the root or assigned set 163depending on the value of 164.Fa level . 165.Fa level 166should be 167.Dv CPU_LEVEL_CPUSET 168or 169.Dv CPU_LEVEL_ROOT 170to get the set id from 171the process or thread specified by the 172.Fa id 173argument. 174Specifying 175.Dv CPU_LEVEL_WHICH 176with a process or thread is unsupported since 177this references the unnumbered anonymous mask. 178.Pp 179The actual contents of the sets may be retrieved or manipulated using 180.Xr cpuset_getaffinity 2 , 181.Xr cpuset_setaffinity 2 , 182.Xr cpuset_getdomain 2 , and 183.Xr cpuset_setdomain 2 . 184The 185.Xr cpuset 9 186macros may be used to manipulate masks of type 187.Ft cpuset_t 188get and set using those APIs. 189See those manual pages for more detail. 190.Sh RETURN VALUES 191.Rv -std 192.Sh EXAMPLES 193In this example, a CPU set mask is configured to limit execution to the first 194CPU using 195.Xr CPU_ZERO 9 196and 197.Xr CPU_SET 9 , 198members of the 199.Xr cpuset 9 200programming interface. 201Then, the mask is applied to a new anonymous CPU set associated with the 202current process using 203.Xr cpuset_setaffinity 2 . 204This mask will be used by the current process, and inherited by any new 205child processes. 206.Bd -literal -offset indent 207#include <sys/param.h> 208#include <sys/cpuset.h> 209 210#include <sysexits.h> 211 212cpuset_t cpuset_mask; 213 214/* Initialize a CPU mask and enable CPU 0. */ 215CPU_ZERO(&cpuset_mask); 216CPU_SET(0, &cpuset_mask); 217 218/* Set affinity for the CPU set for the current process. */ 219if (cpuset_setaffinity(CPU_LEVEL_WHICH, CPU_WHICH_PID, -1, 220 sizeof(cpuset_mask), &cpuset_mask) < 0) 221 err(EX_OSERR, "cpuset_setaffinity"); 222.Ed 223.Pp 224In the next example, a named CPU set is created containing the current 225process, and its affinity similarly configured. 226The resulting CPU set ID can then be used for further external management of 227the affinity of the set. 228.Bd -literal -offset indent 229#include <sys/param.h> 230#include <sys/cpuset.h> 231 232#include <sysexits.h> 233 234cpusetid_t cpuset_id; 235cpuset_t cpuset_mask; 236 237/* Create new cpuset for the current process. */ 238if (cpuset(&cpuset_id) < 0) 239 err(EX_OSERR, "cpuset"); 240 241/* Initialize a CPU mask and enable CPU 0. */ 242CPU_ZERO(&cpuset_mask); 243CPU_SET(0, &cpuset_mask); 244 245/* Set affinity for the CPU set for the current process. */ 246if (cpuset_setaffinity(CPU_LEVEL_SET, CPU_WHICH_CPUSET, cpuset_id, 247 sizeof(cpuset_mask), &cpuset_mask) < 0) 248 err(EX_OSERR, "cpuset_setaffinity"); 249.Ed 250.Sh ERRORS 251The following error codes may be set in 252.Va errno : 253.Bl -tag -width Er 254.It Bq Er EINVAL 255The 256.Fa which 257or 258.Fa level 259argument was not a valid value. 260.It Bq Er EDEADLK 261The 262.Fn cpuset_setid 263call would leave a thread without a valid CPU to run on because the set 264does not overlap with the thread's anonymous mask. 265.It Bq Er EFAULT 266The setid pointer passed to 267.Fn cpuset_getid 268or 269.Fn cpuset 270was invalid. 271.It Bq Er ESRCH 272The object specified by the 273.Fa id 274and 275.Fa which 276arguments could not be found. 277.It Bq Er EPERM 278The calling process did not have the credentials required to complete the 279operation. 280.It Bq Er ENFILE 281There was no free 282.Ft cpusetid_t 283for allocation. 284.El 285.Sh SEE ALSO 286.Xr cpuset 1 , 287.Xr cpuset_getaffinity 2 , 288.Xr cpuset_getdomain 2 , 289.Xr cpuset_setaffinity 2 , 290.Xr cpuset_setdomain 2 , 291.Xr pthread_affinity_np 3 , 292.Xr pthread_attr_affinity_np 3 , 293.Xr CPU_SET 9 , 294.Xr CPU_ZERO 9 , 295.Xr cpuset 9 296.Sh HISTORY 297The 298.Nm 299family of system calls first appeared in 300.Fx 7.1 . 301.Sh AUTHORS 302.An Jeffrey Roberson Aq Mt jeff@FreeBSD.org 303