1.\"- 2.\" Copyright (c) 2017 Robert N. M. Watson 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.Dd July 5, 2018 27.Dt DPCPU 9 28.Os 29.Sh NAME 30.Nm dpcpu 31.Nd Kernel Dynamic Per-CPU Memory Allocator 32.Sh SYNOPSIS 33.In sys/pcpu.h 34.Ss Per-CPU Variable Definition and Declaration 35.Fn DPCPU_DEFINE "type" "name" 36.Fn DPCPU_DEFINE_STATIC "type" "name" 37.Fn DPCPU_DECLARE "type" "name" 38.Ss Current CPU Accessor Functions 39.Fn DPCPU_PTR "name" 40.Fn DPCPU_GET "name" 41.Fn DPCPU_SET "name" "value" 42.Ss Named CPU Accessor Functions 43.Fn DPCPU_ID_PTR "cpu" "name" 44.Fn DPCPU_ID_GET "cpu" "name" 45.Fn DPCPU_ID_SET "cpu" "name" "value" 46.Sh DESCRIPTION 47.Nm 48instantiates one instance of a global variable with each CPU in the system. 49Dynamically allocated per-CPU variables are defined using 50.Fn DPCPU_DEFINE , 51which defines a variable of name 52.Ar name 53and type 54.Ar type . 55Arbitrary C types may be used, including structures and arrays. 56If no initialization is provided, then each per-CPU instance of the variable 57will be zero-filled (i.e., as though allocated in BSS): 58.Bd -literal -offset 1234 59DPCPU_DEFINE(int, foo_int); 60.Ed 61.Pp 62Values may also be initialized statically with the definition, causing each 63per-CPU instance to be initialized with the value: 64.Bd -literal -offset 1234 65DPCPU_DEFINE(int, foo_int) = 1; 66.Ed 67.Pp 68Values that can be defined as 69.Dv static 70must use 71.Fn DPCPU_DEFINE_STATIC : 72.Bd -literal -offset 1234 73DPCPU_DEFINE_STATIC(int, foo_int); 74.Ed 75.Pp 76.Fn DPCPU_DECLARE 77produces a declaration of the per-CPU variable suitable for use in header 78files. 79.Pp 80The current CPU's variable instance can be accessed via 81.Nm DPCPU_PTR 82(which returns a pointer to the per-CPU instance), 83.Nm DPCPU_GET 84(which retrieves the value of the per-CPU instance), 85and 86.Nm DPCPU_SET 87(which sets the value of the per-CPU instance). 88.Pp 89Instances of variables associated with specific CPUs can be accessed via the 90.Nm DPCPU_ID_PTR , 91.Nm DPCPU_ID_GET , 92and 93.Nm DPGPU_ID_SET 94accessor functions, which accept an additional CPU ID argument, 95.Ar cpu . 96.Ss Synchronization 97In addition to the ordinary synchronization concerns associated with global 98variables, which may imply the use of 99.Xr atomic 9 , 100.Xr mutex 9 , 101or other kernel synchronization primitives, it is further the case that 102thread migration could dynamically change the instance of a variable being 103accessed by a thread between operations. 104This requires additional care when reasoning about and protecting per-CPU 105variables. 106.Pp 107For example, it may be desirable to protect access using 108.Xr critical_section 9 109to prevent both preemption and migration during use. 110Alternatively, it may be desirable to cache the CPU ID at the start of a 111sequence of accesses, using suitable synchronization to make non-atomic 112sequences safe in the presence of migration. 113.Bd -literal -offset 1234 114DPCPU_DEFINE_STATIC(int, foo_int); 115DPCPU_DEFINE_STATIC(struct mutex, foo_lock); 116 117void 118foo_int_increment(void) 119{ 120 int cpu, value; 121 122 /* Safe as atomic access. */ 123 atomic_add_int(DPCPU_PTR(foo_int), 1); 124 125 /* 126 * Protect with a critical section, which prevents preemption 127 * and migration. However, access to instances from remote CPUs 128 * is not safe, as critical sections prevent concurrent access 129 * only from the current CPU. 130 */ 131 critical_enter(); 132 value = DPCPU_GET(foo_int); 133 value++; 134 DPCPU_SET(foo_int, value); 135 critical_exit(); 136 137 /* 138 * Protect with a per-CPU mutex, tolerating migration, but 139 * potentially accessing the variable from multiple CPUs if 140 * migration occurs after reading curcpu. Remote access to a 141 * per-CPU variable is safe as long as the correct mutex is 142 * acquired. 143 */ 144 cpu = curcpu; 145 mtx_lock(DPCPU_ID_PTR(cpu, foo_lock)); 146 value = DPCPU_ID_GET(cpu, foo_int); 147 value++; 148 DPCPU_ID_SET(cpu, foo_int); 149 mtx_unlock(DPCPU_ID_PTR(cpu, foo_lock)); 150} 151.Ed 152.Sh SEE ALSO 153.Xr atomic 9 , 154.Xr critical_enter 9 , 155.Xr mutex 9 156.Sh HISTORY 157.Nm 158was first introduced by 159.An Jeff Roberson 160in 161.Fx 8.0 . 162This manual page was written by 163.An Robert N. M. Watson . 164