1.\" Copyright (c) 2000-2001 John H. Baldwin <jhb@FreeBSD.org> 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" THIS SOFTWARE IS PROVIDED BY THE DEVELOPERS ``AS IS'' AND ANY EXPRESS OR 14.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 15.\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 16.\" IN NO EVENT SHALL THE DEVELOPERS BE LIABLE FOR ANY DIRECT, INDIRECT, 17.\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 18.\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 19.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 20.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 21.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 22.\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 23.\" 24.\" $FreeBSD$ 25.\" 26.Dd October 27, 2000 27.Os 28.Dt ATOMIC 9 29.Sh NAME 30.Nm atomic_add , 31.Nm atomic_clear , 32.Nm atomic_cmpset , 33.Nm atomic_load , 34.Nm atomic_readandclear , 35.Nm atomic_set , 36.Nm atomic_subtract , 37.Nm atomic_store 38.Nd atomic operations 39.Sh SYNOPSIS 40.In sys/types.h 41.In machine/atomic.h 42.Ft void 43.Fn atomic_add_[acq_|rel_]<type> "volatile <type> *p" "<type> v" 44.Ft void 45.Fn atomic_clear_[acq_|rel_]<type> "volatile <type> *p" "<type> v" 46.Ft int 47.Fo atomic_cmpset_[acq_|rel_]<type> 48.Fa "volatile <type> *dst" 49.Fa "<type> old" 50.Fa "<type> new" 51.Fc 52.Ft <type> 53.Fn atomic_load_acq_<type> "volatile <type> *p" 54.Ft <type> 55.Fn atomic_readandclear_<type> "volatile <type> *p" 56.Ft void 57.Fn atomic_set_[acq_|rel_]<type> "volatile <type> *p" "<type> v" 58.Ft void 59.Fn atomic_subtract_[acq_|rel_]<type> "volatile <type> *p" "<type> v" 60.Ft void 61.Fn atomic_store_rel_<type> "volatile <type> *p" "<type> v" 62.rm LB RB La Ra 63.Sh DESCRIPTION 64Each of the atomic operations is guaranteed to be atomic in the presence of 65interrupts. 66They can be used to implement reference counts or as building blocks for more 67advanced synchronization primitives such as mutexes. 68.Ss Types 69Each atomic operation operates on a specific 70.Fa type . 71The type to use is indicated in the function name. 72The available types that can be used are: 73.Pp 74.Bl -tag -offset indent -width short -compact 75.It Li int 76unsigned integer 77.It Li long 78unsigned long integer 79.It Li ptr 80unsigned integer the size of a pointer 81.It Li 32 82unsigned 32-bit integer 83.It Li 64 84unsigned 64-bit integer 85.El 86.Pp 87For example, the function to atomically add two integers is called 88.Fn atomic_add_int . 89.Pp 90Certain architectures also provide operations for types smaller than 91.Dq Li int . 92.Pp 93.Bl -tag -offset indent -width short -compact 94.It Li char 95unsigned character 96.It Li short 97unsigned short integer 98.It Li 8 99unsigned 8-bit integer 100.It Li 16 101unsigned 16-bit integer 102.El 103.Pp 104These must not be used in MI code because the instructions to implement them 105efficiently may not be available. 106.Ss Memory Barriers 107Memory barriers are used to guarantee the order of data accesses in 108two ways. 109First, they specify hints to the compiler to not re-order or optimize the 110operations. 111Second, on architectures that do not guarantee ordered data accesses, 112special instructions or special variants of instructions are used to indicate 113to the processor that data accesses need to occur in a certain order. 114As a result, most of the atomic operations have three variants in order to 115include optional memory barriers. 116The first form just performs the operation without any explicit barriers. 117The second form uses a read memory barrier, and the third variant uses a write 118memory barrier. 119.Pp 120The second variant of each operation includes a read memory barrier. 121This barrier ensures that the effects of this operation are completed before the 122effects of any later data accesses. 123As a result, the operation is said to have acquire semantics as it acquires a 124pseudo-lock requiring further operations to wait until it has completed. 125To denote this, the suffix 126.Dq Li _acq 127is inserted into the function name immediately prior to the 128.Dq Li _ Ns Aq Fa type 129suffix. 130For example, to subtract two integers ensuring that any later writes will 131happen after the subtraction is performed, use 132.Fn atomic_subtract_acq_int . 133.Pp 134The third variant of each operation includes a write memory barrier. 135This ensures that all effects of all previous data accesses are completed 136before this operation takes place. 137As a result, the operation is said to have release semantics as it releases 138any pending data accesses to be completed before its operation is performed. 139To denote this, the suffix 140.Dq Li _rel 141is inserted into the function name immediately prior to the 142.Dq Li _ Ns Aq Fa type 143suffix. 144For example, to add two long integers ensuring that all previous 145writes will happen first, use 146.Fn atomic_add_rel_long . 147.Pp 148A practical example of using memory barriers is to ensure that data accesses 149that are protected by a lock are all performed while the lock is held. 150To achieve this, one would use a read barrier when acquiring the lock to 151guarantee that the lock is held before any protected operations are performed. 152Finally, one would use a write barrier when releasing the lock to ensure that 153all of the protected operations are completed before the lock is released. 154.Ss Multiple Processors 155The current set of atomic operations do not necessarily guarantee atomicity 156across multiple processors. 157To guarantee atomicity across processors, not only does the individual 158operation need to be atomic on the processor performing the operation, but 159the result of the operation needs to be pushed out to stable storage and the 160caches of all other processors on the system need to invalidate any cache 161lines that include the affected memory region. 162On the 163.Tn i386 164architecture, the cache coherency model requires that the hardware perform 165this task, thus the atomic operations are atomic across multiple processors. 166On the 167.Tn ia64 168architecture, coherency is only guaranteed for pages that are configured to 169using a caching policy of either uncached or write back. 170.Ss Semantics 171This section describes the semantics of each operation using a C like notation. 172.Bl -hang 173.It Fn atomic_add p v 174.Bd -literal -compact 175*p += v; 176.Ed 177.It Fn atomic_clear p v 178.Bd -literal -compact 179*p &= ~v; 180.Ed 181.It Fn atomic_cmpset dst old new 182.Bd -literal -compact 183if (*dst == old) { 184 *dst = new; 185 return 1; 186} else 187 return 0; 188.Ed 189.El 190.Pp 191The 192.Fn atomic_cmpset 193functions are not implemented for the types 194.Dq Li char , 195.Dq Li short , 196.Dq Li 8 , 197and 198.Dq Li 16 . 199.Bl -hang 200.It Fn atomic_load addr 201.Bd -literal -compact 202return (*addr) 203.Ed 204.El 205.Pp 206The 207.Fn atomic_load 208functions always have acquire semantics. 209.Bl -hang 210.It Fn atomic_readandclear addr 211.Bd -literal -compact 212temp = *addr; 213*addr = 0; 214return (temp); 215.Ed 216.El 217.Pp 218The 219.Fn atomic_readandclear 220functions are not implemented for the types 221.Dq Li char , 222.Dq Li short , 223.Dq Li ptr , 224.Dq Li 8 , 225and 226.Dq Li 16 227and do 228not have any variants with memory barriers at this time. 229.Bl -hang 230.It Fn atomic_set p v 231.Bd -literal -compact 232*p |= v; 233.Ed 234.It Fn atomic_subtract p v 235.Bd -literal -compact 236*p -= v; 237.Ed 238.It Fn atomic_store p v 239.Bd -literal -compact 240*p = v; 241.Ed 242.El 243.Pp 244The 245.Fn atomic_store 246functions always have release semantics. 247.Pp 248The type 249.Dq Li 64 250is currently not implemented for any of the atomic operations on the 251.Tn i386 252architecture. 253.Sh RETURN VALUES 254The 255.Fn atomic_cmpset 256function 257returns the result of the compare operation. 258The 259.Fn atomic_load 260and 261.Fn atomic_readandclear 262functions 263return the value at the specified address. 264.Sh EXAMPLES 265This example uses the 266.Fn atomic_cmpset_acq_ptr 267and 268.Fn atomic_set_ptr 269functions to obtain a sleep mutex and handle recursion. 270Since the 271.Va mtx_lock 272member of a 273.Vt "struct mtx" 274is a pointer, the 275.Dq Li ptr 276type is used. 277.Bd -literal 278#define _obtain_lock(mp, tid) \\ 279 atomic_cmpset_acq_ptr(&(mp)->mtx_lock, (void *)MTX_UNOWNED, (tid)) 280 281/* Get a sleep lock, deal with recursion inline. */ 282#define _getlock_sleep(mp, tid, type) do { \\ 283 if (!_obtain_lock(mp, tid)) { \\ 284 if (((mp)->mtx_lock & MTX_FLAGMASK) != ((uintptr_t)(tid)))\\ 285 mtx_enter_hard(mp, (type) & MTX_HARDOPTS, 0); \\ 286 else { \\ 287 atomic_set_ptr(&(mp)->mtx_lock, MTX_RECURSE); \\ 288 (mp)->mtx_recurse++; \\ 289 } \\ 290 } \\ 291} while (0) 292.Ed 293.Sh HISTORY 294The 295.Fn atomic_add , 296.Fn atomic_clear , 297.Fn atomic_set , 298and 299.Fn atomic_subtract 300operations were first introduced in 301.Fx 3.0 . 302This first set only supported the types 303.Dq Li char , 304.Dq Li short , 305.Dq Li int , 306and 307.Dq Li long . 308The 309.Fn atomic_cmpset , 310.Fn atomic_load , 311.Fn atomic_readandclear , 312and 313.Fn atomic_store 314operations were added in 315.Fx 5.0 . 316The types 317.Dq Li 8 , 318.Dq Li 16 , 319.Dq Li 32 , 320.Dq Li 64 , 321and 322.Dq Li ptr , 323and all of the acquire and release variants 324were added in 325.Fx 5.0 326as well. 327