1.\" Copyright (c) 2007 Stephan Uphoff <ups@FreeBSD.org> 2.\" Copyright (c) 2006 Gleb Smirnoff <glebius@FreeBSD.org> 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" $FreeBSD$ 27.\" 28.\" Based on rwlock.9 man page 29.Dd April 12, 2021 30.Dt RMLOCK 9 31.Os 32.Sh NAME 33.Nm rmlock , 34.Nm rm_init , 35.Nm rm_init_flags , 36.Nm rm_destroy , 37.Nm rm_rlock , 38.Nm rm_try_rlock , 39.Nm rm_wlock , 40.Nm rm_runlock , 41.Nm rm_wunlock , 42.Nm rm_wowned , 43.Nm rm_sleep , 44.Nm rm_assert , 45.Nm RM_SYSINIT , 46.Nm RM_SYSINIT_FLAGS , 47.Nm rms_init , 48.Nm rms_destroy , 49.Nm rms_rlock , 50.Nm rms_wlock , 51.Nm rms_runlock , 52.Nm rms_wunlock 53.Nd kernel reader/writer lock optimized for read-mostly access patterns 54.Sh SYNOPSIS 55.In sys/param.h 56.In sys/lock.h 57.In sys/rmlock.h 58.Ft void 59.Fn rm_init "struct rmlock *rm" "const char *name" 60.Ft void 61.Fn rm_init_flags "struct rmlock *rm" "const char *name" "int opts" 62.Ft void 63.Fn rm_destroy "struct rmlock *rm" 64.Ft void 65.Fn rm_rlock "struct rmlock *rm" "struct rm_priotracker* tracker" 66.Ft int 67.Fn rm_try_rlock "struct rmlock *rm" "struct rm_priotracker* tracker" 68.Ft void 69.Fn rm_wlock "struct rmlock *rm" 70.Ft void 71.Fn rm_runlock "struct rmlock *rm" "struct rm_priotracker* tracker" 72.Ft void 73.Fn rm_wunlock "struct rmlock *rm" 74.Ft int 75.Fn rm_wowned "const struct rmlock *rm" 76.Ft int 77.Fn rm_sleep "void *wchan" "struct rmlock *rm" "int priority" "const char *wmesg" "int timo" 78.Pp 79.Cd "options INVARIANTS" 80.Cd "options INVARIANT_SUPPORT" 81.Ft void 82.Fn rm_assert "struct rmlock *rm" "int what" 83.In sys/kernel.h 84.Fn RM_SYSINIT "name" "struct rmlock *rm" "const char *desc" 85.Fn RM_SYSINIT_FLAGS "name" "struct rmlock *rm" "const char *desc" "int flags" 86.Ft void 87.Fn rms_init "struct rmslock *rms" "const char *name" 88.Ft void 89.Fn rms_destroy "struct rmslock *rms" 90.Ft void 91.Fn rms_rlock "struct rmslock *rms" 92.Ft void 93.Fn rms_wlock "struct rmslock *rms" 94.Ft void 95.Fn rms_runlock "struct rmslock *rms" 96.Ft void 97.Fn rms_wunlock "struct rmslock *rms" 98.Sh DESCRIPTION 99Read-mostly locks allow shared access to protected data by multiple threads, 100or exclusive access by a single thread. 101The threads with shared access are known as 102.Em readers 103since they only read the protected data. 104A thread with exclusive access is known as a 105.Em writer 106since it can modify protected data. 107.Pp 108Read-mostly locks are designed to be efficient for locks almost exclusively 109used as reader locks and as such should be used for protecting data that 110rarely changes. 111Acquiring an exclusive lock after the lock has been locked for shared access 112is an expensive operation. 113.Pp 114Normal read-mostly locks are similar to 115.Xr rwlock 9 116locks and follow the same lock ordering rules as 117.Xr rwlock 9 118locks. 119Read-mostly locks have full priority propagation like mutexes. 120Unlike 121.Xr rwlock 9 , 122read-mostly locks propagate priority to both readers and writers. 123This is implemented via the 124.Va rm_priotracker 125structure argument supplied to 126.Fn rm_rlock 127and 128.Fn rm_runlock . 129Readers can recurse if the lock is initialized with the 130.Dv RM_RECURSE 131option; 132however, writers are never allowed to recurse. 133.Pp 134Sleeping for writers can be allowed by passing 135.Dv RM_SLEEPABLE 136to 137.Fn rm_init_flags . 138It changes lock ordering rules to the same as for 139.Xr sx 9 140locks. 141They do not propagate priority to writers, but they do propagate priority to readers. 142Note that readers are not permitted to sleep regardless of the flag. 143.Pp 144Sleepable read-mostly locks (created with 145.Fn rms_init ) 146allow sleeping for both readers and writers, but don't do priority propagation 147for either. 148They follow 149.Xr sx 9 150lock ordering. 151.Ss Macros and Functions 152.Bl -tag -width indent 153.It Fn rm_init "struct rmlock *rm" "const char *name" 154Initialize the read-mostly lock 155.Fa rm . 156The 157.Fa name 158description is used solely for debugging purposes. 159This function must be called before any other operations 160on the lock. 161.It Fn rm_init_flags "struct rmlock *rm" "const char *name" "int opts" 162Similar to 163.Fn rm_init , 164initialize the read-mostly lock 165.Fa rm 166with a set of optional flags. 167The 168.Fa opts 169arguments contains one or more of the following flags: 170.Bl -tag -width ".Dv RM_NOWITNESS" 171.It Dv RM_NOWITNESS 172Instruct 173.Xr witness 4 174to ignore this lock. 175.It Dv RM_RECURSE 176Allow threads to recursively acquire shared locks for 177.Fa rm . 178.It Dv RM_SLEEPABLE 179Create a sleepable read-mostly lock. 180.It Dv RM_NEW 181If the kernel has been compiled with 182.Cd "option INVARIANTS" , 183.Fn rm_init_flags 184will assert that the 185.Fa rm 186has not been initialized multiple times without intervening calls to 187.Fn rm_destroy 188unless this option is specified. 189.It Dv RM_DUPOK 190.Xr witness 4 191should not log messages about duplicate locks being acquired. 192.El 193.It Fn rm_rlock "struct rmlock *rm" "struct rm_priotracker* tracker" 194Lock 195.Fa rm 196as a reader using 197.Fa tracker 198to track read owners of a lock for priority propagation. 199This data structure is only used internally by 200.Nm 201and must persist until 202.Fn rm_runlock 203has been called. 204This data structure can be allocated on the stack since 205readers cannot sleep. 206If any thread holds this lock exclusively, the current thread blocks, 207and its priority is propagated to the exclusive holder. 208If the lock was initialized with the 209.Dv RM_RECURSE 210option the 211.Fn rm_rlock 212function can be called when the current thread has already acquired reader 213access on 214.Fa rm . 215.It Fn rm_try_rlock "struct rmlock *rm" "struct rm_priotracker* tracker" 216Try to lock 217.Fa rm 218as a reader. 219.Fn rm_try_rlock 220will return 0 if the lock cannot be acquired immediately; 221otherwise, 222the lock will be acquired and a non-zero value will be returned. 223Note that 224.Fn rm_try_rlock 225may fail even while the lock is not currently held by a writer. 226If the lock was initialized with the 227.Dv RM_RECURSE 228option, 229.Fn rm_try_rlock 230will succeed if the current thread has already acquired reader access. 231.It Fn rm_wlock "struct rmlock *rm" 232Lock 233.Fa rm 234as a writer. 235If there are any shared owners of the lock, the current thread blocks. 236The 237.Fn rm_wlock 238function cannot be called recursively. 239.It Fn rm_runlock "struct rmlock *rm" "struct rm_priotracker* tracker" 240This function releases a shared lock previously acquired by 241.Fn rm_rlock . 242The 243.Fa tracker 244argument must match the 245.Fa tracker 246argument used for acquiring the shared lock 247.It Fn rm_wunlock "struct rmlock *rm" 248This function releases an exclusive lock previously acquired by 249.Fn rm_wlock . 250.It Fn rm_destroy "struct rmlock *rm" 251This functions destroys a lock previously initialized with 252.Fn rm_init . 253The 254.Fa rm 255lock must be unlocked. 256.It Fn rm_wowned "const struct rmlock *rm" 257This function returns a non-zero value if the current thread owns an 258exclusive lock on 259.Fa rm . 260.It Fn rm_sleep "void *wchan" "struct rmlock *rm" "int priority" "const char *wmesg" "int timo" 261This function atomically releases 262.Fa rm 263while waiting for an event. 264The 265.Fa rm 266lock must be exclusively locked. 267For more details on the parameters to this function, 268see 269.Xr sleep 9 . 270.It Fn rm_assert "struct rmlock *rm" "int what" 271This function asserts that the 272.Fa rm 273lock is in the state specified by 274.Fa what . 275If the assertions are not true and the kernel is compiled with 276.Cd "options INVARIANTS" 277and 278.Cd "options INVARIANT_SUPPORT" , 279the kernel will panic. 280Currently the following base assertions are supported: 281.Bl -tag -width ".Dv RA_UNLOCKED" 282.It Dv RA_LOCKED 283Assert that current thread holds either a shared or exclusive lock 284of 285.Fa rm . 286.It Dv RA_RLOCKED 287Assert that current thread holds a shared lock of 288.Fa rm . 289.It Dv RA_WLOCKED 290Assert that current thread holds an exclusive lock of 291.Fa rm . 292.It Dv RA_UNLOCKED 293Assert that current thread holds neither a shared nor exclusive lock of 294.Fa rm . 295.El 296.Pp 297In addition, one of the following optional flags may be specified with 298.Dv RA_LOCKED , 299.Dv RA_RLOCKED , 300or 301.Dv RA_WLOCKED : 302.Bl -tag -width ".Dv RA_NOTRECURSED" 303.It Dv RA_RECURSED 304Assert that the current thread holds a recursive lock of 305.Fa rm . 306.It Dv RA_NOTRECURSED 307Assert that the current thread does not hold a recursive lock of 308.Fa rm . 309.El 310.El 311.Bl -tag -width indent 312.It Fn rms_init "struct rmslock *rms" "const char *name" 313Initialize the sleepable read-mostly lock 314.Fa rms . 315The 316.Fa name 317description is used as 318.Fa wmesg 319parameter to the 320.Xr msleep 9 321routine. 322This function must be called before any other operations on the lock. 323.It Fn rms_rlock "struct rmlock *rm" 324Lock 325.Fa rms 326as a reader. 327If any thread holds this lock exclusively, the current thread blocks. 328.It Fn rms_wlock "struct rmslock *rms" 329Lock 330.Fa rms 331as a writer. 332If the lock is already taken, the current thread blocks. 333The 334.Fn rms_wlock 335function cannot be called recursively. 336.It Fn rms_runlock "struct rmslock *rms" 337This function releases a shared lock previously acquired by 338.Fn rms_rlock . 339.It Fn rms_wunlock "struct rmslock *rms" 340This function releases an exclusive lock previously acquired by 341.Fn rms_wlock . 342.It Fn rms_destroy "struct rmslock *rms" 343This functions destroys a lock previously initialized with 344.Fn rms_init . 345The 346.Fa rms 347lock must be unlocked. 348.El 349.Sh SEE ALSO 350.Xr locking 9 , 351.Xr mutex 9 , 352.Xr panic 9 , 353.Xr rwlock 9 , 354.Xr sema 9 , 355.Xr sleep 9 , 356.Xr sx 9 357.Sh HISTORY 358These functions appeared in 359.Fx 7.0 . 360.Sh AUTHORS 361.An -nosplit 362The 363.Nm 364facility was written by 365.An "Stephan Uphoff" . 366This manual page was written by 367.An "Gleb Smirnoff" 368for rwlock and modified to reflect rmlock by 369.An "Stephan Uphoff" . 370.Sh BUGS 371The 372.Nm 373implementation is currently not optimized for single processor systems. 374.Pp 375.Fn rm_try_rlock 376can fail transiently even when there is no writer, while another reader 377updates the state on the local CPU. 378.Pp 379The 380.Nm 381implementation uses a single per CPU list shared by all 382rmlocks in the system. 383If rmlocks become popular, hashing to multiple per CPU queues may 384be needed to speed up the writer lock process. 385