1.\" Copyright (c) 2007 Stephan Uphoff <ups@FreeBSD.org> 2.\" Copyright (c) 2006 Gleb Smirnoff <glebius@FreeBSD.org> 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" Based on rwlock.9 man page 27.Dd April 12, 2021 28.Dt RMLOCK 9 29.Os 30.Sh NAME 31.Nm rmlock , 32.Nm rm_init , 33.Nm rm_init_flags , 34.Nm rm_destroy , 35.Nm rm_rlock , 36.Nm rm_try_rlock , 37.Nm rm_wlock , 38.Nm rm_runlock , 39.Nm rm_wunlock , 40.Nm rm_wowned , 41.Nm rm_sleep , 42.Nm rm_assert , 43.Nm RM_SYSINIT , 44.Nm RM_SYSINIT_FLAGS , 45.Nm rms_init , 46.Nm rms_destroy , 47.Nm rms_rlock , 48.Nm rms_wlock , 49.Nm rms_runlock , 50.Nm rms_wunlock 51.Nd kernel reader/writer lock optimized for read-mostly access patterns 52.Sh SYNOPSIS 53.In sys/param.h 54.In sys/lock.h 55.In sys/rmlock.h 56.Ft void 57.Fn rm_init "struct rmlock *rm" "const char *name" 58.Ft void 59.Fn rm_init_flags "struct rmlock *rm" "const char *name" "int opts" 60.Ft void 61.Fn rm_destroy "struct rmlock *rm" 62.Ft void 63.Fn rm_rlock "struct rmlock *rm" "struct rm_priotracker* tracker" 64.Ft int 65.Fn rm_try_rlock "struct rmlock *rm" "struct rm_priotracker* tracker" 66.Ft void 67.Fn rm_wlock "struct rmlock *rm" 68.Ft void 69.Fn rm_runlock "struct rmlock *rm" "struct rm_priotracker* tracker" 70.Ft void 71.Fn rm_wunlock "struct rmlock *rm" 72.Ft int 73.Fn rm_wowned "const struct rmlock *rm" 74.Ft int 75.Fn rm_sleep "void *wchan" "struct rmlock *rm" "int priority" "const char *wmesg" "int timo" 76.Pp 77.Cd "options INVARIANTS" 78.Cd "options INVARIANT_SUPPORT" 79.Ft void 80.Fn rm_assert "struct rmlock *rm" "int what" 81.In sys/kernel.h 82.Fn RM_SYSINIT "name" "struct rmlock *rm" "const char *desc" 83.Fn RM_SYSINIT_FLAGS "name" "struct rmlock *rm" "const char *desc" "int flags" 84.Ft void 85.Fn rms_init "struct rmslock *rms" "const char *name" 86.Ft void 87.Fn rms_destroy "struct rmslock *rms" 88.Ft void 89.Fn rms_rlock "struct rmslock *rms" 90.Ft void 91.Fn rms_wlock "struct rmslock *rms" 92.Ft void 93.Fn rms_runlock "struct rmslock *rms" 94.Ft void 95.Fn rms_wunlock "struct rmslock *rms" 96.Sh DESCRIPTION 97Read-mostly locks allow shared access to protected data by multiple threads, 98or exclusive access by a single thread. 99The threads with shared access are known as 100.Em readers 101since they only read the protected data. 102A thread with exclusive access is known as a 103.Em writer 104since it can modify protected data. 105.Pp 106Read-mostly locks are designed to be efficient for locks almost exclusively 107used as reader locks and as such should be used for protecting data that 108rarely changes. 109Acquiring an exclusive lock after the lock has been locked for shared access 110is an expensive operation. 111.Pp 112Normal read-mostly locks are similar to 113.Xr rwlock 9 114locks and follow the same lock ordering rules as 115.Xr rwlock 9 116locks. 117Read-mostly locks have full priority propagation like mutexes. 118Unlike 119.Xr rwlock 9 , 120read-mostly locks propagate priority to both readers and writers. 121This is implemented via the 122.Va rm_priotracker 123structure argument supplied to 124.Fn rm_rlock 125and 126.Fn rm_runlock . 127Readers can recurse if the lock is initialized with the 128.Dv RM_RECURSE 129option; 130however, writers are never allowed to recurse. 131.Pp 132Sleeping for writers can be allowed by passing 133.Dv RM_SLEEPABLE 134to 135.Fn rm_init_flags . 136It changes lock ordering rules to the same as for 137.Xr sx 9 138locks. 139They do not propagate priority to writers, but they do propagate priority to readers. 140Note that readers are not permitted to sleep regardless of the flag. 141.Pp 142Sleepable read-mostly locks (created with 143.Fn rms_init ) 144allow sleeping for both readers and writers, but don't do priority propagation 145for either. 146They follow 147.Xr sx 9 148lock ordering. 149.Ss Macros and Functions 150.Bl -tag -width indent 151.It Fn rm_init "struct rmlock *rm" "const char *name" 152Initialize the read-mostly lock 153.Fa rm . 154The 155.Fa name 156description is used solely for debugging purposes. 157This function must be called before any other operations 158on the lock. 159.It Fn rm_init_flags "struct rmlock *rm" "const char *name" "int opts" 160Similar to 161.Fn rm_init , 162initialize the read-mostly lock 163.Fa rm 164with a set of optional flags. 165The 166.Fa opts 167arguments contains one or more of the following flags: 168.Bl -tag -width ".Dv RM_NOWITNESS" 169.It Dv RM_NOWITNESS 170Instruct 171.Xr witness 4 172to ignore this lock. 173.It Dv RM_RECURSE 174Allow threads to recursively acquire shared locks for 175.Fa rm . 176.It Dv RM_SLEEPABLE 177Create a sleepable read-mostly lock. 178.It Dv RM_NEW 179If the kernel has been compiled with 180.Cd "option INVARIANTS" , 181.Fn rm_init_flags 182will assert that the 183.Fa rm 184has not been initialized multiple times without intervening calls to 185.Fn rm_destroy 186unless this option is specified. 187.It Dv RM_DUPOK 188.Xr witness 4 189should not log messages about duplicate locks being acquired. 190.El 191.It Fn rm_rlock "struct rmlock *rm" "struct rm_priotracker* tracker" 192Lock 193.Fa rm 194as a reader using 195.Fa tracker 196to track read owners of a lock for priority propagation. 197This data structure is only used internally by 198.Nm 199and must persist until 200.Fn rm_runlock 201has been called. 202This data structure can be allocated on the stack since 203readers cannot sleep. 204If any thread holds this lock exclusively, the current thread blocks, 205and its priority is propagated to the exclusive holder. 206If the lock was initialized with the 207.Dv RM_RECURSE 208option the 209.Fn rm_rlock 210function can be called when the current thread has already acquired reader 211access on 212.Fa rm . 213.It Fn rm_try_rlock "struct rmlock *rm" "struct rm_priotracker* tracker" 214Try to lock 215.Fa rm 216as a reader. 217.Fn rm_try_rlock 218will return 0 if the lock cannot be acquired immediately; 219otherwise, 220the lock will be acquired and a non-zero value will be returned. 221Note that 222.Fn rm_try_rlock 223may fail even while the lock is not currently held by a writer. 224If the lock was initialized with the 225.Dv RM_RECURSE 226option, 227.Fn rm_try_rlock 228will succeed if the current thread has already acquired reader access. 229.It Fn rm_wlock "struct rmlock *rm" 230Lock 231.Fa rm 232as a writer. 233If there are any shared owners of the lock, the current thread blocks. 234The 235.Fn rm_wlock 236function cannot be called recursively. 237.It Fn rm_runlock "struct rmlock *rm" "struct rm_priotracker* tracker" 238This function releases a shared lock previously acquired by 239.Fn rm_rlock . 240The 241.Fa tracker 242argument must match the 243.Fa tracker 244argument used for acquiring the shared lock 245.It Fn rm_wunlock "struct rmlock *rm" 246This function releases an exclusive lock previously acquired by 247.Fn rm_wlock . 248.It Fn rm_destroy "struct rmlock *rm" 249This functions destroys a lock previously initialized with 250.Fn rm_init . 251The 252.Fa rm 253lock must be unlocked. 254.It Fn rm_wowned "const struct rmlock *rm" 255This function returns a non-zero value if the current thread owns an 256exclusive lock on 257.Fa rm . 258.It Fn rm_sleep "void *wchan" "struct rmlock *rm" "int priority" "const char *wmesg" "int timo" 259This function atomically releases 260.Fa rm 261while waiting for an event. 262The 263.Fa rm 264lock must be exclusively locked. 265For more details on the parameters to this function, 266see 267.Xr sleep 9 . 268.It Fn rm_assert "struct rmlock *rm" "int what" 269This function asserts that the 270.Fa rm 271lock is in the state specified by 272.Fa what . 273If the assertions are not true and the kernel is compiled with 274.Cd "options INVARIANTS" 275and 276.Cd "options INVARIANT_SUPPORT" , 277the kernel will panic. 278Currently the following base assertions are supported: 279.Bl -tag -width ".Dv RA_UNLOCKED" 280.It Dv RA_LOCKED 281Assert that current thread holds either a shared or exclusive lock 282of 283.Fa rm . 284.It Dv RA_RLOCKED 285Assert that current thread holds a shared lock of 286.Fa rm . 287.It Dv RA_WLOCKED 288Assert that current thread holds an exclusive lock of 289.Fa rm . 290.It Dv RA_UNLOCKED 291Assert that current thread holds neither a shared nor exclusive lock of 292.Fa rm . 293.El 294.Pp 295In addition, one of the following optional flags may be specified with 296.Dv RA_LOCKED , 297.Dv RA_RLOCKED , 298or 299.Dv RA_WLOCKED : 300.Bl -tag -width ".Dv RA_NOTRECURSED" 301.It Dv RA_RECURSED 302Assert that the current thread holds a recursive lock of 303.Fa rm . 304.It Dv RA_NOTRECURSED 305Assert that the current thread does not hold a recursive lock of 306.Fa rm . 307.El 308.El 309.Bl -tag -width indent 310.It Fn rms_init "struct rmslock *rms" "const char *name" 311Initialize the sleepable read-mostly lock 312.Fa rms . 313The 314.Fa name 315description is used as 316.Fa wmesg 317parameter to the 318.Xr msleep 9 319routine. 320This function must be called before any other operations on the lock. 321.It Fn rms_rlock "struct rmlock *rm" 322Lock 323.Fa rms 324as a reader. 325If any thread holds this lock exclusively, the current thread blocks. 326.It Fn rms_wlock "struct rmslock *rms" 327Lock 328.Fa rms 329as a writer. 330If the lock is already taken, the current thread blocks. 331The 332.Fn rms_wlock 333function cannot be called recursively. 334.It Fn rms_runlock "struct rmslock *rms" 335This function releases a shared lock previously acquired by 336.Fn rms_rlock . 337.It Fn rms_wunlock "struct rmslock *rms" 338This function releases an exclusive lock previously acquired by 339.Fn rms_wlock . 340.It Fn rms_destroy "struct rmslock *rms" 341This functions destroys a lock previously initialized with 342.Fn rms_init . 343The 344.Fa rms 345lock must be unlocked. 346.El 347.Sh SEE ALSO 348.Xr locking 9 , 349.Xr mutex 9 , 350.Xr panic 9 , 351.Xr rwlock 9 , 352.Xr sema 9 , 353.Xr sleep 9 , 354.Xr sx 9 355.Sh HISTORY 356These functions appeared in 357.Fx 7.0 . 358.Sh AUTHORS 359.An -nosplit 360The 361.Nm 362facility was written by 363.An "Stephan Uphoff" . 364This manual page was written by 365.An "Gleb Smirnoff" 366for rwlock and modified to reflect rmlock by 367.An "Stephan Uphoff" . 368.Sh BUGS 369The 370.Nm 371implementation is currently not optimized for single processor systems. 372.Pp 373.Fn rm_try_rlock 374can fail transiently even when there is no writer, while another reader 375updates the state on the local CPU. 376.Pp 377The 378.Nm 379implementation uses a single per CPU list shared by all 380rmlocks in the system. 381If rmlocks become popular, hashing to multiple per CPU queues may 382be needed to speed up the writer lock process. 383