xref: /freebsd/share/man/man9/locking.9 (revision 815e47727edf9d819ab670fdf6a0a18e797ba909)

.\" Copyright (c) 2007 Julian Elischer  (julian -  freebsd org )
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
.\" ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
.\" SUCH DAMAGE.
.\"
.\" $FreeBSD$
.\"
.Dd January 29, 2010
.Dt LOCKING 9
.Os
.Sh NAME
.Nm locking
.Nd kernel synchronization primitives
.Sh DESCRIPTION
The
.Em FreeBSD
kernel is written to run across multiple CPUs and as such requires
several different synchronization primitives to allow the developers
to safely access and manipulate the many data types required.
.Pp
These include:
.Bl -enum
.It
Mutexes
.It
Spin mutexes
.It
Pool mutexes
.It
Shared/exclusive locks
.It
Reader/writer locks
.It
Read-mostly locks
.It
Counting semaphores
.It
Condition variables
.It
Sleep/wakeup
.It
Giant
.It
Lockmanager locks
.El
.Pp
The primitives interact and have a number of rules regarding how
they can and can not be combined.
Many of these rules are checked using the
.Xr witness 4
code.
.Pp
.Ss Mutexes
Mutexes are the most commonly used synchronization primitive in the kernel.
Thread acquires (locks) a mutex before accessing data shared with other
threads (including interrupt threads), and releases (unlocks) it afterwards.
If the mutex cannot be acquired, the thread requesting it will block.
.Pp
Sleeping while holding mutex is generally prohibited.
You may only call the
.Xr sleep 9
call via
.Fn msleep
or the new
.Fn mtx_sleep
variant.
These will atomically drop the mutex and reacquire it
as part of waking up.
This is often however a
.Em BAD
idea because it generally relies on you having
such a good knowledge of all the call graph above you
and what assumptions it is making that there are a lot
of ways to make hard-to-find mistakes.
For example you MUST re-test all the assumptions you made before,
all the way up the call graph to where you got the lock.
You can not just assume that mtx_sleep can be inserted anywhere.
If any caller above you has any mutex or
rwlock, your sleep, will cause a panic.
If the sleep only happens rarely it may be years before the
bad code path is found.
.Pp
See the
.Xr mutex 9
page for more information.
.Ss Spin mutexes
Spin mutexes are variation of basic mutexes; the main difference between
the two is that spin mutexes never block - instead, they spin, waiting
for the thread holding the lock, which runs on another CPU, to release it.
Differently from ordinary mutex, spin mutexes disable interrupts when acquired.
Since disabling interrupts is expensive, they are also generally slower.
Spin mutexes should only be used to protect data shared with primary
(INTR_FILTER) interrupt code.
You
.Em must not
do anything that deschedules the thread while you
are holding a spin mutex.
.Ss Pool mutexes
With most synchronisaton primitives, such as mutexes, programmer must
provide a piece of allocated memory to hold the primitive.
For example, a mutex may be embedded inside the structure it protects.
Pool mutex is a variant of mutex without this requirement - to lock or unlock
a pool mutex, one uses address of the structure being protected with it,
not the mutex itself.
Pool mutexes are seldom used.
.Pp
See the
.Xr mtx_pool 9
page for more information.
.Ss Reader/writer locks
Reader/writer locks allow shared access to protected data by multiple threads,
or exclusive access by a single thread.
The threads with shared access are known as
.Em readers
since they should only read the protected data.
A thread with exclusive access is known as a
.Em writer
since it may modify protected data.
.Pp
Although reader/writer locks look very similar to
.Xr sx 9
(see below) locks, their usage pattern is different.
Reader/writer locks can be treated as mutexes (see above and
.Xr mutex 9 )
with shared/exclusive semantics.
More specifically, regular mutexes can be
considered to be equivalent to a write-lock on an
.Em rw_lock.
In the future this may in fact
become literally the fact.
An
.Em rw_lock
can be locked while holding a regular mutex, but
can
.Em not
be held while sleeping.
The
.Em rw_lock
locks have priority propagation like mutexes, but priority
can be propagated only to an exclusive holder.
This limitation comes from the fact that shared owners
are anonymous.
Another important property is that shared holders of
.Em rw_lock
can recurse, but exclusive locks are not allowed to recurse.
This ability should not be used lightly and
.Em may go away.
Users of recursion in any locks should be prepared to
defend their decision against vigorous criticism.
.Pp
See the
.Xr rwlock 9
page for more information.
.Ss Read-mostly locks
Mostly reader locks are similar to
.Em Reader/write
locks but optimized for very infrequent
.Em writer
locking.
.Em rm_lock
locks implement full priority propagation by tracking shared owners
using a lock user supplied
.Em tracker
data structure.
.Pp
See the
.Xr rmlock 9
page for more information.
.Ss Shared/exclusive locks
Shared/exclusive locks are used to protect data that are read far more often
than they are written.
Mutexes are inherently more efficient than shared/exclusive locks, so
shared/exclusive locks should be used prudently.
The main reason for using an
.Em sx_lock
is that a thread may hold a shared or exclusive lock on an
.Em sx_lock
lock while sleeping.
As a consequence of this however, an
.Em sx_lock
lock may not be acquired while holding a mutex.
The reason for this is that, if one thread slept while holding an
.Em sx_lock
lock while another thread blocked on the same
.Em sx_lock
lock after acquiring a mutex, then the second thread would effectively
end up sleeping while holding a mutex, which is not allowed.
The
.Em sx_lock
should be considered to be closely related to
.Xr sleep 9 .
In fact it could in some cases be
considered a conditional sleep.
.Pp
See the
.Xr sx 9
page for more information.
.Ss Counting semaphores
Counting semaphores provide a mechanism for synchronizing access
to a pool of resources.
Unlike mutexes, semaphores do not have the concept of an owner,
so they can be useful in situations where one thread needs
to acquire a resource, and another thread needs to release it.
They are largely deprecated.
.Pp
See the
.Xr sema 9
page for more information.
.Ss Condition variables
Condition variables are used in conjunction with mutexes to wait for
conditions to occur.
A thread must hold the mutex before calling the
.Fn cv_wait* ,
functions.
When a thread waits on a condition, the mutex
is atomically released before the thread is blocked, then reacquired
before the function call returns.
.Pp
See the
.Xr condvar 9
page for more information.
.Ss Giant
Giant is a special instance of a sleep lock.
It has several special characteristics.
.Bl -enum
.It
It is recursive.
.It
Drivers can request that Giant be locked around them, but this is
going away.
.It
You can sleep while it has recursed, but other recursive locks cannot.
.It
Giant must be locked first before other locks.
.It
There are places in the kernel that drop Giant and pick it back up
again.
Sleep locks will do this before sleeping.
Parts of the Network or VM code may do this as well, depending on the
setting of a sysctl.
This means that you cannot count on Giant keeping other code from
running if your code sleeps, even if you want it to.
.El
.Ss Sleep/wakeup
The functions
.Fn tsleep ,
.Fn msleep ,
.Fn msleep_spin ,
.Fn pause ,
.Fn wakeup ,
and
.Fn wakeup_one
handle event-based thread blocking.
If a thread must wait for an external event, it is put to sleep by
.Fn tsleep ,
.Fn msleep ,
.Fn msleep_spin ,
or
.Fn pause .
Threads may also wait using one of the locking primitive sleep routines
.Xr mtx_sleep 9 ,
.Xr rw_sleep 9 ,
or
.Xr sx_sleep 9 .
.Pp
The parameter
.Fa chan
is an arbitrary address that uniquely identifies the event on which
the thread is being put to sleep.
All threads sleeping on a single
.Fa chan
are woken up later by
.Fn wakeup ,
often called from inside an interrupt routine, to indicate that the
resource the thread was blocking on is available now.
.Pp
Several of the sleep functions including
.Fn msleep ,
.Fn msleep_spin ,
and the locking primitive sleep routines specify an additional lock
parameter.
The lock will be released before sleeping and reacquired
before the sleep routine returns.
If
.Fa priority
includes the
.Dv PDROP
flag, then the lock will not be reacquired before returning.
The lock is used to ensure that a condition can be checked atomically,
and that the current thread can be suspended without missing a
change to the condition, or an associated wakeup.
In addition, all of the sleep routines will fully drop the
.Va Giant
mutex
(even if recursed)
while the thread is suspended and will reacquire the
.Va Giant
mutex before the function returns.
.Pp
See the
.Xr sleep 9
page for more information.
.Pp
.Ss Lockmanager locks
Shared/exclusive sleep locks, used mostly in
.Xr VFS 9 ,
in particular as a
.Xr vnode 9
lock.
They have features other lock types don't have, such as sleep timeout,
writer starvation avoidance, draining, and interlock mutex, but this makes them
complicated to implement; for this reason, they are deprecated.
.Pp
See the
.Xr lock 9
page for more information.
.Sh INTERACTIONS
.Ss Interaction table.
The following table shows what you can and can not do if you hold
one of the synchronization primitives discussed here:
(someone who knows what they are talking about should write this table)
.Bl -column ".Ic xxxxxxxxxxxxxxxxxxxx" ".Xr XXXXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXXX" ".Xr XXXXX" -offset indent
.It Xo
.Em "You have: You want:" Ta Spin_mtx Ta Slp_mtx Ta sx_lock Ta rw_lock Ta rm_lock Ta sleep
.Xc
.It Ic SPIN mutex  Ta \&ok-1 Ta \&no Ta \&no Ta \&no Ta \&no Ta \&no-3
.It Ic Sleep mutex Ta \&ok Ta \&ok-1 Ta \&no Ta \&ok Ta \&ok Ta \&no-3
.It Ic sx_lock     Ta \&ok Ta \&ok Ta \&ok-2 Ta \&ok Ta \&ok Ta \&ok-4
.It Ic rw_lock     Ta \&ok Ta \&ok Ta \&no Ta \&ok-2 Ta \&ok Ta \&no-3
.It Ic rm_lock     Ta \&ok Ta \&ok Ta \&no Ta \&ok Ta \&ok-2 Ta \&no
.El
.Pp
.Em *1
Recursion is defined per lock.
Lock order is important.
.Pp
.Em *2
readers can recurse though writers can not.
Lock order is important.
.Pp
.Em *3
There are calls atomically release this primitive when going to sleep
and reacquire it on wakeup (e.g.
.Fn mtx_sleep ,
.Fn rw_sleep
and
.Fn msleep_spin
).
.Pp
.Em *4
Though one can sleep holding an sx lock, one can also use
.Fn sx_sleep
which atomically release this primitive when going to sleep and
reacquire it on wakeup.
.Ss Context mode table.
The next table shows what can be used in different contexts.
At this time this is a rather easy to remember table.
.Bl -column ".Ic Xxxxxxxxxxxxxxxxxxxx" ".Xr XXXXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXXX" ".Xr XXXXX" -offset indent
.It Xo
.Em "Context:" Ta Spin_mtx Ta Slp_mtx Ta sx_lock Ta rw_lock Ta rm_lock Ta sleep
.Xc
.It interrupt:  Ta \&ok Ta \&no Ta \&no Ta \&no Ta \&no Ta \&no
.It idle:  Ta \&ok Ta \&no Ta \&no Ta \&no Ta \&no Ta \&no
.El
.Sh SEE ALSO
.Xr condvar 9 ,
.Xr lock 9 ,
.Xr mtx_pool 9 ,
.Xr mutex 9 ,
.Xr rmlock 9 ,
.Xr rwlock 9 ,
.Xr sema 9 ,
.Xr sleep 9 ,
.Xr sx 9 ,
.Xr witness 9 ,
.Xr LOCK_PROFILING 9
.Sh HISTORY
These
functions appeared in
.Bsx 4.1
through
.Fx 7.0
.Sh BUGS
There are too many locking primitives to choose from.