| ec4fad7c | 07-Aug-2026 |
Lyude Paul <lyude@redhat.com> |
rust: sync: Introduce SpinLockIrq::lock_with() and friends
`SpinLockIrq` and `SpinLock` use the exact same underlying C structure,
with the only real difference being that the former uses the
irq_di
rust: sync: Introduce SpinLockIrq::lock_with() and friends
`SpinLockIrq` and `SpinLock` use the exact same underlying C structure,
with the only real difference being that the former uses the
irq_disable() and irq_enable() variants for locking/unlocking. These
variants can introduce some minor overhead in contexts where we already
know that local processor interrupts are disabled, and as such we want a
way to be able to skip modifying processor interrupt state in said
contexts in order to avoid some overhead - just like the current C API
allows us to do.
In order to do this, we add some special functions for SpinLockIrq:
lock_with() and try_lock_with(), which allow acquiring the lock without
changing the interrupt state - as long as the caller can provide a
LocalInterruptDisabled reference to prove that local processor
interrupts have been disabled.
In some hacked-together benchmarks we ran, most of the time this did
actually seem to lead to a noticeable difference in overhead:
From an aarch64 VM running on a MacBook M4:
lock() when irq is disabled, 100 times cost Delta { nanos: 500 }
lock_with() when irq is disabled, 100 times cost Delta { nanos: 292 }
lock() when irq is enabled, 100 times cost Delta { nanos: 834 }
lock() when irq is disabled, 100 times cost Delta { nanos: 459 }
lock_with() when irq is disabled, 100 times cost Delta { nanos: 291 }
lock() when irq is enabled, 100 times cost Delta { nanos: 709 }
From an x86_64 VM (qemu/kvm) running on a i7-13700H
lock() when irq is disabled, 100 times cost Delta { nanos: 1002 }
lock_with() when irq is disabled, 100 times cost Delta { nanos: 729 }
lock() when irq is enabled, 100 times cost Delta { nanos: 1516 }
lock() when irq is disabled, 100 times cost Delta { nanos: 754 }
lock_with() when irq is disabled, 100 times cost Delta { nanos: 966 }
lock() when irq is enabled, 100 times cost Delta { nanos: 1227 }
(note that there were some runs on x86_64 where lock() on irq
disabled vs. lock_with() on irq disabled had equivalent benchmarks,
but it very much appeared to be a minority of test runs.)
While it's not clear how this affects real-world workloads yet, let's
add this for the time being so we can find out.
This makes it so that a `SpinLockIrq` will work like a `SpinLock` if
interrupts are disabled. So a function:
(&'a SpinLockIrq, &'a LocalInterruptDisabled) -> Guard<'a, .., SpinLockBackend>
makes sense. Note that due to `Guard` and `LocalInterruptDisabled`
having the same lifetime, interrupts cannot be enabled while the Guard
exists.
Signed-off-by: Lyude Paul <lyude@redhat.com>
Signed-off-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260807070218.27144-19-boqun@kernel.org
show more ...
|
| 5967f4df | 07-Aug-2026 |
Lyude Paul <lyude@redhat.com> |
rust: sync: Add SpinLockIrq
A variant of `SpinLock` that ensures interrupts are disabled in the
critical section. `lock()` will ensure that either interrupts are
already disabled or disable them. `u
rust: sync: Add SpinLockIrq
A variant of `SpinLock` that ensures interrupts are disabled in the
critical section. `lock()` will ensure that either interrupts are
already disabled or disable them. `unlock()` will reverse the respective
operation.
[Boqun: Port to use spin_lock_irq_disable() and
spin_unlock_irq_enable()]
Signed-off-by: Lyude Paul <lyude@redhat.com>
Signed-off-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260807070218.27144-18-boqun@kernel.org
show more ...
|
| fbd7a5a0 | 25-Nov-2024 |
Lyude Paul <lyude@redhat.com> |
rust: sync: Add lock::Backend::assert_is_held()
Since we've exposed Lock::from_raw() and Guard::new() publically, we
want to be able to make sure that we assert that a lock is actually held
when con
rust: sync: Add lock::Backend::assert_is_held()
Since we've exposed Lock::from_raw() and Guard::new() publically, we
want to be able to make sure that we assert that a lock is actually held
when constructing a Guard for it to handle instances of unsafe
Guard::new() calls outside of our lock module.
Hence add a new method assert_is_held() to Backend, which uses lockdep
to check whether or not a lock has been acquired. When lockdep is
disabled, this has no overhead.
[Boqun: Resolve the conflicts with exposing Guard::new(), reword the
commit log a bit and format "unsafe { <statement>; }" into "unsafe {
<statement> }" for the consistency. ]
Signed-off-by: Lyude Paul <lyude@redhat.com>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Link: https://lore.kernel.org/r/20241125204139.656801-1-lyude@redhat.com
show more ...
|
| eb5ccb03 | 20-Nov-2024 |
Lyude Paul <lyude@redhat.com> |
rust: sync: Add SpinLockGuard type alias
A simple helper alias for code that needs to deal with Guard types returned
from SpinLocks.
Signed-off-by: Lyude Paul <lyude@redhat.com>
Reviewed-by: Alice
rust: sync: Add SpinLockGuard type alias
A simple helper alias for code that needs to deal with Guard types returned
from SpinLocks.
Signed-off-by: Lyude Paul <lyude@redhat.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Link: https://lore.kernel.org/r/20241120222742.2490495-3-lyude@redhat.com
show more ...
|
| 00280272 | 01-Apr-2024 |
Miguel Ojeda <ojeda@kernel.org> |
rust: kernel: remove redundant imports
Rust's `unused_imports` lint covers both unused and redundant imports.
In the upcoming 1.78.0, the lint detects more cases of redundant imports
[1], e.g.:
rust: kernel: remove redundant imports
Rust's `unused_imports` lint covers both unused and redundant imports.
In the upcoming 1.78.0, the lint detects more cases of redundant imports
[1], e.g.:
error: the item `bindings` is imported redundantly
--> rust/kernel/print.rs:38:9
|
38 | use crate::bindings;
| ^^^^^^^^^^^^^^^ the item `bindings` is already defined by prelude
Most cases are `use crate::bindings`, plus a few other items like `Box`.
Thus clean them up.
Note that, in the `bindings` case, the message "defined by prelude"
above means the extern prelude, i.e. the `--extern` flags we pass.
Link: https://github.com/rust-lang/rust/pull/117772 [1]
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Link: https://lore.kernel.org/r/20240401212303.537355-3-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
show more ...
|
| e283ee23 | 29-Jan-2024 |
Alice Ryhl <aliceryhl@google.com> |
rust: kernel: add reexports for macros
Currently, all macros are reexported with #[macro_export] only, which
means that to access `new_work!` from the workqueue, you need to import
it from the path
rust: kernel: add reexports for macros
Currently, all macros are reexported with #[macro_export] only, which
means that to access `new_work!` from the workqueue, you need to import
it from the path `kernel::new_work` instead of importing it from the
workqueue module like all other items in the workqueue. By adding
reexports of the macros, it becomes possible to import the macros from
the correct modules.
It's still possible to import the macros from the root, but I don't
think we can do anything about that.
There is no functional change. This is merely a code cleanliness
improvement.
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Trevor Gross <tmgross@umich.edu>
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Tested-by: Boqun Feng <boqun.feng@gmail.com>
Link: https://lore.kernel.org/r/20240129145837.1419880-1-aliceryhl@google.com
[ Removed new `use kernel::prelude::*`s, reworded title. ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
show more ...
|
| e32cca32 | 27-Mar-2023 |
Wedson Almeida Filho <walmeida@microsoft.com> |
rust: lock: add `Guard::do_unlocked`
It releases the lock, executes some function provided by the caller,
then reacquires the lock. This is preparation for the implementation of
condvars, which will
rust: lock: add `Guard::do_unlocked`
It releases the lock, executes some function provided by the caller,
then reacquires the lock. This is preparation for the implementation of
condvars, which will sleep after between unlocking and relocking.
We need an explicit `relock` method for primitives like `SpinLock` that
have an irqsave variant: we use the guard state to determine if the lock
was originally acquired with the regular `lock` function or
`lock_irqsave`.
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
Link: https://lore.kernel.org/rust-for-linux/20230412121431.41627-1-wedsonaf@gmail.com/
[ Removed the irqsave bits as discussed. ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
show more ...
|