Lines Matching +full:wait +full:- +full:on +full:- +full:read
1 .. SPDX-License-Identifier: GPL-2.0
11 would cause. This list is based on experiences reviewing such patches
14 0. Is RCU being applied to a read-mostly situation? If the data
18 tool for the job. Yes, RCU does reduce read-side overhead by
19 increasing write-side overhead, which is exactly why normal uses
24 is the dynamic NMI code in the Linux 2.6 kernel, at least on
27 Yet another exception is where the low real-time latency of RCU's
28 read-side primitives is critically important.
33 counter-intuitive situation where rcu_read_lock() and
48 memory barriers on weakly ordered machines (pretty much all of
49 them -- even x86 allows later loads to be reordered to precede
53 on large systems (for example, if the task is updating information
54 relating to itself that other tasks can read, there by definition
59 2. Do the RCU read-side critical sections make proper use of
63 under your read-side code, which can greatly increase the
66 As a rough rule of thumb, any dereference of an RCU-protected
68 rcu_read_lock_sched(), or by the appropriate update-side lock.
72 spinlock also enters an RCU read-side critical section.
74 Please note that you *cannot* rely on code known to be built
75 only in non-preemptible kernels. Such code can and will break,
78 Letting RCU-protected pointers "leak" out of an RCU read-side
82 *before* letting them out of the RCU read-side critical section.
89 of ways to handle this concurrency, depending on the situation:
92 primitives to add, remove, and replace elements on
93 an RCU-protected list. Alternatively, use the other
94 RCU-protected data structures that have been added to
99 b. Proceed as in (a) above, but also maintain per-element
101 that guard per-element state. Fields that the readers
114 thus solving the multiple-field problem by imposing an
123 usually liberally sprinkle memory-ordering operations
135 are weakly ordered -- even x86 CPUs allow later loads to be
137 the following measures to prevent memory-corruption problems:
143 on Alpha CPUs.
157 various "_rcu()" list-traversal primitives, such
159 perfectly legal (if redundant) for update-side code to
160 use rcu_dereference() and the "_rcu()" list-traversal
164 of an RCU read-side critical section. See lockdep.rst
168 list-traversal primitives can substitute for a good
180 poisoning from inflicting toxic effects on concurrent
186 in their respective types of RCU-protected lists.
189 type of RCU-protected linked lists.
195 be traversed by an RCU read-side critical section.
212 as the non-expedited forms, but expediting is more CPU intensive.
214 configuration-change operations that would not normally be
215 undertaken while a real-time workload is running. Note that
216 IPI-sensitive real-time workloads can use the rcupdate.rcu_normal
223 a single non-expedited primitive to cover the entire batch.
225 expedited primitive, and will be much much easier on the rest
226 of the system, especially to real-time workloads running on the
231 is RCU-sched for PREEMPTION=n and RCU-preempt for PREEMPTION=y.
235 and re-enables softirq, for example, rcu_read_lock_bh() and
237 and re-enables preemption, for example, rcu_read_lock_sched() and
241 srcu_struct. The rules for the expedited RCU grace-period-wait
242 primitives are the same as for their non-expedited counterparts.
263 when using non-obvious pairs of primitives, commenting is
264 of course a must. One example of non-obvious pairing is
266 network-driver NAPI (softirq) context. BPF relies heavily on RCU
280 synchronize_rcu()'s multi-millisecond latency. So please take
282 memory-freeing capabilities where it applies.
285 primitive is that it automatically self-limits: if grace periods
292 Ways of gaining this self-limiting property when using call_rcu(),
295 a. Keeping a count of the number of data-structure elements
296 used by the RCU-protected data structure, including
298 limit on this number, stalling updates as needed to allow
303 One way to stall the updates is to acquire the update-side
304 mutex. (Don't try this with a spinlock -- other CPUs
305 spinning on the lock could prevent the grace period
311 variations on this theme.
319 c. Trusted update -- if updates can only be done manually by
339 9. All RCU list-traversal primitives, which include
341 list_for_each_safe_rcu(), must be either within an RCU read-side
342 critical section or must be protected by appropriate update-side
343 locks. RCU read-side critical sections are delimited by
349 The reason that it is permissible to use RCU list-traversal
350 primitives when the update-side lock is held is that doing so
359 and the read-side markers (rcu_read_lock() and rcu_read_unlock(),
362 10. Conversely, if you are in an RCU read-side critical section,
363 and you don't hold the appropriate update-side lock, you *must*
370 disable softirq on a given acquisition of that lock will result
378 an issue, the memory-allocator locking handles it). However,
383 Do not assume that RCU callbacks will be executed on the same
387 then that RCU callback will execute on some surviving CPU.
388 (If this was not the case, a self-spawning RCU callback would
391 RCU callbacks executed on some other CPUs, in fact, for some
392 real-time workloads, this is the whole point of using the
396 will be invoked in that order, even if they all are queued on the
397 same CPU. Furthermore, do not assume that same-CPU callbacks will
399 switched between offloaded and de-offloaded callback invocation,
406 SRCU read-side critical section (demarked by srcu_read_lock()
408 Please note that if you don't need to sleep in read-side critical
420 synchronize_srcu() waits only for SRCU read-side critical
423 is what makes sleeping read-side critical sections tolerable --
426 system than RCU would be if RCU's read-side critical sections
429 The ability to sleep in read-side critical sections does not
432 Second, grace-period-detection overhead is amortized only
436 only in extremely read-intensive situations, or in situations
437 requiring SRCU's read-side deadlock immunity or low read-side
442 never sends IPIs to other CPUs, so it is easier on
443 real-time workloads than is synchronize_rcu_expedited().
445 It is also permissible to sleep in RCU Tasks Trace read-side
456 is to wait until all pre-existing readers have finished before
457 carrying out some otherwise-destructive operation. It is
463 Because these primitives only wait for pre-existing readers, it
467 15. The various RCU read-side primitives do *not* necessarily contain
470 read-side critical sections. It is the responsibility of the
471 RCU update-side primitives to deal with this.
481 check that accesses to RCU-protected data structures
482 are carried out under the proper RCU read-side critical
494 of RCU read-side critical sections. This Kconfig
495 option is tough on both performance and scalability,
496 and so is limited to four-CPU systems.
499 tag the pointer to the RCU-protected data structure
509 call_rcu_tasks_trace(), then it is necessary to wait for all
511 Note that it is absolutely *not* sufficient to wait for a grace
513 guaranteed to wait for callbacks registered on other CPUs via
514 call_rcu(). Or even on the current CPU if that CPU recently
519 - call_rcu() -> rcu_barrier()
520 - call_srcu() -> srcu_barrier()
521 - call_rcu_tasks() -> rcu_barrier_tasks()
522 - call_rcu_tasks_trace() -> rcu_barrier_tasks_trace()
525 to wait for a grace period. For example, if there are no
529 So if you need to wait for both a grace period and for all
530 pre-existing callbacks, you will need to invoke both functions,
531 with the pair depending on the flavor of RCU:
533 - Either synchronize_rcu() or synchronize_rcu_expedited(),
535 - Either synchronize_srcu() or synchronize_srcu_expedited(),
537 - synchronize_rcu_tasks() and rcu_barrier_tasks()
538 - synchronize_tasks_trace() and rcu_barrier_tasks_trace()