| /linux/rust/kernel/sync/atomic/ |
| H A D | ordering.rs | 7 //! - [`Acquire`] provides ordering between the load part of the annotated operation and all the
9 //! ordering.
10 //! - [`Release`] provides ordering between all the preceding memory accesses and the store part of
12 //! ordering.
14 //! - It provides ordering between all the preceding memory accesses and the annotated operation.
15 //! - It provides ordering between the annotated operation and all the following memory accesses.
16 //! - It provides ordering between all the preceding memory accesses and all the following memory
19 //! - [`Relaxed`] provides no ordering except the dependency orderings. Dependency orderings are
25 /// The annotation type for relaxed memory ordering, for the description of relaxed memory
26 /// ordering, see [module-level documentation].
[all …]
|
| /linux/tools/memory-model/Documentation/ |
| H A D | ordering.txt | 1 This document gives an overview of the categories of memory-ordering
5 Categories of Ordering
8 This section lists LKMM's three top-level categories of memory-ordering
20 3. Unordered accesses, as the name indicates, have no ordering
23 some of these "unordered" operations provide limited ordering
38 b. Read-modify-write (RMW) ordering augmentation barriers.
50 ordering primitives provided for that purpose. For example, instead of
58 The Linux-kernel primitives that provide full ordering include:
79 memory-ordering primitives. It is surprisingly hard to remember their
82 Second, some RMW atomic operations provide full ordering. These
[all …]
|
| H A D | control-dependencies.txt | 12 Therefore, a load-load control dependency will not preserve ordering
19 This is not guaranteed to provide any ordering because some types of CPUs
31 However, stores are not speculated. This means that ordering is
40 of ordering. But please note that neither the READ_ONCE() nor the
59 It is tempting to try use control dependencies to enforce ordering on
78 WRITE_ONCE(b, 1); /* BUG: No ordering vs. load from a!!! */
91 have been applied. Therefore, if you need ordering in this example,
92 you must use explicit memory ordering, for example, smp_store_release():
103 Without explicit memory ordering, control-dependency-based ordering is
121 preserve ordering. For example:
[all …]
|
| H A D | simple.txt | 2 memory-ordering lives simple, as is necessary for those whose domain
3 is complex. After all, there are bugs other than memory-ordering bugs,
4 and the time spent gaining memory-ordering knowledge is not available
139 memory ordering.
175 2. Operations that did not return a value and provided no ordering,
178 3. Operations that returned a value and provided full ordering, such as
180 value-returning operations provide full ordering only conditionally.
181 For example, cmpxchg() provides ordering only upon success.
184 provide full ordering. These are flagged with either a _relaxed()
185 suffix (providing no ordering), or an _acquire() or _release() suffix
[all …]
|
| H A D | cheatsheet.txt | 25 C: Ordering is cumulative
26 P: Ordering propagates
29 Y: Provides ordering
30 a: Provides ordering given intervening RMW atomic operation
|
| H A D | recipes.txt | 41 your full-ordering warranty, as do undersized accesses that load
161 lock's ordering properties.
163 Ordering can be extended to CPUs not holding the lock by careful use
212 In the absence of any ordering, this goal may not be met, as can be seen
221 the desired MP ordering. The general approach is shown below:
276 The rcu_assign_pointer() macro has the same ordering properties as does
361 absence of any ordering it is quite possible that this may happen, as
438 The ordering in this example is stronger than it needs to be. For
439 example, ordering would still be preserved if CPU1()'s smp_load_acquire()
472 well as simple and powerful, at least as memory-ordering mechanisms go.
[all …]
|
| H A D | README | 18 that the Linux kernel provides: ordering.txt
93 ordering.txt
94 Overview of the Linux kernel's low-level memory-ordering
98 Common memory-ordering patterns.
|
| /linux/rust/kernel/sync/ |
| H A D | atomic.rs | 20 pub mod ordering; module
24 pub use ordering::{Acquire, Full, Relaxed, Release};
28 use ordering::OrderingType;
267 pub fn load<Ordering: ordering::AcquireOrRelaxed>(&self, _: Ordering) -> T { in load()
269 match Ordering::TYPE { in load()
272 _ => build_error!("Wrong ordering"), in load()
297 pub fn store<Ordering: ordering::ReleaseOrRelaxed>(&self, v: T, _: Ordering) { in store() argument
301 match Ordering::TYPE { in store()
304 _ => build_error!("Wrong ordering"), in store()
329 pub fn xchg<Ordering: ordering::Ordering>(&self, v: T, _: Ordering) -> T { in xchg()
[all …]
|
| /linux/Documentation/RCU/Design/Memory-Ordering/ |
| H A D | Tree-RCU-Memory-Ordering.rst | 2 A Tour Through TREE_RCU's Grace-Period Memory Ordering
13 grace-period memory ordering guarantee is provided.
15 What Is Tree RCU's Grace Period Memory Ordering Guarantee?
18 RCU grace periods provide extremely strong memory-ordering guarantees
46 Tree RCU Grace Period Memory Ordering Building Blocks
49 The workhorse for RCU's grace-period memory ordering is the
72 Tree RCU uses these two ordering guarantees to form an ordering
77 The following litmus test exhibits the ordering effects of these
126 | Because we must provide ordering for RCU's polling grace-period |
145 RCU's grace-period memory ordering guarantee to extend to any
[all …]
|
| /linux/arch/mips/include/asm/ |
| H A D | sync.h | 16 * 2) Ordering barriers, which only ensure that affected memory operations
20 * Ordering barriers can be more efficient than completion barriers, since:
22 * a) Ordering barriers only require memory access instructions which precede
31 * b) Multiple variants of ordering barrier are provided which allow the
35 * barrier & don't care about the ordering of loads then the 'wmb'
36 * ordering barrier can be used. Limiting the barrier's effects to stores
57 * we're satisfied that lightweight ordering barriers defined by MIPSr6 are
65 * ...except on Cavium Octeon CPUs, which have been using the 'wmb' ordering
153 * Some Cavium Octeon CPUs suffer from a bug that causes a single wmb ordering
|
| /linux/tools/testing/selftests/bpf/prog_tests/ |
| H A D | cgroup_mprog_opts.c | 53 /* ordering: [fd1] */ in test_prog_attach_detach()
65 /* ordering: [fd2, fd1] */ in test_prog_attach_detach()
78 /* ordering: [fd2, fd3, fd1] */ in test_prog_attach_detach()
90 /* ordering: [fd2, fd3, fd1, fd4] */ in test_prog_attach_detach()
185 /* ordering: [fd1] */ in test_link_attach_detach()
198 /* ordering: [fd2, fd1] */ in test_link_attach_detach()
211 /* ordering: [fd2, fd3, fd1] */ in test_link_attach_detach()
222 /* ordering: [fd2, fd3, fd1, fd4] */ in test_link_attach_detach()
304 /* ordering: [fd1] */ in test_preorder_prog_attach_detach()
316 /* ordering: [fd1, fd2] */ in test_preorder_prog_attach_detach()
[all …]
|
| /linux/scripts/atomic/kerneldoc/ |
| H A D | dec_unless_positive | 3 * ${class}${atomicname}() - atomic decrement unless positive with ${desc_order} ordering
6 * If (@v <= 0), atomically updates @v to (@v - 1) with ${desc_order} ordering.
7 * Otherwise, @v is not modified and relaxed ordering is provided.
|
| H A D | inc_not_zero | 3 * ${class}${atomicname}() - atomic increment unless zero with ${desc_order} ordering
6 * If (@v != 0), atomically updates @v to (@v + 1) with ${desc_order} ordering.
7 * Otherwise, @v is not modified and relaxed ordering is provided.
|
| H A D | inc_unless_negative | 3 * ${class}${atomicname}() - atomic increment unless negative with ${desc_order} ordering
6 * If (@v >= 0), atomically updates @v to (@v + 1) with ${desc_order} ordering.
7 * Otherwise, @v is not modified and relaxed ordering is provided.
|
| H A D | dec_if_positive | 3 * ${class}${atomicname}() - atomic decrement if positive with ${desc_order} ordering
6 * If (@v > 0), atomically updates @v to (@v - 1) with ${desc_order} ordering.
7 * Otherwise, @v is not modified and relaxed ordering is provided.
|
| H A D | cmpxchg | 3 * ${class}${atomicname}() - atomic compare and exchange with ${desc_order} ordering
8 * If (@v == @old), atomically updates @v to @new with ${desc_order} ordering.
9 * Otherwise, @v is not modified and relaxed ordering is provided.
|
| H A D | try_cmpxchg | 3 * ${class}${atomicname}() - atomic compare and exchange with ${desc_order} ordering
8 * If (@v == @old), atomically updates @v to @new with ${desc_order} ordering.
10 * and relaxed ordering is provided.
|
| H A D | add_unless | 7 * ${class}${atomicname}() - atomic add unless value with ${desc_order} ordering
12 * If (@v != @u), atomically updates @v to (@v + @a) with ${desc_order} ordering.
13 * Otherwise, @v is not modified and relaxed ordering is provided.
|
| /linux/lib/ |
| H A D | rcuref.c | 153 * Memory ordering
156 * Memory ordering rules are slightly relaxed wrt regular atomic_t functions
159 * The increments are fully relaxed; these will not provide ordering. The
161 * reference count on will provide the ordering. For locked data
165 * rcuref_get() provides a control dependency ordering future stores which
170 * will be issued before. It also provides a control dependency ordering
174 * object DEAD it also provides acquire ordering.
255 * deconstruction. Provide acquire ordering. in rcuref_put_slowpath()
|
| /linux/fs/jffs2/ |
| H A D | README.Locking | 37 Ordering constraints: See f->sem.
62 Ordering constraints:
67 No ordering rules have been made for doing so.
115 Ordering constraints:
147 Ordering constraints:
168 Ordering constraints:
|
| /linux/tools/perf/pmu-events/arch/x86/goldmont/ |
| H A D | memory.json | 3 "BriefDescription": "Machine clears due to memory ordering issue",
7 …e clears due to memory ordering issues. This occurs when a snoop request happens and the machine …
|
| /linux/tools/perf/pmu-events/arch/x86/goldmontplus/ |
| H A D | memory.json | 3 "BriefDescription": "Machine clears due to memory ordering issue",
7 …e clears due to memory ordering issues. This occurs when a snoop request happens and the machine …
|
| /linux/Documentation/ |
| H A D | memory-barriers.txt | 87 (*) Assumed minimum execution ordering model.
136 abstract CPU, memory operation ordering is very relaxed, and a CPU may actually
364 ordering over the memory operations on either side of the barrier.
386 A write barrier is a partial ordering on stores only; it is not required
411 An address-dependency barrier is a partial ordering on interdependent
425 showing the ordering constraints.
449 A read barrier is a partial ordering on loads only; it is not required to
466 A general memory barrier is a partial ordering over both loads and stores.
662 of dependency ordering is to -prevent- writes to the data structure, along
665 naturally occurring ordering prevents such records from being lost.
[all …]
|
| /linux/include/asm-generic/ |
| H A D | rwonce.h | 6 * particular ordering. One way to make the compiler aware of ordering is to
16 * mutilate accesses that either do not require ordering or that interact
18 * required ordering.
|
| /linux/tools/memory-model/litmus-tests/ |
| H A D | README | 39 Tests whether the ordering provided by a lock-protected S
67 Does a unlock+lock pair provides ordering guarantee between a
98 Does a unlock+lock pair provides ordering guarantee between a
148 Is the ordering provided by a spin_unlock() and a subsequent
149 spin_lock() sufficient to make ordering apparent to accesses
157 Is the ordering provided by a release-acquire chain sufficient
158 to make ordering apparent to accesses by a process that does
|