| /linux/tools/memory-model/Documentation/ |
| H A D | glossary.txt | 1 This document contains brief definitions of LKMM-related terms. Like most
8 based on the value returned by an earlier load, an "address
9 dependency" extends from that load extending to the later access.
10 Address dependencies are quite common in RCU read-side critical
15 3 do_something(p->a);
18 In this case, because the address of "p->a" on line 3 is computed
21 "p->a". In rare cases, optimizing compilers can destroy address
27 Acquire: With respect to a lock, acquiring that lock, for example,
28 using spin_lock(). With respect to a non-lock shared variable,
29 a special operation that includes a load an
[all...] |
| H A D | ordering.txt | 1 This document gives an overview of the categories of memory-ordering
2 operations provided by the Linux-kernel memory model (LKMM).
8 This section lists LKMM's three top-level categories of memory-ordering
38 b. Read-modify-write (RMW) ordering augmentation barriers.
56 --------------------
58 The Linux-kernel primitives that provide full ordering include:
62 o Value-returning RMW atomic operations whose names do not end in
65 o RCU's grace-period primitives.
74 smp_mb(); // Order store to x before load from y.
77 All CPUs will agree that the store to "x" happened before the load
[all …]
|
| H A D | recipes.txt | 20 ------------------------------------
31 2. Compilers are permitted to use the "as-if" rule. That is, a
33 as long as the results of a single-threaded execution appear
41 your full-ordering warranty, as do undersized accesses that load
45 use READ_ONCE() and WRITE_ONCE() or stronger to prevent load/store
46 tearing, load/store fusing, and invented loads and stores.
51 holding the update-side lock, reads from that variable
62 -------
66 locklessly accessing lock-protected shared variables.
68 Locking is well-known and straightforward, at least if you don't think
[all …]
|
| H A D | herd-representation.txt | 3 # R, a Load event
6 # LKR, a Lock-Read event
7 # LKW, a Lock-Write event
9 # LF, a Lock-Fail event
10 # RL, a Read-Locked event
11 # RU, a Read-Unlocked event
12 # R*, a Load event included in RMW
14 # SRCU, a Sleepable-Read-Copy-Updat
[all...] |
| H A D | explanation.txt | 1 Explanation of the Linux-Kernel Memory Consistency Model
15 7. THE PROGRAM ORDER RELATION: po AND po-loc
18 10. THE READS-FROM RELATION: rf, rfi, and rfe
20 12. THE FROM-READS RELATION: fr, fri, and fre
22 14. PROPAGATION ORDER RELATION: cumul-fence
28 20. THE HAPPENS-BEFORE RELATION: hb
29 21. THE PROPAGATES-BEFORE RELATION: pb
30 22. RCU RELATIONS: rcu-link, rcu-gp, rcu-rscsi, rcu-order, rcu-fence, and rb
31 23. SRCU READ-SIDE CRITICAL SECTIONS
39 ------------
[all …]
|
| /linux/tools/memory-model/litmus-tests/ |
| H A D | README | 6 Test of read-read coherence, that is, whether or not two
10 Test of read-write coherence, that is, whether or not a read
15 Test of write-read coherence, that is, whether or not a write
20 Test of write-write coherence, that is, whether or not two
39 Tests whether the ordering provided by a lock-protected S
45 As below, but with store-release replaced with WRITE_ONCE()
46 and load-acquire replaced with READ_ONCE().
49 Can a release-acquire chain order a prior store against
50 a later load?
54 load-buffering litmus test, where each process reads from one
[all …]
|
| H A D | LB+poacquireonce+pooncerelease.litmus | 6 * Does a release-acquire pair suffice for the load-buffering litmus
|
| H A D | S+poonceonces.litmus | 6 * Starting with a two-process release-acquire chain ordering P0()'s
7 * first store against P1()'s final load, if the smp_store_release()
|
| H A D | ISA2+poonceonces.litmus | 6 * Given a release-acquire chain ordering the first process's store
7 * against the last process's load, is ordering preserved if all of the
|
| /linux/Documentation/ |
| H A D | memory-barriers.txt | 19 documentation at tools/memory-model/. Nevertheless, even this memory
37 Note also that it is possible that a barrier may be a no-op for an
48 - Device operations.
49 - Guarantees.
53 - Varieties of memory barrier.
54 - What may not be assumed about memory barriers?
55 - Address-dependency barriers (historical).
56 - Control dependencies.
57 - SMP barrier pairing.
58 - Examples of memory barrier sequences.
[all …]
|
| /linux/Documentation/translations/ko_KR/ |
| H A D | memory-barriers.txt | 2 This is a version of Documentation/memory-barriers.txt translated into Korean.
15 Documentation/memory-barriers.txt
39 일부 이상한 점들은 공식적인 메모리 일관성 모델과 tools/memory-model/ 에 있는
60 해당 배리어의 명시적 사용이 불필요해서 no-op 이 될수도 있음을 알아두시기
76 - 디바이스 오퍼레이션.
77 - 보장사항.
81 - 메모리 배리어의 종류.
82 - 메모리 배리어에 대해 가정해선 안될 것.
83 - 주소 데이터 의존성 배리어 (역사적).
84 - 컨트롤 의존성.
[all …]
|
| /linux/rust/kernel/sync/atomic/ |
| H A D | ordering.rs | 1 // SPDX-License-Identifier: GPL-2.0
7 //! - [`Acquire`] provides ordering between the load part of the annotated operation and all the
10 //! - [`Release`] provides ordering between all the preceding memory accesses and the store part of
11 //! the annotated operation, and if there is a load part, the load part has the [`Relaxed`]
13 //! - [`Full`] means "fully-ordered", that is:
14 //! - It provides ordering between all the preceding memory accesses and the annotated operation.
15 //! - It provides ordering between the annotated operation and all the following memory accesses.
16 //! - It provides ordering between all the preceding memory accesses and all the following memory
18 //! - All the orderings are the same strength as a full memory barrier (i.e. `smp_mb()`).
19 //! - [`Relaxed`] provides no ordering except the dependency orderings. Dependency orderings are
[all …]
|
| /linux/Documentation/translations/sp_SP/ |
| H A D | memory-barriers.txt | 2 This is a version of Documentation/memory-barriers.txt translated into
35 consistencia de memoria formal y documentación en tools/memory-model/. Sin
53 Tenga en cuenta también que es posible que una barrera no valga (sea no-op)
63 - Operaciones del dispositivo.
64 - Garantías.
68 - Variedades de barrera de memoria.
69 - ¿Qué no se puede asumir sobre las barreras de memoria?
70 - Barreras de dirección-dependencia (históricas).
71 - Dependencias de control.
72 - Emparejamiento de barreras smp.
[all …]
|
| /linux/include/linux/ |
| H A D | atomic.h | 1 /* SPDX-License-Identifier: GPL-2.0 */
15 * - Fully ordered: The default implementation, no suffix required.
16 * - Acquire: Provides ACQUIRE semantics, _acquire suffix.
17 * - Release: Provides RELEASE semantics, _release suffix.
18 * - Relaxed: No ordering guarantees, _relaxed suffix.
20 * For compound atomics performing both a load and a store, ACQUIRE
21 * semantics apply only to the load and RELEASE semantics only to the
23 * does -not- imply any memory ordering constraints.
25 * See Documentation/memory-barriers.txt for ACQUIRE/RELEASE definitions.
28 #define atomic_cond_read_acquire(v, c) smp_cond_load_acquire(&(v)->counter, (c))
[all …]
|
| /linux/rust/kernel/sync/ |
| H A D | atomic.rs | 1 // SPDX-License-Identifier: GPL-2.0
13 //! - A normal write from C side is treated as an atomic write if
15 //! - Mixed-size atomic accesses don't cause data races.
17 //! [`LKMM`]: srctree/tools/memory-model/
24 pub use ordering::{Acquire, Full, Relaxed, Release};
40 /// [`Atomic::as_ptr()`], this provides a way to interact with [C-side atomic operations]
49 /// [LKMM]: srctree/tools/memory-model/
50 /// [C-side atomic operations]: srctree/Documentation/atomic_t.txt
59 /// # Round-trip transmutability
61 /// `T` is round-trip transmutable to `U` if and only if both of these properties hold:
[all …]
|
| /linux/tools/perf/pmu-events/arch/arm64/arm/neoverse-n1/ |
| H A D | spec_operation.json | 28 …"PublicDescription": "Counts Load-Exclusive operations that have been speculatively executed. Eg: …
32 …"PublicDescription": "Counts store-exclusive operations that have been speculatively executed and …
36 …"PublicDescription": "Counts store-exclusive operations that have been speculatively executed and …
40 … "PublicDescription": "Counts store-exclusive operations that have been speculatively executed."
44 …ription": "Counts speculatively executed load operations including Single Instruction Multiple Dat…
56 …unts speculatively executed Advanced SIMD operations excluding load, store and move micro-operatio…
80 …udes operations that force a software change of the PC, other than exception-generating operations…
88 …"PublicDescription": "Counts DSB operations that are speculatively issued to Load/Store unit in th…
92 …are speculatively issued to the Load/Store unit in the CPU. This event does not count implied barr…
96 …"PublicDescription": "Counts any load acquire operations that are speculatively executed. Eg: LDAR…
|
| /linux/tools/perf/pmu-events/arch/arm64/arm/neoverse-v1/ |
| H A D | spec_operation.json | 16 …"PublicDescription": "Counts micro-operations speculatively executed. This is the count of the num…
32 …"PublicDescription": "Counts Load-Exclusive operations that have been speculatively executed. Eg: …
36 …"PublicDescription": "Counts store-exclusive operations that have been speculatively executed and …
40 …"PublicDescription": "Counts store-exclusive operations that have been speculatively executed and …
44 … "PublicDescription": "Counts store-exclusive operations that have been speculatively executed."
48 …ription": "Counts speculatively executed load operations including Single Instruction Multiple Dat…
60 …unts speculatively executed Advanced SIMD operations excluding load, store and move micro-operatio…
84 …udes operations that force a software change of the PC, other than exception-generating operations…
92 …"PublicDescription": "Counts DSB operations that are speculatively issued to Load/Store unit in th…
96 …are speculatively issued to the Load/Store unit in the CPU. This event does not count implied barr…
[all …]
|
| /linux/include/asm-generic/ |
| H A D | barrier.h | 1 /* SPDX-License-Identifier: GPL-2.0-or-later */
17 #include <linux/kcsan-checks.h>
223 * smp_acquire__after_ctrl_dep() - Provide ACQUIRE ordering after a control dependency
225 * A control dependency provides a LOAD->STORE order, the additional RMB
226 * provides LOAD->LOAD order, together they provide LOAD->{LOAD,STORE} order,
227 * aka. (load)-ACQUIRE.
229 * Architectures that do not do load speculation can have this be barrier().
236 * smp_cond_load_relaxed() - (Spin) wait for cond with no ordering guarantees
242 * Due to C lacking lambda expressions we load the value of *ptr into a
243 * pre-named variable @VAL to be used in @cond.
[all …]
|
| /linux/arch/arm64/net/ |
| H A D | bpf_jit.h | 1 /* SPDX-License-Identifier: GPL-2.0-only */
5 * Copyright (C) 2014-2016 Zi Shen Lim <zlim.lnx@gmail.com>
12 /* 5-bit Register Operand */
55 /* Load/store register (register offset) */
61 #define A64_LDRB(Wt, Xn, Xm) A64_LS_REG(Wt, Xn, Xm, 8, LOAD)
64 #define A64_LDRH(Wt, Xn, Xm) A64_LS_REG(Wt, Xn, Xm, 16, LOAD)
67 #define A64_LDR32(Wt, Xn, Xm) A64_LS_REG(Wt, Xn, Xm, 32, LOAD)
70 #define A64_LDR64(Xt, Xn, Xm) A64_LS_REG(Xt, Xn, Xm, 64, LOAD)
72 /* Load/store register (immediate offset) */
78 #define A64_LDRBI(Wt, Xn, imm) A64_LS_IMM(Wt, Xn, imm, 8, LOAD)
[all …]
|
| /linux/tools/perf/pmu-events/arch/powerpc/power10/ |
| H A D | marked.json | 60 …n which the marked instruction is the oldest in the pipeline (next-to-finish or next-to-complete)."
105 …ditional store instruction (STCX) finished. LARX and STCX are instructions used to acquire a lock."
125 …a cache was reloaded from local, remote, or distant memory due to a demand miss for a marked load."
130 "BriefDescription": "Marked demand data load miss counted at finish time."
135 …ed from a source other than the local core's L1, L2, or L3 due to a demand miss for a marked load."
155 "BriefDescription": "Marked load instruction completed."
170 …onditional store instruction (STCX) failed. LARX and STCX are instructions used to acquire a lock."
185 …"BriefDescription": "The marked instruction was dependent on a load. It is eligible for issue kill…
190 …": "Marked store completed and sent to nest. Note that this count excludes cache-inhibited stores."
210 …"BriefDescription": "Marked load and reserve instruction (LARX) finished. LARX and STCX are instru…
[all …]
|
| /linux/arch/sparc/include/asm/ |
| H A D | oplib_64.h | 1 /* SPDX-License-Identifier: GPL-2.0 */
85 /* Enter the prom, with no chance of continuation for the stand-alone
90 /* Halt and power-off the machine. */
93 /* Acquire the IDPROM of the root node in the prom device tree. This
153 /* Load explicit I/D TLB entries into the calling processor. */
166 #define PROM_MAP_READ 0x0002 /* Readable - sw */
167 #define PROM_MAP_EXEC 0x0004 /* Executable - sw */
168 #define PROM_MAP_LOCKED 0x0010 /* Locked, use i/dtlb load calls for this instead */
170 #define PROM_MAP_SE 0x0040 /* Side-Effects */
172 #define PROM_MAP_IE 0x0100 /* Invert-Endianness */
[all …]
|
| /linux/arch/arc/include/asm/ |
| H A D | spinlock.h | 1 /* SPDX-License-Identifier: GPL-2.0-only */
3 * Copyright (C) 2004, 2007-2010, 2011-2012 Synopsys, Inc. (www.synopsys.com)
13 #define arch_spin_is_locked(x) ((x)->slock != __ARCH_SPIN_LOCK_UNLOCKED__)
24 " scond %[LOCKED], [%[slock]] \n" /* acquire */ in arch_spin_lock()
28 : [slock] "r" (&(lock->slock)), in arch_spin_lock()
33 * ACQUIRE barrier to ensure load/store after taking the lock in arch_spin_lock()
34 * don't "bleed-up" out of the critical section (leak-in is allowed) in arch_spin_lock()
37 * ARCv2 only has load-load, store-store and all-all barrier in arch_spin_lock()
38 * thus need the full all-all barrier in arch_spin_lock()
43 /* 1 - lock taken successfully */
[all …]
|
| /linux/rust/pin-init/examples/ |
| H A D | mutex.rs | 1 // SPDX-License-Identifier: Apache-2.0 OR MIT
34 pub fn acquire(&self) -> SpinLockGuard<'_> { in acquire() method
37 .compare_exchange(false, true, Ordering::Acquire, Ordering::Relaxed) in acquire()
41 while self.inner.load(Ordering::Relaxed) { in acquire()
50 pub const fn new() -> Self { in new()
78 pub fn new(val: impl PinInit<T>) -> impl PinInit<Self> { in new()
80 wait_list <- ListHead::new(), in new()
83 data <- unsafe { in new()
92 pub fn lock(&self) -> Pin<CMutexGuard<'_, T>> { in lock()
93 let mut sguard = self.spin_lock.acquire(); in lock()
[all …]
|
| /linux/kernel/bpf/ |
| H A D | rqspinlock.c | 1 // SPDX-License-Identifier: GPL-2.0-or-later
5 * (C) Copyright 2013-2015 Hewlett-Packard Development Company, L.P.
6 * (C) Copyright 2013-2014,2018 Red Hat, Inc.
8 * (C) Copyright 2015 Hewlett-Packard Enterprise Development LP
9 * (C) Copyright 2024-202
[all...] |
| /linux/drivers/acpi/acpica/ |
| H A D | tbinstal.c | 1 // SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0
4 * Module Name: tbinstal - ACPI table installation and removal
6 * Copyright (C) 2000 - 2025, Intel Corp.
21 * PARAMETERS: new_table_desc - New table descriptor to install
22 * override - Whether override should be performed
23 * table_index - Where the table index is returned
57 new_table_desc->address, in acpi_tb_install_table_with_override()
58 new_table_desc->flags, in acpi_tb_install_table_with_override()
59 new_table_desc->pointer); in acpi_tb_install_table_with_override()
61 acpi_tb_print_table_header(new_table_desc->address, in acpi_tb_install_table_with_override()
[all …]
|