vdso: Rename struct arch_vdso_data to arch_vdso_time_dataThe struct arch_vdso_data is only about vdso time data. So rename it toarch_vdso_time_data to make it obvious.Non time-related data will b
vdso: Rename struct arch_vdso_data to arch_vdso_time_dataThe struct arch_vdso_data is only about vdso time data. So rename it toarch_vdso_time_data to make it obvious.Non time-related data will be migrated out of these structs soon.Signed-off-by: Nam Cao <namcao@linutronix.de>Signed-off-by: Thomas Gleixner <tglx@linutronix.de>Acked-by: Heiko Carstens <hca@linux.ibm.com> # s390Link: https://lore.kernel.org/all/20241010-vdso-generic-base-v1-28-b64f0842d512@linutronix.de
show more ...
riscv: vdso: Remove timekeeper includeSince the generic VDSO clock mode storage is used, this header file isunused and can be removed.This avoids including a non-VDSO header while building the V
riscv: vdso: Remove timekeeper includeSince the generic VDSO clock mode storage is used, this header file isunused and can be removed.This avoids including a non-VDSO header while building the VDSO,which can lead to compilation errors.Also drop the comment which is out of date and in the wrong place.Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>Signed-off-by: Thomas Gleixner <tglx@linutronix.de>Link: https://lore.kernel.org/all/20241010-vdso-generic-arch_update_vsyscall-v1-5-7fe5a3ea4382@linutronix.de
riscv: Provide a definition for 'pause'If we're going to provide the encoding for 'pause' in cpu_relax()anyway, then we can drop the toolchain checks and just always useit. The advantage of doing
riscv: Provide a definition for 'pause'If we're going to provide the encoding for 'pause' in cpu_relax()anyway, then we can drop the toolchain checks and just always useit. The advantage of doing this is that other code that needpause don't need to also define it (yes, another use is coming).Add the definition to insn-def.h since it's an instructiondefinition and also because insn-def.h doesn't include much, soit's safe to include from asm/vdso/processor.h without concern forcircular dependencies.Signed-off-by: Andrew Jones <ajones@ventanamicro.com>Link: https://lore.kernel.org/r/20240426100820.14762-9-ajones@ventanamicro.comSigned-off-by: Palmer Dabbelt <palmer@rivosinc.com>
riscv: Using TOOLCHAIN_HAS_ZIHINTPAUSE marco replace zihintpauseActually it is a part of Conor'scommit aae538cd03bc ("riscv: fix detection of toolchainZihintpause support").It is looks like a me
riscv: Using TOOLCHAIN_HAS_ZIHINTPAUSE marco replace zihintpauseActually it is a part of Conor'scommit aae538cd03bc ("riscv: fix detection of toolchainZihintpause support").It is looks like a merge issue. Samuel'scommit 0b1d60d6dd9e ("riscv: Fix build withCONFIG_CC_OPTIMIZE_FOR_SIZE=y") do not base on Conor's commit andrevert to __riscv_zihintpause. So this patch can fix it.Signed-off-by: Minda Chen <minda.chen@starfivetech.com>Fixes: 3c349eacc559 ("Merge patch "riscv: Fix build with CONFIG_CC_OPTIMIZE_FOR_SIZE=y"")Reviewed-by: Conor Dooley <conor.dooley@microchip.com>Link: https://lore.kernel.org/r/20230802064215.31111-1-minda.chen@starfivetech.comCc: stable@vger.kernel.orgSigned-off-by: Palmer Dabbelt <palmer@rivosinc.com>
RISC-V: Add hwprobe vDSO function and dataAdd a vDSO function __vdso_riscv_hwprobe, which can sit in front of theriscv_hwprobe syscall and answer common queries. We stash a copy ofstatic answers
RISC-V: Add hwprobe vDSO function and dataAdd a vDSO function __vdso_riscv_hwprobe, which can sit in front of theriscv_hwprobe syscall and answer common queries. We stash a copy ofstatic answers for the "all CPUs" case in the vDSO data page. This datais private to the vDSO, so we can decide later to change what's storedthere or under what conditions we defer to the syscall. Currently alldata can be discovered at boot, so the vDSO function answers all querieswhen the cpumask is set to the "all CPUs" hint.There's also a boolean in the data that lets the vDSO function know thatall CPUs are the same. In that case, the vDSO will also answer queriesfor arbitrary CPU masks in addition to the "all CPUs" hint.Signed-off-by: Evan Green <evan@rivosinc.com>Link: https://lore.kernel.org/r/20230407231103.2622178-7-evan@rivosinc.comSigned-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Merge patch "riscv: Fix build with CONFIG_CC_OPTIMIZE_FOR_SIZE=y"This is a single fix, but it conflicts with some recent features. I'mmerging it on top of the commit it fixes to ease backporting.
Merge patch "riscv: Fix build with CONFIG_CC_OPTIMIZE_FOR_SIZE=y"This is a single fix, but it conflicts with some recent features. I'mmerging it on top of the commit it fixes to ease backporting.* b4-shazam-merge: riscv: Fix build with CONFIG_CC_OPTIMIZE_FOR_SIZE=yLink: https://lore.kernel.org/r/20220922060958.44203-1-samuel@sholland.orgSigned-off-by: Palmer Dabbelt <palmer@rivosinc.com>
riscv: Fix build with CONFIG_CC_OPTIMIZE_FOR_SIZE=ycommit 8eb060e10185 ("arch/riscv: add Zihintpause support") brokebuilding with CONFIG_CC_OPTIMIZE_FOR_SIZE enabled (gcc 11.1.0): CC arch/
riscv: Fix build with CONFIG_CC_OPTIMIZE_FOR_SIZE=ycommit 8eb060e10185 ("arch/riscv: add Zihintpause support") brokebuilding with CONFIG_CC_OPTIMIZE_FOR_SIZE enabled (gcc 11.1.0): CC arch/riscv/kernel/vdso/vgettimeofday.oIn file included from <command-line>:./arch/riscv/include/asm/jump_label.h: In function 'cpu_relax':././include/linux/compiler_types.h:285:33: warning: 'asm' operand 0 probably does not match constraints 285 | #define asm_volatile_goto(x...) asm goto(x) | ^~~./arch/riscv/include/asm/jump_label.h:41:9: note: in expansion of macro 'asm_volatile_goto' 41 | asm_volatile_goto( | ^~~~~~~~~~~~~~~~~././include/linux/compiler_types.h:285:33: error: impossible constraint in 'asm' 285 | #define asm_volatile_goto(x...) asm goto(x) | ^~~./arch/riscv/include/asm/jump_label.h:41:9: note: in expansion of macro 'asm_volatile_goto' 41 | asm_volatile_goto( | ^~~~~~~~~~~~~~~~~make[1]: *** [scripts/Makefile.build:249: arch/riscv/kernel/vdso/vgettimeofday.o] Error 1make: *** [arch/riscv/Makefile:128: vdso_prepare] Error 2Having a static branch in cpu_relax() is problematic because thatfunction is widely inlined, including in some quite complex functionslike in the VDSO. A quick measurement shows this static branch isresponsible by itself for around 40% of the jump table.Drop the static branch, which ends up being the same number ofinstructions anyway. If Zihintpause is supported, we trade the nop fromthe static branch for a div. If Zihintpause is unsupported, we trade thejump from the static branch for (what gets interpreted as) a nop.Fixes: 8eb060e10185 ("arch/riscv: add Zihintpause support")Signed-off-by: Samuel Holland <samuel@sholland.org>Reviewed-by: Conor Dooley <conor.dooley@microchip.com>Cc: stable@vger.kernel.orgSigned-off-by: Palmer Dabbelt <palmer@rivosinc.com>
riscv: fix detection of toolchain Zihintpause supportIt is not sufficient to check if a toolchain supports a particularextension without checking if the linker supports that extensiontoo. For exa
riscv: fix detection of toolchain Zihintpause supportIt is not sufficient to check if a toolchain supports a particularextension without checking if the linker supports that extensiontoo. For example, Clang 15 supports Zihintpause but GNU bintutils2.35.2 does not, leading build errors like so:riscv64-linux-gnu-ld: -march=rv64i2p0_m2p0_a2p0_c2p0_zihintpause2p0: Invalid or unknown z ISA extension: 'zihintpause'Add a TOOLCHAIN_HAS_ZIHINTPAUSE which checks if each of the compiler,assembler and linker support the extension. Replace the ifdef in thevdso with one depending on this new symbol.Fixes: 8eb060e10185 ("arch/riscv: add Zihintpause support")Signed-off-by: Conor Dooley <conor.dooley@microchip.com>Reviewed-by: Heiko Stuebner <heiko@sntech.de>Reviewed-by: Nathan Chancellor <nathan@kernel.org>Link: https://lore.kernel.org/r/20221006173520.1785507-3-conor@kernel.orgSigned-off-by: Palmer Dabbelt <palmer@rivosinc.com>
arch/riscv: add Zihintpause supportImplement support for the ZiHintPause extension.The PAUSE instruction is a HINT that indicates the current hart’s rateof instruction retirement should be tempo
arch/riscv: add Zihintpause supportImplement support for the ZiHintPause extension.The PAUSE instruction is a HINT that indicates the current hart’s rateof instruction retirement should be temporarily reduced or paused.Reviewed-by: Heiko Stuebner <heiko@sntech.de>Tested-by: Heiko Stuebner <heiko@sntech.de>Signed-off-by: Dao Lu <daolu@rivosinc.com>[Palmer: Some minor merge conflicts.]Link: https://lore.kernel.org/all/20220620201530.3929352-1-daolu@rivosinc.com/Link: https://lore.kernel.org/all/20220811053356.17375-1-palmer@rivosinc.com/Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
riscv/vdso: Add support for time namespacesImplement generic vdso time namespace support which also enables timenamespaces for riscv. This is quite similar to what arm64 does.selftest/timens tes
riscv/vdso: Add support for time namespacesImplement generic vdso time namespace support which also enables timenamespaces for riscv. This is quite similar to what arm64 does.selftest/timens test result: 1..10 ok 1 Passed for CLOCK_BOOTTIME (syscall) ok 2 Passed for CLOCK_BOOTTIME (vdso) ok 3 # SKIP CLOCK_BOOTTIME_ALARM isn't supported ok 4 # SKIP CLOCK_BOOTTIME_ALARM isn't supported ok 5 Passed for CLOCK_MONOTONIC (syscall) ok 6 Passed for CLOCK_MONOTONIC (vdso) ok 7 Passed for CLOCK_MONOTONIC_COARSE (syscall) ok 8 Passed for CLOCK_MONOTONIC_COARSE (vdso) ok 9 Passed for CLOCK_MONOTONIC_RAW (syscall) ok 10 Passed for CLOCK_MONOTONIC_RAW (vdso) # Totals: pass:8 fail:0 xfail:0 xpass:0 skip:2 error:0Signed-off-by: Tong Tiangen <tongtiangen@huawei.com>Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
RISC-V: fix barrier() use in <vdso/processor.h>riscv's <vdso/processor.h> uses barrier() so it should include<asm/barrier.h>Fixes this build error: CC [M] drivers/net/ethernet/emulex/benet/be
RISC-V: fix barrier() use in <vdso/processor.h>riscv's <vdso/processor.h> uses barrier() so it should include<asm/barrier.h>Fixes this build error: CC [M] drivers/net/ethernet/emulex/benet/be_main.oIn file included from ./include/vdso/processor.h:10, from ./arch/riscv/include/asm/processor.h:11, from ./include/linux/prefetch.h:15, from drivers/net/ethernet/emulex/benet/be_main.c:14:./arch/riscv/include/asm/vdso/processor.h: In function 'cpu_relax':./arch/riscv/include/asm/vdso/processor.h:14:2: error: implicit declaration of function 'barrier' [-Werror=implicit-function-declaration] 14 | barrier();This happens with a total of 5 networking drivers -- they all use<linux/prefetch.h>.rv64 allmodconfig now builds cleanly after this patch.Fixes fallout from:815f0ddb346c ("include/linux/compiler*.h: make compiler-*.h mutually exclusive")Fixes: ad5d1122b82f ("riscv: use vDSO common flow to reduce the latency of the time-related functions")Reported-by: Andreas Schwab <schwab@linux-m68k.org>Signed-off-by: Randy Dunlap <rdunlap@infradead.org>Acked-by: Arvind Sankar <nivedita@alum.mit.edu>Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
vdso/treewide: Add vdso_data pointer argument to __arch_get_hw_counter()MIPS already uses and S390 will need the vdso data pointer in__arch_get_hw_counter().This works nicely as long as the arch
vdso/treewide: Add vdso_data pointer argument to __arch_get_hw_counter()MIPS already uses and S390 will need the vdso data pointer in__arch_get_hw_counter().This works nicely as long as the architecture does not support timenamespaces in the VDSO. With time namespaces enabled the regularaccessor to the vdso data pointer __arch_get_vdso_data() will return thenamespace specific VDSO data page for tasks which are part of anon-root time namespace. This would cause the architectures which needthe vdso data pointer in __arch_get_hw_counter() to access the wrongvdso data page.Add a vdso_data pointer argument to __arch_get_hw_counter() and hand it infrom the call sites in the core code. For architectures which do not needthe data pointer in their counter accessor function the compiler will justoptimize it out.Fix up all existing architecture implementations and make MIPS utilize thepointer instead of invoking the accessor function.No functional change and no change in the resulting object code (exceptMIPS).Signed-off-by: Thomas Gleixner <tglx@linutronix.de>Link: https://lkml.kernel.org/r/draft-87wo2ekuzn.fsf@nanos.tec.linutronix.de
asm/rwonce: Don't pull <asm/barrier.h> into 'asm-generic/rwonce.h'Now that 'smp_read_barrier_depends()' has gone the way of the NorwegianBlue, drop the inclusion of <asm/barrier.h> in 'asm-generic
asm/rwonce: Don't pull <asm/barrier.h> into 'asm-generic/rwonce.h'Now that 'smp_read_barrier_depends()' has gone the way of the NorwegianBlue, drop the inclusion of <asm/barrier.h> in 'asm-generic/rwonce.h'.This requires fixups to some architecture vdso headers which werepreviously relying on 'asm/barrier.h' coming in via 'linux/compiler.h'.Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>Signed-off-by: Will Deacon <will@kernel.org>
riscv: use vDSO common flow to reduce the latency of the time-related functionsEven if RISC-V has supported the vDSO feature, the latency of the functionsfor obtaining the system time is still exp
riscv: use vDSO common flow to reduce the latency of the time-related functionsEven if RISC-V has supported the vDSO feature, the latency of the functionsfor obtaining the system time is still expensive. It is because thesefunctions still trigger a corresponding system call in the process, whichslows down the response time. If we want to remove the system call toreduce the latency, the kernel should have the ability to output the systemclock information to userspace. This patch introduces the vDSO common flowto enable the kernel to achieve the above feature and uses "rdtime"instruction to obtain the current time in the user space. Under thiscondition, the latency cost by the ecall from U-mode to S-mode can beeliminated. After applying this patch, the latency of gettimeofday()measured on the HiFive unleashed board can be reduced by %61.Signed-off-by: Vincent Chen <vincent.chen@sifive.com>Reviewed-by: Atish Patra <atish.patra@wdc.com>Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>