| 34d5b600 | 08-Oct-2024 |
Yu Liao <liaoyu15@huawei.com> |
selftests: vDSO: Explicitly include sched.h
The previous commit introduced the use of CLONE_NEWTIME without including
<sched.h> which contains its definition.
Add an explicit include of <sched.h> t
selftests: vDSO: Explicitly include sched.h
The previous commit introduced the use of CLONE_NEWTIME without including
<sched.h> which contains its definition.
Add an explicit include of <sched.h> to ensure that CLONE_NEWTIME
is correctly defined before it is used.
Fixes: 2aec90036dcd ("selftests: vDSO: ensure vgetrandom works in a time namespace")
Signed-off-by: Yu Liao <liaoyu15@huawei.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| 3953a1d1 | 07-Oct-2024 |
Jason A. Donenfeld <Jason@zx2c4.com> |
selftests: vDSO: improve getrandom and chacha error messages
Improve the error and skip condition messages to let the developer know
precisely where a test has failed. Also make better use of the ks
selftests: vDSO: improve getrandom and chacha error messages
Improve the error and skip condition messages to let the developer know
precisely where a test has failed. Also make better use of the ksft api
for this.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| fe6305cb | 07-Oct-2024 |
Jason A. Donenfeld <Jason@zx2c4.com> |
selftests: vDSO: unconditionally build getrandom test
Rather than building on supported archs, build on all archs, and then
use the presence of the symbol in the vDSO to either skip the test or
move
selftests: vDSO: unconditionally build getrandom test
Rather than building on supported archs, build on all archs, and then
use the presence of the symbol in the vDSO to either skip the test or
move forward with it.
Note that this means that this test no longer checks whether the symbol
was correctly added to the kernel. But hopefully this will be clear
enough to developers and we'll cross our fingers that symbols aren't
removed by accident and not caught after this change.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| 3b5992ea | 07-Oct-2024 |
Jason A. Donenfeld <Jason@zx2c4.com> |
selftests: vDSO: unconditionally build chacha test
Rather than using symlinks to find the vgetrandom-chacha.S file for each
arch, store this in a file that uses the compiler to determine
architectur
selftests: vDSO: unconditionally build chacha test
Rather than using symlinks to find the vgetrandom-chacha.S file for each
arch, store this in a file that uses the compiler to determine
architecture, and then make use of weak symbols to skip the test on
architectures that don't provide the code.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| 210860e7 | 15-Sep-2024 |
Jason A. Donenfeld <Jason@zx2c4.com> |
selftests: vDSO: check cpu caps before running chacha test
Some archs -- arm64 and s390x -- implemented chacha using instructions
that are available most places, but aren't always available. The ker
selftests: vDSO: check cpu caps before running chacha test
Some archs -- arm64 and s390x -- implemented chacha using instructions
that are available most places, but aren't always available. The kernel
handles this just fine, but the selftest does not. Check the hwcaps
before running, and skip the test if the cpu doesn't support it. As
well, on s390x, always emit the fallback instructions of an alternative
block, to ensure maximum compatibility.
Co-developed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
show more ...
|
| b920aa77 | 13-Sep-2024 |
Heiko Carstens <hca@linux.ibm.com> |
s390/vdso: Wire up getrandom() vdso implementation
Provide the s390 specific vdso getrandom() architecture backend.
_vdso_rng_data required data is placed within the _vdso_data vvar page,
by using
s390/vdso: Wire up getrandom() vdso implementation
Provide the s390 specific vdso getrandom() architecture backend.
_vdso_rng_data required data is placed within the _vdso_data vvar page,
by using a hardcoded offset larger than vdso_data.
As required the chacha20 implementation does not write to the stack.
The implementation follows more or less the arm64 implementations and
makes use of vector instructions. It has a fallback to the getrandom()
system call for machines where the vector facility is not installed.
The check if the vector facility is installed, as well as an
optimization for machines with the vector-enhancements facility 2, is
implemented with alternatives, avoiding runtime checks.
Note that __kernel_getrandom() is implemented without the vdso user
wrapper which would setup a stack frame for odd cases (aka very old
glibc variants) where the caller has not done that. All callers of
__kernel_getrandom() are required to setup a stack frame, like the C ABI
requires it.
The vdso testcases vdso_test_getrandom and vdso_test_chacha pass.
Benchmark on a z16:
$ ./vdso_test_getrandom bench-single
vdso: 25000000 times in 0.493703559 seconds
syscall: 25000000 times in 6.584025337 seconds
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
show more ...
|
| a6e23fb8 | 11-Sep-2024 |
Heiko Carstens <hca@linux.ibm.com> |
selftests: vDSO: fix vdso_config for s390
Running vdso_test_correctness on s390x (aka s390 64 bit) emits a warning:
Warning: failed to find clock_gettime64 in vDSO
This is caused by the "#elif def
selftests: vDSO: fix vdso_config for s390
Running vdso_test_correctness on s390x (aka s390 64 bit) emits a warning:
Warning: failed to find clock_gettime64 in vDSO
This is caused by the "#elif defined (__s390__)" check in vdso_config.h
which the defines VDSO_32BIT.
If __s390x__ is defined also __s390__ is defined. Therefore the correct
check must make sure that only __s390__ is defined.
Therefore add the missing !defined(__s390x__). Also use common
__s390x__ define instead of __s390X__.
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Fixes: 693f5ca08ca0 ("kselftest: Extend vDSO selftest")
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
show more ...
|
| 53cee505 | 02-Sep-2024 |
Christophe Leroy <christophe.leroy@csgroup.eu> |
powerpc/vdso: Wire up getrandom() vDSO implementation on VDSO32
To be consistent with other VDSO functions, the function is called
__kernel_getrandom()
__arch_chacha20_blocks_nostack() fonction is
powerpc/vdso: Wire up getrandom() vDSO implementation on VDSO32
To be consistent with other VDSO functions, the function is called
__kernel_getrandom()
__arch_chacha20_blocks_nostack() fonction is implemented basically
with 32 bits operations. It performs 4 QUARTERROUND operations in
parallele. There are enough registers to avoid using the stack:
On input:
r3: output bytes
r4: 32-byte key input
r5: 8-byte counter input/output
r6: number of 64-byte blocks to write to output
During operation:
stack: pointer to counter (r5) and non-volatile registers (r14-131)
r0: counter of blocks (initialised with r6)
r4: Value '4' after key has been read, used for indexing
r5-r12: key
r14-r15: block counter
r16-r31: chacha state
At the end:
r0, r6-r12: Zeroised
r5, r14-r31: Restored
Performance on powerpc 885 (using kernel selftest):
~# ./vdso_test_getrandom bench-single
vdso: 25000000 times in 62.938002291 seconds
libc: 25000000 times in 535.581916866 seconds
syscall: 25000000 times in 531.525042806 seconds
Performance on powerpc 8321 (using kernel selftest):
~# ./vdso_test_getrandom bench-single
vdso: 25000000 times in 16.899318858 seconds
libc: 25000000 times in 131.050596522 seconds
syscall: 25000000 times in 129.794790389 seconds
This first patch adds support for VDSO32. As selftests cannot easily
be generated only for VDSO32, and because the following patch brings
support for VDSO64 anyway, this patch opts out all code in
__arch_chacha20_blocks_nostack() so that vdso_test_chacha will not
fail to compile and will not crash on PPC64/PPC64LE, allthough the
selftest itself will fail.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Acked-by: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
show more ...
|
| 8bc7c5e5 | 03-Sep-2024 |
Jason A. Donenfeld <Jason@zx2c4.com> |
selftests: vDSO: don't include generated headers for chacha test
It's not correct to use $(top_srcdir) for generated header files, for
builds that are done out of tree via O=, and $(objtree) isn't v
selftests: vDSO: don't include generated headers for chacha test
It's not correct to use $(top_srcdir) for generated header files, for
builds that are done out of tree via O=, and $(objtree) isn't valid in
the selftests context. Instead, just obviate the need for these
generated header files by defining empty stubs in tools/include, which
is the same thing that's done for rwlock.h.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
show more ...
|
| 712676ea | 03-Sep-2024 |
Adhemerval Zanella <adhemerval.zanella@linaro.org> |
arm64: vDSO: Wire up getrandom() vDSO implementation
Hook up the generic vDSO implementation to the aarch64 vDSO data page.
The _vdso_rng_data required data is placed within the _vdso_data vvar
page
arm64: vDSO: Wire up getrandom() vDSO implementation
Hook up the generic vDSO implementation to the aarch64 vDSO data page.
The _vdso_rng_data required data is placed within the _vdso_data vvar
page, by using a offset larger than the vdso_data.
The vDSO function requires a ChaCha20 implementation that does not write
to the stack, and that can do an entire ChaCha20 permutation. The one
provided uses NEON on the permute operation, with a fallback to the
syscall for chips that do not support AdvSIMD.
This also passes the vdso_test_chacha test along with
vdso_test_getrandom. The vdso_test_getrandom bench-single result on
Neoverse-N1 shows:
vdso: 25000000 times in 0.783884250 seconds
libc: 25000000 times in 8.780275399 seconds
syscall: 25000000 times in 8.786581518 seconds
A small fixup to arch/arm64/include/asm/mman.h was required to avoid
pulling kernel code into the vDSO, similar to what's already done in
arch/arm64/include/asm/rwonce.h.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
show more ...
|
| bb10ffe0 | 01-Sep-2024 |
Christophe Leroy <christophe.leroy@csgroup.eu> |
selftests: vDSO: also test counter in vdso_test_chacha
The chacha vDSO selftest doesn't check the way the counter is handled by
__arch_chacha20_blocks_nostack(). It indirectly checks that the counte
selftests: vDSO: also test counter in vdso_test_chacha
The chacha vDSO selftest doesn't check the way the counter is handled by
__arch_chacha20_blocks_nostack(). It indirectly checks that the counter
is writen on exit and read back on new entry, but it doesn't check that
the format is correct. When implementing this function on powerpc, I
missed a case where the counter was writen and read in wrong byte order.
Also, the counter uses two words, but the tests with a zero counter and
uses a small amount of blocks, so at the end the upper part of the
counter is always 0, so it is not checked.
Add a verification of counter's content in addition to the verification
of the output.
Also add two tests where the counter crosses the u32 upper limit. The
first test verifies that the function properly writes back the upper
word, the second test verifies that the function properly reads back the
upper word.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
show more ...
|
| ecb8bd70 | 01-Sep-2024 |
Christophe Leroy <christophe.leroy@csgroup.eu> |
selftests: vDSO: build tests with O2 optimization
Without -O2, the generated code for testing chacha function is awful.
GCC even implements rol32() as a function of 20 instructions instead of
just u
selftests: vDSO: build tests with O2 optimization
Without -O2, the generated code for testing chacha function is awful.
GCC even implements rol32() as a function of 20 instructions instead of
just using the rotlwi instruction.
~# time ./vdso_test_chacha
TAP version 13
1..1
ok 1 chacha: PASS
real 0m 37.16s
user 0m 36.89s
sys 0m 0.26s
Several other selftests directory add -O2, and the kernel is also
always built with optimisation active. Do the same for vDSO selftests.
With this patch the time is reduced by approximately 15%.
~# time ./vdso_test_chacha
TAP version 13
1..1
ok 1 chacha: PASS
real 0m 32.09s
user 0m 31.86s
sys 0m 0.22s
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
show more ...
|
| 67a121ac | 01-Sep-2024 |
Jason A. Donenfeld <Jason@zx2c4.com> |
selftests: vDSO: fix cross build for getrandom and chacha tests
Unlike the check for the standalone x86 test, the check for building the
vDSO getrandom and chacaha tests looks at the architecture fo
selftests: vDSO: fix cross build for getrandom and chacha tests
Unlike the check for the standalone x86 test, the check for building the
vDSO getrandom and chacaha tests looks at the architecture for the host
rather than the architecture for the target when deciding if they should
be built. Since the chacha test includes some assembler code this means
that cross building with x86 as either the target or host is broken.
There's also some additional complications, where ARCH can legitimately
be either x86_64 or x86, but the source code we need to compile lives in
a directory path containing arch/x86. The standard SRCARCH variable
handles that. And actually, all these variables and proper substitutions
are already described in tools/scripts/Makefile.arch, so just include
that to handle it.
Similarly, ARCH=x86 can actually describe ARCH=x86_64,
just with CONFIG_64BIT, so we can't rely on ARCH for selecting
non-32-bit tests. For that, check against $(ARCH)$(CONFIG_X86_32). This
won't help for people manually running this inside the vDSO selftest
directory (which isn't really supported anyway and has problems on
various archs), but it should work for builds of the kselftests, where
the CONFIG_* variables are defined. On x86_64 machines,
$(ARCH)$(CONFIG_X86_32) will evaluate to x86. On arm64 machines, it will
evaluate to arm64. On 32-bit x86 machines, it will evaluate to x86y,
which won't match the filter list.
Reported-by: Mark Brown <broonie@kernel.org>
Reported-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
show more ...
|
| 7fe5b3e4 | 28-Aug-2024 |
Jason A. Donenfeld <Jason@zx2c4.com> |
selftests: vDSO: open code basic chacha instead of linking to libsodium
Linking to libsodium makes building this test annoying in cross
compilation environments and is just way too much. Since this
selftests: vDSO: open code basic chacha instead of linking to libsodium
Linking to libsodium makes building this test annoying in cross
compilation environments and is just way too much. Since this is just a
basic correctness test, simply open code a simple, unoptimized, dumb
chacha, rather than linking to libsodium.
This also fixes a correctness issue on big endian systems. The kernel's
random.c doesn't bother doing a le32_to_cpu operation on the random
bytes that are passed as the key, and consequently neither does
vgetrandom-chacha.S. However, libsodium's chacha _does_ do this, since
it takes the key as an array of bytes. This meant that the test was
broken on big endian systems, which this commit rectifies.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
show more ...
|
| 6fd13b28 | 27-Aug-2024 |
Jason A. Donenfeld <Jason@zx2c4.com> |
random: vDSO: move prototype of arch chacha function to vdso/getrandom.h
Having the prototype for __arch_chacha20_blocks_nostack in
arch/x86/include/asm/vdso/getrandom.h meant that the prototype and
random: vDSO: move prototype of arch chacha function to vdso/getrandom.h
Having the prototype for __arch_chacha20_blocks_nostack in
arch/x86/include/asm/vdso/getrandom.h meant that the prototype and large
doc comment were cloned by every architecture, which has been causing
unnecessary churn. Instead move it into include/vdso/getrandom.h, where
it can be shared by all archs implementing it.
As a side bonus, this then lets us use that prototype in the
vdso_test_chacha self test, to ensure that it matches the source, and
indeed doing so turned up some inconsistencies, which are rectified
here.
Suggested-by: Christophe Leroy <christophe.leroy@csgroup.eu>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
show more ...
|
| 2aec9003 | 05-Sep-2024 |
Jason A. Donenfeld <Jason@zx2c4.com> |
selftests: vDSO: ensure vgetrandom works in a time namespace
After verifying that vDSO getrandom does work, which ensures that the
RNG is initialized, test to see if it also works inside of a time
n
selftests: vDSO: ensure vgetrandom works in a time namespace
After verifying that vDSO getrandom does work, which ensures that the
RNG is initialized, test to see if it also works inside of a time
namespace. This is important to test, because the vvar pages get
swizzled around there. If the arch code isn't careful, the RNG will
appear uninitialized inside of a time namespace.
Because broken code makes the RNG appear uninitialized, test that
everything works by issuing a call to vgetrandom from a fork in a time
namespace, and use ptrace to ensure that the actual syscall getrandom
doesn't get called. If it doesn't get called, then the test succeeds.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
show more ...
|