| c5c30a37 | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/boot: Move startup code out of __head section
Move startup code out of the __head section, now that this no longer has
a special significance. Move everything into .text or .init.text as
appropr
x86/boot: Move startup code out of __head section
Move startup code out of the __head section, now that this no longer has
a special significance. Move everything into .text or .init.text as
appropriate, so that startup code is not kept around unnecessarily.
[ bp: Fold in hunk to fix 32-bit CPU hotplug:
Reported-by: kernel test robot <oliver.sang@intel.com>
Closes: https://lore.kernel.org/oe-lkp/202509022207.56fd97f4-lkp@intel.com ]
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250828102202.1849035-45-ardb+git@google.com
show more ...
|
| 7b38dec3 | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/boot: Create a confined code area for startup code
In order to be able to have tight control over which code may execute
from the early 1:1 mapping of memory, but still link vmlinux as a single
x86/boot: Create a confined code area for startup code
In order to be able to have tight control over which code may execute
from the early 1:1 mapping of memory, but still link vmlinux as a single
executable, prefix all symbol references in startup code with __pi_, and
invoke it from outside using the __pi_ prefix.
Use objtool to check that no absolute symbol references are present in
the startup code, as these cannot be used from code running from the 1:1
mapping.
Note that this also requires disabling the latent-entropy GCC plugin, as
the global symbol references that it injects would require explicit
exports, and given that the startup code rarely executes more than once,
it is not a useful source of entropy anyway.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250828102202.1849035-43-ardb+git@google.com
show more ...
|
| 296650c8 | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/boot: Check startup code for absence of absolute relocations
Invoke objtool on each startup code object individually to check for the
absence of absolute relocations. This is needed because this
x86/boot: Check startup code for absence of absolute relocations
Invoke objtool on each startup code object individually to check for the
absence of absolute relocations. This is needed because this code will
be invoked from the 1:1 mapping of memory before those absolute virtual
addresses (which are derived from the kernel virtual base address
provided to the linker and possibly shifted at boot) are mapped.
Only objects built under arch/x86/boot/startup/ have this restriction,
and once they have been incorporated into vmlinux.o, this distinction is
difficult to make. So force the invocation of objtool for each object
file individually, even if objtool is deferred to vmlinux.o for the rest
of the build. In the latter case, only pass --noabs and nothing else;
otherwise, append it to the existing objtool command line.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250828102202.1849035-40-ardb+git@google.com
show more ...
|
| 05ce314b | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/sev: Export startup routines for later use
Create aliases that expose routines that are part of the startup code to
other code in the core kernel, so that they can be called later as well.
Sign
x86/sev: Export startup routines for later use
Create aliases that expose routines that are part of the startup code to
other code in the core kernel, so that they can be called later as well.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250828102202.1849035-38-ardb+git@google.com
show more ...
|
| d4077e6a | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/sev: Move __sev_[get|put]_ghcb() into separate noinstr object
Rename sev-nmi.c to noinstr.c, and move the get/put GHCB routines into it too,
which are also annotated as 'noinstr' and suffer from
x86/sev: Move __sev_[get|put]_ghcb() into separate noinstr object
Rename sev-nmi.c to noinstr.c, and move the get/put GHCB routines into it too,
which are also annotated as 'noinstr' and suffer from the same problem as the
NMI code, i.e., that GCC may ignore the __no_sanitize_address__ function
attribute implied by 'noinstr' and insert KASAN instrumentation anyway.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250828102202.1849035-37-ardb+git@google.com
show more ...
|
| 9723dd0c | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/sev: Provide PIC aliases for SEV related data objects
Provide PIC aliases for data objects that are shared between the SEV startup
code and the SEV code that executes later. This is needed so th
x86/sev: Provide PIC aliases for SEV related data objects
Provide PIC aliases for data objects that are shared between the SEV startup
code and the SEV code that executes later. This is needed so that the confined
startup code is permitted to access them.
This requires some of these variables to be moved into a source file that is
not part of the startup code, as the PIC alias is already implied, and
exporting variables in the opposite direction is not supported.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250828102202.1849035-36-ardb+git@google.com
show more ...
|
| c54604fb | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/sev: Use boot SVSM CA for all startup and init code
To avoid having to reason about whether or not to use the per-CPU SVSM calling
area when running startup and init code on the boot CPU, reuse
x86/sev: Use boot SVSM CA for all startup and init code
To avoid having to reason about whether or not to use the per-CPU SVSM calling
area when running startup and init code on the boot CPU, reuse the boot SVSM
calling area as the per-CPU area for the BSP.
Thus, remove the need to make the per-CPU variables and associated state in
sev_cfg accessible to the startup code once confined.
[ bp: Massage commit message. ]
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250828102202.1849035-33-ardb+git@google.com
show more ...
|
| 00d25566 | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/sev: Pass SVSM calling area down to early page state change API
The early page state change API is mostly only used very early, when only the
boot time SVSM calling area is in use. However, this
x86/sev: Pass SVSM calling area down to early page state change API
The early page state change API is mostly only used very early, when only the
boot time SVSM calling area is in use. However, this API is also called by the
kexec finishing code, which runs very late, and potentially from a different
CPU (which uses a different calling area).
To avoid pulling the per-CPU SVSM calling area pointers and related SEV state
into the startup code, refactor the page state change API so the SVSM calling
area virtual and physical addresses can be provided by the caller.
No functional change intended.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250828102202.1849035-32-ardb+git@google.com
show more ...
|
| d5949ea5 | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/sev: Share implementation of MSR-based page state change
Both the decompressor and the SEV startup code implement the exact same
sequence for invoking the MSR based communication protocol to eff
x86/sev: Share implementation of MSR-based page state change
Both the decompressor and the SEV startup code implement the exact same
sequence for invoking the MSR based communication protocol to effectuate
a page state change.
Before tweaking the internal APIs used in both versions, merge them and
share them so those tweaks are only needed in a single place.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250828102202.1849035-31-ardb+git@google.com
show more ...
|
| a5f03880 | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/sev: Avoid global variable to store virtual address of SVSM area
The boottime SVSM calling area is used both by the startup code running from
a 1:1 mapping, and potentially later on running from
x86/sev: Avoid global variable to store virtual address of SVSM area
The boottime SVSM calling area is used both by the startup code running from
a 1:1 mapping, and potentially later on running from the ordinary kernel
mapping.
This SVSM calling area is statically allocated, and so its physical address
doesn't change. However, its virtual address depends on the calling context
(1:1 mapping or kernel virtual mapping), and even though the variable that
holds the virtual address of this calling area gets updated from 1:1 address
to kernel address during the boot, it is hard to reason about why this is
guaranteed to be safe.
So instead, take the RIP-relative address of the boottime SVSM calling area
whenever its virtual address is required, and only use a global variable for
the physical address.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Link: https://lore.kernel.org/20250828102202.1849035-30-ardb+git@google.com
show more ...
|
| 37dbd78f | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/sev: Move GHCB page based HV communication out of startup code
Both the decompressor and the core kernel implement an early #VC handler,
which only deals with CPUID instructions, and full featur
x86/sev: Move GHCB page based HV communication out of startup code
Both the decompressor and the core kernel implement an early #VC handler,
which only deals with CPUID instructions, and full featured one, which can
handle any #VC exception.
The former communicates with the hypervisor using the MSR based protocol,
whereas the latter uses a shared GHCB page, which is configured a bit later
during the boot, when the kernel runs from its ordinary virtual mapping,
rather than the 1:1 mapping that the startup code uses.
Accessing this shared GHCB page from the core kernel's startup code is
problematic, because it involves converting the GHCB address provided by the
caller to a physical address. In the startup code, virtual to physical address
translations are problematic, given that the virtual address might be a 1:1
mapped address, and such translations should therefore be avoided.
This means that exposing startup code dealing with the GHCB to callers that
execute from the ordinary kernel virtual mapping should be avoided too. So
move all GHCB page based communication out of the startup code, now that all
communication occurring before the kernel virtual mapping is up relies on the
MSR protocol only.
As an exception, add a flag representing the need to apply the coherency
fix in order to avoid exporting CPUID* helpers because of the code
running too early for the *cpu_has* infrastructure.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250828102202.1849035-29-ardb+git@google.com
show more ...
|
| e349241b | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/sev: Run RMPADJUST on SVSM calling area page to test VMPL
Determining the VMPL at which the kernel runs involves performing a RMPADJUST
operation on an arbitrary page of memory, and observing wh
x86/sev: Run RMPADJUST on SVSM calling area page to test VMPL
Determining the VMPL at which the kernel runs involves performing a RMPADJUST
operation on an arbitrary page of memory, and observing whether it succeeds.
The use of boot_ghcb_page in the core kernel in this case is completely
arbitrary, but results in the need to provide a PIC alias for it. So use
boot_svsm_ca_page instead, which already needs this alias for other reasons.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Link: https://lore.kernel.org/20250828102202.1849035-28-ardb+git@google.com
show more ...
|
| 7cb7b6de | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/sev: Use MSR protocol only for early SVSM PVALIDATE call
The early page state change API performs an SVSM call to PVALIDATE each page
when running under a SVSM, and this involves either a GHCB p
x86/sev: Use MSR protocol only for early SVSM PVALIDATE call
The early page state change API performs an SVSM call to PVALIDATE each page
when running under a SVSM, and this involves either a GHCB page based call or
a call based on the MSR protocol.
The GHCB page based variant involves VA to PA translation of the GHCB address,
and this is best avoided in the startup code, where virtual addresses are
ambiguous (1:1 or kernel virtual).
As this is the last remaining occurrence of svsm_perform_call_protocol() in
the startup code, switch to the MSR protocol exclusively in this particular
case, so that the GHCB based plumbing can be moved out of the startup code
entirely in a subsequent patch.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Link: https://lore.kernel.org/20250828102202.1849035-27-ardb+git@google.com
show more ...
|
| c15a4705 | 28-Aug-2025 |
Ard Biesheuvel <ardb@kernel.org> |
x86/sev: Use MSR protocol for remapping SVSM calling area
As the preceding code comment already indicates, remapping the SVSM
calling area occurs long before the GHCB page is configured, and so
call
x86/sev: Use MSR protocol for remapping SVSM calling area
As the preceding code comment already indicates, remapping the SVSM
calling area occurs long before the GHCB page is configured, and so
calling svsm_perform_call_protocol() is guaranteed to result in a call
to svsm_perform_msr_protocol().
So just call the latter directly. This allows most of the GHCB based API
infrastructure to be moved out of the startup code in a subsequent
patch.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Link: https://lore.kernel.org/20250828102202.1849035-26-ardb+git@google.com
show more ...
|