am64: Allow cpu.h to be included from assembly

Reviewed by: jhibbits, kevans
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45081

arm64: Add a CPU reset hook instead of expecting PSCI

Some SoCs do not include a PSCI for power management and defer it to
something else instead. Add a CPU reset hook to account for this, and
use i

arm64: Add a CPU reset hook instead of expecting PSCI

Some SoCs do not include a PSCI for power management and defer it to
something else instead. Add a CPU reset hook to account for this, and
use it in the psci driver.

Reviewed by: andrew
Obtained from: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D44535

show more ...

arm64: add CPU part identifiers for Apple M1 and M2

Part numbers are sourced from Linux (arch/arm64/include/asm/cputype.h).

MFC after: 2 weeks
Pull Request: https://github.com/freebsd/freebsd-src/p

arm64: add CPU part identifiers for Apple M1 and M2

Part numbers are sourced from Linux (arch/arm64/include/asm/cputype.h).

MFC after: 2 weeks
Pull Request: https://github.com/freebsd/freebsd-src/pull/1144

show more ...

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl s

sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

show more ...

sys: Remove $FreeBSD$: one-line .h pattern

Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/

arm64 lib32: prepare arm64 headers to redirect to arm

In order to compile lib32 libraries and other 32-bit code on arm64,
<machine/foo.h> needs to be redirected to an arm header rather
than arm64 wh

arm64 lib32: prepare arm64 headers to redirect to arm

In order to compile lib32 libraries and other 32-bit code on arm64,
<machine/foo.h> needs to be redirected to an arm header rather
than arm64 when building with -m32. Ifdef the arm64 headers that
are installed in /usr/include/machine and used by user-level software
(including references from /usr/include/*.h) so that if __arm__ is
defined when including the arm64 version, <arm/foo.h> is included
rather than using the rest of the file's contents. Some arm headers
had no arm64 equivalent; headers were added just to do the redirection.
These files use #error if __arm__ is not defined to guard against
confusion. Also add an include/arm Makefile, and modify Makefiles
as needed to install everything, including the arm files in
/usr/include/arm. fenv.h comes from lib/msun/arm/fenv.h.

The new arm64 headers are:
acle-compat.h
cpuinfo.h
sysreg.h

Reviewed by: jrtc27, imp
Differential Revision: https://reviews.freebsd.org/D40944

show more ...

arm64: Add a masked get_kernel_reg()

This lets consumers fetch the value of a system register and apply a
mask over individual fields. That is, each field in the returned value
will be the "smaller

arm64: Add a masked get_kernel_reg()

This lets consumers fetch the value of a system register and apply a
mask over individual fields. That is, each field in the returned value
will be the "smaller" of the two provided by "mask" and the value saved
in kern_cpu_desc. This will be used by vmm to sanitize host system
register fields.

Reviewed by: andrew
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc. (hardware)
Differential Revision: https://reviews.freebsd.org/D40500

show more ...

arm64: Malloc the cpu_desc array

We only need this during boot. Allocate the array before starting CPUs
to reduce the memory usage.

Reviewed by: Zach Leaf <zachary.leaf@arm.com>
Sponsored by: Arm L

arm64: Malloc the cpu_desc array

We only need this during boot. Allocate the array before starting CPUs
to reduce the memory usage.

Reviewed by: Zach Leaf <zachary.leaf@arm.com>
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D40433

show more ...

arm64: Remove duplicated function prototypes for PAC

No functional change intended.

Sponsored by: The FreeBSD Foundation

Add CPU Ident for Qualcomm Kryo 400 (used in MS Dev Kit)

Reviewed by: imp
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D37767

Add the arch field to the arm64 MIDR macros

For completeness add accessors for the MIDR field. As the field is
always 0xf on arm64 it is unneeded in the current MICR handling, but
will be used in th

Add the arch field to the arm64 MIDR macros

For completeness add accessors for the MIDR field. As the field is
always 0xf on arm64 it is unneeded in the current MICR handling, but
will be used in the vmm module for bhyve.

Obtained from: https://github.com/FreeBSD-UPB/freebsd-src (earlier version)
Sponsored by: The FreeBSD Foundation

show more ...

Add more Arm CPUs to the arm64 cpu ident

These are based on CPUs found in https://github.com/ARM-software/data

Sponsored by: The FreeBSD Foundation

Add more Arm CPU IDs

Add more CPU main ID register values for Arm Cortex and Neoverse CPUs

Sponsored by: The FreeBSD Foundation

Add the Ampere and Fujitsu arm64 implementer IDs

Reviewed By: andrew, emaste
MFC after: 3 days
Sponsored by: Ampere Computing
Differential Revision: https://reviews.freebsd.org/D34666

arm64: Add explicit barrier after address translation instruction

Following ARMARM sec D5.2.11, which says:

> Where an instruction results in an update to a System register,
> as is the case with t

arm64: Add explicit barrier after address translation instruction

Following ARMARM sec D5.2.11, which says:

> Where an instruction results in an update to a System register,
> as is the case with the AT * address translation instructions,
> explicit synchronization must be performed before the result is
> guaranteed to be visible to subsequent direct reads of the
> PAR_EL1.

Reviewed By: andrew
MFC after: 3 weeks
Sponsored by: Ampere Computing
Differential Revision: https://reviews.freebsd.org/D34665

show more ...

Add NT_ARM_ADDR_MASK

This can be used by debuggers to find which bits in a virtual address
should be masked off to get a canonical address. This is currently used
by the Pointer Authentication Code

Add NT_ARM_ADDR_MASK

This can be used by debuggers to find which bits in a virtual address
should be masked off to get a canonical address. This is currently used
by the Pointer Authentication Code support to get its mask. It could also
be used if we support Top Byte Ignore for the same purpose.

Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34302

show more ...

Add arm64 pointer authentication support

Pointer authentication allows userspace to add instructions to insert
a Pointer Authentication Code (PAC) into a register based on an address
and modifier an

Add arm64 pointer authentication support

Pointer authentication allows userspace to add instructions to insert
a Pointer Authentication Code (PAC) into a register based on an address
and modifier and check if the PAC is correct. If the check fails it will
either return an invalid address or fault to the kernel.

As many of these instructions are a NOP when disabled and in earlier
revisions of the architecture this can be used, for example, to sign
the return address before pushing it to the stack making Return-oriented
programming (ROP) attack more difficult on hardware that supports them.

The kernel manages five 128 bit signing keys: 2 instruction keys, 2 data
keys, and a generic key. The instructions then use one of these when
signing the registers. Instructions that use the first four store the
PAC in the register being signed, however the instructions that use the
generic key store the PAC in a separate register.

Currently all userspace threads share all the keys within a process
with a new set of userspace keys being generated when executing a new
process. This means a forked child will share its keys with its parent
until it calls an appropriate exec system call.

In the kernel we allow the use of one of the instruction keys, the ia
key. This will be used to sign return addresses in function calls.
Unlike userspace each kernel thread has its own randomly generated.

Thread0 has a static key as does the early code on secondary CPUs.
This should be safe as there is minimal user interaction with these
threads, however we could generate random keys when the Armv8.5
Random number generation instructions are present.

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31261

show more ...

Simplify swi for bus_dma.

When a DMA request using bounce pages completes, a swi is triggered to
schedule pending DMA requests using the just-freed bounce pages. For
a long time this bus_dma swi ha

Simplify swi for bus_dma.

When a DMA request using bounce pages completes, a swi is triggered to
schedule pending DMA requests using the just-freed bounce pages. For
a long time this bus_dma swi has been tied to a "virtual memory" swi
(swi_vm). However, all of the swi_vm implementations are the same and
consist of checking a flag (busdma_swi_pending) which is always true
and if set calling busdma_swi. I suspect this dates back to the
pre-SMPng days and that the intention was for swi_vm to serve as a
mux. However, in the current scheme there's no need for the mux.

Instead, remove swi_vm and vm_ih. Each bus_dma implementation that
uses bounce pages is responsible for creating its own swi (busdma_ih)
which it now schedules directly. This swi invokes busdma_swi directly
removing the need for busdma_swi_pending.

One consequence is that the swi now works on RISC-V which had previously
failed to invoke busdma_swi from swi_vm.

Reviewed by: imp, kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D33447

show more ...

arm64, riscv: Fix TRAF_PC() to return the PC, not the return address.

Reviewed by: mhorne
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D31969

Add the Apple arm64 implementer ID

Sponsored by: The FreeBSD Foundation

Teach the arm64 kernel to identify the Arm AEM

The Arm Architecture Envelope Model is a simulator that models the
architecture rather than any specific implementation. Add its part ID
macro and add

Teach the arm64 kernel to identify the Arm AEM

The Arm Architecture Envelope Model is a simulator that models the
architecture rather than any specific implementation. Add its part ID
macro and add it to the list of Arm CPUs we can decode.

Sponsored by: The FreeBSD Foundation

show more ...

arm64: clean up empty lines in .c and .h files

Read the CPU 0 arm64 ID registers early in initarm

We also update the kernel view early in the boot. This will allow the
use of the common kernel view in ifunc resolvers.

Sponsored by: Innovate UK

Move ID reading signatures to a better header

The functions to read the common user and kernel ID registers should be
in cpu.h rather than undefined.h as they are related to CPU details and
used by

Move ID reading signatures to a better header

The functions to read the common user and kernel ID registers should be
in cpu.h rather than undefined.h as they are related to CPU details and
used by undefined instruction handlers.

Sponsored by: Innovate UK

show more ...

Merge ^/head r358466 through r358677.