arm64: Support mapping a 52-bit physical adddress

When FEAT_LPA2 is enabled the physical address space increases from
48-bits to 52-bits. The top two address bits are moved to the now
unused shareab

arm64: Support mapping a 52-bit physical adddress

When FEAT_LPA2 is enabled the physical address space increases from
48-bits to 52-bits. The top two address bits are moved to the now
unused shareability field.

Update the kernel to support this new larger address space.

Reviewed by: alc, kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46624

show more ...

arm64: fix ID_AA64ISAR2_WFxT_IMPL value

Bits 3:0 of ID_AA64ISAR2_EL1 are reserved for FEAT_WFxT in ARMv8.7, with
bit 1 used to determine whether the feature is supported. The macro,
ID_AA64ISAR2_WFx

arm64: fix ID_AA64ISAR2_WFxT_IMPL value

Bits 3:0 of ID_AA64ISAR2_EL1 are reserved for FEAT_WFxT in ARMv8.7, with
bit 1 used to determine whether the feature is supported. The macro,
ID_AA64ISAR2_WFxT_IMPL can be used to check if this bit is set, however
it is currently incorrectly set to check bit 0. This has now been
corrected.

Reviewed by: andrew
Sponsored by: Arm Ltd
Signed-off-by: Harry Moulton <harry.moulton@arm.com>
Differential Revision: https://reviews.freebsd.org/D48577

show more ...

arm64: Add CTR_EL0 op and CR values

Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47813

arm64: Fix the alt name for ZCR_EL1

Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47810

arm64: Don't enable interrupts when in a spinlock

When we receive an exception while in a spinlock we shouldn't enable
interrupts. When entering a spinlock we disable interrupts so enabling
them her

arm64: Don't enable interrupts when in a spinlock

When we receive an exception while in a spinlock we shouldn't enable
interrupts. When entering a spinlock we disable interrupts so enabling
them here could cause surprising results.

The three cases that could cause this are:
1. A break-before-make sequence
2. Accessing possibly unmapped code with a fault handler
3. Buggy code

1 and 2 are supported later in the data abort handler, and 3 should be
fixed when found.

Reviewed by: mmel, kib, markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46816

show more ...

arm64: Decode CTR_EL0 via a table

Use the same method to decode CTR_EL0 as for the ID registers. This
will allow us to create a common view over all CPUs.

This will also allow us to create a common

arm64: Decode CTR_EL0 via a table

Use the same method to decode CTR_EL0 as for the ID registers. This
will allow us to create a common view over all CPUs.

This will also allow us to create a common view for userspace and the
kernel if we detect a difference on some CPUs, or to handle errata.

Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47124

show more ...

arm64: Add ID register field width values

In preperation for supporting non-4 bit field width to identcpu.c,
e.g. for ctr_el0, add the width of the existing ID register fields.

Reviewed by: imp
Spo

arm64: Add ID register field width values

In preperation for supporting non-4 bit field width to identcpu.c,
e.g. for ctr_el0, add the width of the existing ID register fields.

Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47118

show more ...

arm64: Remove old I-Cache types

The VPIPT and AIVIVT cache types are reserved from Armv8.0. Remove
them as nothing will report these values.

Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revi

arm64: Remove old I-Cache types

The VPIPT and AIVIVT cache types are reserved from Armv8.0. Remove
them as nothing will report these values.

Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47117

show more ...

arm64: Enable SVE in userspace

Report when SVE is present and allow it to be used by calling
sve_restore_state on an SVE exception from userspace.

Reviewed by: kib
Sponsored by: Arm Ltd
Differentia

arm64: Enable SVE in userspace

Report when SVE is present and allow it to be used by calling
sve_restore_state on an SVE exception from userspace.

Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D43310

show more ...

arm64: Initial SVE support

Add initial kernel support for SVE. This detects if SVE is present on
all CPUs, and if so allows for the use of SVE in the future.

As the SVE registers are a superset of

arm64: Initial SVE support

Add initial kernel support for SVE. This detects if SVE is present on
all CPUs, and if so allows for the use of SVE in the future.

As the SVE registers are a superset of the VFP registers we don't need
to restore the VFP registers when SVE is enabled.

Ths interface to enable SVE is provided, but not used until SVE is
supported in signals and with ptrace.

Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D43306

show more ...

arm64: Add support for FIQs

arm64 supports FIQs, fast interrupt requests, which are required by
Apple silicon which hardwires the ARM timers to FIQs. This is needed by
the upcoming Apple Interrupt C

arm64: Add support for FIQs

arm64 supports FIQs, fast interrupt requests, which are required by
Apple silicon which hardwires the ARM timers to FIQs. This is needed by
the upcoming Apple Interrupt Controller. Based on work by andrew@ and
kevans@ in https://reviews.freebsd.org/D40161.

Signed-off-by: Ayrton Munoz <a.munoz3327@gmail.com>
Co-authored-by: Kyle Evans <kevans@FreeBSD.org>
Co-authored-by: Andrew Turner <andrew@FreeBSD.org>
Reviewed-by: imp,mmel,mhorne
Pull-Request: https://github.com/freebsd/freebsd-src/pull/1363

show more ...

arm64: Remove TCR_CACHE_ATTRS and TCR_SMP_ATTRS

These are only used in one place so expand them there.

While here always set TCR_SH0_IS and TCR_SH1_IS. There is no advantage
to not set them in a UP

arm64: Remove TCR_CACHE_ATTRS and TCR_SMP_ATTRS

These are only used in one place so expand them there.

While here always set TCR_SH0_IS and TCR_SH1_IS. There is no advantage
to not set them in a UP kernel.

Reviewed by: alc, kib, markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46393

show more ...

arm64: Add the tcr_el2 ds field

This will be used to support FEAT_LPA2 to allow more than 48 bits of
physical address space.

Reviewed by: alc, kib, markj
Sponsored by: Arm Ltd
Differential Revision

arm64: Add the tcr_el2 ds field

This will be used to support FEAT_LPA2 to allow more than 48 bits of
physical address space.

Reviewed by: alc, kib, markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46392

show more ...

arm64: Fix the ESR_EL1_op2 value

This leads to an unallocated register. Fix the value to point to
ESR_EL1.

Sponsored by: Arm Ltd

arm64: Add counter timer registers to armreg.h

Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46071

arm64: Add EL1 and EL12 register alt names

When adding VHE support we may need to access these registers using
the _EL12 or _EL1 name depending on if VHE is enabled or not.

Add the registers we wil

arm64: Add EL1 and EL12 register alt names

When adding VHE support we may need to access these registers using
the _EL12 or _EL1 name depending on if VHE is enabled or not.

Add the registers we will likely need to access from VHE so these can
be updated in the guest switcher code.

Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45525

show more ...

arm64: Fix indentation to be consistent

Adjust the mair_el1 macro indentation to be consistent with the
surrounding macros.

Reviewed by: emaste
Sponsored by: Arm Ltd
Differential Revision: https://

arm64: Fix indentation to be consistent

Adjust the mair_el1 macro indentation to be consistent with the
surrounding macros.

Reviewed by: emaste
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45524

show more ...

arm64: Add the pointer auth registers to armreg.h

Add the pointer authentication registers to armreg.h. These will be
used to support pointer authentication in a kernel built with GCC.

Reviewed by:

arm64: Add the pointer auth registers to armreg.h

Add the pointer authentication registers to armreg.h. These will be
used to support pointer authentication in a kernel built with GCC.

Reviewed by: jhb
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45262

show more ...

arm64: Use the UL macro in TCR_EL1 defines

While clang can handle numbers with a UL suffix in assembly files
gcc/gas is unable to. Switch to use the UL macro for TCR_EL1 defines as
some are used in

arm64: Use the UL macro in TCR_EL1 defines

While clang can handle numbers with a UL suffix in assembly files
gcc/gas is unable to. Switch to use the UL macro for TCR_EL1 defines as
some are used in locore.S

Reviewed by: brooks, jhb
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45261

show more ...

arm64: add CONTEXTIDR_EL1 reg

CONTEXTIDR_EL1 is used in debug and trace features to identify the
current process or context.

Reviewed by: andrew
Sponsored by: Arm Ltd
Differential Revision: https:/

arm64: add CONTEXTIDR_EL1 reg

CONTEXTIDR_EL1 is used in debug and trace features to identify the
current process or context.

Reviewed by: andrew
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45173

show more ...

arm64: add PMBSR_MSS_{BSC,FSC} status code field

Bits [5:0] of PMBSR_MSS encodes either Buffer Status Code (BSC) or Fault
Status Code (FSC) depending on PMBSR_EC value.

Add PMBSR_MSS_{BSC,FSC} to c

arm64: add PMBSR_MSS_{BSC,FSC} status code field

Bits [5:0] of PMBSR_MSS encodes either Buffer Status Code (BSC) or Fault
Status Code (FSC) depending on PMBSR_EC value.

Add PMBSR_MSS_{BSC,FSC} to cover this field.

Reviewed by: andrew
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45172

show more ...

arm64: make SPE regs use ALT_NAME macro

When the register is not defined in Armv8.0 i.e. added in a later
extension, like SPE added in v8.2, the alternative name format of:
S<op0>_<op1>_C<crn>_C

arm64: make SPE regs use ALT_NAME macro

When the register is not defined in Armv8.0 i.e. added in a later
extension, like SPE added in v8.2, the alternative name format of:
S<op0>_<op1>_C<crn>_C<crm>_<op2>
should be used; otherwise, calls to {READ,WRITE}_SPECIALREG() will
fail.

Use the MRS_REG_ALT_NAME() macro for SPE changing hex to decimal as
required by the macro.

Reviewed by: andrew
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45171

show more ...

arm64: Add MRS_REG_ALT_NAME ID register macros

These can be used even when the compiler is too old for the register
to be included.

Reviewed by: Zachary Leaf <zachary.leaf@arm.com>
Sponsored by: Ar

arm64: Add MRS_REG_ALT_NAME ID register macros

These can be used even when the compiler is too old for the register
to be included.

Reviewed by: Zachary Leaf <zachary.leaf@arm.com>
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D45176

show more ...

arm64: Add EL1 hardware breakpoint exceptions

Reviewed by: jhb
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D44353

arm64: Add ISS_MSR_REG for ESR_ELx.ISS values

Add a macro to get the ESR_ELx ISS value when we trap accessing a
special register.