sys: Remove $FreeBSD$: one-line .c comment pattern

Remove /^/[*/]\s*\$FreeBSD\$.*\n/

AST: rework

Make most AST handlers dynamically registered. This allows to have
subsystem-specific handler source located in the subsystem files,
instead of making subr_trap.c aware of it. For inst

AST: rework

Make most AST handlers dynamically registered. This allows to have
subsystem-specific handler source located in the subsystem files,
instead of making subr_trap.c aware of it. For instance, signal
delivery code on return to userspace is now moved to kern_sig.c.

Also, it allows to have some handlers designated as the cleanup (kclear)
type, which are called both at AST and on thread/process exit. For
instance, ast(), exit1(), and NFS server no longer need to be aware
about UFS softdep processing.

The dynamic registration also allows third-party modules to register AST
handlers if needed. There is one caveat with loadable modules: the
code does not make any effort to ensure that the module is not unloaded
before all threads processed through AST handler in it. In fact, this
is already present behavior for hwpmc.ko and ufs.ko. I do not think it
is worth the efforts and the runtime overhead to try to fix it.

Reviewed by: markj
Tested by: emaste (arm64), pho
Discussed with: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D35888

show more ...

powerpc64: Optimize radix trap handling a little more

Summary:
Since PCPU can live in a GPR for a while longer, let it, rather than
re-getting it in yet another register. MFSPR is an expensive oper

powerpc64: Optimize radix trap handling a little more

Summary:
Since PCPU can live in a GPR for a while longer, let it, rather than
re-getting it in yet another register. MFSPR is an expensive operation,
12 clock latency on POWER9, so the fewer operations we need, the better.

Since the check is tightly coupled to the fetch, by reducing the number
of fetch+check, we reduce the stalls, and improve the performance
marginally. Buildworld was measured at a ~5-7% improvement on a single
run.

Reviewed By: nwhitehorn
Differential Revision: https://reviews.freebsd.org/D30003

show more ...

Add CFI start/end proc directives to arm64, i386, and ppc

Follow-up to r353959 and r368070: do the same for other architectures.

arm32 already seems to use its own .fnstart/.fnend directives, which

Add CFI start/end proc directives to arm64, i386, and ppc

Follow-up to r353959 and r368070: do the same for other architectures.

arm32 already seems to use its own .fnstart/.fnend directives, which
appear to be ARM-specific variants of the same thing. Likewise, MIPS
uses .frame directives.

Reviewed by: arichardson
Differential Revision: https://reviews.freebsd.org/D27387

show more ...

[PowerPC64LE] Fix AP spinup on powernv.

OPAL unconditionally enters secondary CPUs with only HV and SF set.

I tried writing a secondary entry point instead, but OPAL rejected it
and I am unsure why

[PowerPC64LE] Fix AP spinup on powernv.

OPAL unconditionally enters secondary CPUs with only HV and SF set.

I tried writing a secondary entry point instead, but OPAL rejected it
and I am unsure why, so I resorted to making the system reset interrupt
endian-flexible.

This means we take a slight performance hit on wakeup on LE, but it is
a good stopgap until we can figure out a reliable way to make OPAL enter
where we want it to.

It probably makes sense to have it around anyway, because I can imagine
scenarios where the cpu resets itself to BE and does a software reset.

Sponsored by: Tag1 Consulting, Inc.

show more ...

powerpc: Add a (CPU/runtime features) flags set to pcpu struct

Summary:
The point of this addition is to cache CPU behavior 'features', to avoid
having to recompute based on CPU, etc.

The first suc

powerpc: Add a (CPU/runtime features) flags set to pcpu struct

Summary:
The point of this addition is to cache CPU behavior 'features', to avoid
having to recompute based on CPU, etc.

The first such use case is to avoid the unnecessary manipulation of the
SLBs (Segment Lookaside Buffers) when using the Radix pmap on POWER9.
Since we already get the PCPU pointer wherever we swap the SLB entries,
we can use a cached flag to check if it's necessary to perform the
operation anyway, and skip it when not.

Reviewed by: bdragon
Differential Revision: https://reviews.freebsd.org/D24908

show more ...

[PowerPC64] Minor correctness fix in rstcode.

TRAP_ENTRY(0) should be TRAP_GENTRAP(0) here.

However, in practice, it doesn't matter, as the only time TRAP_ENTRY and
TRAP_GENTRAP can differ is when

[PowerPC64] Minor correctness fix in rstcode.

TRAP_ENTRY(0) should be TRAP_GENTRAP(0) here.

However, in practice, it doesn't matter, as the only time TRAP_ENTRY and
TRAP_GENTRAP can differ is when bridge mode is active, which is impossible
on the 64 bit kernel.

Fix it anyway in case we ever need to add a trap preamble on PPC64.

show more ...

powerpc: Add a CPU-custom machine check handler

Summary:
Some machine checks are process-recoverable, others are not. Let a
CPU-specific handler decide what to do.

This works around a machine chec

powerpc: Add a CPU-custom machine check handler

Summary:
Some machine checks are process-recoverable, others are not. Let a
CPU-specific handler decide what to do.

This works around a machine check error hit while building www/firefox
and mail/thunderbird, which would otherwise cause the build to fail.

More work is needed to handle all possible machine check conditions, but
this is sufficient to unblock some ports building.

Differential Revision: https://reviews.freebsd.org/D23731

show more ...

Merge ^/head r356848 through r356919.

D23057: [PowerPC] Fix offset calculations in bridge mode

In rS354701, I replaced text relocations with offsets from &generictrap.

Unfortunately, the magic variable I was using doesn't actually mean

D23057: [PowerPC] Fix offset calculations in bridge mode

In rS354701, I replaced text relocations with offsets from &generictrap.

Unfortunately, the magic variable I was using doesn't actually mean the
address of &generictrap, in bridge mode it actually means &generictrap64.

So, for bridge mode to work, it is necessary to differentiate between
"where do we need to branch to to handle a trap" and "where is &generictrap
for purposes of doing relative math".

Introduce a new TRAP_ENTRY and use it instead of TRAP_GENTRAP for doing
actual calls to the generic trap handler.

Reported by: Mark Millard <marklmi@yahoo.com>
Reviewed by: jhibbits
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D23057

show more ...

powerpc/aim: Fix comment typo

MFHead @348740

Sponsored by: The FreeBSD Foundation

Fix PPC64 kernel build with clang8 + lld8

This patch fixes the following lld link errors:

- unsupported dynamic relocations on read-only sections
- out-of-range TOC references

Submitted by: git_bd

Fix PPC64 kernel build with clang8 + lld8

This patch fixes the following lld link errors:

- unsupported dynamic relocations on read-only sections
- out-of-range TOC references

Submitted by: git_bdragon.rtk0.net
Reviewed by: jhibbits, luporl
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D19352

show more ...

MFHead@r345677

powerpc64: Use medium code model in asm files for TOC references

Summary:
With a sufficiently large TOC, it's possible to index out of range, as
the immediate load instructions only permit 16-bit in

powerpc64: Use medium code model in asm files for TOC references

Summary:
With a sufficiently large TOC, it's possible to index out of range, as
the immediate load instructions only permit 16-bit indices, allowing up
to 64kB range (signed) from the base pointer. Allow +/- 2GB range, with
the medium code model TOC accesses in asm.

Patch originally by Brandon Bergren. The issue appears to impact ELFv2
more than ELFv1.

Reviewed by: luporl
Differential Revision: https://reviews.freebsd.org/D19708

show more ...

MFHead@r345275

MFhead@r344996

powerpc64: Fix early exit with invalid kernel SLB entries

The check for early exit should be checking the SLB entry itself. As
currently written it was checking the address of the SLB, which is alw

powerpc64: Fix early exit with invalid kernel SLB entries

The check for early exit should be checking the SLB entry itself. As
currently written it was checking the address of the SLB, which is always
non-zero, so would go through the kernel SR restore loop regardless.

Submitted by: mmacy
MFC after: 2 weeks

show more ...

Merge ^/head r343712 through r343806.

powerpc64: Add a trap stack area

Currently, the trap code switches to the the temporary stack in the dbtrap
section. It works in most cases, but in the beginning of the execution, the
temp stack is

powerpc64: Add a trap stack area

Currently, the trap code switches to the the temporary stack in the dbtrap
section. It works in most cases, but in the beginning of the execution, the
temp stack is being used, as starting in the powerpc_init() code.

In this current scenario, the stack is being overwritten, which causes the
return of breakpoint() to take abnormal execution.

This current patchset create a small stack to use by the dbtrap: codepath
avoiding the corruption of the temporary stack.

PR: 224872
Submitted by: breno.leitao_gmail.com
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D14484

show more ...

powerpc64: Fix stack setup in dbtrap

r330610 relocated the DMAP from the base of memory to the base of the fourth
quadrant of memory. This broke synthetic traps, such as KDB forced
breakpoints. Us

powerpc64: Fix stack setup in dbtrap

r330610 relocated the DMAP from the base of memory to the base of the fourth
quadrant of memory. This broke synthetic traps, such as KDB forced
breakpoints. Use GET_TOCBASE() so the DMAP offset is handled.

Submitted by: git_bdragon.rkt0.net
Differential Revision: https://reviews.freebsd.org/D15973

show more ...

Add hypervisor trap handling, using HSRR0/HSRR1

Summary:
Some hypervisor exceptions on POWER architecture only save state to HSRR0/HSRR1.
Until we have bhyve on POWER, use a lightweight exception fr

Add hypervisor trap handling, using HSRR0/HSRR1

Summary:
Some hypervisor exceptions on POWER architecture only save state to HSRR0/HSRR1.
Until we have bhyve on POWER, use a lightweight exception frontend which copies
HSRR0/HSRR1 into SRR0/SRR1, and run the normal trap handler.

The first user of this is the Hypervisor Virtualization Interrupt, which targets
the XIVE interrupt controller on POWER9.

Reviewed By: nwhitehorn
Differential Revision: https://reviews.freebsd.org/D15487

show more ...

Move the powerpc64 direct map base address from zero to high memory. This
accomplishes a few things:
- Makes NULL an invalid address in the kernel, which is useful for catching
bugs.
- Lays groundw

Move the powerpc64 direct map base address from zero to high memory. This
accomplishes a few things:
- Makes NULL an invalid address in the kernel, which is useful for catching
bugs.
- Lays groundwork for radix-tree translation on POWER9, which requires the
direct map be at high memory.
- Similarly lays groundwork for a direct map on 64-bit Book-E.

The new base address is chosen as the base of the fourth radix quadrant
(the minimum kernel address in this translation mode) and because all
supported CPUs ignore at least the first two bits of addresses in real
mode, allowing direct-map addresses to be used in real-mode handlers.
This is required by Linux and is part of the architecture standard
starting in POWER ISA 3, so can be relied upon.

Reviewed by: jhibbits, Breno Leitao
Differential Revision: D14499

show more ...

PowerNV: Put processor to power-save state in idle thread

When processor enters power-save state it releases resources shared with other
cpu threads which makes other cores working much faster.

Thi

PowerNV: Put processor to power-save state in idle thread

When processor enters power-save state it releases resources shared with other
cpu threads which makes other cores working much faster.

This patch also implements saving and restoring registers that might get
corrupted in power-save state.

Submitted by: Patryk Duda <pdk@semihalf.com>
Obtained from: Semihalf
Reviewed by: jhibbits, nwhitehorn, wma
Sponsored by: IBM, QCM Technologies
Differential revision: https://reviews.freebsd.org/D14330

show more ...

Correct longjmp

Reviewed by: nwhitehorn
Differential Revision: https://reviews.freebsd.org/D14159