libkern: add ilog2 macro

The kernel source contains several definitions of an ilog2 function;
some are slower than necessary, and one of them is incorrect.
Elimininate them all and define an ilog2 m

libkern: add ilog2 macro

The kernel source contains several definitions of an ilog2 function;
some are slower than necessary, and one of them is incorrect.
Elimininate them all and define an ilog2 macro in libkern to replace
them, in a way that is fast, correct for all argument types, and, in a
GENERIC kernel, includes a check for an invalid zero parameter.

Folks at Microsoft have verified that having a correct ilog2
definition for their MANA driver doesn't break it.

Reviewed by: alc, markj, mhorne (older version), jhibbits (older version)
Differential Revision: https://reviews.freebsd.org/D45170
Differential Revision: https://reviews.freebsd.org/D45235

show more ...

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

Consistently provide ffs/fls using builtins

Use of compiler builtin ffs/ctz functions will result in optimized
instruction sequences when possible, and fall back to calling a function
provided by th

Consistently provide ffs/fls using builtins

Use of compiler builtin ffs/ctz functions will result in optimized
instruction sequences when possible, and fall back to calling a function
provided by the compiler run-time library. We have slowly shifted our
platforms to take advantage of these builtins in 60645781d613 (arm64),
1c76d3a9fbef (arm), 9e319462a03a (powerpc, partial).

Some platforms still rely on the libkern implementations of these
functions provided by libkern, namely riscv, powerpc (ffs*, flsll), and
i386 (ffsll and flsll). These routines are slow, as they perform a
linear search for the bit in question. Even on platforms lacking
dedicated bit-search instructions, such as riscv, the compiler library
will provide better-optimized routines, e.g. by using binary search.

Consolidate all definitions of these functions (whether currently using
builtins or not) to libkern.h. This should result in equivalent or
better performing routines in all cases.

One wart in all of this is the existing HAVE_INLINE_F*** macros, which
we use in a few places to conditionally avoid the slow libkern routines.
These aren't easily removed in one commit. For now, provide these
defines unconditionally, but marked for removal after subsequent
cleanup.

Removal of the now unused libkern routines will follow in the next
commit.

Reviewed by: dougm, imp (previous version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40698

show more ...

x86: cleanup in machine/cpufunc.h

Reduce diffs between amd64 and i386 and improve whitespace

Reviewed by: jhb, imp

Remove checks for <sys/cdefs.h> being included.

These files no longer depend on the macros required when these checks
were added.

PR: 263102 (exp-run)
Reviewed by: brooks, imp, emaste
Differential

Remove checks for <sys/cdefs.h> being included.

These files no longer depend on the macros required when these checks
were added.

PR: 263102 (exp-run)
Reviewed by: brooks, imp, emaste
Differential Revision: https://reviews.freebsd.org/D34804

show more ...

Remove checks for __CC_SUPPORTS__INLINE assuming it is always true.

All supported compilers (modern versions of GCC and clang) support
this.

PR: 263102 (exp-run)
Reviewed by: imp
Differential Revi

Remove checks for __CC_SUPPORTS__INLINE assuming it is always true.

All supported compilers (modern versions of GCC and clang) support
this.

PR: 263102 (exp-run)
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D34802

show more ...

Remove checks for __GNUCLIKE_ASM assuming it is always true.

All supported compilers (modern versions of GCC and clang) support
this.

Many places didn't have an #else so would just silently do the

Remove checks for __GNUCLIKE_ASM assuming it is always true.

All supported compilers (modern versions of GCC and clang) support
this.

Many places didn't have an #else so would just silently do the wrong
thing. Ancient versions of icc (the original motivation for this) are
no longer a compiler FreeBSD supports.

PR: 263102 (exp-run)
Reviewed by: brooks, imp
Differential Revision: https://reviews.freebsd.org/D34797

show more ...

x86: cpufunc: Add rdtsc_ordered()

Add a variant of 'rdtsc()' that performs the ordered version of 'rdtsc'
appropriate for the invoking x86 variant.

Also, expose the 'lfence'-ed and 'mfence'-ed 'rdt

x86: cpufunc: Add rdtsc_ordered()

Add a variant of 'rdtsc()' that performs the ordered version of 'rdtsc'
appropriate for the invoking x86 variant.

Also, expose the 'lfence'-ed and 'mfence'-ed 'rdtsc()' variants needed
by 'rdtsc_ordered()' for general use.

Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D31416

show more ...

x86: cpufunc: Add rdtscp_aux()

Add a variant of 'rdtscp()' that retains and returns the 'IA32_TSC_AUX'
value read by 'rdtscp'.

Sponsored By: Juniper Networks, Inc.
Sponsored By: Klara, Inc.
Reviewe

x86: cpufunc: Add rdtscp_aux()

Add a variant of 'rdtscp()' that retains and returns the 'IA32_TSC_AUX'
value read by 'rdtscp'.

Sponsored By: Juniper Networks, Inc.
Sponsored By: Klara, Inc.
Reviewed by: markj, kib
Differential Revision: https://reviews.freebsd.org/D31415

show more ...

x86: Add rdtscp32() into cpufunc.h.

Suggested by: markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27986

MFhead@r344527

Merge ^/head r344178 through r344512.

Provide convenience C wrappers for RDPKRU and WRPKRU instructions.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Differential revision: https://reviews.fre

Provide convenience C wrappers for RDPKRU and WRPKRU instructions.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Differential revision: https://reviews.freebsd.org/D18893

show more ...

MFhead@r344270

Merge ^/head r343956 through r344177.

Provide userspace versions of do_cpuid() and cpuid_count() on i386.

Some older compilers, when generating PIC code, cannot handle inline
asm that clobbers %ebx (because %ebx is used as the GOT offse

Provide userspace versions of do_cpuid() and cpuid_count() on i386.

Some older compilers, when generating PIC code, cannot handle inline
asm that clobbers %ebx (because %ebx is used as the GOT offset
register). Userspace versions avoid clobbering %ebx by saving it to
stack before executing the CPUID instruction.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

show more ...

Tell the compiler that rdtscp clobbers %ecx.

cpufunc: add rdtscp for x86

Merge ^/head r325999 through r326131.

sys: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for

sys: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

show more ...

MFhead@r322451

Merge ^/head r321383 through r322397.

MFhead@r322057

x86: Tag some intrinsics with __pure2

Some C wrappers for x86 instructions do not touch global memory and only act
on their arguments; they can be marked __pure2, aka __const__. Without this
annota

x86: Tag some intrinsics with __pure2

Some C wrappers for x86 instructions do not touch global memory and only act
on their arguments; they can be marked __pure2, aka __const__. Without this
annotation, Clang 3.9.1 is not intelligent enough on its own to grok that
these functions are __const__.

Submitted by: Anton Rang <anton.rang AT isilon.com>
Sponsored by: Dell EMC Isilon

show more ...

Don't access the reserved registers %dr4 and %dr5 on i386.

On the original i386, %dr[4-5] were unimplemented but not very clearly
reserved, so debuggers read them to print them. i386 was still doin

Don't access the reserved registers %dr4 and %dr5 on i386.

On the original i386, %dr[4-5] were unimplemented but not very clearly
reserved, so debuggers read them to print them. i386 was still doing
this.

On the original athlon64, %dr[4-5] are documented as reserved but are
aliased to %dr[6-7] unless CR4_DE is set, when accessing them traps.

On 2 of my systems, accessing %dr[4-5] trapped sometimes. On my Haswell
system, the apparent randomness was because the boot CPU starts with
CR4_DE set while all other CPUs start with CR4_DE clear. FreeBSD
doesn't support the data breakpoints enabled by CR4_DE and it never
changes this flag, so the flag remains different across CPUs and
the behaviour seemed inconsistent except while booting when the CPU
doesn't change.

The invalid accesses broke:
- read access for printing the registers in ddb "show watches" on CPUs
with CR4_DE set
- read accesses in fill_dbregs() on CPUs with CR4_DE set. This didn't
implement panic(3) since the user case always skipped %dr[4-5].
- write accesses in set_dbregs(). This also didn't affect userland.
When it didn't trap, the aliasing made it fragile.

Don't print the dummy (zero) values of %dr[4-5] in "show watches" for
i386 or amd64. Fix style bugs near this printing.

amd64 also has space in the dbregs struct for the reserved %dr[8-15]
and already didn't print the dummy values for these, and never accessed
any of the 10 reserved debug registers.

Remove cpufuncs for making the invalid accesses. Even amd64 had these.

show more ...