linker: Improve handling of ifuncs when fetching symbol metadata

When looking up symbol values, we map ifunc symbols to the value
returned by the resolver. However, the returned symbol size is stil

linker: Improve handling of ifuncs when fetching symbol metadata

When looking up symbol values, we map ifunc symbols to the value
returned by the resolver. However, the returned symbol size is still
that of the resolver. Be consistent and provide the size of the
implementation symbol as well.

This fixes an inconsistency in dtrace's FBT provider, which enumerates
all function symbols and disassembles their values, using the symbol
size as the bound for the disassembly loop. In particular, for ifuncs,
we were not creating return probes.

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D50683

show more ...

sys: use globals for the ELF kernel and module type strings

Initialize the globals with macros so we can use the same values in the
loader.

Also remove unnecessary "elfN module" checks.

Reviewed b

sys: use globals for the ELF kernel and module type strings

Initialize the globals with macros so we can use the same values in the
loader.

Also remove unnecessary "elfN module" checks.

Reviewed by: imp, kib
Pull Request: https://github.com/freebsd/freebsd-src/pull/1394

show more ...

sys: make the kernel metadata pointer global

The way we got the kernel metadata pointer was by calling
preload_search_by_type with one of the following three: "elf kernel",
"elf32 kernel" and "elf64

sys: make the kernel metadata pointer global

The way we got the kernel metadata pointer was by calling
preload_search_by_type with one of the following three: "elf kernel",
"elf32 kernel" and "elf64 kernel". Which one(s) we used wasn't
consistent though. Sometimes we would only try "elf kernel", and other
times we would try one of the latter two if the first failed. However,
the loader only ever sets "elf kernel" as the kernel type.

Now, the kmdp is a global, preload_kmdp, and it's initialized using
preload_initkmdp in machdep.c (or machdep_boot.c on arm/64).
preload_initkmdp takes a single boolean argument that tells us whether
not finding the kmdp is fatal or not.

Reviewed by: imp, kib
Pull Request: https://github.com/freebsd/freebsd-src/pull/1394

show more ...

kern linker: Don't invoke dtors without having invoked ctors

I have a kernel module which fails to load because of an unrecognized
relocation type. link_elf_load_file() fails before the module's ct

kern linker: Don't invoke dtors without having invoked ctors

I have a kernel module which fails to load because of an unrecognized
relocation type. link_elf_load_file() fails before the module's ctors
are invoked and it calls linker_file_unload(), which causes the module's
dtors to be executed, resulting in a kernel panic.

Add a flag to the linker file to ensure that dtors are not invoked if
unloading due to an error prior to ctors being invoked.

At the moment I only implemented this for link_elf_obj.c since
link_elf.c doesn't invoke dtors, but I refactored link_elf.c to make
them more similar.

Fixes: 9e575fadf491 ("link_elf_obj: Invoke fini callbacks")
Reviewed by: zlei, kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D44559

show more ...

Fix style nits in kern_linker.c

Reported by: jrtc27
Fixes: c21bc6f3c242 ("ddb: Add CTF-based pretty printing")
Approved by: markj (mentor)

ddb: Add CTF-based pretty printing

Add basic CTF support and a CTF-powered pretty-printer to ddb.

The db_ctf.* files expose a basic interface for fetching type
data for ELF symbols, interacting wit

ddb: Add CTF-based pretty printing

Add basic CTF support and a CTF-powered pretty-printer to ddb.

The db_ctf.* files expose a basic interface for fetching type
data for ELF symbols, interacting with the CTF string table,
and translating type identifiers to type data.

The db_pprint.c file uses those interfaces to implement
a pretty-printer for all kernel ELF symbols.
The pretty-printer works with symbol names and arbitrary addresses:
pprint struct thread 0xffffffff8194ad90

Pretty-printing currently only works after the root filesystem
gets mounted because the CTF info is not available during
early boot.

Differential Revision: https://reviews.freebsd.org/D37899
Approved by: markj (mentor)

show more ...

sys/kern: Remove a few sys/cdefs.h around some elf things

These sys/cdefs.h are no longer needed. Remove them in advance of
other work in this area.

Sponsored by: Netflix

sysctl: Update 'master' copy of vnet SYSCTLs on kernel environment variables change

Complete phase three of 3da1cf1e88f8.

With commit 110113bc086f, vnet sysctl variables can be loader tunable
but t

sysctl: Update 'master' copy of vnet SYSCTLs on kernel environment variables change

Complete phase three of 3da1cf1e88f8.

With commit 110113bc086f, vnet sysctl variables can be loader tunable
but the feature is limited. When the kernel modules have been initialized,
any changes (e.g. via kenv) to kernel environment variable will not affect
subsequently created VNETs.

This change relexes the limitation by listening on kernel environment
variable's set / unset events, and then update the 'master' copy of vnet
SYSCTL or restore it to its initial value.

With this change, TUNABLE_XXX_FETCH can be greately eliminated for vnet
loader tunables.

Reviewed by: glebius
Fixes: 110113bc086f sysctl(9): Enable vnet sysctl variables to be loader tunable
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D41825

show more ...

kernel linker: Eliminate unneeded vnet propagation

The module preload happens before vnet0 creation, at this moment the vnet
list is empty thus invoking vnet_data_copy() during preload is a noop.

W

kernel linker: Eliminate unneeded vnet propagation

The module preload happens before vnet0 creation, at this moment the vnet
list is empty thus invoking vnet_data_copy() during preload is a noop.

With recent change 110113bc086f, for dynamic module load, aka via kldload,
linker will do vnet propagation right after registering sysctls which
happens after module load, then previous propagation (during module load)
is redundant.

No functional change intended.

MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D39852

show more ...

sysctl(9): Enable vnet sysctl variables to be loader tunable

Complete phase two of 3da1cf1e88f8.

In 3da1cf1e88f8, the meaning of the flag CTLFLAG_TUN is extended to
automatically check if there is

sysctl(9): Enable vnet sysctl variables to be loader tunable

Complete phase two of 3da1cf1e88f8.

In 3da1cf1e88f8, the meaning of the flag CTLFLAG_TUN is extended to
automatically check if there is a kernel environment variable which
shall initialize the SYSCTL during early boot. It works for all SYSCTL
types both statically and dynamically created ones, except for the
SYSCTLs which belong to VNETs.

This change extends the meaning further, to allow it also works for
the SYSCTLs which belong to VNETs. A typical usage is
```
VNET_DEFINE_STATIC(int, foo) = 0;
SYSCTL_INT(_net, OID_AUTO, foo, CTLFLAG_RWTUN | CTLFLAG_VNET,
&VNET_NAME(foo), 0, "Description of the foo loader tunable");
```

Note that the implementation has a limitation. It behaves the same way
as that of non-vnet loader tunables. That is, after the kernel or modules
being initialized, any changes (e.g. via kenv) to kernel environment
variable will not affect the corresponding vnet variable of subsequently
created VNETs. To overcome it, we can use TUNABLE_XXX_FETCH to fetch
the kernel environment variable into those vnet variables during vnet
constructing.

This change will fix the following SYSCTLs those belong to VNETs and
have CTLFLAG_TUN flag:
```
net.add_addr_allfibs
net.bpf.optimize_writers
net.inet.tcp.fastopen.ccache_buckets
net.link.bridge.inherit_mac
net.link.bridge.ipfw_arp
net.link.bridge.log_stp
net.link.bridge.pfil_bridge
net.link.bridge.pfil_local_phys
net.link.bridge.pfil_member
net.link.bridge.pfil_onlyip
net.link.lagg.default_use_flowid
net.link.lagg.default_use_numa
net.link.lagg.default_flowid_shift
net.link.lagg.lacp.debug
net.link.lagg.lacp.default_strict_mode
```

Although the following vnet SYSCTLs have CTLFLAG_TUN flag, theirs
values are re-fetched via TUNABLE_XXX_FETCH, thus are not affected
by this change.
```
net.inet.ip.reass_hashsize
net.inet.tcp.hostcache.cachelimit
net.inet.tcp.hostcache.hashsize
net.inet.tcp.hostcache.bucketlimit
net.inet.tcp.syncache.bucketlimit
net.inet.tcp.syncache.cachelimit
net.inet.tcp.syncache.hashsize
net.key.spdcache.maxentries
net.key.spdcache.threshold
```

In memoriam: hselasky
Discussed with: hselasky, glebius
Fixes: 3da1cf1e88f8 Extend the meaning of the CTLFLAG_TUN flag ...
MFC after: 2 weeks
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D39638

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Revert "Revert "tslog: Annotate some early boot functions""

Now that <sys/tslog.h> is wrapped in #ifdef _KERNEL, it's safe to have
tslog annotations in files which might be built from userland (i.e.

Revert "Revert "tslog: Annotate some early boot functions""

Now that <sys/tslog.h> is wrapped in #ifdef _KERNEL, it's safe to have
tslog annotations in files which might be built from userland (i.e. in
subr_boot.c, which is built as part of the boot loader).

This reverts commit 59588a546f55523d6fd37ab42eb08b719311d7d6.

show more ...

Revert "tslog: Annotate some early boot functions"

The change to subr_boot.c broke the libsa build because the TSLOG
macros have their own definitions for the boot loader -- I didn't
realize that th

Revert "tslog: Annotate some early boot functions"

The change to subr_boot.c broke the libsa build because the TSLOG
macros have their own definitions for the boot loader -- I didn't
realize that the loader code used subr_boot.c.

I'm currently testing a fix and I'll revert this revert once I'm
satisfied that everything works, but I don't want to leave the
tree broken for too long.

This reverts commit 469cfa3c30ee7a5ddeb597d0a8c3e7cac909b27a.

show more ...

tslog: Annotate some early boot functions

Booting an amd64 kernel on Firecracker with 1 CPU and 128 MB of RAM,
hammer_time takes roughly 2740 us:
* 55 us in xen_pvh_parse_preload_data
* 20 us in b

tslog: Annotate some early boot functions

Booting an amd64 kernel on Firecracker with 1 CPU and 128 MB of RAM,
hammer_time takes roughly 2740 us:
* 55 us in xen_pvh_parse_preload_data
* 20 us in boot_parse_cmdline_delim
* 20 us in boot_env_to_howto
* 15 us in identify_hypervisor
* 1320 us in link_elf_reloc
* 1310 us in relocate_file1 handling ef->rela
* 25 us in init_param1
* 30 us in dpcpu_init
* 355 us in initializecpu
* 255 us in initializecpu calling load_cr4
* 425 us in getmemsize
* 280 us in pmap_bootstrap
* 205 us in create_pagetables
* 10 us in init_param2
* 25 us in pci_early_quirks
* 60 us in cninit
* 90 us in kdb_init
* 105 us in msgbufinit
* 20 us in fpuinit
* 205 us elsewhere in hammer_time

Some of these are unavoidable (e.g. identify_hypervisor uses CPUID and
load_cr4 loads the CR4 register, both of which trap to the hypervisor)
but others may deserve attention.

Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D40325

show more ...

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

link_elf: fix SysV hash function overflow

Quoting from https://maskray.me/blog/2023-04-12-elf-hash-function:

The System V Application Binary Interface (generic ABI) specifies the
ELF object file fo

link_elf: fix SysV hash function overflow

Quoting from https://maskray.me/blog/2023-04-12-elf-hash-function:

The System V Application Binary Interface (generic ABI) specifies the
ELF object file format. When producing an output executable or shared
object needing a dynamic symbol table (.dynsym), a linker generates a
.hash section with type SHT_HASH to hold a symbol hash table. A DT_HASH
tag is produced to hold the address of .hash.

The function is supposed to return a value no larger than 0x0fffffff.
Unfortunately, there is a bug. When unsigned long consists of more than
32 bits, the return value may be larger than UINT32_MAX. For instance,
elf_hash((const unsigned char *)"\xff\x0f\x0f\x0f\x0f\x0f\x12") returns
0x100000002, which is clearly unintended, as the function should behave
the same way regardless of whether long represents a 32-bit integer or
a 64-bit integer.

Reviewed by: kib, Fangrui Song
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39517

show more ...

Retire broken GPROF support from the kernel

The option is not even recognized and with that patched it does not
compile. Even if it did work, it would be prohibitively expensive to
use.

Interested

Retire broken GPROF support from the kernel

The option is not even recognized and with that patched it does not
compile. Even if it did work, it would be prohibitively expensive to
use.

Interested parties can use pmcstat or dtrace instead.

show more ...

vfs: NDFREE(&nd, NDF_ONLY_PNBUF) -> NDFREE_PNBUF(&nd)

Kernel linkers: add emergency sysctl to restore old behavior

allowing linking to static symbols from other files. Default the new
settings to true, delaying the change of the kernel linker behavior

Kernel linkers: add emergency sysctl to restore old behavior

allowing linking to static symbols from other files. Default the new
settings to true, delaying the change of the kernel linker behavior
for other day.

Suggested by: emaste
PR: 207898
Reviewed by: emaste, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32878

show more ...

kernel linker: do not read debug symbol tables for non-debug symbols

In particular, this prevents resolving locals from other files.
To access debug symbol tables, add LINKER_LOOKUP_DEBUG_SYMBOL and

kernel linker: do not read debug symbol tables for non-debug symbols

In particular, this prevents resolving locals from other files.
To access debug symbol tables, add LINKER_LOOKUP_DEBUG_SYMBOL and
LINKER_DEBUG_SYMBOL_VALUES kobj methods, which are allowed to use
any types of present symbols in all tables.

PR: 207898
Reviewed by: emaste, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32878

show more ...

vfs: remove the unused thread argument from NDINIT*

See b4a58fbf640409a1 ("vfs: remove cn_thread")

Bump __FreeBSD_version to 1400043.

Kernel linkers: some style

Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Differential revision: https://reviews.freebsd.org/D32878

Add pmap_change_prot on arm64

Support changing the protection of preloaded kernel modules by
implementing pmap_change_prot on arm64 and calling it from
preload_protect.

Reviewed by: alc (previous v

Add pmap_change_prot on arm64

Support changing the protection of preloaded kernel modules by
implementing pmap_change_prot on arm64 and calling it from
preload_protect.

Reviewed by: alc (previous version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32026

show more ...

vm_map_protect: allow to set prot and max_prot in one go.

This prevents a situation where other thread modifies map entries
permissions between setting max_prot, then relocking, then setting prot,
c

vm_map_protect: allow to set prot and max_prot in one go.

This prevents a situation where other thread modifies map entries
permissions between setting max_prot, then relocking, then setting prot,
confusing the operation outcome. E.g. you can get an error that is not
possible if operation is performed atomic.

Also enable setting rwx for max_prot even if map does not allow to set
effective rwx protection.

Reviewed by: brooks, markj (previous version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D28117

show more ...

malloc: delegate M_EXEC handling to dedicacted routines

It is almost never needed and adds an avoidable branch.

While here do minior clean ups in preparation for larger changes.

Reviewed by: markj

malloc: delegate M_EXEC handling to dedicacted routines

It is almost never needed and adds an avoidable branch.

While here do minior clean ups in preparation for larger changes.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D27019

show more ...