ddb: Add CTF-based pretty printing

Add basic CTF support and a CTF-powered pretty-printer to ddb.

The db_ctf.* files expose a basic interface for fetching type
data for ELF symbols, interacting wit

ddb: Add CTF-based pretty printing

Add basic CTF support and a CTF-powered pretty-printer to ddb.

The db_ctf.* files expose a basic interface for fetching type
data for ELF symbols, interacting with the CTF string table,
and translating type identifiers to type data.

The db_pprint.c file uses those interfaces to implement
a pretty-printer for all kernel ELF symbols.
The pretty-printer works with symbol names and arbitrary addresses:
pprint struct thread 0xffffffff8194ad90

Pretty-printing currently only works after the root filesystem
gets mounted because the CTF info is not available during
early boot.

Differential Revision: https://reviews.freebsd.org/D37899
Approved by: markj (mentor)

show more ...

sysctl(9): Enable vnet sysctl variables to be loader tunable

Complete phase two of 3da1cf1e88f8.

In 3da1cf1e88f8, the meaning of the flag CTLFLAG_TUN is extended to
automatically check if there is

sysctl(9): Enable vnet sysctl variables to be loader tunable

Complete phase two of 3da1cf1e88f8.

In 3da1cf1e88f8, the meaning of the flag CTLFLAG_TUN is extended to
automatically check if there is a kernel environment variable which
shall initialize the SYSCTL during early boot. It works for all SYSCTL
types both statically and dynamically created ones, except for the
SYSCTLs which belong to VNETs.

This change extends the meaning further, to allow it also works for
the SYSCTLs which belong to VNETs. A typical usage is
```
VNET_DEFINE_STATIC(int, foo) = 0;
SYSCTL_INT(_net, OID_AUTO, foo, CTLFLAG_RWTUN | CTLFLAG_VNET,
&VNET_NAME(foo), 0, "Description of the foo loader tunable");
```

Note that the implementation has a limitation. It behaves the same way
as that of non-vnet loader tunables. That is, after the kernel or modules
being initialized, any changes (e.g. via kenv) to kernel environment
variable will not affect the corresponding vnet variable of subsequently
created VNETs. To overcome it, we can use TUNABLE_XXX_FETCH to fetch
the kernel environment variable into those vnet variables during vnet
constructing.

This change will fix the following SYSCTLs those belong to VNETs and
have CTLFLAG_TUN flag:
```
net.add_addr_allfibs
net.bpf.optimize_writers
net.inet.tcp.fastopen.ccache_buckets
net.link.bridge.inherit_mac
net.link.bridge.ipfw_arp
net.link.bridge.log_stp
net.link.bridge.pfil_bridge
net.link.bridge.pfil_local_phys
net.link.bridge.pfil_member
net.link.bridge.pfil_onlyip
net.link.lagg.default_use_flowid
net.link.lagg.default_use_numa
net.link.lagg.default_flowid_shift
net.link.lagg.lacp.debug
net.link.lagg.lacp.default_strict_mode
```

Although the following vnet SYSCTLs have CTLFLAG_TUN flag, theirs
values are re-fetched via TUNABLE_XXX_FETCH, thus are not affected
by this change.
```
net.inet.ip.reass_hashsize
net.inet.tcp.hostcache.cachelimit
net.inet.tcp.hostcache.hashsize
net.inet.tcp.hostcache.bucketlimit
net.inet.tcp.syncache.bucketlimit
net.inet.tcp.syncache.cachelimit
net.inet.tcp.syncache.hashsize
net.key.spdcache.maxentries
net.key.spdcache.threshold
```

In memoriam: hselasky
Discussed with: hselasky, glebius
Fixes: 3da1cf1e88f8 Extend the meaning of the CTLFLAG_TUN flag ...
MFC after: 2 weeks
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D39638

show more ...

sys: Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

kernel linker: do not read debug symbol tables for non-debug symbols

In particular, this prevents resolving locals from other files.
To access debug symbol tables, add LINKER_LOOKUP_DEBUG_SYMBOL and

kernel linker: do not read debug symbol tables for non-debug symbols

In particular, this prevents resolving locals from other files.
To access debug symbol tables, add LINKER_LOOKUP_DEBUG_SYMBOL and
LINKER_DEBUG_SYMBOL_VALUES kobj methods, which are allowed to use
any types of present symbols in all tables.

PR: 207898
Reviewed by: emaste, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32878

show more ...

sys/kern: spelling fixes in comments.

No functional change.

Merge from head

- Merge from HEAD

strict kobj signatures: linker_if fixes

in symtab_get method symtab parameter is made constant as this reflects
actual intention and usage of the method

Reviewed by: imp, current@
Approved by: jhb

strict kobj signatures: linker_if fixes

in symtab_get method symtab parameter is made constant as this reflects
actual intention and usage of the method

Reviewed by: imp, current@
Approved by: jhb (mentor)

show more ...

Add the ksyms(4) pseudo driver. The ksyms driver allows a process to
get a quick snapshot of the kernel's symbol table including the symbols
from any loaded modules (the symbols are all merged into

Add the ksyms(4) pseudo driver. The ksyms driver allows a process to
get a quick snapshot of the kernel's symbol table including the symbols
from any loaded modules (the symbols are all merged into one symbol
table). Unlike like other implementations, this ksyms driver maps
memory in the process memory space to store the snapshot at the time
/dev/ksyms is opened. It also checks to see if the process has already
a snapshot open and won't allow it to open /dev/ksyms it again until it
closes first. This prevents kernel and process memory from being
exhausted. Note that /dev/ksyms is used by the lockstat(1) command.

Reviewed by: gallatin kib (freebsd-arch)
Approved by: gnn (mentor)

show more ...

Add the ctf_get method.

Add a function to list symbols in a file and their values at the
same time rather than having to list the symbols and then go back
and look each one up by name.

/* -> /*- for copyright notices, minor format tweaks as necessary

The method link_preload_finish is not static.

Add the sysctl "kern.function_list", which currently exports all
function symbols in the kernel in a list of C strings, with an extra
nul-termination at the end.

This sysctl requires addition of a n

Add the sysctl "kern.function_list", which currently exports all
function symbols in the kernel in a list of C strings, with an extra
nul-termination at the end.

This sysctl requires addition of a new linker operation. Now,
linker_file_t's need to respond to "each_function_name" to export
their function symbols.

Note that the sysctl doesn't currently allow distinguishing multiple
symbols with the same name from different modules, but could quite
easily without a change to the linker operation. This will be a nicety
to have when it can be used.

Obtained from: NAI Labs CBOSS project
Funded by: DARPA

show more ...

With this commit, I hereby pronounce gensetdefs past its use-by date.

Replace the a.out emulation of 'struct linker_set' with something
a little more flexible. <sys/linker_set.h> now provides macro

With this commit, I hereby pronounce gensetdefs past its use-by date.

Replace the a.out emulation of 'struct linker_set' with something
a little more flexible. <sys/linker_set.h> now provides macros for
accessing elements and completely hides the implementation.

The linker_set.h macros have been on the back burner in various
forms since 1998 and has ideas and code from Mike Smith (SET_FOREACH()),
John Polstra (ELF clue) and myself (cleaned up API and the conversion
of the rest of the kernel to use it).

The macros declare a strongly typed set. They return elements with the
type that you declare the set with, rather than a generic void *.

For ELF, we use the magic ld symbols (__start_<setname> and
__stop_<setname>). Thanks to Richard Henderson <rth@redhat.com> for the
trick about how to force ld to provide them for kld's.

For a.out, we use the old linker_set struct.

NOTE: the item lists are no longer null terminated. This is why
the code impact is high in certain areas.

The runtime linker has a new method to find the linker set
boundaries depending on which backend format is in use.

linker sets are still module/kld unfriendly and should never be used
for anything that may be modular one day.

Reviewed by: eivind

show more ...

First round implementation of a fine grain enhanced module to module
version dependency system. This isn't quite finished, but it is at a
useful stage to do a functional checkpoint.

Highlights:
- v

First round implementation of a fine grain enhanced module to module
version dependency system. This isn't quite finished, but it is at a
useful stage to do a functional checkpoint.

Highlights:
- version and dependency metadata is gathered via linker sets, so things
are handled the same for static kernels and code built to live in a kld.
- The dependencies are at module level (versus at file level).
- Dependencies determine kld symbol search order - this means that you
cannot link against symbols in another file unless you depend on it. This
is so that you cannot accidently unload the target out from underneath
the ones referencing it.
- It is flexible enough that we can put tags in #include files and macros
so that we can get decent hooks for enforcing recompiles on incompatable
ABI changes. eg: if we change struct proc, we could force a recompile
for all kld's that reference the proc struct.
- Tangled dependency references at boot time are sorted. Files are
relocated once all their dependencies are already relocated.

Caveats:
- Loader support is incomplete, but has been worked on seperately.
- Actual enforcement of the version number tags is not active yet - just
the module dependencies are live. The actual structure of versioning
hasn't been agreed on yet. (eg: major.minor, or whatever)
- There is some backwards compatability for old modules without metadata
but I'm not sure how good it is.

This is based on work originally done by Boris Popov (bp@freebsd.org),
but I'm not sure he'd recognize much of it now. Don't blame him. :-)
Also, ideas have been borrowed from Mike Smith.

show more ...

* Rewrite to use kobj(9) instead of hard-coded function tables.
* Report link errors to stdout with uprintf() so that the user can see
what went wrong (PR kern/9214).
* Add support code to allow mo

* Rewrite to use kobj(9) instead of hard-coded function tables.
* Report link errors to stdout with uprintf() so that the user can see
what went wrong (PR kern/9214).
* Add support code to allow module symbols to be loaded into GDB using
the debugger's "sharedlibrary" command.

show more ...

- Merge from HEAD

strict kobj signatures: linker_if fixes

in symtab_get method symtab parameter is made constant as this reflects
actual intention and usage of the method

Reviewed by: imp, current@
Approved by: jhb

strict kobj signatures: linker_if fixes

in symtab_get method symtab parameter is made constant as this reflects
actual intention and usage of the method

Reviewed by: imp, current@
Approved by: jhb (mentor)

show more ...

Add the ksyms(4) pseudo driver. The ksyms driver allows a process to
get a quick snapshot of the kernel's symbol table including the symbols
from any loaded modules (the symbols are all merged into

Add the ksyms(4) pseudo driver. The ksyms driver allows a process to
get a quick snapshot of the kernel's symbol table including the symbols
from any loaded modules (the symbols are all merged into one symbol
table). Unlike like other implementations, this ksyms driver maps
memory in the process memory space to store the snapshot at the time
/dev/ksyms is opened. It also checks to see if the process has already
a snapshot open and won't allow it to open /dev/ksyms it again until it
closes first. This prevents kernel and process memory from being
exhausted. Note that /dev/ksyms is used by the lockstat(1) command.

Reviewed by: gallatin kib (freebsd-arch)
Approved by: gnn (mentor)

show more ...

Add the ctf_get method.

Add a function to list symbols in a file and their values at the
same time rather than having to list the symbols and then go back
and look each one up by name.

/* -> /*- for copyright notices, minor format tweaks as necessary

The method link_preload_finish is not static.

Add the sysctl "kern.function_list", which currently exports all
function symbols in the kernel in a list of C strings, with an extra
nul-termination at the end.

This sysctl requires addition of a n

Add the sysctl "kern.function_list", which currently exports all
function symbols in the kernel in a list of C strings, with an extra
nul-termination at the end.

This sysctl requires addition of a new linker operation. Now,
linker_file_t's need to respond to "each_function_name" to export
their function symbols.

Note that the sysctl doesn't currently allow distinguishing multiple
symbols with the same name from different modules, but could quite
easily without a change to the linker operation. This will be a nicety
to have when it can be used.

Obtained from: NAI Labs CBOSS project
Funded by: DARPA

show more ...