libctf: Handle CTFv3 containers

In general, the patch adds indirection to minimize the amount of code
that needs to know about differences between v2 and v3. Specifically,
some new ctf_get_ctt_* fu

libctf: Handle CTFv3 containers

In general, the patch adds indirection to minimize the amount of code
that needs to know about differences between v2 and v3. Specifically,
some new ctf_get_ctt_* functions are added, and new LCTF_* macros are
added to use the underlying container's version to do the right thing.

CTF containers can have parent/child relationships, wherein a type ID in
one container refers to a type in the parent. It is permitted for the
parent and child to have different versions.

MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34363

show more ...

libctf: Use ctf_type_t instead of struct ctf_type

For consistency with other CTF toolchain code. No functional change
intended.

Fixes: 105fd928b0b5 ("libctf: Improve check for duplicate SOU defini

libctf: Use ctf_type_t instead of struct ctf_type

For consistency with other CTF toolchain code. No functional change
intended.

Fixes: 105fd928b0b5 ("libctf: Improve check for duplicate SOU definitions in ctf_add_type()")
MFC after: 1 week

show more ...

libctf: Improve check for duplicate SOU definitions in ctf_add_type()

When copying a struct or union from one CTF container to another,
ctf_add_type() checks whether it matches an existing type in t

libctf: Improve check for duplicate SOU definitions in ctf_add_type()

When copying a struct or union from one CTF container to another,
ctf_add_type() checks whether it matches an existing type in the
destination container. It does so by looking for a type with the same
name and kind as the new type, and if one exists, it iterates over all
members of the source type and checks whether a member with matching
name and offset exists in the matched destination type. This can
produce false positives, for example because member types are not
compared, but this is not expected to arise in practice. If the match
fails, ctf_add_type() returns an error.

The procedure used for member comparison breaks down in the face of
anonymous struct and union members. ctf_member_iter() visits each
member in the source definition and looks up the corresponding member in
the desination definition by name using ctf_member_info(), but this
function will descend into anonymous members and thus fail to match.
Fix the problem by introducing a custom comparison routine which does
not assume member names are unique. This should also be faster for
types with many members; in the previous scheme, membcmp() would perform
a linear scan of the desination type's members to perform a lookup by
name. The new routine steps through the members of both types in a
single loop.

PR: 258763
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

show more ...

When copying types from one CTF container to another, ensure that we
encode 0-length (i.e. "") structure and union member names as offset 0.
This ensures that we don't confuse other parts of the CTF

When copying types from one CTF container to another, ensure that we
encode 0-length (i.e. "") structure and union member names as offset 0.
This ensures that we don't confuse other parts of the CTF code which
expect this encoding.

This resolves a Dtrace error resolving members of anonymous structs/unions
within the (struct mbuf) type which some users were seeing after r366908.

While here, update the code in ctf_add_generic() to encode 0-length type
names as offset 0.

Reviewed by: markj
MFC after: 2 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D27246

show more ...

When copying types from one CTF container to another, ensure that we
always copy intrinsic data types before copying bitfields which are
based on those types. This ensures the type ordering in the de

When copying types from one CTF container to another, ensure that we
always copy intrinsic data types before copying bitfields which are
based on those types. This ensures the type ordering in the destination
CTF container matches the assumption made elsewhere in the CTF code
that instrinsic data types will always appear before bitfields based on
those types.

This resolves the following error message some users have seen after
r366908:
"/usr/lib/dtrace/ipfw.d", line 121: failed to copy type of 'ip6p':
Conflicting type is already defined

Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D27213

show more ...

Catch up with HEAD (r280229-r284686).

MFH: r282615-r283655

Sponsored by: The FreeBSD Foundation

Merge sync of head

ctf_add_type(): when looking up an integer or floating point type in the
list of pending dynamic type definitions, a match on the type name is not
sufficient - we need to compare the type encodings a

ctf_add_type(): when looking up an integer or floating point type in the
list of pending dynamic type definitions, a match on the type name is not
sufficient - we need to compare the type encodings as well. For example,
bitfields have their own distinct type definitions which share the name of
the underlying integer type, and these types aren't generally
interchangeable.

This bug was causing the following libdtrace error when attempting to trace
the th_flags member of a struct tcphdr:
cg: bad field: off 104 type <32877> bits 539620016

Reported by: rwatson
MFC after: 3 weeks

show more ...

MFH: r278968-r280640

Sponsored by: The FreeBSD Foundation

Merge r278538 through r280226.

Merge from head

Merge ^/head r279759 through r279892.

When copying a type from a source CTF container to a destination container,
ctf_add_type() first performs a by-name lookup of the type in the
destination container. If this lookup returns a forward d

When copying a type from a source CTF container to a destination container,
ctf_add_type() first performs a by-name lookup of the type in the
destination container. If this lookup returns a forward declaration for an
enum, struct, or union, reset dst_type back to CTF_ERR, indicating that the
source type is not in fact present in the destination container. This
ensures that ctf_add_type() will also search the destination container's
dynamic type list for the source type.

Without this change, a pair of mutually recursive struct definitions could
cause infinite recursion in ctf_add_type() if the destination container
only contained forward declarations for the struct types: ctf_add_type()
recursively calls itself on each struct member's type, and the forward
declarations meant that the dynamic type list search would be skipped.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

show more ...

CTF containers use the ctf_dtoldid field as a threshold type index which
indicates the range of type indices which have been committed to the
container by ctf_update(). However, the top bit of the dt

CTF containers use the ctf_dtoldid field as a threshold type index which
indicates the range of type indices which have been committed to the
container by ctf_update(). However, the top bit of the dtd_type field is
not part of the type index; rather, it is a flag used to indicate that the
corresponding CTF container is a parent. This is why the maximum CTF type
index is 2^15 - 1 rather than 2^16 - 1. Therefore, this flag must be masked
off (using the CTF_TYPE_TO_INDEX macro) when comparing a type index with the
ctf_dtoldid field of a container.

This bug was causing libctf to erroneously free committed type definitions
in ctf_discard(). libdtrace holds some references to such types, resulting
in a use-after-free.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

show more ...

ctf_discard(): fetch the next list element before restarting the loop. If
we end up skipping a dynamic type because it has already been committed to
the container, we would previously either set the

ctf_discard(): fetch the next list element before restarting the loop. If
we end up skipping a dynamic type because it has already been committed to
the container, we would previously either set the loop variable to an
uninitialized local variable, or set it to itself, resulting in an infinite
loop.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

show more ...

Merge head

IFC @264767

MFC @ r259205 in preparation for some SVM updates. (for real this time)

Merge head r232040 through r258006.

Quiesce warning assigning to void * from const ctf_header_t * by explicity casting
to void * before assignment.

Submitted as Illumos issue 4287

Merge head r233826 through r256722.

IFC @256277

Approved by: ken (mentor)

IFC @ r255692

Comment out IA32_MISC_ENABLE MSR access - this doesn't exist on AMD.
Need to sort out how arch-specific MSRs will be handled.

Merge from head