| d8f26717 | 01-Jul-2025 |
Giuliano Procida <gprocida@google.com> |
gendwarfksyms: order -T symtypes output by name
When writing symtypes information, we iterate through the entire hash
table containing type expansions. The key order varies unpredictably
as new entr
gendwarfksyms: order -T symtypes output by name
When writing symtypes information, we iterate through the entire hash
table containing type expansions. The key order varies unpredictably
as new entries are added, making it harder to compare symtypes between
builds.
Resolve this by sorting the type expansions by name before output.
Signed-off-by: Giuliano Procida <gprocida@google.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| c9083467 | 08-May-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Add a kABI rule to override type strings
In rare situations where distributions must make significant
changes to otherwise opaque data structures that have
inadvertently been included
gendwarfksyms: Add a kABI rule to override type strings
In rare situations where distributions must make significant
changes to otherwise opaque data structures that have
inadvertently been included in the published ABI, keeping
symbol versions stable using the existing kABI macros can
become tedious.
For example, Android decided to switch to a newer io_uring
implementation in the 5.10 GKI kernel "to resolve a huge number
of potential, and known, problems with the codebase," requiring
"horrible hacks" with genksyms:
"A number of the io_uring structures get used in other core
kernel structures, only as "opaque" pointers, so there is
not any real ABI breakage. But, due to the visibility of
the structures going away, the CRC values of many scheduler
variables and functions were changed."
-- https://r.android.com/2425293
While these specific changes probably could have been hidden
from gendwarfksyms using the existing kABI macros, this may not
always be the case.
Add a last resort kABI rule that allows distribution
maintainers to fully override a type string for a symbol or a
type. Also add a more informative error message in case we find
a non-existent type references when calculating versions.
Suggested-by: Giuliano Procida <gprocida@google.com>
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| db59d74e | 08-May-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Add a kABI rule to override byte_size attributes
A data structure can be partially opaque to modules if its
allocation is handled by the core kernel, and modules only need
to access s
gendwarfksyms: Add a kABI rule to override byte_size attributes
A data structure can be partially opaque to modules if its
allocation is handled by the core kernel, and modules only need
to access some of its members. In this situation, it's possible
to append new members to the structure without breaking the ABI,
as long as the layout for the original members remains unchanged.
For example, consider the following struct:
struct s {
unsigned long a;
void *p;
};
gendwarfksyms --stable --dump-dies produces the following type
expansion:
variable structure_type s {
member base_type long unsigned int byte_size(8) encoding(7) a
data_member_location(0) ,
member pointer_type {
base_type void
} byte_size(8) p data_member_location(8)
} byte_size(16)
To append new members, we can use the KABI_IGNORE() macro to
hide them from gendwarfksyms --stable:
struct s {
/* old members with unchanged layout */
unsigned long a;
void *p;
/* new members not accessed by modules */
KABI_IGNORE(0, unsigned long n);
};
However, we can't hide the fact that adding new members changes
the struct size, as seen in the updated type string:
variable structure_type s {
member base_type long unsigned int byte_size(8) encoding(7) a
data_member_location(0) ,
member pointer_type {
base_type void
} byte_size(8) p data_member_location(8)
} byte_size(24)
In order to support this use case, add a kABI rule that makes it
possible to override the byte_size attribute for types:
/*
* struct s allocation is handled by the kernel, so
* appending new members without changing the original
* layout won't break the ABI.
*/
KABI_BYTE_SIZE(s, 16);
This results in a type string that's unchanged from the original
and therefore, won't change versions for symbols that reference
the changed structure.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| fa624569 | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Add support for symbol type pointers
The compiler may choose not to emit type information in DWARF for
external symbols. Clang, for example, does this for symbols not
defined in the c
gendwarfksyms: Add support for symbol type pointers
The compiler may choose not to emit type information in DWARF for
external symbols. Clang, for example, does this for symbols not
defined in the current TU.
To provide a way to work around this issue, add support for
__gendwarfksyms_ptr_<symbol> pointers that force the compiler to emit
the necessary type information in DWARF also for the missing symbols.
Example usage:
#define GENDWARFKSYMS_PTR(sym) \
static typeof(sym) *__gendwarfksyms_ptr_##sym __used \
__section(".discard.gendwarfksyms") = &sym;
extern int external_symbol(void);
GENDWARFKSYMS_PTR(external_symbol);
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| a9369418 | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Add support for reserved and ignored fields
Distributions that want to maintain a stable kABI need the ability
to make ABI compatible changes to kernel data structures without
affecti
gendwarfksyms: Add support for reserved and ignored fields
Distributions that want to maintain a stable kABI need the ability
to make ABI compatible changes to kernel data structures without
affecting symbol versions, either because of LTS updates or backports.
With genksyms, developers would typically hide these changes from
version calculation with #ifndef __GENKSYMS__, which would result
in the symbol version not changing even though the actual type has
changed. When we process precompiled object files, this isn't an
option.
Change union processing to recognize field name prefixes that allow
the user to ignore the union completely during symbol versioning with
a __kabi_ignored prefix in a field name, or to replace the type of a
placeholder field using a __kabi_reserved field name prefix.
For example, assume we want to add a new field to an existing
alignment hole in a data structure, and ignore the new field when
calculating symbol versions:
struct struct1 {
int a;
/* a 4-byte alignment hole */
unsigned long b;
};
To add `int n` to the alignment hole, we can add a union that includes
a __kabi_ignored field that causes gendwarfksyms to ignore the entire
union:
struct struct1 {
int a;
union {
char __kabi_ignored_0;
int n;
};
unsigned long b;
};
With --stable, both structs produce the same symbol version.
Alternatively, when a distribution expects future modification to a
data structure, they can explicitly add reserved fields:
struct struct2 {
long a;
long __kabi_reserved_0; /* reserved for future use */
};
To take the field into use, we can again replace it with a union, with
one of the fields keeping the __kabi_reserved name prefix to indicate
the original type:
struct struct2 {
long a;
union {
long __kabi_reserved_0;
struct {
int b;
int v;
};
};
Here gendwarfksyms --stable replaces the union with the type of the
placeholder field when calculating versions.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| 936cf61c | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Add support for kABI rules
Distributions that want to maintain a stable kABI need the ability
to make ABI compatible changes to kernel without affecting symbol
versions, either becaus
gendwarfksyms: Add support for kABI rules
Distributions that want to maintain a stable kABI need the ability
to make ABI compatible changes to kernel without affecting symbol
versions, either because of LTS updates or backports.
With genksyms, developers would typically hide these changes from
version calculation with #ifndef __GENKSYMS__, which would result
in the symbol version not changing even though the actual type has
changed. When we process precompiled object files, this isn't an
option.
To support this use case, add a --stable command line flag that
gates kABI stability features that are not needed in mainline
kernels, but can be useful for distributions, and add support for
kABI rules, which can be used to restrict gendwarfksyms output.
The rules are specified as a set of null-terminated strings stored
in the .discard.gendwarfksyms.kabi_rules section. Each rule consists
of four strings as follows:
"version\0type\0target\0value"
The version string ensures the structure can be changed in a
backwards compatible way. The type string indicates the type of the
rule, and target and value strings contain rule-specific data.
Initially support two simple rules:
1. Declaration-only types
A type declaration can change into a full definition when
additional includes are pulled in to the TU, which changes the
versions of any symbol that references the type. Add support
for defining declaration-only types whose definition is not
expanded during versioning.
2. Ignored enumerators
It's possible to add new enum fields without changing the ABI,
but as the fields are included in symbol versioning, this would
change the versions. Add support for ignoring specific fields.
3. Overridden enumerator values
Add support for overriding enumerator values when calculating
versions. This may be needed when the last field of the enum
is used as a sentinel and new fields must be added before it.
Add examples for using the rules under the examples/ directory.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| 71378888 | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Add symbol versioning
Calculate symbol versions from the fully expanded type strings in
type_map, and output the versions in a genksyms-compatible format.
Signed-off-by: Sami Tolvane
gendwarfksyms: Add symbol versioning
Calculate symbol versions from the fully expanded type strings in
type_map, and output the versions in a genksyms-compatible format.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| ab443998 | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Add symtypes output
Add support for producing genksyms-style symtypes files. Process
die_map to find the longest expansions for each type, and use symtypes
references in type definiti
gendwarfksyms: Add symtypes output
Add support for producing genksyms-style symtypes files. Process
die_map to find the longest expansions for each type, and use symtypes
references in type definitions. The basic file format is similar to
genksyms, with two notable exceptions:
1. Type names with spaces (common with Rust) in references are
wrapped in single quotes. E.g.:
s#'core::result::Result<u8, core::num::error::ParseIntError>'
2. The actual type definition is the simple parsed DWARF format we
output with --dump-dies, not the preprocessed C-style format
genksyms produces.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| d2ffdc1c | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Add die_map debugging
Debugging the DWARF processing can be somewhat challenging, so add
more detailed debugging output for die_map operations. Add the
--dump-die-map flag, which adds
gendwarfksyms: Add die_map debugging
Debugging the DWARF processing can be somewhat challenging, so add
more detailed debugging output for die_map operations. Add the
--dump-die-map flag, which adds color coded tags to the output for
die_map changes.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| f936c129 | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Limit structure expansion
Expand each structure type only once per exported symbol. This
is necessary to support self-referential structures, which would
otherwise result in infinite
gendwarfksyms: Limit structure expansion
Expand each structure type only once per exported symbol. This
is necessary to support self-referential structures, which would
otherwise result in infinite recursion, and it's sufficient for
catching ABI changes.
Types defined in .c files are opaque to external users and thus
cannot affect the ABI. Consider type definitions in .c files to
be declarations to prevent opaque types from changing symbol
versions.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| f6bb9245 | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Expand structure types
Recursively expand DWARF structure types, i.e. structs, unions, and
enums. Also include relevant DWARF attributes in type strings to
encode structure layout, fo
gendwarfksyms: Expand structure types
Recursively expand DWARF structure types, i.e. structs, unions, and
enums. Also include relevant DWARF attributes in type strings to
encode structure layout, for example.
Example output with --dump-dies:
subprogram (
formal_parameter structure_type &str {
member pointer_type {
base_type u8 byte_size(1) encoding(7)
} data_ptr data_member_location(0) ,
member base_type usize byte_size(8) encoding(7) length data_member_location(8)
} byte_size(16) alignment(8) msg
)
-> base_type void
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| c772f1d1 | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Expand array_type
Add support for expanding DW_TAG_array_type, and the subrange type
indicating array size.
Example source code:
const char *s[34];
Output with --dump-dies:
va
gendwarfksyms: Expand array_type
Add support for expanding DW_TAG_array_type, and the subrange type
indicating array size.
Example source code:
const char *s[34];
Output with --dump-dies:
variable array_type[34] {
pointer_type {
const_type {
base_type char byte_size(1) encoding(6)
}
} byte_size(8)
}
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| 220a0857 | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Expand subroutine_type
Add support for expanding DW_TAG_subroutine_type and the parameters
in DW_TAG_formal_parameter. Use this to also expand subprograms.
Example output with --dump
gendwarfksyms: Expand subroutine_type
Add support for expanding DW_TAG_subroutine_type and the parameters
in DW_TAG_formal_parameter. Use this to also expand subprograms.
Example output with --dump-dies:
subprogram (
formal_parameter pointer_type {
const_type {
base_type char byte_size(1) encoding(6)
}
}
)
-> base_type unsigned long byte_size(8) encoding(7)
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| 06b8b036 | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Expand type modifiers and typedefs
Add support for expanding DWARF type modifiers, such as pointers,
const values etc., and typedefs. These types all have DW_AT_type
attribute pointin
gendwarfksyms: Expand type modifiers and typedefs
Add support for expanding DWARF type modifiers, such as pointers,
const values etc., and typedefs. These types all have DW_AT_type
attribute pointing to the underlying type, and thus produce similar
output.
Also add linebreaks and indentation to debugging output to make it
more readable.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| 0c1c7627 | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Add a cache for processed DIEs
Basic types in DWARF repeat frequently and traversing the DIEs using
libdw is relatively slow. Add a simple hashtable based cache for the
processed DIEs
gendwarfksyms: Add a cache for processed DIEs
Basic types in DWARF repeat frequently and traversing the DIEs using
libdw is relatively slow. Add a simple hashtable based cache for the
processed DIEs.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| 5b7780e8 | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Expand base_type
Start making gendwarfksyms more useful by adding support for
expanding DW_TAG_base_type types and basic DWARF attributes.
Example:
$ echo loops_per_jiffy | \
gendwarfksyms: Expand base_type
Start making gendwarfksyms more useful by adding support for
expanding DW_TAG_base_type types and basic DWARF attributes.
Example:
$ echo loops_per_jiffy | \
scripts/gendwarfksyms/gendwarfksyms \
--debug --dump-dies vmlinux.o
...
gendwarfksyms: process_symbol: loops_per_jiffy
variable base_type unsigned long byte_size(8) encoding(7)
...
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|
| e982abf4 | 03-Jan-2025 |
Sami Tolvanen <samitolvanen@google.com> |
gendwarfksyms: Add address matching
The compiler may choose not to emit type information in DWARF for all
aliases, but it's possible for each alias to be exported separately.
To ensure we find type
gendwarfksyms: Add address matching
The compiler may choose not to emit type information in DWARF for all
aliases, but it's possible for each alias to be exported separately.
To ensure we find type information for the aliases as well, read
{section, address} tuples from the symbol table and match symbols also
by address.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
show more ...
|