tools/scripts/check_rtime.1onbld

.\" Copyright (c) 2001, 2010, Oracle and/or its affiliates. All rights reserved.
.\"
.\" CDDL HEADER START
.\"
.\" The contents of this file are subject to the terms of the
.\" Common Development and Distribution License (the "License").
.\" You may not use this file except in compliance with the License.
.\"
.\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
.\" or http://www.opensolaris.org/os/licensing.
.\" See the License for the specific language governing permissions
.\" and limitations under the License.
.\"
.\" When distributing Covered Code, include this CDDL HEADER in each
.\" file and include the License file at usr/src/OPENSOLARIS.LICENSE.
.\" If applicable, add the following below this CDDL HEADER, with the
.\" fields enclosed by brackets "[]" replaced with your own identifying
.\" information: Portions Copyright [yyyy] [name of copyright owner]
.\"
.\" CDDL HEADER END
.\"
.Dd February 3, 2020
.Dt CHECK_RTIME 1ONBLD
.Os
.Sh NAME
.Nm check_rtime
.Nd check ELF runtime attributes
.Sh SYNOPSIS
.Nm check_rtime
.Op Fl imosv
.Op Fl D Ar depfile | Fl d depdir
.Op Fl E Ar errfile
.Op Fl e Ar exfile
.Op Fl f Ar listfile
.Op Fl I Ar infofile
.Op Fl w Ar outdir
.Ar file | dir ...
.Sh DESCRIPTION
.Nm check_rtime
attempts to check a number of ELF runtime attributes
for consistency with common build rules.
These checks involve running
.Xr ldd 1
and
.Xr elfdump 1
against a family of dynamic objects.
A dynamic object can be defined explicitly as a
.Ar file
or multiple dynamic objects can be located under the directory
.Ar dir .
.Pp
.Nm check_rtime
is typically called from
.Xr nightly 1ONBLD
when the
.Fl r
option is in effect.
In this case the dynamic objects under
the associated
.Em proto area
.Pq Ev $ROOT
are checked.
.Nm check_rtime
can also be run standalone against any set of dynamic objects.
.Pp
.Nm check_rtime
uses
.Xr  ldd 1
to verify dependencies.
This implies that by default any object inspected will bind to its dependencies
as they are found in the
.Em underlying system .
Use of the
.Fl D ,
.Fl d
option, or the existence of the environment variables
.Ev $CODEMGR_WS
or
.Ev $ROOT
instruct
.Nm check_rtime
to establish an alternative dependency mapping using
runtime configuration files generated with
.Xr crle 1 .
.Pp
.Nm check_rtime
uses
.Xr ldd 1
to completely relocate any dynamic object and thus detect missing
dependencies, unsatisfied symbol relocations, unused and unreferenced
dependencies.
These checks are carried out for the following reasons:
.Bl -bullet
.It
An object that cannot find its dependencies may fail to load
at runtime.
This error condition often goes unnoticed because the existing use of the
object is as a dependency itself, and the objects' dependencies are already
satisfied by the caller.
However, if the object itself is unable to satisfy its dependencies, its use
in new environments may be compromised.
.Pp
A missing or erroneous
.Em runpath
is the typical reason why an object can not locate its dependencies.
Use of the link-editors
.Fl zdefs
option when building a shared object ensures required dependencies are
established.
This flag is inherited from
.Dv $(DYNFLAGS)
in
.Pa lib/Makefile.lib .
Missing dependencies are displayed as:
.Pp
.Dl foo: bar.so.1 => (file not found)  <no -zdefs?>
.It
Unsatisfied symbol relocations indicate that some thread of
execution through the object will fail when it is unable to
locate a referenced symbol.
.Pp
A missing, or mismatched version of a dependency is the typical
reason for unsatisfied symbol relocations (see missing dependency
discussion above). Unsatisfied symbol relocations are displayed as:
.Pp
.Dl foo: symbol not found: bar  <no -zdefs?>
.Pp
Note: Shared objects can make reference to symbol definitions
that are expected to be defined by the caller.
To indicate that such symbols are not undefined in the usual sense, you must
specify these symbols in a
.Em mapfile ,
using the
.Va EXTERN
or
.Va PARENT
symbol attributes.
Without these symbol attributes,
.Xr ldd 1
is unable to determine the symbols special nature, and
.Nm check_rtime
will report these symbols as undefined.
.It
Unused dependencies are wasteful at runtime, as they take time to
load and relocate, but will not be used by the calling object.
They also result in unnecessary processing at link-edit time.
.Pp
Dependency lists (typically defined via
.Dv $(LDLIBS) )
that have been copy and pasted
between
.Pa Makefiles
without verifying their need, are a typicalreason why unused dependencies
exist.
Unused dependencies are displayed as:
.Pp
.Dl foo: unused object=bar.so.1  <remove lib or -zignore?>
.It
Unreferenced dependencies are also wasteful at runtime, although not
to the extent of unused dependencies.
They also result in unnecessary processing at link-edit time.
.Pp
Unreferenced dependency removal guards against a dependency becoming
unused when combined with
different objects, or as the other object dependencies evolve.
Unreferenced dependencies are displayed as:
.Bd -literal
foo: unreferenced object=bar.so.1;  \\
    unused dependency of libfoo.so.1  \\
    <remove lib or -zignore?>
.Ed
.Pp
See also the section
.Sx ENVIRONMENT VARIABLES .
.It
Unused search paths are wasteful at runtime.
Unused search paths are displayed as:
.Bd -literal
foo: unused search path=/usr/foo/lib  \\
    (RUNPATH/RPATH from file libfoo.so.1)  \\
    <remove search path?>
.Ed
.El
.Pp
.Nm check_rtime
uses
.Xr elfdump 1
to look for a concatenated relocation section in shared objects, the existence
of text relocations, whether debugging or symbol table information exists,
whether applications have a non-executable stack defined, duplicate entries in
the symbol sorting sections, and for direct bindings.
These checks are carried out for the following reasons:
.Bl -bullet
.It
A concatenated relocation section
.Pq Em .SUNW_reloc
provides optimal symbol table access at runtime, and thus reduces the overhead
of relocating the shared object.
In past releases, the link-edit of a dynamic object with the
.Fl z Ar combreloc
option was required to generate a combined relocation section.
However, with the integration of 6642769, this section combination is a default behavior of
the link-editor.
.Pp
In past releases, not inheriting
.Dv $(DYNFLAGS)
from
.Pa lib/Makefile.lib
was the typical reason for not having a concatenated relocation section.
The misguided use of the
.Fl z Ar nocombreloc
option will also prevent the creation of a concatenated relocation section.
A missing concatenated relocation section is displayed as:
.Pp
.Dl foo: .SUNW_reloc section missing  <no -zcombreloc?>
.It
Text relocations result in impure text segments.
As text segments are typically read-only, they can be shared between numerous
processes.
If they must be updated as part of the relocation then the updated pages
become unsharable and swap space must be allocated to back these pages.
These events consume unnecessary system resources and reduce overall system
performance.
.Pp
Not inheriting the
.Dv $(PICS)
rules from
.Pa lib/Makefile.lib
is the typical reason for having non-pic code in shared objects.
Text relocations are displayed as:
.Pp
.Dl foo: TEXTREL .dynamic tag  <no -Kpic?>
.It
Debugging information is unnecessary in released objects.
Although extensive when compiled
.Fl g ,
small quantities of debugging information are stored in
.Em .stabs
sections under normal compilations.
This debugging information is geared towards aiding debuggers locate
relocatable objects associated with the dynamic objects being debugged.
As relocatable objects aren't made available as part of a software release
this information has no use.
.Pp
Not inheriting the correct
.Dv $(LDFLAGS)
from
.Pa cmd/Makefile.cmd
.Pq which asserts Fl s
or
.Dv $(POST_PROCESS_SO)
.Pq which asserts Ic strip -x
are typical reasons for not removing debugging information.
Note, removal of debugging information is only enabled
for release builds.
The existence of debugging information is displayed as:
.Bd -literal
foo: debugging sections should be deleted  \\
    <no strip -x?>
.Ed
.It
All objects should retain their full
.Em .symtab
symbol table.
Although this consumes disk space, it provides for more extensive stack
tracing when debugging user applications.
.Pp
Hard coding a
.Fl s
flag with
.Dv $(LDFLAGS) or
.Dv $(DYNFLAGS)
is the typical reason for symbol tables being removed.
Objects that do not contain a symbol table are displayed as:
.Bd -literal
foo.so.1: symbol table should not be stripped  \\
    <remove -s?>
.Ed
.It
Applications should have a non-executable stack defined to make
them less vulnerable to buffer overflow attacks.
.Pp
Not inheriting the
.Dv $(LDFLAGS)
macro in
.Pa cmd/Makefile.cmd
is the typical reason for not having a non-executable stack definition.
Applications without this definition are displayed as:
.Bd -literal
foo: application requires non-executable stack \\
	<no -Mmapfile_noexstk?>
.Ed
.It
x86 applications should have a non-executable data segment defined to make
them less vulnerable to buffer overflow attacks.
.Pp
Not inheriting the
.Dv $(LDFLAGS)
macro in
.Pa cmd/Makefile.cmd
is the typical reason for not having a non-executable data definition.
Applications without this definition are displayed as:
.Bd -literal
foo: application requires non-executable data \\
	<no -Mmapfile_noexdata?>
.Ed
.It
Solaris ELF files contain symbol sort sections used by DTrace to
map addresses in memory to the related function or variable symbols.
There are two such sections,
.Em .SUNW_dynsymsort
for regular symbols, and
.Em .SUNW_dyntlssort
for thread-local symbols.
To ensure that the best names are shown for each such address, and that the
same name is given across Solaris releases,
.Nm check_rtime
enforces the rule that only one symbol can appear in the sort sections for
any given address.
There are two common ways in which multiple symbols
or a given address occur in the ON distribution.
The first is from code written in assembly language.
The second is as a result of using
.Ic #pragma weak
in C to create weak symbols.
The best solution to this situation is to modify the code to avoid symbol
aliasing.
Alternatively, the
.Va NODYNSORT
mapfile attribute can be used to eliminate the unwanted symbol.
.Pp
Duplicate entries in a symbol sort section are
displayed in one of the following ways, depending on
whether the section is for regular or thread-local symbols:
.Bd -literal
foo: .SUNW_dynsymsort: duplicate ADDRESS: sym1, sym2
foo: .SUNW_dyntlssort: duplicate OFFSET: sym1, sym2
.Ed
.It
illumos dynamic ELF objects are expected to employ direct bindings whenever
feasible.
This runtime binding technique helps to avoid accidental interposition
problems, and provides a more optimal runtime symbol search model.
.Pp
Not inheriting the correct
.Dv $(LDFLAGS) from
.Pa cmd/Makefile.cmd ,
or the correct
.Dv $(DYNFLAGS)
from
.Pa lib/Makefile.lib ,
are the typical reasons for not enabling direct bindings.
Dynamic objects that do not contain direct binding information are displayed
as:
.Bd -literal
foo: object has no direct bindings \\
	<no -B direct or -z direct?>
.Ed
.El
.Pp
.Nm check_rtime
also
uses
.Xr elfdump 1
to display useful dynamic entry information under the
.Fl -i
option.
This doesn't necessarily indicate an error condition, but
provides information that is often useful for gatekeepers to track
changes in a release.
Presently the information listed is:
.Bl -bullet
.It
Runpaths are printed for any dynamic object.
This is a historic sanity check to insure compiler supplied runpaths
(typically from
.Nm CC )
are not recorded in any objects.
Runpaths are displayed as:
.Pp
.Dl foo: RPATH=/usr/bar/lib
.It
Needed dependencies are printed for any dynamic object.
In the freeware world this often helps the introducer of a new
shared object discover that an existing binary has become its
consumer, and thus that binaries package dependencies may require updating.
Dependencies are printed as:
.Pp
.Dl foo: NEEDED=bar.so.1
.It
Dependencies may be marked as forbidden
.Pq see Sx EXCEPTION FILE FORMAT
this allows the build to warn should people use them accidentally.
Forbidden dependencies are printed as:
.Pp
.Dl foo: NEEDED=bar.so.1  <forbidden dependency, missing -nodefaultlibs?>
.El
.Pp
.Nm check_rtime
uses
.Xr mcs 1
to inspect an object's
.Em .comment
section.
During development, this section contains numerous file identifiers
marked with the tag
.Qq @(#) .
For release builds these sections are deleted and rewritten under control of
the
.Dv $(POST_PROCESS)
macro to produce a common release identifier.
This identifier typically consists of three lines including a single comment
starting with the string
.Qq @(#) SunOS .
If this common identifier isn't found the following diagnostic is generated:
.Pp
.Dl foo: non-conforming mcs(1) comment  <no $(POST_PROCESS)?>
.Pp
.Nm check_rtime
uses
.Xr pvs 1
to display version definitions under the
.Fl v
option.
Each symbol defined by the object is shown along with the version it belongs to.
Changes to the symbols defined by an object, or the versions they belong to,
do not necessarily indicate an error condition, but
provides information that is often useful for gatekeepers to track
changes in a release.
.Sh OPTIONS
The following options are supported:
.Bl -tag -width indent
.It Fl D Ar depfile
Use
.Ar depfile
to generate an alternative dependency mapping.
.Ar depfile
must be created by
.Ic find_elf -r .
The
.Fl D
and
.Fl d
options are mutually exclusive.
.It Fl d Ar depfile
Use
.Ar depdir
to generate an alternative dependency mapping.
.Xr find_elf 1ONBLD
is used to locate the ELF sharable objects for which alternative mappings are
required.
The
.Fl D
and
.Fl d
options are mutually exclusive.
.It Fl E Ar errfile
Direct error messages for the analyzed objects to
.Ar errfile
instead of stdout.
.It Fl e Ar exfile
An exception file is used to exclude objects from
the usual rules.
See
.Sx EXCEPTION FILE FORMAT .
.It Fl f Ar listfile
Normally,
.Ic interface_check
runs
.Ic find_elf
to locate the ELF objects to analyze.
The
.Fl f
option can be used to instead provide a file containing the list of objects to
analyze, in the format produced by
.Ic find_elf -r .
.It Fl I Ar infofile
Direct informational messages (
.Fl i ,
and
.Fl v
options) for the analyzed objects to
.Ar infofile
instead of stdout.
.It Fl i
Provide dynamic entry information.
Presently only dependencies and runpaths are printed.
.It Fl m
Enable
.Xr mcs 1
checking.
.It Fl o
Produce a one-line output for each condition discovered, prefixed
by the objects name.
This output style is more terse, but is more appropriate for sorting and
diffing with previous build results.
.It Fl s
Determine whether
.Em .stabs
sections exist.
.It Fl v
Provide version definition information.
Each symbol defined by the object is printed along with the version it is
assigned to.
.It Fl w Ar outdir
Interpret the paths of all input and output files relative to
.Ar outdir .
.El
.Sh EXCEPTION FILE FORMAT
Exceptions to the rules enforced by
.Nm check_rtime
are specified using an exception file.
The
.Fl -e
option is used to specify an explicit exception file.
Otherwise, if used in an activated workspace, the default exception file is
.Pa $CODEMGR_WS/exception_list/check_rtime
if that file exists.
If not used in an activated workspace, or if
.Pa $CODEMGR_WS/exception_list/check_rtime
does not exist,
.Nm check_rtime
will use
.Pa /opt/onbld/etc/exception_list/check_rtime
as a fallback default exception file.
.Pp
To run
.Nm check_rtime
without applying exceptions, specify
.Fl e
with a value of
.Pa /dev/null .
.Pp
A
.Ql #
character at the beginning of a line, or at any point in
a line when preceded by whitespace, introduces a comment.
Empty lines, and lines containing only comments, are ignored by
.Nm check_rtime .
Exceptions are specified as space separated keyword, and
.Xr perl 1
regular expression:
.Pp
.Dl keyword  perl-regex
.Pp
Since whitespace is used as a separator, the regular
expression cannot itself contain whitespace.
Use of the
.Ql \es
character class to represent whitespace within the regular expression is
recommended.
.Pp
Before the perl regular expression is used, constructs of the form
.Em MACH(dir)
are expanded into a regular expression that matches the directory given, as
well as any 64-bit architecture subdirectory that might be present
(i.e. amd64, sparcv9). For instance,
.Em MACH(lib)
will match any of the following:
.Bl -tag -width indent
.It Pa lib
.It Pa lib/amd64
.It Pa lib/sparcv9
.El
.Pp
The exceptions understood by
.Nm check_rtime
are:
.Bl -tag -width indent
.It EXEC_DATA
Executables that are not required to have non-executable writable
data segments
.It EXEC_STACK
Executables that are not required to have a non-executable stack
.It NOCRLEALT
Objects that should be skipped when building the alternative dependency
mapping via the
.Fl d
option.
.It NODIRECT
Directories and files that are allowed to have no direct bound symbols.
.It NOSYMSORT
Files for which we skip checking of duplicate addresses in the
symbol sort sections.
.It OLDDEP
Objects that used to contain system functionality that has since
migrated to libc.
We preserve these libraries as pure filters for backward compatibility but
nothing needs to link to them.
.It SKIP
Directories and/or individual objects to skip.
Note that SKIP should be a last resort, used only when one of the other
exceptions will not suffice.
.It STAB
Objects that are allowed to contain debugging information (stabs).
.It TEXTREL
Objects for which we allow relocations to the text segment.
.It UNREF_OBJ
Objects that are allowed to be unreferenced.
.It UNDEF_REF
Objects that are allowed undefined references.
.It UNUSED_DEPS
Objects that are allowed to have unused dependencies.
.It UNUSED_OBJ
Objects that are always allowed to be unused dependencies.
.It UNUSED_RPATH
Objects that are allowed to have unused runpath directories.
.It FORBIDDEN
Specifies that dependencies on a given object are forbidden.
.It FORBIDDEN_DEP
Specifies that a given object is permitted a forbidden dependency.
.El
.Sh ALTERNATIVE DEPENDENCY MAPPING
.Nm check_rtime
was primarily designed to process a nightly builds
.Ev $ROOT
hierarchy.
It is often the case that objects within this hierarchy must bind to
dependencies within the same hierarchy to satisfy their requirements.
.Pp
To achieve this,
.Nm check_rtime
uses the shared objects specified with the
.Fl D
or
.Fl d
options.
If neither option is specified, and the
.Ev $CODEMGR_WS
and
.Ev $ROOT
environment variables are defined, the proto area for the workspace is
used.
The objects found are used to create runtime configuration files via
.Xr crle 1 ,
that establish the new shared objects as alternatives to their underlying
system location.
.Nm check_rtime
passes these configuration files as
.Ev LD_CONFIG
environment variable settings to
.Xr ldd 1
using its
.Fl -e
option.
.Pp
The effect of these configuration files is that the execution of an
object under
.Xr ldd 1
will bind to the dependencies defined as alternatives.
Simply put, an object inspected in the
.Pa proto
area will bind to its dependencies found in the
.Pa proto
area.
Dependencies that have no alternative mapping will continue to bind to the
underlying system.
.Sh ENVIRONMENT VARIABLES
When the
.Fl D
or
.Fl d
option isn't in use,
.Nm check_rtime
uses the following environment variables to
establish an alternative dependency mapping:
.Bl -tag -width indent
.It Ev CODEMGR_WS
The root of your workspace, which is the directory
containing
.Pa .git .
Existence of this environment variable indicates that
.Ev $ROOT
should be investigated.
.It Ev ROOT
Root of the
.Pa proto
area of your workspace.
Any shared objects under this directory will be used to establish an
alternative dependency mapping.
.El
If
.Xr ldd 1
supports the
.Fl U
option, it will be used to determine any unreferenced dependencies.
Otherwise
.Xr ldd 1
uses the older
.Fl u
option which only detects unused references.
If the following environment variable exists, and indicates an earlier release
than \fB5.10\fP then
.Xr ldd 1
also falls back to using the
.Fl u
option.
.Bl -tag -width indent
.It Ev RELEASE
The release version number of the environment being built.
.El
.Sh ERROR CONDITIONS
Inspection of an object with
.Xr ldd 1
assumes it is compatible with the machine on which
.Nm check_rtime
is being run.
Incompatible objects such as a 64-bit object encountered on a 32-bit system,
or an i386 object encountered on a sparc system, can not be fully inspected.
These objects are displayed as:
.Pp
.Dl foo: has wrong class or data encoding
.Sh FILES
.Bl -tag -width indent
.It Pa $CODEMGR_WS/exception_list/check_rtime
.It Pa /opt/onbld/etc/exception_list/check_rtime
.El
.Sh SEE ALSO
.Xr crle 1 ,
.Xr elfdump 1 ,
.Xr ld.so.1 1 ,
.Xr ldd 1 ,
.Xr mcs 1 ,
.Xr find_elf 1ONBLD