Align initial-exec TLS segments to the p_vaddr % align.

This is continuation of D21163/r359634, which handled the alignment
for global mode.

Non-x86 arches are not handled, maintainers are welcomed

Align initial-exec TLS segments to the p_vaddr % align.

This is continuation of D21163/r359634, which handled the alignment
for global mode.

Non-x86 arches are not handled, maintainers are welcomed.

Tested by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D24366

show more ...

Make p_vaddr % p_align == p_offset % p_align for (some) TLS segments.

See https://sourceware.org/bugzilla/show_bug.cgi?id=24606 for the test case.
See https://reviews.llvm.org/D64930 for the backgro

Make p_vaddr % p_align == p_offset % p_align for (some) TLS segments.

See https://sourceware.org/bugzilla/show_bug.cgi?id=24606 for the test case.
See https://reviews.llvm.org/D64930 for the background and more discussion.

Also this fixes another bug in malloc_aligned() where total size of
the allocated memory might be not enough to fit the aligned requested
block after the initial pointer is incremented by the pointer size.

Reviewed by: bdragon
Tested by: antoine (exp-run PR 244866), bdragon, emaste
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D21163

show more ...

Merge ^/head r358269 through r358399.

Remove sparc64 specific parts of rtld-elf.

Merge ^/head r357855 through r357920.

Handle non-plt IRELATIVE relocations, at least for x86.

lld 10.0 seems to generate this relocation for rdtsc_mb() ifunc in our libc.

Reported, reviewed, and tested by: dim (amd64, previous version)

Handle non-plt IRELATIVE relocations, at least for x86.

lld 10.0 seems to generate this relocation for rdtsc_mb() ifunc in our libc.

Reported, reviewed, and tested by: dim (amd64, previous version)
Discussed with: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23652

show more ...

Merge ^/head r357662 through r357854.

Fix indent.

Sponsored by: The FreeBSD Foundation
MFC after: 3 days

Use sigfastblock(2) in rtld.

This allows for rtld to not issue two sigprocmask(2) syscalls for each
symbol binding operation in single-threaded processes. Rtld needs to
block signals as part of loc

Use sigfastblock(2) in rtld.

This allows for rtld to not issue two sigprocmask(2) syscalls for each
symbol binding operation in single-threaded processes. Rtld needs to
block signals as part of locking to ensure signal safety of the bind
process, because signal handlers might need to lazily resolve symbol
references.

As result, number of syscalls issued on startup by simple programs not
using libthr, is typically reduced 2x. For instance, for hello world,
I see:
non-sigfastblock
# (truss ./hello > /dev/null) |& wc -l
63
sigfastblock
# (truss ./hello > /dev/null) |& wc -l
37

Tested by: pho
Disscussed with: cem, emaste, jilles
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D12773

show more ...

rtld: remove hand rolled memset and bzero

They were introduced to take care of ifunc, but right now no architecture
provides ifunc'ed variants. Since rtld uses memset extensively this results in
a p

rtld: remove hand rolled memset and bzero

They were introduced to take care of ifunc, but right now no architecture
provides ifunc'ed variants. Since rtld uses memset extensively this results in
a pessmization. Should someone want to use ifunc here they should provide a
mandatory symbol (e.g., rtld_memset).

See the review for profiling data.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D23176

show more ...

Stop prepending prefix to the result of realpath(3).

The path is already absolute.

Noted and reviewed by: rstone
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https:

Stop prepending prefix to the result of realpath(3).

The path is already absolute.

Noted and reviewed by: rstone
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23121

show more ...

rtld: Return error if $ORIGIN for a dlopen-ed library cannot be resolved ...

instead of killing the process. The same behaviour of terminating
image activation if the $ORIGIN cannot be resolved for

rtld: Return error if $ORIGIN for a dlopen-ed library cannot be resolved ...

instead of killing the process. The same behaviour of terminating
image activation if the $ORIGIN cannot be resolved for the main
object, is kept.

Reported by: Greg V <greg@unrelenting.technology>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23053

show more ...

Resolve relative argv0 for direct exec mode to absolute path for AT_EXECPATH.

We know the binary relative name and can reliably calculate cwd path.
Because realpath(3) was already linked into ld-elf

Resolve relative argv0 for direct exec mode to absolute path for AT_EXECPATH.

We know the binary relative name and can reliably calculate cwd path.
Because realpath(3) was already linked into ld-elf.so.1, reuse it
there to resolve dots and dotdots making the path more canonical.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23014

show more ...

rtld: fix after r356300

binpath local was changed from char array to a char pointer, update
strlcpy/strlcat uses.

Reported by: Coverity through vangyzen
CID: 1412239 and 1412240
Reviewed by: emaste

rtld: fix after r356300

binpath local was changed from char array to a char pointer, update
strlcpy/strlcat uses.

Reported by: Coverity through vangyzen
CID: 1412239 and 1412240
Reviewed by: emaste, imp, vangyzen
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23090

show more ...

rtld: Fix segfault in direct exec mode

When rtld is directly executed with arguments, it has to move the
program arguments, environment and elf aux data up a few slots to
remove its own arguments be

rtld: Fix segfault in direct exec mode

When rtld is directly executed with arguments, it has to move the
program arguments, environment and elf aux data up a few slots to
remove its own arguments before the process being executed sees
them. When copying the environment, rtld was incorrectly testing
whether the location about to be written to currently contained
NULL, when was supposed to check whether it had just copied the
NULL terminator of the environment string. This had the result
that the ELF aux data was mostly treated as environment variables,
and rtld would quickly crash when it tried to access required
ELF aux data that it didn't think was present.

Differential Revision: https://reviews.freebsd.org/D23008
Reviewed by: kib
MFC after: 1 month

show more ...

Fix AT_EXECPATH for direct exec mode.

When activated in direct exec mode, kernel-provided AT_EXECPATH points
to the interpreter. We need to recalculate auxv to point to the
string with the path to

Fix AT_EXECPATH for direct exec mode.

When activated in direct exec mode, kernel-provided AT_EXECPATH points
to the interpreter. We need to recalculate auxv to point to the
string with the path to the executable which is actually executed.

The somewhat problematic case is when the executable path is relative
and either $PATH use is not enabled or it contains '/' so $PATH search
is not performed. In this case resulting AT_EXECPATH is relative, I
might fix this later.

Reported and reviewed by: rstone
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D22894

show more ...

Eliminate the last MI difference in AT_* definitions (for powerpc).

Summary:
As a transition aide, implement an alternative elfN_freebsd_fixup which
is called for old powerpc binaries. Similarly, a

Eliminate the last MI difference in AT_* definitions (for powerpc).

Summary:
As a transition aide, implement an alternative elfN_freebsd_fixup which
is called for old powerpc binaries. Similarly, add a translation to rtld to
convert old values to new ones (as expected by a new rtld).

Translation of old<->new values is incomplete, but sufficient to allow an
installworld of a new userspace from an old one when a new kernel is running.

Test Plan:
Someone needs to see how a new kernel/rtld/libc works with an old
binary. If if works we can probalby ship this. If not we probalby need
some more compat bits.

Submitted by: brooks
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D20799

show more ...

Merge ^/head r352319 through r352435.

Fix aux_info corruption in rtld direct execution mode.

After the aux vector is moved, it is necessary to re-digest aux_info so the
pointers are updated to the new locations.

This was causing thread

Fix aux_info corruption in rtld direct execution mode.

After the aux vector is moved, it is necessary to re-digest aux_info so the
pointers are updated to the new locations.

This was causing thread creation to fail on powerpc64 when using direct
execution due to a nonsense value being read for aux_info[AT_STACKPROT].

Approved by: jhibbits (mentor)
Differential Revision: https://reviews.freebsd.org/D21656

show more ...

MFHead @r350386

Sponsored by: The FreeBSD Foundation

Reduce size of rtld by 22% by pulling in less code from libc

Currently RTLD is linked against libc_nossp_pic which means that any libc
symbol used in rtld can pull in a lot of depedencies. This was

Reduce size of rtld by 22% by pulling in less code from libc

Currently RTLD is linked against libc_nossp_pic which means that any libc
symbol used in rtld can pull in a lot of depedencies. This was causing
symbol such as __libc_interposing and all the pthread stubs to be included
in RTLD even though they are not required. It turns out most of these
dependencies can easily be avoided by providing overrides inside of rtld.

This change is motivated by CHERI, where we have an experimental ABI that
requires additional relocation processing to allow the use of function
pointers inside of rtld. Instead of adding this self-relocation code to
RTLD I attempted to remove most function pointers from RTLD and discovered
that most of them came from the libc dependencies instead of being actually
used inside rtld.

A nice side-effect of this change is that rtld is now 22% smaller on amd64.

text data bss dec hex filename
0x21eb6 0xce0 0xe60 145910 239f6 /home/alr48/ld-elf-x86.before.so.1
0x1a6ed 0x728 0xdd8 113645 1bbed /home/alr48/ld-elf-x86.after.so.1

The number of R_X86_64_RELATIVE relocations that need to be processed on
startup has also gone down from 368 to 187 (almost 50% less).

Reviewed By: kib
Differential Revision: https://reviews.freebsd.org/D20663

show more ...

MFHead @349476

Sponsored by: The FreeBSD Foundation

powerpc: Transition to Secure-PLT, like most other OSs

Summary:
PowerPC has two PLT models: BSS-PLT and Secure-PLT. BSS-PLT uses runtime
code generation to generate the PLT stubs. Secure-PLT was i

powerpc: Transition to Secure-PLT, like most other OSs

Summary:
PowerPC has two PLT models: BSS-PLT and Secure-PLT. BSS-PLT uses runtime
code generation to generate the PLT stubs. Secure-PLT was introduced with
GCC 4.1 and Binutils 2.17 (base has GCC 4.2.1 and Binutils 2.17), and is a
more secure PLT format, using a read-only linkage table, with the dynamic
linker populating a non-executable index table.

This is the libc, rtld, and kernel support only. The toolchain and build
parts will be updated separately.

Reviewed By: nwhitehorn, bdragon, pfg
Differential Revision: https://reviews.freebsd.org/D20598
MFC after: 1 month

show more ...

MFHead @347527

Sponsored by: The FreeBSD Foundation

Switch to use shared vnode locks for text files during image activation.

kern_execve() locks text vnode exclusive to be able to set and clear
VV_TEXT flag. VV_TEXT is mutually exclusive with the v_w

Switch to use shared vnode locks for text files during image activation.

kern_execve() locks text vnode exclusive to be able to set and clear
VV_TEXT flag. VV_TEXT is mutually exclusive with the v_writecount > 0
condition.

The change removes VV_TEXT, replacing it with the condition
v_writecount <= -1, and puts v_writecount under the vnode interlock.
Each text reference decrements v_writecount. To clear the text
reference when the segment is unmapped, it is recorded in the
vm_map_entry backed by the text file as MAP_ENTRY_VN_TEXT flag, and
v_writecount is incremented on the map entry removal

The operations like VOP_ADD_WRITECOUNT() and VOP_SET_TEXT() check that
v_writecount does not contradict the desired change. vn_writecheck()
is now racy and its use was eliminated everywhere except access.
Atomic check for writeability and increment of v_writecount is
performed by the VOP. vn_truncate() now increments v_writecount
around VOP_SETATTR() call, lack of which is arguably a bug on its own.

nullfs bypasses v_writecount to the lower vnode always, so nullfs
vnode has its own v_writecount correct, and lower vnode gets all
references, since object->handle is always lower vnode.

On the text vnode' vm object dealloc, the v_writecount value is reset
to zero, and deadfs vop_unset_text short-circuit the operation.
Reclamation of lowervp always reclaims all nullfs vnodes referencing
lowervp first, so no stray references are left.

Reviewed by: markj, trasz
Tested by: mjg, pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D19923

show more ...