close_range: add audit support

This fixes the closefrom test in sys/audit.

Includes cherry-picks of the following commits from openbsm:

https://github.com/openbsm/openbsm/commit/4dfc628aafe589d688

close_range: add audit support

This fixes the closefrom test in sys/audit.

Includes cherry-picks of the following commits from openbsm:

https://github.com/openbsm/openbsm/commit/4dfc628aafe589d68848f7033f3d3488c4d979e0
https://github.com/openbsm/openbsm/commit/99ff6fe32aebc5a4b8d40d60062b8574697df557
https://github.com/openbsm/openbsm/commit/da48a0399e95448693d3fa2be48454ca564c1be8

Reviewed By: kevans
Differential Revision: https://reviews.freebsd.org/D28388

show more ...

jail: Change both root and working directories in jail_attach(2)

jail_attach(2) performs an internal chroot operation, leaving it up to
the calling process to assure the working directory is inside

jail: Change both root and working directories in jail_attach(2)

jail_attach(2) performs an internal chroot operation, leaving it up to
the calling process to assure the working directory is inside the jail.

Add a matching internal chdir operation to the jail's root. Also
ignore kern.chroot_allow_open_directories, and always disallow the
operation if there are any directory descriptors open.

Reported by: mjg
Approved by: markj, kib
MFC after: 3 days

show more ...

Fix fget_only_user() to return ENOTCAPABLE on a failed capsicum check

After eaad8d1303da500ed691bd774742a4555a05e729 four additional
capsicum-test tests started failing. It turns out this is because

Fix fget_only_user() to return ENOTCAPABLE on a failed capsicum check

After eaad8d1303da500ed691bd774742a4555a05e729 four additional
capsicum-test tests started failing. It turns out this is because
fget_only_user() was returning EBADF on a failed capsicum check instead
of forwarding the return value of cap_check_inline() like
fget_unlocked_seq().

capsicum-test failures before this:
```
[ FAILED ] 7 tests, listed below:
[ FAILED ] Capability.OperationsForked
[ FAILED ] Capability.NoBypassDAC
[ FAILED ] Pdfork.OtherUserForked
[ FAILED ] PipePdfork.WildcardWait
[ FAILED ] OpenatTest.WithFlag
[ FAILED ] ForkedOpenatTest_WithFlagInCapabilityMode._
[ FAILED ] Select.LotsOFileDescriptorsForked
```
After:
```
[ FAILED ] 3 tests, listed below:
[ FAILED ] Capability.NoBypassDAC
[ FAILED ] Pdfork.OtherUserForked
[ FAILED ] PipePdfork.WildcardWait
```

Reviewed By: mjg
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D28691

show more ...

fd: add fget_only_user

This can be used by single-threaded processes which don't share a file
descriptor table to access their file objects without having to
reference them.

For example select cons

fd: add fget_only_user

This can be used by single-threaded processes which don't share a file
descriptor table to access their file objects without having to
reference them.

For example select consumers tend to match the requirement and have
several file descriptors to inspect.

show more ...

fd: add refcount argument to falloc_noinstall

This lets callers avoid atomic ops by initializing the count to required
value from the get go.

While here add falloc_abort to backpedal from this with

fd: add refcount argument to falloc_noinstall

This lets callers avoid atomic ops by initializing the count to required
value from the get go.

While here add falloc_abort to backpedal from this without having to
fdrop.

show more ...

fd: add finstall_refed

Can be used to consume an already existing reference and consequently
avoid atomic ops.

fd: provide a dedicated closef variant for unix socket code

This avoids testing for td != NULL.

fd: use seqc_read_notmodify when translating fds

fd: make fd/fp mandatory

They are both always passed anyway.

fd: inline pwd_get_smr

Tested by: pho

Expose eventfd in the native API/ABI using a new __specialfd syscall

eventfd is a Linux system call that produces special file descriptors
for event notification. When porting Linux software, it is

Expose eventfd in the native API/ABI using a new __specialfd syscall

eventfd is a Linux system call that produces special file descriptors
for event notification. When porting Linux software, it is currently
usually emulated by epoll-shim on top of kqueues. Unfortunately, kqueues
are not passable between processes. And, as noted by the author of
epoll-shim, even if they were, the library state would also have to be
passed somehow. This came up when debugging strange HW video decode
failures in Firefox. A native implementation would avoid these problems
and help with porting Linux software.

Since we now already have an eventfd implementation in the kernel (for
the Linuxulator), it's pretty easy to expose it natively, which is what
this patch does.

Submitted by: greg@unrelenting.technology
Reviewed by: markj (previous version)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D26668

show more ...

fd: reimplement close_range to avoid spurious relocking

audit: rework AUDIT_SYSCLOSE

This in particular avoids spurious lookups on close.

fd: refactor closefp in preparation for close_range rework

fd: remove redundant saturation check from fget_unlocked_seq

refcount_acquire_if_not_zero returns true on saturation.
The case of 0 is handled by looping again, after which the originally
found poin

fd: remove redundant saturation check from fget_unlocked_seq

refcount_acquire_if_not_zero returns true on saturation.
The case of 0 is handled by looping again, after which the originally
found pointer will no longer be there.

Noted by: kib

show more ...

fd: fix fdrop prediction when closing a fd

Most of the time this is the last reference, contrary to typical fdrop use.

fd: make serialization in fdescfree_fds conditional on hold count

p_fd nullification in fdescfree serializes against new threads transitioning
the count 1 -> 2, meaning that fdescfree_fds observing

fd: make serialization in fdescfree_fds conditional on hold count

p_fd nullification in fdescfree serializes against new threads transitioning
the count 1 -> 2, meaning that fdescfree_fds observing the count of 1 can
safely assume there is nobody else using the table. Losing the race and
observing > 1 is harmless.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D27522

show more ...

Plug a race between fd table teardown and several loops

To export information from fd tables we have several loops which do
this:

FILDESC_SLOCK(fdp);
for (i = 0; fdp->fd_refcount > 0 && i <= lastfi

Plug a race between fd table teardown and several loops

To export information from fd tables we have several loops which do
this:

FILDESC_SLOCK(fdp);
for (i = 0; fdp->fd_refcount > 0 && i <= lastfile; i++)
<export info for fd i>;
FILDESC_SUNLOCK(fdp);

Before r367777, fdescfree() acquired the fd table exclusive lock between
decrementing fdp->fd_refcount and freeing table entries. This
serialized with the loop above, so the file at descriptor i would remain
valid until the lock is dropped. Now there is no serialization, so the
loops may race with teardown of file descriptor tables.

Acquire the exclusive fdtable lock after releasing the final table
reference to provide a barrier synchronizing with these loops.

Reported by: pho
Reviewed by: kib (previous version), mjg
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27513

show more ...

Use refcount_load(9) to load fd table reference counts

No functional change intended.

Reviewed by: kib, mjg
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D

Use refcount_load(9) to load fd table reference counts

No functional change intended.

Reviewed by: kib, mjg
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27512

show more ...

kern: never restart syscalls calling closefp(), e.g. close(2)

All paths leading into closefp() will either replace or remove the fd from
the filedesc table, and closefp() will call fo_close methods

kern: never restart syscalls calling closefp(), e.g. close(2)

All paths leading into closefp() will either replace or remove the fd from
the filedesc table, and closefp() will call fo_close methods that can and do
currently sleep without regard for the possibility of an ERESTART. This can
be dangerous in multithreaded applications as another thread could have
opened another file in its place that is subsequently operated on upon
restart.

The following are seemingly the only ones that will pass back ERESTART
in-tree:
- sockets (SO_LINGER)
- fusefs
- nfsclient

Reviewed by: jilles, kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D27310

show more ...

kern: dup: do not assume oldfde is valid

oldfde may be invalidated if the table has grown due to the operation that
we're performing, either via fdalloc() or a direct fdgrowtable_exp().

This was te

kern: dup: do not assume oldfde is valid

oldfde may be invalidated if the table has grown due to the operation that
we're performing, either via fdalloc() or a direct fdgrowtable_exp().

This was technically OK before rS367927 because the old table remained valid
until the filedesc became unused, but now it may be freed immediately if
it's an unshared table in a single-threaded process, so it is no longer a
good assumption to make.

This fixes dup/dup2 invocations that grow the file table; in the initial
report, it manifested as a kernel panic in devel/gmake's configure script.

Reported by: Guy Yur <guyyur gmail com>
Reviewed by: rew
Differential Revision: https://reviews.freebsd.org/D27319

show more ...

fd: free old file descriptor tables when not shared

During the life of a process, new file descriptor tables may be allocated. When
a new table is allocated, the old table is placed in a free list a

fd: free old file descriptor tables when not shared

During the life of a process, new file descriptor tables may be allocated. When
a new table is allocated, the old table is placed in a free list and held onto
until all processes referencing them exit.

When a new file descriptor table is allocated, the old file descriptor table
can be freed when the current process has a single-thread and the file
descriptor table is not being shared with any other processes.

Reviewed by: kevans
Approved by: kevans (mentor)
Differential Revision: https://reviews.freebsd.org/D18617

show more ...

Split out cwd/root/jail, cmask state from filedesc table

No functional change intended.

Tracking these structures separately for each proc enables future work to
correctly emulate clone(2) in linux

Split out cwd/root/jail, cmask state from filedesc table

No functional change intended.

Tracking these structures separately for each proc enables future work to
correctly emulate clone(2) in linux(4).

__FreeBSD_version is bumped (to 1300130) for consumption by, e.g., lsof.

Reviewed by: kib
Discussed with: markj, mjg
Differential Revision: https://reviews.freebsd.org/D27037

show more ...

Fix a pair of races in SIGIO registration

First, funsetownlst() list looks at the first element of the list to see
whether it's processing a process or a process group list. Then it
acquires the gl

Fix a pair of races in SIGIO registration

First, funsetownlst() list looks at the first element of the list to see
whether it's processing a process or a process group list. Then it
acquires the global sigio lock and processes the list. However, nothing
prevents the first sigio tracker from being freed by a concurrent
funsetown() before the sigio lock is acquired.

Fix this by acquiring the global sigio lock immediately after checking
whether the list is empty. Callers of funsetownlst() ensure that new
sigio trackers cannot be added concurrently.

Second, fsetown() uses funsetown() to remove an existing sigio structure
from a file object. However, funsetown() uses a racy check to avoid the
sigio lock, so two threads may call fsetown() on the same file object,
both observe that no sigio tracker is present, and enqueue two sigio
trackers for the same file object. However, if the file object is
destroyed, funsetown() will only remove one sigio tracker, and
funsetownlst() may later trigger a use-after-free when it clears the
file object reference for each entry in the list.

Fix this by introducing funsetown_locked(), which avoids the racy check.

Reviewed by: kib
Reported by: pho
Tested by: pho
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27157

show more ...

fd: make all f_count uses go through refcount_*