Don't store shadow copies of per-process AIO limits.

Previously the AIO subsystem would save a snapshot of the currently
configured per-process limits the first time a process used AIO. The
process

Don't store shadow copies of per-process AIO limits.

Previously the AIO subsystem would save a snapshot of the currently
configured per-process limits the first time a process used AIO. The
process would continue to use the snapshotted limits ignoring any
changes to the global limits during the rest of its lifetime. This
change removes the snapshotted values and changes the AIO code to
always check the global values which can be toggled at runtime.
This means an administrator can now change the effective limits of
existing processes. This is more consistent with how other limits
configured via sysctl work in FreeBSD.

Reviewed by: asomers, kib
MFC after: 2 weeks
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D13819

show more ...

Allow the fast-path for disk AIO requests to fail requests.

- If aio_qphysio() returns a non-zero error code, fail the request rather
than queueing it to the AIO kproc pool to be retried via the s

Allow the fast-path for disk AIO requests to fail requests.

- If aio_qphysio() returns a non-zero error code, fail the request rather
than queueing it to the AIO kproc pool to be retried via the slow path.
Currently this means that if vm_fault_quick_hold_pages() reports an
error, EFAULT is returned from the fast-path rather than retrying the
request in the slow path where it will still fail with EFAULT.
- If aio_qphysio() wishes to use the fast path for a device that doesn't
support unmapped I/O but there are already the maximum number of
such requests in flight, fail with EAGAIN as we do for other AIO
resource limits rather than queueing the request to the AIO kproc pool.
- Move the opcode check for aio_qphysio() out of the caller and into
aio_qphysio() to simplify some logic and remove two goto's while here.
It also uses a whitelist (only supported for LIO_READ / LIO_WRITE)
rather than a blacklist (skipped for LIO_SYNC).

PR: 217261
Submitted by: jkim (an earlier version)
MFC after: 2 weeks
Sponsored by: Chelsio Communications

show more ...

Simplify some logic by merging an if test with a subsequent switch.

Specifically, in aio_queue_file() the code was doing this:

if (opcode == LIO_SYNC) {
...
}

switch (opcode) {

Simplify some logic by merging an if test with a subsequent switch.

Specifically, in aio_queue_file() the code was doing this:

if (opcode == LIO_SYNC) {
...
}

switch (opcode) {
...
case LIO_SYNC:
...
}

This moves the body of the if statement into the LIO_SYNC case of the
switch statement.

MFC after: 2 weeks
Sponsored by: Chelsio Communications

show more ...

Add a counter to track in-flight AIO requests using unmapped I/O.

MFC after: 2 weeks
Sponsored by: Chelsio Communications

Merge ^/head r327169 through r327340.

Do pass removing some write-only variables from the kernel.

This reduces noise when kernel is compiled by newer GCC versions,
such as one used by external toolchain ports.

Reviewed by: kib, andrew(

Do pass removing some write-only variables from the kernel.

This reduces noise when kernel is compiled by newer GCC versions,
such as one used by external toolchain ports.

Reviewed by: kib, andrew(sys/arm and sys/arm64), emaste(partial), erj(partial)
Reviewed by: jhb (sys/dev/pci/* sys/kern/vfs_aio.c and sys/kern/kern_synch.c)
Differential Revision: https://reviews.freebsd.org/D10385

show more ...

sys/kern: adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone

sys/kern: adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

show more ...

Merge ^/head r323559 through r325504.

MFhead@r325119

Fix aio_suspend in 32-bit emulation

An off-by-one error has been present since the system call was first present
in 185878. It additionally became a memory corruption bug after change
324941. The

Fix aio_suspend in 32-bit emulation

An off-by-one error has been present since the system call was first present
in 185878. It additionally became a memory corruption bug after change
324941. The failure is actually revealed by our existing AIO tests.
However, apparently nobody's been running those in 32-bit emulation mode.

Reported by: Coverity, cem
CID: 1382114
MFC after: 18 days
X-MFC-With: 324941
Sponsored by: Spectra Logic Corp

show more ...

MFhead@r325004

Remove artificial restriction on lio_listio's operation count

In r322258 I made p1003_1b.aio_listio_max a tunable. However, further
investigation shows that there was never any good reason for that

Remove artificial restriction on lio_listio's operation count

In r322258 I made p1003_1b.aio_listio_max a tunable. However, further
investigation shows that there was never any good reason for that limit to
exist in the first place. It's used in two completely different ways:

* To size a UMA zone, which globally limits the number of concurrent
aio_suspend calls.

* To artifically limit the number of operations in a single lio_listio call.
There doesn't seem to be any memory allocation associated with this limit.

This change does two things:

* Properly names aio_suspend's UMA zone, and sizes it based on a new constant.

* Eliminates the artifical restriction on lio_listio. Instead, lio_listio
calls will now be limited by the more generous max_aio_queue_per_proc. The
old p1003_1b.aio_listio_max is now an alias for
vfs.aio.max_aio_queue_per_proc, so sysconf(3) will still work with
_SC_AIO_LISTIO_MAX.

Reported by: bde
Reviewed by: jhb
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D12120

show more ...

MFhead@r322451

Merge ^/head r321383 through r322397.

Make p1003_1b.aio_listio_max a tunable

p1003_1b.aio_listio_max is now a tunable. Its value is reflected in the
sysctl of the same name, and the sysconf(3) variable _SC_AIO_LISTIO_MAX.
Its value will

Make p1003_1b.aio_listio_max a tunable

p1003_1b.aio_listio_max is now a tunable. Its value is reflected in the
sysctl of the same name, and the sysconf(3) variable _SC_AIO_LISTIO_MAX.
Its value will be bounded from below by the compile-time constant
AIO_LISTIO_MAX and from above by the compile-time constant
MAX_AIO_QUEUE_PER_PROC and the tunable vfs.aio.max_aio_queue.

Reviewed by: jhb, kib
MFC after: 3 weeks
Relnotes: yes
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D11601

show more ...

Merge ^/head r319973 through 321382.

Merge ^/head r320042 through r320397.

Allow negative aio_offset only for the read and write LIO ops on
device nodes.

Otherwise, the current check of aio_offset == -1LL makes it possible
to pass negative file offsets down to the filesyst

Allow negative aio_offset only for the read and write LIO ops on
device nodes.

Otherwise, the current check of aio_offset == -1LL makes it possible
to pass negative file offsets down to the filesystems. This trips
assertions and is even unsafe for e.g. FFS which keeps metadata at
negative offsets.

Reported and tested by: pho
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D11266

show more ...

Add abstime kqueue(2) timers and expand struct kevent members.

This change implements NOTE_ABSTIME flag for EVFILT_TIMER, which
specifies that the data field contains absolute time to fire the
event

Add abstime kqueue(2) timers and expand struct kevent members.

This change implements NOTE_ABSTIME flag for EVFILT_TIMER, which
specifies that the data field contains absolute time to fire the
event.

To make this useful, data member of the struct kevent must be extended
to 64bit. Using the opportunity, I also added ext members. This
changes struct kevent almost to Apple struct kevent64, except I did
not changed type of ident and udata, the later would cause serious API
incompatibilities.

The type of ident was kept uintptr_t since EVFILT_AIO returns a
pointer in this field, and e.g. CHERI is sensitive to the type
(discussed with brooks, jhb).

Unlike Apple kevent64, symbol versioning allows us to claim ABI
compatibility and still name the new syscall kevent(2). Compat shims
are provided for both host native and compat32.

Requested by: bapt
Reviewed by: bapt, brooks, ngie (previous version)
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D11025

show more ...

Merge ^/head r313644 through r313895.

Rework r313352.

Rename kern_vm_* functions to kern_*. Move the prototypes to
syscallsubr.h. Also change Mach VM types to uintptr_t/size_t as
needed, to avoid headers pollution.

Requested by: alc,

Rework r313352.

Rename kern_vm_* functions to kern_*. Move the prototypes to
syscallsubr.h. Also change Mach VM types to uintptr_t/size_t as
needed, to avoid headers pollution.

Requested by: alc, jhb
Reviewed by: alc
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D9535

show more ...

MFhead @ r304815

Merge ^/head r304236 through r304536.

Remove duplicated code.

aio_aqueue() calls aio_init_aioinfo() as the first action. There is no
need to duplicate the code in kern_aio_fsync().

Also fix indent for aio_aqueue() definition.

Reviewed

Remove duplicated code.

aio_aqueue() calls aio_init_aioinfo() as the first action. There is no
need to duplicate the code in kern_aio_fsync().

Also fix indent for aio_aqueue() definition.

Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D7523

show more ...

Merge ^/head r303250 through r304235.