Merge tag 'sched_ext-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext

Pull sched_ext updates from Tejun Heo:
"Most of this cycle completes the enqueue-path support for hierarc

Merge tag 'sched_ext-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext

Pull sched_ext updates from Tejun Heo:
"Most of this cycle completes the enqueue-path support for hierarchical
sub-scheduling, which makes sub-scheduler support feature complete: a
root BPF scheduler can now hand a cgroup subtree over to a nested
sub-scheduler together with revocable CPU grants, and the
sub-scheduler owns all scheduling decisions for its tasks on those
CPUs.

Development volume was high and a number of changes plugging holes in
the new support landed late in the cycle. Also included are core
scheduling fixes that were completed too late for the v7.2 release and
are routed through this pull request.

Sub-scheduler CPU delegation:

- Parent schedulers now grant and revoke per-CPU capabilities
(enqueueing, preemption, CPU frequency control) on their children,
enforced on every path a scheduler can reach a CPU through.
Previously only dispatching could be delegated; this lets
sub-schedulers fully schedule their CPUs.

- Rescue execution: a task whose scheduler doesn't have access to the
CPUs the task needs to run on starved until the watchdog ejected
the whole scheduler. The kernel now runs such tasks directly on a
small bandwidth budget, turning a scheduler-killing failure into
bounded degradation.

- Cgroup integration: tasks migrating across a sub-scheduler boundary
weren't re-homed to the new owner, causing wrong-scheduler
scheduling and a use-after-free. Sub-schedulers now take over their
cgroup subtree and receive its cgroup callbacks.

- Arena objects now cross the kernel/BPF boundary as typed pointer
arguments, translated transparently by the BPF tree's new arena
argument support, replacing untyped arguments with manual
translation.

- scx_qmap now demonstrates full hierarchical sub-scheduling.

Other fixes and updates:

- Robustness improvements: the abort path is now NMI-safe, fixing
deadlocks when errors are raised from NMI context and making
hardlockup recovery direct. Reenqueue loops that could monopolize a
CPU ahead of the watchdog now eject the offending scheduler, and
stalls are blamed on the scheduler actually responsible.

- Hardening: BPF-writable arena memory is validated before kernel
use, and task slice and vtime writes got explicit synchronization
rules, closing corruption vectors open to buggy or malicious
schedulers.

- Core scheduling: sched_ext dispatching can drop the rq lock inside
the core-wide pick, which let interleaving selections corrupt each
other's state and hard-hang the machine. The selection now restarts
when the lock was released. The task ordering callback was also
invoked with its arguments swapped, and the default ordering is
updated to work across sub-scheduler boundaries. The fixes are
marked for stable.

- Other fixes headed for stable: a task init leak on fork failure
during enable, tooling compat macros that silently failed to detect
newer kernels, and a crash on reenqueueing against a destroyed
dispatch queue.

- Tooling: scx_pair moves off deprecated callbacks, and the
deprecated scx_bpf_cpu_rq() kfunc is removed"

* tag 'sched_ext-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext: (144 commits)
sched_ext: Drop the dead SCX_DEQ_CORE_SCHED_EXEC test in dequeue_task_scx()
sched_ext: Make core-sched task ordering hierarchy-aware
sched_ext: Use runnable_at for the default core-sched task ordering
sched_ext: Fix inverted ops.core_sched_before() invocation
sched_ext: Move the config-off sub-cap kfunc stubs into sub.c
sched_ext: Rename balance-era identifiers to dispatch terms
sched_ext: Drop the stale keep_prev fixup in dispatch_pick()
sched_ext: Keep kick_sync waiting on the rq's own CPU
sched_ext: Make SCHED_CLASS_EXT select GENERIC_ALLOCATOR
sched_ext/scx_flatcg: Fix cvtime true-up on slice expiry
sched_ext: Don't BUG_ON a destroyed DSQ in process_deferred_reenq_users
sched_ext: Fix scx_bpf_dsq_move_to_local___v2 compat detection
sched_ext: Make scx_bpf_events() read the calling scheduler's counters
sched_ext: Drop unlocked scx_rq_clock_invalidate() from scx_root_disable()
selftests/sched_ext: Fix flaky ddsp failure tests on busy systems
selftests/sched_ext: Make numa idle validation race-free
sched_ext: Fix scx_bpf_dsq_reenq___compat kfunc extern prototype
sched_ext/scx_flatcg: expire cached hweights on weight changes
sched_ext: Fix exit_task leak on fork failure during enable
sched_ext: fix stale references in doc comments
...

show more ...

sched_ext: Merge branch 'for-7.3-arena-args' into for-7.3

Pull to receive the __arena argument conversion:

67f1f4a48c24 ("sched_ext: Pass kernel arena pointers to ops_cid callbacks")
a8dc810968af

sched_ext: Merge branch 'for-7.3-arena-args' into for-7.3

Pull to receive the __arena argument conversion:

67f1f4a48c24 ("sched_ext: Pass kernel arena pointers to ops_cid callbacks")
a8dc810968af ("sched_ext: Convert sub-cap kfuncs to __arena cmask arguments")
a05c5b5cb5cf ("sched_ext: Convert scx_bpf_cid_override() to __arena array arguments")

along with the bpf-next branch carrying the __arena argument support they
depend on.

Conflict in kernel/sched/ext/ext.c between:

c384ab8a0b13 ("sched_ext: Move the config-off sub-cap kfunc stubs into sub.c")

and:

a8dc810968af ("sched_ext: Convert sub-cap kfuncs to __arena cmask arguments")

which updated the stubs in their old ext.c location. Resolved by keeping
ext.c without the stubs and applying the prototype conversion to the
relocated stubs in sub.c.

Signed-off-by: Tejun Heo <tj@kernel.org>

show more ...

sched_ext: Convert scx_bpf_cid_override() to __arena array arguments

scx_bpf_cid_override() predates the cid-form arena transition and takes its
arrays as verifier-checked mem+size buffers, forcing

sched_ext: Convert scx_bpf_cid_override() to __arena array arguments

scx_bpf_cid_override() predates the cid-form arena transition and takes its
arrays as verifier-checked mem+size buffers, forcing scx_qmap to keep the
cpu_to_cid and shard_start arrays in writable bss while the rest of its
state lives in the arena. Unify on arena arguments before cid-form
schedulers start seeing real use.

BPF now translates between BPF and kernel arena addresses for __arena
arguments. Take the arrays as __arena arguments, with the counts passed in
entries. The counts now size the snapshot copies and are bounds-checked
before them.

scx_qmap moves the arrays into struct qmap_arena. As the arena is mmapped at
load, the loader populates them between load and attach instead of before
load.

The arena argument address translation is currently implemented only on
x86-64. Schedulers calling this kfunc load only there for now.

Signed-off-by: Tejun Heo <tj@kernel.org>

show more ...

sched_ext: scx_qmap - Add rescue support

A sched holds only the cids its parent granted and nothing guarantees that
they cover its tasks' affinities. A task that can run on none of them has
nowhere

sched_ext: scx_qmap - Add rescue support

A sched holds only the cids its parent granted and nothing guarantees that
they cover its tasks' affinities. A task that can run on none of them has
nowhere to go and qmap stalls out: it force-inserts the task onto its first
allowed cid, but the kernel bounces the insert back and the task parks in
SHARED_DSQ, which is drained only on self cids it can't run on.

Set SCX_ENQ_RESCUE on these inserts so the kernel diverts such tasks to its
rescue path instead of bouncing them. The force-insert covers scheds with
and without children and fires on re-enqueues, and the SHARED_DSQ scan on
every dispatch rescues tasks stranded there - the enqueue-time check misses
a task whose cids were lost while it was already queued. The wrong-cid fault
injection carries the flag too and doubles as a deterministic rescue-traffic
generator.

-B and -q set the root-only rescue bandwidth and quantum ops knobs. -B 0
maps to SCX_RESCUE_DISABLE and turns rescue off kernel-side. Rescue inserts
are counted and reported in the hier stats line.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>

show more ...

sched_ext: Rename the cid-form cgroup ops to cpuctl_*

Two unrelated things go by "cgroup" in the cid form. Sub-schedulers attach
to cgroups, and the cgroup_*() ops deliver cpu controller events. Whi

sched_ext: Rename the cid-form cgroup ops to cpuctl_*

Two unrelated things go by "cgroup" in the cid form. Sub-schedulers attach
to cgroups, and the cgroup_*() ops deliver cpu controller events. While the
ops names suggest cgroup2 hierarchy, they actually operate on the cpu
controller.

Rename them to cpuctl_* in struct sched_ext_ops_cid, which has no users
outside scx_qmap yet. The cpu form is deployed ABI and keeps the old names.
The layout is unchanged and the kernel keeps calling through the cpu-form
union view.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>

show more ...

tools/sched_ext: scx_qmap - Add init fault injection modes

Add -J init-fail which makes ops.init_task() fail with -ENOMEM for tasks
whose comm starts with "qmfail", and -J cgrp-init-fail which does

tools/sched_ext: scx_qmap - Add init fault injection modes

Add -J init-fail which makes ops.init_task() fail with -ENOMEM for tasks
whose comm starts with "qmfail", and -J cgrp-init-fail which does the same
in ops.cgroup_init() for cgroups named "qmfail*".

The former exercises the migration veto path: the cgroup.procs write must
fail with the injected errno while the destination sched stays up and the
task stays put. The latter exercises the ownership-return failure path: a
parent failing to re-init a returned cgroup leaves it unowned, and moves and
set_* ops against it must be skipped instead of dereferencing the missing
owner. Matching on "qmfail" names keeps the injecting scheduler's own enable
unaffected.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>

show more ...

tools/sched_ext: scx_qmap - Consume cgroup weights through set_weight

With the set_* ops delivered to the parent's sched, a parent qmap instance
now receives cgroup_set_weight for its child subs' at

tools/sched_ext: scx_qmap - Consume cgroup weights through set_weight

With the set_* ops delivered to the parent's sched, a parent qmap instance
now receives cgroup_set_weight for its child subs' attach points. Update
the matching sub_sched_ctx weight and redistribute() in-kernel, and drop
the userspace feed_weights() polling. This exercises the knob routing end
to end.

The self weight is fixed at 100: a cgroup's weight is its parent's knob and
not the scheduler's own business. This drops the self-weight polling and the
repartition PROG_RUN poke with it.

sub_attach seeds the slot with the cgroup's current weight, read through
bpf_cgroup_from_id(), so a weight set before the sub attaches is picked up.
A write racing the attach can still be lost until the next value-changing
cpu.weight write. Acceptable for a demo.

While at it, add a traced ops.cgroup_move() so tests can observe move
delivery.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>

show more ...

tools/sched_ext: scx_qmap - Add sub-sched cap fault injection

Add a fault-injection mode to the scx_qmap sub-scheduler that deliberately
dispatches one of its own tasks to a cid it does not hold. Th

tools/sched_ext: scx_qmap - Add sub-sched cap fault injection

Add a fault-injection mode to the scx_qmap sub-scheduler that deliberately
dispatches one of its own tasks to a cid it does not hold. The kernel cap
check must reject it and re-enqueue with SCX_TASK_REENQ_CAP, so the
nr_inject_attempts counter tracks nr_reenq_cap one to one, exercising the
delivery-time cap enforcement.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>

show more ...

tools/sched_ext: scx_qmap - Expand hierarchical sub-scheduling

sched_ext sub-scheduling began as dispatch delegation only: a parent could
call into a child cgroup sub-scheduler's ops.dispatch() from

tools/sched_ext: scx_qmap - Expand hierarchical sub-scheduling

sched_ext sub-scheduling began as dispatch delegation only: a parent could
call into a child cgroup sub-scheduler's ops.dispatch() from its own
dispatch path, but could not delegate cpus to the child for enqueue and the
other paths. sched_ext has since gained cap-based cid delegation, where a
parent grants and revokes a child's per-cid caps. Expand scx_qmap to
demonstrate it.

scx_qmap can now delegate the cids it holds exclusively, split among itself
and its children by cpu.weight. Each gets the floor of its share as
dedicated cids. The leftover from rounding forms a shared pool,
round-robined among them as an ENQ_IMMED time-share.

This shape is deliberate. Exclusive cids exercise the basic grant and revoke
of ownership, and the shared pool exercises time-sharing one cid across
several schedulers. The implemented policy is impractical, but it covers
most of what a practical sub-scheduler would need without overcomplicating
qmap.

Delegation nests. A cid a node receives from its parent only as a
round-robin share stays self-local and is never re-delegated. A node left
with no exclusive cid, e.g. after its cpus went offline, evicts its
children.

v5: Highpri dispatch masked with self_cids, single-read dispatch cgroup_id, feed_weights race comment. (sashiko AI)
v4: Track all idle cids and mask with self_cids at the dispatch pick, dropping the reseed. (sashiko AI)
v3: Dispatch IMMED flags, repartition accounting order, partition-input snapshot. (sashiko AI)
v2: Use __sync_fetch_and_add() for the shared nr_dsps counter. (sashiko AI)

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>

show more ...

sched_ext: Add shard boundaries to scx_bpf_cid_override()

An overridden cid mapping invalidates the auto-generated shard layout, so
the override call has to provide both. Extend scx_bpf_cid_override

sched_ext: Add shard boundaries to scx_bpf_cid_override()

An overridden cid mapping invalidates the auto-generated shard layout, so
the override call has to provide both. Extend scx_bpf_cid_override() with a
shard_start[] array that lists the first cid of each shard (starting at 0,
strictly increasing, last shard implicitly extends to num_possible_cpus()).

A scheduler that wants only custom shards with the auto-generated cid
mapping can read the current mapping and pass it back unchanged.

Overridden shards can span NUMA nodes, so scx_shard_node[] is rebuilt by
majority count: each shard is assigned to the node that owns the most cpus
in it.

v2: Snapshot the caller's cpu_to_cid/shard_start arrays before validating. (sashiko AI)

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>

show more ...

sched_ext: Add ops.init_cids() to finalize the cid layout before init

A cid-form scheduler that calls scx_bpf_cid_override() to install a custom
cid layout can only do so from ops.init(). Enable-pat

sched_ext: Add ops.init_cids() to finalize the cid layout before init

A cid-form scheduler that calls scx_bpf_cid_override() to install a custom
cid layout can only do so from ops.init(). Enable-path setup that depends on
the cid layout thus has to run after ops.init(), and ops.init() itself can't
use anything derived from the final layout, which turned out to be too
restrictive.

Add an ops.init_cids() callback dedicated to finalizing the cid layout. It
runs before the rest of the enable-path setup, so the final layout is in
effect for everything that follows including ops.init(), which now runs
after the arena pool and cmask scratch allocations.

scx_bpf_cid_override() is restricted to ops.init_cids() at load time. It
sits in a kfunc set gated by SCX_KF_ALLOW_INIT_CIDS, a flag set only on the
init_cids op, so the verifier rejects a call from any other context. The
runtime root-only check is dropped as ops.init_cids() only runs during root
enable.

The qmap demo moves its override into a dedicated qmap_init_cids() and,
while at it, introduces an enum for the cid override modes instead of
hard-coded integers.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>

show more ...

tools/sched_ext: scx_qmap - Use bare u64/u32/s32 integer types

scx_qmap.c and the shared scx_qmap.h mixed __u64/__u32/__s32 with the bare
typedefs that scx/common.h provides. Convert the remaining _

tools/sched_ext: scx_qmap - Use bare u64/u32/s32 integer types

scx_qmap.c and the shared scx_qmap.h mixed __u64/__u32/__s32 with the bare
typedefs that scx/common.h provides. Convert the remaining __-prefixed
integer types to the bare forms for consistency. The struct fields become
bare u64 (uint64_t), so the stats printfs that fed them to %llu now cast to
unsigned long long. No functional change.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>

show more ...

Merge tag 'timers-v7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/daniel.lezcano/linux into timers/clocksource

- Use designated initializers for sh_mtu2, sh_cmt, and sh_tmu, and
dro

Merge tag 'timers-v7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/daniel.lezcano/linux into timers/clocksource

- Use designated initializers for sh_mtu2, sh_cmt, and sh_tmu, and
drop the unused initializer in the platform_device_id table for
sh_mtu2 (Uwe Kleine-König)

- Remove redundant dev_err()/dev_err_probe() messages when
devm_request_*_irq() fails, as the helper already logs an error
message (Pan Chuang)

- Fix a boot hang on Allwinner D1 when a forced minimum delta is used
with the sun4i timer (Felix Yan)

- Fix an IRQ leak in the cpuhp_setup_state() error path by freeing the
IRQ on failure in the NXP PIT driver (WenTao Liang)

- Fix incorrect unmapping of shared MMIO between the clocksource and
clockevent drivers. If one of them fails to initialize, the error
path unmaps the shared MMIO region, leaving the other driver with an
invalid mapping on clps711x (Guangshuo Li)

- Make the samsung_pwm driver compatible with PREEMPT_RT by replacing
regular spinlocks with raw_spinlock_t in atomic contexts (Marek
Szyprowski)

- Use __raw_readl() and __raw_writel() instead of ioread32() and
iowrite32() to support SWAP_IO_SPACE in the rtl-otto driver (Rustam
Adilov)

- Fix a missing clk_disable_unprepare() call in the timer
initialization error path of the Armada driver (Yuho Choi)

Link: https://lore.kernel.org/lkml/75feea31-683d-45a1-87f4-ab045e0152ae@oss.qualcomm.com

show more ...

Merge branch 'next' into for-linus

Prepare input updates for 7.3 merge window.

Merge tag 'v7.2-rc3' into next

Sync up with mainline to pull in stable fixes to avoid merge conflicts.

Merge drm/drm-fixes into drm-misc-fixes

Pull in tag v7.2-rc1 so that drm-misc-fixes becomes useful again,
and drm-misc-next-fixes can be closed.

Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>

Merge commit '6beaec3aee9852438b89e4d7891caf5e84d45851' of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux into gpio/for-current

This pulls in the merge commit for MFD updates for v7.2.

Merge commit '6beaec3aee9852438b89e4d7891caf5e84d45851' of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux into gpio/for-current

This pulls in the merge commit for MFD updates for v7.2. The PR contains
a build-time dependency of one of the GPIO commits that will follow.

show more ...

Merge tag 'sched_ext-for-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext

Pull sched_ext updates from Tejun Heo:
"Most of this continues the in-development sub-scheduler support,

Merge tag 'sched_ext-for-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext

Pull sched_ext updates from Tejun Heo:
"Most of this continues the in-development sub-scheduler support, which
lets a root BPF scheduler delegate to nested sub-schedulers. The
dispatch-path building blocks landed in 7.1. A follow-up patchset in
development will complete enqueue-path support for hierarchical
scheduling. This cycle adds most of that infrastructure:

- Topological CPU IDs (cids): a dense, topology-ordered CPU numbering
where the CPUs of a core, LLC, or NUMA node form contiguous ranges,
so a topology unit becomes a (start, length) slice. Raw CPU numbers
are sparse and don't track topological closeness, which makes them
clumsy for sharding work across sub-schedulers and awkward in BPF.

- cmask: bitmaps windowed over a slice of cid space, so a
sub-scheduler can track, for example, the idle cids of its shard
without a full NR_CPUS cpumask.

- A struct_ops variant that cid-form sub-schedulers register with,
along with the cid-form kfuncs they call.

- BPF arena integration, which sub-scheduler support is built on. The
bpf-next additions let the kernel read and write the BPF
scheduler's arena directly, turning it into a real kernel/BPF
shared-memory channel. Shared state like the per-CPU cmask now
lives there.

- scx_qmap is reworked to exercise the new arena and cid interfaces.

Additionally:

- Exit-dump improvements: dump the faulting CPU first, expose the
exit CPU to BPF and userspace, and normalize the dump header.

- Misc kfuncs and cleanups: a task-ID lookup kfunc, __printf checking
on the error and dump formatters, header reorganization, and
assorted fixes"

* tag 'sched_ext-for-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext: (59 commits)
sched_ext: Add scx_arena_to_kaddr() / scx_kaddr_to_arena()
sched_ext: Make scx_bpf_kick_cid() return s32
sched_ext: Add scx_cmask_test() and scx_cmask_for_each_cid()
tools/sched_ext: Order single-cid cmask helpers as (cid, mask)
sched_ext: Order single-cid cmask helpers as (cid, mask)
selftests/sched_ext: Fix dsq_move_to_local check
sched_ext: Guard BPF arena helper calls to fix 32-bit build
sched_ext: idle: Fix errno loss in scx_idle_init()
sched_ext: Convert ops.set_cmask() to arena-resident cmask
sched_ext: Sub-allocator over kernel-claimed BPF arena pages
sched_ext: Require an arena for cid-form schedulers
sched_ext: Add cmask mask ops
sched_ext: Track bits[] storage size in struct scx_cmask
sched_ext: Rename scx_cmask.nr_bits to nr_cids
tools/sched_ext: scx_qmap: Fix qa arena placement
sched_ext: Mark !CONFIG_EXT_SUB_SCHED dummy stubs static inline
sched_ext: Replace tryget_task_struct() with get_task_struct()
sched_ext: Add scx_task_iter_relock() and use it in scx_root_enable_workfn()
sched_ext: Fix ops_cid layout assert
sched_ext: Use offsetofend on both sides of the ops_cid layout assert
...

show more ...

tools/sched_ext: scx_qmap: Port to cid-form struct_ops

Flip qmap's struct_ops to bpf_sched_ext_ops_cid. The kernel now passes
cids and cmasks to callbacks directly, so the per-callback cpu<->cid
tra

tools/sched_ext: scx_qmap: Port to cid-form struct_ops

Flip qmap's struct_ops to bpf_sched_ext_ops_cid. The kernel now passes
cids and cmasks to callbacks directly, so the per-callback cpu<->cid
translations that the prior patch added drop out and cpu_ctxs[] is
reindexed by cid. Cpu-form kfunc calls switch to their cid-form
counterparts.

The cpu-only kfuncs (idle/any pick, cpumask iteration) have no cid
substitute. Their callers already moved to cmask scans against
qa_idle_cids and taskc->cpus_allowed in the prior patch, so the kfunc
calls drop here without behavior changes.

set_cmask is wired up via cmask_copy_from_kernel() to copy the
kernel-supplied cmask into the arena-resident taskc cmask. The
cpuperf monitor iterates the cid-form perf kfuncs.

v4: Match scx_bpf_cid_override()'s 2-arg form, drop the shard test
plumbing, bound nr_cpu_ids for the verifier, and switch mode 3
from bad-mono to bad-range (Changwoo, Andrea).

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Cheng-Yang Chou <yphbchou0911@gmail.com>
Reviewed-by: Changwoo Min <changwoo@igalia.com>
Reviewed-by: Andrea Righi <arighi@nvidia.com>

show more ...

sched_ext: scx_qmap: replace FIFO queue maps with arena-backed lists

Arena simplifies verification and allows more natural programming.
Convert scx_qmap to arena as preparation for further sub-sched

sched_ext: scx_qmap: replace FIFO queue maps with arena-backed lists

Arena simplifies verification and allows more natural programming.
Convert scx_qmap to arena as preparation for further sub-sched work.

Replace the five BPF_MAP_TYPE_QUEUE maps with doubly-linked lists in
arena, threaded through task_ctx. Each queue is a struct qmap_fifo with
head/tail pointers and its own per-queue bpf_res_spin_lock.

qmap_dequeue() now properly removes tasks from the queue instead of
leaving stale entries for dispatch to skip.

v2:
- Remove duplicate QMAP_TOUCH_ARENA() in qmap_dump_task (Andrea).
- Update file-level description for arena-backed lists (Andrea).

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>

show more ...

sched_ext: scx_qmap: move task_ctx into a BPF arena slab

Arena simplifies verification and allows more natural programming.
Convert scx_qmap to arena as preparation for further sub-sched work.

Allo

sched_ext: scx_qmap: move task_ctx into a BPF arena slab

Arena simplifies verification and allows more natural programming.
Convert scx_qmap to arena as preparation for further sub-sched work.

Allocate per-task context from an arena slab instead of storing it
directly in task_storage. task_ctx_stor now holds an arena pointer to
the task's slab entry. Free entries form a singly-linked list protected
by bpf_res_spin_lock; slab exhaustion triggers scx_bpf_error().

The slab size is configurable via the new -N option (default 16384).

Also add bpf_res_spin_lock/unlock declarations to common.bpf.h.

Scheduling logic unchanged.

v2: Add task_ctx_t typedef for struct task_ctx __arena (Emil).

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>

show more ...

sched_ext: scx_qmap: move globals and cpu_ctx into a BPF arena map

Arena simplifies verification and allows more natural programming.
Convert scx_qmap to arena as preparation for further sub-sched w

sched_ext: scx_qmap: move globals and cpu_ctx into a BPF arena map

Arena simplifies verification and allows more natural programming.
Convert scx_qmap to arena as preparation for further sub-sched work.

Move scheduler state from BSS globals and a percpu array map
into a single BPF arena map. A shared struct qmap_arena is declared as
an __arena global so BPF accesses it directly and userspace reaches it
through skel->arena->qa.

Scheduling logic unchanged; only memory backing changes.

v2: Drop "mutable" from comments.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>

show more ...