scripts/jobserver-exec: propagate child exit status

main() called JobserverExec().run() and discarded its return value,
then the script exited with the implicit status 0. As a result, any
Makefile t

scripts/jobserver-exec: propagate child exit status

main() called JobserverExec().run() and discarded its return value,
then the script exited with the implicit status 0. As a result, any
Makefile that wired a build step through jobserver-exec saw the step
silently succeed even when the wrapped command had failed.

Two in-tree callers were affected:

Documentation/devicetree/bindings/Makefile
cmd_chk_style runs a python checker via jobserver-exec and uses
"&& touch $@ || true" so failures leave the stamp file untouched
and the next make rerun reports them again. The swallowed exit
code made the stamp file get created even on failure, caching the
failed run and hiding the reported issues until the inputs change.

scripts/Makefile.vmlinux_o
cmd_gen_initcalls_lds runs scripts/generate_initcall_order.pl via
jobserver-exec; a perl failure was masked by the wrapper.

Return the subprocess exit code from main() and pass it to sys.exit()
so the wrapped command's status reaches make.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Link: https://patch.msgid.link/660368ca16e2d3845577a9fd157d2f37f0e09e85.1779908995.git.daniel@makrotopia.org
Signed-off-by: Rob Herring (Arm) <robh@kernel.org>

show more ...

docs: Move the python libraries to tools/lib/python

"scripts/lib" was always a bit of an awkward place for Python modules. We
already have tools/lib; create a tools/lib/python, move the libraries
t

docs: Move the python libraries to tools/lib/python

"scripts/lib" was always a bit of an awkward place for Python modules. We
already have tools/lib; create a tools/lib/python, move the libraries
there, and update the users accordingly.

While at it, move the contents of tools/docs/lib. Rather than make another
directory, just put these documentation-oriented modules under "kdoc".

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251110220430.726665-2-corbet@lwn.net>

show more ...

scripts/jobserver-exec: add a help message

Currently, calling it without an argument shows an ugly error
message. Instead, print a message using pythondoc as description.

Signed-off-by: Mauro Carva

scripts/jobserver-exec: add a help message

Currently, calling it without an argument shows an ugly error
message. Instead, print a message using pythondoc as description.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <64b0339eac54ac0f2b3de3667a7f4f5becb1c6ae.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>

show more ...

scripts/jobserver-exec: move its class to the lib directory

To make it easier to be re-used, move the JobserverExec class
to the library directory.

Signed-off-by: Mauro Carvalho Chehab <mchehab+hua

scripts/jobserver-exec: move its class to the lib directory

To make it easier to be re-used, move the JobserverExec class
to the library directory.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <6be7b161b6c005a9807162ebfd239af6a4e6fa47.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>

show more ...

scripts/jobserver-exec: move the code to a class

Convert the code inside jobserver-exec to a class and
properly document it.

Using a class allows reusing the jobserver logic on other
scripts.

Whil

scripts/jobserver-exec: move the code to a class

Convert the code inside jobserver-exec to a class and
properly document it.

Using a class allows reusing the jobserver logic on other
scripts.

While the main code remains unchanged, being compatible with
Python 2.6 and 3.0+, its coding style now follows a more
modern standard, having tabs replaced by a 4-spaces
indent, passing autopep8, black and pylint.

The code allows using a pythonic way to enter/exit a python
code, e.g. it now supports:

with JobserverExec() as jobserver:
jobserver.run(sys.argv[1:])

With the new code, the __exit__() function should ensure
that the jobserver slot will be closed at the end, even if
something bad happens somewhere.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <4749921b75d4e0bd85a25d4d94aa2c940fad084e.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>

show more ...

scripts: support GNU make 4.4 in jobserver-exec

Starting with GNU make 4.4, --jobserver-auth newly uses named
pipe (fifo) instead of part of opened file descriptors:
https://www.gnu.org/software/mak

scripts: support GNU make 4.4 in jobserver-exec

Starting with GNU make 4.4, --jobserver-auth newly uses named
pipe (fifo) instead of part of opened file descriptors:
https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html

Support also the new format.

Signed-off-by: Martin Liska <mliska@suse.cz>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

show more ...

scripts/jobserver-exec: parse the last --jobserver-auth= option

In the GNU Make manual, the section "Sharing Job Slots with GNU make"
says:

Be aware that the MAKEFLAGS variable may contain mult

scripts/jobserver-exec: parse the last --jobserver-auth= option

In the GNU Make manual, the section "Sharing Job Slots with GNU make"
says:

Be aware that the MAKEFLAGS variable may contain multiple instances
of the --jobserver-auth= option. Only the last instance is relevant.

Take the last element of the array, not the first.

Link: https://www.gnu.org/software/make/manual/html_node/Job-Slots.html
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

show more ...

scripts/jobserver-exec: Fix a typo ("envirnoment")

Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

kbuild: remove PYTHON variable

Python retired in 2020, and some distributions do not provide the
'python' command any more.

As in commit 51839e29cb59 ("scripts: switch explicitly to Python 3"),
we

kbuild: remove PYTHON variable

Python retired in 2020, and some distributions do not provide the
'python' command any more.

As in commit 51839e29cb59 ("scripts: switch explicitly to Python 3"),
we need to use more specific 'python3' to invoke scripts even if they
are written in a way compatible with both Python 2 and 3.

This commit removes the variable 'PYTHON', and switches the existing
users to 'PYTHON3'.

BTW, PEP 394 (https://www.python.org/dev/peps/pep-0394/) is a helpful
material.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

show more ...

docs, parallelism: Rearrange how jobserver reservations are made

Rasmus correctly observed that the existing jobserver reservation only
worked if no other build targets were specified. The correct a

docs, parallelism: Rearrange how jobserver reservations are made

Rasmus correctly observed that the existing jobserver reservation only
worked if no other build targets were specified. The correct approach
is to hold the jobserver slots until sphinx has finished. To fix this,
the following changes are made:

- refactor (and rename) scripts/jobserver-exec to set an environment
variable for the maximally reserved jobserver slots and exec a
child, to release the slots on exit.

- create Documentation/scripts/parallel-wrapper.sh which examines both
$PARALLELISM and the detected "-jauto" logic from Documentation/Makefile
to decide sphinx's final -j argument.

- chain these together in Documentation/Makefile

Suggested-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Link: https://lore.kernel.org/lkml/eb25959a-9ec4-3530-2031-d9d716b40b20@rasmusvillemoes.dk
Signed-off-by: Kees Cook <keescook@chromium.org>
Link: https://lore.kernel.org/r/20191121205929.40371-4-keescook@chromium.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>

show more ...