Merge tag 'kbuild-v5.6' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild updates from Masahiro Yamada:

- detect missing include guard in UAPI headers

- do not

Merge tag 'kbuild-v5.6' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild updates from Masahiro Yamada:

- detect missing include guard in UAPI headers

- do not create orphan built-in.a or obj-y objects

- generate modules.builtin more simply, and drop tristate.conf

- simplify built-in initramfs creation

- make linux-headers deb package thinner

- optimize the deb package build script

- misc cleanups

* tag 'kbuild-v5.6' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (34 commits)
builddeb: split libc headers deployment out into a function
builddeb: split kernel headers deployment out into a function
builddeb: remove redundant make for ARCH=um
builddeb: avoid invoking sub-shells where possible
builddeb: remove redundant $objtree/
builddeb: match temporary directory name to the package name
builddeb: remove unneeded files in hdrobjfiles for headers package
kbuild: use -S instead of -E for precise cc-option test in Kconfig
builddeb: allow selection of .deb compressor
kbuild: remove 'Building modules, stage 2.' log
kbuild: remove *.tmp file when filechk fails
kbuild: remove PYTHON2 variable
modpost: assume STT_SPARC_REGISTER is defined
gen_initramfs.sh: remove intermediate cpio_list on errors
initramfs: refactor the initramfs build rules
gen_initramfs.sh: always output cpio even without -o option
initramfs: add default_cpio_list, and delete -d option support
initramfs: generate dependency list and cpio at the same time
initramfs: specify $(src)/gen_initramfs.sh as a prerequisite in Makefile
initramfs: make initramfs compression choice non-optional
...

show more ...

Merge branch 'next' into for-linus

Prepare input updates for 5.6 merge window.

kbuild: remove *.tmp file when filechk fails

Bartosz Golaszewski reports that when "make {menu,n,g,x}config" fails
due to missing packages, a temporary file is left over, which is not
ignored by git

kbuild: remove *.tmp file when filechk fails

Bartosz Golaszewski reports that when "make {menu,n,g,x}config" fails
due to missing packages, a temporary file is left over, which is not
ignored by git.

For example, if GTK+ is not installed:

$ make gconfig
*
* Unable to find the GTK+ installation. Please make sure that
* the GTK+ 2.0 development package is correctly installed.
* You need gtk+-2.0 gmodule-2.0 libglade-2.0
*
scripts/kconfig/Makefile:208: recipe for target 'scripts/kconfig/gconf-cfg' failed
make[1]: *** [scripts/kconfig/gconf-cfg] Error 1
Makefile:567: recipe for target 'gconfig' failed
make: *** [gconfig] Error 2
$ git status
HEAD detached at v5.4
Untracked files:
(use "git add <file>..." to include in what will be committed)

scripts/kconfig/gconf-cfg.tmp

nothing added to commit but untracked files present (use "git add" to track)

This is because the check scripts are run with filechk, which misses
to clean up the temporary file on failure.

When the line

{ $(filechk_$(1)); } > $@.tmp;

... fails, it exits immediately due to the 'set -e'. Use trap to make
sure to delete the temporary file on exit.

For extra safety, I replaced $@.tmp with $(dot-target).tmp to make it
a hidden file.

Reported-by: Bartosz Golaszewski <bgolaszewski@baylibre.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

show more ...

Merge tag 'v5.5-rc5' into next

Sync up with mainline to get SPI "delay" API changes.

kbuild: create modules.builtin without Makefile.modbuiltin or tristate.conf

Commit bc081dd6e9f6 ("kbuild: generate modules.builtin") added
infrastructure to generate modules.builtin, the list of all

kbuild: create modules.builtin without Makefile.modbuiltin or tristate.conf

Commit bc081dd6e9f6 ("kbuild: generate modules.builtin") added
infrastructure to generate modules.builtin, the list of all
builtin modules.

Basically, it works like this:

- Kconfig generates include/config/tristate.conf, the list of
tristate CONFIG options with a value in a capital letter.

- scripts/Makefile.modbuiltin makes Kbuild descend into
directories to collect the information of builtin modules.

I am not a big fan of it because Kbuild ends up with traversing
the source tree twice.

I am not sure how perfectly it should work, but this approach cannot
avoid false positives; even if the relevant CONFIG option is tristate,
some Makefiles forces obj-m to obj-y.

Some examples are:

arch/powerpc/platforms/powermac/Makefile:
obj-$(CONFIG_NVRAM:m=y) += nvram.o

net/ipv6/Makefile:
obj-$(subst m,y,$(CONFIG_IPV6)) += inet6_hashtables.o

net/netlabel/Makefile:
obj-$(subst m,y,$(CONFIG_IPV6)) += netlabel_calipso.o

Nobody has complained about (or noticed) it, so it is probably fine to
have false positives in modules.builtin.

This commit simplifies the implementation. Let's exploit the fact
that every module has MODULE_LICENSE(). (modpost shows a warning if
MODULE_LICENSE is missing. If so, 0-day bot would already have blocked
such a module.)

I added MODULE_FILE to <linux/module.h>. When the code is being compiled
as builtin, it will be filled with the file path of the module, and
collected into modules.builtin.info. Then, scripts/link-vmlinux.sh
extracts the list of builtin modules out of it.

This new approach fixes the false-positives above, but adds another
type of false-positives; non-modular code may have MODULE_LICENSE()
by mistake. This is not a big deal, it is just the code is always
orphan. We can clean it up if we like. You can see cleanup examples by:

$ git log --grep='make.* explicitly non-modular'

To sum up, this commits deletes lots of code, but still produces almost
equivalent results. Please note it does not increase the vmlinux size at
all. As you can see in include/asm-generic/vmlinux.lds.h, the .modinfo
section is discarded in the link stage.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

show more ...

kbuild: add stringify helper to quote a string passed to C files

Make $(squote)$(quote)...$(quote)$(squote) a helper macro.
I will reuse it in the next commit.

Signed-off-by: Masahiro Yamada <masah

kbuild: add stringify helper to quote a string passed to C files

Make $(squote)$(quote)...$(quote)$(squote) a helper macro.
I will reuse it in the next commit.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

show more ...

Merge drm/drm-next into drm-misc-next

Requested, and we need v5.5-rc1 backported as our current branch is still based on v5.4.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>

Merge tag 'perf-urgent-for-mingo-5.5-20191216' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/urgent

Pull perf/urgent fixes:

perf top:

Arnaldo Carvalho de Melo:

- Do not

Merge tag 'perf-urgent-for-mingo-5.5-20191216' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/urgent

Pull perf/urgent fixes:

perf top:

Arnaldo Carvalho de Melo:

- Do not bail out when perf_env__read_cpuid() returns ENOSYS, which
has been reported happening on aarch64.

perf metricgroup:

Kajol Jain:

- Fix printing event names of metric group with multiple events

vendor events:

x86:

Ravi Bangoria:

- Fix Kernel_Utilization metric.

s390:

Ed Maste:

- Fix counter long description for DTLB1_GPAGE_WRITES and L1D_RO_EXCL_WRITES.

perf header:

Michael Petlan:

- Fix false warning when there are no duplicate cache entries

libtraceevent:

Sudip Mukherjee:

- Allow custom libdir path

API headers:

Arnaldo Carvalho de Melo:

- Sync linux/kvm.h with the kernel sources.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>

show more ...

Merge remote-tracking branch 'torvalds/master' into perf/urgent

To pick up BPF fixes to allow a clean 'make -C tools/perf build-test':

7c3977d1e804 libbpf: Fix sym->st_value print on 32-bit arche

Merge remote-tracking branch 'torvalds/master' into perf/urgent

To pick up BPF fixes to allow a clean 'make -C tools/perf build-test':

7c3977d1e804 libbpf: Fix sym->st_value print on 32-bit arches
1fd450f99272 libbpf: Fix up generation of bpf_helper_defs.h

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

Merge drm/drm-next into drm-intel-next-queued

Sync up with v5.5-rc1 to get the updated lock_release() API among other
things. Fix the conflict reported by Stephen Rothwell [1].

[1] http://lore.kern

Merge drm/drm-next into drm-intel-next-queued

Sync up with v5.5-rc1 to get the updated lock_release() API among other
things. Fix the conflict reported by Stephen Rothwell [1].

[1] http://lore.kernel.org/r/20191210093957.5120f717@canb.auug.org.au

Signed-off-by: Jani Nikula <jani.nikula@intel.com>

show more ...

Merge tag 'v5.5-rc1' into core/kprobes, to resolve conflicts

Signed-off-by: Ingo Molnar <mingo@kernel.org>

Merge branch 'linus' into sched/urgent, to pick up the latest before merging new patches

Signed-off-by: Ingo Molnar <mingo@kernel.org>

Merge mainline/master into arm/fixes

This brings in the mainline tree right after armsoc contents was merged
this release cycle, so that we can re-run savedefconfig, etc.

Signed-off-by: Olof Johans

Merge mainline/master into arm/fixes

This brings in the mainline tree right after armsoc contents was merged
this release cycle, so that we can re-run savedefconfig, etc.

Signed-off-by: Olof Johansson <olof@lixom.net>

show more ...

Merge tag 'kbuild-v5.5' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild updates from Masahiro Yamada:

- remove unneeded asm headers from hexagon, ia64

- add '

Merge tag 'kbuild-v5.5' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild updates from Masahiro Yamada:

- remove unneeded asm headers from hexagon, ia64

- add 'dir-pkg' target, which works like 'tar-pkg' but skips archiving

- add 'helpnewconfig' target, which shows help for new CONFIG options

- support 'make nsdeps' for external modules

- make rebuilds faster by deleting $(wildcard $^) checks

- remove compile tests for kernel-space headers

- refactor modpost to simplify modversion handling

- make single target builds faster

- optimize and clean up scripts/kallsyms.c

- refactor various Makefiles and scripts

* tag 'kbuild-v5.5' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (59 commits)
MAINTAINERS: update Kbuild/Kconfig maintainer's email address
scripts/kallsyms: remove redundant initializers
scripts/kallsyms: put check_symbol_range() calls close together
scripts/kallsyms: make check_symbol_range() void function
scripts/kallsyms: move ignored symbol types to is_ignored_symbol()
scripts/kallsyms: move more patterns to the ignored_prefixes array
scripts/kallsyms: skip ignored symbols very early
scripts/kallsyms: add const qualifiers where possible
scripts/kallsyms: make find_token() return (unsigned char *)
scripts/kallsyms: replace prefix_underscores_count() with strspn()
scripts/kallsyms: add sym_name() to mitigate cast ugliness
scripts/kallsyms: remove unneeded length check for prefix matching
scripts/kallsyms: remove redundant is_arm_mapping_symbol()
scripts/kallsyms: set relative_base more effectively
scripts/kallsyms: shrink table before sorting it
scripts/kallsyms: fix definitely-lost memory leak
scripts/kallsyms: remove unneeded #ifndef ARRAY_SIZE
kbuild: make single target builds even faster
modpost: respect the previous export when 'exported twice' is warned
modpost: do not set ->preloaded for symbols from Module.symvers
...

show more ...

Merge branch 'for-5.5/core' into for-linus

- hid_have_special_driver[] cleanup for LED devices (Heiner Kallweit)
- HID parser improvements (Blaž Hrastnik, Candle Sun)

Merge branch 'next' into for-linus

Prepare input updates for 5.5 merge window.

Merge tag 'v5.4-rc8' into WIP.x86/mm, to pick up fixes

Signed-off-by: Ingo Molnar <mingo@kernel.org>

Merge tag 'spi-nor/for-5.5' into mtd/next

SPI NOR core changes:
- introduce 'struct spi_nor_controller_ops',
- clean the Register Operations methods,
- use dev_dbg insted of dev_err for low level in

Merge tag 'spi-nor/for-5.5' into mtd/next

SPI NOR core changes:
- introduce 'struct spi_nor_controller_ops',
- clean the Register Operations methods,
- use dev_dbg insted of dev_err for low level info,
- fix retlen handling in sst_write(),
- fix silent truncations in spi_nor_read and spi_nor_read_raw(),
- fix the clearing of QE bit on lock()/unlock(),
- rework the disabling of the block write protection,
- rework the Quad Enable methods,
- make sure nor->spimem and nor->controller_ops are mutually exclusive,
- set default Quad Enable method for ISSI flashes,
- add support for few flashes.

SPI NOR controller drivers changes:
- intel-spi:
- support chips without software sequencer,
- add support for Intel Cannon Lake and Intel Comet Lake-H flashes.

show more ...

Merge tag 'tee-fixes-for-v5.4' of git://git.linaro.org/people/jens.wiklander/linux-tee into arm/fixes

Two OP-TE driver fixes:
- Add proper cleanup on optee_enumerate_devices() failure
- Make sure to

Merge tag 'tee-fixes-for-v5.4' of git://git.linaro.org/people/jens.wiklander/linux-tee into arm/fixes

Two OP-TE driver fixes:
- Add proper cleanup on optee_enumerate_devices() failure
- Make sure to register kernel allocations of dynamic shared memory

* tag 'tee-fixes-for-v5.4' of git://git.linaro.org/people/jens.wiklander/linux-tee: (591 commits)
tee: optee: fix device enumeration error handling
tee: optee: Fix dynamic shm pool allocations
Linux 5.4-rc3
tracing: Initialize iter->seq after zeroing in tracing_read_pipe()
tracing/hwlat: Don't ignore outer-loop duration when calculating max_latency
tracing/hwlat: Report total time spent in all NMIs during the sample
recordmcount: Fix nop_mcount() function
tracing: Do not create tracefs files if tracefs lockdown is in effect
tracing: Add locked_down checks to the open calls of files created for tracefs
tracing: Add tracing_check_open_get_tr()
tracing: Have trace events system open call tracing_open_generic_tr()
tracing: Get trace_array reference for available_tracers files
ftrace: Get a reference counter for the trace_array on filter files
tracefs: Revert ccbd54ff54e8 ("tracefs: Restrict tracefs when the kernel is locked down")
perf/x86/cstate: Add Tiger Lake CPU support
perf/x86/msr: Add Tiger Lake CPU support
perf/x86/intel: Add Tiger Lake CPU support
perf/x86/cstate: Update C-state counters for Ice Lake
perf/x86/msr: Add new CPU model numbers for Ice Lake
perf/x86/cstate: Add Comet Lake CPU support
...

Link: https://lore.kernel.org/r/20191115105353.GA26176@jax
Signed-off-by: Olof Johansson <olof@lixom.net>

show more ...

Merge branch 'linus' into x86/hyperv

Pick up upstream fixes to avoid conflicts.

kbuild: rename any-prereq to newer-prereqs

GNU Make manual says:

$?
The names of all the prerequisites that are newer than the target,
with spaces between them.

To reflect this, rena

kbuild: rename any-prereq to newer-prereqs

GNU Make manual says:

$?
The names of all the prerequisites that are newer than the target,
with spaces between them.

To reflect this, rename any-prereq to newer-prereqs, which is clearer
and more intuitive.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

show more ...

kbuild: drop $(wildcard $^) check in if_changed* for faster rebuild

The incremental build of Linux kernel is pretty slow when lots of
objects are compiled. The rebuild of allmodconfig may take a few

kbuild: drop $(wildcard $^) check in if_changed* for faster rebuild

The incremental build of Linux kernel is pretty slow when lots of
objects are compiled. The rebuild of allmodconfig may take a few
minutes even when none of the objects needs to be rebuilt.

The time-consuming part in the incremental build is the evaluation of
if_changed* macros since they are used in the recipes to compile C and
assembly source files into objects.

I notice the following code in if_changed* is expensive:

$(filter-out $(PHONY) $(wildcard $^),$^)

In the incremental build, every object has its .*.cmd file, which
contains the auto-generated list of included headers. So, $^ are
expanded into the long list of the source file + included headers,
and $(wildcard $^) checks whether they exist.

It may not be clear why this check exists there.

Here is the record of my research.

[1] The first code addition into Kbuild

This code dates back to 2002. It is the pre-git era. So, I copy-pasted
it from the historical git tree.

| commit 4a6db0791528c220655b063cf13fefc8470dbfee (HEAD)
| Author: Kai Germaschewski <kai@tp1.ruhr-uni-bochum.de>
| Date: Mon Jun 17 00:22:37 2002 -0500
|
| kbuild: Handle removed headers
|
| New and old way to handle dependencies would choke when a file
| #include'd by other files was removed, since the dependency on it was
| still recorded, but since it was gone, make has no idea what to do about
| it (and would complain with "No rule to make <file> ...")
|
| We now add targets for all the previously included files, so make will
| just ignore them if they disappear.
|
| diff --git a/Rules.make b/Rules.make
| index 6ef827d3df39..7db5301ea7db 100644
| --- a/Rules.make
| +++ b/Rules.make
| @@ -446,7 +446,7 @@ if_changed = $(if $(strip $? \
| # execute the command and also postprocess generated .d dependencies
| # file
|
| -if_changed_dep = $(if $(strip $? \
| +if_changed_dep = $(if $(strip $? $(filter-out FORCE $(wildcard $^),$^)\
| $(filter-out $(cmd_$(1)),$(cmd_$@))\
| $(filter-out $(cmd_$@),$(cmd_$(1)))),\
| @set -e; \
| diff --git a/scripts/fixdep.c b/scripts/fixdep.c
| index b5d7bee8efc7..db45bd1888c0 100644
| --- a/scripts/fixdep.c
| +++ b/scripts/fixdep.c
| @@ -292,7 +292,7 @@ void parse_dep_file(void *map, size_t len)
| exit(1);
| }
| memcpy(s, m, p-m); s[p-m] = 0;
| - printf("%s: \\\n", target);
| + printf("deps_%s := \\\n", target);
| m = p+1;
|
| clear_config();
| @@ -314,7 +314,8 @@ void parse_dep_file(void *map, size_t len)
| }
| m = p + 1;
| }
| - printf("\n");
| + printf("\n%s: $(deps_%s)\n\n", target, target);
| + printf("$(deps_%s):\n", target);
| }
|
| void print_deps(void)

The "No rule to make <file> ..." error can be solved by passing -MP to
the compiler, but I think the detection of header removal is a good
feature. When a header is removed, all source files that previously
included it should be re-compiled. This makes sure we has correctly
got rid of #include directives of it.

This is also related with the behavior of $?. The GNU Make manual says:

$?
The names of all the prerequisites that are newer than the target,
with spaces between them.

This does not explain whether a non-existent prerequisite is considered
to be newer than the target.

At this point of time, GNU Make 3.7x was used, where the $? did not
include non-existent prerequisites. Therefore,

$(filter-out FORCE $(wildcard $^),$^)

was useful to detect the header removal, and to rebuild the related
objects if it is the case.

[2] Change of $? behavior

Later, the behavior of $? was changed (fixed) to include prerequisites
that did not exist.

First, GNU Make commit 64e16d6c00a5 ("Various changes getting ready for
the release of 3.81.") changed it, but in the release test of 3.81, it
turned out to break the kernel build.

See these:

- http://lists.gnu.org/archive/html/bug-make/2006-03/msg00003.html
- https://savannah.gnu.org/bugs/?16002
- https://savannah.gnu.org/bugs/?16051

Then, GNU Make commit 6d8d9b74d9c5 ("Numerous updates to tests for
issues found on Cygwin and Windows.") reverted it for the 3.81 release
to give Linux kernel time to adjust to the new behavior.

After the 3.81 release, GNU Make commit 7595f38f62af ("Fixed a number
of documentation bugs, plus some build/install issues:") re-added it.

[3] Adjustment to the new $? behavior on Kbuild side

Meanwhile, the kernel build was changed by commit 4f1933620f57 ("kbuild:
change kbuild to not rely on incorrect GNU make behavior") to adjust to
the new $? behavior.

[4] GNU Make 3.82 released in 2010

GNU Make 3.82 was the first release that integrated the correct $?
behavior. At this point, Kbuild dealt with GNU Make versions with
different $? behaviors.

3.81 or older:
$? does not contain any non-existent prerequisite.
$(filter-out $(PHONY) $(wildcard $^),$^) was useful to detect
removed include headers.

3.82 or newer:
$? contains non-existent prerequisites. When a header is removed,
it appears in $?. $(filter-out $(PHONY) $(wildcard $^),$^) became
a redundant check.

With the correct $? behavior, we could have dropped the expensive
check for 3.82 or later, but we did not. (Maybe nobody noticed this
optimization.)

[5] The .SECONDARY special target trips up $?

Some time later, I noticed $? did not work as expected under some
circumstances. As above, $? should contain non-existent prerequisites,
but the ones specified as SECONDARY do not appear in $?.

I asked this in GNU Make ML, and it seems a bug:

https://lists.gnu.org/archive/html/bug-make/2019-01/msg00001.html

Since commit 8e9b61b293d9 ("kbuild: move .SECONDARY special target to
Kbuild.include"), all files, including headers listed in .*.cmd files,
are treated as secondary.

So, we are back into the incorrect $? behavior.

If we Kbuild want to react to the header removal, we need to keep
$(filter-out $(PHONY) $(wildcard $^),$^) but this makes the rebuild
so slow.

[Summary]

- I believe noticing the header removal and recompiling related objects
is a nice feature for the build system.

- If $? worked correctly, $(filter-out $(PHONY),$?) would be enough
to detect the header removal.

- Currently, $? does not work correctly when used with .SECONDARY,
and Kbuild is hit by this bug.

- I filed a bug report for this, but not fixed yet as of writing.

- Currently, the header removal is detected by the following expensive
code:

$(filter-out $(PHONY) $(wildcard $^),$^)

- I do not want to revert commit 8e9b61b293d9 ("kbuild: move
.SECONDARY special target to Kbuild.include"). Specifying
.SECONDARY globally is clean, and it matches to the Kbuild policy.

This commit proactively removes the expensive check since it makes the
incremental build faster. A downside is Kbuild will no longer be able
to notice the header removal.

You can confirm it by the full-build followed by a header removal, and
then re-build.

$ make defconfig all
[ full build ]
$ rm include/linux/device.h
$ make
CALL scripts/checksyscalls.sh
CALL scripts/atomic/check-atomics.sh
DESCEND objtool
CHK include/generated/compile.h
Kernel: arch/x86/boot/bzImage is ready (#11)
Building modules, stage 2.
MODPOST 12 modules

Previously, Kbuild noticed a missing header and emits a build error.
Now, Kbuild is fine with it. This is an unusual corner-case, not a big
deal. Once the $? bug is fixed in GNU Make, everything will work fine.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

show more ...

Merge tag 'v5.4-rc7' into sched/core, to pick up fixes

Signed-off-by: Ingo Molnar <mingo@kernel.org>

Merge tag 'v5.4-rc6' into gpio/for-next

Linux 5.4-rc6

Pull series refactoring quota enabling and disabling code.