Merge branch 'next' into for-linus

Prepare input updates for 6.11 merge window.

Merge tag 'v6.9' into next

Sync up with the mainline to bring in the new cleanup API.

Merge drm/drm-next into drm-intel-gt-next

Some display refactoring patches are needed in order to allow conflict-
less merging.

Signed-off-by: Tvrtko Ursulin <tursulin@ursulin.net>

Merge topic branches 'clkdev' and 'fixes' into for-linus

Merge drm/drm-next into drm-xe-next

Backmerging drm-next in order to get up-to-date and in particular
to access commit 9ca5facd0400f610f3f7f71aeb7fc0b949a48c67.

Signed-off-by: Thomas Hellström <tho

Merge drm/drm-next into drm-xe-next

Backmerging drm-next in order to get up-to-date and in particular
to access commit 9ca5facd0400f610f3f7f71aeb7fc0b949a48c67.

Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>

show more ...

Merge tag 'phy_dp_modes_6.10' into msm-next-lumag

Merge DisplayPort subnode API in order to allow DisplayPort driver to
configure the PHYs either to the DP or eDP mode, depending on hardware
configu

Merge tag 'phy_dp_modes_6.10' into msm-next-lumag

Merge DisplayPort subnode API in order to allow DisplayPort driver to
configure the PHYs either to the DP or eDP mode, depending on hardware
configuration.

Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>

show more ...

Merge drm/drm-next into drm-intel-next

Catching up on 6.9-rc2

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Merge drm/drm-next into drm-misc-next

Backmerging to get v6.9-rc2 changes into drm-misc-next.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>

Merge branch 'topic/sound-devel-6.10' into for-next

Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net

Cross-merge networking fixes after downstream PR.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>

Merge tag 'v6.9-rc1' into sched/core, to pick up fixes and to refresh the branch

Signed-off-by: Ingo Molnar <mingo@kernel.org>

Merge tag 'asoc-fix-v6.9-rc2' of https://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound into for-linus

ASoC: Fixes for v6.9

A relatively large set of fixes here, the biggest piece of it is a

Merge tag 'asoc-fix-v6.9-rc2' of https://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound into for-linus

ASoC: Fixes for v6.9

A relatively large set of fixes here, the biggest piece of it is a
series correcting some problems with the delay reporting for Intel SOF
cards but there's a bunch of other things. Everything here is driver
specific except for a fix in the core for an issue with sign extension
handling volume controls.

show more ...

Merge drm/drm-fixes into drm-misc-fixes

Backmerging to get drm-misc-fixes to the state of v6.9-rc1.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>

Merge tag 'mm-stable-2024-03-13-20-04' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

Pull MM updates from Andrew Morton:

- Sumanth Korikkar has taught s390 to allocate hotplug-time page

Merge tag 'mm-stable-2024-03-13-20-04' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

Pull MM updates from Andrew Morton:

- Sumanth Korikkar has taught s390 to allocate hotplug-time page frames
from hotplugged memory rather than only from main memory. Series
"implement "memmap on memory" feature on s390".

- More folio conversions from Matthew Wilcox in the series

"Convert memcontrol charge moving to use folios"
"mm: convert mm counter to take a folio"

- Chengming Zhou has optimized zswap's rbtree locking, providing
significant reductions in system time and modest but measurable
reductions in overall runtimes. The series is "mm/zswap: optimize the
scalability of zswap rb-tree".

- Chengming Zhou has also provided the series "mm/zswap: optimize zswap
lru list" which provides measurable runtime benefits in some
swap-intensive situations.

- And Chengming Zhou further optimizes zswap in the series "mm/zswap:
optimize for dynamic zswap_pools". Measured improvements are modest.

- zswap cleanups and simplifications from Yosry Ahmed in the series
"mm: zswap: simplify zswap_swapoff()".

- In the series "Add DAX ABI for memmap_on_memory", Vishal Verma has
contributed several DAX cleanups as well as adding a sysfs tunable to
control the memmap_on_memory setting when the dax device is
hotplugged as system memory.

- Johannes Weiner has added the large series "mm: zswap: cleanups",
which does that.

- More DAMON work from SeongJae Park in the series

"mm/damon: make DAMON debugfs interface deprecation unignorable"
"selftests/damon: add more tests for core functionalities and corner cases"
"Docs/mm/damon: misc readability improvements"
"mm/damon: let DAMOS feeds and tame/auto-tune itself"

- In the series "mm/mempolicy: weighted interleave mempolicy and sysfs
extension" Rakie Kim has developed a new mempolicy interleaving
policy wherein we allocate memory across nodes in a weighted fashion
rather than uniformly. This is beneficial in heterogeneous memory
environments appearing with CXL.

- Christophe Leroy has contributed some cleanup and consolidation work
against the ARM pagetable dumping code in the series "mm: ptdump:
Refactor CONFIG_DEBUG_WX and check_wx_pages debugfs attribute".

- Luis Chamberlain has added some additional xarray selftesting in the
series "test_xarray: advanced API multi-index tests".

- Muhammad Usama Anjum has reworked the selftest code to make its
human-readable output conform to the TAP ("Test Anything Protocol")
format. Amongst other things, this opens up the use of third-party
tools to parse and process out selftesting results.

- Ryan Roberts has added fork()-time PTE batching of THP ptes in the
series "mm/memory: optimize fork() with PTE-mapped THP". Mainly
targeted at arm64, this significantly speeds up fork() when the
process has a large number of pte-mapped folios.

- David Hildenbrand also gets in on the THP pte batching game in his
series "mm/memory: optimize unmap/zap with PTE-mapped THP". It
implements batching during munmap() and other pte teardown
situations. The microbenchmark improvements are nice.

- And in the series "Transparent Contiguous PTEs for User Mappings"
Ryan Roberts further utilizes arm's pte's contiguous bit ("contpte
mappings"). Kernel build times on arm64 improved nicely. Ryan's
series "Address some contpte nits" provides some followup work.

- In the series "mm/hugetlb: Restore the reservation" Breno Leitao has
fixed an obscure hugetlb race which was causing unnecessary page
faults. He has also added a reproducer under the selftest code.

- In the series "selftests/mm: Output cleanups for the compaction
test", Mark Brown did what the title claims.

- Kinsey Ho has added the series "mm/mglru: code cleanup and
refactoring".

- Even more zswap material from Nhat Pham. The series "fix and extend
zswap kselftests" does as claimed.

- In the series "Introduce cpu_dcache_is_aliasing() to fix DAX
regression" Mathieu Desnoyers has cleaned up and fixed rather a mess
in our handling of DAX on archiecctures which have virtually aliasing
data caches. The arm architecture is the main beneficiary.

- Lokesh Gidra's series "per-vma locks in userfaultfd" provides
dramatic improvements in worst-case mmap_lock hold times during
certain userfaultfd operations.

- Some page_owner enhancements and maintenance work from Oscar Salvador
in his series

"page_owner: print stacks and their outstanding allocations"
"page_owner: Fixup and cleanup"

- Uladzislau Rezki has contributed some vmalloc scalability
improvements in his series "Mitigate a vmap lock contention". It
realizes a 12x improvement for a certain microbenchmark.

- Some kexec/crash cleanup work from Baoquan He in the series "Split
crash out from kexec and clean up related config items".

- Some zsmalloc maintenance work from Chengming Zhou in the series

"mm/zsmalloc: fix and optimize objects/page migration"
"mm/zsmalloc: some cleanup for get/set_zspage_mapping()"

- Zi Yan has taught the MM to perform compaction on folios larger than
order=0. This a step along the path to implementaton of the merging
of large anonymous folios. The series is named "Enable >0 order folio
memory compaction".

- Christoph Hellwig has done quite a lot of cleanup work in the
pagecache writeback code in his series "convert write_cache_pages()
to an iterator".

- Some modest hugetlb cleanups and speedups in Vishal Moola's series
"Handle hugetlb faults under the VMA lock".

- Zi Yan has changed the page splitting code so we can split huge pages
into sizes other than order-0 to better utilize large folios. The
series is named "Split a folio to any lower order folios".

- David Hildenbrand has contributed the series "mm: remove
total_mapcount()", a cleanup.

- Matthew Wilcox has sought to improve the performance of bulk memory
freeing in his series "Rearrange batched folio freeing".

- Gang Li's series "hugetlb: parallelize hugetlb page init on boot"
provides large improvements in bootup times on large machines which
are configured to use large numbers of hugetlb pages.

- Matthew Wilcox's series "PageFlags cleanups" does that.

- Qi Zheng's series "minor fixes and supplement for ptdesc" does that
also. S390 is affected.

- Cleanups to our pagemap utility functions from Peter Xu in his series
"mm/treewide: Replace pXd_large() with pXd_leaf()".

- Nico Pache has fixed a few things with our hugepage selftests in his
series "selftests/mm: Improve Hugepage Test Handling in MM
Selftests".

- Also, of course, many singleton patches to many things. Please see
the individual changelogs for details.

* tag 'mm-stable-2024-03-13-20-04' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (435 commits)
mm/zswap: remove the memcpy if acomp is not sleepable
crypto: introduce: acomp_is_async to expose if comp drivers might sleep
memtest: use {READ,WRITE}_ONCE in memory scanning
mm: prohibit the last subpage from reusing the entire large folio
mm: recover pud_leaf() definitions in nopmd case
selftests/mm: skip the hugetlb-madvise tests on unmet hugepage requirements
selftests/mm: skip uffd hugetlb tests with insufficient hugepages
selftests/mm: dont fail testsuite due to a lack of hugepages
mm/huge_memory: skip invalid debugfs new_order input for folio split
mm/huge_memory: check new folio order when split a folio
mm, vmscan: retry kswapd's priority loop with cache_trim_mode off on failure
mm: add an explicit smp_wmb() to UFFDIO_CONTINUE
mm: fix list corruption in put_pages_list
mm: remove folio from deferred split list before uncharging it
filemap: avoid unnecessary major faults in filemap_fault()
mm,page_owner: drop unnecessary check
mm,page_owner: check for null stack_record before bumping its refcount
mm: swap: fix race between free_swap_and_cache() and swapoff()
mm/treewide: align up pXd_leaf() retval across archs
mm/treewide: drop pXd_large()
...

show more ...

tools/mm: add thpmaps script to dump THP usage info

With the proliferation of large folios for file-backed memory, and more
recently the introduction of multi-size THP for anonymous memory, it is
be

tools/mm: add thpmaps script to dump THP usage info

With the proliferation of large folios for file-backed memory, and more
recently the introduction of multi-size THP for anonymous memory, it is
becoming useful to be able to see exactly how large folios are mapped into
processes. For some architectures (e.g. arm64), if most memory is mapped
using contpte-sized and -aligned blocks, TLB usage can be optimized so
it's useful to see where these requirements are and are not being met.

thpmaps is a Python utility that reads /proc/<pid>/smaps,
/proc/<pid>/pagemap and /proc/kpageflags to print information about how
transparent huge pages (both file and anon) are mapped to a specified
process or cgroup. It aims to help users debug and optimize their
workloads. In future we may wish to introduce stats directly into the
kernel (e.g. smaps or similar), but for now this provides a short term
solution without the need to introduce any new ABI.

Run with help option for a full listing of the arguments:

# ./thpmaps --help

--8<--
usage: thpmaps [-h] [--pid pid | --cgroup path] [--rollup]
[--cont size[KMG]] [--inc-smaps] [--inc-empty]
[--periodic sleep_ms]

Prints information about how transparent huge pages are mapped, either
system-wide, or for a specified process or cgroup.

When run with --pid, the user explicitly specifies the set of pids to
scan. e.g. "--pid 10 [--pid 134 ...]". When run with --cgroup, the user
passes either a v1 or v2 cgroup and all pids that belong to the cgroup
subtree are scanned. When run with neither --pid nor --cgroup, the full
set of pids on the system is gathered from /proc and scanned as if the
user had provided "--pid 1 --pid 2 ...".

A default set of statistics is always generated for THP mappings.
However, it is also possible to generate additional statistics for
"contiguous block mappings" where the block size is user-defined.

Statistics are maintained independently for anonymous and file-backed
(pagecache) memory and are shown both in kB and as a percentage of either
total anonymous or total file-backed memory as appropriate.

THP Statistics
--------------

Statistics are always generated for fully- and contiguously-mapped THPs
whose mapping address is aligned to their size, for each <size> supported
by the system. Separate counters describe THPs mapped by PTE vs those
mapped by PMD. (Although note a THP can only be mapped by PMD if it is
PMD-sized):

- anon-thp-pte-aligned-<size>kB
- file-thp-pte-aligned-<size>kB
- anon-thp-pmd-aligned-<size>kB
- file-thp-pmd-aligned-<size>kB

Similarly, statistics are always generated for fully- and contiguously-
mapped THPs whose mapping address is *not* aligned to their size, for each
<size> supported by the system. Due to the unaligned mapping, it is
impossible to map by PMD, so there are only PTE counters for this case:

- anon-thp-pte-unaligned-<size>kB
- file-thp-pte-unaligned-<size>kB

Statistics are also always generated for mapped pages that belong to a THP
but where the is THP is *not* fully- and contiguously- mapped. These
"partial" mappings are all counted in the same counter regardless of the
size of the THP that is partially mapped:

- anon-thp-pte-partial
- file-thp-pte-partial

Contiguous Block Statistics
---------------------------

An optional, additional set of statistics is generated for every
contiguous block size specified with `--cont <size>`. These statistics
show how much memory is mapped in contiguous blocks of <size> and also
aligned to <size>. A given contiguous block must all belong to the same
THP, but there is no requirement for it to be the *whole* THP. Separate
counters describe contiguous blocks mapped by PTE vs those mapped by PMD:

- anon-cont-pte-aligned-<size>kB
- file-cont-pte-aligned-<size>kB
- anon-cont-pmd-aligned-<size>kB
- file-cont-pmd-aligned-<size>kB

As an example, if monitoring 64K contiguous blocks (--cont 64K), there are
a number of sources that could provide such blocks: a fully- and
contiguously-mapped 64K THP that is aligned to a 64K boundary would
provide 1 block. A fully- and contiguously-mapped 128K THP that is
aligned to at least a 64K boundary would provide 2 blocks. Or a 128K THP
that maps its first 100K, but contiguously and starting at a 64K boundary
would provide 1 block. A fully- and contiguously-mapped 2M THP would
provide 32 blocks. There are many other possible permutations.

options:
-h, --help show this help message and exit
--pid pid Process id of the target process. Maybe issued
multiple times to scan multiple processes. --pid
and --cgroup are mutually exclusive. If neither
are provided, all processes are scanned to
provide system-wide information.
--cgroup path Path to the target cgroup in sysfs. Iterates
over every pid in the cgroup and its children.
--pid and --cgroup are mutually exclusive. If
neither are provided, all processes are scanned
to provide system-wide information.
--rollup Sum the per-vma statistics to provide a summary
over the whole system, process or cgroup.
--cont size[KMG] Adds stats for memory that is mapped in
contiguous blocks of <size> and also aligned to
<size>. May be issued multiple times to track
multiple sized blocks. Useful to infer e.g.
arm64 contpte and hpa mappings. Size must be a
power-of-2 number of pages.
--inc-smaps Include all numerical, additive
/proc/<pid>/smaps stats in the output.
--inc-empty Show all statistics including those whose value
is 0.
--periodic sleep_ms Run in a loop, polling every sleep_ms
milliseconds.

Requires root privilege to access pagemap and kpageflags.
--8<--

Example command to summarise fully and partially mapped THPs and 64K
contiguous blocks over all VMAs in all processes in the system
(--inc-empty forces printing stats that are 0):

# ./thpmaps --cont 64K --rollup --inc-empty

--8<--
anon-thp-pmd-aligned-2048kB: 139264 kB ( 6%)
file-thp-pmd-aligned-2048kB: 0 kB ( 0%)
anon-thp-pte-aligned-16kB: 0 kB ( 0%)
anon-thp-pte-aligned-32kB: 0 kB ( 0%)
anon-thp-pte-aligned-64kB: 72256 kB ( 3%)
anon-thp-pte-aligned-128kB: 0 kB ( 0%)
anon-thp-pte-aligned-256kB: 0 kB ( 0%)
anon-thp-pte-aligned-512kB: 0 kB ( 0%)
anon-thp-pte-aligned-1024kB: 0 kB ( 0%)
anon-thp-pte-aligned-2048kB: 0 kB ( 0%)
anon-thp-pte-unaligned-16kB: 0 kB ( 0%)
anon-thp-pte-unaligned-32kB: 0 kB ( 0%)
anon-thp-pte-unaligned-64kB: 0 kB ( 0%)
anon-thp-pte-unaligned-128kB: 0 kB ( 0%)
anon-thp-pte-unaligned-256kB: 0 kB ( 0%)
anon-thp-pte-unaligned-512kB: 0 kB ( 0%)
anon-thp-pte-unaligned-1024kB: 0 kB ( 0%)
anon-thp-pte-unaligned-2048kB: 0 kB ( 0%)
anon-thp-pte-partial: 63232 kB ( 3%)
file-thp-pte-aligned-16kB: 809024 kB (47%)
file-thp-pte-aligned-32kB: 43168 kB ( 3%)
file-thp-pte-aligned-64kB: 98496 kB ( 6%)
file-thp-pte-aligned-128kB: 17536 kB ( 1%)
file-thp-pte-aligned-256kB: 0 kB ( 0%)
file-thp-pte-aligned-512kB: 0 kB ( 0%)
file-thp-pte-aligned-1024kB: 0 kB ( 0%)
file-thp-pte-aligned-2048kB: 0 kB ( 0%)
file-thp-pte-unaligned-16kB: 21712 kB ( 1%)
file-thp-pte-unaligned-32kB: 704 kB ( 0%)
file-thp-pte-unaligned-64kB: 896 kB ( 0%)
file-thp-pte-unaligned-128kB: 44928 kB ( 3%)
file-thp-pte-unaligned-256kB: 0 kB ( 0%)
file-thp-pte-unaligned-512kB: 0 kB ( 0%)
file-thp-pte-unaligned-1024kB: 0 kB ( 0%)
file-thp-pte-unaligned-2048kB: 0 kB ( 0%)
file-thp-pte-partial: 9252 kB ( 1%)
anon-cont-pmd-aligned-64kB: 139264 kB ( 6%)
file-cont-pmd-aligned-64kB: 0 kB ( 0%)
anon-cont-pte-aligned-64kB: 100672 kB ( 4%)
file-cont-pte-aligned-64kB: 161856 kB ( 9%)
--8<--

Link: https://lkml.kernel.org/r/20240116141235.960842-1-ryan.roberts@arm.com
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
Tested-by: Barry Song <v-songbaohua@oppo.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Kefeng Wang <wangkefeng.wang@huawei.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: William Kucharski <william.kucharski@oracle.com>
Cc: Zenghui Yu <yuzenghui@huawei.com>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

show more ...