Merge tag 'v5.1' into next

Sync up with mainline to bring in the latest APIs.

Merge branch 'linus' into x86/urgent, to pick up dependent changes

Signed-off-by: Ingo Molnar <mingo@kernel.org>

Merge tag 'xfs-5.2-merge-4' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux

Pull xfs updates from Darrick Wong:
"Here's a big pile of new stuff for XFS for 5.2. XFS has grown the
ability to repo

Merge tag 'xfs-5.2-merge-4' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux

Pull xfs updates from Darrick Wong:
"Here's a big pile of new stuff for XFS for 5.2. XFS has grown the
ability to report metadata health status to userspace after online
fsck checks the filesystem. The online metadata checking code is (I
really hope) feature complete with the addition of checks for the
global fs counters, though it'll remain EXPERIMENTAL for now.

There are also fixes for thundering herds of writeback completions and
some other deadlocks, fixes for theoretical integer overflow attacks
on space accounting, and removal of the long-defunct 'mntpt' option
which was deprecated in the mid-2000s and (it turns out) totally
broken since 2011 (and nobody complained...).

Summary:

- Fix some more buffer deadlocks when performing an unmount after a
hard shutdown.

- Fix some minor space accounting issues.

- Fix some use after free problems.

- Make the (undocumented) FITRIM behavior consistent with other
filesystems.

- Embiggen the xfs geometry ioctl's data structure.

- Introduce a new AG geometry ioctl.

- Introduce a new online health reporting infrastructure and ioctl
for userspace to query a filesystem's health status.

- Enhance online scrub and repair to update the health reports.

- Reduce thundering herd problems when writeback io completes.

- Fix some transaction reservation type errors.

- Fix integer overflow problems with delayed alloc reservation
counters.

- Fix some problems where we would exit to userspace without
unlocking.

- Fix inconsistent behavior when finishing deferred ops fails.

- Strengthen scrub to check incore data against ondisk metadata.

- Remove long-broken mntpt mount option.

- Add an online scrub function for the filesystem summary counters,
which should make online metadata scrub more or less feature
complete for now.

- Various cleanups"

* tag 'xfs-5.2-merge-4' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux: (38 commits)
xfs: change some error-less functions to void types
xfs: add online scrub for superblock counters
xfs: don't parse the mtpt mount option
xfs: always rejoin held resources during defer roll
xfs: add missing error check in xfs_prepare_shift()
xfs: scrub should check incore counters against ondisk headers
xfs: allow scrubbers to pause background reclaim
xfs: rename the speculative block allocation reclaim toggle functions
xfs: track delayed allocation reservations across the filesystem
xfs: fix broken bhold behavior in xrep_roll_ag_trans
xfs: unlock inode when xfs_ioctl_setattr_get_trans can't get transaction
xfs: kill the xfs_dqtrx_t typedef
xfs: widen inode delalloc block counter to 64-bits
xfs: widen quota block counters to 64-bit integers
xfs: abort unaligned nowait directio early
xfs: assert that we don't enter agfl freeing with a non-permanent transaction
xfs: make tr_growdata a permanent transaction
xfs: merge adjacent io completions of the same type
xfs: remove unused m_data_workqueue
xfs: implement per-inode writeback completion queues
...

show more ...

Merge tag 'v5.1-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux into mlx5-next

Linux 5.1-rc1

We forgot to reset the branch last merge window thus mlx5-next is outdated
and stil

Merge tag 'v5.1-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux into mlx5-next

Linux 5.1-rc1

We forgot to reset the branch last merge window thus mlx5-next is outdated
and still based on 5.0-rc2. This merge commit is needed to sync mlx5-next
branch with 5.1-rc1.

Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>

show more ...

xfs: implement per-inode writeback completion queues

When scheduling writeback of dirty file data in the page cache, XFS uses
IO completion workqueue items to ensure that filesystem metadata only
up

xfs: implement per-inode writeback completion queues

When scheduling writeback of dirty file data in the page cache, XFS uses
IO completion workqueue items to ensure that filesystem metadata only
updates after the write completes successfully. This is essential for
converting unwritten extents to real extents at the right time and
performing COW remappings.

Unfortunately, XFS queues each IO completion work item to an unbounded
workqueue, which means that the kernel can spawn dozens of threads to
try to handle the items quickly. These threads need to take the ILOCK
to update file metadata, which results in heavy ILOCK contention if a
large number of the work items target a single file, which is
inefficient.

Worse yet, the writeback completion threads get stuck waiting for the
ILOCK while holding transaction reservations, which can use up all
available log reservation space. When that happens, metadata updates to
other parts of the filesystem grind to a halt, even if the filesystem
could otherwise have handled it.

Even worse, if one of the things grinding to a halt happens to be a
thread in the middle of a defer-ops finish holding the same ILOCK and
trying to obtain more log reservation having exhausted the permanent
reservation, we now have an ABBA deadlock - writeback completion has a
transaction reserved and wants the ILOCK, and someone else has the ILOCK
and wants a transaction reservation.

Therefore, we create a per-inode writeback io completion queue + work
item. When writeback finishes, it can add the ioend to the per-inode
queue and let the single worker item process that queue. This
dramatically cuts down on the number of kworkers and ILOCK contention in
the system, and seems to have eliminated an occasional deadlock I was
seeing while running generic/476.

Testing with a program that simulates a heavy random-write workload to a
single file demonstrates that the number of kworkers drops from
approximately 120 threads per file to 1, without dramatically changing
write bandwidth or pagecache access latency.

Note that we leave the xfs-conv workqueue's max_active alone because we
still want to be able to run ioend processing for as many inodes as the
system can handle.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>

show more ...

Merge tag 'drm-intel-next-2019-04-04' into gvt-next

Merge back drm-intel-next for engine name definition refinement
and 54939ea0bd85 ("drm/i915: Switch to use HWS indices rather than addresses")
tha

Merge tag 'drm-intel-next-2019-04-04' into gvt-next

Merge back drm-intel-next for engine name definition refinement
and 54939ea0bd85 ("drm/i915: Switch to use HWS indices rather than addresses")
that would need gvt fixes to depend on.

Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

show more ...

Merge tag 'asoc-fix-v5.1-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound into for-linus

ASoC: Fixes for v5.1

A few core fixes along with the driver specific ones, mainly fixing

Merge tag 'asoc-fix-v5.1-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound into for-linus

ASoC: Fixes for v5.1

A few core fixes along with the driver specific ones, mainly fixing
small issues that only affect x86 platforms for various reasons (their
unusual machine enumeration mechanisms mainly, plus a fix for error
handling in topology).

There's some of the driver fixes that look larger than they are, like
the hdmi-codec changes which resulted in an indentation change, and most
of the other large changes are for new drivers like the STM32 changes.

show more ...

Merge drm/drm-next into drm-misc-next

Finally have a reason for a backmerge other than "it's been a while"!

Backmerging drm-next to -misc-next to facilitate Rob Herring's work on
Panfrost.

Signed-

Merge drm/drm-next into drm-misc-next

Finally have a reason for a backmerge other than "it's been a while"!

Backmerging drm-next to -misc-next to facilitate Rob Herring's work on
Panfrost.

Signed-off-by: Sean Paul <seanpaul@chromium.org>

show more ...

Merge tag 'v5.1-rc3' of https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux into fbdev-for-next

Linux 5.1-rc3

Sync with upstream (which now contains fbdev-v5.1 changes) to
prepare a base

Merge tag 'v5.1-rc3' of https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux into fbdev-for-next

Linux 5.1-rc3

Sync with upstream (which now contains fbdev-v5.1 changes) to
prepare a base for fbdev-v5.2 changes.

show more ...

Merge drm/drm-next into drm-intel-next-queued

This is needed to get the fourcc code merged without conflicts.

Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>

Merge tag 'v5.1-rc2' into next-general

Merge to Linux 5.1-rc2 for subsystems to work with.

Merge remote-tracking branch 'net/master'

Merge tag 'v5.1-rc1' into spi-5.2

Linux 5.1-rc1

Merge tag 'v5.1-rc1' into regulator-5.2

Linux 5.1-rc1

Merge tag 'v5.1-rc1' into asoc-5.2

Linux 5.1-rc1

Merge tag 'v5.1-rc1' into asoc-5.1

Linux 5.1-rc1

Merge tag 'xfs-5.1-merge-4' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux

Pull xfs updates from Darrick Wong:
"Here are a number of new features and bug fixes for 5.1

They've undergone a week

Merge tag 'xfs-5.1-merge-4' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux

Pull xfs updates from Darrick Wong:
"Here are a number of new features and bug fixes for 5.1

They've undergone a week's worth of fstesting and merge cleanly with
master as of this morning

Most of the changes center on improving metadata validation and fixing
problems with online fsck, though there's also a new cache to speed up
unlinked inode handling and cleanup of the copy on write code in
preparation for future features

Changes for Linux 5.1:

- Fix online fsck to handle inode btrees correctly on 64k block
filesystems

- Teach online fsck to check directory and attribute names for
invalid characters

- Miscellanous fixes for online fsck

- Introduce a new panic mask so that we can halt immediately on
metadata corruption (for debugging purposes)

- Fix a block mapping race during writeback

- Cache unlinked inode list backrefs in memory to speed up list
processing

- Separate the bnobt/cntbt and inobt/finobt buffer verifiers so that
we can detect crosslinked btrees

- Refactor magic number verification so that we can standardize it

- Strengthen ondisk metadata structure offset build time verification

- Fix a memory corruption problem in the listxattr code

- Fix a shutdown problem during log recovery due to unreserved finobt
expansion

- Fix a referential integrity problem where O_TMPFILE inodes were put
on the unlinked list with nlink > 0 which would cause asserts
during log recovery if the system went down immediately

- Refactor the delayed allocation allocator to be more clever about
the possibility that its mapping might be stale

- Various fixes to the copy on write mechanism

- Make CoW preallocation suitable for use even with writes that
wouldn't otherwise require it

- Refactor an internal API

- Fix some statx implementation bugs

- Fix miscellaneous compiler and static checker complaints"

* tag 'xfs-5.1-merge-4' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux: (70 commits)
xfs: fix reporting supported extra file attributes for statx()
xfs: fix backwards endian conversion in scrub
xfs: fix uninitialized error variables
xfs: rework breaking of shared extents in xfs_file_iomap_begin
xfs: don't pass iomap flags to xfs_reflink_allocate_cow
xfs: fix uninitialized error variable
xfs: introduce an always_cow mode
xfs: report IOMAP_F_SHARED from xfs_file_iomap_begin_delay
xfs: make COW fork unwritten extent conversions more robust
xfs: merge COW handling into xfs_file_iomap_begin_delay
xfs: also truncate holes covered by COW blocks
xfs: don't use delalloc extents for COW on files with extsize hints
xfs: fix SEEK_DATA for speculative COW fork preallocation
xfs: make xfs_bmbt_to_iomap more useful
xfs: fix xfs_buf magic number endian checks
xfs: retry COW fork delalloc conversion when no extent was found
xfs: remove the truncate short cut in xfs_map_blocks
xfs: move xfs_iomap_write_allocate to xfs_aops.c
xfs: move stat accounting to xfs_bmapi_convert_delalloc
xfs: move transaction handling to xfs_bmapi_convert_delalloc
..

show more ...

xfs: remove the io_type field from the writeback context and ioend

The io_type field contains what is basically a summary of information
from the inode fork and the imap. But we can just as easily

xfs: remove the io_type field from the writeback context and ioend

The io_type field contains what is basically a summary of information
from the inode fork and the imap. But we can just as easily use that
information directly, simplifying a few bits here and there and
improving the trace points.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

show more ...

Merge tag 'v5.0-rc5' of https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux into fbdev-for-next

Linux 5.0-rc5

Sync with upstream (which now contains fbdev-v5.0-rc3 changes) to
prepare a

Merge tag 'v5.0-rc5' of https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux into fbdev-for-next

Linux 5.0-rc5

Sync with upstream (which now contains fbdev-v5.0-rc3 changes) to
prepare a base for fbdev-v5.1 changes.

show more ...

Merge tag 'v4.20' into for-linus

Sync with mainline to get linux/overflow.h among other things.

Merge tag 'v4.20' into next

Merge with mainline to bring in the new APIs.

Merge drm/drm-next into drm-misc-next

drm-next has been forwarded to 5.0-rc1, and we need it to apply the damage
helper for dirtyfb series from Noralf Trønnes.

Signed-off-by: Maxime Ripard <maxime.

Merge drm/drm-next into drm-misc-next

drm-next has been forwarded to 5.0-rc1, and we need it to apply the damage
helper for dirtyfb series from Noralf Trønnes.

Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>

show more ...

Merge tag 'v5.0-rc1' into next-general

Linux 5.0-rc1

Sync to pick up LSM stacking work (which is based on -rc1).

Merge drm/drm-next into drm-intel-next-queued

Generally catch up with 5.0-rc1, and specifically get the changes:

96d4f267e40f ("Remove 'type' argument from access_ok() function")
0b2c8f8b6b0c ("i91

Merge drm/drm-next into drm-intel-next-queued

Generally catch up with 5.0-rc1, and specifically get the changes:

96d4f267e40f ("Remove 'type' argument from access_ok() function")
0b2c8f8b6b0c ("i915: fix missing user_access_end() in page fault exception case")
594cc251fdd0 ("make 'user_access_begin()' do 'access_ok()'")

Signed-off-by: Jani Nikula <jani.nikula@intel.com>

show more ...

Merge branch 'master' into fixes

We have a fix to apply on top of commit 96d4f267e40f ("Remove 'type'
argument from access_ok() function"), so merge master to get it.