Changes in Makefile

1260ed77798502de9c98020040d2995008de10cc - Merge drm/drm-fixes into drm-misc-fixes

Tue, 08 Apr 2025 10:15:47 +0200

Merge drm/drm-fixes into drm-misc-fixesBackmerging to get updates from v6.15-rc1.Signed-off-by: Thomas Zimmermann List of files: /linux/fs/netfs/Makefile

946661e3bef8efa11ba8079d4ebafe6fc3b0aaad - Merge branch 'next' into for-linus

Sat, 05 Apr 2025 08:04:35 +0200

Merge branch 'next' into for-linusPrepare input updates for 6.15 merge window. List of files: /linux/fs/netfs/Makefile

0b119045b79a672bc6d8f18641c60fc8ce1b4585 - Merge tag 'v6.14-rc4' into next

Wed, 26 Feb 2025 01:03:25 +0100

Merge tag 'v6.14-rc4' into nextSync up with the mainline. List of files: /linux/fs/netfs/Makefile

9e676a024fa1fa2bd8150c2d2ba85478280353bc - Merge tag 'v6.14-rc1' into perf-tools-next

Wed, 05 Feb 2025 23:57:18 +0100

Merge tag 'v6.14-rc1' into perf-tools-nextTo get the various fixes in the current master.Signed-off-by: Namhyung Kim List of files: /linux/fs/netfs/Makefile

0410c6121529409b08e81a77ae3ee58c657e2243 - Merge drm/drm-next into drm-xe-next

Fri, 28 Feb 2025 15:54:14 +0100

Merge drm/drm-next into drm-xe-nextSync to fix conlicts between drm-xe-next and drm-intel-next.Signed-off-by: Lucas De Marchi List of files: /linux/fs/netfs/Makefile

93c7dd1b39444ebd5a6a98e56a363d7a4e646775 - Merge drm/drm-next into drm-misc-next

Thu, 06 Feb 2025 13:47:32 +0100

Merge drm/drm-next into drm-misc-nextBring rc1 to start the new release dev.Signed-off-by: Maxime Ripard List of files: /linux/fs/netfs/Makefile

ea9f8f2b21795a5d80418a655bcb212d5b89e08f - Merge drm/drm-next into drm-intel-next

Wed, 05 Feb 2025 18:12:37 +0100

Merge drm/drm-next into drm-intel-nextSync with v6.14-rc1.Signed-off-by: Jani Nikula List of files: /linux/fs/netfs/Makefile

c771600c6af14749609b49565ffb4cac2959710d - Merge drm/drm-next into drm-intel-gt-next

Wed, 05 Feb 2025 10:29:14 +0100

Merge drm/drm-next into drm-intel-gt-nextWe need4ba4f1afb6a9 ("perf: Generic hotplug support for a PMU with a scope")in order to land a i915 PMU simplification and a fix. That landed in 6.12and we are stuck at 6.9 so lets bump things forward.Signed-off-by: Tvrtko Ursulin List of files: /linux/fs/netfs/Makefile

25768de50b1f2dbb6ea44bd5148a87fe2c9c3688 - Merge branch 'next' into for-linus

Tue, 21 Jan 2025 06:37:39 +0100

Merge branch 'next' into for-linusPrepare input updates for 6.14 merge window. List of files: /linux/fs/netfs/Makefile

6d4a0f4ea72319c9a37c1a7191695467006dd272 - Merge tag 'v6.13-rc3' into next

Tue, 17 Dec 2024 18:40:45 +0100

Merge tag 'v6.13-rc3' into nextSync up with the mainline. List of files: /linux/fs/netfs/Makefile

ca56a74a31e26d81a481304ed2f631e65883372b - Merge tag 'vfs-6.14-rc1.netfs' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs

Mon, 20 Jan 2025 18:29:11 +0100

Merge tag 'vfs-6.14-rc1.netfs' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfsPull vfs netfs updates from Christian Brauner: "This contains read performance improvements and support for monolithic single-blob objects that have to be read/written as such (e.g. AFS directory contents). The implementation of the two parts is interwoven as each makes the other possible. - Read performance improvements The read performance improvements are intended to speed up some loss of performance detected in cifs and to a lesser extend in afs. The problem is that we queue too many work items during the collection of read results: each individual subrequest is collected by its own work item, and then they have to interact with each other when a series of subrequests don't exactly align with the pattern of folios that are being read by the overall request. Whilst the processing of the pages covered by individual subrequests as they complete potentially allows folios to be woken in parallel and with minimum delay, it can shuffle wakeups for sequential reads out of order - and that is the most common I/O pattern. The final assessment and cleanup of an operation is then held up until the last I/O completes - and for a synchronous sequential operation, this means the bouncing around of work items just adds latency. Two changes have been made to make this work: (1) All collection is now done in a single "work item" that works progressively through the subrequests as they complete (and also dispatches retries as necessary). (2) For readahead and AIO, this work item be done on a workqueue and can run in parallel with the ultimate consumer of the data; for synchronous direct or unbuffered reads, the collection is run in the application thread and not offloaded. Functions such as smb2_readv_callback() then just tell netfslib that the subrequest has terminated; netfslib does a minimal bit of processing on the spot - stat counting and tracing mostly - and then queues/wakes up the worker. This simplifies the logic as the collector just walks sequentially through the subrequests as they complete and walks through the folios, if buffered, unlocking them as it goes. It also keeps to a minimum the amount of latency injected into the filesystem's low-level I/O handling The way netfs supports filesystems using the deprecated PG_private_2 flag is changed: folios are flagged and added to a write request as they complete and that takes care of scheduling the writes to the cache. The originating read request can then just unlock the pages whatever happens. - Single-blob object support Single-blob objects are files for which the content of the file must be read from or written to the server in a single operation because reading them in parts may yield inconsistent results. AFS directories are an example of this as there exists the possibility that the contents are generated on the fly and would differ between reads or might change due to third party interference. Such objects will be written to and retrieved from the cache if one is present, though we allow/may need to propose multiple subrequests to do so. The important part is that read from/write to the *server* is monolithic. Single blob reading is, for the moment, fully synchronous and does result collection in the application thread and, also for the moment, the API is supplied the buffer in the form of a folio_queue chain rather than using the pagecache. - Related afs changes This series makes a number of changes to the kafs filesystem, primarily in the area of directory handling: - AFS's FetchData RPC reply processing is made partially asynchronous which allows the netfs_io_request's outstanding operation counter to be removed as part of reducing the collection to a single work item. - Directory and symlink reading are plumbed through netfslib using the single-blob object API and are now cacheable with fscache. This also allows the afs_read struct to be eliminated and netfs_io_subrequest to be used directly instead. - Directory and symlink content are now stored in a folio_queue buffer rather than in the pagecache. This means we don't require the RCU read lock and xarray iteration to access it, and folios won't randomly disappear under us because the VM wants them back. - The vnode operation lock is changed from a mutex struct to a private lock implementation. The problem is that the lock now needs to be dropped in a separate thread and mutexes don't permit that. - When a new directory or symlink is created, we now initialise it locally and mark it valid rather than downloading it (we know what it's likely to look like). - We now use the in-directory hashtable to reduce the number of entries we need to scan when doing a lookup. The edit routines have to maintain the hash chains. - Cancellation (e.g. by signal) of an async call after the rxrpc_call has been set up is now offloaded to the worker thread as there will be a notification from rxrpc upon completion. This avoids a double cleanup. - A "rolling buffer" implementation is created to abstract out the two separate folio_queue chaining implementations I had (one for read and one for write). - Functions are provided to create/extend a buffer in a folio_queue chain and tear it down again. This is used to handle AFS directories, but could also be used to create bounce buffers for content crypto and transport crypto. - The was_async argument is dropped from netfs_read_subreq_terminated() Instead we wake the read collection work item by either queuing it or waking up the app thread. - We don't need to use BH-excluding locks when communicating between the issuing thread and the collection thread as neither of them now run in BH context. - Also included are a number of new tracepoints; a split of the netfslib write collection code to put retrying into its own file (it gets more complicated with content encryption). - There are also some minor fixes AFS included, including fixing the AFS directory format struct layout, reducing some directory over-invalidation and making afs_mkdir() translate EEXIST to ENOTEMPY (which is not available on all systems the servers support). - Finally, there's a patch to try and detect entry into the folio unlock function with no folio_queue structs in the buffer (which isn't allowed in the cases that can get there). This is a debugging patch, but should be minimal overhead"* tag 'vfs-6.14-rc1.netfs' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (31 commits) netfs: Report on NULL folioq in netfs_writeback_unlock_folios() afs: Add a tracepoint for afs_read_receive() afs: Locally initialise the contents of a new symlink on creation afs: Use the contained hashtable to search a directory afs: Make afs_mkdir() locally initialise a new directory's content netfs: Change the read result collector to only use one work item afs: Make {Y,}FS.FetchData an asynchronous operation afs: Fix cleanup of immediately failed async calls afs: Eliminate afs_read afs: Use netfslib for symlinks, allowing them to be cached afs: Use netfslib for directories afs: Make afs_init_request() get a key if not given a file netfs: Add support for caching single monolithic objects such as AFS dirs netfs: Add functions to build/clean a buffer in a folio_queue afs: Add more tracepoints to do with tracking validity cachefiles: Add auxiliary data trace cachefiles: Add some subrequest tracepoints netfs: Remove some extraneous directory invalidations afs: Fix directory format encoding struct afs: Fix EEXIST error returned from afs_rmdir() to be ENOTEMPTY ... List of files: /linux/fs/netfs/Makefile

7a47db23a9f003614e15c687d2a5425c175a9ca8 - Merge patch series "netfs: Read performance improvements and "single-blob" support"

Fri, 20 Dec 2024 22:34:18 +0100

Merge patch series "netfs: Read performance improvements and "single-blob" support"David Howells says:This set of patches is primarily about two things: improving readperformance and supporting monolithic single-blob objects that have to beread/written as such (e.g. AFS directory contents). The implementation ofthe two parts is interwoven as each makes the other possible.READ PERFORMANCE================The read performance improvements are intended to speed up some loss ofperformance detected in cifs and to a lesser extend in afs. The problem isthat we queue too many work items during the collection of read results:each individual subrequest is collected by its own work item, and then theyhave to interact with each other when a series of subrequests don't exactlyalign with the pattern of folios that are being read by the overallrequest.Whilst the processing of the pages covered by individual subrequests asthey complete potentially allows folios to be woken in parallel and withminimum delay, it can shuffle wakeups for sequential reads out of order -and that is the most common I/O pattern.The final assessment and cleanup of an operation is then held up until thelast I/O completes - and for a synchronous sequential operation, this meansthe bouncing around of work items just adds latency.Two changes have been made to make this work: (1) All collection is now done in a single "work item" that works progressively through the subrequests as they complete (and also dispatches retries as necessary). (2) For readahead and AIO, this work item be done on a workqueue and can run in parallel with the ultimate consumer of the data; for synchronous direct or unbuffered reads, the collection is run in the application thread and not offloaded.Functions such as smb2_readv_callback() then just tell netfslib that thesubrequest has terminated; netfslib does a minimal bit of processing on thespot - stat counting and tracing mostly - and then queues/wakes up theworker. This simplifies the logic as the collector just walks sequentiallythrough the subrequests as they complete and walks through the folios, ifbuffered, unlocking them as it goes. It also keeps to a minimum the amountof latency injected into the filesystem's low-level I/O handlingThe way netfs supports filesystems using the deprecated PG_private_2 flagis changed: folios are flagged and added to a write request as theycomplete and that takes care of scheduling the writes to the cache. Theoriginating read request can then just unlock the pages whatever happens.SINGLE-BLOB OBJECT SUPPORT==========================Single-blob objects are files for which the content of the file must beread from or written to the server in a single operation because readingthem in parts may yield inconsistent results. AFS directories are anexample of this as there exists the possibility that the contents aregenerated on the fly and would differ between reads or might change due tothird party interference.Such objects will be written to and retrieved from the cache if one ispresent, though we allow/may need to propose multiple subrequests to do so.The important part is that read from/write to the *server* is monolithic.Single blob reading is, for the moment, fully synchronous and does resultcollection in the application thread and, also for the moment, the API issupplied the buffer in the form of a folio_queue chain rather than usingthe pagecache.AFS CHANGES===========This series makes a number of changes to the kafs filesystem, primarily inthe area of directory handling: (1) AFS's FetchData RPC reply processing is made partially asynchronous which allows the netfs_io_request's outstanding operation counter to be removed as part of reducing the collection to a single work item. (2) Directory and symlink reading are plumbed through netfslib using the single-blob object API and are now cacheable with fscache. This also allows the afs_read struct to be eliminated and netfs_io_subrequest to be used directly instead. (3) Directory and symlink content are now stored in a folio_queue buffer rather than in the pagecache. This means we don't require the RCU read lock and xarray iteration to access it, and folios won't randomly disappear under us because the VM wants them back. There are some downsides to this, though: the storage folios are no longer known to the VM, drop_caches can't flush them, the folios are not migrateable. The inode must also be marked dirty manually to get the data written to the cache in the background. (4) The vnode operation lock is changed from a mutex struct to a private lock implementation. The problem is that the lock now needs to be dropped in a separate thread and mutexes don't permit that. (5) When a new directory or symlink is created, we now initialise it locally and mark it valid rather than downloading it (we know what it's likely to look like). (6) We now use the in-directory hashtable to reduce the number of entries we need to scan when doing a lookup. The edit routines have to maintain the hash chains. (7) Cancellation (e.g. by signal) of an async call after the rxrpc_call has been set up is now offloaded to the worker thread as there will be a notification from rxrpc upon completion. This avoids a double cleanup.SUPPORTING CHANGES==================To support the above some other changes are also made: (1) A "rolling buffer" implementation is created to abstract out the two separate folio_queue chaining implementations I had (one for read and one for write). (2) Functions are provided to create/extend a buffer in a folio_queue chain and tear it down again. This is used to handle AFS directories, but could also be used to create bounce buffers for content crypto and transport crypto. (3) The was_async argument is dropped from netfs_read_subreq_terminated(). Instead we wake the read collection work item by either queuing it or waking up the app thread. (4) We don't need to use BH-excluding locks when communicating between the issuing thread and the collection thread as neither of them now run in BH context.MISCELLANY==========Also included are a number of new tracepoints; a split of the netfslibwrite collection code to put retrying into its own file (it gets morecomplicated with content encryption).There are also some minor fixes AFS included, including fixing the AFSdirectory format struct layout, reducing some directory over-invalidationand making afs_mkdir() translate EEXIST to ENOTEMPY (which is not availableon all systems the servers support).Finally, there's a patch to try and detect entry into the folio unlockfunction with no folio_queue structs in the buffer (which isn't allowed inthe cases that can get there). This is a debugging patch, but should beminimal overhead.* patches from https://lore.kernel.org/r/20241216204124.3752367-1-dhowells@redhat.com: (31 commits) netfs: Report on NULL folioq in netfs_writeback_unlock_folios() afs: Add a tracepoint for afs_read_receive() afs: Locally initialise the contents of a new symlink on creation afs: Use the contained hashtable to search a directory afs: Make afs_mkdir() locally initialise a new directory's content netfs: Change the read result collector to only use one work item afs: Make {Y,}FS.FetchData an asynchronous operation afs: Fix cleanup of immediately failed async calls afs: Eliminate afs_read afs: Use netfslib for symlinks, allowing them to be cached afs: Use netfslib for directories afs: Make afs_init_request() get a key if not given a file netfs: Add support for caching single monolithic objects such as AFS dirs netfs: Add functions to build/clean a buffer in a folio_queue afs: Add more tracepoints to do with tracking validity cachefiles: Add auxiliary data trace cachefiles: Add some subrequest tracepoints netfs: Remove some extraneous directory invalidations afs: Fix directory format encoding struct afs: Fix EEXIST error returned from afs_rmdir() to be ENOTEMPTY ...Link: https://lore.kernel.org/r/20241216204124.3752367-1-dhowells@redhat.comSigned-off-by: Christian Brauner List of files: /linux/fs/netfs/Makefile

49866ce7ea8d41a3dc198f519cc9caa2d6be1891 - netfs: Add support for caching single monolithic objects such as AFS dirs

Mon, 16 Dec 2024 21:41:09 +0100

netfs: Add support for caching single monolithic objects such as AFS dirsAdd support for caching the content of a file that contains a singlemonolithic object that must be read/written with a single I/O operation,such as an AFS directory.Signed-off-by: David Howells Link: https://lore.kernel.org/r/20241216204124.3752367-20-dhowells@redhat.comcc: Jeff Layton cc: Marc Dionne cc: netfs@lists.linux.devcc: linux-afs@lists.infradead.orgcc: linux-fsdevel@vger.kernel.orgSigned-off-by: Christian Brauner List of files: /linux/fs/netfs/Makefile

751e213f9f8a24e1bd5988b9cb8043b0b2f017f0 - netfs: Split retry code out of fs/netfs/write_collect.c

Mon, 16 Dec 2024 21:40:57 +0100

netfs: Split retry code out of fs/netfs/write_collect.cSplit write-retry code out of fs/netfs/write_collect.c as it will becomemore elaborate when content crypto is introduced.Signed-off-by: David Howells Link: https://lore.kernel.org/r/20241216204124.3752367-8-dhowells@redhat.comcc: Jeff Layton cc: netfs@lists.linux.devcc: linux-fsdevel@vger.kernel.orgSigned-off-by: Christian Brauner List of files: /linux/fs/netfs/Makefile

06fa229ceb36898e68022b5654c017d2c6582d7d - netfs: Abstract out a rolling folio buffer implementation

Mon, 16 Dec 2024 21:40:55 +0100

netfs: Abstract out a rolling folio buffer implementationA rolling buffer is a series of folios held in a list of folio_queues. Newfolios and folio_queue structs may be inserted at the head simultaneouslywith spent ones being removed from the tail without the need for locking.The rolling buffer includes an iov_iter and it has to be careful managingthis as the list of folio_queues is extended such that an oops doesn'tincurred because the iterator was pointing to the end of a folio_queuesegment that got appended to and then removed.We need to use the mechanism twice, once for read and once for write, and,in future patches, we will use a second rolling buffer to handle bouncebuffering for content encryption.Signed-off-by: David Howells Link: https://lore.kernel.org/r/20241216204124.3752367-6-dhowells@redhat.comcc: Jeff Layton cc: netfs@lists.linux.devcc: linux-fsdevel@vger.kernel.orgSigned-off-by: Christian Brauner List of files: /linux/fs/netfs/Makefile

77b679453d3364688ff3e5153c0be5b2b52672b7 - Merge tag 'v6.12-rc3' into perf-tools-next

Mon, 14 Oct 2024 19:45:28 +0200

Merge tag 'v6.12-rc3' into perf-tools-nextTo get the fixes in the current perf-tools tree.Signed-off-by: Namhyung Kim List of files: /linux/fs/netfs/Makefile

3fd6c59042dbba50391e30862beac979491145fe - Merge tag 'v6.12-rc1' into clk-meson-next

Mon, 30 Sep 2024 11:28:07 +0200

Merge tag 'v6.12-rc1' into clk-meson-nextLinux 6.12-rc1 List of files: /linux/fs/netfs/Makefile

a0efa2f362a69e47b9d8b48f770ef3a0249a7911 - Merge net-next/main to resolve conflicts

Wed, 09 Oct 2024 08:59:14 +0200

Merge net-next/main to resolve conflictsThe wireless-next tree was based on something older, and thereare now conflicts between -rc2 and work here. Merge net-next,which has enough of -rc2 for the conflicts to happen, resolvingthem in the process.Signed-off-by: Johannes Berg List of files: /linux/fs/netfs/Makefile

b88132ceb3faccdd785809df75f9d490ebaab459 - Merge drm/drm-next into drm-xe-next

Fri, 04 Oct 2024 11:29:21 +0200

Merge drm/drm-next into drm-xe-nextBackmerging to resolve a conflict with core locally.Signed-off-by: Thomas Hellström List of files: /linux/fs/netfs/Makefile

2dd0ef5d951e9b565ddb324fe26c531b6a40bf82 - Merge drm/drm-next into drm-misc-next

Mon, 30 Sep 2024 10:50:54 +0200

Merge drm/drm-next into drm-misc-nextGet drm-misc-next to up v6.12-rc1.Signed-off-by: Thomas Zimmermann List of files: /linux/fs/netfs/Makefile