1.\" 2.\" The contents of this file are subject to the terms of the Common Development 3.\" and Distribution License (the "License"). You may not use this file except 4.\" in compliance with the License. You can obtain a copy of the license at 5.\" usr/src/OPENSOLARIS.LICENSE or https://opensource.org/licenses/CDDL-1.0. 6.\" 7.\" See the License for the specific language governing permissions and 8.\" limitations under the License. When distributing Covered Code, include this 9.\" CDDL HEADER in each file and include the License file at 10.\" usr/src/OPENSOLARIS.LICENSE. If applicable, add the following below this 11.\" CDDL HEADER, with the fields enclosed by brackets "[]" replaced with your 12.\" own identifying information: 13.\" Portions Copyright [yyyy] [name of copyright owner] 14.\" 15.\" Copyright 2013 Turbo Fredriksson <turbo@bayour.com>. All rights reserved. 16.\" 17.Dd August 24, 2020 18.Dt SPL 4 19.Os 20. 21.Sh NAME 22.Nm spl 23.Nd parameters of the SPL kernel module 24. 25.Sh DESCRIPTION 26.Bl -tag -width Ds 27.It Sy spl_kmem_cache_kmem_threads Ns = Ns Sy 4 Pq uint 28The number of threads created for the spl_kmem_cache task queue. 29This task queue is responsible for allocating new slabs 30for use by the kmem caches. 31For the majority of systems and workloads only a small number of threads are 32required. 33. 34.It Sy spl_kmem_cache_obj_per_slab Ns = Ns Sy 8 Pq uint 35The preferred number of objects per slab in the cache. 36In general, a larger value will increase the caches memory footprint 37while decreasing the time required to perform an allocation. 38Conversely, a smaller value will minimize the footprint 39and improve cache reclaim time but individual allocations may take longer. 40. 41.It Sy spl_kmem_cache_max_size Ns = Ns Sy 32 Po 64-bit Pc or Sy 4 Po 32-bit Pc Pq uint 42The maximum size of a kmem cache slab in MiB. 43This effectively limits the maximum cache object size to 44.Sy spl_kmem_cache_max_size Ns / Ns Sy spl_kmem_cache_obj_per_slab . 45.Pp 46Caches may not be created with 47object sized larger than this limit. 48. 49.It Sy spl_kmem_cache_slab_limit Ns = Ns Sy 16384 Pq uint 50For small objects the Linux slab allocator should be used to make the most 51efficient use of the memory. 52However, large objects are not supported by 53the Linux slab and therefore the SPL implementation is preferred. 54This value is used to determine the cutoff between a small and large object. 55.Pp 56Objects of size 57.Sy spl_kmem_cache_slab_limit 58or smaller will be allocated using the Linux slab allocator, 59large objects use the SPL allocator. 60A cutoff of 16K was determined to be optimal for architectures using 4K pages. 61. 62.It Sy spl_kmem_alloc_warn Ns = Ns Sy 32768 Pq uint 63As a general rule 64.Fn kmem_alloc 65allocations should be small, 66preferably just a few pages, since they must by physically contiguous. 67Therefore, a rate limited warning will be printed to the console for any 68.Fn kmem_alloc 69which exceeds a reasonable threshold. 70.Pp 71The default warning threshold is set to eight pages but capped at 32K to 72accommodate systems using large pages. 73This value was selected to be small enough to ensure 74the largest allocations are quickly noticed and fixed. 75But large enough to avoid logging any warnings when a allocation size is 76larger than optimal but not a serious concern. 77Since this value is tunable, developers are encouraged to set it lower 78when testing so any new largish allocations are quickly caught. 79These warnings may be disabled by setting the threshold to zero. 80. 81.It Sy spl_kmem_alloc_max Ns = Ns Sy KMALLOC_MAX_SIZE Ns / Ns Sy 4 Pq uint 82Large 83.Fn kmem_alloc 84allocations will fail if they exceed 85.Sy KMALLOC_MAX_SIZE . 86Allocations which are marginally smaller than this limit may succeed but 87should still be avoided due to the expense of locating a contiguous range 88of free pages. 89Therefore, a maximum kmem size with reasonable safely margin of 4x is set. 90.Fn kmem_alloc 91allocations larger than this maximum will quickly fail. 92.Fn vmem_alloc 93allocations less than or equal to this value will use 94.Fn kmalloc , 95but shift to 96.Fn vmalloc 97when exceeding this value. 98. 99.It Sy spl_kmem_cache_magazine_size Ns = Ns Sy 0 Pq uint 100Cache magazines are an optimization designed to minimize the cost of 101allocating memory. 102They do this by keeping a per-cpu cache of recently 103freed objects, which can then be reallocated without taking a lock. 104This can improve performance on highly contended caches. 105However, because objects in magazines will prevent otherwise empty slabs 106from being immediately released this may not be ideal for low memory machines. 107.Pp 108For this reason, 109.Sy spl_kmem_cache_magazine_size 110can be used to set a maximum magazine size. 111When this value is set to 0 the magazine size will 112be automatically determined based on the object size. 113Otherwise magazines will be limited to 2-256 objects per magazine (i.e per cpu). 114Magazines may never be entirely disabled in this implementation. 115. 116.It Sy spl_hostid Ns = Ns Sy 0 Pq ulong 117The system hostid, when set this can be used to uniquely identify a system. 118By default this value is set to zero which indicates the hostid is disabled. 119It can be explicitly enabled by placing a unique non-zero value in 120.Pa /etc/hostid . 121. 122.It Sy spl_hostid_path Ns = Ns Pa /etc/hostid Pq charp 123The expected path to locate the system hostid when specified. 124This value may be overridden for non-standard configurations. 125. 126.It Sy spl_panic_halt Ns = Ns Sy 0 Pq uint 127Cause a kernel panic on assertion failures. 128When not enabled, the thread is halted to facilitate further debugging. 129.Pp 130Set to a non-zero value to enable. 131. 132.It Sy spl_taskq_kick Ns = Ns Sy 0 Pq uint 133Kick stuck taskq to spawn threads. 134When writing a non-zero value to it, it will scan all the taskqs. 135If any of them have a pending task more than 5 seconds old, 136it will kick it to spawn more threads. 137This can be used if you find a rare 138deadlock occurs because one or more taskqs didn't spawn a thread when it should. 139. 140.It Sy spl_taskq_thread_bind Ns = Ns Sy 0 Pq int 141Bind taskq threads to specific CPUs. 142When enabled all taskq threads will be distributed evenly 143across the available CPUs. 144By default, this behavior is disabled to allow the Linux scheduler 145the maximum flexibility to determine where a thread should run. 146. 147.It Sy spl_taskq_thread_dynamic Ns = Ns Sy 1 Pq int 148Allow dynamic taskqs. 149When enabled taskqs which set the 150.Sy TASKQ_DYNAMIC 151flag will by default create only a single thread. 152New threads will be created on demand up to a maximum allowed number 153to facilitate the completion of outstanding tasks. 154Threads which are no longer needed will be promptly destroyed. 155By default this behavior is enabled but it can be disabled to 156aid performance analysis or troubleshooting. 157. 158.It Sy spl_taskq_thread_priority Ns = Ns Sy 1 Pq int 159Allow newly created taskq threads to set a non-default scheduler priority. 160When enabled, the priority specified when a taskq is created will be applied 161to all threads created by that taskq. 162When disabled all threads will use the default Linux kernel thread priority. 163By default, this behavior is enabled. 164. 165.It Sy spl_taskq_thread_sequential Ns = Ns Sy 4 Pq int 166The number of items a taskq worker thread must handle without interruption 167before requesting a new worker thread be spawned. 168This is used to control 169how quickly taskqs ramp up the number of threads processing the queue. 170Because Linux thread creation and destruction are relatively inexpensive a 171small default value has been selected. 172This means that normally threads will be created aggressively which is 173desirable. 174Increasing this value will 175result in a slower thread creation rate which may be preferable for some 176configurations. 177. 178.It Sy spl_max_show_tasks Ns = Ns Sy 512 Pq uint 179The maximum number of tasks per pending list in each taskq shown in 180.Pa /proc/spl/taskq{,-all} . 181Write 182.Sy 0 183to turn off the limit. 184The proc file will walk the lists with lock held, 185reading it could cause a lock-up if the list grow too large 186without limiting the output. 187"(truncated)" will be shown if the list is larger than the limit. 188. 189.It Sy spl_taskq_thread_timeout_ms Ns = Ns Sy 5000 Pq uint 190Minimum idle threads exit interval for dynamic taskqs. 191Smaller values allow idle threads exit more often and potentially be 192respawned again on demand, causing more churn. 193.El 194