1.\" 2.\" Copyright (c) 2010 Spectra Logic Corporation 3.\" Copyright (c) 2014 LSI Corp 4.\" Copyright (c) 2015-2017 Avago Technologies 5.\" Copyright (c) 2015-2022 Broadcom Ltd. 6.\" All rights reserved. 7.\" 8.\" Redistribution and use in source and binary forms, with or without 9.\" modification, are permitted provided that the following conditions 10.\" are met: 11.\" 1. Redistributions of source code must retain the above copyright 12.\" notice, this list of conditions, and the following disclaimer, 13.\" without modification. 14.\" 2. Redistributions in binary form must reproduce at minimum a disclaimer 15.\" substantially similar to the "NO WARRANTY" disclaimer below 16.\" ("Disclaimer") and any redistribution must be conditioned upon 17.\" including a substantially similar Disclaimer requirement for further 18.\" binary redistribution. 19.\" 20.\" NO WARRANTY 21.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 22.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 23.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR 24.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 25.\" HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 26.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 27.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 28.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, 29.\" STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING 30.\" IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 31.\" POSSIBILITY OF SUCH DAMAGES. 32.\" 33.\" mpr driver man page. 34.\" 35.\" Author: Ken Merry <ken@FreeBSD.org> 36.\" Author: Stephen McConnell <slm@FreeBSD.org> 37.\" 38.\" $Id$ 39.\" $FreeBSD$ 40.\" 41.Dd June 1, 2019 42.Dt MPR 4 43.Os 44.Sh NAME 45.Nm mpr 46.Nd "LSI Fusion-MPT 3/3.5 IT/IR 12Gb/s Serial Attached SCSI/SATA/PCIe driver" 47.Sh SYNOPSIS 48To compile this driver into the kernel, place these lines in the kernel 49configuration file: 50.Bd -ragged -offset indent 51.Cd "device pci" 52.Cd "device scbus" 53.Cd "device mpr" 54.Ed 55.Pp 56The driver can be loaded as a module at boot time by placing this line in 57.Xr loader.conf 5 : 58.Bd -literal -offset indent 59mpr_load="YES" 60.Ed 61.Sh DESCRIPTION 62The 63.Nm 64driver provides support for Broadcom Ltd./Avago Tech (LSI) 65Fusion-MPT 3/3.5 IT/IR 66.Tn SAS/PCIe 67controllers. 68.Sh HARDWARE 69These controllers are supported by the 70.Nm 71driver: 72.Pp 73.Bl -bullet -compact 74.It 75Broadcom Ltd./Avago Tech (LSI) SAS 3004 (4 Port SAS) 76.It 77Broadcom Ltd./Avago Tech (LSI) SAS 3008 (8 Port SAS) 78.It 79Broadcom Ltd./Avago Tech (LSI) SAS 3108 (8 Port SAS) 80.It 81Broadcom Ltd./Avago Tech (LSI) SAS 3216 (16 Port SAS) 82.It 83Broadcom Ltd./Avago Tech (LSI) SAS 3224 (24 Port SAS) 84.It 85Broadcom Ltd./Avago Tech (LSI) SAS 3316 (16 Port SAS) 86.It 87Broadcom Ltd./Avago Tech (LSI) SAS 3324 (24 Port SAS) 88.It 89Broadcom Ltd./Avago Tech (LSI) SAS 3408 (8 Port SAS/PCIe) 90.It 91Broadcom Ltd./Avago Tech (LSI) SAS 3416 (16 Port SAS/PCIe) 92.It 93Broadcom Ltd./Avago Tech (LSI) SAS 3508 (8 Port SAS/PCIe) 94.It 95Broadcom Ltd./Avago Tech (LSI) SAS 3516 (16 Port SAS/PCIe) 96.It 97Broadcom Ltd./Avago Tech (LSI) SAS 3616 (16 Port SAS/PCIe) 98.It 99Broadcom Ltd./Avago Tech (LSI) SAS 3708 (8 Port SAS/PCIe) 100.It 101Broadcom Ltd./Avago Tech (LSI) SAS 3716 (16 Port SAS/PCIe) 102.It 103Broadcom Ltd./Avago Tech (LSI) SAS 3816 (16 Port SAS/PCIe) 104.It 105Broadcom Ltd./Avago Tech (LSI) SAS 3916 (16 Port SAS/PCIe) 106.El 107.Sh CONFIGURATION 108In all tunable descriptions below, X represents the adapter number. 109.Pp 110To disable MSI interrupts for all 111.Nm 112driver instances, set this tunable value in 113.Xr loader.conf 5 : 114.Bd -literal -offset indent 115hw.mpr.disable_msi=1 116.Ed 117.Pp 118To disable MSI interrupts for a specific 119.Nm 120driver instance, set this tunable value in 121.Xr loader.conf 5 : 122.Bd -literal -offset indent 123dev.mpr.X.disable_msi=1 124.Ed 125.Pp 126To disable MSI-X interrupts for all 127.Nm 128driver instances, set this tunable value in 129.Xr loader.conf 5 : 130.Bd -literal -offset indent 131hw.mpr.disable_msix=1 132.Ed 133.Pp 134To disable MSI-X interrupts for a specific 135.Nm 136driver instance, set this tunable value in 137.Xr loader.conf 5 : 138.Bd -literal -offset indent 139dev.mpr.X.disable_msix=1 140.Ed 141.Pp 142To set the maximum number of DMA chains allocated for all adapters, set 143this tunable in 144.Xr loader.conf 5 : 145.Bd -literal -offset indent 146hw.mpr.max_chains=NNNN 147.Ed 148.Pp 149To set the maximum number of DMA chains allocated for a specific adapter, 150set this tunable in 151.Xr loader.conf 5 : 152.Bd -literal -offset indent 153dev.mpr.X.max_chains=NNNN 154.Ed 155.Pp 156The default max_chains value is 16384. 157.Pp 158The current number of free chain frames is stored in the 159dev.mpr.X.chain_free 160.Xr sysctl 8 161variable. 162.Pp 163The lowest number of free chain frames seen since boot is stored in the 164dev.mpr.X.chain_free_lowwater 165.Xr sysctl 8 166variable. 167.Pp 168The number of times that chain frame allocations have failed since boot is 169stored in the 170dev.mpr.X.chain_alloc_fail 171.Xr sysctl 8 172variable. 173This can be used to determine whether the max_chains tunable should be 174increased to help performance. 175.Pp 176The current number of active I/O commands is shown in the 177dev.mpr.X.io_cmds_active 178.Xr sysctl 8 179variable. 180.Pp 181The current number of free PRP pages is stored in the 182dev.mpr.X.prp_pages_free 183.Xr sysctl 8 184variable. 185PRP pages are used by NVMe devices for I/O transfers, much like Scatter/Gather 186lists. 187.Pp 188The lowest number of free PRP pages seen since boot is stored in the 189dev.mpr.X.prp_pages_free_lowwater 190.Xr sysctl 8 191variable. 192.Pp 193The number of times that PRP page allocations have failed since boot is 194stored in the 195dev.mpr.X.prp_page_alloc_fail 196.Xr sysctl 8 197variable. 198.Pp 199To set the maximum number of pages that will be used per I/O for all adapters, 200set this tunable in 201.Xr loader.conf 5 : 202.Bd -literal -offset indent 203hw.mpr.max_io_pages=NNNN 204.Ed 205.Pp 206To set the maximum number of pages that will be used per I/O for a specific 207adapter, set this tunable in 208.Xr loader.conf 5 : 209.Bd -literal -offset indent 210dev.mpr.X.max_io_pages=NNNN 211.Ed 212.Pp 213The default max_io_pages value is -1, meaning that the maximum I/O size that 214will be used per I/O will be calculated using the IOCFacts values stored in 215the controller. 216The lowest value that the driver will use for max_io_pages is 1, otherwise 217IOCFacts will be used to calculate the maximum I/O size. 218The smaller I/O size calculated from either max_io_pages or IOCFacts will be the 219maximum I/O size used by the driver. 220.Pp 221The highest number of active I/O commands seen since boot is stored in the 222dev.mpr.X.io_cmds_highwater 223.Xr sysctl 8 224variable. 225.Pp 226Devices can be excluded from 227.Nm 228control for all adapters by setting this tunable in 229.Xr loader.conf 5 : 230.Bd -literal -offset indent 231hw.mpr.exclude_ids=Y 232.Ed 233.Pp 234Y represents the target ID of the device. 235If more than one device is to be excluded, target IDs are separated by commas. 236.Pp 237Devices can be excluded from 238.Nm 239control for a specific adapter by setting this tunable in 240.Xr loader.conf 5 : 241.Bd -literal -offset indent 242dev.mpr.X.exclude_ids=Y 243.Ed 244.Pp 245Y represents the target ID of the device. 246If more than one device is to be excluded, target IDs are separated by commas. 247.Pp 248The adapter can issue the 249.Sy StartStopUnit 250SCSI command to SATA direct-access devices during shutdown. 251This allows the device to quiesce powering down. 252To control this feature for all adapters, set the 253.Bd -literal -offset indent 254hw.mpr.enable_ssu 255.Ed 256.Pp 257tunable in 258.Xr loader.conf 5 259to one of these values: 260.Bl -tag -width 6n -offset indent 261.It 0 262Do not send SSU to either HDDs or SSDs. 263.It 1 264Send SSU to SSDs, but not to HDDs. 265This is the default value. 266.It 2 267Send SSU to HDDs, but not to SSDs. 268.It 3 269Send SSU to both HDDs and SSDs. 270.El 271.Pp 272To control this feature for a specific adapter, set this tunable value in 273.Xr loader.conf 5 : 274.Bd -literal -offset indent 275dev.mpr.X.enable_ssu 276.Ed 277.Pp 278The same set of values are valid as when setting this tunable for all adapters. 279.Pp 280SATA disks that take several seconds to spin up and fail the SATA Identify 281command might not be discovered by the driver. 282This problem can sometimes be overcome by increasing the value of the spinup 283wait time in 284.Xr loader.conf 5 285with the 286.Bd -literal -offset indent 287hw.mpr.spinup_wait_time=NNNN 288.Ed 289.Pp 290tunable. 291NNNN represents the number of seconds to wait for SATA devices to spin up when 292the device fails the initial SATA Identify command. 293.Pp 294Spinup wait times can be set for specific adapters in 295.Xr loader.conf 5 : 296with the 297.Bd -literal -offset indent 298dev.mpr.X.spinup_wait_time=NNNN 299.Ed 300.Pp 301tunable. 302NNNN is the number of seconds to wait for SATA devices to spin up when they fail 303the initial SATA Identify command. 304.Pp 305The driver can map devices discovered by the adapter so that target IDs 306corresponding to a specific device persist across resets and reboots. 307In some cases it is possible for devices to lose their mapped IDs due to 308unexpected behavior from certain hardware, such as some types of enclosures. 309To overcome this problem, a tunable is provided that will force the driver to 310map devices using the Phy number associated with the device. 311This feature is not recommended if the topology includes multiple 312enclosures/expanders. 313If multiple enclosures/expanders are present in the topology, Phy numbers are 314repeated, causing all devices at these Phy numbers except the first device to 315fail enumeration. 316To control this feature for all adapters, set the 317.Bd -literal -offset indent 318hw.mpr.use_phy_num 319.Ed 320.Pp 321tunable in 322.Xr loader.conf 5 323to one of these values: 324.Bl -tag -width 6n -offset indent 325.It -1 326Only use Phy numbers to map devices and bypass the driver's mapping logic. 327.It 0 328Never use Phy numbers to map devices. 329.It 1 330Use Phy numbers to map devices, but only if the driver's mapping logic fails 331to map the device that is being enumerated. 332This is the default value. 333.El 334.Pp 335To control this feature for a specific adapter, set this tunable value in 336.Xr loader.conf 5 : 337.Bd -literal -offset indent 338dev.mpr.X.use_phy_num 339.Ed 340.Pp 341The same set of values are valid as when setting this tunable for all adapters. 342.Sh DEBUGGING 343Driver diagnostic printing is controlled in 344.Xr loader.conf 5 345by using the global 346.Va hw.mpr.debug_level 347and per-device 348.Va dev.mpr.X.debug_level 349tunables. 350One can alter the debug level for any adapter at run-time using the 351.Xr sysctl 8 352variable 353.Va dev.mpr.X.debug_level . 354.Pp 355All 356.Va debug_level 357variables can be named by either an integer value or a text string. 358Multiple values can be specified together by either ORing the 359integer values or by providing a comma-separated list of names. 360A text string prefixed by 361.Qq + 362adds the specified debug levels to the existing set, while the prefix 363.Qq - 364removes them from the existing set. 365The current 366.Va debug_level 367status is reported in both formats for convenience. 368The following levels are available: 369.Bl -column "FlagXX" "NameXXXX" "Description" -offset indent 370.It Em Flag Ta Em Name Ta Em Description 371.It 0x0001 Ta info Ta Basic information (enabled by default) 372.It 0x0002 Ta fault Ta Driver faults (enabled by default) 373.It 0x0004 Ta event Ta Controller events 374.It 0x0008 Ta log Ta Logging data from controller 375.It 0x0010 Ta recovery Ta Tracing of recovery operations 376.It 0x0020 Ta error Ta Parameter errors and programming bugs 377.It 0x0040 Ta init Ta System initialization operations 378.It 0x0080 Ta xinfo Ta More detailed information 379.It 0x0100 Ta user Ta Tracing of user-generated commands (IOCTL) 380.It 0x0200 Ta mapping Ta Tracing of device mapping 381.It 0x0400 Ta trace Ta Tracing through driver functions 382.El 383.Sh SEE ALSO 384.Xr cam 4 , 385.Xr cd 4 , 386.Xr ch 4 , 387.Xr da 4 , 388.Xr mps 4 , 389.Xr mpt 4 , 390.Xr pci 4 , 391.Xr sa 4 , 392.Xr scsi 4 , 393.Xr targ 4 , 394.Xr loader.conf 5 , 395.Xr sysctl 8 396.Sh HISTORY 397The 398.Nm 399driver first appeared in 400.Fx 9.3 . 401.Sh AUTHORS 402The 403.Nm 404driver was originally written by 405.An -nosplit 406.An Scott Long Aq Mt scottl@FreeBSD.org . 407It has been improved and tested by LSI Corporation, 408Avago Technologies (formally LSI), and Broadcom Ltd. (formally Avago). 409.Pp 410This man page was written by 411.An Ken Merry Aq Mt ken@FreeBSD.org 412with additional input from 413.An Stephen McConnell Aq Mt slm@FreeBSD.org . 414