1.\" Copyright (c) 1996 2.\" Julian Elischer <julian@FreeBSD.org>. All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" $FreeBSD$ 27.Dd June 18, 2020 28.Dt CAM 4 29.Os 30.Sh NAME 31.Nm CAM 32.Nd Common Access Method Storage subsystem 33.Sh SYNOPSIS 34.Cd "device scbus" 35.Cd "device ada" 36.Cd "device cd" 37.Cd "device ch" 38.Cd "device da" 39.Cd "device pass" 40.Cd "device pt" 41.Cd "device sa" 42.Cd "options CAMDEBUG" 43.Cd "options CAM_DEBUG_BUS=-1" 44.Cd "options CAM_DEBUG_TARGET=-1" 45.Cd "options CAM_DEBUG_LUN=-1" 46.Cd "options CAM_DEBUG_COMPILE=CAM_DEBUG_INFO|CAM_DEBUG_CDB|CAM_DEBUG_PROBE" 47.Cd "options CAM_DEBUG_FLAGS=CAM_DEBUG_INFO|CAM_DEBUG_CDB" 48.Cd "options CAM_MAX_HIGHPOWER=4" 49.Cd "options SCSI_NO_SENSE_STRINGS" 50.Cd "options SCSI_NO_OP_STRINGS" 51.Cd "options SCSI_DELAY=8000" 52.Sh DESCRIPTION 53The 54.Nm 55subsystem provides a uniform and modular system for the implementation 56of drivers to control various 57.Tn SCSI , 58.Tn ATA , 59.Tn NVMe , 60and 61.Tn MMC / SD 62devices, and to utilize different 63.Tn SCSI , 64.Tn ATA , 65.Tn NVMe , 66and 67.Tn MMC / SD 68host adapters through host adapter drivers. 69When the system probes buses, it attaches any devices it finds to the 70appropriate drivers. 71The 72.Xr pass 4 73driver, if it is configured in the kernel, will attach to all devices. 74.Sh KERNEL CONFIGURATION 75There are a number of generic kernel configuration options for the 76.Nm 77subsystem: 78.Bl -tag -width SCSI_NO_SENSE_STRINGS 79.It Dv CAM_BOOT_DELAY 80Additional time to wait after the static parts of the kernel have run to allow 81for discovery of additional devices which may take time to connect, 82such as USB attached storage. 83.It Dv CAM_IOSCHED_DYNAMIC 84Enable dynamic decisions in the I/O scheduler based on hints and the current 85performance of the storage devices. 86.It Dv CAM_IO_STATS 87Enable collection of statistics for periph devices. 88.It Dv CAM_TEST_FAILURE 89Enable ability to simulate I/O failures. 90.It Dv CAMDEBUG 91This option compiles in all the 92.Nm 93debugging printf code. 94This will not actually 95cause any debugging information to be printed out when included by itself. 96See below for details. 97.It Dv "CAM_MAX_HIGHPOWER=4" 98This sets the maximum allowable number of concurrent "high power" commands. 99A "high power" command is a command that takes more electrical power than 100most to complete. 101An example of this is the 102.Tn SCSI 103START UNIT command. 104Starting a disk often takes significantly more electrical power than normal 105operation. 106This option allows the 107user to specify how many concurrent high power commands may be outstanding 108without overloading the power supply on his computer. 109.It Dv SCSI_NO_SENSE_STRINGS 110This eliminates text descriptions of each 111.Tn SCSI 112Additional Sense Code and Additional Sense Code Qualifier pair. 113Since this 114is a fairly large text database, eliminating it reduces the size of the 115kernel somewhat. 116This is primarily necessary for boot floppies and other 117low disk space or low memory space environments. 118In most cases, though, 119this should be enabled, since it speeds the interpretation of 120.Tn SCSI 121error messages. 122Do not let the "kernel bloat" zealots get to you -- leave 123the sense descriptions in your kernel! 124.It Dv SCSI_NO_OP_STRINGS 125This disables text descriptions of each 126.Tn SCSI 127opcode. 128This option, like the sense string option above, is primarily 129useful for environments like a boot floppy where kernel size is critical. 130Enabling this option for normal use is not recommended, since it slows 131debugging of 132.Tn SCSI 133problems. 134.It Dv SCSI_DELAY=8000 135This is the 136.Tn SCSI 137"bus settle delay." 138In 139.Nm , 140it is specified in 141.Em milliseconds , 142not seconds like the old 143.Tn SCSI 144layer used to do. 145When the kernel boots, it sends a bus reset to each 146.Tn SCSI 147bus to tell each device to reset itself to a default set of transfer 148negotiations and other settings. 149Most 150.Tn SCSI 151devices need some amount of time to recover from a bus reset. 152Newer disks 153may need as little as 100ms, while old, slow devices may need much longer. 154If the 155.Dv SCSI_DELAY 156is not specified, it defaults to 2 seconds. 157The minimum allowable value for 158.Dv SCSI_DELAY 159is "100", or 100ms. 160One special case is that if the 161.Dv SCSI_DELAY 162is set to 0, that will be taken to mean the "lowest possible value." 163In that case, the 164.Dv SCSI_DELAY 165will be reset to 100ms. 166.El 167.Pp 168All devices and buses support dynamic allocation so that 169an upper number of devices and controllers does not need to be configured; 170.Cd "device da" 171will suffice for any number of disk drivers. 172.Pp 173The devices are either 174.Em wired 175so they appear as a particular device unit or 176.Em counted 177so that they appear as the next available unused unit. 178.Pp 179Units are wired down by setting kernel environment hints. 180This is usually done either interactively from the 181.Xr loader 8 , 182or automatically via the 183.Pa /boot/device.hints 184file. 185The basic syntax is: 186.Bd -literal -offset indent 187hint.device.unit.property="value" 188.Ed 189.Pp 190Individual 191.Nm 192bus numbers can be wired down to specific controllers with 193a config line similar to the following: 194.Bd -literal -offset indent 195hint.scbus.0.at="ahd1" 196.Ed 197.Pp 198This assigns 199.Nm 200bus number 0 to the 201.Em ahd1 202driver instance. 203For controllers supporting more than one bus, a particular bus can be assigned 204as follows: 205.Bd -literal -offset indent 206hint.scbus.0.at="ahc1" 207hint.scbus.0.bus="1" 208.Ed 209.Pp 210This assigns 211.Nm 212bus 0 to the bus 1 instance on 213.Em ahc1 . 214Peripheral drivers can be wired to a specific bus, target, and lun as so: 215.Bd -literal -offset indent 216hint.da.0.at="scbus0" 217hint.da.0.target="0" 218hint.da.0.unit="0" 219.Ed 220.Pp 221This assigns 222.Em da0 223to target 0, unit (lun) 0 of scbus 0. 224Omitting the target or unit hints will instruct 225.Nm 226to treat them as wildcards 227and use the first respective counted instances. 228These examples can be combined together to allow a peripheral device to be 229wired to any particular controller, bus, target, and/or unit instance. 230.Pp 231This also works with 232.Xr nvme 4 233drives as well. 234.Bd -literal -offset indent 235hint.nvme.4.at="pci7:0:0" 236hint.scbus.10.at="nvme4" 237hint.nda.10.at="scbus10" 238hint.nda.10.target="1" 239hint.nda.10.unit="12" 240hint.nda.11.at="scbus10" 241hint.nda.11.target="1" 242hint.nda.11.unit="2" 243.Ed 244.Pp 245This assigns the NVMe card living at PCI bus 7 to scbus 10 (in PCIe, 246slot and function are rarely used and usually 0). 247The target for 248.Xr nda 4 249devices is always 1. 250The unit is the namespace identifier from the drive. 251The namespace id 1 is exported as 252.Tn nda10 253and namespace id 2 is exported as 254.Tn nda11 . 255.Pp 256When you have a mixture of wired down and counted devices then the 257counting begins with the first non-wired down unit for a particular 258type. 259That is, if you have a disk wired down as 260.Em "device da1" , 261then the first non-wired disk shall come on line as 262.Em da2 . 263.Sh ADAPTERS 264The system allows common device drivers to work through many different 265types of adapters. 266The adapters take requests from the upper layers and do 267all IO between the 268.Tn SCSI , 269.Tn ATA , 270.Tn NVMe , 271or 272.Tn MMC / SD 273bus and the system. 274The maximum size of a transfer is governed by the 275adapter. 276Most adapters can transfer 64KB in a single operation, however 277many can transfer larger amounts. 278.Sh TARGET MODE 279Some adapters support 280.Em target mode 281in which the system is capable of operating as a device, responding to 282operations initiated by another system. 283Target mode is supported for 284some adapters, but is not yet complete for this version of the 285.Nm 286.Tn SCSI 287subsystem. 288.Sh ARCHITECTURE 289The 290.Nm 291subsystem glues together the upper layers of the system to the storage devices. 292PERIPH devices accept storage requests from GEOM and other upper layers of the 293system and translates them into protocol requests. 294XPT (transport) dispatches these protocol requests to a SIM driver. 295A SIM driver takes protocol requests and translates them into hardware commands 296the host adapter understands to transfer the protocol requests, and data (if 297any) to the storage device. 298The CCB transports these requests around as messages. 299.Ss CAM 300The Common Access Method was a standard defined in the 1990s to talk to disk 301drives. 302.Fx 303is one of the few operating systems to fully implement this model. 304The interface between different parts of CAM is the CCB (or CAM Control Block). 305Each CCB has a standard header, which contains the type of request and dispatch 306information, and a command specific portion. 307A CAM Periph generates requests. 308The XPT layer dispatches these requests to the appropriate SIM. 309Some CCBs are sent directly to the SIM for immediate processing, while others 310are queued and complete when the I/O has finished. 311A SIM takes CCBs and translates them into hardware specific commands to push the 312SCSI CDB or other protocol control block to the peripheral, along with setting 313up the DMA for the associated data. 314.Ss Periph Devices 315A periph driver knows how to translate standard requests into protocol messages 316that a SIM can deliver to hardware. 317These requests can come from any upper layer source, but primarily come in via 318GEOM as a bio request. 319They can also come in directly from character device requests for tapes and pass 320through commands. 321.Pp 322Disk devices, or direct access (da) in CAM, are one type of peripheral. 323These devices present themselves to the kernel a device ending in 324.Dq da . 325Each protocol has a unique device name: 326.Bl -tag -width 4 327.It Xr da 4 328SCSI or SAS device, or devices that accept SCSI CDBs for I/O. 329.It Xr ada 4 330ATA or SATA device 331.It Xr nda 4 332NVME device 333.It Xr sdda 4 334An SD or MMC block storage device. 335.El 336.Pp 337Tape devices are called serial access 338.Po 339.Xr sa 4 340.Pc 341in CAM. 342They interface to the system via a character device and provide 343.Xr ioctl 2 344control for tape drives. 345.Pp 346The 347.Xr pass 4 348device will pass through CCB requests from userland to the SIM directly. 349The device is used to send commands other than read, write, trim or flush to a 350device. 351The 352.Xr camcontrol 8 353command uses this device. 354.Ss XPT drivers 355The transport driver connects the periph to the SIM. 356It is not configured separately. 357It is also responsible for device discovery for those SIM drivers that do not 358enumerate themselves. 359.Ss SIM driver 360SIM used to stand for SCSI Interface Module. 361Now it is just SIM because it understands protocols other than SCSI. 362There are two types of SIM drivers: virtual and physical. 363Physical SIMs are typically called host bus adapters (HBA), but not universally. 364Virtual SIM drivers are for communicating with virtual machine hosts. 365.Sh FILES 366see other 367.Nm 368device entries. 369.Sh DIAGNOSTICS 370An XPT_DEBUG CCB can be used to enable various amounts of tracing information 371on any specific bus/device from the list of options compiled into the kernel. 372There are currently seven debugging flags that may be compiled in and used: 373.Bl -tag -width CAM_DEBUG_SUBTRACE 374.It Dv CAM_DEBUG_INFO 375This flag enables general informational printfs for the device 376or devices in question. 377.It Dv CAM_DEBUG_TRACE 378This flag enables function-level command flow tracing i.e., 379kernel printfs will happen at the entrance and exit of various functions. 380.It Dv CAM_DEBUG_SUBTRACE 381This flag enables debugging output internal to various functions. 382.It Dv CAM_DEBUG_CDB 383This flag will cause the kernel to print out all 384.Tn ATA 385and 386.Tn SCSI 387commands sent to a particular device or devices. 388.It Dv CAM_DEBUG_XPT 389This flag will enable command scheduler tracing. 390.It Dv CAM_DEBUG_PERIPH 391This flag will enable peripheral drivers messages. 392.It Dv CAM_DEBUG_PROBE 393This flag will enable devices probe process tracing. 394.El 395.Pp 396Some of these flags, most notably 397.Dv CAM_DEBUG_TRACE 398and 399.Dv CAM_DEBUG_SUBTRACE , 400will produce kernel printfs in EXTREME numbers. 401.Pp 402Users can enable debugging from their kernel config file, by using 403the following kernel config options: 404.Bl -tag -width CAM_DEBUG_COMPILE 405.It Dv CAMDEBUG 406This builds into the kernel all possible 407.Nm 408debugging. 409.It Dv CAM_DEBUG_COMPILE 410This specifies support for which debugging flags described above 411should be built into the kernel. 412Flags may be ORed together if the user wishes to 413see printfs for multiple debugging levels. 414.It Dv CAM_DEBUG_FLAGS 415This sets the various debugging flags from a kernel config file. 416.It Dv CAM_DEBUG_BUS 417Specify a bus to debug. 418To debug all buses, set this to -1. 419.It Dv CAM_DEBUG_TARGET 420Specify a target to debug. 421To debug all targets, set this to -1. 422.It Dv CAM_DEBUG_LUN 423Specify a lun to debug. 424To debug all luns, set this to -1. 425.El 426.Pp 427Users may also enable debugging on the fly by using the 428.Xr camcontrol 8 429utility, if wanted options built into the kernel. 430See 431.Xr camcontrol 8 432for details. 433.Sh SEE ALSO 434.Bl -tag -width 20 435.It Sy Commands: 436.Xr camcontrol 8 , 437.Xr camdd 8 438.It Sy Libraries: 439.Xr cam 3 440.It Sy Periph Drivers: 441.Xr ada 4 , 442.Xr da 4 , 443.Xr nda 4 , 444.\" .Xr sdda 4 , 445.Xr pass 4 , 446.Xr sa 4 447.Pp 448.It Sy SIM Devices: 449.Xr aac 4 , 450.Xr aacraid 4 , 451.Xr ahc 4 , 452.Xr ahci 4 , 453.Xr ata 4 , 454.Xr aw_mmc 4 , 455.Xr ciss 4 , 456.Xr hv_storvsc 4 , 457.Xr isci 4 , 458.Xr iscsi 4 , 459.Xr isp 4 , 460.\" .Xr mmcnull 4 , 461.Xr mpr 4 , 462.Xr mps 4 , 463.Xr mpt 4 , 464.Xr mrsas 4 , 465.Xr mvs 4 , 466.Xr nvme 4 , 467.Xr pms 4 , 468.Xr pvscsi 4 , 469.Xr sdhci 4 , 470.Xr smartpqi 4 , 471.Xr sym 4 , 472.Xr tws 4 , 473.Xr umass 4 , 474.Xr virtio_scsi 4 475.It Sy Deprecated or Poorly Supported SIM Devices: 476.Xr ahd 4 , 477.Xr amr 4 , 478.Xr arcmsr 4 , 479.Xr esp 4 , 480.\" .Xr fslsata 4 , 481.Xr hpt27xx 4 , 482.Xr hptiop 4 , 483.Xr hptmv 4 , 484.Xr hptnr 4 , 485.\" .Xr htprr 4 , 486.Xr iir 4 487.Xr mfi 4 , 488.\" .Xr osc 4 , 489.\" .Xr ps3cdrom 4 , 490.Xr sbp 4 , 491.Xr twa 4 492.El 493.Sh HISTORY 494The 495.Nm 496.Tn SCSI 497subsystem first appeared in 498.Fx 3.0 . 499The 500.Nm 501ATA support was added in 502.Fx 8.0 . 503.Sh AUTHORS 504.An -nosplit 505The 506.Nm 507.Tn SCSI 508subsystem was written by 509.An Justin Gibbs 510and 511.An Kenneth Merry . 512The 513.Nm 514.Tn ATA 515support was added by 516.An Alexander Motin Aq Mt mav@FreeBSD.org . 517The 518.Nm 519.Tn NVMe 520support was added by 521.An Warner Losh Aq Mt imp@FreeBSD.org . 522