1.\" Copyright (c) 2013 Edward Tomasz Napierala 2.\" Copyright (c) 2015-2017 Alexander Motin <mav@FreeBSD.org> 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" $FreeBSD$ 27.Dd March 19, 2017 28.Dt CTL 4 29.Os 30.Sh NAME 31.Nm ctl 32.Nd CAM Target Layer / iSCSI target subsystem 33.Sh SYNOPSIS 34To compile this driver into the kernel, 35place the following line in your 36kernel configuration file: 37.Bd -ragged -offset indent 38.Cd "device iscsi" 39.Cd "device ctl" 40.Ed 41.Pp 42Alternatively, to load the driver as a 43module at boot time, place the following line in 44.Xr loader.conf 5 : 45.Bd -literal -offset indent 46ctl_load="YES" 47.Ed 48.Sh DESCRIPTION 49The 50.Nm 51subsystem provides SCSI target devices emulation. 52It supports features such as: 53.Pp 54.Bl -bullet -compact 55.It 56Disk, CD-ROM and processor device emulation 57.It 58Tagged queueing 59.It 60SCSI task attribute support (ordered, head of queue, simple tags) 61.It 62SCSI implicit command ordering support 63.It 64Full task management support (abort, query, reset, etc.) 65.It 66Support for multiple ports, initiators, targets and backing stores 67.It 68Support for VMWare VAAI and Microsoft ODX offload (COMPARE AND WRITE, 69XCOPY, POPULATE TOKEN/WRITE USING TOKEN, WRITE SAME and UNMAP) 70.It 71Persistent reservation support 72.It 73Extensive VPD/mode/log pages support 74.It 75Featured error reporting, error injection and basic SMART support 76.It 77High Availability clustering support with ALUA 78.It 79All I/O handled in-kernel, no userland context switch overhead 80.El 81.Pp 82The 83.Nm 84subsystem includes multiple frontends to provide access using different 85transport protocols and implementations: 86.Bl -tag -width cfumass 87.It camsim 88Provides access for local system via virtual initiator mode 89.Xr CAM 4 90SIM. 91.It camtgt 92Provides access for remote systems via target mode 93.Xr CAM 4 94SIMs, such as Fibre Channel 95.Xr isp 4 96and 97.Xr mpt 4 . 98.It cfumass 99Provides access for remote systems via USB Mass Storage Class 100Bulk Only (BBB) Transport. 101.It ha 102Internal frontend used to receive requests from other node ports in 103High Availability cluster. 104.It ioctl 105Provides access for local user-level applications via 106.Xr ioctl 2 107based API. 108.It iscsi 109Combined with 110.Xr iscsi 4 111and 112.Xr ctld 8 , 113provides access for remote systems via iSCSI protocol. 114.It tpc 115Internal frontend used to receive requests from Third Party Copy engine, 116implementing copy offload operations. 117.El 118.Pp 119The 120.Nm 121subsystem includes two backends to create logical units using different 122kinds of backing stores: 123.Bl -tag -width ramdisk 124.It block 125Stores data in ZFS ZVOLs, files or raw block devices. 126.It ramdisk 127Stores data in RAM, that makes it mostly useful for performance testing. 128Depending on configured capacity can work as black hole, thin or thick 129provisioned disk. 130.El 131.Sh SYSCTL VARIABLES 132The following variables are available as both 133.Xr sysctl 8 134variables and 135.Xr loader 8 136tunables: 137.Bl -tag -width indent 138.It Va kern.cam.ctl.debug 139Bit mask of enabled CTL log levels: 140.Bl -tag -offset indent -compact 141.It 1 142log commands with errors; 143.It 2 144log all commands; 145.It 4 146log data for commands other then READ/WRITE. 147.El 148Defaults to 0. 149.It Va kern.cam.ctl.ha_id 150Specifies unique position of this node within High Availability cluster. 151Default is 0 -- no HA, 1 and 2 -- HA enabled at specified position. 152.It Va kern.cam.ctl.ha_mode 153Specifies High Availability cluster operation mode: 154.Bl -tag -offset indent -compact 155.It 0 156Active/Standby -- primary node has backend access and processes requests, 157while secondary can only do basic LUN discovery and reservation; 158.It 1 159Active/Active -- both nodes have backend access and process requests, 160while secondary node synchronizes processing with primary one; 161.It 2 162Active/Active -- primary node has backend access and processes requests, 163while secondary node forwards all requests and data to primary one; 164.El 165All above modes require established connection between HA cluster nodes. 166If connection is not configured, secondary node will report Unavailable 167state; if configured but not established -- Transitioning state. 168Defaults to 0. 169.It Va kern.cam.ctl.ha_peer 170String value, specifying method to establish connection to peer HA node. 171Can be "listen IP:port", "connect IP:port" or empty. 172.It Va kern.cam.ctl.ha_link 173Reports present state of connection between HA cluster nodes: 174.Bl -tag -offset indent -compact 175.It 0 176not configured; 177.It 1 178configured but not established; 179.It 2 180established. 181.El 182.It Va kern.cam.ctl.ha_role 183Specifies default role of this node: 184.Bl -tag -offset indent -compact 185.It 0 186primary; 187.It 1 188secondary. 189.El 190This role can be overridden on per-LUN basis using "ha_role" LUN option, 191so that for one LUN one node is primary, while for another -- another. 192Role change from primary to secondary for HA modes 0 and 2 closes backends, 193the opposite change -- opens. 194If there is no primary node (both nodes are secondary, or secondary node has 195no connection to primary one), secondary node(s) report Transitioning state. 196State with two primary nodes is illegal (split brain condition). 197.It Va kern.cam.ctl.iscsi.debug 198Verbosity level for log messages from the kernel part of iSCSI target. 199Set to 0 to disable logging or 1 to warn about potential problems. 200Larger values enable debugging output. 201Defaults to 1. 202.It Va kern.cam.ctl.iscsi.maxtags 203The number of outstanding commands to advertise to each iSCSI initiator. 204Current implementation is not very accurate, so do not set this below 2. 205Defaults to 256. 206.It Va kern.cam.ctl.iscsi.ping_timeout 207The number of seconds to wait for the iSCSI initiator to respond to a NOP-In 208PDU. 209In the event that there is no response within that time the session gets 210forcibly terminated. 211Set to 0 to disable sending NOP-In PDUs. 212Defaults to 5. 213.El 214.Sh SEE ALSO 215.Xr cfumass 4 , 216.Xr ctladm 8 , 217.Xr ctld 8 , 218.Xr ctlstat 8 219.Sh HISTORY 220The 221.Nm 222subsystem first appeared in 223.Fx 9.1 . 224.Sh AUTHORS 225The 226.Nm 227subsystem was originally written by 228.An Kenneth Merry Aq Mt ken@FreeBSD.org . 229Later work was done by 230.An Alexander Motin Aq Mt mav@FreeBSD.org . 231