1/* $FreeBSD$ */ 2 3 Driver Theory of Operation Manual 4 51. Introduction 6 7This is a short text document that will describe the background, goals 8for, and current theory of operation for the joint Fibre Channel/SCSI 9HBA driver for QLogic hardware. 10 11Because this driver is an ongoing project, do not expect this manual 12to remain entirely up to date. Like a lot of software engineering, the 13ultimate documentation is the driver source. However, this manual should 14serve as a solid basis for attempting to understand where the driver 15started and what is trying to be accomplished with the current source. 16 17The reader is expected to understand the basics of SCSI and Fibre Channel 18and to be familiar with the range of platforms that Solaris, Linux and 19the variant "BSD" Open Source systems are available on. A glossary and 20a few references will be placed at the end of the document. 21 22There will be references to functions and structures within the body of 23this document. These can be easily found within the source using editor 24tags or grep. There will be few code examples here as the code already 25exists where the reader can easily find it. 26 272. A Brief History for this Driver 28 29This driver originally started as part of work funded by NASA Ames 30Research Center's Numerical Aerodynamic Simulation center ("NAS" for 31short) for the QLogic PCI 1020 and 1040 SCSI Host Adapters as part of my 32work at porting the NetBSD Operating System to the Alpha architectures 33(specifically the AlphaServer 8200 and 8400 platforms). In short, it 34started just as simple single SCSI HBA driver for just the purpose of 35running off a SCSI disk. This work took place starting in January, 1997. 36 37Because the first implementation was for NetBSD, which runs on a very 38large number of platforms, and because NetBSD supported both systems with 39SBus cards (e.g., Sun SPARC systems) as well as systems with PCI cards, 40and because the QLogic SCSI cards came in both SBus and PCI versions, the 41initial implementation followed the very thoughtful NetBSD design tenet 42of splitting drivers into what are called MI (for Machine Independent) 43and MD (Machine Dependent) portions. The original design therefore was 44from the premise that the driver would drive both SBus and PCI card 45variants. These busses are similar but have quite different constraints, 46and while the QLogic SBus and PCI cards are very similar, there are some 47significant differences. 48 49After this initial goal had been met, there began to be some talk about 50looking into implementing Fibre Channel mass storage at NAS. At this time 51the QLogic 2100 FC/AL HBA was about to become available. After looking at 52the way it was designed I concluded that it was so darned close to being 53just like the SCSI HBAs that it would be insane to *not* leverage off of 54the existing driver. So, we ended up with a driver for NetBSD that drove 55PCI and SBus SCSI cards, and now also drove the QLogic 2100 FC-AL HBA. 56 57After this, ports to non-NetBSD platforms became interesting as well. 58This took the driver out of the interest with NAS and into interested 59support from a number of other places. Since the original NetBSD 60development, the driver has been ported to FreeBSD, OpenBSD, Linux, 61Solaris, and two proprietary systems. Following from the original MI/MD 62design of NetBSD, a rather successful attempt has been made to keep the 63Operating System Platform differences segregated and to a minimum. 64 65Along the way, support for the 2200 as well as full fabric and target 66mode support has been added, and 2300 support as well as an FC-IP stack 67are planned. 68 693. Driver Design Goals 70 71The driver has not started out as one normally would do such an effort. 72Normally you design via top-down methodologies and set an initial goal 73and meet it. This driver has had a design goal that changes from almost 74the very first. This has been an extremely peculiar, if not risque, 75experience. As a consequence, this section of this document contains 76a bit of "reconstruction after the fact" in that the design goals are 77as I perceive them to be now- not necessarily what they started as. 78 79The primary design goal now is to have a driver that can run both the 80SCSI and Fibre Channel SCSI prototocols on multiple OS platforms with 81as little OS platform support code as possible. 82 83The intended support targets for SCSI HBAs is to support the single and 84dual channel PCI Ultra2 and PCI Ultra3 cards as well as the older PCI 85Ultra single channel cards and SBus cards. 86 87The intended support targets for Fibre Channel HBAs is the 2100, 2200 88and 2300 PCI cards. 89 90Fibre Channel support should include complete fabric and public loop 91as well as private loop and private loop, direct-attach topologies. 92FC-IP support is also a goal. 93 94For both SCSI and Fibre Channel, simultaneous target/initiator mode support 95is a goal. 96 97Pure, raw, performance is not a primary goal of this design. This design, 98because it has a tremendous amount of code common across multiple 99platforms, will undoubtedly never be able to beat the performance of a 100driver that is specifically designed for a single platform and a single 101card. However, it is a good strong secondary goal to make the performance 102penalties in this design as small as possible. 103 104Another primary aim, which almost need not be stated, is that the 105implementation of platform differences must not clutter up the common 106code with platform specific defines. Instead, some reasonable layering 107semantics are defined such that platform specifics can be kept in the 108platform specific code. 109 1104. QLogic Hardware Architecture 111 112In order to make the design of this driver more intelligible, some 113description of the Qlogic hardware architecture is in order. This will 114not be an exhaustive description of how this card works, but will 115note enough of the important features so that the driver design is 116hopefully clearer. 117 1184.1 Basic QLogic hardware 119 120The QLogic HBA cards all contain a tiny 16-bit RISC-like processor and 121varying sizes of SRAM. Each card contains a Bus Interface Unit (BIU) 122as appropriate for the host bus (SBus or PCI). The BIUs allow access 123to a set of dual-ranked 16 bit incoming and outgoing mailbox registers 124as well as access to control registers that control the RISC or access 125other portions of the card (e.g., Flash BIOS). The term 'dual-ranked' 126means that at the same host visible address if you write a mailbox 127register, that is a write to an (incoming, to the HBA) mailbox register, 128while a read to the same address reads another (outgoing, to the HBA) 129mailbox register with completely different data. Each HBA also then has 130core and auxiliary logic which either is used to interface to a SCSI bus 131(or to external bus drivers that connect to a SCSI bus), or to connect 132to a Fibre Channel bus. 133 1344.2 Basic Control Interface 135 136There are two principle I/O control mechanisms by which the driver 137communicates with and controls the QLogic HBA. The first mechanism is to 138use the incoming mailbox registers to interrupt and issue commands to 139the RISC processor (with results usually, but not always, ending up in 140the ougtoing mailbox registers). The second mechanism is to establish, 141via mailbox commands, circular request and response queues in system 142memory that are then shared between the QLogic and the driver. The 143request queue is used to queue requests (e.g., I/O requests) for the 144QLogic HBA's RISC engine to copy into the HBA memory and process. The 145result queue is used by the QLogic HBA's RISC engine to place results of 146requests read from the request queue, as well as to place notification 147of asynchronous events (e.g., incoming commands in target mode). 148 149To give a bit more precise scale to the preceding description, the QLogic 150HBA has 8 dual-ranked 16 bit mailbox registers, mostly for out-of-band 151control purposes. The QLogic HBA then utilizes a circular request queue 152of 64 byte fixed size Queue Entries to receive normal initiator mode 153I/O commands (or continue target mode requests). The request queue may 154be up to 256 elements for the QLogic 1020 and 1040 chipsets, but may 155be quite larger for the QLogic 12X0/12160 SCSI and QLogic 2X00 Fibre 156Channel chipsets. 157 158In addition to synchronously initiated usage of mailbox commands by 159the host system, the QLogic may also deliver asynchronous notifications 160solely in outgoing mailbox registers. These asynchronous notifications in 161mailboxes may be things like notification of SCSI Bus resets, or that the 162Fabric Name server has sent a change notification, or even that a specific 163I/O command completed without error (this is called 'Fast Posting' 164and saves the QLogic HBA from having to write a response queue entry). 165 166The QLogic HBA is an interrupting card, and when servicing an interrupt 167you really only have to check for either a mailbox interrupt or an 168interrupt notification that the response queue has an entry to 169be dequeued. 170 1714.3 Fibre Channel SCSI out of SCSI 172 173QLogic took the approach in introducing the 2X00 cards to just treat 174FC-AL as a 'fat' SCSI bus (a SCSI bus with more than 15 targets). All 175of the things that you really need to do with Fibre Channel with respect 176to providing FC-4 services on top of a Class 3 connection are performed 177by the RISC engine on the QLogic card itself. This means that from 178an HBA driver point of view, very little needs to change that would 179distinguish addressing a Fibre Channel disk from addressing a plain 180old SCSI disk. 181 182However, in the details it's not *quite* that simple. For example, in 183order to manage Fabric Connections, the HBA driver has to do explicit 184binding of entities it's queried from the name server to specific 'target' 185ids (targets, in this case, being a virtual entity). 186 187Still- the HBA firmware does really nearly all of the tedious management 188of Fibre Channel login state. The corollary to this sometimes is the 189lack of ability to say why a particular login connection to a Fibre 190Channel disk is not working well. 191 192There are clear limits with the QLogic card in managing fabric devices. 193The QLogic manages local loop devices (LoopID or Target 0..126) itself, 194but for the management of fabric devices, it has an absolute limit of 195253 simultaneous connections (256 entries less 3 reserved entries). 196 1975. Driver Architecture 198 1995.1 Driver Assumptions 200 201The first basic assumption for this driver is that the requirements for 202a SCSI HBA driver for any system is that of a 2 or 3 layer model where 203there are SCSI target device drivers (drivers which drive SCSI disks, 204SCSI tapes, and so on), possibly a middle services layer, and a bottom 205layer that manages the transport of SCSI CDB's out a SCSI bus (or across 206Fibre Channel) to a SCSI device. It's assumed that each SCSI command is 207a separate structure (or pointer to a structure) that contains the SCSI 208CDB and a place to store SCSI Status and SCSI Sense Data. 209 210This turns out to be a pretty good assumption. All of the Open Source 211systems (*BSD and Linux) and most of the proprietary systems have this 212kind of structure. This has been the way to manage SCSI subsystems for 213at least ten years. 214 215There are some additional basic assumptions that this driver makes- primarily 216in the arena of basic simple services like memory zeroing, memory copying, 217delay, sleep, microtime functions. It doesn't assume much more than this. 218 2195.2 Overall Driver Architecture 220 221The driver is split into a core (machine independent) module and platform 222and bus specific outer modules (machine dependent). 223 224The core code (in the files isp.c, isp_inline.h, ispvar.h, ispreg.h and 225ispmbox.h) handles: 226 227 + Chipset recognition and reset and firmware download (isp_reset) 228 + Board Initialization (isp_init) 229 + First level interrupt handling (response retrieval) (isp_intr) 230 + A SCSI command queueing entry point (isp_start) 231 + A set of control services accessed either via local requirements within 232 the core module or via an externally visible control entry point 233 (isp_control). 234 235The platform/bus specific modules (and definitions) depend on each 236platform, and they provide both definitions and functions for the core 237module's use. Generally a platform module set is split into a bus 238dependent module (where configuration is begun from and bus specific 239support functions reside) and relatively thin platform specific layer 240which serves as the interconnect with the rest of this platform's SCSI 241subsystem. 242 243For ease of bus specific access issues, a centralized soft state 244structure is maintained for each HBA instance (struct ispsoftc). This 245soft state structure contains a machine/bus dependent vector (mdvec) 246for functions that read and write hardware registers, set up DMA for the 247request/response queues and fibre channel scratch area, set up and tear 248down DMA mappings for a SCSI command, provide a pointer to firmware to 249load, and other minor things. 250 251The machine dependent outer module must provide functional entry points 252for the core module: 253 254 + A SCSI command completion handoff point (isp_done) 255 + An asynchronous event handler (isp_async) 256 + A logging/printing function (isp_prt) 257 258The machine dependent outer module code must also provide a set of 259abstracting definitions which is what the core module utilizes heavily 260to do its job. These are discussed in detail in the comments in the 261file ispvar.h, but to give a sense of the range of what is required, 262let's illustrate two basic classes of these defines. 263 264The first class are "structure definition/access" class. An 265example of these would be: 266 267 XS_T Platform SCSI transaction type (i.e., command for HBA) 268 .. 269 XS_TGT(xs) gets the target from an XS_T 270 .. 271 XS_TAG_TYPE(xs) which type of tag to use 272 .. 273 274The second class are 'functional' class definitions. Some examples of 275this class are: 276 277 MEMZERO(dst, src) platform zeroing function 278 .. 279 MBOX_WAIT_COMPLETE(struct ispsoftc *) wait for mailbox cmd to be done 280 281Note that the former is likely to be simple replacement with bzero or 282memset on most systems, while the latter could be quite complex. 283 284This soft state structure also contains different parameter information 285based upon whether this is a SCSI HBA or a Fibre Channel HBA (which is 286filled in by the code module). 287 288In order to clear up what is undoubtedly a seeming confusion of 289interconnects, a description of the typical flow of code that performs 290boards initialization and command transactions may help. 291 2925.3 Initialization Code Flow 293 294Typically a bus specific module for a platform (e.g., one that wants 295to configure a PCI card) is entered via that platform's configuration 296methods. If this module recognizes a card and can utilize or construct the 297space for the HBA instance softc, it does so, and initializes the machine 298dependent vector as well as any other platform specific information that 299can be hidden in or associated with this structure. 300 301Configuration at this point usually involves mapping in board registers 302and registering an interrupt. It's quite possible that the core module's 303isp_intr function is adequate to be the interrupt entry point, but often 304it's more useful have a bus specific wrapper module that calls isp_intr. 305 306After mapping and interrupt registry is done, isp_reset is called. 307Part of the isp_reset call may cause callbacks out to the bus dependent 308module to perform allocation and/or mapping of Request and Response 309queues (as well as a Fibre Channel scratch area if this is a Fibre 310Channel HBA). The reason this is considered 'bus dependent' is that 311only the bus dependent module may have the information that says how 312one could perform I/O mapping and dependent (e.g., on a Solaris system) 313on the Request and Response queues. Another callback can enable the *use* 314of interrupts should this platform be able to finish configuration in 315interrupt driven mode. 316 317If isp_reset is successful at resetting the QLogic chipset and downloading 318new firmware (if available) and setting it running, isp_init is called. If 319isp_init is successful in doing initial board setups (including reading 320NVRAM from the QLogic card), then this bus specicic module will call the 321platform dependent module that takes the appropriate steps to 'register' 322this HBA with this platform's SCSI subsystem. Examining either the 323OpenBSD or the NetBSD isp_pci.c or isp_sbus.c files may assist the reader 324here in clarifying some of this. 325 3265.4 Initiator Mode Command Code Flow 327 328A successful execution of isp_init will lead to the driver 'registering' 329itself with this platform's SCSI subsystem. One assumed action for this 330is the registry of a function that the SCSI subsystem for this platform 331will call when it has a SCSI command to run. 332 333The platform specific module function that receives this will do whatever 334it needs to to prepare this command for execution in the core module. This 335sounds vague, but it's also very flexible. In principle, this could be 336a complete marshalling/demarshalling of this platform's SCSI command 337structure (should it be impossible to represent in an XS_T). In addition, 338this function can also block commands from running (if, e.g., Fibre 339Channel loop state would preclude successful starting of the command). 340 341When it's ready to do so, the function isp_start is called with this 342command. This core module tries to allocate request queue space for 343this command. It also calls through the machine dependent vector 344function to make sure any DMA mapping for this command is done. 345 346Now, DMA mapping here is possibly a misnomer, as more than just 347DMA mapping can be done in this bus dependent function. This is 348also the place where any endian byte-swizzling will be done. At any 349rate, this function is called last because the process of establishing 350DMA addresses for any command may in fact consume more Request Queue 351entries than there are currently available. If the mapping and other 352functions are successful, the QLogic mailbox inbox pointer register 353is updated to indicate to the QLogic that it has a new request to 354read. 355 356If this function is unsuccessful, policy as to what to do at this point is 357left to the machine dependent platform function which called isp_start. In 358some platforms, temporary resource shortages can be handled by the main 359SCSI subsystem. In other platforms, the machine dependent code has to 360handle this. 361 362In order to keep track of commands that are in progress, the soft state 363structure contains an array of 'handles' that are associated with each 364active command. When you send a command to the QLogic firmware, a portion 365of the Request Queue entry can contain a non-zero handle identifier so 366that at a later point in time in reading either a Response Queue entry 367or from a Fast Posting mailbox completion interrupt, you can take this 368handle to find the command you were waiting on. It should be noted that 369this is probably one of the most dangerous areas of this driver. Corrupted 370handles will lead to system panics. 371 372At some later point in time an interrupt will occur. Eventually, 373isp_intr will be called. This core module will determine what the cause 374of the interrupt is, and if it is for a completing command. That is, 375it'll determine the handle and fetch the pointer to the command out of 376storage within the soft state structure. Skipping over a lot of details, 377the machine dependent code supplied function isp_done is called with the 378pointer to the completing command. This would then be the glue layer that 379informs the SCSI subsystem for this platform that a command is complete. 380 3815.5 Asynchronous Events 382 383Interrupts occur for events other than commands (mailbox or request queue 384started commands) completing. These are called Asynchronous Mailbox 385interrupts. When some external event causes the SCSI bus to be reset, 386or when a Fibre Channel loop changes state (e.g., a LIP is observed), 387this generates such an asynchronous event. 388 389Each platform module has to provide an isp_async entry point that will 390handle a set of these. This isp_async entry point also handles things 391which aren't properly async events but are simply natural outgrowths 392of code flow for another core function (see discussion on fabric device 393management below). 394 3955.6 Target Mode Code Flow 396 397This section could use a lot of expansion, but this covers the basics. 398 399The QLogic cards, when operating in target mode, follow a code flow that is 400essentially the inverse of that for intiator mode describe above. In this 401scenario, an interrupt occurs, and present on the Response Queue is a 402queue entry element defining a new command arriving from an initiator. 403 404This is passed to possibly external target mode handler. This driver 405provides some handling for this in a core module, but also leaves 406things open enough that a completely different target mode handler 407may accept this incoming queue entry. 408 409The external target mode handler then turns around forms up a response 410to this 'response' that just arrived which is then placed on the Request 411Queue and handled very much like an initiator mode command (i.e., calling 412the bus dependent DMA mapping function). If this entry completes the 413command, no more need occur. But often this handles only part of the 414requested command, so the QLogic firmware will rewrite the response 415to the initial 'response' again onto the Response Queue, whereupon the 416target mode handler will respond to that, and so on until the command 417is completely handled. 418 419Because almost no platform provides basic SCSI Subsystem target mode 420support, this design has been left extremely open ended, and as such 421it's a bit hard to describe in more detail than this. 422 4235.7 Locking Assumptions 424 425The observant reader by now is likely to have asked the question, "but what 426about locking? Or interrupt masking" by now. 427 428The basic assumption about this is that the core module does not know 429anything directly about locking or interrupt masking. It may assume that 430upon entry (e.g., via isp_start, isp_control, isp_intr) that appropriate 431locking and interrupt masking has been done. 432 433The platform dependent code may also therefore assume that if it is 434called (e.g., isp_done or isp_async) that any locking or masking that 435was in place upon the entry to the core module is still there. It is up 436to the platform dependent code to worry about avoiding any lock nesting 437issues. As an example of this, the Linux implementation simply queues 438up commands completed via the callout to isp_done, which it then pushes 439out to the SCSI subsystem after a return from it's calling isp_intr is 440executed (and locks dropped appropriately, as well as avoidance of deep 441interrupt stacks). 442 443Recent changes in the design have now eased what had been an original 444requirement that the while in the core module no locks or interrupt 445masking could be dropped. It's now up to each platform to figure out how 446to implement this. This is principally used in the execution of mailbox 447commands (which are principally used for Loop and Fabric management via 448the isp_control function). 449 4505.8 SCSI Specifics 451 452The driver core or platform dependent architecture issues that are specific 453to SCSI are few. There is a basic assumption that the QLogic firmware 454supported Automatic Request sense will work- there is no particular provision 455for disabling it's usage on a per-command basis. 456 4575.9 Fibre Channel Specifics 458 459Fibre Channel presents an interesting challenge here. The QLogic firmware 460architecture for dealing with Fibre Channel as just a 'fat' SCSI bus 461is fine on the face of it, but there are some subtle and not so subtle 462problems here. 463 4645.9.1 Firmware State 465 466Part of the initialization (isp_init) for Fibre Channel HBAs involves 467sending a command (Initialize Control Block) that establishes Node 468and Port WWNs as well as topology preferences. After this occurs, 469the QLogic firmware tries to traverese through serveral states: 470 471 FW_CONFIG_WAIT 472 FW_WAIT_AL_PA 473 FW_WAIT_LOGIN 474 FW_READY 475 FW_LOSS_OF_SYNC 476 FW_ERROR 477 FW_REINIT 478 FW_NON_PART 479 480It starts with FW_CONFIG_WAIT, attempts to get an AL_PA (if on an FC-AL 481loop instead of being connected as an N-port), waits to log into all 482FC-AL loop entities and then hopefully transitions to FW_READY state. 483 484Clearly, no command should be attempted prior to FW_READY state is 485achieved. The core internal function isp_fclink_test (reachable via 486isp_control with the ISPCTL_FCLINK_TEST function code). This function 487also determines connection topology (i.e., whether we're attached to a 488fabric or not). 489 4905.9.2. Loop State Transitions- From Nil to Ready 491 492Once the firmware has transitioned to a ready state, then the state of the 493connection to either arbitrated loop or to a fabric has to be ascertained, 494and the identity of all loop members (and fabric members validated). 495 496This can be very complicated, and it isn't made easy in that the QLogic 497firmware manages PLOGI and PRLI to devices that are on a local loop, but 498it is the driver that must manage PLOGI/PRLI with devices on the fabric. 499 500In order to manage this state an eight level staging of current "Loop" 501(where "Loop" is taken to mean FC-AL or N- or F-port connections) states 502in the following ascending order: 503 504 LOOP_NIL 505 LOOP_LIP_RCVD 506 LOOP_PDB_RCVD 507 LOOP_SCANNING_FABRIC 508 LOOP_FSCAN_DONE 509 LOOP_SCANNING_LOOP 510 LOOP_LSCAN_DONE 511 LOOP_SYNCING_PDB 512 LOOP_READY 513 514When the core code initializes the QLogic firmware, it sets the loop 515state to LOOP_NIL. The first 'LIP Received' asynchronous event sets state 516to LOOP_LIP_RCVD. This should be followed by a "Port Database Changed" 517asynchronous event which will set the state to LOOP_PDB_RCVD. Each of 518these states, when entered, causes an isp_async event call to the 519machine dependent layers with the ISPASYNC_CHANGE_NOTIFY code. 520 521After the state of LOOP_PDB_RCVD is reached, the internal core function 522isp_scan_fabric (reachable via isp_control(..ISPCTL_SCAN_FABRIC)) will, 523if the connection is to a fabric, use Simple Name Server mailbox mediated 524commands to dump the entire fabric contents. For each new entity, an 525isp_async event will be generated that says a Fabric device has arrived 526(ISPASYNC_FABRIC_DEV). The function that isp_async must perform in this 527step is to insert possibly remove devices that it wants to have the 528QLogic firmware log into (at LOOP_SYNCING_PDB state level)). 529 530After this has occurred, the state LOOP_FSCAN_DONE is set, and then the 531internal function isp_scan_loop (isp_control(...ISPCTL_SCAN_LOOP)) can 532be called which will then scan for any local (FC-AL) entries by asking 533for each possible local loop id the QLogic firmware for a Port Database 534entry. It's at this level some entries cached locally are purged 535or shifting loopids are managed (see section 5.9.4). 536 537The final step after this is to call the internal function isp_pdb_sync 538(isp_control(..ISPCTL_PDB_SYNC)). The purpose of this function is to 539then perform the PLOGI/PRLI functions for fabric devices. The next state 540entered after this is LOOP_READY, which means that the driver is ready 541to process commands to send to Fibre Channel devices. 542 5435.9.3 Fibre Channel variants of Initiator Mode Code Flow 544 545The code flow in isp_start for Fibre Channel devices is the same as it is 546for SCSI devices, but with a notable exception. 547 548Maintained within the fibre channel specific portion of the driver soft 549state structure is a distillation of the existing population of both 550local loop and fabric devices. Because Loop IDs can shift on a local 551loop but we wish to retain a 'constant' Target ID (see 5.9.4), this 552is indexed directly via the Target ID for the command (XS_TGT(xs)). 553 554If there is a valid entry for this Target ID, the command is started 555(with the stored 'Loop ID'). If not the command is completed with 556the error that is just like a SCSI Selection Timeout error. 557 558This code is currently somewhat in transition. Some platforms to 559do firmware and loop state management (as described above) at this 560point. Other platforms manage this from the machine dependent layers. The 561important function to watch in this respect is isp_fc_runstate (in 562isp_inline.h). 563 5645.9.4 "Target" in Fibre Channel is a fixed virtual construct 565 566Very few systems can cope with the notion that "Target" for a disk 567device can change while you're using it. But one of the properties of 568for arbitrated loop is that the physical bus address for a loop member 569(the AL_PA) can change depending on when and how things are inserted in 570the loop. 571 572To illustrate this, let's take an example. Let's say you start with a 573loop that has 5 disks in it. At boot time, the system will likely find 574them and see them in this order: 575 576disk# Loop ID Target ID 577disk0 0 0 578disk1 1 1 579disk2 2 2 580disk3 3 3 581disk4 4 4 582 583The driver uses 'Loop ID' when it forms requests to send a comamnd to 584each disk. However, it reports to NetBSD that things exist as 'Target 585ID'. As you can see here, there is perfect correspondence between disk, 586Loop ID and Target ID. 587 588Let's say you add a new disk between disk2 and disk3 while things are 589running. You don't really often see this, but you *could* see this where 590the loop has to renegotiate, and you end up with: 591 592disk# Loop ID Target ID 593disk0 0 0 594disk1 1 1 595disk2 2 2 596diskN 3 ? 597disk3 4 ? 598disk4 5 ? 599 600Clearly, you don't want disk3 and disk4's "Target ID" to change while you're 601running since currently mounted filesystems will get trashed. 602 603What the driver is supposed to do (this is the function of isp_scan_loop), 604is regenerate things such that the following then occurs: 605 606disk# Loop ID Target ID 607disk0 0 0 608disk1 1 1 609disk2 2 2 610diskN 3 5 611disk3 4 3 612disk4 5 4 613 614So, "Target" is a virtual entity that is maintained while you're running. 615 6166. Glossary 617 618HBA - Host Bus Adapter 619 620SCSI - Small Computer 621 6227. References 623 624Various URLs of interest: 625 626http://www.netbsd.org - NetBSD's Web Page 627http://www.openbsd.org - OpenBSD's Web Page 628https://www.freebsd.org - FreeBSD's Web Page 629 630http://www.t10.org - ANSI SCSI Commitee's Web Page 631 (SCSI Specs) 632http://www.t11.org - NCITS Device Interface Web Page 633 (Fibre Channel Specs) 634 635