1 2 Hardware that is Known To or Should Work with This Driver 3 4 50. Intro 6 7 This is not an endorsement for hardware vendors (there will be 8 no "where to buy" URLs here with a couple of exception). This 9 is simply a list of things I know work, or should work, plus 10 maybe a couple of notes as to what you should do to make it 11 work. Corrections accepted. Even better would be to send me 12 hardware to I can test it. 13 14 I'll put a rough range of costs in US$ that I know about. No doubt 15 it'll differ from your expectations. 16 171. HBAs 18 19Qlogic 2100, 2102 20 2200, 2202, 2204 21 22 There are various suffices that indicate copper or optical 23 connectors, or 33 vs. 66MHz PCI bus operation. None of these 24 have a software impact. 25 26 Approx cost: 1K$ for a 2200 27 28Qlogic 2300, 2312 29 30 These are the new 2-Gigabit cards. Optical only. 31 32 Approx cost: ?????? 33 34 35Antares P-0033, P-0034, P-0036 36 37 There many other vendors that use the Qlogic 2X00 chipset. Some older 38 2100 boards (not on this list) have a bug in the ROM that causes a 39 failure to download newer firmware that is larger than 0x7fff words. 40 41 Approx cost: 850$ for a P-0036 42 43 44 45 In general, the 2200 class chip is to be preferred. 46 47 482. Hubs 49 50Vixel 1000 51Vixel 2000 52 Of the two, the 1000 (7 ports, vs. 12 ports) has had fewer problems- 53 it's an old workhorse. 54 55 56 Approx cost: 1.5K$ for Vixel 1000, 2.5K$ for 2000 57 58Gadzoox Cappellix 3000 59 Don't forget to use telnet to configure the Cappellix ports 60 to the role you're using them for- otherwise things don't 61 work well at all. 62 63 (cost: I have no idea... certainly less than a switch) 64 653. Switches 66 67Brocade Silkworm II 68Brocade 2400 69(other brocades should be fine) 70 71 Especially with revision 2 or higher f/w, this is now best 72 of breed for fabrics or segmented loop (which Brocade 73 calls "QuickLoop"). 74 75 For the Silkworm II, set operating mode to "Tachyon" (mode 3). 76 77 The web interace isn't good- but telnet is what I prefer anyhow. 78 79 You can't connect a Silkworm II and the other Brocades together 80 as E-ports to make a large fabric (at least with the f/w *I* 81 had for the Silkworm II). 82 83 Approx cost of a Brocade 2400 with no GBICs is about 8K$ when 84 I recently checked the US Government SEWP price list- no doubt 85 it'll be a bit more for others. I'd assume around 10K$. 86 87ANCOR SA-8 88 89 This also is a fine switch, but you have to use a browser 90 with working java to manage it- which is a bit of a pain. 91 This also supports fabric and segmented loop. 92 93 These switches don't form E-ports with each other for a larger 94 fabric. 95 96 (cost: no idea) 97 98McData (model unknown) 99 100 I tried one exactly once for 30 minutes. Seemed to work once 101 I added the "register FC4 types" command to the driver. 102 103 (cost: very very expensive, 40K$ plus) 104 1054. Cables/GBICs 106 107 Multimode optical is adequate for Fibre Channel- the same cable is 108 used for Gigabit Ethernet. 109 110 Copper DB-9 and Copper HSS-DC connectors are also fine. Copper && 111 Optical both are rated to 1.026Gbit- copper is naturally shorter 112 (the longest I've used is a 15meter cable but it's supposed to go 113 longer). 114 115 The reason to use copper instead of optical is that if step on one of 116 the really fat DB-9 cables you can get, it'll survive. Optical usually 117 dies quickly if you step on it. 118 119 Approx cost: I don't know what optical is- you can expect to pay maybe 120 a 100$ for a 3m copper cable. 121 122GBICs- 123 124 I use Finisar copper and IBM Opticals. 125 126 Approx Cost: Copper GBICs are 70$ each. Opticals are twice that or more. 127 128 129Vendor: (this is the one exception I'll make because it turns out to be 130 an incredible pain to find FC copper cabling and GBICs- the source I 131 use for GBICs and copper cables is http://www.scsi-cables.com) 132 133 134Other: 135 There now is apparently a source for little connector boards 136 to connect to bare drives: http://www.cinonic.com. 137 138 1395. Storage JBODs/RAID 140 141JMR 4-Bay 142 143 Rinky-tink, but a solid 4 bay loop only entry model. 144 145 I paid 1000$ for mine- overprice, IMO. 146 147JMR Fortra 148 149 I rather like this box. The blue LEDs are a very nice touch- you 150 can see them very clearly from 50 feet away. 151 152 I paid 2000$ for one used. 153 154Sun A5X00 155 156 Very expensive (in my opinion) but well crafted. Has two SES 157 instances, so you can use the ses driver (and the example 158 code in /usr/share/examples) for power/thermal/slot monitoring. 159 160 Approx Cost: The last I saw for a price list item on this was 22K$ 161 for an unpopulated (no disk drive) A5X00. 162 163 164DataDirect E1000 RAID 165 166 Don't connect both SCSI and FC interfaces at the same time- a SCSI 167 reset will cause the DataDirect to think you want to use the SCSI 168 interface and a LIP on the FC interface will cause it to think you 169 want to use the FC interface. Use only one connector at a time so 170 both you and the DataDirect are sure about what you want. 171 172 Cost: I have no idea. 173 174Veritas ServPoint 175 176 This is a software storage virtualization engine that 177 runs on Sparc/Solaris in target mode for frontend 178 and with other FC or SCSI as the backend storage. FreeBSD 179 has been used extensively to test it. 180 181 182 Cost: I have no idea. 183 1846. Disk Drives 185 186 I have used lots of different Seagate and a few IBM drives and 187 typically have had few problems with them. These are the bare 188 drives with 40-pin SCA connectors in back. They go into the JBODs 189 you assemble. 190 191 Seagate does make, but I can no longer find, a little paddleboard 192 single drive connector that goes from DB-9 FC to the 40-pin SCA 193 connector- primarily for you to try and evaluate a single FC drive. 194 195 All FC-AL disk drives are dual ported (i.e., have separte 'A' and 196 'B' ports- which are completely separate loops). This seems to work 197 reasonably enough, but I haven't tested it much. It really depends 198 on the JBOD you put them to carry this dual port to the outside 199 world. The JMR boxes have it. The Sun A5X00 you have to pay for 200 an extra IB card to carry it out. 201 202 Approx Cost: You'll find that FC drives are the same cost if not 203 slightly cheaper than the equivalent Ultra3 SCSI drives. 204 2057. Recommended Configurations 206 207These are recommendations that are biased toward the cautious side. They 208do not represent formal engineering commitments- just suggestions as to 209what I would expect to work. 210 211A. The simpletst form of a connection topology I can suggest for 212a small SAN (i.e., replacement for SCSI JBOD/RAID): 213 214HOST 2152xxx <----------> Single Unit of Storage (JBOD, RAID) 216 217This is called a PL_DA (Private Loop, Direct Attach) topology. 218 219B. The next most simple form of a connection topology I can suggest for 220a medium local SAN (where you do not plan to do dynamic insertion 221and removal of devices while I/Os are active): 222 223HOST 2242xxx <----------> +-------- 225 | Vixel | 226 | 1000 | 227 | +<---> Storage 228 | | 229 | +<---> Storage 230 | | 231 | +<---> Storage 232 -------- 233 234This is a Private Loop topology. Remember that this can get very unstable 235if you make it too long. A good practice is to try it in a staged fashion. 236 237It is possible with some units to "daisy chain", e.g.: 238 239HOST 2402xxx <----------> (JBOD, RAID) <--------> (JBOD, RAID) 241 242In practice I have had poor results with these configurations. They *should* 243work fine, but for both the JMR and the Sun A5X00 I tend to get LIP storms 244and so the second unit just isn't seen and the loop isn't stable. 245 246Now, this could simply be my lack of clean, newer, h/w (or, in general, 247a lack of h/w), but I would recommend the use of a hub if you want to 248stay with Private Loop and have more than one FC target. 249 250You should also note this can begin to be the basis for a shared SAN 251solution. For example, the above configuration can be extended to be: 252 253HOST 2542xxx <----------> +-------- 255 | Vixel | 256 | 1000 | 257 | +<---> Storage 258 | | 259 | +<---> Storage 260 | | 261 | +<---> Storage 262HOST | | 2632xxx <----------> +-------- 264 265However, note that there is nothing to mediate locking of devices, and 266it is also conceivable that the reboot of one host can, by causing 267a LIP storm, cause problems with the I/Os from the other host. 268(in other words, this topology hasn't really been made safe yet for 269this driver). 270 271D. You can repeat the topology in #B with a switch that is set to be 272in segmented loop mode. This avoids LIPs propagating where you don't 273want them to- and this makes for a much more reliable, if more expensive, 274SAN. 275 276E. The next level of complexity is a Switched Fabric. The following topology 277is good when you start to begin to get to want more performance. Private 278and Public Arbitrated Loop, while 100MB/s, is a shared medium. Direct 279connections to a switch can run full-duplex at full speed. 280 281HOST 2822xxx <----------> +--------- 283 | Brocade| 284 | 2400 | 285 | +<---> Storage 286 | | 287 | +<---> Storage 288 | | 289 | +<---> Storage 290HOST | | 2912xxx <----------> +--------- 292 293 294I would call this the best configuration available now. It can expand 295substantially if you cascade switches. 296 297There is a hard limit of about 253 devices for each Qlogic HBA- and the 298fabric login policy is simplistic (log them in as you find them). If 299somebody actually runs into a configuration that's larger, let me know 300and I'll work on some tools that would allow you some policy choices 301as to which would be interesting devices to actually connect to. 302 303 304