1/* $FreeBSD$ */ 2 3 Hardware that is Known To or Should Work with This Driver 4 5 60. Intro 7 8 This is not an endorsement for hardware vendors (there will be 9 no "where to buy" URLs here with a couple of exception). This 10 is simply a list of things I know work, or should work, plus 11 maybe a couple of notes as to what you should do to make it 12 work. Corrections accepted. Even better would be to send me 13 hardware to I can test it. 14 15 I'll put a rough range of costs in US$ that I know about. No doubt 16 it'll differ from your expectations. 17 181. HBAs 19 20Qlogic 2100, 2102 21 2200, 2202, 2204 22 23 There are various suffices that indicate copper or optical 24 connectors, or 33 vs. 66MHz PCI bus operation. None of these 25 have a software impact. 26 27 Approx cost: 1K$ for a 2200 28 29Qlogic 2300, 2312 30 31 These are the new 2-Gigabit cards. Optical only. 32 33 Approx cost: ?????? 34 35 36Antares P-0033, P-0034, P-0036 37 38 There many other vendors that use the Qlogic 2X00 chipset. Some older 39 2100 boards (not on this list) have a bug in the ROM that causes a 40 failure to download newer firmware that is larger than 0x7fff words. 41 42 Approx cost: 850$ for a P-0036 43 44 45 46 In general, the 2200 class chip is to be preferred. 47 48 492. Hubs 50 51Vixel 1000 52Vixel 2000 53 Of the two, the 1000 (7 ports, vs. 12 ports) has had fewer problems- 54 it's an old workhorse. 55 56 57 Approx cost: 1.5K$ for Vixel 1000, 2.5K$ for 2000 58 59Gadzoox Cappellix 3000 60 Don't forget to use telnet to configure the Cappellix ports 61 to the role you're using them for- otherwise things don't 62 work well at all. 63 64 (cost: I have no idea... certainly less than a switch) 65 663. Switches 67 68Brocade Silkworm II 69Brocade 2400 70(other brocades should be fine) 71 72 Especially with revision 2 or higher f/w, this is now best 73 of breed for fabrics or segmented loop (which Brocade 74 calls "QuickLoop"). 75 76 For the Silkworm II, set operating mode to "Tachyon" (mode 3). 77 78 The web interace isn't good- but telnet is what I prefer anyhow. 79 80 You can't connect a Silkworm II and the other Brocades together 81 as E-ports to make a large fabric (at least with the f/w *I* 82 had for the Silkworm II). 83 84 Approx cost of a Brocade 2400 with no GBICs is about 8K$ when 85 I recently checked the US Government SEWP price list- no doubt 86 it'll be a bit more for others. I'd assume around 10K$. 87 88ANCOR SA-8 89 90 This also is a fine switch, but you have to use a browser 91 with working java to manage it- which is a bit of a pain. 92 This also supports fabric and segmented loop. 93 94 These switches don't form E-ports with each other for a larger 95 fabric. 96 97 (cost: no idea) 98 99McData (model unknown) 100 101 I tried one exactly once for 30 minutes. Seemed to work once 102 I added the "register FC4 types" command to the driver. 103 104 (cost: very very expensive, 40K$ plus) 105 1064. Cables/GBICs 107 108 Multimode optical is adequate for Fibre Channel- the same cable is 109 used for Gigabit Ethernet. 110 111 Copper DB-9 and Copper HSS-DC connectors are also fine. Copper && 112 Optical both are rated to 1.026Gbit- copper is naturally shorter 113 (the longest I've used is a 15meter cable but it's supposed to go 114 longer). 115 116 The reason to use copper instead of optical is that if step on one of 117 the really fat DB-9 cables you can get, it'll survive. Optical usually 118 dies quickly if you step on it. 119 120 Approx cost: I don't know what optical is- you can expect to pay maybe 121 a 100$ for a 3m copper cable. 122 123GBICs- 124 125 I use Finisar copper and IBM Opticals. 126 127 Approx Cost: Copper GBICs are 70$ each. Opticals are twice that or more. 128 129 130Vendor: (this is the one exception I'll make because it turns out to be 131 an incredible pain to find FC copper cabling and GBICs- the source I 132 use for GBICs and copper cables is http://www.scsi-cables.com) 133 134 135Other: 136 There now is apparently a source for little connector boards 137 to connect to bare drives: http://www.cinonic.com. 138 139 1405. Storage JBODs/RAID 141 142JMR 4-Bay 143 144 Rinky-tink, but a solid 4 bay loop only entry model. 145 146 I paid 1000$ for mine- overprice, IMO. 147 148JMR Fortra 149 150 I rather like this box. The blue LEDs are a very nice touch- you 151 can see them very clearly from 50 feet away. 152 153 I paid 2000$ for one used. 154 155Sun A5X00 156 157 Very expensive (in my opinion) but well crafted. Has two SES 158 instances, so you can use the ses driver (and the example 159 code in /usr/share/examples) for power/thermal/slot monitoring. 160 161 Approx Cost: The last I saw for a price list item on this was 22K$ 162 for an unpopulated (no disk drive) A5X00. 163 164 165DataDirect E1000 RAID 166 167 Don't connect both SCSI and FC interfaces at the same time- a SCSI 168 reset will cause the DataDirect to think you want to use the SCSI 169 interface and a LIP on the FC interface will cause it to think you 170 want to use the FC interface. Use only one connector at a time so 171 both you and the DataDirect are sure about what you want. 172 173 Cost: I have no idea. 174 175Veritas ServPoint 176 177 This is a software storage virtualization engine that 178 runs on Sparc/Solaris in target mode for frontend 179 and with other FC or SCSI as the backend storage. FreeBSD 180 has been used extensively to test it. 181 182 183 Cost: I have no idea. 184 1856. Disk Drives 186 187 I have used lots of different Seagate and a few IBM drives and 188 typically have had few problems with them. These are the bare 189 drives with 40-pin SCA connectors in back. They go into the JBODs 190 you assemble. 191 192 Seagate does make, but I can no longer find, a little paddleboard 193 single drive connector that goes from DB-9 FC to the 40-pin SCA 194 connector- primarily for you to try and evaluate a single FC drive. 195 196 All FC-AL disk drives are dual ported (i.e., have separte 'A' and 197 'B' ports- which are completely separate loops). This seems to work 198 reasonably enough, but I haven't tested it much. It really depends 199 on the JBOD you put them to carry this dual port to the outside 200 world. The JMR boxes have it. The Sun A5X00 you have to pay for 201 an extra IB card to carry it out. 202 203 Approx Cost: You'll find that FC drives are the same cost if not 204 slightly cheaper than the equivalent Ultra3 SCSI drives. 205 2067. Recommended Configurations 207 208These are recommendations that are biased toward the cautious side. They 209do not represent formal engineering commitments- just suggestions as to 210what I would expect to work. 211 212A. The simpletst form of a connection topology I can suggest for 213a small SAN (i.e., replacement for SCSI JBOD/RAID): 214 215HOST 2162xxx <----------> Single Unit of Storage (JBOD, RAID) 217 218This is called a PL_DA (Private Loop, Direct Attach) topology. 219 220B. The next most simple form of a connection topology I can suggest for 221a medium local SAN (where you do not plan to do dynamic insertion 222and removal of devices while I/Os are active): 223 224HOST 2252xxx <----------> +-------- 226 | Vixel | 227 | 1000 | 228 | +<---> Storage 229 | | 230 | +<---> Storage 231 | | 232 | +<---> Storage 233 -------- 234 235This is a Private Loop topology. Remember that this can get very unstable 236if you make it too long. A good practice is to try it in a staged fashion. 237 238It is possible with some units to "daisy chain", e.g.: 239 240HOST 2412xxx <----------> (JBOD, RAID) <--------> (JBOD, RAID) 242 243In practice I have had poor results with these configurations. They *should* 244work fine, but for both the JMR and the Sun A5X00 I tend to get LIP storms 245and so the second unit just isn't seen and the loop isn't stable. 246 247Now, this could simply be my lack of clean, newer, h/w (or, in general, 248a lack of h/w), but I would recommend the use of a hub if you want to 249stay with Private Loop and have more than one FC target. 250 251You should also note this can begin to be the basis for a shared SAN 252solution. For example, the above configuration can be extended to be: 253 254HOST 2552xxx <----------> +-------- 256 | Vixel | 257 | 1000 | 258 | +<---> Storage 259 | | 260 | +<---> Storage 261 | | 262 | +<---> Storage 263HOST | | 2642xxx <----------> +-------- 265 266However, note that there is nothing to mediate locking of devices, and 267it is also conceivable that the reboot of one host can, by causing 268a LIP storm, cause problems with the I/Os from the other host. 269(in other words, this topology hasn't really been made safe yet for 270this driver). 271 272D. You can repeat the topology in #B with a switch that is set to be 273in segmented loop mode. This avoids LIPs propagating where you don't 274want them to- and this makes for a much more reliable, if more expensive, 275SAN. 276 277E. The next level of complexity is a Switched Fabric. The following topology 278is good when you start to begin to get to want more performance. Private 279and Public Arbitrated Loop, while 100MB/s, is a shared medium. Direct 280connections to a switch can run full-duplex at full speed. 281 282HOST 2832xxx <----------> +--------- 284 | Brocade| 285 | 2400 | 286 | +<---> Storage 287 | | 288 | +<---> Storage 289 | | 290 | +<---> Storage 291HOST | | 2922xxx <----------> +--------- 293 294 295I would call this the best configuration available now. It can expand 296substantially if you cascade switches. 297 298There is a hard limit of about 253 devices for each Qlogic HBA- and the 299fabric login policy is simplistic (log them in as you find them). If 300somebody actually runs into a configuration that's larger, let me know 301and I'll work on some tools that would allow you some policy choices 302as to which would be interesting devices to actually connect to. 303 304 305