1 /* 2 * CDDL HEADER START 3 * 4 * The contents of this file are subject to the terms of the 5 * Common Development and Distribution License (the "License"). 6 * You may not use this file except in compliance with the License. 7 * 8 * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE 9 * or http://www.opensolaris.org/os/licensing. 10 * See the License for the specific language governing permissions 11 * and limitations under the License. 12 * 13 * When distributing Covered Code, include this CDDL HEADER in each 14 * file and include the License file at usr/src/OPENSOLARIS.LICENSE. 15 * If applicable, add the following below this CDDL HEADER, with the 16 * fields enclosed by brackets "[]" replaced with your own identifying 17 * information: Portions Copyright [yyyy] [name of copyright owner] 18 * 19 * CDDL HEADER END 20 */ 21 22 /* 23 * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. 24 * Copyright 2019 Joyent, Inc. 25 * Copyright 2015 Garrett D'Amore <garrett@damore.org> 26 */ 27 28 /* 29 * MAC Services Module 30 * 31 * The GLDv3 framework locking - The MAC layer 32 * -------------------------------------------- 33 * 34 * The MAC layer is central to the GLD framework and can provide the locking 35 * framework needed for itself and for the use of MAC clients. MAC end points 36 * are fairly disjoint and don't share a lot of state. So a coarse grained 37 * multi-threading scheme is to single thread all create/modify/delete or set 38 * type of control operations on a per mac end point while allowing data threads 39 * concurrently. 40 * 41 * Control operations (set) that modify a mac end point are always serialized on 42 * a per mac end point basis, We have at most 1 such thread per mac end point 43 * at a time. 44 * 45 * All other operations that are not serialized are essentially multi-threaded. 46 * For example a control operation (get) like getting statistics which may not 47 * care about reading values atomically or data threads sending or receiving 48 * data. Mostly these type of operations don't modify the control state. Any 49 * state these operations care about are protected using traditional locks. 50 * 51 * The perimeter only serializes serial operations. It does not imply there 52 * aren't any other concurrent operations. However a serialized operation may 53 * sometimes need to make sure it is the only thread. In this case it needs 54 * to use reference counting mechanisms to cv_wait until any current data 55 * threads are done. 56 * 57 * The mac layer itself does not hold any locks across a call to another layer. 58 * The perimeter is however held across a down call to the driver to make the 59 * whole control operation atomic with respect to other control operations. 60 * Also the data path and get type control operations may proceed concurrently. 61 * These operations synchronize with the single serial operation on a given mac 62 * end point using regular locks. The perimeter ensures that conflicting 63 * operations like say a mac_multicast_add and a mac_multicast_remove on the 64 * same mac end point don't interfere with each other and also ensures that the 65 * changes in the mac layer and the call to the underlying driver to say add a 66 * multicast address are done atomically without interference from a thread 67 * trying to delete the same address. 68 * 69 * For example, consider 70 * mac_multicst_add() 71 * { 72 * mac_perimeter_enter(); serialize all control operations 73 * 74 * grab list lock protect against access by data threads 75 * add to list 76 * drop list lock 77 * 78 * call driver's mi_multicst 79 * 80 * mac_perimeter_exit(); 81 * } 82 * 83 * To lessen the number of serialization locks and simplify the lock hierarchy, 84 * we serialize all the control operations on a per mac end point by using a 85 * single serialization lock called the perimeter. We allow recursive entry into 86 * the perimeter to facilitate use of this mechanism by both the mac client and 87 * the MAC layer itself. 88 * 89 * MAC client means an entity that does an operation on a mac handle 90 * obtained from a mac_open/mac_client_open. Similarly MAC driver means 91 * an entity that does an operation on a mac handle obtained from a 92 * mac_register. An entity could be both client and driver but on different 93 * handles eg. aggr. and should only make the corresponding mac interface calls 94 * i.e. mac driver interface or mac client interface as appropriate for that 95 * mac handle. 96 * 97 * General rules. 98 * ------------- 99 * 100 * R1. The lock order of upcall threads is natually opposite to downcall 101 * threads. Hence upcalls must not hold any locks across layers for fear of 102 * recursive lock enter and lock order violation. This applies to all layers. 103 * 104 * R2. The perimeter is just another lock. Since it is held in the down 105 * direction, acquiring the perimeter in an upcall is prohibited as it would 106 * cause a deadlock. This applies to all layers. 107 * 108 * Note that upcalls that need to grab the mac perimeter (for example 109 * mac_notify upcalls) can still achieve that by posting the request to a 110 * thread, which can then grab all the required perimeters and locks in the 111 * right global order. Note that in the above example the mac layer iself 112 * won't grab the mac perimeter in the mac_notify upcall, instead the upcall 113 * to the client must do that. Please see the aggr code for an example. 114 * 115 * MAC client rules 116 * ---------------- 117 * 118 * R3. A MAC client may use the MAC provided perimeter facility to serialize 119 * control operations on a per mac end point. It does this by by acquring 120 * and holding the perimeter across a sequence of calls to the mac layer. 121 * This ensures atomicity across the entire block of mac calls. In this 122 * model the MAC client must not hold any client locks across the calls to 123 * the mac layer. This model is the preferred solution. 124 * 125 * R4. However if a MAC client has a lot of global state across all mac end 126 * points the per mac end point serialization may not be sufficient. In this 127 * case the client may choose to use global locks or use its own serialization. 128 * To avoid deadlocks, these client layer locks held across the mac calls 129 * in the control path must never be acquired by the data path for the reason 130 * mentioned below. 131 * 132 * (Assume that a control operation that holds a client lock blocks in the 133 * mac layer waiting for upcall reference counts to drop to zero. If an upcall 134 * data thread that holds this reference count, tries to acquire the same 135 * client lock subsequently it will deadlock). 136 * 137 * A MAC client may follow either the R3 model or the R4 model, but can't 138 * mix both. In the former, the hierarchy is Perim -> client locks, but in 139 * the latter it is client locks -> Perim. 140 * 141 * R5. MAC clients must make MAC calls (excluding data calls) in a cv_wait'able 142 * context since they may block while trying to acquire the perimeter. 143 * In addition some calls may block waiting for upcall refcnts to come down to 144 * zero. 145 * 146 * R6. MAC clients must make sure that they are single threaded and all threads 147 * from the top (in particular data threads) have finished before calling 148 * mac_client_close. The MAC framework does not track the number of client 149 * threads using the mac client handle. Also mac clients must make sure 150 * they have undone all the control operations before calling mac_client_close. 151 * For example mac_unicast_remove/mac_multicast_remove to undo the corresponding 152 * mac_unicast_add/mac_multicast_add. 153 * 154 * MAC framework rules 155 * ------------------- 156 * 157 * R7. The mac layer itself must not hold any mac layer locks (except the mac 158 * perimeter) across a call to any other layer from the mac layer. The call to 159 * any other layer could be via mi_* entry points, classifier entry points into 160 * the driver or via upcall pointers into layers above. The mac perimeter may 161 * be acquired or held only in the down direction, for e.g. when calling into 162 * a mi_* driver enty point to provide atomicity of the operation. 163 * 164 * R8. Since it is not guaranteed (see R14) that drivers won't hold locks across 165 * mac driver interfaces, the MAC layer must provide a cut out for control 166 * interfaces like upcall notifications and start them in a separate thread. 167 * 168 * R9. Note that locking order also implies a plumbing order. For example 169 * VNICs are allowed to be created over aggrs, but not vice-versa. An attempt 170 * to plumb in any other order must be failed at mac_open time, otherwise it 171 * could lead to deadlocks due to inverse locking order. 172 * 173 * R10. MAC driver interfaces must not block since the driver could call them 174 * in interrupt context. 175 * 176 * R11. Walkers must preferably not hold any locks while calling walker 177 * callbacks. Instead these can operate on reference counts. In simple 178 * callbacks it may be ok to hold a lock and call the callbacks, but this is 179 * harder to maintain in the general case of arbitrary callbacks. 180 * 181 * R12. The MAC layer must protect upcall notification callbacks using reference 182 * counts rather than holding locks across the callbacks. 183 * 184 * R13. Given the variety of drivers, it is preferable if the MAC layer can make 185 * sure that any pointers (such as mac ring pointers) it passes to the driver 186 * remain valid until mac unregister time. Currently the mac layer achieves 187 * this by using generation numbers for rings and freeing the mac rings only 188 * at unregister time. The MAC layer must provide a layer of indirection and 189 * must not expose underlying driver rings or driver data structures/pointers 190 * directly to MAC clients. 191 * 192 * MAC driver rules 193 * ---------------- 194 * 195 * R14. It would be preferable if MAC drivers don't hold any locks across any 196 * mac call. However at a minimum they must not hold any locks across data 197 * upcalls. They must also make sure that all references to mac data structures 198 * are cleaned up and that it is single threaded at mac_unregister time. 199 * 200 * R15. MAC driver interfaces don't block and so the action may be done 201 * asynchronously in a separate thread as for example handling notifications. 202 * The driver must not assume that the action is complete when the call 203 * returns. 204 * 205 * R16. Drivers must maintain a generation number per Rx ring, and pass it 206 * back to mac_rx_ring(); They are expected to increment the generation 207 * number whenever the ring's stop routine is invoked. 208 * See comments in mac_rx_ring(); 209 * 210 * R17 Similarly mi_stop is another synchronization point and the driver must 211 * ensure that all upcalls are done and there won't be any future upcall 212 * before returning from mi_stop. 213 * 214 * R18. The driver may assume that all set/modify control operations via 215 * the mi_* entry points are single threaded on a per mac end point. 216 * 217 * Lock and Perimeter hierarchy scenarios 218 * --------------------------------------- 219 * 220 * i_mac_impl_lock -> mi_rw_lock -> srs_lock -> s_ring_lock[i_mac_tx_srs_notify] 221 * 222 * ft_lock -> fe_lock [mac_flow_lookup] 223 * 224 * mi_rw_lock -> fe_lock [mac_bcast_send] 225 * 226 * srs_lock -> mac_bw_lock [mac_rx_srs_drain_bw] 227 * 228 * cpu_lock -> mac_srs_g_lock -> srs_lock -> s_ring_lock [mac_walk_srs_and_bind] 229 * 230 * i_dls_devnet_lock -> mac layer locks [dls_devnet_rename] 231 * 232 * Perimeters are ordered P1 -> P2 -> P3 from top to bottom in order of mac 233 * client to driver. In the case of clients that explictly use the mac provided 234 * perimeter mechanism for its serialization, the hierarchy is 235 * Perimeter -> mac layer locks, since the client never holds any locks across 236 * the mac calls. In the case of clients that use its own locks the hierarchy 237 * is Client locks -> Mac Perim -> Mac layer locks. The client never explicitly 238 * calls mac_perim_enter/exit in this case. 239 * 240 * Subflow creation rules 241 * --------------------------- 242 * o In case of a user specified cpulist present on underlying link and flows, 243 * the flows cpulist must be a subset of the underlying link. 244 * o In case of a user specified fanout mode present on link and flow, the 245 * subflow fanout count has to be less than or equal to that of the 246 * underlying link. The cpu-bindings for the subflows will be a subset of 247 * the underlying link. 248 * o In case if no cpulist specified on both underlying link and flow, the 249 * underlying link relies on a MAC tunable to provide out of box fanout. 250 * The subflow will have no cpulist (the subflow will be unbound) 251 * o In case if no cpulist is specified on the underlying link, a subflow can 252 * carry either a user-specified cpulist or fanout count. The cpu-bindings 253 * for the subflow will not adhere to restriction that they need to be subset 254 * of the underlying link. 255 * o In case where the underlying link is carrying either a user specified 256 * cpulist or fanout mode and for a unspecified subflow, the subflow will be 257 * created unbound. 258 * o While creating unbound subflows, bandwidth mode changes attempt to 259 * figure a right fanout count. In such cases the fanout count will override 260 * the unbound cpu-binding behavior. 261 * o In addition to this, while cycling between flow and link properties, we 262 * impose a restriction that if a link property has a subflow with 263 * user-specified attributes, we will not allow changing the link property. 264 * The administrator needs to reset all the user specified properties for the 265 * subflows before attempting a link property change. 266 * Some of the above rules can be overridden by specifying additional command 267 * line options while creating or modifying link or subflow properties. 268 * 269 * Datapath 270 * -------- 271 * 272 * For information on the datapath, the world of soft rings, hardware rings, how 273 * it is structured, and the path of an mblk_t between a driver and a mac 274 * client, see mac_sched.c. 275 */ 276 277 #include <sys/types.h> 278 #include <sys/conf.h> 279 #include <sys/id_space.h> 280 #include <sys/esunddi.h> 281 #include <sys/stat.h> 282 #include <sys/mkdev.h> 283 #include <sys/stream.h> 284 #include <sys/strsun.h> 285 #include <sys/strsubr.h> 286 #include <sys/dlpi.h> 287 #include <sys/list.h> 288 #include <sys/modhash.h> 289 #include <sys/mac_provider.h> 290 #include <sys/mac_client_impl.h> 291 #include <sys/mac_soft_ring.h> 292 #include <sys/mac_stat.h> 293 #include <sys/mac_impl.h> 294 #include <sys/mac.h> 295 #include <sys/dls.h> 296 #include <sys/dld.h> 297 #include <sys/modctl.h> 298 #include <sys/fs/dv_node.h> 299 #include <sys/thread.h> 300 #include <sys/proc.h> 301 #include <sys/callb.h> 302 #include <sys/cpuvar.h> 303 #include <sys/atomic.h> 304 #include <sys/bitmap.h> 305 #include <sys/sdt.h> 306 #include <sys/mac_flow.h> 307 #include <sys/ddi_intr_impl.h> 308 #include <sys/disp.h> 309 #include <sys/sdt.h> 310 #include <sys/vnic.h> 311 #include <sys/vnic_impl.h> 312 #include <sys/vlan.h> 313 #include <inet/ip.h> 314 #include <inet/ip6.h> 315 #include <sys/exacct.h> 316 #include <sys/exacct_impl.h> 317 #include <inet/nd.h> 318 #include <sys/ethernet.h> 319 #include <sys/pool.h> 320 #include <sys/pool_pset.h> 321 #include <sys/cpupart.h> 322 #include <inet/wifi_ioctl.h> 323 #include <net/wpa.h> 324 325 #define IMPL_HASHSZ 67 /* prime */ 326 327 kmem_cache_t *i_mac_impl_cachep; 328 mod_hash_t *i_mac_impl_hash; 329 krwlock_t i_mac_impl_lock; 330 uint_t i_mac_impl_count; 331 static kmem_cache_t *mac_ring_cache; 332 static id_space_t *minor_ids; 333 static uint32_t minor_count; 334 static pool_event_cb_t mac_pool_event_reg; 335 336 /* 337 * Logging stuff. Perhaps mac_logging_interval could be broken into 338 * mac_flow_log_interval and mac_link_log_interval if we want to be 339 * able to schedule them differently. 340 */ 341 uint_t mac_logging_interval; 342 boolean_t mac_flow_log_enable; 343 boolean_t mac_link_log_enable; 344 timeout_id_t mac_logging_timer; 345 346 #define MACTYPE_KMODDIR "mac" 347 #define MACTYPE_HASHSZ 67 348 static mod_hash_t *i_mactype_hash; 349 /* 350 * i_mactype_lock synchronizes threads that obtain references to mactype_t 351 * structures through i_mactype_getplugin(). 352 */ 353 static kmutex_t i_mactype_lock; 354 355 /* 356 * mac_tx_percpu_cnt 357 * 358 * Number of per cpu locks per mac_client_impl_t. Used by the transmit side 359 * in mac_tx to reduce lock contention. This is sized at boot time in mac_init. 360 * mac_tx_percpu_cnt_max is settable in /etc/system and must be a power of 2. 361 * Per cpu locks may be disabled by setting mac_tx_percpu_cnt_max to 1. 362 */ 363 int mac_tx_percpu_cnt; 364 int mac_tx_percpu_cnt_max = 128; 365 366 /* 367 * Call back functions for the bridge module. These are guaranteed to be valid 368 * when holding a reference on a link or when holding mip->mi_bridge_lock and 369 * mi_bridge_link is non-NULL. 370 */ 371 mac_bridge_tx_t mac_bridge_tx_cb; 372 mac_bridge_rx_t mac_bridge_rx_cb; 373 mac_bridge_ref_t mac_bridge_ref_cb; 374 mac_bridge_ls_t mac_bridge_ls_cb; 375 376 static int i_mac_constructor(void *, void *, int); 377 static void i_mac_destructor(void *, void *); 378 static int i_mac_ring_ctor(void *, void *, int); 379 static void i_mac_ring_dtor(void *, void *); 380 static mblk_t *mac_rx_classify(mac_impl_t *, mac_resource_handle_t, mblk_t *); 381 void mac_tx_client_flush(mac_client_impl_t *); 382 void mac_tx_client_block(mac_client_impl_t *); 383 static void mac_rx_ring_quiesce(mac_ring_t *, uint_t); 384 static int mac_start_group_and_rings(mac_group_t *); 385 static void mac_stop_group_and_rings(mac_group_t *); 386 static void mac_pool_event_cb(pool_event_t, int, void *); 387 388 typedef struct netinfo_s { 389 list_node_t ni_link; 390 void *ni_record; 391 int ni_size; 392 int ni_type; 393 } netinfo_t; 394 395 /* 396 * Module initialization functions. 397 */ 398 399 void 400 mac_init(void) 401 { 402 mac_tx_percpu_cnt = ((boot_max_ncpus == -1) ? max_ncpus : 403 boot_max_ncpus); 404 405 /* Upper bound is mac_tx_percpu_cnt_max */ 406 if (mac_tx_percpu_cnt > mac_tx_percpu_cnt_max) 407 mac_tx_percpu_cnt = mac_tx_percpu_cnt_max; 408 409 if (mac_tx_percpu_cnt < 1) { 410 /* Someone set max_tx_percpu_cnt_max to 0 or less */ 411 mac_tx_percpu_cnt = 1; 412 } 413 414 ASSERT(mac_tx_percpu_cnt >= 1); 415 mac_tx_percpu_cnt = (1 << highbit(mac_tx_percpu_cnt - 1)); 416 /* 417 * Make it of the form 2**N - 1 in the range 418 * [0 .. mac_tx_percpu_cnt_max - 1] 419 */ 420 mac_tx_percpu_cnt--; 421 422 i_mac_impl_cachep = kmem_cache_create("mac_impl_cache", 423 sizeof (mac_impl_t), 0, i_mac_constructor, i_mac_destructor, 424 NULL, NULL, NULL, 0); 425 ASSERT(i_mac_impl_cachep != NULL); 426 427 mac_ring_cache = kmem_cache_create("mac_ring_cache", 428 sizeof (mac_ring_t), 0, i_mac_ring_ctor, i_mac_ring_dtor, NULL, 429 NULL, NULL, 0); 430 ASSERT(mac_ring_cache != NULL); 431 432 i_mac_impl_hash = mod_hash_create_extended("mac_impl_hash", 433 IMPL_HASHSZ, mod_hash_null_keydtor, mod_hash_null_valdtor, 434 mod_hash_bystr, NULL, mod_hash_strkey_cmp, KM_SLEEP); 435 rw_init(&i_mac_impl_lock, NULL, RW_DEFAULT, NULL); 436 437 mac_flow_init(); 438 mac_soft_ring_init(); 439 mac_bcast_init(); 440 mac_client_init(); 441 442 i_mac_impl_count = 0; 443 444 i_mactype_hash = mod_hash_create_extended("mactype_hash", 445 MACTYPE_HASHSZ, 446 mod_hash_null_keydtor, mod_hash_null_valdtor, 447 mod_hash_bystr, NULL, mod_hash_strkey_cmp, KM_SLEEP); 448 449 /* 450 * Allocate an id space to manage minor numbers. The range of the 451 * space will be from MAC_MAX_MINOR+1 to MAC_PRIVATE_MINOR-1. This 452 * leaves half of the 32-bit minors available for driver private use. 453 */ 454 minor_ids = id_space_create("mac_minor_ids", MAC_MAX_MINOR+1, 455 MAC_PRIVATE_MINOR-1); 456 ASSERT(minor_ids != NULL); 457 minor_count = 0; 458 459 /* Let's default to 20 seconds */ 460 mac_logging_interval = 20; 461 mac_flow_log_enable = B_FALSE; 462 mac_link_log_enable = B_FALSE; 463 mac_logging_timer = 0; 464 465 /* Register to be notified of noteworthy pools events */ 466 mac_pool_event_reg.pec_func = mac_pool_event_cb; 467 mac_pool_event_reg.pec_arg = NULL; 468 pool_event_cb_register(&mac_pool_event_reg); 469 } 470 471 int 472 mac_fini(void) 473 { 474 475 if (i_mac_impl_count > 0 || minor_count > 0) 476 return (EBUSY); 477 478 pool_event_cb_unregister(&mac_pool_event_reg); 479 480 id_space_destroy(minor_ids); 481 mac_flow_fini(); 482 483 mod_hash_destroy_hash(i_mac_impl_hash); 484 rw_destroy(&i_mac_impl_lock); 485 486 mac_client_fini(); 487 kmem_cache_destroy(mac_ring_cache); 488 489 mod_hash_destroy_hash(i_mactype_hash); 490 mac_soft_ring_finish(); 491 492 493 return (0); 494 } 495 496 /* 497 * Initialize a GLDv3 driver's device ops. A driver that manages its own ops 498 * (e.g. softmac) may pass in a NULL ops argument. 499 */ 500 void 501 mac_init_ops(struct dev_ops *ops, const char *name) 502 { 503 major_t major = ddi_name_to_major((char *)name); 504 505 /* 506 * By returning on error below, we are not letting the driver continue 507 * in an undefined context. The mac_register() function will faill if 508 * DN_GLDV3_DRIVER isn't set. 509 */ 510 if (major == DDI_MAJOR_T_NONE) 511 return; 512 LOCK_DEV_OPS(&devnamesp[major].dn_lock); 513 devnamesp[major].dn_flags |= (DN_GLDV3_DRIVER | DN_NETWORK_DRIVER); 514 UNLOCK_DEV_OPS(&devnamesp[major].dn_lock); 515 if (ops != NULL) 516 dld_init_ops(ops, name); 517 } 518 519 void 520 mac_fini_ops(struct dev_ops *ops) 521 { 522 dld_fini_ops(ops); 523 } 524 525 /*ARGSUSED*/ 526 static int 527 i_mac_constructor(void *buf, void *arg, int kmflag) 528 { 529 mac_impl_t *mip = buf; 530 531 bzero(buf, sizeof (mac_impl_t)); 532 533 mip->mi_linkstate = LINK_STATE_UNKNOWN; 534 535 rw_init(&mip->mi_rw_lock, NULL, RW_DRIVER, NULL); 536 mutex_init(&mip->mi_notify_lock, NULL, MUTEX_DRIVER, NULL); 537 mutex_init(&mip->mi_promisc_lock, NULL, MUTEX_DRIVER, NULL); 538 mutex_init(&mip->mi_ring_lock, NULL, MUTEX_DEFAULT, NULL); 539 540 mip->mi_notify_cb_info.mcbi_lockp = &mip->mi_notify_lock; 541 cv_init(&mip->mi_notify_cb_info.mcbi_cv, NULL, CV_DRIVER, NULL); 542 mip->mi_promisc_cb_info.mcbi_lockp = &mip->mi_promisc_lock; 543 cv_init(&mip->mi_promisc_cb_info.mcbi_cv, NULL, CV_DRIVER, NULL); 544 545 mutex_init(&mip->mi_bridge_lock, NULL, MUTEX_DEFAULT, NULL); 546 547 return (0); 548 } 549 550 /*ARGSUSED*/ 551 static void 552 i_mac_destructor(void *buf, void *arg) 553 { 554 mac_impl_t *mip = buf; 555 mac_cb_info_t *mcbi; 556 557 ASSERT(mip->mi_ref == 0); 558 ASSERT(mip->mi_active == 0); 559 ASSERT(mip->mi_linkstate == LINK_STATE_UNKNOWN); 560 ASSERT(mip->mi_devpromisc == 0); 561 ASSERT(mip->mi_ksp == NULL); 562 ASSERT(mip->mi_kstat_count == 0); 563 ASSERT(mip->mi_nclients == 0); 564 ASSERT(mip->mi_nactiveclients == 0); 565 ASSERT(mip->mi_single_active_client == NULL); 566 ASSERT(mip->mi_state_flags == 0); 567 ASSERT(mip->mi_factory_addr == NULL); 568 ASSERT(mip->mi_factory_addr_num == 0); 569 ASSERT(mip->mi_default_tx_ring == NULL); 570 571 mcbi = &mip->mi_notify_cb_info; 572 ASSERT(mcbi->mcbi_del_cnt == 0 && mcbi->mcbi_walker_cnt == 0); 573 ASSERT(mip->mi_notify_bits == 0); 574 ASSERT(mip->mi_notify_thread == NULL); 575 ASSERT(mcbi->mcbi_lockp == &mip->mi_notify_lock); 576 mcbi->mcbi_lockp = NULL; 577 578 mcbi = &mip->mi_promisc_cb_info; 579 ASSERT(mcbi->mcbi_del_cnt == 0 && mip->mi_promisc_list == NULL); 580 ASSERT(mip->mi_promisc_list == NULL); 581 ASSERT(mcbi->mcbi_lockp == &mip->mi_promisc_lock); 582 mcbi->mcbi_lockp = NULL; 583 584 ASSERT(mip->mi_bcast_ngrps == 0 && mip->mi_bcast_grp == NULL); 585 ASSERT(mip->mi_perim_owner == NULL && mip->mi_perim_ocnt == 0); 586 587 rw_destroy(&mip->mi_rw_lock); 588 589 mutex_destroy(&mip->mi_promisc_lock); 590 cv_destroy(&mip->mi_promisc_cb_info.mcbi_cv); 591 mutex_destroy(&mip->mi_notify_lock); 592 cv_destroy(&mip->mi_notify_cb_info.mcbi_cv); 593 mutex_destroy(&mip->mi_ring_lock); 594 595 ASSERT(mip->mi_bridge_link == NULL); 596 } 597 598 /* ARGSUSED */ 599 static int 600 i_mac_ring_ctor(void *buf, void *arg, int kmflag) 601 { 602 mac_ring_t *ring = (mac_ring_t *)buf; 603 604 bzero(ring, sizeof (mac_ring_t)); 605 cv_init(&ring->mr_cv, NULL, CV_DEFAULT, NULL); 606 mutex_init(&ring->mr_lock, NULL, MUTEX_DEFAULT, NULL); 607 ring->mr_state = MR_FREE; 608 return (0); 609 } 610 611 /* ARGSUSED */ 612 static void 613 i_mac_ring_dtor(void *buf, void *arg) 614 { 615 mac_ring_t *ring = (mac_ring_t *)buf; 616 617 cv_destroy(&ring->mr_cv); 618 mutex_destroy(&ring->mr_lock); 619 } 620 621 /* 622 * Common functions to do mac callback addition and deletion. Currently this is 623 * used by promisc callbacks and notify callbacks. List addition and deletion 624 * need to take care of list walkers. List walkers in general, can't hold list 625 * locks and make upcall callbacks due to potential lock order and recursive 626 * reentry issues. Instead list walkers increment the list walker count to mark 627 * the presence of a walker thread. Addition can be carefully done to ensure 628 * that the list walker always sees either the old list or the new list. 629 * However the deletion can't be done while the walker is active, instead the 630 * deleting thread simply marks the entry as logically deleted. The last walker 631 * physically deletes and frees up the logically deleted entries when the walk 632 * is complete. 633 */ 634 void 635 mac_callback_add(mac_cb_info_t *mcbi, mac_cb_t **mcb_head, 636 mac_cb_t *mcb_elem) 637 { 638 mac_cb_t *p; 639 mac_cb_t **pp; 640 641 /* Verify it is not already in the list */ 642 for (pp = mcb_head; (p = *pp) != NULL; pp = &p->mcb_nextp) { 643 if (p == mcb_elem) 644 break; 645 } 646 VERIFY(p == NULL); 647 648 /* 649 * Add it to the head of the callback list. The membar ensures that 650 * the following list pointer manipulations reach global visibility 651 * in exactly the program order below. 652 */ 653 ASSERT(MUTEX_HELD(mcbi->mcbi_lockp)); 654 655 mcb_elem->mcb_nextp = *mcb_head; 656 membar_producer(); 657 *mcb_head = mcb_elem; 658 } 659 660 /* 661 * Mark the entry as logically deleted. If there aren't any walkers unlink 662 * from the list. In either case return the corresponding status. 663 */ 664 boolean_t 665 mac_callback_remove(mac_cb_info_t *mcbi, mac_cb_t **mcb_head, 666 mac_cb_t *mcb_elem) 667 { 668 mac_cb_t *p; 669 mac_cb_t **pp; 670 671 ASSERT(MUTEX_HELD(mcbi->mcbi_lockp)); 672 /* 673 * Search the callback list for the entry to be removed 674 */ 675 for (pp = mcb_head; (p = *pp) != NULL; pp = &p->mcb_nextp) { 676 if (p == mcb_elem) 677 break; 678 } 679 VERIFY(p != NULL); 680 681 /* 682 * If there are walkers just mark it as deleted and the last walker 683 * will remove from the list and free it. 684 */ 685 if (mcbi->mcbi_walker_cnt != 0) { 686 p->mcb_flags |= MCB_CONDEMNED; 687 mcbi->mcbi_del_cnt++; 688 return (B_FALSE); 689 } 690 691 ASSERT(mcbi->mcbi_del_cnt == 0); 692 *pp = p->mcb_nextp; 693 p->mcb_nextp = NULL; 694 return (B_TRUE); 695 } 696 697 /* 698 * Wait for all pending callback removals to be completed 699 */ 700 void 701 mac_callback_remove_wait(mac_cb_info_t *mcbi) 702 { 703 ASSERT(MUTEX_HELD(mcbi->mcbi_lockp)); 704 while (mcbi->mcbi_del_cnt != 0) { 705 DTRACE_PROBE1(need_wait, mac_cb_info_t *, mcbi); 706 cv_wait(&mcbi->mcbi_cv, mcbi->mcbi_lockp); 707 } 708 } 709 710 /* 711 * The last mac callback walker does the cleanup. Walk the list and unlik 712 * all the logically deleted entries and construct a temporary list of 713 * removed entries. Return the list of removed entries to the caller. 714 */ 715 mac_cb_t * 716 mac_callback_walker_cleanup(mac_cb_info_t *mcbi, mac_cb_t **mcb_head) 717 { 718 mac_cb_t *p; 719 mac_cb_t **pp; 720 mac_cb_t *rmlist = NULL; /* List of removed elements */ 721 int cnt = 0; 722 723 ASSERT(MUTEX_HELD(mcbi->mcbi_lockp)); 724 ASSERT(mcbi->mcbi_del_cnt != 0 && mcbi->mcbi_walker_cnt == 0); 725 726 pp = mcb_head; 727 while (*pp != NULL) { 728 if ((*pp)->mcb_flags & MCB_CONDEMNED) { 729 p = *pp; 730 *pp = p->mcb_nextp; 731 p->mcb_nextp = rmlist; 732 rmlist = p; 733 cnt++; 734 continue; 735 } 736 pp = &(*pp)->mcb_nextp; 737 } 738 739 ASSERT(mcbi->mcbi_del_cnt == cnt); 740 mcbi->mcbi_del_cnt = 0; 741 return (rmlist); 742 } 743 744 boolean_t 745 mac_callback_lookup(mac_cb_t **mcb_headp, mac_cb_t *mcb_elem) 746 { 747 mac_cb_t *mcb; 748 749 /* Verify it is not already in the list */ 750 for (mcb = *mcb_headp; mcb != NULL; mcb = mcb->mcb_nextp) { 751 if (mcb == mcb_elem) 752 return (B_TRUE); 753 } 754 755 return (B_FALSE); 756 } 757 758 boolean_t 759 mac_callback_find(mac_cb_info_t *mcbi, mac_cb_t **mcb_headp, mac_cb_t *mcb_elem) 760 { 761 boolean_t found; 762 763 mutex_enter(mcbi->mcbi_lockp); 764 found = mac_callback_lookup(mcb_headp, mcb_elem); 765 mutex_exit(mcbi->mcbi_lockp); 766 767 return (found); 768 } 769 770 /* Free the list of removed callbacks */ 771 void 772 mac_callback_free(mac_cb_t *rmlist) 773 { 774 mac_cb_t *mcb; 775 mac_cb_t *mcb_next; 776 777 for (mcb = rmlist; mcb != NULL; mcb = mcb_next) { 778 mcb_next = mcb->mcb_nextp; 779 kmem_free(mcb->mcb_objp, mcb->mcb_objsize); 780 } 781 } 782 783 /* 784 * The promisc callbacks are in 2 lists, one off the 'mip' and another off the 785 * 'mcip' threaded by mpi_mi_link and mpi_mci_link respectively. However there 786 * is only a single shared total walker count, and an entry can't be physically 787 * unlinked if a walker is active on either list. The last walker does this 788 * cleanup of logically deleted entries. 789 */ 790 void 791 i_mac_promisc_walker_cleanup(mac_impl_t *mip) 792 { 793 mac_cb_t *rmlist; 794 mac_cb_t *mcb; 795 mac_cb_t *mcb_next; 796 mac_promisc_impl_t *mpip; 797 798 /* 799 * Construct a temporary list of deleted callbacks by walking the 800 * the mi_promisc_list. Then for each entry in the temporary list, 801 * remove it from the mci_promisc_list and free the entry. 802 */ 803 rmlist = mac_callback_walker_cleanup(&mip->mi_promisc_cb_info, 804 &mip->mi_promisc_list); 805 806 for (mcb = rmlist; mcb != NULL; mcb = mcb_next) { 807 mcb_next = mcb->mcb_nextp; 808 mpip = (mac_promisc_impl_t *)mcb->mcb_objp; 809 VERIFY(mac_callback_remove(&mip->mi_promisc_cb_info, 810 &mpip->mpi_mcip->mci_promisc_list, &mpip->mpi_mci_link)); 811 mcb->mcb_flags = 0; 812 mcb->mcb_nextp = NULL; 813 kmem_cache_free(mac_promisc_impl_cache, mpip); 814 } 815 } 816 817 void 818 i_mac_notify(mac_impl_t *mip, mac_notify_type_t type) 819 { 820 mac_cb_info_t *mcbi; 821 822 /* 823 * Signal the notify thread even after mi_ref has become zero and 824 * mi_disabled is set. The synchronization with the notify thread 825 * happens in mac_unregister and that implies the driver must make 826 * sure it is single-threaded (with respect to mac calls) and that 827 * all pending mac calls have returned before it calls mac_unregister 828 */ 829 rw_enter(&i_mac_impl_lock, RW_READER); 830 if (mip->mi_state_flags & MIS_DISABLED) 831 goto exit; 832 833 /* 834 * Guard against incorrect notifications. (Running a newer 835 * mac client against an older implementation?) 836 */ 837 if (type >= MAC_NNOTE) 838 goto exit; 839 840 mcbi = &mip->mi_notify_cb_info; 841 mutex_enter(mcbi->mcbi_lockp); 842 mip->mi_notify_bits |= (1 << type); 843 cv_broadcast(&mcbi->mcbi_cv); 844 mutex_exit(mcbi->mcbi_lockp); 845 846 exit: 847 rw_exit(&i_mac_impl_lock); 848 } 849 850 /* 851 * Mac serialization primitives. Please see the block comment at the 852 * top of the file. 853 */ 854 void 855 i_mac_perim_enter(mac_impl_t *mip) 856 { 857 mac_client_impl_t *mcip; 858 859 if (mip->mi_state_flags & MIS_IS_VNIC) { 860 /* 861 * This is a VNIC. Return the lower mac since that is what 862 * we want to serialize on. 863 */ 864 mcip = mac_vnic_lower(mip); 865 mip = mcip->mci_mip; 866 } 867 868 mutex_enter(&mip->mi_perim_lock); 869 if (mip->mi_perim_owner == curthread) { 870 mip->mi_perim_ocnt++; 871 mutex_exit(&mip->mi_perim_lock); 872 return; 873 } 874 875 while (mip->mi_perim_owner != NULL) 876 cv_wait(&mip->mi_perim_cv, &mip->mi_perim_lock); 877 878 mip->mi_perim_owner = curthread; 879 ASSERT(mip->mi_perim_ocnt == 0); 880 mip->mi_perim_ocnt++; 881 #ifdef DEBUG 882 mip->mi_perim_stack_depth = getpcstack(mip->mi_perim_stack, 883 MAC_PERIM_STACK_DEPTH); 884 #endif 885 mutex_exit(&mip->mi_perim_lock); 886 } 887 888 int 889 i_mac_perim_enter_nowait(mac_impl_t *mip) 890 { 891 /* 892 * The vnic is a special case, since the serialization is done based 893 * on the lower mac. If the lower mac is busy, it does not imply the 894 * vnic can't be unregistered. But in the case of other drivers, 895 * a busy perimeter or open mac handles implies that the mac is busy 896 * and can't be unregistered. 897 */ 898 if (mip->mi_state_flags & MIS_IS_VNIC) { 899 i_mac_perim_enter(mip); 900 return (0); 901 } 902 903 mutex_enter(&mip->mi_perim_lock); 904 if (mip->mi_perim_owner != NULL) { 905 mutex_exit(&mip->mi_perim_lock); 906 return (EBUSY); 907 } 908 ASSERT(mip->mi_perim_ocnt == 0); 909 mip->mi_perim_owner = curthread; 910 mip->mi_perim_ocnt++; 911 mutex_exit(&mip->mi_perim_lock); 912 913 return (0); 914 } 915 916 void 917 i_mac_perim_exit(mac_impl_t *mip) 918 { 919 mac_client_impl_t *mcip; 920 921 if (mip->mi_state_flags & MIS_IS_VNIC) { 922 /* 923 * This is a VNIC. Return the lower mac since that is what 924 * we want to serialize on. 925 */ 926 mcip = mac_vnic_lower(mip); 927 mip = mcip->mci_mip; 928 } 929 930 ASSERT(mip->mi_perim_owner == curthread && mip->mi_perim_ocnt != 0); 931 932 mutex_enter(&mip->mi_perim_lock); 933 if (--mip->mi_perim_ocnt == 0) { 934 mip->mi_perim_owner = NULL; 935 cv_signal(&mip->mi_perim_cv); 936 } 937 mutex_exit(&mip->mi_perim_lock); 938 } 939 940 /* 941 * Returns whether the current thread holds the mac perimeter. Used in making 942 * assertions. 943 */ 944 boolean_t 945 mac_perim_held(mac_handle_t mh) 946 { 947 mac_impl_t *mip = (mac_impl_t *)mh; 948 mac_client_impl_t *mcip; 949 950 if (mip->mi_state_flags & MIS_IS_VNIC) { 951 /* 952 * This is a VNIC. Return the lower mac since that is what 953 * we want to serialize on. 954 */ 955 mcip = mac_vnic_lower(mip); 956 mip = mcip->mci_mip; 957 } 958 return (mip->mi_perim_owner == curthread); 959 } 960 961 /* 962 * mac client interfaces to enter the mac perimeter of a mac end point, given 963 * its mac handle, or macname or linkid. 964 */ 965 void 966 mac_perim_enter_by_mh(mac_handle_t mh, mac_perim_handle_t *mphp) 967 { 968 mac_impl_t *mip = (mac_impl_t *)mh; 969 970 i_mac_perim_enter(mip); 971 /* 972 * The mac_perim_handle_t returned encodes the 'mip' and whether a 973 * mac_open has been done internally while entering the perimeter. 974 * This information is used in mac_perim_exit 975 */ 976 MAC_ENCODE_MPH(*mphp, mip, 0); 977 } 978 979 int 980 mac_perim_enter_by_macname(const char *name, mac_perim_handle_t *mphp) 981 { 982 int err; 983 mac_handle_t mh; 984 985 if ((err = mac_open(name, &mh)) != 0) 986 return (err); 987 988 mac_perim_enter_by_mh(mh, mphp); 989 MAC_ENCODE_MPH(*mphp, mh, 1); 990 return (0); 991 } 992 993 int 994 mac_perim_enter_by_linkid(datalink_id_t linkid, mac_perim_handle_t *mphp) 995 { 996 int err; 997 mac_handle_t mh; 998 999 if ((err = mac_open_by_linkid(linkid, &mh)) != 0) 1000 return (err); 1001 1002 mac_perim_enter_by_mh(mh, mphp); 1003 MAC_ENCODE_MPH(*mphp, mh, 1); 1004 return (0); 1005 } 1006 1007 void 1008 mac_perim_exit(mac_perim_handle_t mph) 1009 { 1010 mac_impl_t *mip; 1011 boolean_t need_close; 1012 1013 MAC_DECODE_MPH(mph, mip, need_close); 1014 i_mac_perim_exit(mip); 1015 if (need_close) 1016 mac_close((mac_handle_t)mip); 1017 } 1018 1019 int 1020 mac_hold(const char *macname, mac_impl_t **pmip) 1021 { 1022 mac_impl_t *mip; 1023 int err; 1024 1025 /* 1026 * Check the device name length to make sure it won't overflow our 1027 * buffer. 1028 */ 1029 if (strlen(macname) >= MAXNAMELEN) 1030 return (EINVAL); 1031 1032 /* 1033 * Look up its entry in the global hash table. 1034 */ 1035 rw_enter(&i_mac_impl_lock, RW_WRITER); 1036 err = mod_hash_find(i_mac_impl_hash, (mod_hash_key_t)macname, 1037 (mod_hash_val_t *)&mip); 1038 1039 if (err != 0) { 1040 rw_exit(&i_mac_impl_lock); 1041 return (ENOENT); 1042 } 1043 1044 if (mip->mi_state_flags & MIS_DISABLED) { 1045 rw_exit(&i_mac_impl_lock); 1046 return (ENOENT); 1047 } 1048 1049 if (mip->mi_state_flags & MIS_EXCLUSIVE_HELD) { 1050 rw_exit(&i_mac_impl_lock); 1051 return (EBUSY); 1052 } 1053 1054 mip->mi_ref++; 1055 rw_exit(&i_mac_impl_lock); 1056 1057 *pmip = mip; 1058 return (0); 1059 } 1060 1061 void 1062 mac_rele(mac_impl_t *mip) 1063 { 1064 rw_enter(&i_mac_impl_lock, RW_WRITER); 1065 ASSERT(mip->mi_ref != 0); 1066 if (--mip->mi_ref == 0) { 1067 ASSERT(mip->mi_nactiveclients == 0 && 1068 !(mip->mi_state_flags & MIS_EXCLUSIVE)); 1069 } 1070 rw_exit(&i_mac_impl_lock); 1071 } 1072 1073 /* 1074 * Private GLDv3 function to start a MAC instance. 1075 */ 1076 int 1077 mac_start(mac_handle_t mh) 1078 { 1079 mac_impl_t *mip = (mac_impl_t *)mh; 1080 int err = 0; 1081 mac_group_t *defgrp; 1082 1083 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 1084 ASSERT(mip->mi_start != NULL); 1085 1086 /* 1087 * Check whether the device is already started. 1088 */ 1089 if (mip->mi_active++ == 0) { 1090 mac_ring_t *ring = NULL; 1091 1092 /* 1093 * Start the device. 1094 */ 1095 err = mip->mi_start(mip->mi_driver); 1096 if (err != 0) { 1097 mip->mi_active--; 1098 return (err); 1099 } 1100 1101 /* 1102 * Start the default tx ring. 1103 */ 1104 if (mip->mi_default_tx_ring != NULL) { 1105 1106 ring = (mac_ring_t *)mip->mi_default_tx_ring; 1107 if (ring->mr_state != MR_INUSE) { 1108 err = mac_start_ring(ring); 1109 if (err != 0) { 1110 mip->mi_active--; 1111 return (err); 1112 } 1113 } 1114 } 1115 1116 if ((defgrp = MAC_DEFAULT_RX_GROUP(mip)) != NULL) { 1117 /* 1118 * Start the default ring, since it will be needed 1119 * to receive broadcast and multicast traffic for 1120 * both primary and non-primary MAC clients. 1121 */ 1122 ASSERT(defgrp->mrg_state == MAC_GROUP_STATE_REGISTERED); 1123 err = mac_start_group_and_rings(defgrp); 1124 if (err != 0) { 1125 mip->mi_active--; 1126 if ((ring != NULL) && 1127 (ring->mr_state == MR_INUSE)) 1128 mac_stop_ring(ring); 1129 return (err); 1130 } 1131 mac_set_group_state(defgrp, MAC_GROUP_STATE_SHARED); 1132 } 1133 } 1134 1135 return (err); 1136 } 1137 1138 /* 1139 * Private GLDv3 function to stop a MAC instance. 1140 */ 1141 void 1142 mac_stop(mac_handle_t mh) 1143 { 1144 mac_impl_t *mip = (mac_impl_t *)mh; 1145 mac_group_t *grp; 1146 1147 ASSERT(mip->mi_stop != NULL); 1148 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 1149 1150 /* 1151 * Check whether the device is still needed. 1152 */ 1153 ASSERT(mip->mi_active != 0); 1154 if (--mip->mi_active == 0) { 1155 if ((grp = MAC_DEFAULT_RX_GROUP(mip)) != NULL) { 1156 /* 1157 * There should be no more active clients since the 1158 * MAC is being stopped. Stop the default RX group 1159 * and transition it back to registered state. 1160 * 1161 * When clients are torn down, the groups 1162 * are release via mac_release_rx_group which 1163 * knows the the default group is always in 1164 * started mode since broadcast uses it. So 1165 * we can assert that their are no clients 1166 * (since mac_bcast_add doesn't register itself 1167 * as a client) and group is in SHARED state. 1168 */ 1169 ASSERT(grp->mrg_state == MAC_GROUP_STATE_SHARED); 1170 ASSERT(MAC_GROUP_NO_CLIENT(grp) && 1171 mip->mi_nactiveclients == 0); 1172 mac_stop_group_and_rings(grp); 1173 mac_set_group_state(grp, MAC_GROUP_STATE_REGISTERED); 1174 } 1175 1176 if (mip->mi_default_tx_ring != NULL) { 1177 mac_ring_t *ring; 1178 1179 ring = (mac_ring_t *)mip->mi_default_tx_ring; 1180 if (ring->mr_state == MR_INUSE) { 1181 mac_stop_ring(ring); 1182 ring->mr_flag = 0; 1183 } 1184 } 1185 1186 /* 1187 * Stop the device. 1188 */ 1189 mip->mi_stop(mip->mi_driver); 1190 } 1191 } 1192 1193 int 1194 i_mac_promisc_set(mac_impl_t *mip, boolean_t on) 1195 { 1196 int err = 0; 1197 1198 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 1199 ASSERT(mip->mi_setpromisc != NULL); 1200 1201 if (on) { 1202 /* 1203 * Enable promiscuous mode on the device if not yet enabled. 1204 */ 1205 if (mip->mi_devpromisc++ == 0) { 1206 err = mip->mi_setpromisc(mip->mi_driver, B_TRUE); 1207 if (err != 0) { 1208 mip->mi_devpromisc--; 1209 return (err); 1210 } 1211 i_mac_notify(mip, MAC_NOTE_DEVPROMISC); 1212 } 1213 } else { 1214 if (mip->mi_devpromisc == 0) 1215 return (EPROTO); 1216 1217 /* 1218 * Disable promiscuous mode on the device if this is the last 1219 * enabling. 1220 */ 1221 if (--mip->mi_devpromisc == 0) { 1222 err = mip->mi_setpromisc(mip->mi_driver, B_FALSE); 1223 if (err != 0) { 1224 mip->mi_devpromisc++; 1225 return (err); 1226 } 1227 i_mac_notify(mip, MAC_NOTE_DEVPROMISC); 1228 } 1229 } 1230 1231 return (0); 1232 } 1233 1234 /* 1235 * The promiscuity state can change any time. If the caller needs to take 1236 * actions that are atomic with the promiscuity state, then the caller needs 1237 * to bracket the entire sequence with mac_perim_enter/exit 1238 */ 1239 boolean_t 1240 mac_promisc_get(mac_handle_t mh) 1241 { 1242 mac_impl_t *mip = (mac_impl_t *)mh; 1243 1244 /* 1245 * Return the current promiscuity. 1246 */ 1247 return (mip->mi_devpromisc != 0); 1248 } 1249 1250 /* 1251 * Invoked at MAC instance attach time to initialize the list 1252 * of factory MAC addresses supported by a MAC instance. This function 1253 * builds a local cache in the mac_impl_t for the MAC addresses 1254 * supported by the underlying hardware. The MAC clients themselves 1255 * use the mac_addr_factory*() functions to query and reserve 1256 * factory MAC addresses. 1257 */ 1258 void 1259 mac_addr_factory_init(mac_impl_t *mip) 1260 { 1261 mac_capab_multifactaddr_t capab; 1262 uint8_t *addr; 1263 int i; 1264 1265 /* 1266 * First round to see how many factory MAC addresses are available. 1267 */ 1268 bzero(&capab, sizeof (capab)); 1269 if (!i_mac_capab_get((mac_handle_t)mip, MAC_CAPAB_MULTIFACTADDR, 1270 &capab) || (capab.mcm_naddr == 0)) { 1271 /* 1272 * The MAC instance doesn't support multiple factory 1273 * MAC addresses, we're done here. 1274 */ 1275 return; 1276 } 1277 1278 /* 1279 * Allocate the space and get all the factory addresses. 1280 */ 1281 addr = kmem_alloc(capab.mcm_naddr * MAXMACADDRLEN, KM_SLEEP); 1282 capab.mcm_getaddr(mip->mi_driver, capab.mcm_naddr, addr); 1283 1284 mip->mi_factory_addr_num = capab.mcm_naddr; 1285 mip->mi_factory_addr = kmem_zalloc(mip->mi_factory_addr_num * 1286 sizeof (mac_factory_addr_t), KM_SLEEP); 1287 1288 for (i = 0; i < capab.mcm_naddr; i++) { 1289 bcopy(addr + i * MAXMACADDRLEN, 1290 mip->mi_factory_addr[i].mfa_addr, 1291 mip->mi_type->mt_addr_length); 1292 mip->mi_factory_addr[i].mfa_in_use = B_FALSE; 1293 } 1294 1295 kmem_free(addr, capab.mcm_naddr * MAXMACADDRLEN); 1296 } 1297 1298 void 1299 mac_addr_factory_fini(mac_impl_t *mip) 1300 { 1301 if (mip->mi_factory_addr == NULL) { 1302 ASSERT(mip->mi_factory_addr_num == 0); 1303 return; 1304 } 1305 1306 kmem_free(mip->mi_factory_addr, mip->mi_factory_addr_num * 1307 sizeof (mac_factory_addr_t)); 1308 1309 mip->mi_factory_addr = NULL; 1310 mip->mi_factory_addr_num = 0; 1311 } 1312 1313 /* 1314 * Reserve a factory MAC address. If *slot is set to -1, the function 1315 * attempts to reserve any of the available factory MAC addresses and 1316 * returns the reserved slot id. If no slots are available, the function 1317 * returns ENOSPC. If *slot is not set to -1, the function reserves 1318 * the specified slot if it is available, or returns EBUSY is the slot 1319 * is already used. Returns ENOTSUP if the underlying MAC does not 1320 * support multiple factory addresses. If the slot number is not -1 but 1321 * is invalid, returns EINVAL. 1322 */ 1323 int 1324 mac_addr_factory_reserve(mac_client_handle_t mch, int *slot) 1325 { 1326 mac_client_impl_t *mcip = (mac_client_impl_t *)mch; 1327 mac_impl_t *mip = mcip->mci_mip; 1328 int i, ret = 0; 1329 1330 i_mac_perim_enter(mip); 1331 /* 1332 * Protect against concurrent readers that may need a self-consistent 1333 * view of the factory addresses 1334 */ 1335 rw_enter(&mip->mi_rw_lock, RW_WRITER); 1336 1337 if (mip->mi_factory_addr_num == 0) { 1338 ret = ENOTSUP; 1339 goto bail; 1340 } 1341 1342 if (*slot != -1) { 1343 /* check the specified slot */ 1344 if (*slot < 1 || *slot > mip->mi_factory_addr_num) { 1345 ret = EINVAL; 1346 goto bail; 1347 } 1348 if (mip->mi_factory_addr[*slot-1].mfa_in_use) { 1349 ret = EBUSY; 1350 goto bail; 1351 } 1352 } else { 1353 /* pick the next available slot */ 1354 for (i = 0; i < mip->mi_factory_addr_num; i++) { 1355 if (!mip->mi_factory_addr[i].mfa_in_use) 1356 break; 1357 } 1358 1359 if (i == mip->mi_factory_addr_num) { 1360 ret = ENOSPC; 1361 goto bail; 1362 } 1363 *slot = i+1; 1364 } 1365 1366 mip->mi_factory_addr[*slot-1].mfa_in_use = B_TRUE; 1367 mip->mi_factory_addr[*slot-1].mfa_client = mcip; 1368 1369 bail: 1370 rw_exit(&mip->mi_rw_lock); 1371 i_mac_perim_exit(mip); 1372 return (ret); 1373 } 1374 1375 /* 1376 * Release the specified factory MAC address slot. 1377 */ 1378 void 1379 mac_addr_factory_release(mac_client_handle_t mch, uint_t slot) 1380 { 1381 mac_client_impl_t *mcip = (mac_client_impl_t *)mch; 1382 mac_impl_t *mip = mcip->mci_mip; 1383 1384 i_mac_perim_enter(mip); 1385 /* 1386 * Protect against concurrent readers that may need a self-consistent 1387 * view of the factory addresses 1388 */ 1389 rw_enter(&mip->mi_rw_lock, RW_WRITER); 1390 1391 ASSERT(slot > 0 && slot <= mip->mi_factory_addr_num); 1392 ASSERT(mip->mi_factory_addr[slot-1].mfa_in_use); 1393 1394 mip->mi_factory_addr[slot-1].mfa_in_use = B_FALSE; 1395 1396 rw_exit(&mip->mi_rw_lock); 1397 i_mac_perim_exit(mip); 1398 } 1399 1400 /* 1401 * Stores in mac_addr the value of the specified MAC address. Returns 1402 * 0 on success, or EINVAL if the slot number is not valid for the MAC. 1403 * The caller must provide a string of at least MAXNAMELEN bytes. 1404 */ 1405 void 1406 mac_addr_factory_value(mac_handle_t mh, int slot, uchar_t *mac_addr, 1407 uint_t *addr_len, char *client_name, boolean_t *in_use_arg) 1408 { 1409 mac_impl_t *mip = (mac_impl_t *)mh; 1410 boolean_t in_use; 1411 1412 ASSERT(slot > 0 && slot <= mip->mi_factory_addr_num); 1413 1414 /* 1415 * Readers need to hold mi_rw_lock. Writers need to hold mac perimeter 1416 * and mi_rw_lock 1417 */ 1418 rw_enter(&mip->mi_rw_lock, RW_READER); 1419 bcopy(mip->mi_factory_addr[slot-1].mfa_addr, mac_addr, MAXMACADDRLEN); 1420 *addr_len = mip->mi_type->mt_addr_length; 1421 in_use = mip->mi_factory_addr[slot-1].mfa_in_use; 1422 if (in_use && client_name != NULL) { 1423 bcopy(mip->mi_factory_addr[slot-1].mfa_client->mci_name, 1424 client_name, MAXNAMELEN); 1425 } 1426 if (in_use_arg != NULL) 1427 *in_use_arg = in_use; 1428 rw_exit(&mip->mi_rw_lock); 1429 } 1430 1431 /* 1432 * Returns the number of factory MAC addresses (in addition to the 1433 * primary MAC address), 0 if the underlying MAC doesn't support 1434 * that feature. 1435 */ 1436 uint_t 1437 mac_addr_factory_num(mac_handle_t mh) 1438 { 1439 mac_impl_t *mip = (mac_impl_t *)mh; 1440 1441 return (mip->mi_factory_addr_num); 1442 } 1443 1444 1445 void 1446 mac_rx_group_unmark(mac_group_t *grp, uint_t flag) 1447 { 1448 mac_ring_t *ring; 1449 1450 for (ring = grp->mrg_rings; ring != NULL; ring = ring->mr_next) 1451 ring->mr_flag &= ~flag; 1452 } 1453 1454 /* 1455 * The following mac_hwrings_xxx() functions are private mac client functions 1456 * used by the aggr driver to access and control the underlying HW Rx group 1457 * and rings. In this case, the aggr driver has exclusive control of the 1458 * underlying HW Rx group/rings, it calls the following functions to 1459 * start/stop the HW Rx rings, disable/enable polling, add/remove mac' 1460 * addresses, or set up the Rx callback. 1461 */ 1462 /* ARGSUSED */ 1463 static void 1464 mac_hwrings_rx_process(void *arg, mac_resource_handle_t srs, 1465 mblk_t *mp_chain, boolean_t loopback) 1466 { 1467 mac_soft_ring_set_t *mac_srs = (mac_soft_ring_set_t *)srs; 1468 mac_srs_rx_t *srs_rx = &mac_srs->srs_rx; 1469 mac_direct_rx_t proc; 1470 void *arg1; 1471 mac_resource_handle_t arg2; 1472 1473 proc = srs_rx->sr_func; 1474 arg1 = srs_rx->sr_arg1; 1475 arg2 = mac_srs->srs_mrh; 1476 1477 proc(arg1, arg2, mp_chain, NULL); 1478 } 1479 1480 /* 1481 * This function is called to get the list of HW rings that are reserved by 1482 * an exclusive mac client. 1483 * 1484 * Return value: the number of HW rings. 1485 */ 1486 int 1487 mac_hwrings_get(mac_client_handle_t mch, mac_group_handle_t *hwgh, 1488 mac_ring_handle_t *hwrh, mac_ring_type_t rtype) 1489 { 1490 mac_client_impl_t *mcip = (mac_client_impl_t *)mch; 1491 flow_entry_t *flent = mcip->mci_flent; 1492 mac_group_t *grp; 1493 mac_ring_t *ring; 1494 int cnt = 0; 1495 1496 if (rtype == MAC_RING_TYPE_RX) { 1497 grp = flent->fe_rx_ring_group; 1498 } else if (rtype == MAC_RING_TYPE_TX) { 1499 grp = flent->fe_tx_ring_group; 1500 } else { 1501 ASSERT(B_FALSE); 1502 return (-1); 1503 } 1504 /* 1505 * The mac client did not reserve any RX group, return directly. 1506 * This is probably because the underlying MAC does not support 1507 * any groups. 1508 */ 1509 if (hwgh != NULL) 1510 *hwgh = NULL; 1511 if (grp == NULL) 1512 return (0); 1513 /* 1514 * This group must be reserved by this mac client. 1515 */ 1516 ASSERT((grp->mrg_state == MAC_GROUP_STATE_RESERVED) && 1517 (mcip == MAC_GROUP_ONLY_CLIENT(grp))); 1518 1519 for (ring = grp->mrg_rings; ring != NULL; ring = ring->mr_next, cnt++) { 1520 ASSERT(cnt < MAX_RINGS_PER_GROUP); 1521 hwrh[cnt] = (mac_ring_handle_t)ring; 1522 } 1523 if (hwgh != NULL) 1524 *hwgh = (mac_group_handle_t)grp; 1525 1526 return (cnt); 1527 } 1528 1529 /* 1530 * This function is called to get info about Tx/Rx rings. 1531 * 1532 * Return value: returns uint_t which will have various bits set 1533 * that indicates different properties of the ring. 1534 */ 1535 uint_t 1536 mac_hwring_getinfo(mac_ring_handle_t rh) 1537 { 1538 mac_ring_t *ring = (mac_ring_t *)rh; 1539 mac_ring_info_t *info = &ring->mr_info; 1540 1541 return (info->mri_flags); 1542 } 1543 1544 /* 1545 * Export ddi interrupt handles from the HW ring to the pseudo ring and 1546 * setup the RX callback of the mac client which exclusively controls 1547 * HW ring. 1548 */ 1549 void 1550 mac_hwring_setup(mac_ring_handle_t hwrh, mac_resource_handle_t prh, 1551 mac_ring_handle_t pseudo_rh) 1552 { 1553 mac_ring_t *hw_ring = (mac_ring_t *)hwrh; 1554 mac_ring_t *pseudo_ring; 1555 mac_soft_ring_set_t *mac_srs = hw_ring->mr_srs; 1556 1557 if (pseudo_rh != NULL) { 1558 pseudo_ring = (mac_ring_t *)pseudo_rh; 1559 /* Export the ddi handles to pseudo ring */ 1560 pseudo_ring->mr_info.mri_intr.mi_ddi_handle = 1561 hw_ring->mr_info.mri_intr.mi_ddi_handle; 1562 pseudo_ring->mr_info.mri_intr.mi_ddi_shared = 1563 hw_ring->mr_info.mri_intr.mi_ddi_shared; 1564 /* 1565 * Save a pointer to pseudo ring in the hw ring. If 1566 * interrupt handle changes, the hw ring will be 1567 * notified of the change (see mac_ring_intr_set()) 1568 * and the appropriate change has to be made to 1569 * the pseudo ring that has exported the ddi handle. 1570 */ 1571 hw_ring->mr_prh = pseudo_rh; 1572 } 1573 1574 if (hw_ring->mr_type == MAC_RING_TYPE_RX) { 1575 ASSERT(!(mac_srs->srs_type & SRST_TX)); 1576 mac_srs->srs_mrh = prh; 1577 mac_srs->srs_rx.sr_lower_proc = mac_hwrings_rx_process; 1578 } 1579 } 1580 1581 void 1582 mac_hwring_teardown(mac_ring_handle_t hwrh) 1583 { 1584 mac_ring_t *hw_ring = (mac_ring_t *)hwrh; 1585 mac_soft_ring_set_t *mac_srs; 1586 1587 if (hw_ring == NULL) 1588 return; 1589 hw_ring->mr_prh = NULL; 1590 if (hw_ring->mr_type == MAC_RING_TYPE_RX) { 1591 mac_srs = hw_ring->mr_srs; 1592 ASSERT(!(mac_srs->srs_type & SRST_TX)); 1593 mac_srs->srs_rx.sr_lower_proc = mac_rx_srs_process; 1594 mac_srs->srs_mrh = NULL; 1595 } 1596 } 1597 1598 int 1599 mac_hwring_disable_intr(mac_ring_handle_t rh) 1600 { 1601 mac_ring_t *rr_ring = (mac_ring_t *)rh; 1602 mac_intr_t *intr = &rr_ring->mr_info.mri_intr; 1603 1604 return (intr->mi_disable(intr->mi_handle)); 1605 } 1606 1607 int 1608 mac_hwring_enable_intr(mac_ring_handle_t rh) 1609 { 1610 mac_ring_t *rr_ring = (mac_ring_t *)rh; 1611 mac_intr_t *intr = &rr_ring->mr_info.mri_intr; 1612 1613 return (intr->mi_enable(intr->mi_handle)); 1614 } 1615 1616 int 1617 mac_hwring_start(mac_ring_handle_t rh) 1618 { 1619 mac_ring_t *rr_ring = (mac_ring_t *)rh; 1620 1621 MAC_RING_UNMARK(rr_ring, MR_QUIESCE); 1622 return (0); 1623 } 1624 1625 void 1626 mac_hwring_stop(mac_ring_handle_t rh) 1627 { 1628 mac_ring_t *rr_ring = (mac_ring_t *)rh; 1629 1630 mac_rx_ring_quiesce(rr_ring, MR_QUIESCE); 1631 } 1632 1633 mblk_t * 1634 mac_hwring_poll(mac_ring_handle_t rh, int bytes_to_pickup) 1635 { 1636 mac_ring_t *rr_ring = (mac_ring_t *)rh; 1637 mac_ring_info_t *info = &rr_ring->mr_info; 1638 1639 return (info->mri_poll(info->mri_driver, bytes_to_pickup)); 1640 } 1641 1642 /* 1643 * Send packets through a selected tx ring. 1644 */ 1645 mblk_t * 1646 mac_hwring_tx(mac_ring_handle_t rh, mblk_t *mp) 1647 { 1648 mac_ring_t *ring = (mac_ring_t *)rh; 1649 mac_ring_info_t *info = &ring->mr_info; 1650 1651 ASSERT(ring->mr_type == MAC_RING_TYPE_TX && 1652 ring->mr_state >= MR_INUSE); 1653 return (info->mri_tx(info->mri_driver, mp)); 1654 } 1655 1656 /* 1657 * Query stats for a particular rx/tx ring 1658 */ 1659 int 1660 mac_hwring_getstat(mac_ring_handle_t rh, uint_t stat, uint64_t *val) 1661 { 1662 mac_ring_t *ring = (mac_ring_t *)rh; 1663 mac_ring_info_t *info = &ring->mr_info; 1664 1665 return (info->mri_stat(info->mri_driver, stat, val)); 1666 } 1667 1668 /* 1669 * Private function that is only used by aggr to send packets through 1670 * a port/Tx ring. Since aggr exposes a pseudo Tx ring even for ports 1671 * that does not expose Tx rings, aggr_ring_tx() entry point needs 1672 * access to mac_impl_t to send packets through m_tx() entry point. 1673 * It accomplishes this by calling mac_hwring_send_priv() function. 1674 */ 1675 mblk_t * 1676 mac_hwring_send_priv(mac_client_handle_t mch, mac_ring_handle_t rh, mblk_t *mp) 1677 { 1678 mac_client_impl_t *mcip = (mac_client_impl_t *)mch; 1679 mac_impl_t *mip = mcip->mci_mip; 1680 1681 MAC_TX(mip, rh, mp, mcip); 1682 return (mp); 1683 } 1684 1685 /* 1686 * Private function that is only used by aggr to update the default transmission 1687 * ring. Because aggr exposes a pseudo Tx ring even for ports that may 1688 * temporarily be down, it may need to update the default ring that is used by 1689 * MAC such that it refers to a link that can actively be used to send traffic. 1690 * Note that this is different from the case where the port has been removed 1691 * from the group. In those cases, all of the rings will be torn down because 1692 * the ring will no longer exist. It's important to give aggr a case where the 1693 * rings can still exist such that it may be able to continue to send LACP PDUs 1694 * to potentially restore the link. 1695 * 1696 * Finally, we explicitly don't do anything if the ring hasn't been enabled yet. 1697 * This is to help out aggr which doesn't really know the internal state that 1698 * MAC does about the rings and can't know that it's not quite ready for use 1699 * yet. 1700 */ 1701 void 1702 mac_hwring_set_default(mac_handle_t mh, mac_ring_handle_t rh) 1703 { 1704 mac_impl_t *mip = (mac_impl_t *)mh; 1705 mac_ring_t *ring = (mac_ring_t *)rh; 1706 1707 ASSERT(MAC_PERIM_HELD(mh)); 1708 VERIFY(mip->mi_state_flags & MIS_IS_AGGR); 1709 1710 if (ring->mr_state != MR_INUSE) 1711 return; 1712 1713 mip->mi_default_tx_ring = rh; 1714 } 1715 1716 int 1717 mac_hwgroup_addmac(mac_group_handle_t gh, const uint8_t *addr) 1718 { 1719 mac_group_t *group = (mac_group_t *)gh; 1720 1721 return (mac_group_addmac(group, addr)); 1722 } 1723 1724 int 1725 mac_hwgroup_remmac(mac_group_handle_t gh, const uint8_t *addr) 1726 { 1727 mac_group_t *group = (mac_group_t *)gh; 1728 1729 return (mac_group_remmac(group, addr)); 1730 } 1731 1732 /* 1733 * Set the RX group to be shared/reserved. Note that the group must be 1734 * started/stopped outside of this function. 1735 */ 1736 void 1737 mac_set_group_state(mac_group_t *grp, mac_group_state_t state) 1738 { 1739 /* 1740 * If there is no change in the group state, just return. 1741 */ 1742 if (grp->mrg_state == state) 1743 return; 1744 1745 switch (state) { 1746 case MAC_GROUP_STATE_RESERVED: 1747 /* 1748 * Successfully reserved the group. 1749 * 1750 * Given that there is an exclusive client controlling this 1751 * group, we enable the group level polling when available, 1752 * so that SRSs get to turn on/off individual rings they's 1753 * assigned to. 1754 */ 1755 ASSERT(MAC_PERIM_HELD(grp->mrg_mh)); 1756 1757 if (grp->mrg_type == MAC_RING_TYPE_RX && 1758 GROUP_INTR_DISABLE_FUNC(grp) != NULL) { 1759 GROUP_INTR_DISABLE_FUNC(grp)(GROUP_INTR_HANDLE(grp)); 1760 } 1761 break; 1762 1763 case MAC_GROUP_STATE_SHARED: 1764 /* 1765 * Set all rings of this group to software classified. 1766 * If the group has an overriding interrupt, then re-enable it. 1767 */ 1768 ASSERT(MAC_PERIM_HELD(grp->mrg_mh)); 1769 1770 if (grp->mrg_type == MAC_RING_TYPE_RX && 1771 GROUP_INTR_ENABLE_FUNC(grp) != NULL) { 1772 GROUP_INTR_ENABLE_FUNC(grp)(GROUP_INTR_HANDLE(grp)); 1773 } 1774 /* The ring is not available for reservations any more */ 1775 break; 1776 1777 case MAC_GROUP_STATE_REGISTERED: 1778 /* Also callable from mac_register, perim is not held */ 1779 break; 1780 1781 default: 1782 ASSERT(B_FALSE); 1783 break; 1784 } 1785 1786 grp->mrg_state = state; 1787 } 1788 1789 /* 1790 * Quiesce future hardware classified packets for the specified Rx ring 1791 */ 1792 static void 1793 mac_rx_ring_quiesce(mac_ring_t *rx_ring, uint_t ring_flag) 1794 { 1795 ASSERT(rx_ring->mr_classify_type == MAC_HW_CLASSIFIER); 1796 ASSERT(ring_flag == MR_CONDEMNED || ring_flag == MR_QUIESCE); 1797 1798 mutex_enter(&rx_ring->mr_lock); 1799 rx_ring->mr_flag |= ring_flag; 1800 while (rx_ring->mr_refcnt != 0) 1801 cv_wait(&rx_ring->mr_cv, &rx_ring->mr_lock); 1802 mutex_exit(&rx_ring->mr_lock); 1803 } 1804 1805 /* 1806 * Please see mac_tx for details about the per cpu locking scheme 1807 */ 1808 static void 1809 mac_tx_lock_all(mac_client_impl_t *mcip) 1810 { 1811 int i; 1812 1813 for (i = 0; i <= mac_tx_percpu_cnt; i++) 1814 mutex_enter(&mcip->mci_tx_pcpu[i].pcpu_tx_lock); 1815 } 1816 1817 static void 1818 mac_tx_unlock_all(mac_client_impl_t *mcip) 1819 { 1820 int i; 1821 1822 for (i = mac_tx_percpu_cnt; i >= 0; i--) 1823 mutex_exit(&mcip->mci_tx_pcpu[i].pcpu_tx_lock); 1824 } 1825 1826 static void 1827 mac_tx_unlock_allbutzero(mac_client_impl_t *mcip) 1828 { 1829 int i; 1830 1831 for (i = mac_tx_percpu_cnt; i > 0; i--) 1832 mutex_exit(&mcip->mci_tx_pcpu[i].pcpu_tx_lock); 1833 } 1834 1835 static int 1836 mac_tx_sum_refcnt(mac_client_impl_t *mcip) 1837 { 1838 int i; 1839 int refcnt = 0; 1840 1841 for (i = 0; i <= mac_tx_percpu_cnt; i++) 1842 refcnt += mcip->mci_tx_pcpu[i].pcpu_tx_refcnt; 1843 1844 return (refcnt); 1845 } 1846 1847 /* 1848 * Stop future Tx packets coming down from the client in preparation for 1849 * quiescing the Tx side. This is needed for dynamic reclaim and reassignment 1850 * of rings between clients 1851 */ 1852 void 1853 mac_tx_client_block(mac_client_impl_t *mcip) 1854 { 1855 mac_tx_lock_all(mcip); 1856 mcip->mci_tx_flag |= MCI_TX_QUIESCE; 1857 while (mac_tx_sum_refcnt(mcip) != 0) { 1858 mac_tx_unlock_allbutzero(mcip); 1859 cv_wait(&mcip->mci_tx_cv, &mcip->mci_tx_pcpu[0].pcpu_tx_lock); 1860 mutex_exit(&mcip->mci_tx_pcpu[0].pcpu_tx_lock); 1861 mac_tx_lock_all(mcip); 1862 } 1863 mac_tx_unlock_all(mcip); 1864 } 1865 1866 void 1867 mac_tx_client_unblock(mac_client_impl_t *mcip) 1868 { 1869 mac_tx_lock_all(mcip); 1870 mcip->mci_tx_flag &= ~MCI_TX_QUIESCE; 1871 mac_tx_unlock_all(mcip); 1872 /* 1873 * We may fail to disable flow control for the last MAC_NOTE_TX 1874 * notification because the MAC client is quiesced. Send the 1875 * notification again. 1876 */ 1877 i_mac_notify(mcip->mci_mip, MAC_NOTE_TX); 1878 } 1879 1880 /* 1881 * Wait for an SRS to quiesce. The SRS worker will signal us when the 1882 * quiesce is done. 1883 */ 1884 static void 1885 mac_srs_quiesce_wait(mac_soft_ring_set_t *srs, uint_t srs_flag) 1886 { 1887 mutex_enter(&srs->srs_lock); 1888 while (!(srs->srs_state & srs_flag)) 1889 cv_wait(&srs->srs_quiesce_done_cv, &srs->srs_lock); 1890 mutex_exit(&srs->srs_lock); 1891 } 1892 1893 /* 1894 * Quiescing an Rx SRS is achieved by the following sequence. The protocol 1895 * works bottom up by cutting off packet flow from the bottommost point in the 1896 * mac, then the SRS, and then the soft rings. There are 2 use cases of this 1897 * mechanism. One is a temporary quiesce of the SRS, such as say while changing 1898 * the Rx callbacks. Another use case is Rx SRS teardown. In the former case 1899 * the QUIESCE prefix/suffix is used and in the latter the CONDEMNED is used 1900 * for the SRS and MR flags. In the former case the threads pause waiting for 1901 * a restart, while in the latter case the threads exit. The Tx SRS teardown 1902 * is also mostly similar to the above. 1903 * 1904 * 1. Stop future hardware classified packets at the lowest level in the mac. 1905 * Remove any hardware classification rule (CONDEMNED case) and mark the 1906 * rings as CONDEMNED or QUIESCE as appropriate. This prevents the mr_refcnt 1907 * from increasing. Upcalls from the driver that come through hardware 1908 * classification will be dropped in mac_rx from now on. Then we wait for 1909 * the mr_refcnt to drop to zero. When the mr_refcnt reaches zero we are 1910 * sure there aren't any upcall threads from the driver through hardware 1911 * classification. In the case of SRS teardown we also remove the 1912 * classification rule in the driver. 1913 * 1914 * 2. Stop future software classified packets by marking the flow entry with 1915 * FE_QUIESCE or FE_CONDEMNED as appropriate which prevents the refcnt from 1916 * increasing. We also remove the flow entry from the table in the latter 1917 * case. Then wait for the fe_refcnt to reach an appropriate quiescent value 1918 * that indicates there aren't any active threads using that flow entry. 1919 * 1920 * 3. Quiesce the SRS and softrings by signaling the SRS. The SRS poll thread, 1921 * SRS worker thread, and the soft ring threads are quiesced in sequence 1922 * with the SRS worker thread serving as a master controller. This 1923 * mechansim is explained in mac_srs_worker_quiesce(). 1924 * 1925 * The restart mechanism to reactivate the SRS and softrings is explained 1926 * in mac_srs_worker_restart(). Here we just signal the SRS worker to start the 1927 * restart sequence. 1928 */ 1929 void 1930 mac_rx_srs_quiesce(mac_soft_ring_set_t *srs, uint_t srs_quiesce_flag) 1931 { 1932 flow_entry_t *flent = srs->srs_flent; 1933 uint_t mr_flag, srs_done_flag; 1934 1935 ASSERT(MAC_PERIM_HELD((mac_handle_t)FLENT_TO_MIP(flent))); 1936 ASSERT(!(srs->srs_type & SRST_TX)); 1937 1938 if (srs_quiesce_flag == SRS_CONDEMNED) { 1939 mr_flag = MR_CONDEMNED; 1940 srs_done_flag = SRS_CONDEMNED_DONE; 1941 if (srs->srs_type & SRST_CLIENT_POLL_ENABLED) 1942 mac_srs_client_poll_disable(srs->srs_mcip, srs); 1943 } else { 1944 ASSERT(srs_quiesce_flag == SRS_QUIESCE); 1945 mr_flag = MR_QUIESCE; 1946 srs_done_flag = SRS_QUIESCE_DONE; 1947 if (srs->srs_type & SRST_CLIENT_POLL_ENABLED) 1948 mac_srs_client_poll_quiesce(srs->srs_mcip, srs); 1949 } 1950 1951 if (srs->srs_ring != NULL) { 1952 mac_rx_ring_quiesce(srs->srs_ring, mr_flag); 1953 } else { 1954 /* 1955 * SRS is driven by software classification. In case 1956 * of CONDEMNED, the top level teardown functions will 1957 * deal with flow removal. 1958 */ 1959 if (srs_quiesce_flag != SRS_CONDEMNED) { 1960 FLOW_MARK(flent, FE_QUIESCE); 1961 mac_flow_wait(flent, FLOW_DRIVER_UPCALL); 1962 } 1963 } 1964 1965 /* 1966 * Signal the SRS to quiesce itself, and then cv_wait for the 1967 * SRS quiesce to complete. The SRS worker thread will wake us 1968 * up when the quiesce is complete 1969 */ 1970 mac_srs_signal(srs, srs_quiesce_flag); 1971 mac_srs_quiesce_wait(srs, srs_done_flag); 1972 } 1973 1974 /* 1975 * Remove an SRS. 1976 */ 1977 void 1978 mac_rx_srs_remove(mac_soft_ring_set_t *srs) 1979 { 1980 flow_entry_t *flent = srs->srs_flent; 1981 int i; 1982 1983 mac_rx_srs_quiesce(srs, SRS_CONDEMNED); 1984 /* 1985 * Locate and remove our entry in the fe_rx_srs[] array, and 1986 * adjust the fe_rx_srs array entries and array count by 1987 * moving the last entry into the vacated spot. 1988 */ 1989 mutex_enter(&flent->fe_lock); 1990 for (i = 0; i < flent->fe_rx_srs_cnt; i++) { 1991 if (flent->fe_rx_srs[i] == srs) 1992 break; 1993 } 1994 1995 ASSERT(i != 0 && i < flent->fe_rx_srs_cnt); 1996 if (i != flent->fe_rx_srs_cnt - 1) { 1997 flent->fe_rx_srs[i] = 1998 flent->fe_rx_srs[flent->fe_rx_srs_cnt - 1]; 1999 i = flent->fe_rx_srs_cnt - 1; 2000 } 2001 2002 flent->fe_rx_srs[i] = NULL; 2003 flent->fe_rx_srs_cnt--; 2004 mutex_exit(&flent->fe_lock); 2005 2006 mac_srs_free(srs); 2007 } 2008 2009 static void 2010 mac_srs_clear_flag(mac_soft_ring_set_t *srs, uint_t flag) 2011 { 2012 mutex_enter(&srs->srs_lock); 2013 srs->srs_state &= ~flag; 2014 mutex_exit(&srs->srs_lock); 2015 } 2016 2017 void 2018 mac_rx_srs_restart(mac_soft_ring_set_t *srs) 2019 { 2020 flow_entry_t *flent = srs->srs_flent; 2021 mac_ring_t *mr; 2022 2023 ASSERT(MAC_PERIM_HELD((mac_handle_t)FLENT_TO_MIP(flent))); 2024 ASSERT((srs->srs_type & SRST_TX) == 0); 2025 2026 /* 2027 * This handles a change in the number of SRSs between the quiesce and 2028 * and restart operation of a flow. 2029 */ 2030 if (!SRS_QUIESCED(srs)) 2031 return; 2032 2033 /* 2034 * Signal the SRS to restart itself. Wait for the restart to complete 2035 * Note that we only restart the SRS if it is not marked as 2036 * permanently quiesced. 2037 */ 2038 if (!SRS_QUIESCED_PERMANENT(srs)) { 2039 mac_srs_signal(srs, SRS_RESTART); 2040 mac_srs_quiesce_wait(srs, SRS_RESTART_DONE); 2041 mac_srs_clear_flag(srs, SRS_RESTART_DONE); 2042 2043 mac_srs_client_poll_restart(srs->srs_mcip, srs); 2044 } 2045 2046 /* Finally clear the flags to let the packets in */ 2047 mr = srs->srs_ring; 2048 if (mr != NULL) { 2049 MAC_RING_UNMARK(mr, MR_QUIESCE); 2050 /* In case the ring was stopped, safely restart it */ 2051 if (mr->mr_state != MR_INUSE) 2052 (void) mac_start_ring(mr); 2053 } else { 2054 FLOW_UNMARK(flent, FE_QUIESCE); 2055 } 2056 } 2057 2058 /* 2059 * Temporary quiesce of a flow and associated Rx SRS. 2060 * Please see block comment above mac_rx_classify_flow_rem. 2061 */ 2062 /* ARGSUSED */ 2063 int 2064 mac_rx_classify_flow_quiesce(flow_entry_t *flent, void *arg) 2065 { 2066 int i; 2067 2068 for (i = 0; i < flent->fe_rx_srs_cnt; i++) { 2069 mac_rx_srs_quiesce((mac_soft_ring_set_t *)flent->fe_rx_srs[i], 2070 SRS_QUIESCE); 2071 } 2072 return (0); 2073 } 2074 2075 /* 2076 * Restart a flow and associated Rx SRS that has been quiesced temporarily 2077 * Please see block comment above mac_rx_classify_flow_rem 2078 */ 2079 /* ARGSUSED */ 2080 int 2081 mac_rx_classify_flow_restart(flow_entry_t *flent, void *arg) 2082 { 2083 int i; 2084 2085 for (i = 0; i < flent->fe_rx_srs_cnt; i++) 2086 mac_rx_srs_restart((mac_soft_ring_set_t *)flent->fe_rx_srs[i]); 2087 2088 return (0); 2089 } 2090 2091 void 2092 mac_srs_perm_quiesce(mac_client_handle_t mch, boolean_t on) 2093 { 2094 mac_client_impl_t *mcip = (mac_client_impl_t *)mch; 2095 flow_entry_t *flent = mcip->mci_flent; 2096 mac_impl_t *mip = mcip->mci_mip; 2097 mac_soft_ring_set_t *mac_srs; 2098 int i; 2099 2100 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 2101 2102 if (flent == NULL) 2103 return; 2104 2105 for (i = 0; i < flent->fe_rx_srs_cnt; i++) { 2106 mac_srs = flent->fe_rx_srs[i]; 2107 mutex_enter(&mac_srs->srs_lock); 2108 if (on) 2109 mac_srs->srs_state |= SRS_QUIESCE_PERM; 2110 else 2111 mac_srs->srs_state &= ~SRS_QUIESCE_PERM; 2112 mutex_exit(&mac_srs->srs_lock); 2113 } 2114 } 2115 2116 void 2117 mac_rx_client_quiesce(mac_client_handle_t mch) 2118 { 2119 mac_client_impl_t *mcip = (mac_client_impl_t *)mch; 2120 mac_impl_t *mip = mcip->mci_mip; 2121 2122 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 2123 2124 if (MCIP_DATAPATH_SETUP(mcip)) { 2125 (void) mac_rx_classify_flow_quiesce(mcip->mci_flent, 2126 NULL); 2127 (void) mac_flow_walk_nolock(mcip->mci_subflow_tab, 2128 mac_rx_classify_flow_quiesce, NULL); 2129 } 2130 } 2131 2132 void 2133 mac_rx_client_restart(mac_client_handle_t mch) 2134 { 2135 mac_client_impl_t *mcip = (mac_client_impl_t *)mch; 2136 mac_impl_t *mip = mcip->mci_mip; 2137 2138 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 2139 2140 if (MCIP_DATAPATH_SETUP(mcip)) { 2141 (void) mac_rx_classify_flow_restart(mcip->mci_flent, NULL); 2142 (void) mac_flow_walk_nolock(mcip->mci_subflow_tab, 2143 mac_rx_classify_flow_restart, NULL); 2144 } 2145 } 2146 2147 /* 2148 * This function only quiesces the Tx SRS and softring worker threads. Callers 2149 * need to make sure that there aren't any mac client threads doing current or 2150 * future transmits in the mac before calling this function. 2151 */ 2152 void 2153 mac_tx_srs_quiesce(mac_soft_ring_set_t *srs, uint_t srs_quiesce_flag) 2154 { 2155 mac_client_impl_t *mcip = srs->srs_mcip; 2156 2157 ASSERT(MAC_PERIM_HELD((mac_handle_t)mcip->mci_mip)); 2158 2159 ASSERT(srs->srs_type & SRST_TX); 2160 ASSERT(srs_quiesce_flag == SRS_CONDEMNED || 2161 srs_quiesce_flag == SRS_QUIESCE); 2162 2163 /* 2164 * Signal the SRS to quiesce itself, and then cv_wait for the 2165 * SRS quiesce to complete. The SRS worker thread will wake us 2166 * up when the quiesce is complete 2167 */ 2168 mac_srs_signal(srs, srs_quiesce_flag); 2169 mac_srs_quiesce_wait(srs, srs_quiesce_flag == SRS_QUIESCE ? 2170 SRS_QUIESCE_DONE : SRS_CONDEMNED_DONE); 2171 } 2172 2173 void 2174 mac_tx_srs_restart(mac_soft_ring_set_t *srs) 2175 { 2176 /* 2177 * Resizing the fanout could result in creation of new SRSs. 2178 * They may not necessarily be in the quiesced state in which 2179 * case it need be restarted 2180 */ 2181 if (!SRS_QUIESCED(srs)) 2182 return; 2183 2184 mac_srs_signal(srs, SRS_RESTART); 2185 mac_srs_quiesce_wait(srs, SRS_RESTART_DONE); 2186 mac_srs_clear_flag(srs, SRS_RESTART_DONE); 2187 } 2188 2189 /* 2190 * Temporary quiesce of a flow and associated Rx SRS. 2191 * Please see block comment above mac_rx_srs_quiesce 2192 */ 2193 /* ARGSUSED */ 2194 int 2195 mac_tx_flow_quiesce(flow_entry_t *flent, void *arg) 2196 { 2197 /* 2198 * The fe_tx_srs is null for a subflow on an interface that is 2199 * not plumbed 2200 */ 2201 if (flent->fe_tx_srs != NULL) 2202 mac_tx_srs_quiesce(flent->fe_tx_srs, SRS_QUIESCE); 2203 return (0); 2204 } 2205 2206 /* ARGSUSED */ 2207 int 2208 mac_tx_flow_restart(flow_entry_t *flent, void *arg) 2209 { 2210 /* 2211 * The fe_tx_srs is null for a subflow on an interface that is 2212 * not plumbed 2213 */ 2214 if (flent->fe_tx_srs != NULL) 2215 mac_tx_srs_restart(flent->fe_tx_srs); 2216 return (0); 2217 } 2218 2219 static void 2220 i_mac_tx_client_quiesce(mac_client_handle_t mch, uint_t srs_quiesce_flag) 2221 { 2222 mac_client_impl_t *mcip = (mac_client_impl_t *)mch; 2223 2224 ASSERT(MAC_PERIM_HELD((mac_handle_t)mcip->mci_mip)); 2225 2226 mac_tx_client_block(mcip); 2227 if (MCIP_TX_SRS(mcip) != NULL) { 2228 mac_tx_srs_quiesce(MCIP_TX_SRS(mcip), srs_quiesce_flag); 2229 (void) mac_flow_walk_nolock(mcip->mci_subflow_tab, 2230 mac_tx_flow_quiesce, NULL); 2231 } 2232 } 2233 2234 void 2235 mac_tx_client_quiesce(mac_client_handle_t mch) 2236 { 2237 i_mac_tx_client_quiesce(mch, SRS_QUIESCE); 2238 } 2239 2240 void 2241 mac_tx_client_condemn(mac_client_handle_t mch) 2242 { 2243 i_mac_tx_client_quiesce(mch, SRS_CONDEMNED); 2244 } 2245 2246 void 2247 mac_tx_client_restart(mac_client_handle_t mch) 2248 { 2249 mac_client_impl_t *mcip = (mac_client_impl_t *)mch; 2250 2251 ASSERT(MAC_PERIM_HELD((mac_handle_t)mcip->mci_mip)); 2252 2253 mac_tx_client_unblock(mcip); 2254 if (MCIP_TX_SRS(mcip) != NULL) { 2255 mac_tx_srs_restart(MCIP_TX_SRS(mcip)); 2256 (void) mac_flow_walk_nolock(mcip->mci_subflow_tab, 2257 mac_tx_flow_restart, NULL); 2258 } 2259 } 2260 2261 void 2262 mac_tx_client_flush(mac_client_impl_t *mcip) 2263 { 2264 ASSERT(MAC_PERIM_HELD((mac_handle_t)mcip->mci_mip)); 2265 2266 mac_tx_client_quiesce((mac_client_handle_t)mcip); 2267 mac_tx_client_restart((mac_client_handle_t)mcip); 2268 } 2269 2270 void 2271 mac_client_quiesce(mac_client_impl_t *mcip) 2272 { 2273 mac_rx_client_quiesce((mac_client_handle_t)mcip); 2274 mac_tx_client_quiesce((mac_client_handle_t)mcip); 2275 } 2276 2277 void 2278 mac_client_restart(mac_client_impl_t *mcip) 2279 { 2280 mac_rx_client_restart((mac_client_handle_t)mcip); 2281 mac_tx_client_restart((mac_client_handle_t)mcip); 2282 } 2283 2284 /* 2285 * Allocate a minor number. 2286 */ 2287 minor_t 2288 mac_minor_hold(boolean_t sleep) 2289 { 2290 id_t id; 2291 2292 /* 2293 * Grab a value from the arena. 2294 */ 2295 atomic_inc_32(&minor_count); 2296 2297 if (sleep) 2298 return ((uint_t)id_alloc(minor_ids)); 2299 2300 if ((id = id_alloc_nosleep(minor_ids)) == -1) { 2301 atomic_dec_32(&minor_count); 2302 return (0); 2303 } 2304 2305 return ((uint_t)id); 2306 } 2307 2308 /* 2309 * Release a previously allocated minor number. 2310 */ 2311 void 2312 mac_minor_rele(minor_t minor) 2313 { 2314 /* 2315 * Return the value to the arena. 2316 */ 2317 id_free(minor_ids, minor); 2318 atomic_dec_32(&minor_count); 2319 } 2320 2321 uint32_t 2322 mac_no_notification(mac_handle_t mh) 2323 { 2324 mac_impl_t *mip = (mac_impl_t *)mh; 2325 2326 return (((mip->mi_state_flags & MIS_LEGACY) != 0) ? 2327 mip->mi_capab_legacy.ml_unsup_note : 0); 2328 } 2329 2330 /* 2331 * Prevent any new opens of this mac in preparation for unregister 2332 */ 2333 int 2334 i_mac_disable(mac_impl_t *mip) 2335 { 2336 mac_client_impl_t *mcip; 2337 2338 rw_enter(&i_mac_impl_lock, RW_WRITER); 2339 if (mip->mi_state_flags & MIS_DISABLED) { 2340 /* Already disabled, return success */ 2341 rw_exit(&i_mac_impl_lock); 2342 return (0); 2343 } 2344 /* 2345 * See if there are any other references to this mac_t (e.g., VLAN's). 2346 * If so return failure. If all the other checks below pass, then 2347 * set mi_disabled atomically under the i_mac_impl_lock to prevent 2348 * any new VLAN's from being created or new mac client opens of this 2349 * mac end point. 2350 */ 2351 if (mip->mi_ref > 0) { 2352 rw_exit(&i_mac_impl_lock); 2353 return (EBUSY); 2354 } 2355 2356 /* 2357 * mac clients must delete all multicast groups they join before 2358 * closing. bcast groups are reference counted, the last client 2359 * to delete the group will wait till the group is physically 2360 * deleted. Since all clients have closed this mac end point 2361 * mi_bcast_ngrps must be zero at this point 2362 */ 2363 ASSERT(mip->mi_bcast_ngrps == 0); 2364 2365 /* 2366 * Don't let go of this if it has some flows. 2367 * All other code guarantees no flows are added to a disabled 2368 * mac, therefore it is sufficient to check for the flow table 2369 * only here. 2370 */ 2371 mcip = mac_primary_client_handle(mip); 2372 if ((mcip != NULL) && mac_link_has_flows((mac_client_handle_t)mcip)) { 2373 rw_exit(&i_mac_impl_lock); 2374 return (ENOTEMPTY); 2375 } 2376 2377 mip->mi_state_flags |= MIS_DISABLED; 2378 rw_exit(&i_mac_impl_lock); 2379 return (0); 2380 } 2381 2382 int 2383 mac_disable_nowait(mac_handle_t mh) 2384 { 2385 mac_impl_t *mip = (mac_impl_t *)mh; 2386 int err; 2387 2388 if ((err = i_mac_perim_enter_nowait(mip)) != 0) 2389 return (err); 2390 err = i_mac_disable(mip); 2391 i_mac_perim_exit(mip); 2392 return (err); 2393 } 2394 2395 int 2396 mac_disable(mac_handle_t mh) 2397 { 2398 mac_impl_t *mip = (mac_impl_t *)mh; 2399 int err; 2400 2401 i_mac_perim_enter(mip); 2402 err = i_mac_disable(mip); 2403 i_mac_perim_exit(mip); 2404 2405 /* 2406 * Clean up notification thread and wait for it to exit. 2407 */ 2408 if (err == 0) 2409 i_mac_notify_exit(mip); 2410 2411 return (err); 2412 } 2413 2414 /* 2415 * Called when the MAC instance has a non empty flow table, to de-multiplex 2416 * incoming packets to the right flow. 2417 * The MAC's rw lock is assumed held as a READER. 2418 */ 2419 /* ARGSUSED */ 2420 static mblk_t * 2421 mac_rx_classify(mac_impl_t *mip, mac_resource_handle_t mrh, mblk_t *mp) 2422 { 2423 flow_entry_t *flent = NULL; 2424 uint_t flags = FLOW_INBOUND; 2425 int err; 2426 2427 /* 2428 * If the mac is a port of an aggregation, pass FLOW_IGNORE_VLAN 2429 * to mac_flow_lookup() so that the VLAN packets can be successfully 2430 * passed to the non-VLAN aggregation flows. 2431 * 2432 * Note that there is possibly a race between this and 2433 * mac_unicast_remove/add() and VLAN packets could be incorrectly 2434 * classified to non-VLAN flows of non-aggregation mac clients. These 2435 * VLAN packets will be then filtered out by the mac module. 2436 */ 2437 if ((mip->mi_state_flags & MIS_EXCLUSIVE) != 0) 2438 flags |= FLOW_IGNORE_VLAN; 2439 2440 err = mac_flow_lookup(mip->mi_flow_tab, mp, flags, &flent); 2441 if (err != 0) { 2442 /* no registered receive function */ 2443 return (mp); 2444 } else { 2445 mac_client_impl_t *mcip; 2446 2447 /* 2448 * This flent might just be an additional one on the MAC client, 2449 * i.e. for classification purposes (different fdesc), however 2450 * the resources, SRS et. al., are in the mci_flent, so if 2451 * this isn't the mci_flent, we need to get it. 2452 */ 2453 if ((mcip = flent->fe_mcip) != NULL && 2454 mcip->mci_flent != flent) { 2455 FLOW_REFRELE(flent); 2456 flent = mcip->mci_flent; 2457 FLOW_TRY_REFHOLD(flent, err); 2458 if (err != 0) 2459 return (mp); 2460 } 2461 (flent->fe_cb_fn)(flent->fe_cb_arg1, flent->fe_cb_arg2, mp, 2462 B_FALSE); 2463 FLOW_REFRELE(flent); 2464 } 2465 return (NULL); 2466 } 2467 2468 mblk_t * 2469 mac_rx_flow(mac_handle_t mh, mac_resource_handle_t mrh, mblk_t *mp_chain) 2470 { 2471 mac_impl_t *mip = (mac_impl_t *)mh; 2472 mblk_t *bp, *bp1, **bpp, *list = NULL; 2473 2474 /* 2475 * We walk the chain and attempt to classify each packet. 2476 * The packets that couldn't be classified will be returned 2477 * back to the caller. 2478 */ 2479 bp = mp_chain; 2480 bpp = &list; 2481 while (bp != NULL) { 2482 bp1 = bp; 2483 bp = bp->b_next; 2484 bp1->b_next = NULL; 2485 2486 if (mac_rx_classify(mip, mrh, bp1) != NULL) { 2487 *bpp = bp1; 2488 bpp = &bp1->b_next; 2489 } 2490 } 2491 return (list); 2492 } 2493 2494 static int 2495 mac_tx_flow_srs_wakeup(flow_entry_t *flent, void *arg) 2496 { 2497 mac_ring_handle_t ring = arg; 2498 2499 if (flent->fe_tx_srs) 2500 mac_tx_srs_wakeup(flent->fe_tx_srs, ring); 2501 return (0); 2502 } 2503 2504 void 2505 i_mac_tx_srs_notify(mac_impl_t *mip, mac_ring_handle_t ring) 2506 { 2507 mac_client_impl_t *cclient; 2508 mac_soft_ring_set_t *mac_srs; 2509 2510 /* 2511 * After grabbing the mi_rw_lock, the list of clients can't change. 2512 * If there are any clients mi_disabled must be B_FALSE and can't 2513 * get set since there are clients. If there aren't any clients we 2514 * don't do anything. In any case the mip has to be valid. The driver 2515 * must make sure that it goes single threaded (with respect to mac 2516 * calls) and wait for all pending mac calls to finish before calling 2517 * mac_unregister. 2518 */ 2519 rw_enter(&i_mac_impl_lock, RW_READER); 2520 if (mip->mi_state_flags & MIS_DISABLED) { 2521 rw_exit(&i_mac_impl_lock); 2522 return; 2523 } 2524 2525 /* 2526 * Get MAC tx srs from walking mac_client_handle list. 2527 */ 2528 rw_enter(&mip->mi_rw_lock, RW_READER); 2529 for (cclient = mip->mi_clients_list; cclient != NULL; 2530 cclient = cclient->mci_client_next) { 2531 if ((mac_srs = MCIP_TX_SRS(cclient)) != NULL) { 2532 mac_tx_srs_wakeup(mac_srs, ring); 2533 } else { 2534 /* 2535 * Aggr opens underlying ports in exclusive mode 2536 * and registers flow control callbacks using 2537 * mac_tx_client_notify(). When opened in 2538 * exclusive mode, Tx SRS won't be created 2539 * during mac_unicast_add(). 2540 */ 2541 if (cclient->mci_state_flags & MCIS_EXCLUSIVE) { 2542 mac_tx_invoke_callbacks(cclient, 2543 (mac_tx_cookie_t)ring); 2544 } 2545 } 2546 (void) mac_flow_walk(cclient->mci_subflow_tab, 2547 mac_tx_flow_srs_wakeup, ring); 2548 } 2549 rw_exit(&mip->mi_rw_lock); 2550 rw_exit(&i_mac_impl_lock); 2551 } 2552 2553 /* ARGSUSED */ 2554 void 2555 mac_multicast_refresh(mac_handle_t mh, mac_multicst_t refresh, void *arg, 2556 boolean_t add) 2557 { 2558 mac_impl_t *mip = (mac_impl_t *)mh; 2559 2560 i_mac_perim_enter((mac_impl_t *)mh); 2561 /* 2562 * If no specific refresh function was given then default to the 2563 * driver's m_multicst entry point. 2564 */ 2565 if (refresh == NULL) { 2566 refresh = mip->mi_multicst; 2567 arg = mip->mi_driver; 2568 } 2569 2570 mac_bcast_refresh(mip, refresh, arg, add); 2571 i_mac_perim_exit((mac_impl_t *)mh); 2572 } 2573 2574 void 2575 mac_promisc_refresh(mac_handle_t mh, mac_setpromisc_t refresh, void *arg) 2576 { 2577 mac_impl_t *mip = (mac_impl_t *)mh; 2578 2579 /* 2580 * If no specific refresh function was given then default to the 2581 * driver's m_promisc entry point. 2582 */ 2583 if (refresh == NULL) { 2584 refresh = mip->mi_setpromisc; 2585 arg = mip->mi_driver; 2586 } 2587 ASSERT(refresh != NULL); 2588 2589 /* 2590 * Call the refresh function with the current promiscuity. 2591 */ 2592 refresh(arg, (mip->mi_devpromisc != 0)); 2593 } 2594 2595 /* 2596 * The mac client requests that the mac not to change its margin size to 2597 * be less than the specified value. If "current" is B_TRUE, then the client 2598 * requests the mac not to change its margin size to be smaller than the 2599 * current size. Further, return the current margin size value in this case. 2600 * 2601 * We keep every requested size in an ordered list from largest to smallest. 2602 */ 2603 int 2604 mac_margin_add(mac_handle_t mh, uint32_t *marginp, boolean_t current) 2605 { 2606 mac_impl_t *mip = (mac_impl_t *)mh; 2607 mac_margin_req_t **pp, *p; 2608 int err = 0; 2609 2610 rw_enter(&(mip->mi_rw_lock), RW_WRITER); 2611 if (current) 2612 *marginp = mip->mi_margin; 2613 2614 /* 2615 * If the current margin value cannot satisfy the margin requested, 2616 * return ENOTSUP directly. 2617 */ 2618 if (*marginp > mip->mi_margin) { 2619 err = ENOTSUP; 2620 goto done; 2621 } 2622 2623 /* 2624 * Check whether the given margin is already in the list. If so, 2625 * bump the reference count. 2626 */ 2627 for (pp = &mip->mi_mmrp; (p = *pp) != NULL; pp = &p->mmr_nextp) { 2628 if (p->mmr_margin == *marginp) { 2629 /* 2630 * The margin requested is already in the list, 2631 * so just bump the reference count. 2632 */ 2633 p->mmr_ref++; 2634 goto done; 2635 } 2636 if (p->mmr_margin < *marginp) 2637 break; 2638 } 2639 2640 2641 p = kmem_zalloc(sizeof (mac_margin_req_t), KM_SLEEP); 2642 p->mmr_margin = *marginp; 2643 p->mmr_ref++; 2644 p->mmr_nextp = *pp; 2645 *pp = p; 2646 2647 done: 2648 rw_exit(&(mip->mi_rw_lock)); 2649 return (err); 2650 } 2651 2652 /* 2653 * The mac client requests to cancel its previous mac_margin_add() request. 2654 * We remove the requested margin size from the list. 2655 */ 2656 int 2657 mac_margin_remove(mac_handle_t mh, uint32_t margin) 2658 { 2659 mac_impl_t *mip = (mac_impl_t *)mh; 2660 mac_margin_req_t **pp, *p; 2661 int err = 0; 2662 2663 rw_enter(&(mip->mi_rw_lock), RW_WRITER); 2664 /* 2665 * Find the entry in the list for the given margin. 2666 */ 2667 for (pp = &(mip->mi_mmrp); (p = *pp) != NULL; pp = &(p->mmr_nextp)) { 2668 if (p->mmr_margin == margin) { 2669 if (--p->mmr_ref == 0) 2670 break; 2671 2672 /* 2673 * There is still a reference to this address so 2674 * there's nothing more to do. 2675 */ 2676 goto done; 2677 } 2678 } 2679 2680 /* 2681 * We did not find an entry for the given margin. 2682 */ 2683 if (p == NULL) { 2684 err = ENOENT; 2685 goto done; 2686 } 2687 2688 ASSERT(p->mmr_ref == 0); 2689 2690 /* 2691 * Remove it from the list. 2692 */ 2693 *pp = p->mmr_nextp; 2694 kmem_free(p, sizeof (mac_margin_req_t)); 2695 done: 2696 rw_exit(&(mip->mi_rw_lock)); 2697 return (err); 2698 } 2699 2700 boolean_t 2701 mac_margin_update(mac_handle_t mh, uint32_t margin) 2702 { 2703 mac_impl_t *mip = (mac_impl_t *)mh; 2704 uint32_t margin_needed = 0; 2705 2706 rw_enter(&(mip->mi_rw_lock), RW_WRITER); 2707 2708 if (mip->mi_mmrp != NULL) 2709 margin_needed = mip->mi_mmrp->mmr_margin; 2710 2711 if (margin_needed <= margin) 2712 mip->mi_margin = margin; 2713 2714 rw_exit(&(mip->mi_rw_lock)); 2715 2716 if (margin_needed <= margin) 2717 i_mac_notify(mip, MAC_NOTE_MARGIN); 2718 2719 return (margin_needed <= margin); 2720 } 2721 2722 /* 2723 * MAC clients use this interface to request that a MAC device not change its 2724 * MTU below the specified amount. At this time, that amount must be within the 2725 * range of the device's current minimum and the device's current maximum. eg. a 2726 * client cannot request a 3000 byte MTU when the device's MTU is currently 2727 * 2000. 2728 * 2729 * If "current" is set to B_TRUE, then the request is to simply to reserve the 2730 * current underlying mac's maximum for this mac client and return it in mtup. 2731 */ 2732 int 2733 mac_mtu_add(mac_handle_t mh, uint32_t *mtup, boolean_t current) 2734 { 2735 mac_impl_t *mip = (mac_impl_t *)mh; 2736 mac_mtu_req_t *prev, *cur; 2737 mac_propval_range_t mpr; 2738 int err; 2739 2740 i_mac_perim_enter(mip); 2741 rw_enter(&mip->mi_rw_lock, RW_WRITER); 2742 2743 if (current == B_TRUE) 2744 *mtup = mip->mi_sdu_max; 2745 mpr.mpr_count = 1; 2746 err = mac_prop_info(mh, MAC_PROP_MTU, "mtu", NULL, 0, &mpr, NULL); 2747 if (err != 0) { 2748 rw_exit(&mip->mi_rw_lock); 2749 i_mac_perim_exit(mip); 2750 return (err); 2751 } 2752 2753 if (*mtup > mip->mi_sdu_max || 2754 *mtup < mpr.mpr_range_uint32[0].mpur_min) { 2755 rw_exit(&mip->mi_rw_lock); 2756 i_mac_perim_exit(mip); 2757 return (ENOTSUP); 2758 } 2759 2760 prev = NULL; 2761 for (cur = mip->mi_mtrp; cur != NULL; cur = cur->mtr_nextp) { 2762 if (*mtup == cur->mtr_mtu) { 2763 cur->mtr_ref++; 2764 rw_exit(&mip->mi_rw_lock); 2765 i_mac_perim_exit(mip); 2766 return (0); 2767 } 2768 2769 if (*mtup > cur->mtr_mtu) 2770 break; 2771 2772 prev = cur; 2773 } 2774 2775 cur = kmem_alloc(sizeof (mac_mtu_req_t), KM_SLEEP); 2776 cur->mtr_mtu = *mtup; 2777 cur->mtr_ref = 1; 2778 if (prev != NULL) { 2779 cur->mtr_nextp = prev->mtr_nextp; 2780 prev->mtr_nextp = cur; 2781 } else { 2782 cur->mtr_nextp = mip->mi_mtrp; 2783 mip->mi_mtrp = cur; 2784 } 2785 2786 rw_exit(&mip->mi_rw_lock); 2787 i_mac_perim_exit(mip); 2788 return (0); 2789 } 2790 2791 int 2792 mac_mtu_remove(mac_handle_t mh, uint32_t mtu) 2793 { 2794 mac_impl_t *mip = (mac_impl_t *)mh; 2795 mac_mtu_req_t *cur, *prev; 2796 2797 i_mac_perim_enter(mip); 2798 rw_enter(&mip->mi_rw_lock, RW_WRITER); 2799 2800 prev = NULL; 2801 for (cur = mip->mi_mtrp; cur != NULL; cur = cur->mtr_nextp) { 2802 if (cur->mtr_mtu == mtu) { 2803 ASSERT(cur->mtr_ref > 0); 2804 cur->mtr_ref--; 2805 if (cur->mtr_ref == 0) { 2806 if (prev == NULL) { 2807 mip->mi_mtrp = cur->mtr_nextp; 2808 } else { 2809 prev->mtr_nextp = cur->mtr_nextp; 2810 } 2811 kmem_free(cur, sizeof (mac_mtu_req_t)); 2812 } 2813 rw_exit(&mip->mi_rw_lock); 2814 i_mac_perim_exit(mip); 2815 return (0); 2816 } 2817 2818 prev = cur; 2819 } 2820 2821 rw_exit(&mip->mi_rw_lock); 2822 i_mac_perim_exit(mip); 2823 return (ENOENT); 2824 } 2825 2826 /* 2827 * MAC Type Plugin functions. 2828 */ 2829 2830 mactype_t * 2831 mactype_getplugin(const char *pname) 2832 { 2833 mactype_t *mtype = NULL; 2834 boolean_t tried_modload = B_FALSE; 2835 2836 mutex_enter(&i_mactype_lock); 2837 2838 find_registered_mactype: 2839 if (mod_hash_find(i_mactype_hash, (mod_hash_key_t)pname, 2840 (mod_hash_val_t *)&mtype) != 0) { 2841 if (!tried_modload) { 2842 /* 2843 * If the plugin has not yet been loaded, then 2844 * attempt to load it now. If modload() succeeds, 2845 * the plugin should have registered using 2846 * mactype_register(), in which case we can go back 2847 * and attempt to find it again. 2848 */ 2849 if (modload(MACTYPE_KMODDIR, (char *)pname) != -1) { 2850 tried_modload = B_TRUE; 2851 goto find_registered_mactype; 2852 } 2853 } 2854 } else { 2855 /* 2856 * Note that there's no danger that the plugin we've loaded 2857 * could be unloaded between the modload() step and the 2858 * reference count bump here, as we're holding 2859 * i_mactype_lock, which mactype_unregister() also holds. 2860 */ 2861 atomic_inc_32(&mtype->mt_ref); 2862 } 2863 2864 mutex_exit(&i_mactype_lock); 2865 return (mtype); 2866 } 2867 2868 mactype_register_t * 2869 mactype_alloc(uint_t mactype_version) 2870 { 2871 mactype_register_t *mtrp; 2872 2873 /* 2874 * Make sure there isn't a version mismatch between the plugin and 2875 * the framework. In the future, if multiple versions are 2876 * supported, this check could become more sophisticated. 2877 */ 2878 if (mactype_version != MACTYPE_VERSION) 2879 return (NULL); 2880 2881 mtrp = kmem_zalloc(sizeof (mactype_register_t), KM_SLEEP); 2882 mtrp->mtr_version = mactype_version; 2883 return (mtrp); 2884 } 2885 2886 void 2887 mactype_free(mactype_register_t *mtrp) 2888 { 2889 kmem_free(mtrp, sizeof (mactype_register_t)); 2890 } 2891 2892 int 2893 mactype_register(mactype_register_t *mtrp) 2894 { 2895 mactype_t *mtp; 2896 mactype_ops_t *ops = mtrp->mtr_ops; 2897 2898 /* Do some sanity checking before we register this MAC type. */ 2899 if (mtrp->mtr_ident == NULL || ops == NULL) 2900 return (EINVAL); 2901 2902 /* 2903 * Verify that all mandatory callbacks are set in the ops 2904 * vector. 2905 */ 2906 if (ops->mtops_unicst_verify == NULL || 2907 ops->mtops_multicst_verify == NULL || 2908 ops->mtops_sap_verify == NULL || 2909 ops->mtops_header == NULL || 2910 ops->mtops_header_info == NULL) { 2911 return (EINVAL); 2912 } 2913 2914 mtp = kmem_zalloc(sizeof (*mtp), KM_SLEEP); 2915 mtp->mt_ident = mtrp->mtr_ident; 2916 mtp->mt_ops = *ops; 2917 mtp->mt_type = mtrp->mtr_mactype; 2918 mtp->mt_nativetype = mtrp->mtr_nativetype; 2919 mtp->mt_addr_length = mtrp->mtr_addrlen; 2920 if (mtrp->mtr_brdcst_addr != NULL) { 2921 mtp->mt_brdcst_addr = kmem_alloc(mtrp->mtr_addrlen, KM_SLEEP); 2922 bcopy(mtrp->mtr_brdcst_addr, mtp->mt_brdcst_addr, 2923 mtrp->mtr_addrlen); 2924 } 2925 2926 mtp->mt_stats = mtrp->mtr_stats; 2927 mtp->mt_statcount = mtrp->mtr_statcount; 2928 2929 mtp->mt_mapping = mtrp->mtr_mapping; 2930 mtp->mt_mappingcount = mtrp->mtr_mappingcount; 2931 2932 if (mod_hash_insert(i_mactype_hash, 2933 (mod_hash_key_t)mtp->mt_ident, (mod_hash_val_t)mtp) != 0) { 2934 kmem_free(mtp->mt_brdcst_addr, mtp->mt_addr_length); 2935 kmem_free(mtp, sizeof (*mtp)); 2936 return (EEXIST); 2937 } 2938 return (0); 2939 } 2940 2941 int 2942 mactype_unregister(const char *ident) 2943 { 2944 mactype_t *mtp; 2945 mod_hash_val_t val; 2946 int err; 2947 2948 /* 2949 * Let's not allow MAC drivers to use this plugin while we're 2950 * trying to unregister it. Holding i_mactype_lock also prevents a 2951 * plugin from unregistering while a MAC driver is attempting to 2952 * hold a reference to it in i_mactype_getplugin(). 2953 */ 2954 mutex_enter(&i_mactype_lock); 2955 2956 if ((err = mod_hash_find(i_mactype_hash, (mod_hash_key_t)ident, 2957 (mod_hash_val_t *)&mtp)) != 0) { 2958 /* A plugin is trying to unregister, but it never registered. */ 2959 err = ENXIO; 2960 goto done; 2961 } 2962 2963 if (mtp->mt_ref != 0) { 2964 err = EBUSY; 2965 goto done; 2966 } 2967 2968 err = mod_hash_remove(i_mactype_hash, (mod_hash_key_t)ident, &val); 2969 ASSERT(err == 0); 2970 if (err != 0) { 2971 /* This should never happen, thus the ASSERT() above. */ 2972 err = EINVAL; 2973 goto done; 2974 } 2975 ASSERT(mtp == (mactype_t *)val); 2976 2977 if (mtp->mt_brdcst_addr != NULL) 2978 kmem_free(mtp->mt_brdcst_addr, mtp->mt_addr_length); 2979 kmem_free(mtp, sizeof (mactype_t)); 2980 done: 2981 mutex_exit(&i_mactype_lock); 2982 return (err); 2983 } 2984 2985 /* 2986 * Checks the size of the value size specified for a property as 2987 * part of a property operation. Returns B_TRUE if the size is 2988 * correct, B_FALSE otherwise. 2989 */ 2990 boolean_t 2991 mac_prop_check_size(mac_prop_id_t id, uint_t valsize, boolean_t is_range) 2992 { 2993 uint_t minsize = 0; 2994 2995 if (is_range) 2996 return (valsize >= sizeof (mac_propval_range_t)); 2997 2998 switch (id) { 2999 case MAC_PROP_ZONE: 3000 minsize = sizeof (dld_ioc_zid_t); 3001 break; 3002 case MAC_PROP_AUTOPUSH: 3003 if (valsize != 0) 3004 minsize = sizeof (struct dlautopush); 3005 break; 3006 case MAC_PROP_TAGMODE: 3007 minsize = sizeof (link_tagmode_t); 3008 break; 3009 case MAC_PROP_RESOURCE: 3010 case MAC_PROP_RESOURCE_EFF: 3011 minsize = sizeof (mac_resource_props_t); 3012 break; 3013 case MAC_PROP_DUPLEX: 3014 minsize = sizeof (link_duplex_t); 3015 break; 3016 case MAC_PROP_SPEED: 3017 minsize = sizeof (uint64_t); 3018 break; 3019 case MAC_PROP_STATUS: 3020 minsize = sizeof (link_state_t); 3021 break; 3022 case MAC_PROP_AUTONEG: 3023 case MAC_PROP_EN_AUTONEG: 3024 minsize = sizeof (uint8_t); 3025 break; 3026 case MAC_PROP_MTU: 3027 case MAC_PROP_LLIMIT: 3028 case MAC_PROP_LDECAY: 3029 minsize = sizeof (uint32_t); 3030 break; 3031 case MAC_PROP_FLOWCTRL: 3032 minsize = sizeof (link_flowctrl_t); 3033 break; 3034 case MAC_PROP_ADV_5000FDX_CAP: 3035 case MAC_PROP_EN_5000FDX_CAP: 3036 case MAC_PROP_ADV_2500FDX_CAP: 3037 case MAC_PROP_EN_2500FDX_CAP: 3038 case MAC_PROP_ADV_100GFDX_CAP: 3039 case MAC_PROP_EN_100GFDX_CAP: 3040 case MAC_PROP_ADV_50GFDX_CAP: 3041 case MAC_PROP_EN_50GFDX_CAP: 3042 case MAC_PROP_ADV_40GFDX_CAP: 3043 case MAC_PROP_EN_40GFDX_CAP: 3044 case MAC_PROP_ADV_25GFDX_CAP: 3045 case MAC_PROP_EN_25GFDX_CAP: 3046 case MAC_PROP_ADV_10GFDX_CAP: 3047 case MAC_PROP_EN_10GFDX_CAP: 3048 case MAC_PROP_ADV_1000HDX_CAP: 3049 case MAC_PROP_EN_1000HDX_CAP: 3050 case MAC_PROP_ADV_100FDX_CAP: 3051 case MAC_PROP_EN_100FDX_CAP: 3052 case MAC_PROP_ADV_100HDX_CAP: 3053 case MAC_PROP_EN_100HDX_CAP: 3054 case MAC_PROP_ADV_10FDX_CAP: 3055 case MAC_PROP_EN_10FDX_CAP: 3056 case MAC_PROP_ADV_10HDX_CAP: 3057 case MAC_PROP_EN_10HDX_CAP: 3058 case MAC_PROP_ADV_100T4_CAP: 3059 case MAC_PROP_EN_100T4_CAP: 3060 minsize = sizeof (uint8_t); 3061 break; 3062 case MAC_PROP_PVID: 3063 minsize = sizeof (uint16_t); 3064 break; 3065 case MAC_PROP_IPTUN_HOPLIMIT: 3066 minsize = sizeof (uint32_t); 3067 break; 3068 case MAC_PROP_IPTUN_ENCAPLIMIT: 3069 minsize = sizeof (uint32_t); 3070 break; 3071 case MAC_PROP_MAX_TX_RINGS_AVAIL: 3072 case MAC_PROP_MAX_RX_RINGS_AVAIL: 3073 case MAC_PROP_MAX_RXHWCLNT_AVAIL: 3074 case MAC_PROP_MAX_TXHWCLNT_AVAIL: 3075 minsize = sizeof (uint_t); 3076 break; 3077 case MAC_PROP_WL_ESSID: 3078 minsize = sizeof (wl_linkstatus_t); 3079 break; 3080 case MAC_PROP_WL_BSSID: 3081 minsize = sizeof (wl_bssid_t); 3082 break; 3083 case MAC_PROP_WL_BSSTYPE: 3084 minsize = sizeof (wl_bss_type_t); 3085 break; 3086 case MAC_PROP_WL_LINKSTATUS: 3087 minsize = sizeof (wl_linkstatus_t); 3088 break; 3089 case MAC_PROP_WL_DESIRED_RATES: 3090 minsize = sizeof (wl_rates_t); 3091 break; 3092 case MAC_PROP_WL_SUPPORTED_RATES: 3093 minsize = sizeof (wl_rates_t); 3094 break; 3095 case MAC_PROP_WL_AUTH_MODE: 3096 minsize = sizeof (wl_authmode_t); 3097 break; 3098 case MAC_PROP_WL_ENCRYPTION: 3099 minsize = sizeof (wl_encryption_t); 3100 break; 3101 case MAC_PROP_WL_RSSI: 3102 minsize = sizeof (wl_rssi_t); 3103 break; 3104 case MAC_PROP_WL_PHY_CONFIG: 3105 minsize = sizeof (wl_phy_conf_t); 3106 break; 3107 case MAC_PROP_WL_CAPABILITY: 3108 minsize = sizeof (wl_capability_t); 3109 break; 3110 case MAC_PROP_WL_WPA: 3111 minsize = sizeof (wl_wpa_t); 3112 break; 3113 case MAC_PROP_WL_SCANRESULTS: 3114 minsize = sizeof (wl_wpa_ess_t); 3115 break; 3116 case MAC_PROP_WL_POWER_MODE: 3117 minsize = sizeof (wl_ps_mode_t); 3118 break; 3119 case MAC_PROP_WL_RADIO: 3120 minsize = sizeof (wl_radio_t); 3121 break; 3122 case MAC_PROP_WL_ESS_LIST: 3123 minsize = sizeof (wl_ess_list_t); 3124 break; 3125 case MAC_PROP_WL_KEY_TAB: 3126 minsize = sizeof (wl_wep_key_tab_t); 3127 break; 3128 case MAC_PROP_WL_CREATE_IBSS: 3129 minsize = sizeof (wl_create_ibss_t); 3130 break; 3131 case MAC_PROP_WL_SETOPTIE: 3132 minsize = sizeof (wl_wpa_ie_t); 3133 break; 3134 case MAC_PROP_WL_DELKEY: 3135 minsize = sizeof (wl_del_key_t); 3136 break; 3137 case MAC_PROP_WL_KEY: 3138 minsize = sizeof (wl_key_t); 3139 break; 3140 case MAC_PROP_WL_MLME: 3141 minsize = sizeof (wl_mlme_t); 3142 break; 3143 case MAC_PROP_VN_PROMISC_FILTERED: 3144 minsize = sizeof (boolean_t); 3145 break; 3146 } 3147 3148 return (valsize >= minsize); 3149 } 3150 3151 /* 3152 * mac_set_prop() sets MAC or hardware driver properties: 3153 * 3154 * - MAC-managed properties such as resource properties include maxbw, 3155 * priority, and cpu binding list, as well as the default port VID 3156 * used by bridging. These properties are consumed by the MAC layer 3157 * itself and not passed down to the driver. For resource control 3158 * properties, this function invokes mac_set_resources() which will 3159 * cache the property value in mac_impl_t and may call 3160 * mac_client_set_resource() to update property value of the primary 3161 * mac client, if it exists. 3162 * 3163 * - Properties which act on the hardware and must be passed to the 3164 * driver, such as MTU, through the driver's mc_setprop() entry point. 3165 */ 3166 int 3167 mac_set_prop(mac_handle_t mh, mac_prop_id_t id, char *name, void *val, 3168 uint_t valsize) 3169 { 3170 int err = ENOTSUP; 3171 mac_impl_t *mip = (mac_impl_t *)mh; 3172 3173 ASSERT(MAC_PERIM_HELD(mh)); 3174 3175 switch (id) { 3176 case MAC_PROP_RESOURCE: { 3177 mac_resource_props_t *mrp; 3178 3179 /* call mac_set_resources() for MAC properties */ 3180 ASSERT(valsize >= sizeof (mac_resource_props_t)); 3181 mrp = kmem_zalloc(sizeof (*mrp), KM_SLEEP); 3182 bcopy(val, mrp, sizeof (*mrp)); 3183 err = mac_set_resources(mh, mrp); 3184 kmem_free(mrp, sizeof (*mrp)); 3185 break; 3186 } 3187 3188 case MAC_PROP_PVID: 3189 ASSERT(valsize >= sizeof (uint16_t)); 3190 if (mip->mi_state_flags & MIS_IS_VNIC) 3191 return (EINVAL); 3192 err = mac_set_pvid(mh, *(uint16_t *)val); 3193 break; 3194 3195 case MAC_PROP_MTU: { 3196 uint32_t mtu; 3197 3198 ASSERT(valsize >= sizeof (uint32_t)); 3199 bcopy(val, &mtu, sizeof (mtu)); 3200 err = mac_set_mtu(mh, mtu, NULL); 3201 break; 3202 } 3203 3204 case MAC_PROP_LLIMIT: 3205 case MAC_PROP_LDECAY: { 3206 uint32_t learnval; 3207 3208 if (valsize < sizeof (learnval) || 3209 (mip->mi_state_flags & MIS_IS_VNIC)) 3210 return (EINVAL); 3211 bcopy(val, &learnval, sizeof (learnval)); 3212 if (learnval == 0 && id == MAC_PROP_LDECAY) 3213 return (EINVAL); 3214 if (id == MAC_PROP_LLIMIT) 3215 mip->mi_llimit = learnval; 3216 else 3217 mip->mi_ldecay = learnval; 3218 err = 0; 3219 break; 3220 } 3221 3222 default: 3223 /* For other driver properties, call driver's callback */ 3224 if (mip->mi_callbacks->mc_callbacks & MC_SETPROP) { 3225 err = mip->mi_callbacks->mc_setprop(mip->mi_driver, 3226 name, id, valsize, val); 3227 } 3228 } 3229 return (err); 3230 } 3231 3232 /* 3233 * mac_get_prop() gets MAC or device driver properties. 3234 * 3235 * If the property is a driver property, mac_get_prop() calls driver's callback 3236 * entry point to get it. 3237 * If the property is a MAC property, mac_get_prop() invokes mac_get_resources() 3238 * which returns the cached value in mac_impl_t. 3239 */ 3240 int 3241 mac_get_prop(mac_handle_t mh, mac_prop_id_t id, char *name, void *val, 3242 uint_t valsize) 3243 { 3244 int err = ENOTSUP; 3245 mac_impl_t *mip = (mac_impl_t *)mh; 3246 uint_t rings; 3247 uint_t vlinks; 3248 3249 bzero(val, valsize); 3250 3251 switch (id) { 3252 case MAC_PROP_RESOURCE: { 3253 mac_resource_props_t *mrp; 3254 3255 /* If mac property, read from cache */ 3256 ASSERT(valsize >= sizeof (mac_resource_props_t)); 3257 mrp = kmem_zalloc(sizeof (*mrp), KM_SLEEP); 3258 mac_get_resources(mh, mrp); 3259 bcopy(mrp, val, sizeof (*mrp)); 3260 kmem_free(mrp, sizeof (*mrp)); 3261 return (0); 3262 } 3263 case MAC_PROP_RESOURCE_EFF: { 3264 mac_resource_props_t *mrp; 3265 3266 /* If mac effective property, read from client */ 3267 ASSERT(valsize >= sizeof (mac_resource_props_t)); 3268 mrp = kmem_zalloc(sizeof (*mrp), KM_SLEEP); 3269 mac_get_effective_resources(mh, mrp); 3270 bcopy(mrp, val, sizeof (*mrp)); 3271 kmem_free(mrp, sizeof (*mrp)); 3272 return (0); 3273 } 3274 3275 case MAC_PROP_PVID: 3276 ASSERT(valsize >= sizeof (uint16_t)); 3277 if (mip->mi_state_flags & MIS_IS_VNIC) 3278 return (EINVAL); 3279 *(uint16_t *)val = mac_get_pvid(mh); 3280 return (0); 3281 3282 case MAC_PROP_LLIMIT: 3283 case MAC_PROP_LDECAY: 3284 ASSERT(valsize >= sizeof (uint32_t)); 3285 if (mip->mi_state_flags & MIS_IS_VNIC) 3286 return (EINVAL); 3287 if (id == MAC_PROP_LLIMIT) 3288 bcopy(&mip->mi_llimit, val, sizeof (mip->mi_llimit)); 3289 else 3290 bcopy(&mip->mi_ldecay, val, sizeof (mip->mi_ldecay)); 3291 return (0); 3292 3293 case MAC_PROP_MTU: { 3294 uint32_t sdu; 3295 3296 ASSERT(valsize >= sizeof (uint32_t)); 3297 mac_sdu_get2(mh, NULL, &sdu, NULL); 3298 bcopy(&sdu, val, sizeof (sdu)); 3299 3300 return (0); 3301 } 3302 case MAC_PROP_STATUS: { 3303 link_state_t link_state; 3304 3305 if (valsize < sizeof (link_state)) 3306 return (EINVAL); 3307 link_state = mac_link_get(mh); 3308 bcopy(&link_state, val, sizeof (link_state)); 3309 3310 return (0); 3311 } 3312 3313 case MAC_PROP_MAX_RX_RINGS_AVAIL: 3314 case MAC_PROP_MAX_TX_RINGS_AVAIL: 3315 ASSERT(valsize >= sizeof (uint_t)); 3316 rings = id == MAC_PROP_MAX_RX_RINGS_AVAIL ? 3317 mac_rxavail_get(mh) : mac_txavail_get(mh); 3318 bcopy(&rings, val, sizeof (uint_t)); 3319 return (0); 3320 3321 case MAC_PROP_MAX_RXHWCLNT_AVAIL: 3322 case MAC_PROP_MAX_TXHWCLNT_AVAIL: 3323 ASSERT(valsize >= sizeof (uint_t)); 3324 vlinks = id == MAC_PROP_MAX_RXHWCLNT_AVAIL ? 3325 mac_rxhwlnksavail_get(mh) : mac_txhwlnksavail_get(mh); 3326 bcopy(&vlinks, val, sizeof (uint_t)); 3327 return (0); 3328 3329 case MAC_PROP_RXRINGSRANGE: 3330 case MAC_PROP_TXRINGSRANGE: 3331 /* 3332 * The value for these properties are returned through 3333 * the MAC_PROP_RESOURCE property. 3334 */ 3335 return (0); 3336 3337 default: 3338 break; 3339 3340 } 3341 3342 /* If driver property, request from driver */ 3343 if (mip->mi_callbacks->mc_callbacks & MC_GETPROP) { 3344 err = mip->mi_callbacks->mc_getprop(mip->mi_driver, name, id, 3345 valsize, val); 3346 } 3347 3348 return (err); 3349 } 3350 3351 /* 3352 * Helper function to initialize the range structure for use in 3353 * mac_get_prop. If the type can be other than uint32, we can 3354 * pass that as an arg. 3355 */ 3356 static void 3357 _mac_set_range(mac_propval_range_t *range, uint32_t min, uint32_t max) 3358 { 3359 range->mpr_count = 1; 3360 range->mpr_type = MAC_PROPVAL_UINT32; 3361 range->mpr_range_uint32[0].mpur_min = min; 3362 range->mpr_range_uint32[0].mpur_max = max; 3363 } 3364 3365 /* 3366 * Returns information about the specified property, such as default 3367 * values or permissions. 3368 */ 3369 int 3370 mac_prop_info(mac_handle_t mh, mac_prop_id_t id, char *name, 3371 void *default_val, uint_t default_size, mac_propval_range_t *range, 3372 uint_t *perm) 3373 { 3374 mac_prop_info_state_t state; 3375 mac_impl_t *mip = (mac_impl_t *)mh; 3376 uint_t max; 3377 3378 /* 3379 * A property is read/write by default unless the driver says 3380 * otherwise. 3381 */ 3382 if (perm != NULL) 3383 *perm = MAC_PROP_PERM_RW; 3384 3385 if (default_val != NULL) 3386 bzero(default_val, default_size); 3387 3388 /* 3389 * First, handle framework properties for which we don't need to 3390 * involve the driver. 3391 */ 3392 switch (id) { 3393 case MAC_PROP_RESOURCE: 3394 case MAC_PROP_PVID: 3395 case MAC_PROP_LLIMIT: 3396 case MAC_PROP_LDECAY: 3397 return (0); 3398 3399 case MAC_PROP_MAX_RX_RINGS_AVAIL: 3400 case MAC_PROP_MAX_TX_RINGS_AVAIL: 3401 case MAC_PROP_MAX_RXHWCLNT_AVAIL: 3402 case MAC_PROP_MAX_TXHWCLNT_AVAIL: 3403 if (perm != NULL) 3404 *perm = MAC_PROP_PERM_READ; 3405 return (0); 3406 3407 case MAC_PROP_RXRINGSRANGE: 3408 case MAC_PROP_TXRINGSRANGE: 3409 /* 3410 * Currently, we support range for RX and TX rings properties. 3411 * When we extend this support to maxbw, cpus and priority, 3412 * we should move this to mac_get_resources. 3413 * There is no default value for RX or TX rings. 3414 */ 3415 if ((mip->mi_state_flags & MIS_IS_VNIC) && 3416 mac_is_vnic_primary(mh)) { 3417 /* 3418 * We don't support setting rings for a VLAN 3419 * data link because it shares its ring with the 3420 * primary MAC client. 3421 */ 3422 if (perm != NULL) 3423 *perm = MAC_PROP_PERM_READ; 3424 if (range != NULL) 3425 range->mpr_count = 0; 3426 } else if (range != NULL) { 3427 if (mip->mi_state_flags & MIS_IS_VNIC) 3428 mh = mac_get_lower_mac_handle(mh); 3429 mip = (mac_impl_t *)mh; 3430 if ((id == MAC_PROP_RXRINGSRANGE && 3431 mip->mi_rx_group_type == MAC_GROUP_TYPE_STATIC) || 3432 (id == MAC_PROP_TXRINGSRANGE && 3433 mip->mi_tx_group_type == MAC_GROUP_TYPE_STATIC)) { 3434 if (id == MAC_PROP_RXRINGSRANGE) { 3435 if ((mac_rxhwlnksavail_get(mh) + 3436 mac_rxhwlnksrsvd_get(mh)) <= 1) { 3437 /* 3438 * doesn't support groups or 3439 * rings 3440 */ 3441 range->mpr_count = 0; 3442 } else { 3443 /* 3444 * supports specifying groups, 3445 * but not rings 3446 */ 3447 _mac_set_range(range, 0, 0); 3448 } 3449 } else { 3450 if ((mac_txhwlnksavail_get(mh) + 3451 mac_txhwlnksrsvd_get(mh)) <= 1) { 3452 /* 3453 * doesn't support groups or 3454 * rings 3455 */ 3456 range->mpr_count = 0; 3457 } else { 3458 /* 3459 * supports specifying groups, 3460 * but not rings 3461 */ 3462 _mac_set_range(range, 0, 0); 3463 } 3464 } 3465 } else { 3466 max = id == MAC_PROP_RXRINGSRANGE ? 3467 mac_rxavail_get(mh) + mac_rxrsvd_get(mh) : 3468 mac_txavail_get(mh) + mac_txrsvd_get(mh); 3469 if (max <= 1) { 3470 /* 3471 * doesn't support groups or 3472 * rings 3473 */ 3474 range->mpr_count = 0; 3475 } else { 3476 /* 3477 * -1 because we have to leave out the 3478 * default ring. 3479 */ 3480 _mac_set_range(range, 1, max - 1); 3481 } 3482 } 3483 } 3484 return (0); 3485 3486 case MAC_PROP_STATUS: 3487 if (perm != NULL) 3488 *perm = MAC_PROP_PERM_READ; 3489 return (0); 3490 } 3491 3492 /* 3493 * Get the property info from the driver if it implements the 3494 * property info entry point. 3495 */ 3496 bzero(&state, sizeof (state)); 3497 3498 if (mip->mi_callbacks->mc_callbacks & MC_PROPINFO) { 3499 state.pr_default = default_val; 3500 state.pr_default_size = default_size; 3501 3502 /* 3503 * The caller specifies the maximum number of ranges 3504 * it can accomodate using mpr_count. We don't touch 3505 * this value until the driver returns from its 3506 * mc_propinfo() callback, and ensure we don't exceed 3507 * this number of range as the driver defines 3508 * supported range from its mc_propinfo(). 3509 * 3510 * pr_range_cur_count keeps track of how many ranges 3511 * were defined by the driver from its mc_propinfo() 3512 * entry point. 3513 * 3514 * On exit, the user-specified range mpr_count returns 3515 * the number of ranges specified by the driver on 3516 * success, or the number of ranges it wanted to 3517 * define if that number of ranges could not be 3518 * accomodated by the specified range structure. In 3519 * the latter case, the caller will be able to 3520 * allocate a larger range structure, and query the 3521 * property again. 3522 */ 3523 state.pr_range_cur_count = 0; 3524 state.pr_range = range; 3525 3526 mip->mi_callbacks->mc_propinfo(mip->mi_driver, name, id, 3527 (mac_prop_info_handle_t)&state); 3528 3529 if (state.pr_flags & MAC_PROP_INFO_RANGE) 3530 range->mpr_count = state.pr_range_cur_count; 3531 3532 /* 3533 * The operation could fail if the buffer supplied by 3534 * the user was too small for the range or default 3535 * value of the property. 3536 */ 3537 if (state.pr_errno != 0) 3538 return (state.pr_errno); 3539 3540 if (perm != NULL && state.pr_flags & MAC_PROP_INFO_PERM) 3541 *perm = state.pr_perm; 3542 } 3543 3544 /* 3545 * The MAC layer may want to provide default values or allowed 3546 * ranges for properties if the driver does not provide a 3547 * property info entry point, or that entry point exists, but 3548 * it did not provide a default value or allowed ranges for 3549 * that property. 3550 */ 3551 switch (id) { 3552 case MAC_PROP_MTU: { 3553 uint32_t sdu; 3554 3555 mac_sdu_get2(mh, NULL, &sdu, NULL); 3556 3557 if (range != NULL && !(state.pr_flags & 3558 MAC_PROP_INFO_RANGE)) { 3559 /* MTU range */ 3560 _mac_set_range(range, sdu, sdu); 3561 } 3562 3563 if (default_val != NULL && !(state.pr_flags & 3564 MAC_PROP_INFO_DEFAULT)) { 3565 if (mip->mi_info.mi_media == DL_ETHER) 3566 sdu = ETHERMTU; 3567 /* default MTU value */ 3568 bcopy(&sdu, default_val, sizeof (sdu)); 3569 } 3570 } 3571 } 3572 3573 return (0); 3574 } 3575 3576 int 3577 mac_fastpath_disable(mac_handle_t mh) 3578 { 3579 mac_impl_t *mip = (mac_impl_t *)mh; 3580 3581 if ((mip->mi_state_flags & MIS_LEGACY) == 0) 3582 return (0); 3583 3584 return (mip->mi_capab_legacy.ml_fastpath_disable(mip->mi_driver)); 3585 } 3586 3587 void 3588 mac_fastpath_enable(mac_handle_t mh) 3589 { 3590 mac_impl_t *mip = (mac_impl_t *)mh; 3591 3592 if ((mip->mi_state_flags & MIS_LEGACY) == 0) 3593 return; 3594 3595 mip->mi_capab_legacy.ml_fastpath_enable(mip->mi_driver); 3596 } 3597 3598 void 3599 mac_register_priv_prop(mac_impl_t *mip, char **priv_props) 3600 { 3601 uint_t nprops, i; 3602 3603 if (priv_props == NULL) 3604 return; 3605 3606 nprops = 0; 3607 while (priv_props[nprops] != NULL) 3608 nprops++; 3609 if (nprops == 0) 3610 return; 3611 3612 3613 mip->mi_priv_prop = kmem_zalloc(nprops * sizeof (char *), KM_SLEEP); 3614 3615 for (i = 0; i < nprops; i++) { 3616 mip->mi_priv_prop[i] = kmem_zalloc(MAXLINKPROPNAME, KM_SLEEP); 3617 (void) strlcpy(mip->mi_priv_prop[i], priv_props[i], 3618 MAXLINKPROPNAME); 3619 } 3620 3621 mip->mi_priv_prop_count = nprops; 3622 } 3623 3624 void 3625 mac_unregister_priv_prop(mac_impl_t *mip) 3626 { 3627 uint_t i; 3628 3629 if (mip->mi_priv_prop_count == 0) { 3630 ASSERT(mip->mi_priv_prop == NULL); 3631 return; 3632 } 3633 3634 for (i = 0; i < mip->mi_priv_prop_count; i++) 3635 kmem_free(mip->mi_priv_prop[i], MAXLINKPROPNAME); 3636 kmem_free(mip->mi_priv_prop, mip->mi_priv_prop_count * 3637 sizeof (char *)); 3638 3639 mip->mi_priv_prop = NULL; 3640 mip->mi_priv_prop_count = 0; 3641 } 3642 3643 /* 3644 * mac_ring_t 'mr' macros. Some rogue drivers may access ring structure 3645 * (by invoking mac_rx()) even after processing mac_stop_ring(). In such 3646 * cases if MAC free's the ring structure after mac_stop_ring(), any 3647 * illegal access to the ring structure coming from the driver will panic 3648 * the system. In order to protect the system from such inadverent access, 3649 * we maintain a cache of rings in the mac_impl_t after they get free'd up. 3650 * When packets are received on free'd up rings, MAC (through the generation 3651 * count mechanism) will drop such packets. 3652 */ 3653 static mac_ring_t * 3654 mac_ring_alloc(mac_impl_t *mip) 3655 { 3656 mac_ring_t *ring; 3657 3658 mutex_enter(&mip->mi_ring_lock); 3659 if (mip->mi_ring_freelist != NULL) { 3660 ring = mip->mi_ring_freelist; 3661 mip->mi_ring_freelist = ring->mr_next; 3662 bzero(ring, sizeof (mac_ring_t)); 3663 mutex_exit(&mip->mi_ring_lock); 3664 } else { 3665 mutex_exit(&mip->mi_ring_lock); 3666 ring = kmem_cache_alloc(mac_ring_cache, KM_SLEEP); 3667 } 3668 ASSERT((ring != NULL) && (ring->mr_state == MR_FREE)); 3669 return (ring); 3670 } 3671 3672 static void 3673 mac_ring_free(mac_impl_t *mip, mac_ring_t *ring) 3674 { 3675 ASSERT(ring->mr_state == MR_FREE); 3676 3677 mutex_enter(&mip->mi_ring_lock); 3678 ring->mr_state = MR_FREE; 3679 ring->mr_flag = 0; 3680 ring->mr_next = mip->mi_ring_freelist; 3681 ring->mr_mip = NULL; 3682 mip->mi_ring_freelist = ring; 3683 mac_ring_stat_delete(ring); 3684 mutex_exit(&mip->mi_ring_lock); 3685 } 3686 3687 static void 3688 mac_ring_freeall(mac_impl_t *mip) 3689 { 3690 mac_ring_t *ring_next; 3691 mutex_enter(&mip->mi_ring_lock); 3692 mac_ring_t *ring = mip->mi_ring_freelist; 3693 while (ring != NULL) { 3694 ring_next = ring->mr_next; 3695 kmem_cache_free(mac_ring_cache, ring); 3696 ring = ring_next; 3697 } 3698 mip->mi_ring_freelist = NULL; 3699 mutex_exit(&mip->mi_ring_lock); 3700 } 3701 3702 int 3703 mac_start_ring(mac_ring_t *ring) 3704 { 3705 int rv = 0; 3706 3707 ASSERT(ring->mr_state == MR_FREE); 3708 3709 if (ring->mr_start != NULL) { 3710 rv = ring->mr_start(ring->mr_driver, ring->mr_gen_num); 3711 if (rv != 0) 3712 return (rv); 3713 } 3714 3715 ring->mr_state = MR_INUSE; 3716 return (rv); 3717 } 3718 3719 void 3720 mac_stop_ring(mac_ring_t *ring) 3721 { 3722 ASSERT(ring->mr_state == MR_INUSE); 3723 3724 if (ring->mr_stop != NULL) 3725 ring->mr_stop(ring->mr_driver); 3726 3727 ring->mr_state = MR_FREE; 3728 3729 /* 3730 * Increment the ring generation number for this ring. 3731 */ 3732 ring->mr_gen_num++; 3733 } 3734 3735 int 3736 mac_start_group(mac_group_t *group) 3737 { 3738 int rv = 0; 3739 3740 if (group->mrg_start != NULL) 3741 rv = group->mrg_start(group->mrg_driver); 3742 3743 return (rv); 3744 } 3745 3746 void 3747 mac_stop_group(mac_group_t *group) 3748 { 3749 if (group->mrg_stop != NULL) 3750 group->mrg_stop(group->mrg_driver); 3751 } 3752 3753 /* 3754 * Called from mac_start() on the default Rx group. Broadcast and multicast 3755 * packets are received only on the default group. Hence the default group 3756 * needs to be up even if the primary client is not up, for the other groups 3757 * to be functional. We do this by calling this function at mac_start time 3758 * itself. However the broadcast packets that are received can't make their 3759 * way beyond mac_rx until a mac client creates a broadcast flow. 3760 */ 3761 static int 3762 mac_start_group_and_rings(mac_group_t *group) 3763 { 3764 mac_ring_t *ring; 3765 int rv = 0; 3766 3767 ASSERT(group->mrg_state == MAC_GROUP_STATE_REGISTERED); 3768 if ((rv = mac_start_group(group)) != 0) 3769 return (rv); 3770 3771 for (ring = group->mrg_rings; ring != NULL; ring = ring->mr_next) { 3772 ASSERT(ring->mr_state == MR_FREE); 3773 if ((rv = mac_start_ring(ring)) != 0) 3774 goto error; 3775 ring->mr_classify_type = MAC_SW_CLASSIFIER; 3776 } 3777 return (0); 3778 3779 error: 3780 mac_stop_group_and_rings(group); 3781 return (rv); 3782 } 3783 3784 /* Called from mac_stop on the default Rx group */ 3785 static void 3786 mac_stop_group_and_rings(mac_group_t *group) 3787 { 3788 mac_ring_t *ring; 3789 3790 for (ring = group->mrg_rings; ring != NULL; ring = ring->mr_next) { 3791 if (ring->mr_state != MR_FREE) { 3792 mac_stop_ring(ring); 3793 ring->mr_flag = 0; 3794 ring->mr_classify_type = MAC_NO_CLASSIFIER; 3795 } 3796 } 3797 mac_stop_group(group); 3798 } 3799 3800 3801 static mac_ring_t * 3802 mac_init_ring(mac_impl_t *mip, mac_group_t *group, int index, 3803 mac_capab_rings_t *cap_rings) 3804 { 3805 mac_ring_t *ring, *rnext; 3806 mac_ring_info_t ring_info; 3807 ddi_intr_handle_t ddi_handle; 3808 3809 ring = mac_ring_alloc(mip); 3810 3811 /* Prepare basic information of ring */ 3812 3813 /* 3814 * Ring index is numbered to be unique across a particular device. 3815 * Ring index computation makes following assumptions: 3816 * - For drivers with static grouping (e.g. ixgbe, bge), 3817 * ring index exchanged with the driver (e.g. during mr_rget) 3818 * is unique only across the group the ring belongs to. 3819 * - Drivers with dynamic grouping (e.g. nxge), start 3820 * with single group (mrg_index = 0). 3821 */ 3822 ring->mr_index = group->mrg_index * group->mrg_info.mgi_count + index; 3823 ring->mr_type = group->mrg_type; 3824 ring->mr_gh = (mac_group_handle_t)group; 3825 3826 /* Insert the new ring to the list. */ 3827 ring->mr_next = group->mrg_rings; 3828 group->mrg_rings = ring; 3829 3830 /* Zero to reuse the info data structure */ 3831 bzero(&ring_info, sizeof (ring_info)); 3832 3833 /* Query ring information from driver */ 3834 cap_rings->mr_rget(mip->mi_driver, group->mrg_type, group->mrg_index, 3835 index, &ring_info, (mac_ring_handle_t)ring); 3836 3837 ring->mr_info = ring_info; 3838 3839 /* 3840 * The interrupt handle could be shared among multiple rings. 3841 * Thus if there is a bunch of rings that are sharing an 3842 * interrupt, then only one ring among the bunch will be made 3843 * available for interrupt re-targeting; the rest will have 3844 * ddi_shared flag set to TRUE and would not be available for 3845 * be interrupt re-targeting. 3846 */ 3847 if ((ddi_handle = ring_info.mri_intr.mi_ddi_handle) != NULL) { 3848 rnext = ring->mr_next; 3849 while (rnext != NULL) { 3850 if (rnext->mr_info.mri_intr.mi_ddi_handle == 3851 ddi_handle) { 3852 /* 3853 * If default ring (mr_index == 0) is part 3854 * of a group of rings sharing an 3855 * interrupt, then set ddi_shared flag for 3856 * the default ring and give another ring 3857 * the chance to be re-targeted. 3858 */ 3859 if (rnext->mr_index == 0 && 3860 !rnext->mr_info.mri_intr.mi_ddi_shared) { 3861 rnext->mr_info.mri_intr.mi_ddi_shared = 3862 B_TRUE; 3863 } else { 3864 ring->mr_info.mri_intr.mi_ddi_shared = 3865 B_TRUE; 3866 } 3867 break; 3868 } 3869 rnext = rnext->mr_next; 3870 } 3871 /* 3872 * If rnext is NULL, then no matching ddi_handle was found. 3873 * Rx rings get registered first. So if this is a Tx ring, 3874 * then go through all the Rx rings and see if there is a 3875 * matching ddi handle. 3876 */ 3877 if (rnext == NULL && ring->mr_type == MAC_RING_TYPE_TX) { 3878 mac_compare_ddi_handle(mip->mi_rx_groups, 3879 mip->mi_rx_group_count, ring); 3880 } 3881 } 3882 3883 /* Update ring's status */ 3884 ring->mr_state = MR_FREE; 3885 ring->mr_flag = 0; 3886 3887 /* Update the ring count of the group */ 3888 group->mrg_cur_count++; 3889 3890 /* Create per ring kstats */ 3891 if (ring->mr_stat != NULL) { 3892 ring->mr_mip = mip; 3893 mac_ring_stat_create(ring); 3894 } 3895 3896 return (ring); 3897 } 3898 3899 /* 3900 * Rings are chained together for easy regrouping. 3901 */ 3902 static void 3903 mac_init_group(mac_impl_t *mip, mac_group_t *group, int size, 3904 mac_capab_rings_t *cap_rings) 3905 { 3906 int index; 3907 3908 /* 3909 * Initialize all ring members of this group. Size of zero will not 3910 * enter the loop, so it's safe for initializing an empty group. 3911 */ 3912 for (index = size - 1; index >= 0; index--) 3913 (void) mac_init_ring(mip, group, index, cap_rings); 3914 } 3915 3916 int 3917 mac_init_rings(mac_impl_t *mip, mac_ring_type_t rtype) 3918 { 3919 mac_capab_rings_t *cap_rings; 3920 mac_group_t *group; 3921 mac_group_t *groups; 3922 mac_group_info_t group_info; 3923 uint_t group_free = 0; 3924 uint_t ring_left; 3925 mac_ring_t *ring; 3926 int g; 3927 int err = 0; 3928 uint_t grpcnt; 3929 boolean_t pseudo_txgrp = B_FALSE; 3930 3931 switch (rtype) { 3932 case MAC_RING_TYPE_RX: 3933 ASSERT(mip->mi_rx_groups == NULL); 3934 3935 cap_rings = &mip->mi_rx_rings_cap; 3936 cap_rings->mr_type = MAC_RING_TYPE_RX; 3937 break; 3938 case MAC_RING_TYPE_TX: 3939 ASSERT(mip->mi_tx_groups == NULL); 3940 3941 cap_rings = &mip->mi_tx_rings_cap; 3942 cap_rings->mr_type = MAC_RING_TYPE_TX; 3943 break; 3944 default: 3945 ASSERT(B_FALSE); 3946 } 3947 3948 if (!i_mac_capab_get((mac_handle_t)mip, MAC_CAPAB_RINGS, cap_rings)) 3949 return (0); 3950 grpcnt = cap_rings->mr_gnum; 3951 3952 /* 3953 * If we have multiple TX rings, but only one TX group, we can 3954 * create pseudo TX groups (one per TX ring) in the MAC layer, 3955 * except for an aggr. For an aggr currently we maintain only 3956 * one group with all the rings (for all its ports), going 3957 * forwards we might change this. 3958 */ 3959 if (rtype == MAC_RING_TYPE_TX && 3960 cap_rings->mr_gnum == 0 && cap_rings->mr_rnum > 0 && 3961 (mip->mi_state_flags & MIS_IS_AGGR) == 0) { 3962 /* 3963 * The -1 here is because we create a default TX group 3964 * with all the rings in it. 3965 */ 3966 grpcnt = cap_rings->mr_rnum - 1; 3967 pseudo_txgrp = B_TRUE; 3968 } 3969 3970 /* 3971 * Allocate a contiguous buffer for all groups. 3972 */ 3973 groups = kmem_zalloc(sizeof (mac_group_t) * (grpcnt+ 1), KM_SLEEP); 3974 3975 ring_left = cap_rings->mr_rnum; 3976 3977 /* 3978 * Get all ring groups if any, and get their ring members 3979 * if any. 3980 */ 3981 for (g = 0; g < grpcnt; g++) { 3982 group = groups + g; 3983 3984 /* Prepare basic information of the group */ 3985 group->mrg_index = g; 3986 group->mrg_type = rtype; 3987 group->mrg_state = MAC_GROUP_STATE_UNINIT; 3988 group->mrg_mh = (mac_handle_t)mip; 3989 group->mrg_next = group + 1; 3990 3991 /* Zero to reuse the info data structure */ 3992 bzero(&group_info, sizeof (group_info)); 3993 3994 if (pseudo_txgrp) { 3995 /* 3996 * This is a pseudo group that we created, apart 3997 * from setting the state there is nothing to be 3998 * done. 3999 */ 4000 group->mrg_state = MAC_GROUP_STATE_REGISTERED; 4001 group_free++; 4002 continue; 4003 } 4004 /* Query group information from driver */ 4005 cap_rings->mr_gget(mip->mi_driver, rtype, g, &group_info, 4006 (mac_group_handle_t)group); 4007 4008 switch (cap_rings->mr_group_type) { 4009 case MAC_GROUP_TYPE_DYNAMIC: 4010 if (cap_rings->mr_gaddring == NULL || 4011 cap_rings->mr_gremring == NULL) { 4012 DTRACE_PROBE3( 4013 mac__init__rings_no_addremring, 4014 char *, mip->mi_name, 4015 mac_group_add_ring_t, 4016 cap_rings->mr_gaddring, 4017 mac_group_add_ring_t, 4018 cap_rings->mr_gremring); 4019 err = EINVAL; 4020 goto bail; 4021 } 4022 4023 switch (rtype) { 4024 case MAC_RING_TYPE_RX: 4025 /* 4026 * The first RX group must have non-zero 4027 * rings, and the following groups must 4028 * have zero rings. 4029 */ 4030 if (g == 0 && group_info.mgi_count == 0) { 4031 DTRACE_PROBE1( 4032 mac__init__rings__rx__def__zero, 4033 char *, mip->mi_name); 4034 err = EINVAL; 4035 goto bail; 4036 } 4037 if (g > 0 && group_info.mgi_count != 0) { 4038 DTRACE_PROBE3( 4039 mac__init__rings__rx__nonzero, 4040 char *, mip->mi_name, 4041 int, g, int, group_info.mgi_count); 4042 err = EINVAL; 4043 goto bail; 4044 } 4045 break; 4046 case MAC_RING_TYPE_TX: 4047 /* 4048 * All TX ring groups must have zero rings. 4049 */ 4050 if (group_info.mgi_count != 0) { 4051 DTRACE_PROBE3( 4052 mac__init__rings__tx__nonzero, 4053 char *, mip->mi_name, 4054 int, g, int, group_info.mgi_count); 4055 err = EINVAL; 4056 goto bail; 4057 } 4058 break; 4059 } 4060 break; 4061 case MAC_GROUP_TYPE_STATIC: 4062 /* 4063 * Note that an empty group is allowed, e.g., an aggr 4064 * would start with an empty group. 4065 */ 4066 break; 4067 default: 4068 /* unknown group type */ 4069 DTRACE_PROBE2(mac__init__rings__unknown__type, 4070 char *, mip->mi_name, 4071 int, cap_rings->mr_group_type); 4072 err = EINVAL; 4073 goto bail; 4074 } 4075 4076 4077 /* 4078 * Driver must register group->mgi_addmac/remmac() for rx groups 4079 * to support multiple MAC addresses. 4080 */ 4081 if (rtype == MAC_RING_TYPE_RX && 4082 ((group_info.mgi_addmac == NULL) || 4083 (group_info.mgi_remmac == NULL))) { 4084 err = EINVAL; 4085 goto bail; 4086 } 4087 4088 /* Cache driver-supplied information */ 4089 group->mrg_info = group_info; 4090 4091 /* Update the group's status and group count. */ 4092 mac_set_group_state(group, MAC_GROUP_STATE_REGISTERED); 4093 group_free++; 4094 4095 group->mrg_rings = NULL; 4096 group->mrg_cur_count = 0; 4097 mac_init_group(mip, group, group_info.mgi_count, cap_rings); 4098 ring_left -= group_info.mgi_count; 4099 4100 /* The current group size should be equal to default value */ 4101 ASSERT(group->mrg_cur_count == group_info.mgi_count); 4102 } 4103 4104 /* Build up a dummy group for free resources as a pool */ 4105 group = groups + grpcnt; 4106 4107 /* Prepare basic information of the group */ 4108 group->mrg_index = -1; 4109 group->mrg_type = rtype; 4110 group->mrg_state = MAC_GROUP_STATE_UNINIT; 4111 group->mrg_mh = (mac_handle_t)mip; 4112 group->mrg_next = NULL; 4113 4114 /* 4115 * If there are ungrouped rings, allocate a continuous buffer for 4116 * remaining resources. 4117 */ 4118 if (ring_left != 0) { 4119 group->mrg_rings = NULL; 4120 group->mrg_cur_count = 0; 4121 mac_init_group(mip, group, ring_left, cap_rings); 4122 4123 /* The current group size should be equal to ring_left */ 4124 ASSERT(group->mrg_cur_count == ring_left); 4125 4126 ring_left = 0; 4127 4128 /* Update this group's status */ 4129 mac_set_group_state(group, MAC_GROUP_STATE_REGISTERED); 4130 } else 4131 group->mrg_rings = NULL; 4132 4133 ASSERT(ring_left == 0); 4134 4135 bail: 4136 4137 /* Cache other important information to finalize the initialization */ 4138 switch (rtype) { 4139 case MAC_RING_TYPE_RX: 4140 mip->mi_rx_group_type = cap_rings->mr_group_type; 4141 mip->mi_rx_group_count = cap_rings->mr_gnum; 4142 mip->mi_rx_groups = groups; 4143 mip->mi_rx_donor_grp = groups; 4144 if (mip->mi_rx_group_type == MAC_GROUP_TYPE_DYNAMIC) { 4145 /* 4146 * The default ring is reserved since it is 4147 * used for sending the broadcast etc. packets. 4148 */ 4149 mip->mi_rxrings_avail = 4150 mip->mi_rx_groups->mrg_cur_count - 1; 4151 mip->mi_rxrings_rsvd = 1; 4152 } 4153 /* 4154 * The default group cannot be reserved. It is used by 4155 * all the clients that do not have an exclusive group. 4156 */ 4157 mip->mi_rxhwclnt_avail = mip->mi_rx_group_count - 1; 4158 mip->mi_rxhwclnt_used = 1; 4159 break; 4160 case MAC_RING_TYPE_TX: 4161 mip->mi_tx_group_type = pseudo_txgrp ? MAC_GROUP_TYPE_DYNAMIC : 4162 cap_rings->mr_group_type; 4163 mip->mi_tx_group_count = grpcnt; 4164 mip->mi_tx_group_free = group_free; 4165 mip->mi_tx_groups = groups; 4166 4167 group = groups + grpcnt; 4168 ring = group->mrg_rings; 4169 /* 4170 * The ring can be NULL in the case of aggr. Aggr will 4171 * have an empty Tx group which will get populated 4172 * later when pseudo Tx rings are added after 4173 * mac_register() is done. 4174 */ 4175 if (ring == NULL) { 4176 ASSERT(mip->mi_state_flags & MIS_IS_AGGR); 4177 /* 4178 * pass the group to aggr so it can add Tx 4179 * rings to the group later. 4180 */ 4181 cap_rings->mr_gget(mip->mi_driver, rtype, 0, NULL, 4182 (mac_group_handle_t)group); 4183 /* 4184 * Even though there are no rings at this time 4185 * (rings will come later), set the group 4186 * state to registered. 4187 */ 4188 group->mrg_state = MAC_GROUP_STATE_REGISTERED; 4189 } else { 4190 /* 4191 * Ring 0 is used as the default one and it could be 4192 * assigned to a client as well. 4193 */ 4194 while ((ring->mr_index != 0) && (ring->mr_next != NULL)) 4195 ring = ring->mr_next; 4196 ASSERT(ring->mr_index == 0); 4197 mip->mi_default_tx_ring = (mac_ring_handle_t)ring; 4198 } 4199 if (mip->mi_tx_group_type == MAC_GROUP_TYPE_DYNAMIC) { 4200 mip->mi_txrings_avail = group->mrg_cur_count - 1; 4201 /* 4202 * The default ring cannot be reserved. 4203 */ 4204 mip->mi_txrings_rsvd = 1; 4205 } 4206 /* 4207 * The default group cannot be reserved. It will be shared 4208 * by clients that do not have an exclusive group. 4209 */ 4210 mip->mi_txhwclnt_avail = mip->mi_tx_group_count; 4211 mip->mi_txhwclnt_used = 1; 4212 break; 4213 default: 4214 ASSERT(B_FALSE); 4215 } 4216 4217 if (err != 0) 4218 mac_free_rings(mip, rtype); 4219 4220 return (err); 4221 } 4222 4223 /* 4224 * The ddi interrupt handle could be shared amoung rings. If so, compare 4225 * the new ring's ddi handle with the existing ones and set ddi_shared 4226 * flag. 4227 */ 4228 void 4229 mac_compare_ddi_handle(mac_group_t *groups, uint_t grpcnt, mac_ring_t *cring) 4230 { 4231 mac_group_t *group; 4232 mac_ring_t *ring; 4233 ddi_intr_handle_t ddi_handle; 4234 int g; 4235 4236 ddi_handle = cring->mr_info.mri_intr.mi_ddi_handle; 4237 for (g = 0; g < grpcnt; g++) { 4238 group = groups + g; 4239 for (ring = group->mrg_rings; ring != NULL; 4240 ring = ring->mr_next) { 4241 if (ring == cring) 4242 continue; 4243 if (ring->mr_info.mri_intr.mi_ddi_handle == 4244 ddi_handle) { 4245 if (cring->mr_type == MAC_RING_TYPE_RX && 4246 ring->mr_index == 0 && 4247 !ring->mr_info.mri_intr.mi_ddi_shared) { 4248 ring->mr_info.mri_intr.mi_ddi_shared = 4249 B_TRUE; 4250 } else { 4251 cring->mr_info.mri_intr.mi_ddi_shared = 4252 B_TRUE; 4253 } 4254 return; 4255 } 4256 } 4257 } 4258 } 4259 4260 /* 4261 * Called to free all groups of particular type (RX or TX). It's assumed that 4262 * no clients are using these groups. 4263 */ 4264 void 4265 mac_free_rings(mac_impl_t *mip, mac_ring_type_t rtype) 4266 { 4267 mac_group_t *group, *groups; 4268 uint_t group_count; 4269 4270 switch (rtype) { 4271 case MAC_RING_TYPE_RX: 4272 if (mip->mi_rx_groups == NULL) 4273 return; 4274 4275 groups = mip->mi_rx_groups; 4276 group_count = mip->mi_rx_group_count; 4277 4278 mip->mi_rx_groups = NULL; 4279 mip->mi_rx_donor_grp = NULL; 4280 mip->mi_rx_group_count = 0; 4281 break; 4282 case MAC_RING_TYPE_TX: 4283 ASSERT(mip->mi_tx_group_count == mip->mi_tx_group_free); 4284 4285 if (mip->mi_tx_groups == NULL) 4286 return; 4287 4288 groups = mip->mi_tx_groups; 4289 group_count = mip->mi_tx_group_count; 4290 4291 mip->mi_tx_groups = NULL; 4292 mip->mi_tx_group_count = 0; 4293 mip->mi_tx_group_free = 0; 4294 mip->mi_default_tx_ring = NULL; 4295 break; 4296 default: 4297 ASSERT(B_FALSE); 4298 } 4299 4300 for (group = groups; group != NULL; group = group->mrg_next) { 4301 mac_ring_t *ring; 4302 4303 if (group->mrg_cur_count == 0) 4304 continue; 4305 4306 ASSERT(group->mrg_rings != NULL); 4307 4308 while ((ring = group->mrg_rings) != NULL) { 4309 group->mrg_rings = ring->mr_next; 4310 mac_ring_free(mip, ring); 4311 } 4312 } 4313 4314 /* Free all the cached rings */ 4315 mac_ring_freeall(mip); 4316 /* Free the block of group data strutures */ 4317 kmem_free(groups, sizeof (mac_group_t) * (group_count + 1)); 4318 } 4319 4320 /* 4321 * Associate a MAC address with a receive group. 4322 * 4323 * The return value of this function should always be checked properly, because 4324 * any type of failure could cause unexpected results. A group can be added 4325 * or removed with a MAC address only after it has been reserved. Ideally, 4326 * a successful reservation always leads to calling mac_group_addmac() to 4327 * steer desired traffic. Failure of adding an unicast MAC address doesn't 4328 * always imply that the group is functioning abnormally. 4329 * 4330 * Currently this function is called everywhere, and it reflects assumptions 4331 * about MAC addresses in the implementation. CR 6735196. 4332 */ 4333 int 4334 mac_group_addmac(mac_group_t *group, const uint8_t *addr) 4335 { 4336 ASSERT(group->mrg_type == MAC_RING_TYPE_RX); 4337 ASSERT(group->mrg_info.mgi_addmac != NULL); 4338 4339 return (group->mrg_info.mgi_addmac(group->mrg_info.mgi_driver, addr)); 4340 } 4341 4342 /* 4343 * Remove the association between MAC address and receive group. 4344 */ 4345 int 4346 mac_group_remmac(mac_group_t *group, const uint8_t *addr) 4347 { 4348 ASSERT(group->mrg_type == MAC_RING_TYPE_RX); 4349 ASSERT(group->mrg_info.mgi_remmac != NULL); 4350 4351 return (group->mrg_info.mgi_remmac(group->mrg_info.mgi_driver, addr)); 4352 } 4353 4354 /* 4355 * This is the entry point for packets transmitted through the bridging code. 4356 * If no bridge is in place, MAC_RING_TX transmits using tx ring. The 'rh' 4357 * pointer may be NULL to select the default ring. 4358 */ 4359 mblk_t * 4360 mac_bridge_tx(mac_impl_t *mip, mac_ring_handle_t rh, mblk_t *mp) 4361 { 4362 mac_handle_t mh; 4363 4364 /* 4365 * Once we take a reference on the bridge link, the bridge 4366 * module itself can't unload, so the callback pointers are 4367 * stable. 4368 */ 4369 mutex_enter(&mip->mi_bridge_lock); 4370 if ((mh = mip->mi_bridge_link) != NULL) 4371 mac_bridge_ref_cb(mh, B_TRUE); 4372 mutex_exit(&mip->mi_bridge_lock); 4373 if (mh == NULL) { 4374 MAC_RING_TX(mip, rh, mp, mp); 4375 } else { 4376 mp = mac_bridge_tx_cb(mh, rh, mp); 4377 mac_bridge_ref_cb(mh, B_FALSE); 4378 } 4379 4380 return (mp); 4381 } 4382 4383 /* 4384 * Find a ring from its index. 4385 */ 4386 mac_ring_handle_t 4387 mac_find_ring(mac_group_handle_t gh, int index) 4388 { 4389 mac_group_t *group = (mac_group_t *)gh; 4390 mac_ring_t *ring = group->mrg_rings; 4391 4392 for (ring = group->mrg_rings; ring != NULL; ring = ring->mr_next) 4393 if (ring->mr_index == index) 4394 break; 4395 4396 return ((mac_ring_handle_t)ring); 4397 } 4398 /* 4399 * Add a ring to an existing group. 4400 * 4401 * The ring must be either passed directly (for example if the ring 4402 * movement is initiated by the framework), or specified through a driver 4403 * index (for example when the ring is added by the driver. 4404 * 4405 * The caller needs to call mac_perim_enter() before calling this function. 4406 */ 4407 int 4408 i_mac_group_add_ring(mac_group_t *group, mac_ring_t *ring, int index) 4409 { 4410 mac_impl_t *mip = (mac_impl_t *)group->mrg_mh; 4411 mac_capab_rings_t *cap_rings; 4412 boolean_t driver_call = (ring == NULL); 4413 mac_group_type_t group_type; 4414 int ret = 0; 4415 flow_entry_t *flent; 4416 4417 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 4418 4419 switch (group->mrg_type) { 4420 case MAC_RING_TYPE_RX: 4421 cap_rings = &mip->mi_rx_rings_cap; 4422 group_type = mip->mi_rx_group_type; 4423 break; 4424 case MAC_RING_TYPE_TX: 4425 cap_rings = &mip->mi_tx_rings_cap; 4426 group_type = mip->mi_tx_group_type; 4427 break; 4428 default: 4429 ASSERT(B_FALSE); 4430 } 4431 4432 /* 4433 * There should be no ring with the same ring index in the target 4434 * group. 4435 */ 4436 ASSERT(mac_find_ring((mac_group_handle_t)group, 4437 driver_call ? index : ring->mr_index) == NULL); 4438 4439 if (driver_call) { 4440 /* 4441 * The function is called as a result of a request from 4442 * a driver to add a ring to an existing group, for example 4443 * from the aggregation driver. Allocate a new mac_ring_t 4444 * for that ring. 4445 */ 4446 ring = mac_init_ring(mip, group, index, cap_rings); 4447 ASSERT(group->mrg_state > MAC_GROUP_STATE_UNINIT); 4448 } else { 4449 /* 4450 * The function is called as a result of a MAC layer request 4451 * to add a ring to an existing group. In this case the 4452 * ring is being moved between groups, which requires 4453 * the underlying driver to support dynamic grouping, 4454 * and the mac_ring_t already exists. 4455 */ 4456 ASSERT(group_type == MAC_GROUP_TYPE_DYNAMIC); 4457 ASSERT(group->mrg_driver == NULL || 4458 cap_rings->mr_gaddring != NULL); 4459 ASSERT(ring->mr_gh == NULL); 4460 } 4461 4462 /* 4463 * At this point the ring should not be in use, and it should be 4464 * of the right for the target group. 4465 */ 4466 ASSERT(ring->mr_state < MR_INUSE); 4467 ASSERT(ring->mr_srs == NULL); 4468 ASSERT(ring->mr_type == group->mrg_type); 4469 4470 if (!driver_call) { 4471 /* 4472 * Add the driver level hardware ring if the process was not 4473 * initiated by the driver, and the target group is not the 4474 * group. 4475 */ 4476 if (group->mrg_driver != NULL) { 4477 cap_rings->mr_gaddring(group->mrg_driver, 4478 ring->mr_driver, ring->mr_type); 4479 } 4480 4481 /* 4482 * Insert the ring ahead existing rings. 4483 */ 4484 ring->mr_next = group->mrg_rings; 4485 group->mrg_rings = ring; 4486 ring->mr_gh = (mac_group_handle_t)group; 4487 group->mrg_cur_count++; 4488 } 4489 4490 /* 4491 * If the group has not been actively used, we're done. 4492 */ 4493 if (group->mrg_index != -1 && 4494 group->mrg_state < MAC_GROUP_STATE_RESERVED) 4495 return (0); 4496 4497 /* 4498 * Start the ring if needed. Failure causes to undo the grouping action. 4499 */ 4500 if (ring->mr_state != MR_INUSE) { 4501 if ((ret = mac_start_ring(ring)) != 0) { 4502 if (!driver_call) { 4503 cap_rings->mr_gremring(group->mrg_driver, 4504 ring->mr_driver, ring->mr_type); 4505 } 4506 group->mrg_cur_count--; 4507 group->mrg_rings = ring->mr_next; 4508 4509 ring->mr_gh = NULL; 4510 4511 if (driver_call) 4512 mac_ring_free(mip, ring); 4513 4514 return (ret); 4515 } 4516 } 4517 4518 /* 4519 * Set up SRS/SR according to the ring type. 4520 */ 4521 switch (ring->mr_type) { 4522 case MAC_RING_TYPE_RX: 4523 /* 4524 * Setup SRS on top of the new ring if the group is 4525 * reserved for someones exclusive use. 4526 */ 4527 if (group->mrg_state == MAC_GROUP_STATE_RESERVED) { 4528 mac_client_impl_t *mcip; 4529 4530 mcip = MAC_GROUP_ONLY_CLIENT(group); 4531 /* 4532 * Even though this group is reserved we migth still 4533 * have multiple clients, i.e a VLAN shares the 4534 * group with the primary mac client. 4535 */ 4536 if (mcip != NULL) { 4537 flent = mcip->mci_flent; 4538 ASSERT(flent->fe_rx_srs_cnt > 0); 4539 mac_rx_srs_group_setup(mcip, flent, SRST_LINK); 4540 mac_fanout_setup(mcip, flent, 4541 MCIP_RESOURCE_PROPS(mcip), mac_rx_deliver, 4542 mcip, NULL, NULL); 4543 } else { 4544 ring->mr_classify_type = MAC_SW_CLASSIFIER; 4545 } 4546 } 4547 break; 4548 case MAC_RING_TYPE_TX: 4549 { 4550 mac_grp_client_t *mgcp = group->mrg_clients; 4551 mac_client_impl_t *mcip; 4552 mac_soft_ring_set_t *mac_srs; 4553 mac_srs_tx_t *tx; 4554 4555 if (MAC_GROUP_NO_CLIENT(group)) { 4556 if (ring->mr_state == MR_INUSE) 4557 mac_stop_ring(ring); 4558 ring->mr_flag = 0; 4559 break; 4560 } 4561 /* 4562 * If the rings are being moved to a group that has 4563 * clients using it, then add the new rings to the 4564 * clients SRS. 4565 */ 4566 while (mgcp != NULL) { 4567 boolean_t is_aggr; 4568 4569 mcip = mgcp->mgc_client; 4570 flent = mcip->mci_flent; 4571 is_aggr = (mcip->mci_state_flags & MCIS_IS_AGGR); 4572 mac_srs = MCIP_TX_SRS(mcip); 4573 tx = &mac_srs->srs_tx; 4574 mac_tx_client_quiesce((mac_client_handle_t)mcip); 4575 /* 4576 * If we are growing from 1 to multiple rings. 4577 */ 4578 if (tx->st_mode == SRS_TX_BW || 4579 tx->st_mode == SRS_TX_SERIALIZE || 4580 tx->st_mode == SRS_TX_DEFAULT) { 4581 mac_ring_t *tx_ring = tx->st_arg2; 4582 4583 tx->st_arg2 = NULL; 4584 mac_tx_srs_stat_recreate(mac_srs, B_TRUE); 4585 mac_tx_srs_add_ring(mac_srs, tx_ring); 4586 if (mac_srs->srs_type & SRST_BW_CONTROL) { 4587 tx->st_mode = is_aggr ? SRS_TX_BW_AGGR : 4588 SRS_TX_BW_FANOUT; 4589 } else { 4590 tx->st_mode = is_aggr ? SRS_TX_AGGR : 4591 SRS_TX_FANOUT; 4592 } 4593 tx->st_func = mac_tx_get_func(tx->st_mode); 4594 } 4595 mac_tx_srs_add_ring(mac_srs, ring); 4596 mac_fanout_setup(mcip, flent, MCIP_RESOURCE_PROPS(mcip), 4597 mac_rx_deliver, mcip, NULL, NULL); 4598 mac_tx_client_restart((mac_client_handle_t)mcip); 4599 mgcp = mgcp->mgc_next; 4600 } 4601 break; 4602 } 4603 default: 4604 ASSERT(B_FALSE); 4605 } 4606 /* 4607 * For aggr, the default ring will be NULL to begin with. If it 4608 * is NULL, then pick the first ring that gets added as the 4609 * default ring. Any ring in an aggregation can be removed at 4610 * any time (by the user action of removing a link) and if the 4611 * current default ring gets removed, then a new one gets 4612 * picked (see i_mac_group_rem_ring()). 4613 */ 4614 if (mip->mi_state_flags & MIS_IS_AGGR && 4615 mip->mi_default_tx_ring == NULL && 4616 ring->mr_type == MAC_RING_TYPE_TX) { 4617 mip->mi_default_tx_ring = (mac_ring_handle_t)ring; 4618 } 4619 4620 MAC_RING_UNMARK(ring, MR_INCIPIENT); 4621 return (0); 4622 } 4623 4624 /* 4625 * Remove a ring from it's current group. MAC internal function for dynamic 4626 * grouping. 4627 * 4628 * The caller needs to call mac_perim_enter() before calling this function. 4629 */ 4630 void 4631 i_mac_group_rem_ring(mac_group_t *group, mac_ring_t *ring, 4632 boolean_t driver_call) 4633 { 4634 mac_impl_t *mip = (mac_impl_t *)group->mrg_mh; 4635 mac_capab_rings_t *cap_rings = NULL; 4636 mac_group_type_t group_type; 4637 4638 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 4639 4640 ASSERT(mac_find_ring((mac_group_handle_t)group, 4641 ring->mr_index) == (mac_ring_handle_t)ring); 4642 ASSERT((mac_group_t *)ring->mr_gh == group); 4643 ASSERT(ring->mr_type == group->mrg_type); 4644 4645 if (ring->mr_state == MR_INUSE) 4646 mac_stop_ring(ring); 4647 switch (ring->mr_type) { 4648 case MAC_RING_TYPE_RX: 4649 group_type = mip->mi_rx_group_type; 4650 cap_rings = &mip->mi_rx_rings_cap; 4651 4652 /* 4653 * Only hardware classified packets hold a reference to the 4654 * ring all the way up the Rx path. mac_rx_srs_remove() 4655 * will take care of quiescing the Rx path and removing the 4656 * SRS. The software classified path neither holds a reference 4657 * nor any association with the ring in mac_rx. 4658 */ 4659 if (ring->mr_srs != NULL) { 4660 mac_rx_srs_remove(ring->mr_srs); 4661 ring->mr_srs = NULL; 4662 } 4663 4664 break; 4665 case MAC_RING_TYPE_TX: 4666 { 4667 mac_grp_client_t *mgcp; 4668 mac_client_impl_t *mcip; 4669 mac_soft_ring_set_t *mac_srs; 4670 mac_srs_tx_t *tx; 4671 mac_ring_t *rem_ring; 4672 mac_group_t *defgrp; 4673 uint_t ring_info = 0; 4674 4675 /* 4676 * For TX this function is invoked in three 4677 * cases: 4678 * 4679 * 1) In the case of a failure during the 4680 * initial creation of a group when a share is 4681 * associated with a MAC client. So the SRS is not 4682 * yet setup, and will be setup later after the 4683 * group has been reserved and populated. 4684 * 4685 * 2) From mac_release_tx_group() when freeing 4686 * a TX SRS. 4687 * 4688 * 3) In the case of aggr, when a port gets removed, 4689 * the pseudo Tx rings that it exposed gets removed. 4690 * 4691 * In the first two cases the SRS and its soft 4692 * rings are already quiesced. 4693 */ 4694 if (driver_call) { 4695 mac_client_impl_t *mcip; 4696 mac_soft_ring_set_t *mac_srs; 4697 mac_soft_ring_t *sringp; 4698 mac_srs_tx_t *srs_tx; 4699 4700 if (mip->mi_state_flags & MIS_IS_AGGR && 4701 mip->mi_default_tx_ring == 4702 (mac_ring_handle_t)ring) { 4703 /* pick a new default Tx ring */ 4704 mip->mi_default_tx_ring = 4705 (group->mrg_rings != ring) ? 4706 (mac_ring_handle_t)group->mrg_rings : 4707 (mac_ring_handle_t)(ring->mr_next); 4708 } 4709 /* Presently only aggr case comes here */ 4710 if (group->mrg_state != MAC_GROUP_STATE_RESERVED) 4711 break; 4712 4713 mcip = MAC_GROUP_ONLY_CLIENT(group); 4714 ASSERT(mcip != NULL); 4715 ASSERT(mcip->mci_state_flags & MCIS_IS_AGGR); 4716 mac_srs = MCIP_TX_SRS(mcip); 4717 ASSERT(mac_srs->srs_tx.st_mode == SRS_TX_AGGR || 4718 mac_srs->srs_tx.st_mode == SRS_TX_BW_AGGR); 4719 srs_tx = &mac_srs->srs_tx; 4720 /* 4721 * Wakeup any callers blocked on this 4722 * Tx ring due to flow control. 4723 */ 4724 sringp = srs_tx->st_soft_rings[ring->mr_index]; 4725 ASSERT(sringp != NULL); 4726 mac_tx_invoke_callbacks(mcip, (mac_tx_cookie_t)sringp); 4727 mac_tx_client_quiesce((mac_client_handle_t)mcip); 4728 mac_tx_srs_del_ring(mac_srs, ring); 4729 mac_tx_client_restart((mac_client_handle_t)mcip); 4730 break; 4731 } 4732 ASSERT(ring != (mac_ring_t *)mip->mi_default_tx_ring); 4733 group_type = mip->mi_tx_group_type; 4734 cap_rings = &mip->mi_tx_rings_cap; 4735 /* 4736 * See if we need to take it out of the MAC clients using 4737 * this group 4738 */ 4739 if (MAC_GROUP_NO_CLIENT(group)) 4740 break; 4741 mgcp = group->mrg_clients; 4742 defgrp = MAC_DEFAULT_TX_GROUP(mip); 4743 while (mgcp != NULL) { 4744 mcip = mgcp->mgc_client; 4745 mac_srs = MCIP_TX_SRS(mcip); 4746 tx = &mac_srs->srs_tx; 4747 mac_tx_client_quiesce((mac_client_handle_t)mcip); 4748 /* 4749 * If we are here when removing rings from the 4750 * defgroup, mac_reserve_tx_ring would have 4751 * already deleted the ring from the MAC 4752 * clients in the group. 4753 */ 4754 if (group != defgrp) { 4755 mac_tx_invoke_callbacks(mcip, 4756 (mac_tx_cookie_t) 4757 mac_tx_srs_get_soft_ring(mac_srs, ring)); 4758 mac_tx_srs_del_ring(mac_srs, ring); 4759 } 4760 /* 4761 * Additionally, if we are left with only 4762 * one ring in the group after this, we need 4763 * to modify the mode etc. to. (We haven't 4764 * yet taken the ring out, so we check with 2). 4765 */ 4766 if (group->mrg_cur_count == 2) { 4767 if (ring->mr_next == NULL) 4768 rem_ring = group->mrg_rings; 4769 else 4770 rem_ring = ring->mr_next; 4771 mac_tx_invoke_callbacks(mcip, 4772 (mac_tx_cookie_t) 4773 mac_tx_srs_get_soft_ring(mac_srs, 4774 rem_ring)); 4775 mac_tx_srs_del_ring(mac_srs, rem_ring); 4776 if (rem_ring->mr_state != MR_INUSE) { 4777 (void) mac_start_ring(rem_ring); 4778 } 4779 tx->st_arg2 = (void *)rem_ring; 4780 mac_tx_srs_stat_recreate(mac_srs, B_FALSE); 4781 ring_info = mac_hwring_getinfo( 4782 (mac_ring_handle_t)rem_ring); 4783 /* 4784 * We are shrinking from multiple 4785 * to 1 ring. 4786 */ 4787 if (mac_srs->srs_type & SRST_BW_CONTROL) { 4788 tx->st_mode = SRS_TX_BW; 4789 } else if (mac_tx_serialize || 4790 (ring_info & MAC_RING_TX_SERIALIZE)) { 4791 tx->st_mode = SRS_TX_SERIALIZE; 4792 } else { 4793 tx->st_mode = SRS_TX_DEFAULT; 4794 } 4795 tx->st_func = mac_tx_get_func(tx->st_mode); 4796 } 4797 mac_tx_client_restart((mac_client_handle_t)mcip); 4798 mgcp = mgcp->mgc_next; 4799 } 4800 break; 4801 } 4802 default: 4803 ASSERT(B_FALSE); 4804 } 4805 4806 /* 4807 * Remove the ring from the group. 4808 */ 4809 if (ring == group->mrg_rings) 4810 group->mrg_rings = ring->mr_next; 4811 else { 4812 mac_ring_t *pre; 4813 4814 pre = group->mrg_rings; 4815 while (pre->mr_next != ring) 4816 pre = pre->mr_next; 4817 pre->mr_next = ring->mr_next; 4818 } 4819 group->mrg_cur_count--; 4820 4821 if (!driver_call) { 4822 ASSERT(group_type == MAC_GROUP_TYPE_DYNAMIC); 4823 ASSERT(group->mrg_driver == NULL || 4824 cap_rings->mr_gremring != NULL); 4825 4826 /* 4827 * Remove the driver level hardware ring. 4828 */ 4829 if (group->mrg_driver != NULL) { 4830 cap_rings->mr_gremring(group->mrg_driver, 4831 ring->mr_driver, ring->mr_type); 4832 } 4833 } 4834 4835 ring->mr_gh = NULL; 4836 if (driver_call) 4837 mac_ring_free(mip, ring); 4838 else 4839 ring->mr_flag = 0; 4840 } 4841 4842 /* 4843 * Move a ring to the target group. If needed, remove the ring from the group 4844 * that it currently belongs to. 4845 * 4846 * The caller need to enter MAC's perimeter by calling mac_perim_enter(). 4847 */ 4848 static int 4849 mac_group_mov_ring(mac_impl_t *mip, mac_group_t *d_group, mac_ring_t *ring) 4850 { 4851 mac_group_t *s_group = (mac_group_t *)ring->mr_gh; 4852 int rv; 4853 4854 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 4855 ASSERT(d_group != NULL); 4856 ASSERT(s_group->mrg_mh == d_group->mrg_mh); 4857 4858 if (s_group == d_group) 4859 return (0); 4860 4861 /* 4862 * Remove it from current group first. 4863 */ 4864 if (s_group != NULL) 4865 i_mac_group_rem_ring(s_group, ring, B_FALSE); 4866 4867 /* 4868 * Add it to the new group. 4869 */ 4870 rv = i_mac_group_add_ring(d_group, ring, 0); 4871 if (rv != 0) { 4872 /* 4873 * Failed to add ring back to source group. If 4874 * that fails, the ring is stuck in limbo, log message. 4875 */ 4876 if (i_mac_group_add_ring(s_group, ring, 0)) { 4877 cmn_err(CE_WARN, "%s: failed to move ring %p\n", 4878 mip->mi_name, (void *)ring); 4879 } 4880 } 4881 4882 return (rv); 4883 } 4884 4885 /* 4886 * Find a MAC address according to its value. 4887 */ 4888 mac_address_t * 4889 mac_find_macaddr(mac_impl_t *mip, uint8_t *mac_addr) 4890 { 4891 mac_address_t *map; 4892 4893 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 4894 4895 for (map = mip->mi_addresses; map != NULL; map = map->ma_next) { 4896 if (bcmp(mac_addr, map->ma_addr, map->ma_len) == 0) 4897 break; 4898 } 4899 4900 return (map); 4901 } 4902 4903 /* 4904 * Check whether the MAC address is shared by multiple clients. 4905 */ 4906 boolean_t 4907 mac_check_macaddr_shared(mac_address_t *map) 4908 { 4909 ASSERT(MAC_PERIM_HELD((mac_handle_t)map->ma_mip)); 4910 4911 return (map->ma_nusers > 1); 4912 } 4913 4914 /* 4915 * Remove the specified MAC address from the MAC address list and free it. 4916 */ 4917 static void 4918 mac_free_macaddr(mac_address_t *map) 4919 { 4920 mac_impl_t *mip = map->ma_mip; 4921 4922 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 4923 ASSERT(mip->mi_addresses != NULL); 4924 4925 map = mac_find_macaddr(mip, map->ma_addr); 4926 4927 ASSERT(map != NULL); 4928 ASSERT(map->ma_nusers == 0); 4929 4930 if (map == mip->mi_addresses) { 4931 mip->mi_addresses = map->ma_next; 4932 } else { 4933 mac_address_t *pre; 4934 4935 pre = mip->mi_addresses; 4936 while (pre->ma_next != map) 4937 pre = pre->ma_next; 4938 pre->ma_next = map->ma_next; 4939 } 4940 4941 kmem_free(map, sizeof (mac_address_t)); 4942 } 4943 4944 /* 4945 * Add a MAC address reference for a client. If the desired MAC address 4946 * exists, add a reference to it. Otherwise, add the new address by adding 4947 * it to a reserved group or setting promiscuous mode. Won't try different 4948 * group is the group is non-NULL, so the caller must explictly share 4949 * default group when needed. 4950 * 4951 * Note, the primary MAC address is initialized at registration time, so 4952 * to add it to default group only need to activate it if its reference 4953 * count is still zero. Also, some drivers may not have advertised RINGS 4954 * capability. 4955 */ 4956 int 4957 mac_add_macaddr(mac_impl_t *mip, mac_group_t *group, uint8_t *mac_addr, 4958 boolean_t use_hw) 4959 { 4960 mac_address_t *map; 4961 int err = 0; 4962 boolean_t allocated_map = B_FALSE; 4963 4964 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 4965 4966 map = mac_find_macaddr(mip, mac_addr); 4967 4968 /* 4969 * If the new MAC address has not been added. Allocate a new one 4970 * and set it up. 4971 */ 4972 if (map == NULL) { 4973 map = kmem_zalloc(sizeof (mac_address_t), KM_SLEEP); 4974 map->ma_len = mip->mi_type->mt_addr_length; 4975 bcopy(mac_addr, map->ma_addr, map->ma_len); 4976 map->ma_nusers = 0; 4977 map->ma_group = group; 4978 map->ma_mip = mip; 4979 4980 /* add the new MAC address to the head of the address list */ 4981 map->ma_next = mip->mi_addresses; 4982 mip->mi_addresses = map; 4983 4984 allocated_map = B_TRUE; 4985 } 4986 4987 ASSERT(map->ma_group == NULL || map->ma_group == group); 4988 if (map->ma_group == NULL) 4989 map->ma_group = group; 4990 4991 /* 4992 * If the MAC address is already in use, simply account for the 4993 * new client. 4994 */ 4995 if (map->ma_nusers++ > 0) 4996 return (0); 4997 4998 /* 4999 * Activate this MAC address by adding it to the reserved group. 5000 */ 5001 if (group != NULL) { 5002 err = mac_group_addmac(group, (const uint8_t *)mac_addr); 5003 if (err == 0) { 5004 map->ma_type = MAC_ADDRESS_TYPE_UNICAST_CLASSIFIED; 5005 return (0); 5006 } 5007 } 5008 5009 /* 5010 * The MAC address addition failed. If the client requires a 5011 * hardware classified MAC address, fail the operation. 5012 */ 5013 if (use_hw) { 5014 err = ENOSPC; 5015 goto bail; 5016 } 5017 5018 /* 5019 * Try promiscuous mode. 5020 * 5021 * For drivers that don't advertise RINGS capability, do 5022 * nothing for the primary address. 5023 */ 5024 if ((group == NULL) && 5025 (bcmp(map->ma_addr, mip->mi_addr, map->ma_len) == 0)) { 5026 map->ma_type = MAC_ADDRESS_TYPE_UNICAST_CLASSIFIED; 5027 return (0); 5028 } 5029 5030 /* 5031 * Enable promiscuous mode in order to receive traffic 5032 * to the new MAC address. 5033 */ 5034 if ((err = i_mac_promisc_set(mip, B_TRUE)) == 0) { 5035 map->ma_type = MAC_ADDRESS_TYPE_UNICAST_PROMISC; 5036 return (0); 5037 } 5038 5039 /* 5040 * Free the MAC address that could not be added. Don't free 5041 * a pre-existing address, it could have been the entry 5042 * for the primary MAC address which was pre-allocated by 5043 * mac_init_macaddr(), and which must remain on the list. 5044 */ 5045 bail: 5046 map->ma_nusers--; 5047 if (allocated_map) 5048 mac_free_macaddr(map); 5049 return (err); 5050 } 5051 5052 /* 5053 * Remove a reference to a MAC address. This may cause to remove the MAC 5054 * address from an associated group or to turn off promiscuous mode. 5055 * The caller needs to handle the failure properly. 5056 */ 5057 int 5058 mac_remove_macaddr(mac_address_t *map) 5059 { 5060 mac_impl_t *mip = map->ma_mip; 5061 int err = 0; 5062 5063 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 5064 5065 ASSERT(map == mac_find_macaddr(mip, map->ma_addr)); 5066 5067 /* 5068 * If it's not the last client using this MAC address, only update 5069 * the MAC clients count. 5070 */ 5071 if (--map->ma_nusers > 0) 5072 return (0); 5073 5074 /* 5075 * The MAC address is no longer used by any MAC client, so remove 5076 * it from its associated group, or turn off promiscuous mode 5077 * if it was enabled for the MAC address. 5078 */ 5079 switch (map->ma_type) { 5080 case MAC_ADDRESS_TYPE_UNICAST_CLASSIFIED: 5081 /* 5082 * Don't free the preset primary address for drivers that 5083 * don't advertise RINGS capability. 5084 */ 5085 if (map->ma_group == NULL) 5086 return (0); 5087 5088 err = mac_group_remmac(map->ma_group, map->ma_addr); 5089 if (err == 0) 5090 map->ma_group = NULL; 5091 break; 5092 case MAC_ADDRESS_TYPE_UNICAST_PROMISC: 5093 err = i_mac_promisc_set(mip, B_FALSE); 5094 break; 5095 default: 5096 ASSERT(B_FALSE); 5097 } 5098 5099 if (err != 0) 5100 return (err); 5101 5102 /* 5103 * We created MAC address for the primary one at registration, so we 5104 * won't free it here. mac_fini_macaddr() will take care of it. 5105 */ 5106 if (bcmp(map->ma_addr, mip->mi_addr, map->ma_len) != 0) 5107 mac_free_macaddr(map); 5108 5109 return (0); 5110 } 5111 5112 /* 5113 * Update an existing MAC address. The caller need to make sure that the new 5114 * value has not been used. 5115 */ 5116 int 5117 mac_update_macaddr(mac_address_t *map, uint8_t *mac_addr) 5118 { 5119 mac_impl_t *mip = map->ma_mip; 5120 int err = 0; 5121 5122 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 5123 ASSERT(mac_find_macaddr(mip, mac_addr) == NULL); 5124 5125 switch (map->ma_type) { 5126 case MAC_ADDRESS_TYPE_UNICAST_CLASSIFIED: 5127 /* 5128 * Update the primary address for drivers that are not 5129 * RINGS capable. 5130 */ 5131 if (mip->mi_rx_groups == NULL) { 5132 err = mip->mi_unicst(mip->mi_driver, (const uint8_t *) 5133 mac_addr); 5134 if (err != 0) 5135 return (err); 5136 break; 5137 } 5138 5139 /* 5140 * If this MAC address is not currently in use, 5141 * simply break out and update the value. 5142 */ 5143 if (map->ma_nusers == 0) 5144 break; 5145 5146 /* 5147 * Need to replace the MAC address associated with a group. 5148 */ 5149 err = mac_group_remmac(map->ma_group, map->ma_addr); 5150 if (err != 0) 5151 return (err); 5152 5153 err = mac_group_addmac(map->ma_group, mac_addr); 5154 5155 /* 5156 * Failure hints hardware error. The MAC layer needs to 5157 * have error notification facility to handle this. 5158 * Now, simply try to restore the value. 5159 */ 5160 if (err != 0) 5161 (void) mac_group_addmac(map->ma_group, map->ma_addr); 5162 5163 break; 5164 case MAC_ADDRESS_TYPE_UNICAST_PROMISC: 5165 /* 5166 * Need to do nothing more if in promiscuous mode. 5167 */ 5168 break; 5169 default: 5170 ASSERT(B_FALSE); 5171 } 5172 5173 /* 5174 * Successfully replaced the MAC address. 5175 */ 5176 if (err == 0) 5177 bcopy(mac_addr, map->ma_addr, map->ma_len); 5178 5179 return (err); 5180 } 5181 5182 /* 5183 * Freshen the MAC address with new value. Its caller must have updated the 5184 * hardware MAC address before calling this function. 5185 * This funcitons is supposed to be used to handle the MAC address change 5186 * notification from underlying drivers. 5187 */ 5188 void 5189 mac_freshen_macaddr(mac_address_t *map, uint8_t *mac_addr) 5190 { 5191 mac_impl_t *mip = map->ma_mip; 5192 5193 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 5194 ASSERT(mac_find_macaddr(mip, mac_addr) == NULL); 5195 5196 /* 5197 * Freshen the MAC address with new value. 5198 */ 5199 bcopy(mac_addr, map->ma_addr, map->ma_len); 5200 bcopy(mac_addr, mip->mi_addr, map->ma_len); 5201 5202 /* 5203 * Update all MAC clients that share this MAC address. 5204 */ 5205 mac_unicast_update_clients(mip, map); 5206 } 5207 5208 /* 5209 * Set up the primary MAC address. 5210 */ 5211 void 5212 mac_init_macaddr(mac_impl_t *mip) 5213 { 5214 mac_address_t *map; 5215 5216 /* 5217 * The reference count is initialized to zero, until it's really 5218 * activated. 5219 */ 5220 map = kmem_zalloc(sizeof (mac_address_t), KM_SLEEP); 5221 map->ma_len = mip->mi_type->mt_addr_length; 5222 bcopy(mip->mi_addr, map->ma_addr, map->ma_len); 5223 5224 /* 5225 * If driver advertises RINGS capability, it shouldn't have initialized 5226 * its primary MAC address. For other drivers, including VNIC, the 5227 * primary address must work after registration. 5228 */ 5229 if (mip->mi_rx_groups == NULL) 5230 map->ma_type = MAC_ADDRESS_TYPE_UNICAST_CLASSIFIED; 5231 5232 map->ma_mip = mip; 5233 5234 mip->mi_addresses = map; 5235 } 5236 5237 /* 5238 * Clean up the primary MAC address. Note, only one primary MAC address 5239 * is allowed. All other MAC addresses must have been freed appropriately. 5240 */ 5241 void 5242 mac_fini_macaddr(mac_impl_t *mip) 5243 { 5244 mac_address_t *map = mip->mi_addresses; 5245 5246 if (map == NULL) 5247 return; 5248 5249 /* 5250 * If mi_addresses is initialized, there should be exactly one 5251 * entry left on the list with no users. 5252 */ 5253 ASSERT(map->ma_nusers == 0); 5254 ASSERT(map->ma_next == NULL); 5255 5256 kmem_free(map, sizeof (mac_address_t)); 5257 mip->mi_addresses = NULL; 5258 } 5259 5260 /* 5261 * Logging related functions. 5262 * 5263 * Note that Kernel statistics have been extended to maintain fine 5264 * granularity of statistics viz. hardware lane, software lane, fanout 5265 * stats etc. However, extended accounting continues to support only 5266 * aggregate statistics like before. 5267 */ 5268 5269 /* Write the flow description to a netinfo_t record */ 5270 static netinfo_t * 5271 mac_write_flow_desc(flow_entry_t *flent, mac_client_impl_t *mcip) 5272 { 5273 netinfo_t *ninfo; 5274 net_desc_t *ndesc; 5275 flow_desc_t *fdesc; 5276 mac_resource_props_t *mrp; 5277 5278 ninfo = kmem_zalloc(sizeof (netinfo_t), KM_NOSLEEP); 5279 if (ninfo == NULL) 5280 return (NULL); 5281 ndesc = kmem_zalloc(sizeof (net_desc_t), KM_NOSLEEP); 5282 if (ndesc == NULL) { 5283 kmem_free(ninfo, sizeof (netinfo_t)); 5284 return (NULL); 5285 } 5286 5287 /* 5288 * Grab the fe_lock to see a self-consistent fe_flow_desc. 5289 * Updates to the fe_flow_desc are done under the fe_lock 5290 */ 5291 mutex_enter(&flent->fe_lock); 5292 fdesc = &flent->fe_flow_desc; 5293 mrp = &flent->fe_resource_props; 5294 5295 ndesc->nd_name = flent->fe_flow_name; 5296 ndesc->nd_devname = mcip->mci_name; 5297 bcopy(fdesc->fd_src_mac, ndesc->nd_ehost, ETHERADDRL); 5298 bcopy(fdesc->fd_dst_mac, ndesc->nd_edest, ETHERADDRL); 5299 ndesc->nd_sap = htonl(fdesc->fd_sap); 5300 ndesc->nd_isv4 = (uint8_t)fdesc->fd_ipversion == IPV4_VERSION; 5301 ndesc->nd_bw_limit = mrp->mrp_maxbw; 5302 if (ndesc->nd_isv4) { 5303 ndesc->nd_saddr[3] = htonl(fdesc->fd_local_addr.s6_addr32[3]); 5304 ndesc->nd_daddr[3] = htonl(fdesc->fd_remote_addr.s6_addr32[3]); 5305 } else { 5306 bcopy(&fdesc->fd_local_addr, ndesc->nd_saddr, IPV6_ADDR_LEN); 5307 bcopy(&fdesc->fd_remote_addr, ndesc->nd_daddr, IPV6_ADDR_LEN); 5308 } 5309 ndesc->nd_sport = htons(fdesc->fd_local_port); 5310 ndesc->nd_dport = htons(fdesc->fd_remote_port); 5311 ndesc->nd_protocol = (uint8_t)fdesc->fd_protocol; 5312 mutex_exit(&flent->fe_lock); 5313 5314 ninfo->ni_record = ndesc; 5315 ninfo->ni_size = sizeof (net_desc_t); 5316 ninfo->ni_type = EX_NET_FLDESC_REC; 5317 5318 return (ninfo); 5319 } 5320 5321 /* Write the flow statistics to a netinfo_t record */ 5322 static netinfo_t * 5323 mac_write_flow_stats(flow_entry_t *flent) 5324 { 5325 netinfo_t *ninfo; 5326 net_stat_t *nstat; 5327 mac_soft_ring_set_t *mac_srs; 5328 mac_rx_stats_t *mac_rx_stat; 5329 mac_tx_stats_t *mac_tx_stat; 5330 int i; 5331 5332 ninfo = kmem_zalloc(sizeof (netinfo_t), KM_NOSLEEP); 5333 if (ninfo == NULL) 5334 return (NULL); 5335 nstat = kmem_zalloc(sizeof (net_stat_t), KM_NOSLEEP); 5336 if (nstat == NULL) { 5337 kmem_free(ninfo, sizeof (netinfo_t)); 5338 return (NULL); 5339 } 5340 5341 nstat->ns_name = flent->fe_flow_name; 5342 for (i = 0; i < flent->fe_rx_srs_cnt; i++) { 5343 mac_srs = (mac_soft_ring_set_t *)flent->fe_rx_srs[i]; 5344 mac_rx_stat = &mac_srs->srs_rx.sr_stat; 5345 5346 nstat->ns_ibytes += mac_rx_stat->mrs_intrbytes + 5347 mac_rx_stat->mrs_pollbytes + mac_rx_stat->mrs_lclbytes; 5348 nstat->ns_ipackets += mac_rx_stat->mrs_intrcnt + 5349 mac_rx_stat->mrs_pollcnt + mac_rx_stat->mrs_lclcnt; 5350 nstat->ns_oerrors += mac_rx_stat->mrs_ierrors; 5351 } 5352 5353 mac_srs = (mac_soft_ring_set_t *)(flent->fe_tx_srs); 5354 if (mac_srs != NULL) { 5355 mac_tx_stat = &mac_srs->srs_tx.st_stat; 5356 5357 nstat->ns_obytes = mac_tx_stat->mts_obytes; 5358 nstat->ns_opackets = mac_tx_stat->mts_opackets; 5359 nstat->ns_oerrors = mac_tx_stat->mts_oerrors; 5360 } 5361 5362 ninfo->ni_record = nstat; 5363 ninfo->ni_size = sizeof (net_stat_t); 5364 ninfo->ni_type = EX_NET_FLSTAT_REC; 5365 5366 return (ninfo); 5367 } 5368 5369 /* Write the link description to a netinfo_t record */ 5370 static netinfo_t * 5371 mac_write_link_desc(mac_client_impl_t *mcip) 5372 { 5373 netinfo_t *ninfo; 5374 net_desc_t *ndesc; 5375 flow_entry_t *flent = mcip->mci_flent; 5376 5377 ninfo = kmem_zalloc(sizeof (netinfo_t), KM_NOSLEEP); 5378 if (ninfo == NULL) 5379 return (NULL); 5380 ndesc = kmem_zalloc(sizeof (net_desc_t), KM_NOSLEEP); 5381 if (ndesc == NULL) { 5382 kmem_free(ninfo, sizeof (netinfo_t)); 5383 return (NULL); 5384 } 5385 5386 ndesc->nd_name = mcip->mci_name; 5387 ndesc->nd_devname = mcip->mci_name; 5388 ndesc->nd_isv4 = B_TRUE; 5389 /* 5390 * Grab the fe_lock to see a self-consistent fe_flow_desc. 5391 * Updates to the fe_flow_desc are done under the fe_lock 5392 * after removing the flent from the flow table. 5393 */ 5394 mutex_enter(&flent->fe_lock); 5395 bcopy(flent->fe_flow_desc.fd_src_mac, ndesc->nd_ehost, ETHERADDRL); 5396 mutex_exit(&flent->fe_lock); 5397 5398 ninfo->ni_record = ndesc; 5399 ninfo->ni_size = sizeof (net_desc_t); 5400 ninfo->ni_type = EX_NET_LNDESC_REC; 5401 5402 return (ninfo); 5403 } 5404 5405 /* Write the link statistics to a netinfo_t record */ 5406 static netinfo_t * 5407 mac_write_link_stats(mac_client_impl_t *mcip) 5408 { 5409 netinfo_t *ninfo; 5410 net_stat_t *nstat; 5411 flow_entry_t *flent; 5412 mac_soft_ring_set_t *mac_srs; 5413 mac_rx_stats_t *mac_rx_stat; 5414 mac_tx_stats_t *mac_tx_stat; 5415 int i; 5416 5417 ninfo = kmem_zalloc(sizeof (netinfo_t), KM_NOSLEEP); 5418 if (ninfo == NULL) 5419 return (NULL); 5420 nstat = kmem_zalloc(sizeof (net_stat_t), KM_NOSLEEP); 5421 if (nstat == NULL) { 5422 kmem_free(ninfo, sizeof (netinfo_t)); 5423 return (NULL); 5424 } 5425 5426 nstat->ns_name = mcip->mci_name; 5427 flent = mcip->mci_flent; 5428 if (flent != NULL) { 5429 for (i = 0; i < flent->fe_rx_srs_cnt; i++) { 5430 mac_srs = (mac_soft_ring_set_t *)flent->fe_rx_srs[i]; 5431 mac_rx_stat = &mac_srs->srs_rx.sr_stat; 5432 5433 nstat->ns_ibytes += mac_rx_stat->mrs_intrbytes + 5434 mac_rx_stat->mrs_pollbytes + 5435 mac_rx_stat->mrs_lclbytes; 5436 nstat->ns_ipackets += mac_rx_stat->mrs_intrcnt + 5437 mac_rx_stat->mrs_pollcnt + mac_rx_stat->mrs_lclcnt; 5438 nstat->ns_oerrors += mac_rx_stat->mrs_ierrors; 5439 } 5440 } 5441 5442 mac_srs = (mac_soft_ring_set_t *)(mcip->mci_flent->fe_tx_srs); 5443 if (mac_srs != NULL) { 5444 mac_tx_stat = &mac_srs->srs_tx.st_stat; 5445 5446 nstat->ns_obytes = mac_tx_stat->mts_obytes; 5447 nstat->ns_opackets = mac_tx_stat->mts_opackets; 5448 nstat->ns_oerrors = mac_tx_stat->mts_oerrors; 5449 } 5450 5451 ninfo->ni_record = nstat; 5452 ninfo->ni_size = sizeof (net_stat_t); 5453 ninfo->ni_type = EX_NET_LNSTAT_REC; 5454 5455 return (ninfo); 5456 } 5457 5458 typedef struct i_mac_log_state_s { 5459 boolean_t mi_last; 5460 int mi_fenable; 5461 int mi_lenable; 5462 list_t *mi_list; 5463 } i_mac_log_state_t; 5464 5465 /* 5466 * For a given flow, if the description has not been logged before, do it now. 5467 * If it is a VNIC, then we have collected information about it from the MAC 5468 * table, so skip it. 5469 * 5470 * Called through mac_flow_walk_nolock() 5471 * 5472 * Return 0 if successful. 5473 */ 5474 static int 5475 mac_log_flowinfo(flow_entry_t *flent, void *arg) 5476 { 5477 mac_client_impl_t *mcip = flent->fe_mcip; 5478 i_mac_log_state_t *lstate = arg; 5479 netinfo_t *ninfo; 5480 5481 if (mcip == NULL) 5482 return (0); 5483 5484 /* 5485 * If the name starts with "vnic", and fe_user_generated is true (to 5486 * exclude the mcast and active flow entries created implicitly for 5487 * a vnic, it is a VNIC flow. i.e. vnic1 is a vnic flow, 5488 * vnic/bge1/mcast1 is not and neither is vnic/bge1/active. 5489 */ 5490 if (strncasecmp(flent->fe_flow_name, "vnic", 4) == 0 && 5491 (flent->fe_type & FLOW_USER) != 0) { 5492 return (0); 5493 } 5494 5495 if (!flent->fe_desc_logged) { 5496 /* 5497 * We don't return error because we want to continue the 5498 * walk in case this is the last walk which means we 5499 * need to reset fe_desc_logged in all the flows. 5500 */ 5501 if ((ninfo = mac_write_flow_desc(flent, mcip)) == NULL) 5502 return (0); 5503 list_insert_tail(lstate->mi_list, ninfo); 5504 flent->fe_desc_logged = B_TRUE; 5505 } 5506 5507 /* 5508 * Regardless of the error, we want to proceed in case we have to 5509 * reset fe_desc_logged. 5510 */ 5511 ninfo = mac_write_flow_stats(flent); 5512 if (ninfo == NULL) 5513 return (-1); 5514 5515 list_insert_tail(lstate->mi_list, ninfo); 5516 5517 if (mcip != NULL && !(mcip->mci_state_flags & MCIS_DESC_LOGGED)) 5518 flent->fe_desc_logged = B_FALSE; 5519 5520 return (0); 5521 } 5522 5523 /* 5524 * Log the description for each mac client of this mac_impl_t, if it 5525 * hasn't already been done. Additionally, log statistics for the link as 5526 * well. Walk the flow table and log information for each flow as well. 5527 * If it is the last walk (mci_last), then we turn off mci_desc_logged (and 5528 * also fe_desc_logged, if flow logging is on) since we want to log the 5529 * description if and when logging is restarted. 5530 * 5531 * Return 0 upon success or -1 upon failure 5532 */ 5533 static int 5534 i_mac_impl_log(mac_impl_t *mip, i_mac_log_state_t *lstate) 5535 { 5536 mac_client_impl_t *mcip; 5537 netinfo_t *ninfo; 5538 5539 i_mac_perim_enter(mip); 5540 /* 5541 * Only walk the client list for NIC and etherstub 5542 */ 5543 if ((mip->mi_state_flags & MIS_DISABLED) || 5544 ((mip->mi_state_flags & MIS_IS_VNIC) && 5545 (mac_get_lower_mac_handle((mac_handle_t)mip) != NULL))) { 5546 i_mac_perim_exit(mip); 5547 return (0); 5548 } 5549 5550 for (mcip = mip->mi_clients_list; mcip != NULL; 5551 mcip = mcip->mci_client_next) { 5552 if (!MCIP_DATAPATH_SETUP(mcip)) 5553 continue; 5554 if (lstate->mi_lenable) { 5555 if (!(mcip->mci_state_flags & MCIS_DESC_LOGGED)) { 5556 ninfo = mac_write_link_desc(mcip); 5557 if (ninfo == NULL) { 5558 /* 5559 * We can't terminate it if this is the last 5560 * walk, else there might be some links with 5561 * mi_desc_logged set to true, which means 5562 * their description won't be logged the next 5563 * time logging is started (similarly for the 5564 * flows within such links). We can continue 5565 * without walking the flow table (i.e. to 5566 * set fe_desc_logged to false) because we 5567 * won't have written any flow stuff for this 5568 * link as we haven't logged the link itself. 5569 */ 5570 i_mac_perim_exit(mip); 5571 if (lstate->mi_last) 5572 return (0); 5573 else 5574 return (-1); 5575 } 5576 mcip->mci_state_flags |= MCIS_DESC_LOGGED; 5577 list_insert_tail(lstate->mi_list, ninfo); 5578 } 5579 } 5580 5581 ninfo = mac_write_link_stats(mcip); 5582 if (ninfo == NULL && !lstate->mi_last) { 5583 i_mac_perim_exit(mip); 5584 return (-1); 5585 } 5586 list_insert_tail(lstate->mi_list, ninfo); 5587 5588 if (lstate->mi_last) 5589 mcip->mci_state_flags &= ~MCIS_DESC_LOGGED; 5590 5591 if (lstate->mi_fenable) { 5592 if (mcip->mci_subflow_tab != NULL) { 5593 (void) mac_flow_walk_nolock( 5594 mcip->mci_subflow_tab, mac_log_flowinfo, 5595 lstate); 5596 } 5597 } 5598 } 5599 i_mac_perim_exit(mip); 5600 return (0); 5601 } 5602 5603 /* 5604 * modhash walker function to add a mac_impl_t to a list 5605 */ 5606 /*ARGSUSED*/ 5607 static uint_t 5608 i_mac_impl_list_walker(mod_hash_key_t key, mod_hash_val_t *val, void *arg) 5609 { 5610 list_t *list = (list_t *)arg; 5611 mac_impl_t *mip = (mac_impl_t *)val; 5612 5613 if ((mip->mi_state_flags & MIS_DISABLED) == 0) { 5614 list_insert_tail(list, mip); 5615 mip->mi_ref++; 5616 } 5617 5618 return (MH_WALK_CONTINUE); 5619 } 5620 5621 void 5622 i_mac_log_info(list_t *net_log_list, i_mac_log_state_t *lstate) 5623 { 5624 list_t mac_impl_list; 5625 mac_impl_t *mip; 5626 netinfo_t *ninfo; 5627 5628 /* Create list of mac_impls */ 5629 ASSERT(RW_LOCK_HELD(&i_mac_impl_lock)); 5630 list_create(&mac_impl_list, sizeof (mac_impl_t), offsetof(mac_impl_t, 5631 mi_node)); 5632 mod_hash_walk(i_mac_impl_hash, i_mac_impl_list_walker, &mac_impl_list); 5633 rw_exit(&i_mac_impl_lock); 5634 5635 /* Create log entries for each mac_impl */ 5636 for (mip = list_head(&mac_impl_list); mip != NULL; 5637 mip = list_next(&mac_impl_list, mip)) { 5638 if (i_mac_impl_log(mip, lstate) != 0) 5639 continue; 5640 } 5641 5642 /* Remove elements and destroy list of mac_impls */ 5643 rw_enter(&i_mac_impl_lock, RW_WRITER); 5644 while ((mip = list_remove_tail(&mac_impl_list)) != NULL) { 5645 mip->mi_ref--; 5646 } 5647 rw_exit(&i_mac_impl_lock); 5648 list_destroy(&mac_impl_list); 5649 5650 /* 5651 * Write log entries to files outside of locks, free associated 5652 * structures, and remove entries from the list. 5653 */ 5654 while ((ninfo = list_head(net_log_list)) != NULL) { 5655 (void) exacct_commit_netinfo(ninfo->ni_record, ninfo->ni_type); 5656 list_remove(net_log_list, ninfo); 5657 kmem_free(ninfo->ni_record, ninfo->ni_size); 5658 kmem_free(ninfo, sizeof (*ninfo)); 5659 } 5660 list_destroy(net_log_list); 5661 } 5662 5663 /* 5664 * The timer thread that runs every mac_logging_interval seconds and logs 5665 * link and/or flow information. 5666 */ 5667 /* ARGSUSED */ 5668 void 5669 mac_log_linkinfo(void *arg) 5670 { 5671 i_mac_log_state_t lstate; 5672 list_t net_log_list; 5673 5674 list_create(&net_log_list, sizeof (netinfo_t), 5675 offsetof(netinfo_t, ni_link)); 5676 5677 rw_enter(&i_mac_impl_lock, RW_READER); 5678 if (!mac_flow_log_enable && !mac_link_log_enable) { 5679 rw_exit(&i_mac_impl_lock); 5680 return; 5681 } 5682 lstate.mi_fenable = mac_flow_log_enable; 5683 lstate.mi_lenable = mac_link_log_enable; 5684 lstate.mi_last = B_FALSE; 5685 lstate.mi_list = &net_log_list; 5686 5687 /* Write log entries for each mac_impl in the list */ 5688 i_mac_log_info(&net_log_list, &lstate); 5689 5690 if (mac_flow_log_enable || mac_link_log_enable) { 5691 mac_logging_timer = timeout(mac_log_linkinfo, NULL, 5692 SEC_TO_TICK(mac_logging_interval)); 5693 } 5694 } 5695 5696 typedef struct i_mac_fastpath_state_s { 5697 boolean_t mf_disable; 5698 int mf_err; 5699 } i_mac_fastpath_state_t; 5700 5701 /* modhash walker function to enable or disable fastpath */ 5702 /*ARGSUSED*/ 5703 static uint_t 5704 i_mac_fastpath_walker(mod_hash_key_t key, mod_hash_val_t *val, 5705 void *arg) 5706 { 5707 i_mac_fastpath_state_t *state = arg; 5708 mac_handle_t mh = (mac_handle_t)val; 5709 5710 if (state->mf_disable) 5711 state->mf_err = mac_fastpath_disable(mh); 5712 else 5713 mac_fastpath_enable(mh); 5714 5715 return (state->mf_err == 0 ? MH_WALK_CONTINUE : MH_WALK_TERMINATE); 5716 } 5717 5718 /* 5719 * Start the logging timer. 5720 */ 5721 int 5722 mac_start_logusage(mac_logtype_t type, uint_t interval) 5723 { 5724 i_mac_fastpath_state_t dstate = {B_TRUE, 0}; 5725 i_mac_fastpath_state_t estate = {B_FALSE, 0}; 5726 int err; 5727 5728 rw_enter(&i_mac_impl_lock, RW_WRITER); 5729 switch (type) { 5730 case MAC_LOGTYPE_FLOW: 5731 if (mac_flow_log_enable) { 5732 rw_exit(&i_mac_impl_lock); 5733 return (0); 5734 } 5735 /* FALLTHRU */ 5736 case MAC_LOGTYPE_LINK: 5737 if (mac_link_log_enable) { 5738 rw_exit(&i_mac_impl_lock); 5739 return (0); 5740 } 5741 break; 5742 default: 5743 ASSERT(0); 5744 } 5745 5746 /* Disable fastpath */ 5747 mod_hash_walk(i_mac_impl_hash, i_mac_fastpath_walker, &dstate); 5748 if ((err = dstate.mf_err) != 0) { 5749 /* Reenable fastpath */ 5750 mod_hash_walk(i_mac_impl_hash, i_mac_fastpath_walker, &estate); 5751 rw_exit(&i_mac_impl_lock); 5752 return (err); 5753 } 5754 5755 switch (type) { 5756 case MAC_LOGTYPE_FLOW: 5757 mac_flow_log_enable = B_TRUE; 5758 /* FALLTHRU */ 5759 case MAC_LOGTYPE_LINK: 5760 mac_link_log_enable = B_TRUE; 5761 break; 5762 } 5763 5764 mac_logging_interval = interval; 5765 rw_exit(&i_mac_impl_lock); 5766 mac_log_linkinfo(NULL); 5767 return (0); 5768 } 5769 5770 /* 5771 * Stop the logging timer if both link and flow logging are turned off. 5772 */ 5773 void 5774 mac_stop_logusage(mac_logtype_t type) 5775 { 5776 i_mac_log_state_t lstate; 5777 i_mac_fastpath_state_t estate = {B_FALSE, 0}; 5778 list_t net_log_list; 5779 5780 list_create(&net_log_list, sizeof (netinfo_t), 5781 offsetof(netinfo_t, ni_link)); 5782 5783 rw_enter(&i_mac_impl_lock, RW_WRITER); 5784 5785 lstate.mi_fenable = mac_flow_log_enable; 5786 lstate.mi_lenable = mac_link_log_enable; 5787 lstate.mi_list = &net_log_list; 5788 5789 /* Last walk */ 5790 lstate.mi_last = B_TRUE; 5791 5792 switch (type) { 5793 case MAC_LOGTYPE_FLOW: 5794 if (lstate.mi_fenable) { 5795 ASSERT(mac_link_log_enable); 5796 mac_flow_log_enable = B_FALSE; 5797 mac_link_log_enable = B_FALSE; 5798 break; 5799 } 5800 /* FALLTHRU */ 5801 case MAC_LOGTYPE_LINK: 5802 if (!lstate.mi_lenable || mac_flow_log_enable) { 5803 rw_exit(&i_mac_impl_lock); 5804 return; 5805 } 5806 mac_link_log_enable = B_FALSE; 5807 break; 5808 default: 5809 ASSERT(0); 5810 } 5811 5812 /* Reenable fastpath */ 5813 mod_hash_walk(i_mac_impl_hash, i_mac_fastpath_walker, &estate); 5814 5815 (void) untimeout(mac_logging_timer); 5816 mac_logging_timer = 0; 5817 5818 /* Write log entries for each mac_impl in the list */ 5819 i_mac_log_info(&net_log_list, &lstate); 5820 } 5821 5822 /* 5823 * Walk the rx and tx SRS/SRs for a flow and update the priority value. 5824 */ 5825 void 5826 mac_flow_update_priority(mac_client_impl_t *mcip, flow_entry_t *flent) 5827 { 5828 pri_t pri; 5829 int count; 5830 mac_soft_ring_set_t *mac_srs; 5831 5832 if (flent->fe_rx_srs_cnt <= 0) 5833 return; 5834 5835 if (((mac_soft_ring_set_t *)flent->fe_rx_srs[0])->srs_type == 5836 SRST_FLOW) { 5837 pri = FLOW_PRIORITY(mcip->mci_min_pri, 5838 mcip->mci_max_pri, 5839 flent->fe_resource_props.mrp_priority); 5840 } else { 5841 pri = mcip->mci_max_pri; 5842 } 5843 5844 for (count = 0; count < flent->fe_rx_srs_cnt; count++) { 5845 mac_srs = flent->fe_rx_srs[count]; 5846 mac_update_srs_priority(mac_srs, pri); 5847 } 5848 /* 5849 * If we have a Tx SRS, we need to modify all the threads associated 5850 * with it. 5851 */ 5852 if (flent->fe_tx_srs != NULL) 5853 mac_update_srs_priority(flent->fe_tx_srs, pri); 5854 } 5855 5856 /* 5857 * RX and TX rings are reserved according to different semantics depending 5858 * on the requests from the MAC clients and type of rings: 5859 * 5860 * On the Tx side, by default we reserve individual rings, independently from 5861 * the groups. 5862 * 5863 * On the Rx side, the reservation is at the granularity of the group 5864 * of rings, and used for v12n level 1 only. It has a special case for the 5865 * primary client. 5866 * 5867 * If a share is allocated to a MAC client, we allocate a TX group and an 5868 * RX group to the client, and assign TX rings and RX rings to these 5869 * groups according to information gathered from the driver through 5870 * the share capability. 5871 * 5872 * The foreseable evolution of Rx rings will handle v12n level 2 and higher 5873 * to allocate individual rings out of a group and program the hw classifier 5874 * based on IP address or higher level criteria. 5875 */ 5876 5877 /* 5878 * mac_reserve_tx_ring() 5879 * Reserve a unused ring by marking it with MR_INUSE state. 5880 * As reserved, the ring is ready to function. 5881 * 5882 * Notes for Hybrid I/O: 5883 * 5884 * If a specific ring is needed, it is specified through the desired_ring 5885 * argument. Otherwise that argument is set to NULL. 5886 * If the desired ring was previous allocated to another client, this 5887 * function swaps it with a new ring from the group of unassigned rings. 5888 */ 5889 mac_ring_t * 5890 mac_reserve_tx_ring(mac_impl_t *mip, mac_ring_t *desired_ring) 5891 { 5892 mac_group_t *group; 5893 mac_grp_client_t *mgcp; 5894 mac_client_impl_t *mcip; 5895 mac_soft_ring_set_t *srs; 5896 5897 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 5898 5899 /* 5900 * Find an available ring and start it before changing its status. 5901 * The unassigned rings are at the end of the mi_tx_groups 5902 * array. 5903 */ 5904 group = MAC_DEFAULT_TX_GROUP(mip); 5905 5906 /* Can't take the default ring out of the default group */ 5907 ASSERT(desired_ring != (mac_ring_t *)mip->mi_default_tx_ring); 5908 5909 if (desired_ring->mr_state == MR_FREE) { 5910 ASSERT(MAC_GROUP_NO_CLIENT(group)); 5911 if (mac_start_ring(desired_ring) != 0) 5912 return (NULL); 5913 return (desired_ring); 5914 } 5915 /* 5916 * There are clients using this ring, so let's move the clients 5917 * away from using this ring. 5918 */ 5919 for (mgcp = group->mrg_clients; mgcp != NULL; mgcp = mgcp->mgc_next) { 5920 mcip = mgcp->mgc_client; 5921 mac_tx_client_quiesce((mac_client_handle_t)mcip); 5922 srs = MCIP_TX_SRS(mcip); 5923 ASSERT(mac_tx_srs_ring_present(srs, desired_ring)); 5924 mac_tx_invoke_callbacks(mcip, 5925 (mac_tx_cookie_t)mac_tx_srs_get_soft_ring(srs, 5926 desired_ring)); 5927 mac_tx_srs_del_ring(srs, desired_ring); 5928 mac_tx_client_restart((mac_client_handle_t)mcip); 5929 } 5930 return (desired_ring); 5931 } 5932 5933 /* 5934 * For a reserved group with multiple clients, return the primary client. 5935 */ 5936 static mac_client_impl_t * 5937 mac_get_grp_primary(mac_group_t *grp) 5938 { 5939 mac_grp_client_t *mgcp = grp->mrg_clients; 5940 mac_client_impl_t *mcip; 5941 5942 while (mgcp != NULL) { 5943 mcip = mgcp->mgc_client; 5944 if (mcip->mci_flent->fe_type & FLOW_PRIMARY_MAC) 5945 return (mcip); 5946 mgcp = mgcp->mgc_next; 5947 } 5948 return (NULL); 5949 } 5950 5951 /* 5952 * Hybrid I/O specifies the ring that should be given to a share. 5953 * If the ring is already used by clients, then we need to release 5954 * the ring back to the default group so that we can give it to 5955 * the share. This means the clients using this ring now get a 5956 * replacement ring. If there aren't any replacement rings, this 5957 * function returns a failure. 5958 */ 5959 static int 5960 mac_reclaim_ring_from_grp(mac_impl_t *mip, mac_ring_type_t ring_type, 5961 mac_ring_t *ring, mac_ring_t **rings, int nrings) 5962 { 5963 mac_group_t *group = (mac_group_t *)ring->mr_gh; 5964 mac_resource_props_t *mrp; 5965 mac_client_impl_t *mcip; 5966 mac_group_t *defgrp; 5967 mac_ring_t *tring; 5968 mac_group_t *tgrp; 5969 int i; 5970 int j; 5971 5972 mcip = MAC_GROUP_ONLY_CLIENT(group); 5973 if (mcip == NULL) 5974 mcip = mac_get_grp_primary(group); 5975 ASSERT(mcip != NULL); 5976 ASSERT(mcip->mci_share == 0); 5977 5978 mrp = MCIP_RESOURCE_PROPS(mcip); 5979 if (ring_type == MAC_RING_TYPE_RX) { 5980 defgrp = mip->mi_rx_donor_grp; 5981 if ((mrp->mrp_mask & MRP_RX_RINGS) == 0) { 5982 /* Need to put this mac client in the default group */ 5983 if (mac_rx_switch_group(mcip, group, defgrp) != 0) 5984 return (ENOSPC); 5985 } else { 5986 /* 5987 * Switch this ring with some other ring from 5988 * the default group. 5989 */ 5990 for (tring = defgrp->mrg_rings; tring != NULL; 5991 tring = tring->mr_next) { 5992 if (tring->mr_index == 0) 5993 continue; 5994 for (j = 0; j < nrings; j++) { 5995 if (rings[j] == tring) 5996 break; 5997 } 5998 if (j >= nrings) 5999 break; 6000 } 6001 if (tring == NULL) 6002 return (ENOSPC); 6003 if (mac_group_mov_ring(mip, group, tring) != 0) 6004 return (ENOSPC); 6005 if (mac_group_mov_ring(mip, defgrp, ring) != 0) { 6006 (void) mac_group_mov_ring(mip, defgrp, tring); 6007 return (ENOSPC); 6008 } 6009 } 6010 ASSERT(ring->mr_gh == (mac_group_handle_t)defgrp); 6011 return (0); 6012 } 6013 6014 defgrp = MAC_DEFAULT_TX_GROUP(mip); 6015 if (ring == (mac_ring_t *)mip->mi_default_tx_ring) { 6016 /* 6017 * See if we can get a spare ring to replace the default 6018 * ring. 6019 */ 6020 if (defgrp->mrg_cur_count == 1) { 6021 /* 6022 * Need to get a ring from another client, see if 6023 * there are any clients that can be moved to 6024 * the default group, thereby freeing some rings. 6025 */ 6026 for (i = 0; i < mip->mi_tx_group_count; i++) { 6027 tgrp = &mip->mi_tx_groups[i]; 6028 if (tgrp->mrg_state == 6029 MAC_GROUP_STATE_REGISTERED) { 6030 continue; 6031 } 6032 mcip = MAC_GROUP_ONLY_CLIENT(tgrp); 6033 if (mcip == NULL) 6034 mcip = mac_get_grp_primary(tgrp); 6035 ASSERT(mcip != NULL); 6036 mrp = MCIP_RESOURCE_PROPS(mcip); 6037 if ((mrp->mrp_mask & MRP_TX_RINGS) == 0) { 6038 ASSERT(tgrp->mrg_cur_count == 1); 6039 /* 6040 * If this ring is part of the 6041 * rings asked by the share we cannot 6042 * use it as the default ring. 6043 */ 6044 for (j = 0; j < nrings; j++) { 6045 if (rings[j] == tgrp->mrg_rings) 6046 break; 6047 } 6048 if (j < nrings) 6049 continue; 6050 mac_tx_client_quiesce( 6051 (mac_client_handle_t)mcip); 6052 mac_tx_switch_group(mcip, tgrp, 6053 defgrp); 6054 mac_tx_client_restart( 6055 (mac_client_handle_t)mcip); 6056 break; 6057 } 6058 } 6059 /* 6060 * All the rings are reserved, can't give up the 6061 * default ring. 6062 */ 6063 if (defgrp->mrg_cur_count <= 1) 6064 return (ENOSPC); 6065 } 6066 /* 6067 * Swap the default ring with another. 6068 */ 6069 for (tring = defgrp->mrg_rings; tring != NULL; 6070 tring = tring->mr_next) { 6071 /* 6072 * If this ring is part of the rings asked by the 6073 * share we cannot use it as the default ring. 6074 */ 6075 for (j = 0; j < nrings; j++) { 6076 if (rings[j] == tring) 6077 break; 6078 } 6079 if (j >= nrings) 6080 break; 6081 } 6082 ASSERT(tring != NULL); 6083 mip->mi_default_tx_ring = (mac_ring_handle_t)tring; 6084 return (0); 6085 } 6086 /* 6087 * The Tx ring is with a group reserved by a MAC client. See if 6088 * we can swap it. 6089 */ 6090 ASSERT(group->mrg_state == MAC_GROUP_STATE_RESERVED); 6091 mcip = MAC_GROUP_ONLY_CLIENT(group); 6092 if (mcip == NULL) 6093 mcip = mac_get_grp_primary(group); 6094 ASSERT(mcip != NULL); 6095 mrp = MCIP_RESOURCE_PROPS(mcip); 6096 mac_tx_client_quiesce((mac_client_handle_t)mcip); 6097 if ((mrp->mrp_mask & MRP_TX_RINGS) == 0) { 6098 ASSERT(group->mrg_cur_count == 1); 6099 /* Put this mac client in the default group */ 6100 mac_tx_switch_group(mcip, group, defgrp); 6101 } else { 6102 /* 6103 * Switch this ring with some other ring from 6104 * the default group. 6105 */ 6106 for (tring = defgrp->mrg_rings; tring != NULL; 6107 tring = tring->mr_next) { 6108 if (tring == (mac_ring_t *)mip->mi_default_tx_ring) 6109 continue; 6110 /* 6111 * If this ring is part of the rings asked by the 6112 * share we cannot use it for swapping. 6113 */ 6114 for (j = 0; j < nrings; j++) { 6115 if (rings[j] == tring) 6116 break; 6117 } 6118 if (j >= nrings) 6119 break; 6120 } 6121 if (tring == NULL) { 6122 mac_tx_client_restart((mac_client_handle_t)mcip); 6123 return (ENOSPC); 6124 } 6125 if (mac_group_mov_ring(mip, group, tring) != 0) { 6126 mac_tx_client_restart((mac_client_handle_t)mcip); 6127 return (ENOSPC); 6128 } 6129 if (mac_group_mov_ring(mip, defgrp, ring) != 0) { 6130 (void) mac_group_mov_ring(mip, defgrp, tring); 6131 mac_tx_client_restart((mac_client_handle_t)mcip); 6132 return (ENOSPC); 6133 } 6134 } 6135 mac_tx_client_restart((mac_client_handle_t)mcip); 6136 ASSERT(ring->mr_gh == (mac_group_handle_t)defgrp); 6137 return (0); 6138 } 6139 6140 /* 6141 * Populate a zero-ring group with rings. If the share is non-NULL, 6142 * the rings are chosen according to that share. 6143 * Invoked after allocating a new RX or TX group through 6144 * mac_reserve_rx_group() or mac_reserve_tx_group(), respectively. 6145 * Returns zero on success, an errno otherwise. 6146 */ 6147 int 6148 i_mac_group_allocate_rings(mac_impl_t *mip, mac_ring_type_t ring_type, 6149 mac_group_t *src_group, mac_group_t *new_group, mac_share_handle_t share, 6150 uint32_t ringcnt) 6151 { 6152 mac_ring_t **rings, *ring; 6153 uint_t nrings; 6154 int rv = 0, i = 0, j; 6155 6156 ASSERT((ring_type == MAC_RING_TYPE_RX && 6157 mip->mi_rx_group_type == MAC_GROUP_TYPE_DYNAMIC) || 6158 (ring_type == MAC_RING_TYPE_TX && 6159 mip->mi_tx_group_type == MAC_GROUP_TYPE_DYNAMIC)); 6160 6161 /* 6162 * First find the rings to allocate to the group. 6163 */ 6164 if (share != 0) { 6165 /* get rings through ms_squery() */ 6166 mip->mi_share_capab.ms_squery(share, ring_type, NULL, &nrings); 6167 ASSERT(nrings != 0); 6168 rings = kmem_alloc(nrings * sizeof (mac_ring_handle_t), 6169 KM_SLEEP); 6170 mip->mi_share_capab.ms_squery(share, ring_type, 6171 (mac_ring_handle_t *)rings, &nrings); 6172 for (i = 0; i < nrings; i++) { 6173 /* 6174 * If we have given this ring to a non-default 6175 * group, we need to check if we can get this 6176 * ring. 6177 */ 6178 ring = rings[i]; 6179 if (ring->mr_gh != (mac_group_handle_t)src_group || 6180 ring == (mac_ring_t *)mip->mi_default_tx_ring) { 6181 if (mac_reclaim_ring_from_grp(mip, ring_type, 6182 ring, rings, nrings) != 0) { 6183 rv = ENOSPC; 6184 goto bail; 6185 } 6186 } 6187 } 6188 } else { 6189 /* 6190 * Pick one ring from default group. 6191 * 6192 * for now pick the second ring which requires the first ring 6193 * at index 0 to stay in the default group, since it is the 6194 * ring which carries the multicast traffic. 6195 * We need a better way for a driver to indicate this, 6196 * for example a per-ring flag. 6197 */ 6198 rings = kmem_alloc(ringcnt * sizeof (mac_ring_handle_t), 6199 KM_SLEEP); 6200 for (ring = src_group->mrg_rings; ring != NULL; 6201 ring = ring->mr_next) { 6202 if (ring_type == MAC_RING_TYPE_RX && 6203 ring->mr_index == 0) { 6204 continue; 6205 } 6206 if (ring_type == MAC_RING_TYPE_TX && 6207 ring == (mac_ring_t *)mip->mi_default_tx_ring) { 6208 continue; 6209 } 6210 rings[i++] = ring; 6211 if (i == ringcnt) 6212 break; 6213 } 6214 ASSERT(ring != NULL); 6215 nrings = i; 6216 /* Not enough rings as required */ 6217 if (nrings != ringcnt) { 6218 rv = ENOSPC; 6219 goto bail; 6220 } 6221 } 6222 6223 switch (ring_type) { 6224 case MAC_RING_TYPE_RX: 6225 if (src_group->mrg_cur_count - nrings < 1) { 6226 /* we ran out of rings */ 6227 rv = ENOSPC; 6228 goto bail; 6229 } 6230 6231 /* move receive rings to new group */ 6232 for (i = 0; i < nrings; i++) { 6233 rv = mac_group_mov_ring(mip, new_group, rings[i]); 6234 if (rv != 0) { 6235 /* move rings back on failure */ 6236 for (j = 0; j < i; j++) { 6237 (void) mac_group_mov_ring(mip, 6238 src_group, rings[j]); 6239 } 6240 goto bail; 6241 } 6242 } 6243 break; 6244 6245 case MAC_RING_TYPE_TX: { 6246 mac_ring_t *tmp_ring; 6247 6248 /* move the TX rings to the new group */ 6249 for (i = 0; i < nrings; i++) { 6250 /* get the desired ring */ 6251 tmp_ring = mac_reserve_tx_ring(mip, rings[i]); 6252 if (tmp_ring == NULL) { 6253 rv = ENOSPC; 6254 goto bail; 6255 } 6256 ASSERT(tmp_ring == rings[i]); 6257 rv = mac_group_mov_ring(mip, new_group, rings[i]); 6258 if (rv != 0) { 6259 /* cleanup on failure */ 6260 for (j = 0; j < i; j++) { 6261 (void) mac_group_mov_ring(mip, 6262 MAC_DEFAULT_TX_GROUP(mip), 6263 rings[j]); 6264 } 6265 goto bail; 6266 } 6267 } 6268 break; 6269 } 6270 } 6271 6272 /* add group to share */ 6273 if (share != 0) 6274 mip->mi_share_capab.ms_sadd(share, new_group->mrg_driver); 6275 6276 bail: 6277 /* free temporary array of rings */ 6278 kmem_free(rings, nrings * sizeof (mac_ring_handle_t)); 6279 6280 return (rv); 6281 } 6282 6283 void 6284 mac_group_add_client(mac_group_t *grp, mac_client_impl_t *mcip) 6285 { 6286 mac_grp_client_t *mgcp; 6287 6288 for (mgcp = grp->mrg_clients; mgcp != NULL; mgcp = mgcp->mgc_next) { 6289 if (mgcp->mgc_client == mcip) 6290 break; 6291 } 6292 6293 VERIFY(mgcp == NULL); 6294 6295 mgcp = kmem_zalloc(sizeof (mac_grp_client_t), KM_SLEEP); 6296 mgcp->mgc_client = mcip; 6297 mgcp->mgc_next = grp->mrg_clients; 6298 grp->mrg_clients = mgcp; 6299 6300 } 6301 6302 void 6303 mac_group_remove_client(mac_group_t *grp, mac_client_impl_t *mcip) 6304 { 6305 mac_grp_client_t *mgcp, **pprev; 6306 6307 for (pprev = &grp->mrg_clients, mgcp = *pprev; mgcp != NULL; 6308 pprev = &mgcp->mgc_next, mgcp = *pprev) { 6309 if (mgcp->mgc_client == mcip) 6310 break; 6311 } 6312 6313 ASSERT(mgcp != NULL); 6314 6315 *pprev = mgcp->mgc_next; 6316 kmem_free(mgcp, sizeof (mac_grp_client_t)); 6317 } 6318 6319 /* 6320 * mac_reserve_rx_group() 6321 * 6322 * Finds an available group and exclusively reserves it for a client. 6323 * The group is chosen to suit the flow's resource controls (bandwidth and 6324 * fanout requirements) and the address type. 6325 * If the requestor is the pimary MAC then return the group with the 6326 * largest number of rings, otherwise the default ring when available. 6327 */ 6328 mac_group_t * 6329 mac_reserve_rx_group(mac_client_impl_t *mcip, uint8_t *mac_addr, boolean_t move) 6330 { 6331 mac_share_handle_t share = mcip->mci_share; 6332 mac_impl_t *mip = mcip->mci_mip; 6333 mac_group_t *grp = NULL; 6334 int i; 6335 int err = 0; 6336 mac_address_t *map; 6337 mac_resource_props_t *mrp = MCIP_RESOURCE_PROPS(mcip); 6338 int nrings; 6339 int donor_grp_rcnt; 6340 boolean_t need_exclgrp = B_FALSE; 6341 int need_rings = 0; 6342 mac_group_t *candidate_grp = NULL; 6343 mac_client_impl_t *gclient; 6344 mac_resource_props_t *gmrp; 6345 mac_group_t *donorgrp = NULL; 6346 boolean_t rxhw = mrp->mrp_mask & MRP_RX_RINGS; 6347 boolean_t unspec = mrp->mrp_mask & MRP_RXRINGS_UNSPEC; 6348 boolean_t isprimary; 6349 6350 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip)); 6351 6352 isprimary = mcip->mci_flent->fe_type & FLOW_PRIMARY_MAC; 6353 6354 /* 6355 * Check if a group already has this mac address (case of VLANs) 6356 * unless we are moving this MAC client from one group to another. 6357 */ 6358 if (!move && (map = mac_find_macaddr(mip, mac_addr)) != NULL) { 6359 if (map->ma_group != NULL) 6360 return (map->ma_group); 6361 } 6362 if (mip->mi_rx_groups == NULL || mip->mi_rx_group_count == 0) 6363 return (NULL); 6364 /* 6365 * If exclusive open, return NULL which will enable the 6366 * caller to use the default group. 6367 */ 6368 if (mcip->mci_state_flags & MCIS_EXCLUSIVE) 6369 return (NULL); 6370 6371 /* For dynamic groups default unspecified to 1 */ 6372 if (rxhw && unspec && 6373 mip->mi_rx_group_type == MAC_GROUP_TYPE_DYNAMIC) { 6374 mrp->mrp_nrxrings = 1; 6375 } 6376 /* 6377 * For static grouping we allow only specifying rings=0 and 6378 * unspecified 6379 */ 6380 if (rxhw && mrp->mrp_nrxrings > 0 && 6381 mip->mi_rx_group_type == MAC_GROUP_TYPE_STATIC) { 6382 return (NULL); 6383 } 6384 if (rxhw) { 6385 /* 6386 * We have explicitly asked for a group (with nrxrings, 6387 * if unspec). 6388 */ 6389 if (unspec || mrp->mrp_nrxrings > 0) { 6390 need_exclgrp = B_TRUE; 6391 need_rings = mrp->mrp_nrxrings; 6392 } else if (mrp->mrp_nrxrings == 0) { 6393 /* 6394 * We have asked for a software group. 6395 */ 6396 return (NULL); 6397 } 6398 } else if (isprimary && mip->mi_nactiveclients == 1 && 6399 mip->mi_rx_group_type == MAC_GROUP_TYPE_DYNAMIC) { 6400 /* 6401 * If the primary is the only active client on this 6402 * mip and we have not asked for any rings, we give 6403 * it the default group so that the primary gets to 6404 * use all the rings. 6405 */ 6406 return (NULL); 6407 } 6408 6409 /* The group that can donate rings */ 6410 donorgrp = mip->mi_rx_donor_grp; 6411 6412 /* 6413 * The number of rings that the default group can donate. 6414 * We need to leave at least one ring. 6415 */ 6416 donor_grp_rcnt = donorgrp->mrg_cur_count - 1; 6417 6418 /* 6419 * Try to exclusively reserve a RX group. 6420 * 6421 * For flows requiring HW_DEFAULT_RING (unicast flow of the primary 6422 * client), try to reserve the a non-default RX group and give 6423 * it all the rings from the donor group, except the default ring 6424 * 6425 * For flows requiring HW_RING (unicast flow of other clients), try 6426 * to reserve non-default RX group with the specified number of 6427 * rings, if available. 6428 * 6429 * For flows that have not asked for software or hardware ring, 6430 * try to reserve a non-default group with 1 ring, if available. 6431 */ 6432 for (i = 1; i < mip->mi_rx_group_count; i++) { 6433 grp = &mip->mi_rx_groups[i]; 6434 6435 DTRACE_PROBE3(rx__group__trying, char *, mip->mi_name, 6436 int, grp->mrg_index, mac_group_state_t, grp->mrg_state); 6437 6438 /* 6439 * Check if this group could be a candidate group for 6440 * eviction if we need a group for this MAC client, 6441 * but there aren't any. A candidate group is one 6442 * that didn't ask for an exclusive group, but got 6443 * one and it has enough rings (combined with what 6444 * the donor group can donate) for the new MAC 6445 * client 6446 */ 6447 if (grp->mrg_state >= MAC_GROUP_STATE_RESERVED) { 6448 /* 6449 * If the primary/donor group is not the default 6450 * group, don't bother looking for a candidate group. 6451 * If we don't have enough rings we will check 6452 * if the primary group can be vacated. 6453 */ 6454 if (candidate_grp == NULL && 6455 donorgrp == MAC_DEFAULT_RX_GROUP(mip)) { 6456 ASSERT(!MAC_GROUP_NO_CLIENT(grp)); 6457 gclient = MAC_GROUP_ONLY_CLIENT(grp); 6458 if (gclient == NULL) 6459 gclient = mac_get_grp_primary(grp); 6460 ASSERT(gclient != NULL); 6461 gmrp = MCIP_RESOURCE_PROPS(gclient); 6462 if (gclient->mci_share == 0 && 6463 (gmrp->mrp_mask & MRP_RX_RINGS) == 0 && 6464 (unspec || 6465 (grp->mrg_cur_count + donor_grp_rcnt >= 6466 need_rings))) { 6467 candidate_grp = grp; 6468 } 6469 } 6470 continue; 6471 } 6472 /* 6473 * This group could already be SHARED by other multicast 6474 * flows on this client. In that case, the group would 6475 * be shared and has already been started. 6476 */ 6477 ASSERT(grp->mrg_state != MAC_GROUP_STATE_UNINIT); 6478 6479 if ((grp->mrg_state == MAC_GROUP_STATE_REGISTERED) && 6480 (mac_start_group(grp) != 0)) { 6481 continue; 6482 } 6483 6484 if (mip->mi_rx_group_type != MAC_GROUP_TYPE_DYNAMIC) 6485 break; 6486 ASSERT(grp->mrg_cur_count == 0); 6487 6488 /* 6489 * Populate the group. Rings should be taken 6490 * from the donor group. 6491 */ 6492 nrings = rxhw ? need_rings : isprimary ? donor_grp_rcnt: 1; 6493 6494 /* 6495 * If the donor group can't donate, let's just walk and 6496 * see if someone can vacate a group, so that we have 6497 * enough rings for this, unless we already have 6498 * identified a candiate group.. 6499 */ 6500 if (nrings <= donor_grp_rcnt) { 6501 err = i_mac_group_allocate_rings(mip, MAC_RING_TYPE_RX, 6502 donorgrp, grp, share, nrings); 6503 if (err == 0) { 6504 /* 6505 * For a share i_mac_group_allocate_rings gets 6506 * the rings from the driver, let's populate 6507 * the property for the client now. 6508 */ 6509 if (share != 0) { 6510 mac_client_set_rings( 6511 (mac_client_handle_t)mcip, 6512 grp->mrg_cur_count, -1); 6513 } 6514 if (mac_is_primary_client(mcip) && !rxhw) 6515 mip->mi_rx_donor_grp = grp; 6516 break; 6517 } 6518 } 6519 6520 DTRACE_PROBE3(rx__group__reserve__alloc__rings, char *, 6521 mip->mi_name, int, grp->mrg_index, int, err); 6522 6523 /* 6524 * It's a dynamic group but the grouping operation 6525 * failed. 6526 */ 6527 mac_stop_group(grp); 6528 } 6529 /* We didn't find an exclusive group for this MAC client */ 6530 if (i >= mip->mi_rx_group_count) { 6531 6532 if (!need_exclgrp) 6533 return (NULL); 6534 6535 /* 6536 * If we found a candidate group then we switch the 6537 * MAC client from the candidate_group to the default 6538 * group and give the group to this MAC client. If 6539 * we didn't find a candidate_group, check if the 6540 * primary is in its own group and if it can make way 6541 * for this MAC client. 6542 */ 6543 if (candidate_grp == NULL && 6544 donorgrp != MAC_DEFAULT_RX_GROUP(mip) && 6545 donorgrp->mrg_cur_count >= need_rings) { 6546 candidate_grp = donorgrp; 6547 } 6548 if (candidate_grp != NULL) { 6549 boolean_t prim_grp = B_FALSE; 6550 6551 /* 6552 * Switch the MAC client from the candidate group 6553 * to the default group.. If this group was the 6554 * donor group, then after the switch we need 6555 * to update the donor group too. 6556 */ 6557 grp = candidate_grp; 6558 gclient = MAC_GROUP_ONLY_CLIENT(grp); 6559 if (gclient == NULL) 6560 gclient = mac_get_grp_primary(grp); 6561 if (grp == mip->mi_rx_donor_grp) 6562 prim_grp = B_TRUE; 6563 if (mac_rx_switch_group(gclient, grp, 6564 MAC_DEFAULT_RX_GROUP(mip)) != 0) { 6565 return (NULL); 6566 } 6567 if (prim_grp) { 6568 mip->mi_rx_donor_grp = 6569 MAC_DEFAULT_RX_GROUP(mip); 6570 donorgrp = MAC_DEFAULT_RX_GROUP(mip); 6571 } 6572 6573 6574 /* 6575 * Now give this group with the required rings 6576 * to this MAC client. 6577 */ 6578 ASSERT(grp->mrg_state == MAC_GROUP_STATE_REGISTERED); 6579 if (mac_start_group(grp) != 0) 6580 return (NULL); 6581 6582 if (mip->mi_rx_group_type != MAC_GROUP_TYPE_DYNAMIC) 6583 return (grp); 6584 6585 donor_grp_rcnt = donorgrp->mrg_cur_count - 1; 6586 ASSERT(grp->mrg_cur_count == 0); 6587 ASSERT(donor_grp_rcnt >= need_rings); 6588 err = i_mac_group_allocate_rings(mip, MAC_RING_TYPE_RX, 6589 donorgrp, grp, share, need_rings); 6590 if (err == 0) { 6591 /* 6592 * For a share i_mac_group_allocate_rings gets 6593 * the rings from the driver, let's populate 6594 * the property for the client now. 6595 */ 6596 if (share != 0) { 6597 mac_client_set_rings( 6598 (mac_client_handle_t)mcip, 6599 grp->mrg_cur_count, -1); 6600 } 6601 DTRACE_PROBE2(rx__group__reserved, 6602 char *, mip->mi_name, int, grp->mrg_index); 6603 return (grp); 6604 } 6605 DTRACE_PROBE3(rx__group__reserve__alloc__rings, char *, 6606 mip->mi_name, int, grp->mrg_index, int, err); 6607 mac_stop_group(grp); 6608 } 6609 return (NULL); 6610 } 6611 ASSERT(grp != NULL); 6612 6613 DTRACE_PROBE2(rx__group__reserved, 6614 char *, mip->mi_name, int, grp->mrg_index); 6615 return (grp); 6616 } 6617 6618 /* 6619 * mac_rx_release_group() 6620 * 6621 * This is called when there are no clients left for the group. 6622 * The group is stopped and marked MAC_GROUP_STATE_REGISTERED, 6623 * and if it is a non default group, the shares are removed and 6624 * all rings are assigned back to default group. 6625 */ 6626 void 6627 mac_release_rx_group(mac_client_impl_t *mcip, mac_group_t *group) 6628 { 6629 mac_impl_t *mip = mcip->mci_mip; 6630 mac_ring_t *ring; 6631 6632 ASSERT(group != MAC_DEFAULT_RX_GROUP(mip)); 6633 6634 if (mip->mi_rx_donor_grp == group) 6635 mip->mi_rx_donor_grp = MAC_DEFAULT_RX_GROUP(mip); 6636 6637 /* 6638 * This is the case where there are no clients left. Any 6639 * SRS etc on this group have also be quiesced. 6640 */ 6641 for (ring = group->mrg_rings; ring != NULL; ring = ring->mr_next) { 6642 if (ring->mr_classify_type == MAC_HW_CLASSIFIER) { 6643 ASSERT(group->mrg_state == MAC_GROUP_STATE_RESERVED); 6644 /* 6645 * Remove the SRS associated with the HW ring. 6646 * As a result, polling will be disabled. 6647 */ 6648 ring->mr_srs = NULL; 6649 } 6650 ASSERT(group->mrg_state < MAC_GROUP_STATE_RESERVED || 6651 ring->mr_state == MR_INUSE); 6652 if (ring->mr_state == MR_INUSE) { 6653 mac_stop_ring(ring); 6654 ring->mr_flag = 0; 6655 } 6656 } 6657 6658 /* remove group from share */ 6659 if (mcip->mci_share != 0) { 6660 mip->mi_share_capab.ms_sremove(mcip->mci_share, 6661 group->mrg_driver); 6662 } 6663 6664 if (mip->mi_rx_group_type == MAC_GROUP_TYPE_DYNAMIC) { 6665 mac_ring_t *ring; 6666 6667 /* 6668 * Rings were dynamically allocated to group. 6669 * Move rings back to default group. 6670 */ 6671 while ((ring = group->mrg_rings) != NULL) { 6672 (void) mac_group_mov_ring(mip, mip->mi_rx_donor_grp, 6673 ring); 6674 } 6675 } 6676 mac_stop_group(group); 6677 /* 6678 * Possible improvement: See if we can assign the group just released 6679 * to a another client of the mip 6680 */ 6681 } 6682 6683 /* 6684 * When we move the primary's mac address between groups, we need to also 6685 * take all the clients sharing the same mac address along with it (VLANs) 6686 * We remove the mac address for such clients from the group after quiescing 6687 * them. When we add the mac address we restart the client. Note that 6688 * the primary's mac address is removed from the group after all the 6689 * other clients sharing the address are removed. Similarly, the primary's 6690 * mac address is added before all the other client's mac address are 6691 * added. While grp is the group where the clients reside, tgrp is 6692 * the group where the addresses have to be added. 6693 */ 6694 static void 6695 mac_rx_move_macaddr_prim(mac_client_impl_t *mcip, mac_group_t *grp, 6696 mac_group_t *tgrp, uint8_t *maddr, boolean_t add) 6697 { 6698 mac_impl_t *mip = mcip->mci_mip; 6699 mac_grp_client_t *mgcp = grp->mrg_clients; 6700 mac_client_impl_t *gmcip; 6701 boolean_t prim; 6702 6703 prim = (mcip->mci_state_flags & MCIS_UNICAST_HW) != 0; 6704 6705 /* 6706 * If the clients are in a non-default group, we just have to 6707 * walk the group's client list. If it is in the default group 6708 * (which will be shared by other clients as well, we need to 6709 * check if the unicast address matches mcip's unicast. 6710 */ 6711 while (mgcp != NULL) { 6712 gmcip = mgcp->mgc_client; 6713 if (gmcip != mcip && 6714 (grp != MAC_DEFAULT_RX_GROUP(mip) || 6715 mcip->mci_unicast == gmcip->mci_unicast)) { 6716 if (!add) { 6717 mac_rx_client_quiesce( 6718 (mac_client_handle_t)gmcip); 6719 (void) mac_remove_macaddr(mcip->mci_unicast); 6720 } else { 6721 (void) mac_add_macaddr(mip, tgrp, maddr, prim); 6722 mac_rx_client_restart( 6723 (mac_client_handle_t)gmcip); 6724 } 6725 } 6726 mgcp = mgcp->mgc_next; 6727 } 6728 } 6729 6730 6731 /* 6732 * Move the MAC address from fgrp to tgrp. If this is the primary client, 6733 * we need to take any VLANs etc. together too. 6734 */ 6735 static int 6736 mac_rx_move_macaddr(mac_client_impl_t *mcip, mac_group_t *fgrp, 6737 mac_group_t *tgrp) 6738 { 6739 mac_impl_t *mip = mcip->mci_mip; 6740 uint8_t maddr[MAXMACADDRLEN]; 6741 int err = 0; 6742 boolean_t prim; 6743 boolean_t multiclnt = B_FALSE; 6744 6745 mac_rx_client_quiesce((mac_client_handle_t)mcip); 6746 ASSERT(mcip->mci_unicast != NULL); 6747 bcopy(mcip->mci_unicast->ma_addr, maddr, mcip->mci_unicast->ma_len); 6748 6749 prim = (mcip->mci_state_flags & MCIS_UNICAST_HW) != 0; 6750 if (mcip->mci_unicast->ma_nusers > 1) { 6751 mac_rx_move_macaddr_prim(mcip, fgrp, NULL, maddr, B_FALSE); 6752 multiclnt = B_TRUE; 6753 } 6754 ASSERT(mcip->mci_unicast->ma_nusers == 1); 6755 err = mac_remove_macaddr(mcip->mci_unicast); 6756 if (err != 0) { 6757 mac_rx_client_restart((mac_client_handle_t)mcip); 6758 if (multiclnt) { 6759 mac_rx_move_macaddr_prim(mcip, fgrp, fgrp, maddr, 6760 B_TRUE); 6761 } 6762 return (err); 6763 } 6764 /* 6765 * Program the H/W Classifier first, if this fails we need 6766 * not proceed with the other stuff. 6767 */ 6768 if ((err = mac_add_macaddr(mip, tgrp, maddr, prim)) != 0) { 6769 /* Revert back the H/W Classifier */ 6770 if ((err = mac_add_macaddr(mip, fgrp, maddr, prim)) != 0) { 6771 /* 6772 * This should not fail now since it worked earlier, 6773 * should we panic? 6774 */ 6775 cmn_err(CE_WARN, 6776 "mac_rx_switch_group: switching %p back" 6777 " to group %p failed!!", (void *)mcip, 6778 (void *)fgrp); 6779 } 6780 mac_rx_client_restart((mac_client_handle_t)mcip); 6781 if (multiclnt) { 6782 mac_rx_move_macaddr_prim(mcip, fgrp, fgrp, maddr, 6783 B_TRUE); 6784 } 6785 return (err); 6786 } 6787 mcip->mci_unicast = mac_find_macaddr(mip, maddr); 6788 mac_rx_client_restart((mac_client_handle_t)mcip); 6789 if (multiclnt) 6790 mac_rx_move_macaddr_prim(mcip, fgrp, tgrp, maddr, B_TRUE); 6791 return (err); 6792 } 6793 6794 /* 6795 * Switch the MAC client from one group to another. This means we need 6796 * to remove the MAC address from the group, remove the MAC client, 6797 * teardown the SRSs and revert the group state. Then, we add the client 6798 * to the destination group, set the SRSs, and add the MAC address to the 6799 * group. 6800 */ 6801 int 6802 mac_rx_switch_group(mac_client_impl_t *mcip, mac_group_t *fgrp, 6803 mac_group_t *tgrp) 6804 { 6805 int err; 6806 mac_group_state_t next_state; 6807 mac_client_impl_t *group_only_mcip; 6808 mac_client_impl_t *gmcip; 6809 mac_impl_t *mip = mcip->mci_mip; 6810 mac_grp_client_t *mgcp; 6811 6812 ASSERT(fgrp == mcip->mci_flent->fe_rx_ring_group); 6813 6814 if ((err = mac_rx_move_macaddr(mcip, fgrp, tgrp)) != 0) 6815 return (err); 6816 6817 /* 6818 * The group might be reserved, but SRSs may not be set up, e.g. 6819 * primary and its vlans using a reserved group. 6820 */ 6821 if (fgrp->mrg_state == MAC_GROUP_STATE_RESERVED && 6822 MAC_GROUP_ONLY_CLIENT(fgrp) != NULL) { 6823 mac_rx_srs_group_teardown(mcip->mci_flent, B_TRUE); 6824 } 6825 if (fgrp != MAC_DEFAULT_RX_GROUP(mip)) { 6826 mgcp = fgrp->mrg_clients; 6827 while (mgcp != NULL) { 6828 gmcip = mgcp->mgc_client; 6829 mgcp = mgcp->mgc_next; 6830 mac_group_remove_client(fgrp, gmcip); 6831 mac_group_add_client(tgrp, gmcip); 6832 gmcip->mci_flent->fe_rx_ring_group = tgrp; 6833 } 6834 mac_release_rx_group(mcip, fgrp); 6835 ASSERT(MAC_GROUP_NO_CLIENT(fgrp)); 6836 mac_set_group_state(fgrp, MAC_GROUP_STATE_REGISTERED); 6837 } else { 6838 mac_group_remove_client(fgrp, mcip); 6839 mac_group_add_client(tgrp, mcip); 6840 mcip->mci_flent->fe_rx_ring_group = tgrp; 6841 /* 6842 * If there are other clients (VLANs) sharing this address 6843 * we should be here only for the primary. 6844 */ 6845 if (mcip->mci_unicast->ma_nusers > 1) { 6846 /* 6847 * We need to move all the clients that are using 6848 * this h/w address. 6849 */ 6850 mgcp = fgrp->mrg_clients; 6851 while (mgcp != NULL) { 6852 gmcip = mgcp->mgc_client; 6853 mgcp = mgcp->mgc_next; 6854 if (mcip->mci_unicast == gmcip->mci_unicast) { 6855 mac_group_remove_client(fgrp, gmcip); 6856 mac_group_add_client(tgrp, gmcip); 6857 gmcip->mci_flent->fe_rx_ring_group = 6858 tgrp; 6859 } 6860 } 6861 } 6862 /* 6863 * The default group will still take the multicast, 6864 * broadcast traffic etc., so it won't go to 6865 * MAC_GROUP_STATE_REGISTERED. 6866 */ 6867 if (fgrp->mrg_state == MAC_GROUP_STATE_RESERVED) 6868 mac_rx_group_unmark(fgrp, MR_CONDEMNED); 6869 mac_set_group_state(fgrp, MAC_GROUP_STATE_SHARED); 6870 } 6871 next_state = mac_group_next_state(tgrp, &group_only_mcip, 6872 MAC_DEFAULT_RX_GROUP(mip), B_TRUE); 6873 mac_set_group_state(tgrp, next_state); 6874 /* 6875 * If the destination group is reserved, setup the SRSs etc. 6876 */ 6877 if (tgrp->mrg_state == MAC_GROUP_STATE_RESERVED) { 6878 mac_rx_srs_group_setup(mcip, mcip->mci_flent, SRST_LINK); 6879 mac_fanout_setup(mcip, mcip->mci_flent, 6880 MCIP_RESOURCE_PROPS(mcip), mac_rx_deliver, mcip, NULL, 6881 NULL); 6882 mac_rx_group_unmark(tgrp, MR_INCIPIENT); 6883 } else { 6884 mac_rx_switch_grp_to_sw(tgrp); 6885 } 6886 return (0); 6887 } 6888 6889 /* 6890 * Reserves a TX group for the specified share. Invoked by mac_tx_srs_setup() 6891 * when a share was allocated to the client. 6892 */ 6893 mac_group_t * 6894 mac_reserve_tx_group(mac_client_impl_t *mcip, boolean_t move) 6895 { 6896 mac_impl_t *mip = mcip->mci_mip; 6897 mac_group_t *grp = NULL; 6898 int rv; 6899 int i; 6900 int err; 6901 mac_group_t *defgrp; 6902 mac_share_handle_t share = mcip->mci_share; 6903 mac_resource_props_t *mrp = MCIP_RESOURCE_PROPS(mcip); 6904 int nrings; 6905 int defnrings; 6906 boolean_t need_exclgrp = B_FALSE; 6907 int need_rings = 0; 6908 mac_group_t *candidate_grp = NULL; 6909 mac_client_impl_t *gclient; 6910 mac_resource_props_t *gmrp; 6911 boolean_t txhw = mrp->mrp_mask & MRP_TX_RINGS; 6912 boolean_t unspec = mrp->mrp_mask & MRP_TXRINGS_UNSPEC; 6913 boolean_t isprimary; 6914 6915 isprimary = mcip->mci_flent->fe_type & FLOW_PRIMARY_MAC; 6916 /* 6917 * When we come here for a VLAN on the primary (dladm create-vlan), 6918 * we need to pair it along with the primary (to keep it consistent 6919 * with the RX side). So, we check if the primary is already assigned 6920 * to a group and return the group if so. The other way is also 6921 * true, i.e. the VLAN is already created and now we are plumbing 6922 * the primary. 6923 */ 6924 if (!move && isprimary) { 6925 for (gclient = mip->mi_clients_list; gclient != NULL; 6926 gclient = gclient->mci_client_next) { 6927 if (gclient->mci_flent->fe_type & FLOW_PRIMARY_MAC && 6928 gclient->mci_flent->fe_tx_ring_group != NULL) { 6929 return (gclient->mci_flent->fe_tx_ring_group); 6930 } 6931 } 6932 } 6933 6934 if (mip->mi_tx_groups == NULL || mip->mi_tx_group_count == 0) 6935 return (NULL); 6936 6937 /* For dynamic groups, default unspec to 1 */ 6938 if (txhw && unspec && 6939 mip->mi_tx_group_type == MAC_GROUP_TYPE_DYNAMIC) { 6940 mrp->mrp_ntxrings = 1; 6941 } 6942 /* 6943 * For static grouping we allow only specifying rings=0 and 6944 * unspecified 6945 */ 6946 if (txhw && mrp->mrp_ntxrings > 0 && 6947 mip->mi_tx_group_type == MAC_GROUP_TYPE_STATIC) { 6948 return (NULL); 6949 } 6950 6951 if (txhw) { 6952 /* 6953 * We have explicitly asked for a group (with ntxrings, 6954 * if unspec). 6955 */ 6956 if (unspec || mrp->mrp_ntxrings > 0) { 6957 need_exclgrp = B_TRUE; 6958 need_rings = mrp->mrp_ntxrings; 6959 } else if (mrp->mrp_ntxrings == 0) { 6960 /* 6961 * We have asked for a software group. 6962 */ 6963 return (NULL); 6964 } 6965 } 6966 defgrp = MAC_DEFAULT_TX_GROUP(mip); 6967 /* 6968 * The number of rings that the default group can donate. 6969 * We need to leave at least one ring - the default ring - in 6970 * this group. 6971 */ 6972 defnrings = defgrp->mrg_cur_count - 1; 6973 6974 /* 6975 * Primary gets default group unless explicitly told not 6976 * to (i.e. rings > 0). 6977 */ 6978 if (isprimary && !need_exclgrp) 6979 return (NULL); 6980 6981 nrings = (mrp->mrp_mask & MRP_TX_RINGS) != 0 ? mrp->mrp_ntxrings : 1; 6982 for (i = 0; i < mip->mi_tx_group_count; i++) { 6983 grp = &mip->mi_tx_groups[i]; 6984 if ((grp->mrg_state == MAC_GROUP_STATE_RESERVED) || 6985 (grp->mrg_state == MAC_GROUP_STATE_UNINIT)) { 6986 /* 6987 * Select a candidate for replacement if we don't 6988 * get an exclusive group. A candidate group is one 6989 * that didn't ask for an exclusive group, but got 6990 * one and it has enough rings (combined with what 6991 * the default group can donate) for the new MAC 6992 * client. 6993 */ 6994 if (grp->mrg_state == MAC_GROUP_STATE_RESERVED && 6995 candidate_grp == NULL) { 6996 gclient = MAC_GROUP_ONLY_CLIENT(grp); 6997 if (gclient == NULL) 6998 gclient = mac_get_grp_primary(grp); 6999 gmrp = MCIP_RESOURCE_PROPS(gclient); 7000 if (gclient->mci_share == 0 && 7001 (gmrp->mrp_mask & MRP_TX_RINGS) == 0 && 7002 (unspec || 7003 (grp->mrg_cur_count + defnrings) >= 7004 need_rings)) { 7005 candidate_grp = grp; 7006 } 7007 } 7008 continue; 7009 } 7010 /* 7011 * If the default can't donate let's just walk and 7012 * see if someone can vacate a group, so that we have 7013 * enough rings for this. 7014 */ 7015 if (mip->mi_tx_group_type != MAC_GROUP_TYPE_DYNAMIC || 7016 nrings <= defnrings) { 7017 if (grp->mrg_state == MAC_GROUP_STATE_REGISTERED) { 7018 rv = mac_start_group(grp); 7019 ASSERT(rv == 0); 7020 } 7021 break; 7022 } 7023 } 7024 7025 /* The default group */ 7026 if (i >= mip->mi_tx_group_count) { 7027 /* 7028 * If we need an exclusive group and have identified a 7029 * candidate group we switch the MAC client from the 7030 * candidate group to the default group and give the 7031 * candidate group to this client. 7032 */ 7033 if (need_exclgrp && candidate_grp != NULL) { 7034 /* 7035 * Switch the MAC client from the candidate group 7036 * to the default group. 7037 */ 7038 grp = candidate_grp; 7039 gclient = MAC_GROUP_ONLY_CLIENT(grp); 7040 if (gclient == NULL) 7041 gclient = mac_get_grp_primary(grp); 7042 mac_tx_client_quiesce((mac_client_handle_t)gclient); 7043 mac_tx_switch_group(gclient, grp, defgrp); 7044 mac_tx_client_restart((mac_client_handle_t)gclient); 7045 7046 /* 7047 * Give the candidate group with the specified number 7048 * of rings to this MAC client. 7049 */ 7050 ASSERT(grp->mrg_state == MAC_GROUP_STATE_REGISTERED); 7051 rv = mac_start_group(grp); 7052 ASSERT(rv == 0); 7053 7054 if (mip->mi_tx_group_type != MAC_GROUP_TYPE_DYNAMIC) 7055 return (grp); 7056 7057 ASSERT(grp->mrg_cur_count == 0); 7058 ASSERT(defgrp->mrg_cur_count > need_rings); 7059 7060 err = i_mac_group_allocate_rings(mip, MAC_RING_TYPE_TX, 7061 defgrp, grp, share, need_rings); 7062 if (err == 0) { 7063 /* 7064 * For a share i_mac_group_allocate_rings gets 7065 * the rings from the driver, let's populate 7066 * the property for the client now. 7067 */ 7068 if (share != 0) { 7069 mac_client_set_rings( 7070 (mac_client_handle_t)mcip, -1, 7071 grp->mrg_cur_count); 7072 } 7073 mip->mi_tx_group_free--; 7074 return (grp); 7075 } 7076 DTRACE_PROBE3(tx__group__reserve__alloc__rings, char *, 7077 mip->mi_name, int, grp->mrg_index, int, err); 7078 mac_stop_group(grp); 7079 } 7080 return (NULL); 7081 } 7082 /* 7083 * We got an exclusive group, but it is not dynamic. 7084 */ 7085 if (mip->mi_tx_group_type != MAC_GROUP_TYPE_DYNAMIC) { 7086 mip->mi_tx_group_free--; 7087 return (grp); 7088 } 7089 7090 rv = i_mac_group_allocate_rings(mip, MAC_RING_TYPE_TX, defgrp, grp, 7091 share, nrings); 7092 if (rv != 0) { 7093 DTRACE_PROBE3(tx__group__reserve__alloc__rings, 7094 char *, mip->mi_name, int, grp->mrg_index, int, rv); 7095 mac_stop_group(grp); 7096 return (NULL); 7097 } 7098 /* 7099 * For a share i_mac_group_allocate_rings gets the rings from the 7100 * driver, let's populate the property for the client now. 7101 */ 7102 if (share != 0) { 7103 mac_client_set_rings((mac_client_handle_t)mcip, -1, 7104 grp->mrg_cur_count); 7105 } 7106 mip->mi_tx_group_free--; 7107 return (grp); 7108 } 7109 7110 void 7111 mac_release_tx_group(mac_client_impl_t *mcip, mac_group_t *grp) 7112 { 7113 mac_impl_t *mip = mcip->mci_mip; 7114 mac_share_handle_t share = mcip->mci_share; 7115 mac_ring_t *ring; 7116 mac_soft_ring_set_t *srs = MCIP_TX_SRS(mcip); 7117 mac_group_t *defgrp; 7118 7119 defgrp = MAC_DEFAULT_TX_GROUP(mip); 7120 if (srs != NULL) { 7121 if (srs->srs_soft_ring_count > 0) { 7122 for (ring = grp->mrg_rings; ring != NULL; 7123 ring = ring->mr_next) { 7124 ASSERT(mac_tx_srs_ring_present(srs, ring)); 7125 mac_tx_invoke_callbacks(mcip, 7126 (mac_tx_cookie_t) 7127 mac_tx_srs_get_soft_ring(srs, ring)); 7128 mac_tx_srs_del_ring(srs, ring); 7129 } 7130 } else { 7131 ASSERT(srs->srs_tx.st_arg2 != NULL); 7132 srs->srs_tx.st_arg2 = NULL; 7133 mac_srs_stat_delete(srs); 7134 } 7135 } 7136 if (share != 0) 7137 mip->mi_share_capab.ms_sremove(share, grp->mrg_driver); 7138 7139 /* move the ring back to the pool */ 7140 if (mip->mi_tx_group_type == MAC_GROUP_TYPE_DYNAMIC) { 7141 while ((ring = grp->mrg_rings) != NULL) 7142 (void) mac_group_mov_ring(mip, defgrp, ring); 7143 } 7144 mac_stop_group(grp); 7145 mip->mi_tx_group_free++; 7146 } 7147 7148 /* 7149 * Disassociate a MAC client from a group, i.e go through the rings in the 7150 * group and delete all the soft rings tied to them. 7151 */ 7152 static void 7153 mac_tx_dismantle_soft_rings(mac_group_t *fgrp, flow_entry_t *flent) 7154 { 7155 mac_client_impl_t *mcip = flent->fe_mcip; 7156 mac_soft_ring_set_t *tx_srs; 7157 mac_srs_tx_t *tx; 7158 mac_ring_t *ring; 7159 7160 tx_srs = flent->fe_tx_srs; 7161 tx = &tx_srs->srs_tx; 7162 7163 /* Single ring case we haven't created any soft rings */ 7164 if (tx->st_mode == SRS_TX_BW || tx->st_mode == SRS_TX_SERIALIZE || 7165 tx->st_mode == SRS_TX_DEFAULT) { 7166 tx->st_arg2 = NULL; 7167 mac_srs_stat_delete(tx_srs); 7168 /* Fanout case, where we have to dismantle the soft rings */ 7169 } else { 7170 for (ring = fgrp->mrg_rings; ring != NULL; 7171 ring = ring->mr_next) { 7172 ASSERT(mac_tx_srs_ring_present(tx_srs, ring)); 7173 mac_tx_invoke_callbacks(mcip, 7174 (mac_tx_cookie_t)mac_tx_srs_get_soft_ring(tx_srs, 7175 ring)); 7176 mac_tx_srs_del_ring(tx_srs, ring); 7177 } 7178 ASSERT(tx->st_arg2 == NULL); 7179 } 7180 } 7181 7182 /* 7183 * Switch the MAC client from one group to another. This means we need 7184 * to remove the MAC client, teardown the SRSs and revert the group state. 7185 * Then, we add the client to the destination roup, set the SRSs etc. 7186 */ 7187 void 7188 mac_tx_switch_group(mac_client_impl_t *mcip, mac_group_t *fgrp, 7189 mac_group_t *tgrp) 7190 { 7191 mac_client_impl_t *group_only_mcip; 7192 mac_impl_t *mip = mcip->mci_mip; 7193 flow_entry_t *flent = mcip->mci_flent; 7194 mac_group_t *defgrp; 7195 mac_grp_client_t *mgcp; 7196 mac_client_impl_t *gmcip; 7197 flow_entry_t *gflent; 7198 7199 defgrp = MAC_DEFAULT_TX_GROUP(mip); 7200 ASSERT(fgrp == flent->fe_tx_ring_group); 7201 7202 if (fgrp == defgrp) { 7203 /* 7204 * If this is the primary we need to find any VLANs on 7205 * the primary and move them too. 7206 */ 7207 mac_group_remove_client(fgrp, mcip); 7208 mac_tx_dismantle_soft_rings(fgrp, flent); 7209 if (mcip->mci_unicast->ma_nusers > 1) { 7210 mgcp = fgrp->mrg_clients; 7211 while (mgcp != NULL) { 7212 gmcip = mgcp->mgc_client; 7213 mgcp = mgcp->mgc_next; 7214 if (mcip->mci_unicast != gmcip->mci_unicast) 7215 continue; 7216 mac_tx_client_quiesce( 7217 (mac_client_handle_t)gmcip); 7218 7219 gflent = gmcip->mci_flent; 7220 mac_group_remove_client(fgrp, gmcip); 7221 mac_tx_dismantle_soft_rings(fgrp, gflent); 7222 7223 mac_group_add_client(tgrp, gmcip); 7224 gflent->fe_tx_ring_group = tgrp; 7225 /* We could directly set this to SHARED */ 7226 tgrp->mrg_state = mac_group_next_state(tgrp, 7227 &group_only_mcip, defgrp, B_FALSE); 7228 7229 mac_tx_srs_group_setup(gmcip, gflent, 7230 SRST_LINK); 7231 mac_fanout_setup(gmcip, gflent, 7232 MCIP_RESOURCE_PROPS(gmcip), mac_rx_deliver, 7233 gmcip, NULL, NULL); 7234 7235 mac_tx_client_restart( 7236 (mac_client_handle_t)gmcip); 7237 } 7238 } 7239 if (MAC_GROUP_NO_CLIENT(fgrp)) { 7240 mac_ring_t *ring; 7241 int cnt; 7242 int ringcnt; 7243 7244 fgrp->mrg_state = MAC_GROUP_STATE_REGISTERED; 7245 /* 7246 * Additionally, we also need to stop all 7247 * the rings in the default group, except 7248 * the default ring. The reason being 7249 * this group won't be released since it is 7250 * the default group, so the rings won't 7251 * be stopped otherwise. 7252 */ 7253 ringcnt = fgrp->mrg_cur_count; 7254 ring = fgrp->mrg_rings; 7255 for (cnt = 0; cnt < ringcnt; cnt++) { 7256 if (ring->mr_state == MR_INUSE && 7257 ring != 7258 (mac_ring_t *)mip->mi_default_tx_ring) { 7259 mac_stop_ring(ring); 7260 ring->mr_flag = 0; 7261 } 7262 ring = ring->mr_next; 7263 } 7264 } else if (MAC_GROUP_ONLY_CLIENT(fgrp) != NULL) { 7265 fgrp->mrg_state = MAC_GROUP_STATE_RESERVED; 7266 } else { 7267 ASSERT(fgrp->mrg_state == MAC_GROUP_STATE_SHARED); 7268 } 7269 } else { 7270 /* 7271 * We could have VLANs sharing the non-default group with 7272 * the primary. 7273 */ 7274 mgcp = fgrp->mrg_clients; 7275 while (mgcp != NULL) { 7276 gmcip = mgcp->mgc_client; 7277 mgcp = mgcp->mgc_next; 7278 if (gmcip == mcip) 7279 continue; 7280 mac_tx_client_quiesce((mac_client_handle_t)gmcip); 7281 gflent = gmcip->mci_flent; 7282 7283 mac_group_remove_client(fgrp, gmcip); 7284 mac_tx_dismantle_soft_rings(fgrp, gflent); 7285 7286 mac_group_add_client(tgrp, gmcip); 7287 gflent->fe_tx_ring_group = tgrp; 7288 /* We could directly set this to SHARED */ 7289 tgrp->mrg_state = mac_group_next_state(tgrp, 7290 &group_only_mcip, defgrp, B_FALSE); 7291 mac_tx_srs_group_setup(gmcip, gflent, SRST_LINK); 7292 mac_fanout_setup(gmcip, gflent, 7293 MCIP_RESOURCE_PROPS(gmcip), mac_rx_deliver, 7294 gmcip, NULL, NULL); 7295 7296 mac_tx_client_restart((mac_client_handle_t)gmcip); 7297 } 7298 mac_group_remove_client(fgrp, mcip); 7299 mac_release_tx_group(mcip, fgrp); 7300 fgrp->mrg_state = MAC_GROUP_STATE_REGISTERED; 7301 } 7302 7303 /* Add it to the tgroup */ 7304 mac_group_add_client(tgrp, mcip); 7305 flent->fe_tx_ring_group = tgrp; 7306 tgrp->mrg_state = mac_group_next_state(tgrp, &group_only_mcip, 7307 defgrp, B_FALSE); 7308 7309 mac_tx_srs_group_setup(mcip, flent, SRST_LINK); 7310 mac_fanout_setup(mcip, flent, MCIP_RESOURCE_PROPS(mcip), 7311 mac_rx_deliver, mcip, NULL, NULL); 7312 } 7313 7314 /* 7315 * This is a 1-time control path activity initiated by the client (IP). 7316 * The mac perimeter protects against other simultaneous control activities, 7317 * for example an ioctl that attempts to change the degree of fanout and 7318 * increase or decrease the number of softrings associated with this Tx SRS. 7319 */ 7320 static mac_tx_notify_cb_t * 7321 mac_client_tx_notify_add(mac_client_impl_t *mcip, 7322 mac_tx_notify_t notify, void *arg) 7323 { 7324 mac_cb_info_t *mcbi; 7325 mac_tx_notify_cb_t *mtnfp; 7326 7327 ASSERT(MAC_PERIM_HELD((mac_handle_t)mcip->mci_mip)); 7328 7329 mtnfp = kmem_zalloc(sizeof (mac_tx_notify_cb_t), KM_SLEEP); 7330 mtnfp->mtnf_fn = notify; 7331 mtnfp->mtnf_arg = arg; 7332 mtnfp->mtnf_link.mcb_objp = mtnfp; 7333 mtnfp->mtnf_link.mcb_objsize = sizeof (mac_tx_notify_cb_t); 7334 mtnfp->mtnf_link.mcb_flags = MCB_TX_NOTIFY_CB_T; 7335 7336 mcbi = &mcip->mci_tx_notify_cb_info; 7337 mutex_enter(mcbi->mcbi_lockp); 7338 mac_callback_add(mcbi, &mcip->mci_tx_notify_cb_list, &mtnfp->mtnf_link); 7339 mutex_exit(mcbi->mcbi_lockp); 7340 return (mtnfp); 7341 } 7342 7343 static void 7344 mac_client_tx_notify_remove(mac_client_impl_t *mcip, mac_tx_notify_cb_t *mtnfp) 7345 { 7346 mac_cb_info_t *mcbi; 7347 mac_cb_t **cblist; 7348 7349 ASSERT(MAC_PERIM_HELD((mac_handle_t)mcip->mci_mip)); 7350 7351 if (!mac_callback_find(&mcip->mci_tx_notify_cb_info, 7352 &mcip->mci_tx_notify_cb_list, &mtnfp->mtnf_link)) { 7353 cmn_err(CE_WARN, 7354 "mac_client_tx_notify_remove: callback not " 7355 "found, mcip 0x%p mtnfp 0x%p", (void *)mcip, (void *)mtnfp); 7356 return; 7357 } 7358 7359 mcbi = &mcip->mci_tx_notify_cb_info; 7360 cblist = &mcip->mci_tx_notify_cb_list; 7361 mutex_enter(mcbi->mcbi_lockp); 7362 if (mac_callback_remove(mcbi, cblist, &mtnfp->mtnf_link)) 7363 kmem_free(mtnfp, sizeof (mac_tx_notify_cb_t)); 7364 else 7365 mac_callback_remove_wait(&mcip->mci_tx_notify_cb_info); 7366 mutex_exit(mcbi->mcbi_lockp); 7367 } 7368 7369 /* 7370 * mac_client_tx_notify(): 7371 * call to add and remove flow control callback routine. 7372 */ 7373 mac_tx_notify_handle_t 7374 mac_client_tx_notify(mac_client_handle_t mch, mac_tx_notify_t callb_func, 7375 void *ptr) 7376 { 7377 mac_client_impl_t *mcip = (mac_client_impl_t *)mch; 7378 mac_tx_notify_cb_t *mtnfp = NULL; 7379 7380 i_mac_perim_enter(mcip->mci_mip); 7381 7382 if (callb_func != NULL) { 7383 /* Add a notify callback */ 7384 mtnfp = mac_client_tx_notify_add(mcip, callb_func, ptr); 7385 } else { 7386 mac_client_tx_notify_remove(mcip, (mac_tx_notify_cb_t *)ptr); 7387 } 7388 i_mac_perim_exit(mcip->mci_mip); 7389 7390 return ((mac_tx_notify_handle_t)mtnfp); 7391 } 7392 7393 void 7394 mac_bridge_vectors(mac_bridge_tx_t txf, mac_bridge_rx_t rxf, 7395 mac_bridge_ref_t reff, mac_bridge_ls_t lsf) 7396 { 7397 mac_bridge_tx_cb = txf; 7398 mac_bridge_rx_cb = rxf; 7399 mac_bridge_ref_cb = reff; 7400 mac_bridge_ls_cb = lsf; 7401 } 7402 7403 int 7404 mac_bridge_set(mac_handle_t mh, mac_handle_t link) 7405 { 7406 mac_impl_t *mip = (mac_impl_t *)mh; 7407 int retv; 7408 7409 mutex_enter(&mip->mi_bridge_lock); 7410 if (mip->mi_bridge_link == NULL) { 7411 mip->mi_bridge_link = link; 7412 retv = 0; 7413 } else { 7414 retv = EBUSY; 7415 } 7416 mutex_exit(&mip->mi_bridge_lock); 7417 if (retv == 0) { 7418 mac_poll_state_change(mh, B_FALSE); 7419 mac_capab_update(mh); 7420 } 7421 return (retv); 7422 } 7423 7424 /* 7425 * Disable bridging on the indicated link. 7426 */ 7427 void 7428 mac_bridge_clear(mac_handle_t mh, mac_handle_t link) 7429 { 7430 mac_impl_t *mip = (mac_impl_t *)mh; 7431 7432 mutex_enter(&mip->mi_bridge_lock); 7433 ASSERT(mip->mi_bridge_link == link); 7434 mip->mi_bridge_link = NULL; 7435 mutex_exit(&mip->mi_bridge_lock); 7436 mac_poll_state_change(mh, B_TRUE); 7437 mac_capab_update(mh); 7438 } 7439 7440 void 7441 mac_no_active(mac_handle_t mh) 7442 { 7443 mac_impl_t *mip = (mac_impl_t *)mh; 7444 7445 i_mac_perim_enter(mip); 7446 mip->mi_state_flags |= MIS_NO_ACTIVE; 7447 i_mac_perim_exit(mip); 7448 } 7449 7450 /* 7451 * Walk the primary VLAN clients whenever the primary's rings property 7452 * changes and update the mac_resource_props_t for the VLAN's client. 7453 * We need to do this since we don't support setting these properties 7454 * on the primary's VLAN clients, but the VLAN clients have to 7455 * follow the primary w.r.t the rings property; 7456 */ 7457 void 7458 mac_set_prim_vlan_rings(mac_impl_t *mip, mac_resource_props_t *mrp) 7459 { 7460 mac_client_impl_t *vmcip; 7461 mac_resource_props_t *vmrp; 7462 7463 for (vmcip = mip->mi_clients_list; vmcip != NULL; 7464 vmcip = vmcip->mci_client_next) { 7465 if (!(vmcip->mci_flent->fe_type & FLOW_PRIMARY_MAC) || 7466 mac_client_vid((mac_client_handle_t)vmcip) == 7467 VLAN_ID_NONE) { 7468 continue; 7469 } 7470 vmrp = MCIP_RESOURCE_PROPS(vmcip); 7471 7472 vmrp->mrp_nrxrings = mrp->mrp_nrxrings; 7473 if (mrp->mrp_mask & MRP_RX_RINGS) 7474 vmrp->mrp_mask |= MRP_RX_RINGS; 7475 else if (vmrp->mrp_mask & MRP_RX_RINGS) 7476 vmrp->mrp_mask &= ~MRP_RX_RINGS; 7477 7478 vmrp->mrp_ntxrings = mrp->mrp_ntxrings; 7479 if (mrp->mrp_mask & MRP_TX_RINGS) 7480 vmrp->mrp_mask |= MRP_TX_RINGS; 7481 else if (vmrp->mrp_mask & MRP_TX_RINGS) 7482 vmrp->mrp_mask &= ~MRP_TX_RINGS; 7483 7484 if (mrp->mrp_mask & MRP_RXRINGS_UNSPEC) 7485 vmrp->mrp_mask |= MRP_RXRINGS_UNSPEC; 7486 else 7487 vmrp->mrp_mask &= ~MRP_RXRINGS_UNSPEC; 7488 7489 if (mrp->mrp_mask & MRP_TXRINGS_UNSPEC) 7490 vmrp->mrp_mask |= MRP_TXRINGS_UNSPEC; 7491 else 7492 vmrp->mrp_mask &= ~MRP_TXRINGS_UNSPEC; 7493 } 7494 } 7495 7496 /* 7497 * We are adding or removing ring(s) from a group. The source for taking 7498 * rings is the default group. The destination for giving rings back is 7499 * the default group. 7500 */ 7501 int 7502 mac_group_ring_modify(mac_client_impl_t *mcip, mac_group_t *group, 7503 mac_group_t *defgrp) 7504 { 7505 mac_resource_props_t *mrp = MCIP_RESOURCE_PROPS(mcip); 7506 uint_t modify; 7507 int count; 7508 mac_ring_t *ring; 7509 mac_ring_t *next; 7510 mac_impl_t *mip = mcip->mci_mip; 7511 mac_ring_t **rings; 7512 uint_t ringcnt; 7513 int i = 0; 7514 boolean_t rx_group = group->mrg_type == MAC_RING_TYPE_RX; 7515 int start; 7516 int end; 7517 mac_group_t *tgrp; 7518 int j; 7519 int rv = 0; 7520 7521 /* 7522 * If we are asked for just a group, we give 1 ring, else 7523 * the specified number of rings. 7524 */ 7525 if (rx_group) { 7526 ringcnt = (mrp->mrp_mask & MRP_RXRINGS_UNSPEC) ? 1: 7527 mrp->mrp_nrxrings; 7528 } else { 7529 ringcnt = (mrp->mrp_mask & MRP_TXRINGS_UNSPEC) ? 1: 7530 mrp->mrp_ntxrings; 7531 } 7532 7533 /* don't allow modifying rings for a share for now. */ 7534 ASSERT(mcip->mci_share == 0); 7535 7536 if (ringcnt == group->mrg_cur_count) 7537 return (0); 7538 7539 if (group->mrg_cur_count > ringcnt) { 7540 modify = group->mrg_cur_count - ringcnt; 7541 if (rx_group) { 7542 if (mip->mi_rx_donor_grp == group) { 7543 ASSERT(mac_is_primary_client(mcip)); 7544 mip->mi_rx_donor_grp = defgrp; 7545 } else { 7546 defgrp = mip->mi_rx_donor_grp; 7547 } 7548 } 7549 ring = group->mrg_rings; 7550 rings = kmem_alloc(modify * sizeof (mac_ring_handle_t), 7551 KM_SLEEP); 7552 j = 0; 7553 for (count = 0; count < modify; count++) { 7554 next = ring->mr_next; 7555 rv = mac_group_mov_ring(mip, defgrp, ring); 7556 if (rv != 0) { 7557 /* cleanup on failure */ 7558 for (j = 0; j < count; j++) { 7559 (void) mac_group_mov_ring(mip, group, 7560 rings[j]); 7561 } 7562 break; 7563 } 7564 rings[j++] = ring; 7565 ring = next; 7566 } 7567 kmem_free(rings, modify * sizeof (mac_ring_handle_t)); 7568 return (rv); 7569 } 7570 if (ringcnt >= MAX_RINGS_PER_GROUP) 7571 return (EINVAL); 7572 7573 modify = ringcnt - group->mrg_cur_count; 7574 7575 if (rx_group) { 7576 if (group != mip->mi_rx_donor_grp) 7577 defgrp = mip->mi_rx_donor_grp; 7578 else 7579 /* 7580 * This is the donor group with all the remaining 7581 * rings. Default group now gets to be the donor 7582 */ 7583 mip->mi_rx_donor_grp = defgrp; 7584 start = 1; 7585 end = mip->mi_rx_group_count; 7586 } else { 7587 start = 0; 7588 end = mip->mi_tx_group_count - 1; 7589 } 7590 /* 7591 * If the default doesn't have any rings, lets see if we can 7592 * take rings given to an h/w client that doesn't need it. 7593 * For now, we just see if there is any one client that can donate 7594 * all the required rings. 7595 */ 7596 if (defgrp->mrg_cur_count < (modify + 1)) { 7597 for (i = start; i < end; i++) { 7598 if (rx_group) { 7599 tgrp = &mip->mi_rx_groups[i]; 7600 if (tgrp == group || tgrp->mrg_state < 7601 MAC_GROUP_STATE_RESERVED) { 7602 continue; 7603 } 7604 mcip = MAC_GROUP_ONLY_CLIENT(tgrp); 7605 if (mcip == NULL) 7606 mcip = mac_get_grp_primary(tgrp); 7607 ASSERT(mcip != NULL); 7608 mrp = MCIP_RESOURCE_PROPS(mcip); 7609 if ((mrp->mrp_mask & MRP_RX_RINGS) != 0) 7610 continue; 7611 if ((tgrp->mrg_cur_count + 7612 defgrp->mrg_cur_count) < (modify + 1)) { 7613 continue; 7614 } 7615 if (mac_rx_switch_group(mcip, tgrp, 7616 defgrp) != 0) { 7617 return (ENOSPC); 7618 } 7619 } else { 7620 tgrp = &mip->mi_tx_groups[i]; 7621 if (tgrp == group || tgrp->mrg_state < 7622 MAC_GROUP_STATE_RESERVED) { 7623 continue; 7624 } 7625 mcip = MAC_GROUP_ONLY_CLIENT(tgrp); 7626 if (mcip == NULL) 7627 mcip = mac_get_grp_primary(tgrp); 7628 mrp = MCIP_RESOURCE_PROPS(mcip); 7629 if ((mrp->mrp_mask & MRP_TX_RINGS) != 0) 7630 continue; 7631 if ((tgrp->mrg_cur_count + 7632 defgrp->mrg_cur_count) < (modify + 1)) { 7633 continue; 7634 } 7635 /* OK, we can switch this to s/w */ 7636 mac_tx_client_quiesce( 7637 (mac_client_handle_t)mcip); 7638 mac_tx_switch_group(mcip, tgrp, defgrp); 7639 mac_tx_client_restart( 7640 (mac_client_handle_t)mcip); 7641 } 7642 } 7643 if (defgrp->mrg_cur_count < (modify + 1)) 7644 return (ENOSPC); 7645 } 7646 if ((rv = i_mac_group_allocate_rings(mip, group->mrg_type, defgrp, 7647 group, mcip->mci_share, modify)) != 0) { 7648 return (rv); 7649 } 7650 return (0); 7651 } 7652 7653 /* 7654 * Given the poolname in mac_resource_props, find the cpupart 7655 * that is associated with this pool. The cpupart will be used 7656 * later for finding the cpus to be bound to the networking threads. 7657 * 7658 * use_default is set B_TRUE if pools are enabled and pool_default 7659 * is returned. This avoids a 2nd lookup to set the poolname 7660 * for pool-effective. 7661 * 7662 * returns: 7663 * 7664 * NULL - pools are disabled or if the 'cpus' property is set. 7665 * cpupart of pool_default - pools are enabled and the pool 7666 * is not available or poolname is blank 7667 * cpupart of named pool - pools are enabled and the pool 7668 * is available. 7669 */ 7670 cpupart_t * 7671 mac_pset_find(mac_resource_props_t *mrp, boolean_t *use_default) 7672 { 7673 pool_t *pool; 7674 cpupart_t *cpupart; 7675 7676 *use_default = B_FALSE; 7677 7678 /* CPUs property is set */ 7679 if (mrp->mrp_mask & MRP_CPUS) 7680 return (NULL); 7681 7682 ASSERT(pool_lock_held()); 7683 7684 /* Pools are disabled, no pset */ 7685 if (pool_state == POOL_DISABLED) 7686 return (NULL); 7687 7688 /* Pools property is set */ 7689 if (mrp->mrp_mask & MRP_POOL) { 7690 if ((pool = pool_lookup_pool_by_name(mrp->mrp_pool)) == NULL) { 7691 /* Pool not found */ 7692 DTRACE_PROBE1(mac_pset_find_no_pool, char *, 7693 mrp->mrp_pool); 7694 *use_default = B_TRUE; 7695 pool = pool_default; 7696 } 7697 /* Pools property is not set */ 7698 } else { 7699 *use_default = B_TRUE; 7700 pool = pool_default; 7701 } 7702 7703 /* Find the CPU pset that corresponds to the pool */ 7704 mutex_enter(&cpu_lock); 7705 if ((cpupart = cpupart_find(pool->pool_pset->pset_id)) == NULL) { 7706 DTRACE_PROBE1(mac_find_pset_no_pset, psetid_t, 7707 pool->pool_pset->pset_id); 7708 } 7709 mutex_exit(&cpu_lock); 7710 7711 return (cpupart); 7712 } 7713 7714 void 7715 mac_set_pool_effective(boolean_t use_default, cpupart_t *cpupart, 7716 mac_resource_props_t *mrp, mac_resource_props_t *emrp) 7717 { 7718 ASSERT(pool_lock_held()); 7719 7720 if (cpupart != NULL) { 7721 emrp->mrp_mask |= MRP_POOL; 7722 if (use_default) { 7723 (void) strcpy(emrp->mrp_pool, 7724 "pool_default"); 7725 } else { 7726 ASSERT(strlen(mrp->mrp_pool) != 0); 7727 (void) strcpy(emrp->mrp_pool, 7728 mrp->mrp_pool); 7729 } 7730 } else { 7731 emrp->mrp_mask &= ~MRP_POOL; 7732 bzero(emrp->mrp_pool, MAXPATHLEN); 7733 } 7734 } 7735 7736 struct mac_pool_arg { 7737 char mpa_poolname[MAXPATHLEN]; 7738 pool_event_t mpa_what; 7739 }; 7740 7741 /*ARGSUSED*/ 7742 static uint_t 7743 mac_pool_link_update(mod_hash_key_t key, mod_hash_val_t *val, void *arg) 7744 { 7745 struct mac_pool_arg *mpa = arg; 7746 mac_impl_t *mip = (mac_impl_t *)val; 7747 mac_client_impl_t *mcip; 7748 mac_resource_props_t *mrp, *emrp; 7749 boolean_t pool_update = B_FALSE; 7750 boolean_t pool_clear = B_FALSE; 7751 boolean_t use_default = B_FALSE; 7752 cpupart_t *cpupart = NULL; 7753 7754 mrp = kmem_zalloc(sizeof (*mrp), KM_SLEEP); 7755 i_mac_perim_enter(mip); 7756 for (mcip = mip->mi_clients_list; mcip != NULL; 7757 mcip = mcip->mci_client_next) { 7758 pool_update = B_FALSE; 7759 pool_clear = B_FALSE; 7760 use_default = B_FALSE; 7761 mac_client_get_resources((mac_client_handle_t)mcip, mrp); 7762 emrp = MCIP_EFFECTIVE_PROPS(mcip); 7763 7764 /* 7765 * When pools are enabled 7766 */ 7767 if ((mpa->mpa_what == POOL_E_ENABLE) && 7768 ((mrp->mrp_mask & MRP_CPUS) == 0)) { 7769 mrp->mrp_mask |= MRP_POOL; 7770 pool_update = B_TRUE; 7771 } 7772 7773 /* 7774 * When pools are disabled 7775 */ 7776 if ((mpa->mpa_what == POOL_E_DISABLE) && 7777 ((mrp->mrp_mask & MRP_CPUS) == 0)) { 7778 mrp->mrp_mask |= MRP_POOL; 7779 pool_clear = B_TRUE; 7780 } 7781 7782 /* 7783 * Look for links with the pool property set and the poolname 7784 * matching the one which is changing. 7785 */ 7786 if (strcmp(mrp->mrp_pool, mpa->mpa_poolname) == 0) { 7787 /* 7788 * The pool associated with the link has changed. 7789 */ 7790 if (mpa->mpa_what == POOL_E_CHANGE) { 7791 mrp->mrp_mask |= MRP_POOL; 7792 pool_update = B_TRUE; 7793 } 7794 } 7795 7796 /* 7797 * This link is associated with pool_default and 7798 * pool_default has changed. 7799 */ 7800 if ((mpa->mpa_what == POOL_E_CHANGE) && 7801 (strcmp(emrp->mrp_pool, "pool_default") == 0) && 7802 (strcmp(mpa->mpa_poolname, "pool_default") == 0)) { 7803 mrp->mrp_mask |= MRP_POOL; 7804 pool_update = B_TRUE; 7805 } 7806 7807 /* 7808 * Get new list of cpus for the pool, bind network 7809 * threads to new list of cpus and update resources. 7810 */ 7811 if (pool_update) { 7812 if (MCIP_DATAPATH_SETUP(mcip)) { 7813 pool_lock(); 7814 cpupart = mac_pset_find(mrp, &use_default); 7815 mac_fanout_setup(mcip, mcip->mci_flent, mrp, 7816 mac_rx_deliver, mcip, NULL, cpupart); 7817 mac_set_pool_effective(use_default, cpupart, 7818 mrp, emrp); 7819 pool_unlock(); 7820 } 7821 mac_update_resources(mrp, MCIP_RESOURCE_PROPS(mcip), 7822 B_FALSE); 7823 } 7824 7825 /* 7826 * Clear the effective pool and bind network threads 7827 * to any available CPU. 7828 */ 7829 if (pool_clear) { 7830 if (MCIP_DATAPATH_SETUP(mcip)) { 7831 emrp->mrp_mask &= ~MRP_POOL; 7832 bzero(emrp->mrp_pool, MAXPATHLEN); 7833 mac_fanout_setup(mcip, mcip->mci_flent, mrp, 7834 mac_rx_deliver, mcip, NULL, NULL); 7835 } 7836 mac_update_resources(mrp, MCIP_RESOURCE_PROPS(mcip), 7837 B_FALSE); 7838 } 7839 } 7840 i_mac_perim_exit(mip); 7841 kmem_free(mrp, sizeof (*mrp)); 7842 return (MH_WALK_CONTINUE); 7843 } 7844 7845 static void 7846 mac_pool_update(void *arg) 7847 { 7848 mod_hash_walk(i_mac_impl_hash, mac_pool_link_update, arg); 7849 kmem_free(arg, sizeof (struct mac_pool_arg)); 7850 } 7851 7852 /* 7853 * Callback function to be executed when a noteworthy pool event 7854 * takes place. 7855 */ 7856 /* ARGSUSED */ 7857 static void 7858 mac_pool_event_cb(pool_event_t what, poolid_t id, void *arg) 7859 { 7860 pool_t *pool; 7861 char *poolname = NULL; 7862 struct mac_pool_arg *mpa; 7863 7864 pool_lock(); 7865 mpa = kmem_zalloc(sizeof (struct mac_pool_arg), KM_SLEEP); 7866 7867 switch (what) { 7868 case POOL_E_ENABLE: 7869 case POOL_E_DISABLE: 7870 break; 7871 7872 case POOL_E_CHANGE: 7873 pool = pool_lookup_pool_by_id(id); 7874 if (pool == NULL) { 7875 kmem_free(mpa, sizeof (struct mac_pool_arg)); 7876 pool_unlock(); 7877 return; 7878 } 7879 pool_get_name(pool, &poolname); 7880 (void) strlcpy(mpa->mpa_poolname, poolname, 7881 sizeof (mpa->mpa_poolname)); 7882 break; 7883 7884 default: 7885 kmem_free(mpa, sizeof (struct mac_pool_arg)); 7886 pool_unlock(); 7887 return; 7888 } 7889 pool_unlock(); 7890 7891 mpa->mpa_what = what; 7892 7893 mac_pool_update(mpa); 7894 } 7895 7896 /* 7897 * Set effective rings property. This could be called from datapath_setup/ 7898 * datapath_teardown or set-linkprop. 7899 * If the group is reserved we just go ahead and set the effective rings. 7900 * Additionally, for TX this could mean the default group has lost/gained 7901 * some rings, so if the default group is reserved, we need to adjust the 7902 * effective rings for the default group clients. For RX, if we are working 7903 * with the non-default group, we just need * to reset the effective props 7904 * for the default group clients. 7905 */ 7906 void 7907 mac_set_rings_effective(mac_client_impl_t *mcip) 7908 { 7909 mac_impl_t *mip = mcip->mci_mip; 7910 mac_group_t *grp; 7911 mac_group_t *defgrp; 7912 flow_entry_t *flent = mcip->mci_flent; 7913 mac_resource_props_t *emrp = MCIP_EFFECTIVE_PROPS(mcip); 7914 mac_grp_client_t *mgcp; 7915 mac_client_impl_t *gmcip; 7916 7917 grp = flent->fe_rx_ring_group; 7918 if (grp != NULL) { 7919 defgrp = MAC_DEFAULT_RX_GROUP(mip); 7920 /* 7921 * If we have reserved a group, set the effective rings 7922 * to the ring count in the group. 7923 */ 7924 if (grp->mrg_state == MAC_GROUP_STATE_RESERVED) { 7925 emrp->mrp_mask |= MRP_RX_RINGS; 7926 emrp->mrp_nrxrings = grp->mrg_cur_count; 7927 } 7928 7929 /* 7930 * We go through the clients in the shared group and 7931 * reset the effective properties. It is possible this 7932 * might have already been done for some client (i.e. 7933 * if some client is being moved to a group that is 7934 * already shared). The case where the default group is 7935 * RESERVED is taken care of above (note in the RX side if 7936 * there is a non-default group, the default group is always 7937 * SHARED). 7938 */ 7939 if (grp != defgrp || grp->mrg_state == MAC_GROUP_STATE_SHARED) { 7940 if (grp->mrg_state == MAC_GROUP_STATE_SHARED) 7941 mgcp = grp->mrg_clients; 7942 else 7943 mgcp = defgrp->mrg_clients; 7944 while (mgcp != NULL) { 7945 gmcip = mgcp->mgc_client; 7946 emrp = MCIP_EFFECTIVE_PROPS(gmcip); 7947 if (emrp->mrp_mask & MRP_RX_RINGS) { 7948 emrp->mrp_mask &= ~MRP_RX_RINGS; 7949 emrp->mrp_nrxrings = 0; 7950 } 7951 mgcp = mgcp->mgc_next; 7952 } 7953 } 7954 } 7955 7956 /* Now the TX side */ 7957 grp = flent->fe_tx_ring_group; 7958 if (grp != NULL) { 7959 defgrp = MAC_DEFAULT_TX_GROUP(mip); 7960 7961 if (grp->mrg_state == MAC_GROUP_STATE_RESERVED) { 7962 emrp->mrp_mask |= MRP_TX_RINGS; 7963 emrp->mrp_ntxrings = grp->mrg_cur_count; 7964 } else if (grp->mrg_state == MAC_GROUP_STATE_SHARED) { 7965 mgcp = grp->mrg_clients; 7966 while (mgcp != NULL) { 7967 gmcip = mgcp->mgc_client; 7968 emrp = MCIP_EFFECTIVE_PROPS(gmcip); 7969 if (emrp->mrp_mask & MRP_TX_RINGS) { 7970 emrp->mrp_mask &= ~MRP_TX_RINGS; 7971 emrp->mrp_ntxrings = 0; 7972 } 7973 mgcp = mgcp->mgc_next; 7974 } 7975 } 7976 7977 /* 7978 * If the group is not the default group and the default 7979 * group is reserved, the ring count in the default group 7980 * might have changed, update it. 7981 */ 7982 if (grp != defgrp && 7983 defgrp->mrg_state == MAC_GROUP_STATE_RESERVED) { 7984 gmcip = MAC_GROUP_ONLY_CLIENT(defgrp); 7985 emrp = MCIP_EFFECTIVE_PROPS(gmcip); 7986 emrp->mrp_ntxrings = defgrp->mrg_cur_count; 7987 } 7988 } 7989 emrp = MCIP_EFFECTIVE_PROPS(mcip); 7990 } 7991 7992 /* 7993 * Check if the primary is in the default group. If so, see if we 7994 * can give it a an exclusive group now that another client is 7995 * being configured. We take the primary out of the default group 7996 * because the multicast/broadcast packets for the all the clients 7997 * will land in the default ring in the default group which means 7998 * any client in the default group, even if it is the only on in 7999 * the group, will lose exclusive access to the rings, hence 8000 * polling. 8001 */ 8002 mac_client_impl_t * 8003 mac_check_primary_relocation(mac_client_impl_t *mcip, boolean_t rxhw) 8004 { 8005 mac_impl_t *mip = mcip->mci_mip; 8006 mac_group_t *defgrp = MAC_DEFAULT_RX_GROUP(mip); 8007 flow_entry_t *flent = mcip->mci_flent; 8008 mac_resource_props_t *mrp = MCIP_RESOURCE_PROPS(mcip); 8009 uint8_t *mac_addr; 8010 mac_group_t *ngrp; 8011 8012 /* 8013 * Check if the primary is in the default group, if not 8014 * or if it is explicitly configured to be in the default 8015 * group OR set the RX rings property, return. 8016 */ 8017 if (flent->fe_rx_ring_group != defgrp || mrp->mrp_mask & MRP_RX_RINGS) 8018 return (NULL); 8019 8020 /* 8021 * If the new client needs an exclusive group and we 8022 * don't have another for the primary, return. 8023 */ 8024 if (rxhw && mip->mi_rxhwclnt_avail < 2) 8025 return (NULL); 8026 8027 mac_addr = flent->fe_flow_desc.fd_dst_mac; 8028 /* 8029 * We call this when we are setting up the datapath for 8030 * the first non-primary. 8031 */ 8032 ASSERT(mip->mi_nactiveclients == 2); 8033 /* 8034 * OK, now we have the primary that needs to be relocated. 8035 */ 8036 ngrp = mac_reserve_rx_group(mcip, mac_addr, B_TRUE); 8037 if (ngrp == NULL) 8038 return (NULL); 8039 if (mac_rx_switch_group(mcip, defgrp, ngrp) != 0) { 8040 mac_stop_group(ngrp); 8041 return (NULL); 8042 } 8043 return (mcip); 8044 } 8045 8046 void 8047 mac_transceiver_init(mac_impl_t *mip) 8048 { 8049 if (mac_capab_get((mac_handle_t)mip, MAC_CAPAB_TRANSCEIVER, 8050 &mip->mi_transceiver)) { 8051 /* 8052 * The driver set a flag that we don't know about. In this case, 8053 * we need to warn about that case and ignore this capability. 8054 */ 8055 if (mip->mi_transceiver.mct_flags != 0) { 8056 dev_err(mip->mi_dip, CE_WARN, "driver set transceiver " 8057 "flags to invalid value: 0x%x, ignoring " 8058 "capability", mip->mi_transceiver.mct_flags); 8059 bzero(&mip->mi_transceiver, 8060 sizeof (mac_capab_transceiver_t)); 8061 } 8062 } else { 8063 bzero(&mip->mi_transceiver, 8064 sizeof (mac_capab_transceiver_t)); 8065 } 8066 } 8067 8068 int 8069 mac_transceiver_count(mac_handle_t mh, uint_t *countp) 8070 { 8071 mac_impl_t *mip = (mac_impl_t *)mh; 8072 8073 ASSERT(MAC_PERIM_HELD(mh)); 8074 8075 if (mip->mi_transceiver.mct_ntransceivers == 0) 8076 return (ENOTSUP); 8077 8078 *countp = mip->mi_transceiver.mct_ntransceivers; 8079 return (0); 8080 } 8081 8082 int 8083 mac_transceiver_info(mac_handle_t mh, uint_t tranid, boolean_t *present, 8084 boolean_t *usable) 8085 { 8086 int ret; 8087 mac_transceiver_info_t info; 8088 8089 mac_impl_t *mip = (mac_impl_t *)mh; 8090 8091 ASSERT(MAC_PERIM_HELD(mh)); 8092 8093 if (mip->mi_transceiver.mct_info == NULL || 8094 mip->mi_transceiver.mct_ntransceivers == 0) 8095 return (ENOTSUP); 8096 8097 if (tranid >= mip->mi_transceiver.mct_ntransceivers) 8098 return (EINVAL); 8099 8100 bzero(&info, sizeof (mac_transceiver_info_t)); 8101 if ((ret = mip->mi_transceiver.mct_info(mip->mi_driver, tranid, 8102 &info)) != 0) { 8103 return (ret); 8104 } 8105 8106 *present = info.mti_present; 8107 *usable = info.mti_usable; 8108 return (0); 8109 } 8110 8111 int 8112 mac_transceiver_read(mac_handle_t mh, uint_t tranid, uint_t page, void *buf, 8113 size_t nbytes, off_t offset, size_t *nread) 8114 { 8115 int ret; 8116 size_t nr; 8117 mac_impl_t *mip = (mac_impl_t *)mh; 8118 8119 ASSERT(MAC_PERIM_HELD(mh)); 8120 8121 if (mip->mi_transceiver.mct_read == NULL) 8122 return (ENOTSUP); 8123 8124 if (tranid >= mip->mi_transceiver.mct_ntransceivers) 8125 return (EINVAL); 8126 8127 /* 8128 * All supported pages today are 256 bytes wide. Make sure offset + 8129 * nbytes never exceeds that. 8130 */ 8131 if (offset < 0 || offset >= 256 || nbytes > 256 || 8132 offset + nbytes > 256) 8133 return (EINVAL); 8134 8135 if (nread == NULL) 8136 nread = &nr; 8137 ret = mip->mi_transceiver.mct_read(mip->mi_driver, tranid, page, buf, 8138 nbytes, offset, nread); 8139 if (ret == 0 && *nread > nbytes) { 8140 dev_err(mip->mi_dip, CE_PANIC, "driver wrote %lu bytes into " 8141 "%lu byte sized buffer, possible memory corruption", 8142 *nread, nbytes); 8143 } 8144 8145 return (ret); 8146 } 8147 8148 void 8149 mac_led_init(mac_impl_t *mip) 8150 { 8151 mip->mi_led_modes = MAC_LED_DEFAULT; 8152 8153 if (!mac_capab_get((mac_handle_t)mip, MAC_CAPAB_LED, &mip->mi_led)) { 8154 bzero(&mip->mi_led, sizeof (mac_capab_led_t)); 8155 return; 8156 } 8157 8158 if (mip->mi_led.mcl_flags != 0) { 8159 dev_err(mip->mi_dip, CE_WARN, "driver set led capability " 8160 "flags to invalid value: 0x%x, ignoring " 8161 "capability", mip->mi_transceiver.mct_flags); 8162 bzero(&mip->mi_led, sizeof (mac_capab_led_t)); 8163 return; 8164 } 8165 8166 if ((mip->mi_led.mcl_modes & ~MAC_LED_ALL) != 0) { 8167 dev_err(mip->mi_dip, CE_WARN, "driver set led capability " 8168 "supported modes to invalid value: 0x%x, ignoring " 8169 "capability", mip->mi_transceiver.mct_flags); 8170 bzero(&mip->mi_led, sizeof (mac_capab_led_t)); 8171 return; 8172 } 8173 } 8174 8175 int 8176 mac_led_get(mac_handle_t mh, mac_led_mode_t *supported, mac_led_mode_t *active) 8177 { 8178 mac_impl_t *mip = (mac_impl_t *)mh; 8179 8180 ASSERT(MAC_PERIM_HELD(mh)); 8181 8182 if (mip->mi_led.mcl_set == NULL) 8183 return (ENOTSUP); 8184 8185 *supported = mip->mi_led.mcl_modes; 8186 *active = mip->mi_led_modes; 8187 8188 return (0); 8189 } 8190 8191 /* 8192 * Update and multiplex the various LED requests. We only ever send one LED to 8193 * the underlying driver at a time. As such, we end up multiplexing all 8194 * requested states and picking one to send down to the driver. 8195 */ 8196 int 8197 mac_led_set(mac_handle_t mh, mac_led_mode_t desired) 8198 { 8199 int ret; 8200 mac_led_mode_t driver; 8201 8202 mac_impl_t *mip = (mac_impl_t *)mh; 8203 8204 ASSERT(MAC_PERIM_HELD(mh)); 8205 8206 /* 8207 * If we've been passed a desired value of zero, that indicates that 8208 * we're basically resetting to the value of zero, which is our default 8209 * value. 8210 */ 8211 if (desired == 0) 8212 desired = MAC_LED_DEFAULT; 8213 8214 if (mip->mi_led.mcl_set == NULL) 8215 return (ENOTSUP); 8216 8217 /* 8218 * Catch both values that we don't know about and those that the driver 8219 * doesn't support. 8220 */ 8221 if ((desired & ~MAC_LED_ALL) != 0) 8222 return (EINVAL); 8223 8224 if ((desired & ~mip->mi_led.mcl_modes) != 0) 8225 return (ENOTSUP); 8226 8227 /* 8228 * If we have the same value, then there is nothing to do. 8229 */ 8230 if (desired == mip->mi_led_modes) 8231 return (0); 8232 8233 /* 8234 * Based on the desired value, determine what to send to the driver. We 8235 * only will send a single bit to the driver at any given time. IDENT 8236 * takes priority over OFF or ON. We also let OFF take priority over the 8237 * rest. 8238 */ 8239 if (desired & MAC_LED_IDENT) { 8240 driver = MAC_LED_IDENT; 8241 } else if (desired & MAC_LED_OFF) { 8242 driver = MAC_LED_OFF; 8243 } else if (desired & MAC_LED_ON) { 8244 driver = MAC_LED_ON; 8245 } else { 8246 driver = MAC_LED_DEFAULT; 8247 } 8248 8249 if ((ret = mip->mi_led.mcl_set(mip->mi_driver, driver, 0)) == 0) { 8250 mip->mi_led_modes = desired; 8251 } 8252 8253 return (ret); 8254 } 8255