1.\" $NetBSD: altq.9,v 1.8 2002/05/28 11:41:45 wiz Exp $ 2.\" $OpenBSD: altq.9,v 1.4 2001/07/12 12:41:42 itojun Exp $ 3.\" 4.\" Copyright (C) 2004 Max Laier. All rights reserved. 5.\" Copyright (C) 2001 6.\" Sony Computer Science Laboratories Inc. All rights reserved. 7.\" 8.\" Redistribution and use in source and binary forms, with or without 9.\" modification, are permitted provided that the following conditions 10.\" are met: 11.\" 1. Redistributions of source code must retain the above copyright 12.\" notice, this list of conditions and the following disclaimer. 13.\" 2. Redistributions in binary form must reproduce the above copyright 14.\" notice, this list of conditions and the following disclaimer in the 15.\" documentation and/or other materials provided with the distribution. 16.\" 17.\" THIS SOFTWARE IS PROVIDED BY SONY CSL AND CONTRIBUTORS ``AS IS'' AND 18.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 19.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 20.\" ARE DISCLAIMED. IN NO EVENT SHALL SONY CSL OR CONTRIBUTORS BE LIABLE 21.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 23.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 24.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 25.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 26.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 27.\" SUCH DAMAGE. 28.\" 29.\" $FreeBSD$ 30.\" 31.Dd August 25, 2004 32.Dt ALTQ 9 33.Os 34.\" 35.Sh NAME 36.Nm ALTQ 37.Nd kernel interfaces for manipulating output queues on network interfaces 38.Sh SYNOPSIS 39.In sys/types.h 40.In sys/socket.h 41.In net/if.h 42.In net/if_var.h 43.\" 44.Ss Enqueue macros 45.Fn IFQ_ENQUEUE "struct ifaltq *ifq" "struct mbuf *m" "int error" 46.Fn IFQ_HANDOFF "struct ifnet *ifp" "struct mbuf *m" "int error" 47.Fo IFQ_HANDOFF_ADJ 48.Fa "struct ifnet *ifp" "struct mbuf *m" "int adjust" "int error" 49.Fc 50.\" 51.Ss Dequeue macros 52.Fn IFQ_DEQUEUE "struct ifaltq *ifq" "struct mbuf *m" 53.Fn IFQ_POLL_NOLOCK "struct ifaltq *ifq" "struct mbuf *m" 54.Fn IFQ_PURGE "struct ifaltq *ifq" 55.Fn IFQ_IS_EMPTY "struct ifaltq *ifq" 56.\" 57.Ss Driver managed dequeue macros 58.Fn IFQ_DRV_DEQUEUE "struct ifaltq *ifq" "struct mbuf *m" 59.Fn IFQ_DRV_PREPEND "struct ifaltq *ifq" "struct mbuf *m" 60.Fn IFQ_DRV_PURGE "struct ifaltq *ifq" 61.Fn IFQ_DRV_IS_EMPTY "struct ifaltq *ifq" 62.\" 63.Ss General setup macros 64.Fn IFQ_SET_MAXLEN "struct ifaltq *ifq" "int len" 65.Fn IFQ_INC_LEN "struct ifaltq *ifq" 66.Fn IFQ_DEC_LEN "struct ifaltq *ifq" 67.Fn IFQ_INC_DROPS "struct ifaltq *ifq" 68.Fn IFQ_SET_READY "struct ifaltq *ifq" 69.Sh DESCRIPTION 70The 71.Nm 72system is a framework to manage queuing disciplines on network 73interfaces. 74.Nm 75introduces new macros to manipulate output queues. 76The output queue macros are used to abstract queue operations and not to 77touch the internal fields of the output queue structure. 78The macros are independent from the 79.Nm 80implementation, and compatible with the traditional 81.Vt ifqueue 82macros for ease of transition. 83.Pp 84.Fn IFQ_ENQUEUE , 85.Fn IFQ_HANDOFF 86and 87.Fn IFQ_HANDOFF_ADJ 88enqueue a packet 89.Fa m 90to the queue 91.Fa ifq . 92The underlying queuing discipline may discard the packet. 93The 94.Fa error 95argument is set to 0 on success, or 96.Er ENOBUFS 97if the packet is discarded. 98The packet pointed to by 99.Fa m 100will be freed by the device driver on success, or by the queuing discipline on 101failure, so the caller should not touch 102.Fa m 103after enqueuing. 104.Fn IFQ_HANDOFF 105and 106.Fn IFQ_HANDOFF_ADJ 107combine the enqueue operation with statistic generation and call 108.Fn if_start 109upon successful enqueue to initiate the actual send. 110.Pp 111.Fn IFQ_DEQUEUE 112dequeues a packet from the queue. 113The dequeued packet is returned in 114.Fa m , 115or 116.Fa m 117is set to 118.Dv NULL 119if no packet is dequeued. 120The caller must always check 121.Fa m 122since a non-empty queue could return 123.Dv NULL 124under rate-limiting. 125.Pp 126.Fn IFQ_POLL_NOLOCK 127returns the next packet without removing it from the queue. 128The caller must hold the queue mutex when calling 129.Fn IFQ_POLL_NOLOCK 130in order to guarantee that a subsequent call to 131.Fn IFQ_DEQUEUE_NOLOCK 132dequeues the same packet. 133.Pp 134.Fn IFQ_*_NOLOCK 135variants (if available) always assume that the caller holds the queue mutex. 136They can be grabbed with 137.Fn IFQ_LOCK 138and released with 139.Fn IFQ_UNLOCK . 140.Pp 141.Fn IFQ_PURGE 142discards all the packets in the queue. 143The purge operation is needed since a non-work conserving queue cannot be 144emptied by a dequeue loop. 145.Pp 146.Fn IFQ_IS_EMPTY 147can be used to check if the queue is empty. 148Note that 149.Fn IFQ_DEQUEUE 150could still return 151.Dv NULL 152if the queuing discipline is non-work conserving. 153.Pp 154.Fn IFQ_DRV_DEQUEUE 155moves up to 156.Fa ifq->ifq_drv_maxlen 157packets from the queue to the 158.Dq "driver managed" 159queue and returns the first one via 160.Fa m . 161As for 162.Fn IFQ_DEQUEUE , 163.Fa m 164can be 165.Dv NULL 166even for a non-empty queue. 167Subsequent calls to 168.Fn IFQ_DRV_DEQUEUE 169pass the packets from the 170.Dq "driver managed" 171queue without obtaining the queue mutex. 172It is the responsibility of the caller to protect against concurrent access. 173Enabling 174.Nm 175for a given queue sets 176.Va ifq_drv_maxlen 177to 0 as the 178.Dq "bulk dequeue" 179performed by 180.Fn IFQ_DRV_DEQUEUE 181for higher values of 182.Va ifq_drv_maxlen 183is adverse to 184.Nm ALTQ Ns 's 185internal timing. 186Note that a driver must not mix 187.Fn IFQ_DRV_* 188macros with the default dequeue macros as the default macros do not look at the 189.Dq "driver managed" 190queue which might lead to an mbuf leak. 191.Pp 192.Fn IFQ_DRV_PREPEND 193prepends 194.Fa m 195to the 196.Dq "driver managed" 197queue from where it will be obtained with the next call to 198.Fn IFQ_DRV_DEQUEUE . 199.Pp 200.Fn IFQ_DRV_PURGE 201flushes all packets in the 202.Dq "driver managed" 203queue and calls to 204.Fn IFQ_PURGE 205afterwards. 206.Pp 207.Fn IFQ_DRV_IS_EMPTY 208checks for packets in the 209.Dq "driver managed" 210part of the queue. 211If it is empty, it forwards to 212.Fn IFQ_IS_EMPTY . 213.Pp 214.Fn IFQ_SET_MAXLEN 215sets the queue length limit to the default FIFO queue. 216The 217.Va ifq_drv_maxlen 218member of the 219.Vt ifaltq 220structure controls the length limit of the 221.Dq "driver managed" 222queue. 223.Pp 224.Fn IFQ_INC_LEN 225and 226.Fn IFQ_DEC_LEN 227increment or decrement the current queue length in packets. 228This is mostly for internal purposes. 229.Pp 230.Fn IFQ_INC_DROPS 231increments the drop counter and is identical to 232.Fn IF_DROP . 233It is defined for naming consistency only. 234.Pp 235.Fn IFQ_SET_READY 236sets a flag to indicate that a driver was converted to use the new macros. 237.Nm 238can be enabled only on interfaces with this flag. 239.Sh COMPATIBILITY 240.Ss Vt ifaltq structure 241In order to keep compatibility with the existing code, the new 242output queue structure 243.Vt ifaltq 244has the same fields. 245The traditional 246.Fn IF_* 247macros and the code directly referencing the fields within 248.Va if_snd 249still work with 250.Vt ifaltq . 251.Bd -literal 252 ##old-style## ##new-style## 253 | 254 struct ifqueue { | struct ifaltq { 255 struct mbuf *ifq_head; | struct mbuf *ifq_head; 256 struct mbuf *ifq_tail; | struct mbuf *ifq_tail; 257 int ifq_len; | int ifq_len; 258 int ifq_maxlen; | int ifq_maxlen; 259 }; | /* driver queue fields */ 260 | ...... 261 | /* altq related fields */ 262 | ...... 263 | }; 264 | 265.Ed 266The new structure replaces 267.Vt "struct ifqueue" 268in 269.Vt "struct ifnet" . 270.Bd -literal 271 ##old-style## ##new-style## 272 | 273 struct ifnet { | struct ifnet { 274 .... | .... 275 | 276 struct ifqueue if_snd; | struct ifaltq if_snd; 277 | 278 .... | .... 279 }; | }; 280 | 281.Ed 282The (simplified) new 283.Fn IFQ_* 284macros look like: 285.Bd -literal 286 #define IFQ_DEQUEUE(ifq, m) \e 287 if (ALTQ_IS_ENABLED((ifq)) \e 288 ALTQ_DEQUEUE((ifq), (m)); \e 289 else \e 290 IF_DEQUEUE((ifq), (m)); 291.Ed 292.Ss Enqueue operation 293The semantics of the enqueue operation is changed. 294In the new style, 295enqueue and packet drop are combined since they cannot be easily 296separated in many queuing disciplines. 297The new enqueue operation corresponds to the following macro that is 298written with the old macros. 299.Bd -literal 300#define IFQ_ENQUEUE(ifq, m, error) \e 301do { \e 302 if (IF_QFULL((ifq))) { \e 303 m_freem((m)); \e 304 (error) = ENOBUFS; \e 305 IF_DROP(ifq); \e 306 } else { \e 307 IF_ENQUEUE((ifq), (m)); \e 308 (error) = 0; \e 309 } \e 310} while (0) 311.Ed 312.Pp 313.Fn IFQ_ENQUEUE 314does the following: 315.Pp 316.Bl -hyphen -compact 317.It 318queue a packet, 319.It 320drop (and free) a packet if the enqueue operation fails. 321.El 322.Pp 323If the enqueue operation fails, 324.Fa error 325is set to 326.Er ENOBUFS . 327The 328.Fa m 329mbuf is freed by the queuing discipline. 330The caller should not touch mbuf after calling 331.Fn IFQ_ENQUEUE 332so that the caller may need to copy 333.Va m_pkthdr.len 334or 335.Va m_flags 336field beforehand for statistics. 337.Fn IFQ_HANDOFF 338and 339.Fn IFQ_HANDOFF_ADJ 340can be used if only default interface statistics and an immediate call to 341.Fn if_start 342are desired. 343The caller should not use 344.Fn senderr 345since mbuf was already freed. 346.Pp 347The new style 348.Fn if_output 349looks as follows: 350.Bd -literal 351 ##old-style## ##new-style## 352 | 353 int | int 354 ether_output(ifp, m0, dst, rt0) | ether_output(ifp, m0, dst, rt0) 355 { | { 356 ...... | ...... 357 | 358 | mflags = m->m_flags; 359 | len = m->m_pkthdr.len; 360 s = splimp(); | s = splimp(); 361 if (IF_QFULL(&ifp->if_snd)) { | IFQ_ENQUEUE(&ifp->if_snd, m, 362 | error); 363 IF_DROP(&ifp->if_snd); | if (error != 0) { 364 splx(s); | splx(s); 365 senderr(ENOBUFS); | return (error); 366 } | } 367 IF_ENQUEUE(&ifp->if_snd, m); | 368 ifp->if_obytes += | ifp->if_obytes += len; 369 m->m_pkthdr.len; | 370 if (m->m_flags & M_MCAST) | if (mflags & M_MCAST) 371 ifp->if_omcasts++; | ifp->if_omcasts++; 372 | 373 if ((ifp->if_flags & IFF_OACTIVE) | if ((ifp->if_flags & IFF_OACTIVE) 374 == 0) | == 0) 375 (*ifp->if_start)(ifp); | (*ifp->if_start)(ifp); 376 splx(s); | splx(s); 377 return (error); | return (error); 378 | 379 bad: | bad: 380 if (m) | if (m) 381 m_freem(m); | m_freem(m); 382 return (error); | return (error); 383 } | } 384 | 385.Ed 386.Sh HOW TO CONVERT THE EXISTING DRIVERS 387First, make sure the corresponding 388.Fn if_output 389is already converted to the new style. 390.Pp 391Look for 392.Va if_snd 393in the driver. 394Probably, you need to make changes to the lines that include 395.Va if_snd . 396.Ss Empty check operation 397If the code checks 398.Va ifq_head 399to see whether the queue is empty or not, use 400.Fn IFQ_IS_EMPTY . 401.Bd -literal 402 ##old-style## ##new-style## 403 | 404 if (ifp->if_snd.ifq_head != NULL) | if (!IFQ_IS_EMPTY(&ifp->if_snd)) 405 | 406.Ed 407.Fn IFQ_IS_EMPTY 408only checks if there is any packet stored in the queue. 409Note that even when 410.Fn IFQ_IS_EMPTY 411is 412.Dv FALSE , 413.Fn IFQ_DEQUEUE 414could still return 415.Dv NULL 416if the queue is under rate-limiting. 417.Ss Dequeue operation 418Replace 419.Fn IF_DEQUEUE 420by 421.Fn IFQ_DEQUEUE . 422Always check whether the dequeued mbuf is 423.Dv NULL 424or not. 425Note that even when 426.Fn IFQ_IS_EMPTY 427is 428.Dv FALSE , 429.Fn IFQ_DEQUEUE 430could return 431.Dv NULL 432due to rate-limiting. 433.Bd -literal 434 ##old-style## ##new-style## 435 | 436 IF_DEQUEUE(&ifp->if_snd, m); | IFQ_DEQUEUE(&ifp->if_snd, m); 437 | if (m == NULL) 438 | return; 439 | 440.Ed 441A driver is supposed to call 442.Fn if_start 443from transmission complete interrupts in order to trigger the next dequeue. 444.Ss Poll-and-dequeue operation 445If the code polls the packet at the head of the queue and actually uses 446the packet before dequeuing it, use 447.Fn IFQ_POLL_NOLOCK 448and 449.Fn IFQ_DEQUEUE_NOLOCK . 450.Bd -literal 451 ##old-style## ##new-style## 452 | 453 | IFQ_LOCK(&ifp->if_snd); 454 m = ifp->if_snd.ifq_head; | IFQ_POLL_NOLOCK(&ifp->if_snd, m); 455 if (m != NULL) { | if (m != NULL) { 456 | 457 /* use m to get resources */ | /* use m to get resources */ 458 if (something goes wrong) | if (something goes wrong) 459 | IFQ_UNLOCK(&ifp->if_snd); 460 return; | return; 461 | 462 IF_DEQUEUE(&ifp->if_snd, m); | IFQ_DEQUEUE_NOLOCK(&ifp->if_snd, m); 463 | IFQ_UNLOCK(&ifp->if_snd); 464 | 465 /* kick the hardware */ | /* kick the hardware */ 466 } | } 467 | 468.Ed 469It is guaranteed that 470.Fn IFQ_DEQUEUE_NOLOCK 471under the same lock as a previous 472.Fn IFQ_POLL_NOLOCK 473returns the same packet. 474Note that they need to be guarded by 475.Fn IFQ_LOCK . 476.Ss Eliminating Fn IF_PREPEND 477If the code uses 478.Fn IF_PREPEND , 479you have to eliminate it unless you can use a 480.Dq "driver managed" 481queue which allows the use of 482.Fn IFQ_DRV_PREPEND 483as a substitute. 484A common usage of 485.Fn IF_PREPEND 486is to cancel the previous dequeue operation. 487You have to convert the logic into poll-and-dequeue. 488.Bd -literal 489 ##old-style## ##new-style## 490 | 491 | IFQ_LOCK(&ifp->if_snd); 492 IF_DEQUEUE(&ifp->if_snd, m); | IFQ_POLL_NOLOCK(&ifp->if_snd, m); 493 if (m != NULL) { | if (m != NULL) { 494 | 495 if (something_goes_wrong) { | if (something_goes_wrong) { 496 IF_PREPEND(&ifp->if_snd, m); | IFQ_UNLOCK(&ifp->if_snd); 497 return; | return; 498 } | } 499 | 500 | /* at this point, the driver 501 | * is committed to send this 502 | * packet. 503 | */ 504 | IFQ_DEQUEUE_NOLOCK(&ifp->if_snd, m); 505 | IFQ_UNLOCK(&ifp->if_snd); 506 | 507 /* kick the hardware */ | /* kick the hardware */ 508 } | } 509 | 510.Ed 511.Ss Purge operation 512Use 513.Fn IFQ_PURGE 514to empty the queue. 515Note that a non-work conserving queue cannot be emptied by a dequeue loop. 516.Bd -literal 517 ##old-style## ##new-style## 518 | 519 while (ifp->if_snd.ifq_head != NULL) {| IFQ_PURGE(&ifp->if_snd); 520 IF_DEQUEUE(&ifp->if_snd, m); | 521 m_freem(m); | 522 } | 523 | 524.Ed 525.Ss Conversion using a driver managed queue 526Convert 527.Fn IF_* 528macros to their equivalent 529.Fn IFQ_DRV_* 530and employ 531.Fn IFQ_DRV_IS_EMPTY 532where appropriate. 533.Bd -literal 534 ##old-style## ##new-style## 535 | 536 if (ifp->if_snd.ifq_head != NULL) | if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) 537 | 538.Ed 539Make sure that calls to 540.Fn IFQ_DRV_DEQUEUE , 541.Fn IFQ_DRV_PREPEND 542and 543.Fn IFQ_DRV_PURGE 544are protected with a mutex of some kind. 545.Ss Attach routine 546Use 547.Fn IFQ_SET_MAXLEN 548to set 549.Va ifq_maxlen 550to 551.Fa len . 552Initialize 553.Va ifq_drv_maxlen 554with a sensible value if you plan to use the 555.Fn IFQ_DRV_* 556macros. 557Add 558.Fn IFQ_SET_READY 559to show this driver is converted to the new style. 560(This is used to distinguish new-style drivers.) 561.Bd -literal 562 ##old-style## ##new-style## 563 | 564 ifp->if_snd.ifq_maxlen = qsize; | IFQ_SET_MAXLEN(&ifp->if_snd, qsize); 565 | ifp->if_snd.ifq_drv_maxlen = qsize; 566 | IFQ_SET_READY(&ifp->if_snd); 567 if_attach(ifp); | if_attach(ifp); 568 | 569.Ed 570.Ss Other issues 571The new macros for statistics: 572.Bd -literal 573 ##old-style## ##new-style## 574 | 575 IF_DROP(&ifp->if_snd); | IFQ_INC_DROPS(&ifp->if_snd); 576 | 577 ifp->if_snd.ifq_len++; | IFQ_INC_LEN(&ifp->if_snd); 578 | 579 ifp->if_snd.ifq_len--; | IFQ_DEC_LEN(&ifp->if_snd); 580 | 581.Ed 582.Sh QUEUING DISCIPLINES 583Queuing disciplines need to maintain 584.Fa ifq_len 585(used by 586.Fn IFQ_IS_EMPTY ) . 587Queuing disciplines also need to guarantee that the same mbuf is returned if 588.Fn IFQ_DEQUEUE 589is called immediately after 590.Fn IFQ_POLL . 591.Sh SEE ALSO 592.Xr pf 4 , 593.Xr pf.conf 5 , 594.Xr pfctl 8 595.Sh HISTORY 596The 597.Nm 598system first appeared in March 1997. 599