1.\" $NetBSD: altq.9,v 1.8 2002/05/28 11:41:45 wiz Exp $ 2.\" $OpenBSD: altq.9,v 1.4 2001/07/12 12:41:42 itojun Exp $ 3.\" 4.\" Copyright (C) 2004 Max Laier. All rights reserved. 5.\" Copyright (C) 2001 6.\" Sony Computer Science Laboratories Inc. All rights reserved. 7.\" 8.\" Redistribution and use in source and binary forms, with or without 9.\" modification, are permitted provided that the following conditions 10.\" are met: 11.\" 1. Redistributions of source code must retain the above copyright 12.\" notice, this list of conditions and the following disclaimer. 13.\" 2. Redistributions in binary form must reproduce the above copyright 14.\" notice, this list of conditions and the following disclaimer in the 15.\" documentation and/or other materials provided with the distribution. 16.\" 17.\" THIS SOFTWARE IS PROVIDED BY SONY CSL AND CONTRIBUTORS ``AS IS'' AND 18.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 19.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 20.\" ARE DISCLAIMED. IN NO EVENT SHALL SONY CSL OR CONTRIBUTORS BE LIABLE 21.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 23.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 24.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 25.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 26.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 27.\" SUCH DAMAGE. 28.\" 29.Dd March 20, 2018 30.Dt ALTQ 9 31.Os 32.\" 33.Sh NAME 34.Nm ALTQ 35.Nd kernel interfaces for manipulating output queues on network interfaces 36.Sh SYNOPSIS 37.In sys/types.h 38.In sys/socket.h 39.In net/if.h 40.In net/if_var.h 41.\" 42.Ss Enqueue macros 43.Fn IFQ_ENQUEUE "struct ifaltq *ifq" "struct mbuf *m" "int error" 44.Fn IFQ_HANDOFF "struct ifnet *ifp" "struct mbuf *m" "int error" 45.Fo IFQ_HANDOFF_ADJ 46.Fa "struct ifnet *ifp" "struct mbuf *m" "int adjust" "int error" 47.Fc 48.\" 49.Ss Dequeue macros 50.Fn IFQ_DEQUEUE "struct ifaltq *ifq" "struct mbuf *m" 51.Fn IFQ_POLL_NOLOCK "struct ifaltq *ifq" "struct mbuf *m" 52.Fn IFQ_PURGE "struct ifaltq *ifq" 53.Fn IFQ_IS_EMPTY "struct ifaltq *ifq" 54.\" 55.Ss Driver managed dequeue macros 56.Fn IFQ_DRV_DEQUEUE "struct ifaltq *ifq" "struct mbuf *m" 57.Fn IFQ_DRV_PREPEND "struct ifaltq *ifq" "struct mbuf *m" 58.Fn IFQ_DRV_PURGE "struct ifaltq *ifq" 59.Fn IFQ_DRV_IS_EMPTY "struct ifaltq *ifq" 60.\" 61.Ss General setup macros 62.Fn IFQ_SET_MAXLEN "struct ifaltq *ifq" "int len" 63.Fn IFQ_INC_LEN "struct ifaltq *ifq" 64.Fn IFQ_DEC_LEN "struct ifaltq *ifq" 65.Fn IFQ_INC_DROPS "struct ifaltq *ifq" 66.Fn IFQ_SET_READY "struct ifaltq *ifq" 67.Sh DESCRIPTION 68The 69.Nm 70system is a framework to manage queuing disciplines on network 71interfaces. 72.Nm 73introduces new macros to manipulate output queues. 74The output queue macros are used to abstract queue operations and not to 75touch the internal fields of the output queue structure. 76The macros are independent from the 77.Nm 78implementation, and compatible with the traditional 79.Vt ifqueue 80macros for ease of transition. 81.Pp 82.Fn IFQ_ENQUEUE , 83.Fn IFQ_HANDOFF 84and 85.Fn IFQ_HANDOFF_ADJ 86enqueue a packet 87.Fa m 88to the queue 89.Fa ifq . 90The underlying queuing discipline may discard the packet. 91The 92.Fa error 93argument is set to 0 on success, or 94.Er ENOBUFS 95if the packet is discarded. 96The packet pointed to by 97.Fa m 98will be freed by the device driver on success, or by the queuing discipline on 99failure, so the caller should not touch 100.Fa m 101after enqueuing. 102.Fn IFQ_HANDOFF 103and 104.Fn IFQ_HANDOFF_ADJ 105combine the enqueue operation with statistic generation and call 106.Fn if_start 107upon successful enqueue to initiate the actual send. 108.Pp 109.Fn IFQ_DEQUEUE 110dequeues a packet from the queue. 111The dequeued packet is returned in 112.Fa m , 113or 114.Fa m 115is set to 116.Dv NULL 117if no packet is dequeued. 118The caller must always check 119.Fa m 120since a non-empty queue could return 121.Dv NULL 122under rate-limiting. 123.Pp 124.Fn IFQ_POLL_NOLOCK 125returns the next packet without removing it from the queue. 126The caller must hold the queue mutex when calling 127.Fn IFQ_POLL_NOLOCK 128in order to guarantee that a subsequent call to 129.Fn IFQ_DEQUEUE_NOLOCK 130dequeues the same packet. 131.Pp 132.Fn IFQ_*_NOLOCK 133variants (if available) always assume that the caller holds the queue mutex. 134They can be grabbed with 135.Fn IFQ_LOCK 136and released with 137.Fn IFQ_UNLOCK . 138.Pp 139.Fn IFQ_PURGE 140discards all the packets in the queue. 141The purge operation is needed since a non-work conserving queue cannot be 142emptied by a dequeue loop. 143.Pp 144.Fn IFQ_IS_EMPTY 145can be used to check if the queue is empty. 146Note that 147.Fn IFQ_DEQUEUE 148could still return 149.Dv NULL 150if the queuing discipline is non-work conserving. 151.Pp 152.Fn IFQ_DRV_DEQUEUE 153moves up to 154.Fa ifq->ifq_drv_maxlen 155packets from the queue to the 156.Dq "driver managed" 157queue and returns the first one via 158.Fa m . 159As for 160.Fn IFQ_DEQUEUE , 161.Fa m 162can be 163.Dv NULL 164even for a non-empty queue. 165Subsequent calls to 166.Fn IFQ_DRV_DEQUEUE 167pass the packets from the 168.Dq "driver managed" 169queue without obtaining the queue mutex. 170It is the responsibility of the caller to protect against concurrent access. 171Enabling 172.Nm 173for a given queue sets 174.Va ifq_drv_maxlen 175to 0 as the 176.Dq "bulk dequeue" 177performed by 178.Fn IFQ_DRV_DEQUEUE 179for higher values of 180.Va ifq_drv_maxlen 181is adverse to 182.Nm ALTQ Ns 's 183internal timing. 184Note that a driver must not mix 185.Fn IFQ_DRV_* 186macros with the default dequeue macros as the default macros do not look at the 187.Dq "driver managed" 188queue which might lead to an mbuf leak. 189.Pp 190.Fn IFQ_DRV_PREPEND 191prepends 192.Fa m 193to the 194.Dq "driver managed" 195queue from where it will be obtained with the next call to 196.Fn IFQ_DRV_DEQUEUE . 197.Pp 198.Fn IFQ_DRV_PURGE 199flushes all packets in the 200.Dq "driver managed" 201queue and calls to 202.Fn IFQ_PURGE 203afterwards. 204.Pp 205.Fn IFQ_DRV_IS_EMPTY 206checks for packets in the 207.Dq "driver managed" 208part of the queue. 209If it is empty, it forwards to 210.Fn IFQ_IS_EMPTY . 211.Pp 212.Fn IFQ_SET_MAXLEN 213sets the queue length limit to the default FIFO queue. 214The 215.Va ifq_drv_maxlen 216member of the 217.Vt ifaltq 218structure controls the length limit of the 219.Dq "driver managed" 220queue. 221.Pp 222.Fn IFQ_INC_LEN 223and 224.Fn IFQ_DEC_LEN 225increment or decrement the current queue length in packets. 226This is mostly for internal purposes. 227.Pp 228.Fn IFQ_INC_DROPS 229increments the drop counter and is identical to 230.Fn IF_DROP . 231It is defined for naming consistency only. 232.Pp 233.Fn IFQ_SET_READY 234sets a flag to indicate that a driver was converted to use the new macros. 235.Nm 236can be enabled only on interfaces with this flag. 237.Sh COMPATIBILITY 238.Ss Vt ifaltq structure 239In order to keep compatibility with the existing code, the new 240output queue structure 241.Vt ifaltq 242has the same fields. 243The traditional 244.Fn IF_* 245macros and the code directly referencing the fields within 246.Va if_snd 247still work with 248.Vt ifaltq . 249.Bd -literal 250 ##old-style## ##new-style## 251 | 252 struct ifqueue { | struct ifaltq { 253 struct mbuf *ifq_head; | struct mbuf *ifq_head; 254 struct mbuf *ifq_tail; | struct mbuf *ifq_tail; 255 int ifq_len; | int ifq_len; 256 int ifq_maxlen; | int ifq_maxlen; 257 }; | /* driver queue fields */ 258 | ...... 259 | /* altq related fields */ 260 | ...... 261 | }; 262 | 263.Ed 264The new structure replaces 265.Vt "struct ifqueue" 266in 267.Vt "struct ifnet" . 268.Bd -literal 269 ##old-style## ##new-style## 270 | 271 struct ifnet { | struct ifnet { 272 .... | .... 273 | 274 struct ifqueue if_snd; | struct ifaltq if_snd; 275 | 276 .... | .... 277 }; | }; 278 | 279.Ed 280The (simplified) new 281.Fn IFQ_* 282macros look like: 283.Bd -literal 284 #define IFQ_DEQUEUE(ifq, m) \e 285 if (ALTQ_IS_ENABLED((ifq)) \e 286 ALTQ_DEQUEUE((ifq), (m)); \e 287 else \e 288 IF_DEQUEUE((ifq), (m)); 289.Ed 290.Ss Enqueue operation 291The semantics of the enqueue operation is changed. 292In the new style, 293enqueue and packet drop are combined since they cannot be easily 294separated in many queuing disciplines. 295The new enqueue operation corresponds to the following macro that is 296written with the old macros. 297.Bd -literal 298#define IFQ_ENQUEUE(ifq, m, error) \e 299do { \e 300 if (IF_QFULL((ifq))) { \e 301 m_freem((m)); \e 302 (error) = ENOBUFS; \e 303 IF_DROP(ifq); \e 304 } else { \e 305 IF_ENQUEUE((ifq), (m)); \e 306 (error) = 0; \e 307 } \e 308} while (0) 309.Ed 310.Pp 311.Fn IFQ_ENQUEUE 312does the following: 313.Pp 314.Bl -hyphen -compact 315.It 316queue a packet, 317.It 318drop (and free) a packet if the enqueue operation fails. 319.El 320.Pp 321If the enqueue operation fails, 322.Fa error 323is set to 324.Er ENOBUFS . 325The 326.Fa m 327mbuf is freed by the queuing discipline. 328The caller should not touch mbuf after calling 329.Fn IFQ_ENQUEUE 330so that the caller may need to copy 331.Va m_pkthdr.len 332or 333.Va m_flags 334field beforehand for statistics. 335.Fn IFQ_HANDOFF 336and 337.Fn IFQ_HANDOFF_ADJ 338can be used if only default interface statistics and an immediate call to 339.Fn if_start 340are desired. 341The caller should not use 342.Fn senderr 343since mbuf was already freed. 344.Pp 345The new style 346.Fn if_output 347looks as follows: 348.Bd -literal 349 ##old-style## ##new-style## 350 | 351 int | int 352 ether_output(ifp, m0, dst, rt0) | ether_output(ifp, m0, dst, rt0) 353 { | { 354 ...... | ...... 355 | 356 | mflags = m->m_flags; 357 | len = m->m_pkthdr.len; 358 s = splimp(); | s = splimp(); 359 if (IF_QFULL(&ifp->if_snd)) { | IFQ_ENQUEUE(&ifp->if_snd, m, 360 | error); 361 IF_DROP(&ifp->if_snd); | if (error != 0) { 362 splx(s); | splx(s); 363 senderr(ENOBUFS); | return (error); 364 } | } 365 IF_ENQUEUE(&ifp->if_snd, m); | 366 ifp->if_obytes += | ifp->if_obytes += len; 367 m->m_pkthdr.len; | 368 if (m->m_flags & M_MCAST) | if (mflags & M_MCAST) 369 ifp->if_omcasts++; | ifp->if_omcasts++; 370 | 371 if ((ifp->if_flags & IFF_OACTIVE) | if ((ifp->if_flags & IFF_OACTIVE) 372 == 0) | == 0) 373 (*ifp->if_start)(ifp); | (*ifp->if_start)(ifp); 374 splx(s); | splx(s); 375 return (error); | return (error); 376 | 377 bad: | bad: 378 if (m) | if (m) 379 m_freem(m); | m_freem(m); 380 return (error); | return (error); 381 } | } 382 | 383.Ed 384.Sh HOW TO CONVERT THE EXISTING DRIVERS 385First, make sure the corresponding 386.Fn if_output 387is already converted to the new style. 388.Pp 389Look for 390.Va if_snd 391in the driver. 392Probably, you need to make changes to the lines that include 393.Va if_snd . 394.Ss Empty check operation 395If the code checks 396.Va ifq_head 397to see whether the queue is empty or not, use 398.Fn IFQ_IS_EMPTY . 399.Bd -literal 400 ##old-style## ##new-style## 401 | 402 if (ifp->if_snd.ifq_head != NULL) | if (!IFQ_IS_EMPTY(&ifp->if_snd)) 403 | 404.Ed 405.Fn IFQ_IS_EMPTY 406only checks if there is any packet stored in the queue. 407Note that even when 408.Fn IFQ_IS_EMPTY 409is 410.Dv FALSE , 411.Fn IFQ_DEQUEUE 412could still return 413.Dv NULL 414if the queue is under rate-limiting. 415.Ss Dequeue operation 416Replace 417.Fn IF_DEQUEUE 418by 419.Fn IFQ_DEQUEUE . 420Always check whether the dequeued mbuf is 421.Dv NULL 422or not. 423Note that even when 424.Fn IFQ_IS_EMPTY 425is 426.Dv FALSE , 427.Fn IFQ_DEQUEUE 428could return 429.Dv NULL 430due to rate-limiting. 431.Bd -literal 432 ##old-style## ##new-style## 433 | 434 IF_DEQUEUE(&ifp->if_snd, m); | IFQ_DEQUEUE(&ifp->if_snd, m); 435 | if (m == NULL) 436 | return; 437 | 438.Ed 439A driver is supposed to call 440.Fn if_start 441from transmission complete interrupts in order to trigger the next dequeue. 442.Ss Poll-and-dequeue operation 443If the code polls the packet at the head of the queue and actually uses 444the packet before dequeuing it, use 445.Fn IFQ_POLL_NOLOCK 446and 447.Fn IFQ_DEQUEUE_NOLOCK . 448.Bd -literal 449 ##old-style## ##new-style## 450 | 451 | IFQ_LOCK(&ifp->if_snd); 452 m = ifp->if_snd.ifq_head; | IFQ_POLL_NOLOCK(&ifp->if_snd, m); 453 if (m != NULL) { | if (m != NULL) { 454 | 455 /* use m to get resources */ | /* use m to get resources */ 456 if (something goes wrong) | if (something goes wrong) 457 | IFQ_UNLOCK(&ifp->if_snd); 458 return; | return; 459 | 460 IF_DEQUEUE(&ifp->if_snd, m); | IFQ_DEQUEUE_NOLOCK(&ifp->if_snd, m); 461 | IFQ_UNLOCK(&ifp->if_snd); 462 | 463 /* kick the hardware */ | /* kick the hardware */ 464 } | } 465 | 466.Ed 467It is guaranteed that 468.Fn IFQ_DEQUEUE_NOLOCK 469under the same lock as a previous 470.Fn IFQ_POLL_NOLOCK 471returns the same packet. 472Note that they need to be guarded by 473.Fn IFQ_LOCK . 474.Ss Eliminating Fn IF_PREPEND 475If the code uses 476.Fn IF_PREPEND , 477you have to eliminate it unless you can use a 478.Dq "driver managed" 479queue which allows the use of 480.Fn IFQ_DRV_PREPEND 481as a substitute. 482A common usage of 483.Fn IF_PREPEND 484is to cancel the previous dequeue operation. 485You have to convert the logic into poll-and-dequeue. 486.Bd -literal 487 ##old-style## ##new-style## 488 | 489 | IFQ_LOCK(&ifp->if_snd); 490 IF_DEQUEUE(&ifp->if_snd, m); | IFQ_POLL_NOLOCK(&ifp->if_snd, m); 491 if (m != NULL) { | if (m != NULL) { 492 | 493 if (something_goes_wrong) { | if (something_goes_wrong) { 494 IF_PREPEND(&ifp->if_snd, m); | IFQ_UNLOCK(&ifp->if_snd); 495 return; | return; 496 } | } 497 | 498 | /* at this point, the driver 499 | * is committed to send this 500 | * packet. 501 | */ 502 | IFQ_DEQUEUE_NOLOCK(&ifp->if_snd, m); 503 | IFQ_UNLOCK(&ifp->if_snd); 504 | 505 /* kick the hardware */ | /* kick the hardware */ 506 } | } 507 | 508.Ed 509.Ss Purge operation 510Use 511.Fn IFQ_PURGE 512to empty the queue. 513Note that a non-work conserving queue cannot be emptied by a dequeue loop. 514.Bd -literal 515 ##old-style## ##new-style## 516 | 517 while (ifp->if_snd.ifq_head != NULL) {| IFQ_PURGE(&ifp->if_snd); 518 IF_DEQUEUE(&ifp->if_snd, m); | 519 m_freem(m); | 520 } | 521 | 522.Ed 523.Ss Conversion using a driver managed queue 524Convert 525.Fn IF_* 526macros to their equivalent 527.Fn IFQ_DRV_* 528and employ 529.Fn IFQ_DRV_IS_EMPTY 530where appropriate. 531.Bd -literal 532 ##old-style## ##new-style## 533 | 534 if (ifp->if_snd.ifq_head != NULL) | if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) 535 | 536.Ed 537Make sure that calls to 538.Fn IFQ_DRV_DEQUEUE , 539.Fn IFQ_DRV_PREPEND 540and 541.Fn IFQ_DRV_PURGE 542are protected with a mutex of some kind. 543.Ss Attach routine 544Use 545.Fn IFQ_SET_MAXLEN 546to set 547.Va ifq_maxlen 548to 549.Fa len . 550Initialize 551.Va ifq_drv_maxlen 552with a sensible value if you plan to use the 553.Fn IFQ_DRV_* 554macros. 555Add 556.Fn IFQ_SET_READY 557to show this driver is converted to the new style. 558(This is used to distinguish new-style drivers.) 559.Bd -literal 560 ##old-style## ##new-style## 561 | 562 ifp->if_snd.ifq_maxlen = qsize; | IFQ_SET_MAXLEN(&ifp->if_snd, qsize); 563 | ifp->if_snd.ifq_drv_maxlen = qsize; 564 | IFQ_SET_READY(&ifp->if_snd); 565 if_attach(ifp); | if_attach(ifp); 566 | 567.Ed 568.Ss Other issues 569The new macros for statistics: 570.Bd -literal 571 ##old-style## ##new-style## 572 | 573 IF_DROP(&ifp->if_snd); | IFQ_INC_DROPS(&ifp->if_snd); 574 | 575 ifp->if_snd.ifq_len++; | IFQ_INC_LEN(&ifp->if_snd); 576 | 577 ifp->if_snd.ifq_len--; | IFQ_DEC_LEN(&ifp->if_snd); 578 | 579.Ed 580.Sh QUEUING DISCIPLINES 581Queuing disciplines need to maintain 582.Fa ifq_len 583(used by 584.Fn IFQ_IS_EMPTY ) . 585Queuing disciplines also need to guarantee that the same mbuf is returned if 586.Fn IFQ_DEQUEUE 587is called immediately after 588.Fn IFQ_POLL . 589.Sh SEE ALSO 590.Xr pf 4 , 591.Xr pf.conf 5 , 592.Xr pfctl 8 593.Sh HISTORY 594The 595.Nm 596system first appeared in March 1997 and found home in the KAME project 597(https://www.kame.net). 598It was imported to 599.Fx 600in 5.3 . 601