1.\" 2.\" Copyright (c) 1999 Kenneth D. Merry. 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. The name of the author may not be used to endorse or promote products 11.\" derived from this software without specific prior written permission. 12.\" 13.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 14.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 17.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23.\" SUCH DAMAGE. 24.\" 25.\" $FreeBSD$ 26.\" 27.Dd June 14, 2018 28.Dt PCI 4 29.Os 30.Sh NAME 31.Nm pci 32.Nd generic PCI bus driver 33.Sh SYNOPSIS 34To compile the PCI bus driver into the kernel, 35place the following line in your 36kernel configuration file: 37.Bd -ragged -offset indent 38.Cd device pci 39.Ed 40.Pp 41To compile in support for Single Root I/O Virtualization 42.Pq SR-IOV : 43.Bd -ragged -offset indent 44.Cd options PCI_IOV 45.Ed 46.Pp 47To compile in support for native PCI-express HotPlug: 48.Bd -ragged -offset indent 49.Cd options PCI_HP 50.Ed 51.Sh DESCRIPTION 52The 53.Nm 54driver provides support for 55.Tn PCI 56devices in the kernel and limited access to 57.Tn PCI 58devices for userland. 59.Pp 60The 61.Nm 62driver provides a 63.Pa /dev/pci 64character device that can be used by userland programs to read and write 65.Tn PCI 66configuration registers. 67Programs can also use this device to get a list of all 68.Tn PCI 69devices, or all 70.Tn PCI 71devices that match various patterns. 72.Pp 73Since the 74.Nm 75driver provides a write interface for 76.Tn PCI 77configuration registers, system administrators should exercise caution when 78granting access to the 79.Nm 80device. 81If used improperly, this driver can allow userland applications to 82crash a machine or cause data loss. 83.Pp 84The 85.Nm 86driver implements the 87.Tn PCI 88bus in the kernel. 89It enumerates any devices on the 90.Tn PCI 91bus and gives 92.Tn PCI 93client drivers the chance to attach to them. 94It assigns resources to children, when the BIOS does not. 95It takes care of routing interrupts when necessary. 96It reprobes the unattached 97.Tn PCI 98children when 99.Tn PCI 100client drivers are dynamically 101loaded at runtime. 102The 103.Nm 104driver also includes support for PCI-PCI bridges, 105various platform-specific Host-PCI bridges, 106and basic support for 107.Tn PCI 108VGA adapters. 109.Sh IOCTLS 110The following 111.Xr ioctl 2 112calls are supported by the 113.Nm 114driver. 115They are defined in the header file 116.In sys/pciio.h . 117.Bl -tag -width 012345678901234 118.It PCIOCGETCONF 119This 120.Xr ioctl 2 121takes a 122.Va pci_conf_io 123structure. 124It allows the user to retrieve information on all 125.Tn PCI 126devices in the system, or on 127.Tn PCI 128devices matching patterns supplied by the user. 129The call may set 130.Va errno 131to any value specified in either 132.Xr copyin 9 133or 134.Xr copyout 9 . 135The 136.Va pci_conf_io 137structure consists of a number of fields: 138.Bl -tag -width match_buf_len 139.It pat_buf_len 140The length, in bytes, of the buffer filled with user-supplied patterns. 141.It num_patterns 142The number of user-supplied patterns. 143.It patterns 144Pointer to a buffer filled with user-supplied patterns. 145.Va patterns 146is a pointer to 147.Va num_patterns 148.Va pci_match_conf 149structures. 150The 151.Va pci_match_conf 152structure consists of the following elements: 153.Bl -tag -width pd_vendor 154.It pc_sel 155.Tn PCI 156domain, bus, slot and function. 157.It pd_name 158.Tn PCI 159device driver name. 160.It pd_unit 161.Tn PCI 162device driver unit number. 163.It pc_vendor 164.Tn PCI 165vendor ID. 166.It pc_device 167.Tn PCI 168device ID. 169.It pc_class 170.Tn PCI 171device class. 172.It flags 173The flags describe which of the fields the kernel should match against. 174A device must match all specified fields in order to be returned. 175The match flags are enumerated in the 176.Va pci_getconf_flags 177structure. 178Hopefully the flag values are obvious enough that they do not need to 179described in detail. 180.El 181.It match_buf_len 182Length of the 183.Va matches 184buffer allocated by the user to hold the results of the 185.Dv PCIOCGETCONF 186query. 187.It num_matches 188Number of matches returned by the kernel. 189.It matches 190Buffer containing matching devices returned by the kernel. 191The items in this buffer are of type 192.Va pci_conf , 193which consists of the following items: 194.Bl -tag -width pc_subvendor 195.It pc_sel 196.Tn PCI 197domain, bus, slot and function. 198.It pc_hdr 199.Tn PCI 200header type. 201.It pc_subvendor 202.Tn PCI 203subvendor ID. 204.It pc_subdevice 205.Tn PCI 206subdevice ID. 207.It pc_vendor 208.Tn PCI 209vendor ID. 210.It pc_device 211.Tn PCI 212device ID. 213.It pc_class 214.Tn PCI 215device class. 216.It pc_subclass 217.Tn PCI 218device subclass. 219.It pc_progif 220.Tn PCI 221device programming interface. 222.It pc_revid 223.Tn PCI 224revision ID. 225.It pd_name 226Driver name. 227.It pd_unit 228Driver unit number. 229.El 230.It offset 231The offset is passed in by the user to tell the kernel where it should 232start traversing the device list. 233The value passed out by the kernel 234points to the record immediately after the last one returned. 235The user may 236pass the value returned by the kernel in subsequent calls to the 237.Dv PCIOCGETCONF 238ioctl. 239If the user does not intend to use the offset, it must be set to zero. 240.It generation 241.Tn PCI 242configuration generation. 243This value only needs to be set if the offset is set. 244The kernel will compare the current generation number of its internal 245device list to the generation passed in by the user to determine whether 246its device list has changed since the user last called the 247.Dv PCIOCGETCONF 248ioctl. 249If the device list has changed, a status of 250.Va PCI_GETCONF_LIST_CHANGED 251will be passed back. 252.It status 253The status tells the user the disposition of his request for a device list. 254The possible status values are: 255.Bl -ohang 256.It PCI_GETCONF_LAST_DEVICE 257This means that there are no more devices in the PCI device list matching 258the specified criteria after the 259ones returned in the 260.Va matches 261buffer. 262.It PCI_GETCONF_LIST_CHANGED 263This status tells the user that the 264.Tn PCI 265device list has changed since his last call to the 266.Dv PCIOCGETCONF 267ioctl and he must reset the 268.Va offset 269and 270.Va generation 271to zero to start over at the beginning of the list. 272.It PCI_GETCONF_MORE_DEVS 273This tells the user that his buffer was not large enough to hold all of the 274remaining devices in the device list that match his criteria. 275.It PCI_GETCONF_ERROR 276This indicates a general error while servicing the user's request. 277If the 278.Va pat_buf_len 279is not equal to 280.Va num_patterns 281times 282.Fn sizeof "struct pci_match_conf" , 283.Va errno 284will be set to 285.Er EINVAL . 286.El 287.El 288.It PCIOCREAD 289This 290.Xr ioctl 2 291reads the 292.Tn PCI 293configuration registers specified by the passed-in 294.Va pci_io 295structure. 296The 297.Va pci_io 298structure consists of the following fields: 299.Bl -tag -width pi_width 300.It pi_sel 301A 302.Va pcisel 303structure which specifies the domain, bus, slot and function the user would 304like to query. 305If the specific bus is not found, errno will be set to ENODEV and -1 returned 306from the ioctl. 307.It pi_reg 308The 309.Tn PCI 310configuration register the user would like to access. 311.It pi_width 312The width, in bytes, of the data the user would like to read. 313This value 314may be either 1, 2, or 4. 3153-byte reads and reads larger than 4 bytes are 316not supported. 317If an invalid width is passed, errno will be set to EINVAL. 318.It pi_data 319The data returned by the kernel. 320.El 321.It PCIOCWRITE 322This 323.Xr ioctl 2 324allows users to write to the 325.Tn PCI 326specified in the passed-in 327.Va pci_io 328structure. 329The 330.Va pci_io 331structure is described above. 332The limitations on data width described for 333reading registers, above, also apply to writing 334.Tn PCI 335configuration registers. 336.It PCIOCBARMMAP 337This 338.Xr ioctl 2 339command allows userspace processes to 340.Xr mmap 2 341the memory-mapped PCI BAR into its address space. 342The input parameters and results are passed in the 343.Va pci_bar_mmap 344structure, which has the following fields: 345.Bl -tag -width Vt struct pcise pbm_sel 346.It Vt uint64_t pbm_map_base 347Reports the established mapping base to the caller. 348If 349.Va PCIIO_BAR_MMAP_FIXED 350flag was specified, then this field must be filled before the call 351with the desired address for the mapping. 352.It Vt uint64_t pbm_map_length 353Reports the mapped length of the BAR, in bytes. 354Its .Vt uint64_t value is always multiple of machine pages. 355.It Vt int64_t pbm_bar_length 356Reports length of the bar as exposed by the device. 357.It Vt int pbm_bar_off 358Reports offset from the mapped base to the start of the 359first register in the bar. 360.It Vt struct pcisel pbm_sel 361Should be filled before the call. 362Describes the device to operate on. 363.It Vt int pbm_reg 364The BAR index to mmap. 365.It Vt int pbm_flags 366Flags which augments the operation. 367See below. 368.It Vt int pbm_memattr 369The caching attribute for the mapping. 370Typical values are 371.Dv VM_MEMATTR_UNCACHEABLE 372for control registers BARs, and 373.Dv VM_MEMATTR_WRITE_COMBINING 374for frame buffers. 375Regular memory-like BAR should be mapped with 376.Dv VM_MEMATTR_DEFAULT 377attribute. 378.El 379.Pp 380Currently defined flags are: 381.Bl -tag -width PCIIO_BAR_MMAP_ACTIVATE 382.It PCIIO_BAR_MMAP_FIXED 383The resulted mappings should be established at the address 384specified by the 385.Va pbm_map_base 386member, otherwise fail. 387.It PCIIO_BAR_MMAP_EXCL 388Must be used together with 389.Vd PCIIO_BAR_MMAP_FIXED 390If the specified base contains already established mappings, the 391operation fails instead of implicitly unmapping them. 392.It PCIIO_BAR_MMAP_RW 393The requested mapping allows both reading and writing. 394Without the flag, read-only mapping is established. 395Note that it is common for the device registers to have side-effects 396even on reads. 397.It PCIIO_BAR_MMAP_ACTIVATE 398(Unimplemented) If the BAR is not activated, activate it in the course 399of mapping. 400Currently attempt to mmap an inactive BAR results in error. 401.El 402.El 403.Sh LOADER TUNABLES 404Tunables can be set at the 405.Xr loader 8 406prompt before booting the kernel, or stored in 407.Xr loader.conf 5 . 408The current value of these tunables can be examined at runtime via 409.Xr sysctl 8 410nodes of the same name. 411Unless otherwise specified, 412each of these tunables is a boolean that can be enabled by setting the 413tunable to a non-zero value. 414.Bl -tag -width indent 415.It Va hw.pci.clear_bars Pq Defaults to 0 416Ignore any firmware-assigned memory and I/O port resources. 417This forces the 418.Tn PCI 419bus driver to allocate resource ranges for memory and I/O port resources 420from scratch. 421.It Va hw.pci.clear_buses Pq Defaults to 0 422Ignore any firmware-assigned bus number registers in PCI-PCI bridges. 423This forces the 424.Tn PCI 425bus driver and PCI-PCI bridge driver to allocate bus numbers for secondary 426buses behind PCI-PCI bridges. 427.It Va hw.pci.clear_pcib Pq Defaults to 0 428Ignore any firmware-assigned memory and I/O port resource windows in PCI-PCI 429bridges. 430This forces the PCI-PCI bridge driver to allocate memory and I/O port resources 431for resource windows from scratch. 432.Pp 433By default the PCI-PCI bridge driver will allocate windows that 434contain the firmware-assigned resources devices behind the bridge. 435In addition, the PCI-PCI bridge driver will suballocate from existing window 436regions when possible to satisfy a resource request. 437As a result, 438both 439.Va hw.pci.clear_bars 440and 441.Va hw.pci.clear_pcib 442must be enabled to fully ignore firmware-supplied resource assignments. 443.It Va hw.pci.default_vgapci_unit Pq Defaults to -1 444By default, 445the first 446.Tn PCI 447VGA adapter encountered by the system is assumed to be the boot display device. 448This tunable can be set to choose a specific VGA adapter by specifying the 449unit number of the associated 450.Va vgapci Ns Ar X 451device. 452.It Va hw.pci.do_power_nodriver Pq Defaults to 0 453Place devices into a low power state 454.Pq D3 455when a suitable device driver is not found. 456Can be set to one of the following values: 457.Bl -tag -width indent 458.It 3 459Powers down all 460.Tn PCI 461devices without a device driver. 462.It 2 463Powers down most devices without a device driver. 464PCI devices with the display, memory, and base peripheral device classes 465are not powered down. 466.It 1 467Similar to a setting of 2 except that storage controllers are also not 468powered down. 469.It 0 470All devices are left fully powered. 471.El 472.Pp 473A 474.Tn PCI 475device must support power management to be powered down. 476Placing a device into a low power state may not reduce power consumption. 477.It Va hw.pci.do_power_resume Pq Defaults to 1 478Place 479.Tn PCI 480devices into the fully powered state when resuming either the system or an 481individual device. 482Setting this to zero is discouraged as the system will not attempt to power 483up non-powered PCI devices after a suspend. 484.It Va hw.pci.do_power_suspend Pq Defaults to 1 485Place 486.Tn PCI 487devices into a low power state when suspending either the system or individual 488devices. 489Normally the D3 state is used as the low power state, 490but firmware may override the desired power state during a system suspend. 491.It Va hw.pci.enable_ari Pq Defaults to 1 492Enable support for PCI-express Alternative RID Interpretation. 493This is often used in conjunction with SR-IOV. 494.It Va hw.pci.enable_io_modes Pq Defaults to 1 495Enable memory or I/O port decoding in a PCI device's command register if it has 496firmware-assigned memory or I/O port resources. 497The firmware 498.Pq BIOS 499in some systems does not enable memory or I/O port decoding for some devices 500even when it has assigned resources to the device. 501This enables decoding for such resources during bus probe. 502.It Va hw.pci.enable_msi Pq Defaults to 1 503Enable support for Message Signalled Interrupts 504.Pq MSI . 505MSI interrupts can be disabled by setting this tunable to 0. 506.It Va hw.pci.enable_msix Pq Defaults to 1 507Enable support for extended Message Signalled Interrupts 508.Pq MSI-X . 509MSI-X interrupts can be disabled by setting this tunable to 0. 510.It Va hw.pci.enable_pcie_hp Pq Defaults to 1 511Enable support for native PCI-express HotPlug. 512.It Va hw.pci.honor_msi_blacklist Pq Defaults to 1 513MSI and MSI-X interrupts are disabled for certain chipsets known to have 514broken MSI and MSI-X implementations when this tunable is set. 515It can be set to zero to permit use of MSI and MSI-X interrupts if the 516chipset match is a false positive. 517.It Va hw.pci.iov_max_config Pq Defaults to 1MB 518The maximum amount of memory permitted for the configuration parameters 519used when creating Virtual Functions via SR-IOV. 520This tunable can also be changed at runtime via 521.Xr sysctl 8 . 522.It Va hw.pci.realloc_bars Pq Defaults to 0 523Attempt to allocate a new resource range during the initial device scan 524for any memory or I/O port resources with firmware-assigned ranges that 525conflict with another active resource. 526.It Va hw.pci.usb_early_takeover Pq Defaults to 1 on Tn amd64 and Tn i386 527Disable legacy device emulation of USB devices during the initial device 528scan. 529Set this tunable to zero to use USB devices via legacy emulation when 530using a custom kernel without USB controller drivers. 531.It Va hw.pci<D>.<B>.<S>.INT<P>.irq 532These tunables can be used to override the interrupt routing for legacy 533PCI INTx interrupts. 534Unlike other tunables in this list, 535these do not have corresponding sysctl nodes. 536The tunable name includes the address of the PCI device as well as the 537pin of the desired INTx IRQ to override: 538.Bl -tag -width indent 539.It <D> 540The domain 541.Pq or segment 542of the PCI device in decimal. 543.It <B> 544The bus address of the PCI device in decimal. 545.It <S> 546The slot of the PCI device in decimal. 547.It <P> 548The interrupt pin of the PCI slot to override. 549One of 550.Ql A , 551.Ql B , 552.Ql C , 553or 554.Ql D . 555.El 556.Pp 557The value of the tunable is the raw IRQ value to use for the INTx interrupt 558pin identified by the tunable name. 559Mapping of IRQ values to platform interrupt sources is machine dependent. 560.El 561.Sh DEVICE WIRING 562You can wire the device unit at a given location with device.hints. 563Entries of the form 564.Va hints.<name>.<unit>.at="pci<B>:<S>:<F>" 565or 566.Va hints.<name>.<unit>.at="pci<D>:<B>:<S>:<F>" 567will force the driver 568.Va name 569to probe and attach at unit 570.Va unit 571for any PCI device found to match the specification, where: 572.Bl -tag -width -indent 573.It <D> 574The domain 575.Pq or segment 576of the PCI device in decimal. 577Defaults to 0 if unspecified 578.It <B> 579The bus address of the PCI device in decimal. 580.It <S> 581The slot of the PCI device in decimal. 582.It <F> 583The function of the PCI device in decimal. 584.El 585.Pp 586The code to do the matching requires an exact string match. 587Do not specify the angle brackets 588.Pq < > 589in the hints file. 590Wiring multiple devices to the same 591.Va name 592and 593.Va unit 594produces undefined results. 595.Ss Examples 596Given the following lines in 597.Pa /boot/device.hints : 598.Cd hint.nvme.3.at="pci6:0:0" 599.Cd hint.igb.8.at="pci14:0:0" 600If there is a device that supports 601.Xr igb 4 602at PCI bus 14 slot 0 function 0, 603then it will be assigned igb8 for probe and attach. 604Likewise, if there is an 605.Xr nvme 4 606card at PCI bus 6 slot 0 function 0, 607then it will be assigned nvme3 for probe and attach. 608If another type of card is in either of these locations, the name and 609unit of that card will be the default names and will be unaffected by 610these hints. 611If other igb or nvme cards are located elsewhere, they will be 612assigned their unit numbers sequentially, skipping the unit numbers 613that have 'at' hints. 614.Sh FILES 615.Bl -tag -width /dev/pci -compact 616.It Pa /dev/pci 617Character device for the 618.Nm 619driver. 620.El 621.Sh SEE ALSO 622.Xr pciconf 8 623.Sh HISTORY 624The 625.Nm 626driver (not the kernel's 627.Tn PCI 628support code) first appeared in 629.Fx 2.2 , 630and was written by Stefan Esser and Garrett Wollman. 631Support for device listing and matching was re-implemented by 632Kenneth Merry, and first appeared in 633.Fx 3.0 . 634.Sh AUTHORS 635.An Kenneth Merry Aq Mt ken@FreeBSD.org 636.Sh BUGS 637It is not possible for users to specify an accurate offset into the device 638list without calling the 639.Dv PCIOCGETCONF 640at least once, since they have no way of knowing the current generation 641number otherwise. 642This probably is not a serious problem, though, since 643users can easily narrow their search by specifying a pattern or patterns 644for the kernel to match against. 645