1.\" SPDX-License-Identifier: BSD-3-Clause 2.\" 3.\" Copyright (c) 2023 Google LLC 4.\" 5.\" Redistribution and use in source and binary forms, with or without modification, 6.\" are permitted provided that the following conditions are met: 7.\" 8.\" 1. Redistributions of source code must retain the above copyright notice, this 9.\" list of conditions and the following disclaimer. 10.\" 11.\" 2. Redistributions in binary form must reproduce the above copyright notice, 12.\" this list of conditions and the following disclaimer in the documentation 13.\" and/or other materials provided with the distribution. 14.\" 15.\" 3. Neither the name of the copyright holder nor the names of its contributors 16.\" may be used to endorse or promote products derived from this software without 17.\" specific prior written permission. 18.\" 19.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 20.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 21.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 22.\" DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR 23.\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 24.\" (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 25.\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON 26.\" ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 27.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 28.\" SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29.Dd April 26, 2023 30.Dt GVE 4 31.Os 32.Sh NAME 33.Nm gve 34.Nd "Ethernet driver for Google Virtual NIC (gVNIC)" 35.Sh SYNOPSIS 36To compile this driver into the kernel, 37place the following lines in your 38kernel configuration file: 39.Bd -ragged -offset indent 40.Cd "device gve" 41.Ed 42.Pp 43Alternatively, to load the driver as a 44module at boot time, place the following line in 45.Xr loader.conf 5 : 46.Bd -literal -offset indent 47if_gve_load="YES" 48.Ed 49.Sh DESCRIPTION 50gVNIC is a virtual network interface designed specifically for Google Compute Engine (GCE). 51It is required to support per-VM Tier-1 networking performance, and for using certain VM shapes on GCE. 52.Pp 53.Nm 54is the driver for gVNIC. 55It supports the following features: 56.Pp 57.Bl -bullet -compact 58.It 59RX checksum offload 60.It 61TX chesksum offload 62.It 63TCP Segmentation Offload (TSO) 64.It 65Large Receive Offload (LRO) in software 66.It 67Jumbo frames 68.It 69Receive Side Scaling (RSS) 70.El 71.Pp 72For more information on configuring this device, see 73.Xr ifconfig 8 . 74.Sh HARDWARE 75.Nm 76binds to a single PCI device ID presented by gVNIC: 77.Pp 78.Bl -bullet -compact 79.It 800x1AE0:0x0042 81.El 82.Sh DIAGNOSTICS 83The following messages are recorded during driver initialization: 84.Bl -diag 85.It "Enabled MSIX with %d vectors" 86.It "Configured device resources" 87.It "Successfully attached %s" 88.It "Deconfigured device resources" 89.El 90.Pp 91These messages are seen if driver initialization fails. 92Global (across-queues) allocation failures: 93.Bl -diag 94.It "Failed to configure device resources: err=%d" 95.It "No compatible queue formats" 96.It "Failed to allocate ifnet struct" 97.It "Failed to allocate admin queue mem" 98.It "Failed to alloc DMA mem for DescribeDevice" 99.It "Failed to allocate QPL page" 100.El 101.Pp 102irq and BAR allocation failures: 103.Bl -diag 104.It "Failed to acquire any msix vectors" 105.It "Tried to acquire %d msix vectors, got only %d" 106.It "Failed to setup irq %d for Tx queue %d " 107.It "Failed to setup irq %d for Rx queue %d " 108.It "Failed to allocate irq %d for mgmnt queue" 109.It "Failed to setup irq %d for mgmnt queue, err: %d" 110.It "Failed to allocate BAR0" 111.It "Failed to allocate BAR2" 112.It "Failed to allocate msix table" 113.El 114.Pp 115Rx queue-specific allocation failures: 116.Bl -diag 117.It "No QPL left for rx ring %d" 118.It "Failed to alloc queue resources for rx ring %d" 119.It "Failed to alloc desc ring for rx ring %d" 120.It "Failed to alloc data ring for rx ring %d" 121.El 122.Pp 123Tx queue-specific allocation failures: 124.Bl -diag 125.It "No QPL left for tx ring %d" 126.It "Failed to alloc queue resources for tx ring %d" 127.It "Failed to alloc desc ring for tx ring %d" 128.It "Failed to vmap fifo, qpl_id = %d" 129.El 130.El 131.Pp 132The following messages are recorded when the interface detach fails: 133.Bl -diag 134.It "Failed to deconfigure device resources: err=%d" 135.El 136.Pp 137If bootverbose is on, the following messages are recorded when the interface is being brought up: 138.Bl -diag 139.It "Created %d rx queues" 140.It "Created %d tx queues" 141.It "MTU set to %d" 142.El 143.Pp 144The following messages are recorded when the interface is being brought down: 145.Bl -diag 146.It "Destroyed %d rx queues" 147.It "Destroyed %d tx queues" 148.El 149.Pp 150These messages are seen if errors are encountered when bringing the interface up or down: 151.Bl -diag 152.It "Failed to destroy rxq %d, err: %d" 153.It "Failed to destroy txq %d, err: %d" 154.It "Failed to create rxq %d, err: %d" 155.It "Failed to create txq %d, err: %d" 156.It "Failed to set MTU to %d" 157.It "Invalid new MTU setting. new mtu: %d max mtu: %d min mtu: %d" 158.It "Cannot bring the iface up when detached" 159.It "Reached max number of registered pages %lu > %lu" 160.It "Failed to init lro for rx ring %d" 161.El 162.Pp 163These messages are seen if any admin queue command fails: 164.Bl -diag 165.It "AQ command(%u): failed with status %d" 166.It "AQ command(%u): unknown status code %d" 167.It "AQ commands timed out, need to reset AQ" 168.It "Unknown AQ command opcode %d" 169.El 170.Pp 171These messages are recorded when the device is being reset due to an error: 172.Bl -diag 173.It "Scheduling reset task!" 174.It "Waiting until admin queue is released." 175.It "Admin queue released" 176.El 177.Pp 178If it was the NIC that requested the reset, this message is recorded: 179.Bl -diag 180.It "Device requested reset" 181.El 182.Pp 183If the reset fails during the reinitialization phase, this message is recorded: 184.Bl -diag 185.It "Restore failed!" 186.El 187.Pp 188These two messages correspond to the NIC alerting the driver to link state changes: 189.Bl -diag 190.It "Device link is up." 191.It "Device link is down." 192.El 193.Pp 194Apart from these messages, the driver exposes per-queue packet and error counters as sysctl nodes. 195Global (across queues) counters can be read using 196.Xr netstat 8 . 197.Sh LIMITATIONS 198.Nm 199does not support the transmission of VLAN-tagged packets. 200All VLAN-tagged traffic is dropped. 201.Sh SUPPORT 202Please email gvnic-drivers@google.com with the specifics of the issue encountered. 203.El 204.Sh SEE ALSO 205.Xr ifconfig 8 , 206.Xr netstat 8 207.Sh HISTORY 208The 209.Nm 210device driver first appeared in 211.Fx 13.3 . 212.Sh AUTHORS 213The 214.Nm 215driver was written by Google. 216