1d16c0cd2SRicardo Cañuelo.. SPDX-License-Identifier: GPL-2.0 2d16c0cd2SRicardo Cañuelo 3d16c0cd2SRicardo Cañuelo.. _virtio: 4d16c0cd2SRicardo Cañuelo 5d16c0cd2SRicardo Cañuelo=============== 6d16c0cd2SRicardo CañueloVirtio on Linux 7d16c0cd2SRicardo Cañuelo=============== 8d16c0cd2SRicardo Cañuelo 9d16c0cd2SRicardo CañueloIntroduction 10d16c0cd2SRicardo Cañuelo============ 11d16c0cd2SRicardo Cañuelo 12d16c0cd2SRicardo CañueloVirtio is an open standard that defines a protocol for communication 13d16c0cd2SRicardo Cañuelobetween drivers and devices of different types, see Chapter 5 ("Device 14fb25d456SBagas SanjayaTypes") of the virtio spec (`[1]`_). Originally developed as a standard 15d16c0cd2SRicardo Cañuelofor paravirtualized devices implemented by a hypervisor, it can be used 16d16c0cd2SRicardo Cañueloto interface any compliant device (real or emulated) with a driver. 17d16c0cd2SRicardo Cañuelo 18d16c0cd2SRicardo CañueloFor illustrative purposes, this document will focus on the common case 19d16c0cd2SRicardo Cañueloof a Linux kernel running in a virtual machine and using paravirtualized 20d16c0cd2SRicardo Cañuelodevices provided by the hypervisor, which exposes them as virtio devices 21d16c0cd2SRicardo Cañuelovia standard mechanisms such as PCI. 22d16c0cd2SRicardo Cañuelo 23d16c0cd2SRicardo Cañuelo 24d16c0cd2SRicardo CañueloDevice - Driver communication: virtqueues 25d16c0cd2SRicardo Cañuelo========================================= 26d16c0cd2SRicardo Cañuelo 27d16c0cd2SRicardo CañueloAlthough the virtio devices are really an abstraction layer in the 28d16c0cd2SRicardo Cañuelohypervisor, they're exposed to the guest as if they are physical devices 29d16c0cd2SRicardo Cañuelousing a specific transport method -- PCI, MMIO or CCW -- that is 30d16c0cd2SRicardo Cañueloorthogonal to the device itself. The virtio spec defines these transport 31d16c0cd2SRicardo Cañuelomethods in detail, including device discovery, capabilities and 32d16c0cd2SRicardo Cañuelointerrupt handling. 33d16c0cd2SRicardo Cañuelo 34d16c0cd2SRicardo CañueloThe communication between the driver in the guest OS and the device in 35d16c0cd2SRicardo Cañuelothe hypervisor is done through shared memory (that's what makes virtio 36d16c0cd2SRicardo Cañuelodevices so efficient) using specialized data structures called 37d16c0cd2SRicardo Cañuelovirtqueues, which are actually ring buffers [#f1]_ of buffer descriptors 38d16c0cd2SRicardo Cañuelosimilar to the ones used in a network device: 39d16c0cd2SRicardo Cañuelo 40d16c0cd2SRicardo Cañuelo.. kernel-doc:: include/uapi/linux/virtio_ring.h 41d16c0cd2SRicardo Cañuelo :identifiers: struct vring_desc 42d16c0cd2SRicardo Cañuelo 43d16c0cd2SRicardo CañueloAll the buffers the descriptors point to are allocated by the guest and 44d16c0cd2SRicardo Cañueloused by the host either for reading or for writing but not for both. 45d16c0cd2SRicardo Cañuelo 46fb25d456SBagas SanjayaRefer to Chapter 2.5 ("Virtqueues") of the virtio spec (`[1]`_) for the 47fb25d456SBagas Sanjayareference definitions of virtqueues and "Virtqueues and virtio ring: How 48fb25d456SBagas Sanjayathe data travels" blog post (`[2]`_) for an illustrated overview of how 49fb25d456SBagas Sanjayathe host device and the guest driver communicate. 50d16c0cd2SRicardo Cañuelo 51d16c0cd2SRicardo CañueloThe :c:type:`vring_virtqueue` struct models a virtqueue, including the 52d16c0cd2SRicardo Cañueloring buffers and management data. Embedded in this struct is the 53d16c0cd2SRicardo Cañuelo:c:type:`virtqueue` struct, which is the data structure that's 54d16c0cd2SRicardo Cañueloultimately used by virtio drivers: 55d16c0cd2SRicardo Cañuelo 56d16c0cd2SRicardo Cañuelo.. kernel-doc:: include/linux/virtio.h 57d16c0cd2SRicardo Cañuelo :identifiers: struct virtqueue 58d16c0cd2SRicardo Cañuelo 59d16c0cd2SRicardo CañueloThe callback function pointed by this struct is triggered when the 60d16c0cd2SRicardo Cañuelodevice has consumed the buffers provided by the driver. More 61d16c0cd2SRicardo Cañuelospecifically, the trigger will be an interrupt issued by the hypervisor 62d16c0cd2SRicardo Cañuelo(see vring_interrupt()). Interrupt request handlers are registered for 63d16c0cd2SRicardo Cañueloa virtqueue during the virtqueue setup process (transport-specific). 64d16c0cd2SRicardo Cañuelo 65d16c0cd2SRicardo Cañuelo.. kernel-doc:: drivers/virtio/virtio_ring.c 66d16c0cd2SRicardo Cañuelo :identifiers: vring_interrupt 67d16c0cd2SRicardo Cañuelo 68d16c0cd2SRicardo Cañuelo 69d16c0cd2SRicardo CañueloDevice discovery and probing 70d16c0cd2SRicardo Cañuelo============================ 71d16c0cd2SRicardo Cañuelo 72d16c0cd2SRicardo CañueloIn the kernel, the virtio core contains the virtio bus driver and 73d16c0cd2SRicardo Cañuelotransport-specific drivers like `virtio-pci` and `virtio-mmio`. Then 74d16c0cd2SRicardo Cañuelothere are individual virtio drivers for specific device types that are 75d16c0cd2SRicardo Cañueloregistered to the virtio bus driver. 76d16c0cd2SRicardo Cañuelo 77d16c0cd2SRicardo CañueloHow a virtio device is found and configured by the kernel depends on how 78d16c0cd2SRicardo Cañuelothe hypervisor defines it. Taking the `QEMU virtio-console 79d16c0cd2SRicardo Cañuelo<https://gitlab.com/qemu-project/qemu/-/blob/master/hw/char/virtio-console.c>`__ 80d16c0cd2SRicardo Cañuelodevice as an example. When using PCI as a transport method, the device 81d16c0cd2SRicardo Cañuelowill present itself on the PCI bus with vendor 0x1af4 (Red Hat, Inc.) 82d16c0cd2SRicardo Cañueloand device id 0x1003 (virtio console), as defined in the spec, so the 83d16c0cd2SRicardo Cañuelokernel will detect it as it would do with any other PCI device. 84d16c0cd2SRicardo Cañuelo 85d16c0cd2SRicardo CañueloDuring the PCI enumeration process, if a device is found to match the 86d16c0cd2SRicardo Cañuelovirtio-pci driver (according to the virtio-pci device table, any PCI 87d16c0cd2SRicardo Cañuelodevice with vendor id = 0x1af4):: 88d16c0cd2SRicardo Cañuelo 89d16c0cd2SRicardo Cañuelo /* Qumranet donated their vendor ID for devices 0x1000 thru 0x10FF. */ 90d16c0cd2SRicardo Cañuelo static const struct pci_device_id virtio_pci_id_table[] = { 91d16c0cd2SRicardo Cañuelo { PCI_DEVICE(PCI_VENDOR_ID_REDHAT_QUMRANET, PCI_ANY_ID) }, 92d16c0cd2SRicardo Cañuelo { 0 } 93d16c0cd2SRicardo Cañuelo }; 94d16c0cd2SRicardo Cañuelo 95d16c0cd2SRicardo Cañuelothen the virtio-pci driver is probed and, if the probing goes well, the 96d16c0cd2SRicardo Cañuelodevice is registered to the virtio bus:: 97d16c0cd2SRicardo Cañuelo 98d16c0cd2SRicardo Cañuelo static int virtio_pci_probe(struct pci_dev *pci_dev, 99d16c0cd2SRicardo Cañuelo const struct pci_device_id *id) 100d16c0cd2SRicardo Cañuelo { 101d16c0cd2SRicardo Cañuelo ... 102d16c0cd2SRicardo Cañuelo 103d16c0cd2SRicardo Cañuelo if (force_legacy) { 104d16c0cd2SRicardo Cañuelo rc = virtio_pci_legacy_probe(vp_dev); 105d16c0cd2SRicardo Cañuelo /* Also try modern mode if we can't map BAR0 (no IO space). */ 106d16c0cd2SRicardo Cañuelo if (rc == -ENODEV || rc == -ENOMEM) 107d16c0cd2SRicardo Cañuelo rc = virtio_pci_modern_probe(vp_dev); 108d16c0cd2SRicardo Cañuelo if (rc) 109d16c0cd2SRicardo Cañuelo goto err_probe; 110d16c0cd2SRicardo Cañuelo } else { 111d16c0cd2SRicardo Cañuelo rc = virtio_pci_modern_probe(vp_dev); 112d16c0cd2SRicardo Cañuelo if (rc == -ENODEV) 113d16c0cd2SRicardo Cañuelo rc = virtio_pci_legacy_probe(vp_dev); 114d16c0cd2SRicardo Cañuelo if (rc) 115d16c0cd2SRicardo Cañuelo goto err_probe; 116d16c0cd2SRicardo Cañuelo } 117d16c0cd2SRicardo Cañuelo 118d16c0cd2SRicardo Cañuelo ... 119d16c0cd2SRicardo Cañuelo 120d16c0cd2SRicardo Cañuelo rc = register_virtio_device(&vp_dev->vdev); 121d16c0cd2SRicardo Cañuelo 122d16c0cd2SRicardo CañueloWhen the device is registered to the virtio bus the kernel will look 123d16c0cd2SRicardo Cañuelofor a driver in the bus that can handle the device and call that 124d16c0cd2SRicardo Cañuelodriver's ``probe`` method. 125d16c0cd2SRicardo Cañuelo 126ae8d2247SBagas SanjayaAt this point, the virtqueues will be allocated and configured by 127ae8d2247SBagas Sanjayacalling the appropriate ``virtio_find`` helper function, such as 128ae8d2247SBagas Sanjayavirtio_find_single_vq() or virtio_find_vqs(), which will end up calling 129ae8d2247SBagas Sanjayaa transport-specific ``find_vqs`` method. 130d16c0cd2SRicardo Cañuelo 131d16c0cd2SRicardo Cañuelo 132d16c0cd2SRicardo CañueloReferences 133d16c0cd2SRicardo Cañuelo========== 134d16c0cd2SRicardo Cañuelo 135d16c0cd2SRicardo Cañuelo_`[1]` Virtio Spec v1.2: 136d16c0cd2SRicardo Cañuelohttps://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html 137d16c0cd2SRicardo Cañuelo 138*2b034e82SBagas Sanjaya.. Check for later versions of the spec as well. 139d16c0cd2SRicardo Cañuelo 140d16c0cd2SRicardo Cañuelo_`[2]` Virtqueues and virtio ring: How the data travels 141d16c0cd2SRicardo Cañuelohttps://www.redhat.com/en/blog/virtqueues-and-virtio-ring-how-data-travels 142d16c0cd2SRicardo Cañuelo 143d16c0cd2SRicardo Cañuelo.. rubric:: Footnotes 144d16c0cd2SRicardo Cañuelo 145d16c0cd2SRicardo Cañuelo.. [#f1] that's why they may be also referred to as virtrings. 146