1b13e60daSPeter Grehan.\" Copyright (c) 2013 Peter Grehan 2b13e60daSPeter Grehan.\" All rights reserved. 3b13e60daSPeter Grehan.\" 4b13e60daSPeter Grehan.\" Redistribution and use in source and binary forms, with or without 5b13e60daSPeter Grehan.\" modification, are permitted provided that the following conditions 6b13e60daSPeter Grehan.\" are met: 7b13e60daSPeter Grehan.\" 1. Redistributions of source code must retain the above copyright 8b13e60daSPeter Grehan.\" notice, this list of conditions and the following disclaimer. 9b13e60daSPeter Grehan.\" 2. Redistributions in binary form must reproduce the above copyright 10b13e60daSPeter Grehan.\" notice, this list of conditions and the following disclaimer in the 11b13e60daSPeter Grehan.\" documentation and/or other materials provided with the distribution. 12b13e60daSPeter Grehan.\" 13b13e60daSPeter Grehan.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND 14b13e60daSPeter Grehan.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15b13e60daSPeter Grehan.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16b13e60daSPeter Grehan.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE 17b13e60daSPeter Grehan.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18b13e60daSPeter Grehan.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19b13e60daSPeter Grehan.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20b13e60daSPeter Grehan.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21b13e60daSPeter Grehan.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22b13e60daSPeter Grehan.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23b13e60daSPeter Grehan.\" SUCH DAMAGE. 24b13e60daSPeter Grehan.\" 25b13e60daSPeter Grehan.\" $FreeBSD$ 26b13e60daSPeter Grehan.\" 27*9d0c4e17SPeter Grehan.Dd April 2, 2014 28b13e60daSPeter Grehan.Dt BHYVE 8 29b13e60daSPeter Grehan.Os 30b13e60daSPeter Grehan.Sh NAME 31b13e60daSPeter Grehan.Nm bhyve 32b13e60daSPeter Grehan.Nd "run a guest operating system inside a virtual machine" 33b13e60daSPeter Grehan.Sh SYNOPSIS 34b13e60daSPeter Grehan.Nm 359777ca20SNeel Natu.Op Fl aehwxAHPW 36b13e60daSPeter Grehan.Op Fl c Ar numcpus 37b13e60daSPeter Grehan.Op Fl g Ar gdbport 38b13e60daSPeter Grehan.Op Fl p Ar pinnedcpu 39b13e60daSPeter Grehan.Op Fl s Ar slot,emulation Ns Op , Ns Ar conf 40b13e60daSPeter Grehan.Op Fl l Ar lpcdev Ns Op , Ns Ar conf 41b13e60daSPeter Grehan.Ar vmname 42b13e60daSPeter Grehan.Sh DESCRIPTION 43b13e60daSPeter Grehan.Nm 44b13e60daSPeter Grehanis an experimental hypervisor that runs guest operating systems inside a 45b13e60daSPeter Grehanvirtual machine. 46b13e60daSPeter Grehan.Pp 47b13e60daSPeter GrehanParameters such as the number of virtual CPUs, amount of guest memory, and 48b13e60daSPeter GrehanI/O connectivity can be specified with command-line parameters. 49b13e60daSPeter Grehan.Pp 50b13e60daSPeter GrehanThe guest operating system must be loaded with 51b13e60daSPeter Grehan.Xr bhyveload 4 52b13e60daSPeter Grehanor a similar boot loader before running 53b13e60daSPeter Grehan.Nm . 54b13e60daSPeter Grehan.Pp 55b13e60daSPeter Grehan.Nm 56b13e60daSPeter Grehanruns until the guest operating system reboots or an unhandled hypervisor 57b13e60daSPeter Grehanexit is detected. 58b13e60daSPeter Grehan.Sh OPTIONS 59b13e60daSPeter Grehan.Bl -tag -width 10n 60b13e60daSPeter Grehan.It Fl a 619777ca20SNeel NatuThe guest's local APIC is configured in xAPIC mode. 629777ca20SNeel Natu 639777ca20SNeel NatuThe xAPIC mode is the default setting so this option is redundant. It will be 649777ca20SNeel Natudeprecated in a future version. 65b13e60daSPeter Grehan.It Fl A 66b13e60daSPeter GrehanGenerate ACPI tables. 67b13e60daSPeter GrehanRequired for 68b13e60daSPeter Grehan.Fx Ns /amd64 69b13e60daSPeter Grehanguests. 70b13e60daSPeter Grehan.It Fl c Ar numcpus 71b13e60daSPeter GrehanNumber of guest virtual CPUs. 72b13e60daSPeter GrehanThe default is 1 and the maximum is 16. 73b13e60daSPeter Grehan.It Fl H 74b13e60daSPeter GrehanYield the virtual CPU thread when a HLT instruction is detected. 75b13e60daSPeter GrehanIf this option is not specified, virtual CPUs will use 100% of a host CPU. 76b13e60daSPeter Grehan.It Fl g Ar gdbport 77b13e60daSPeter GrehanFor 78b13e60daSPeter Grehan.Fx Ns /amd64 kernels compiled with 79b13e60daSPeter Grehan.Cd "option bvmdebug" , 80b13e60daSPeter Grehanallow a remote kernel kgdb to be relayed to the guest kernel gdb stub 81b13e60daSPeter Grehanvia a local IPv4 address and this port. 82b13e60daSPeter GrehanThis option will be deprecated in a future version. 83b13e60daSPeter Grehan.It Fl p Ar pinnedcpu 84b13e60daSPeter GrehanForce guest virtual CPUs to be pinned to host CPUs. 85b13e60daSPeter GrehanVirtual CPU 86b13e60daSPeter Grehan.Em n 87b13e60daSPeter Grehanis pinned to host CPU 88b13e60daSPeter Grehan.Em pinnedcpu+n . 89b13e60daSPeter Grehan.It Fl P 90b13e60daSPeter GrehanForce the guest virtual CPU to exit when a PAUSE instruction is detected. 91b13e60daSPeter Grehan.It Fl W 92b13e60daSPeter GrehanForce virtio PCI device emulations to use MSI interrupts instead of MSI-X 93b13e60daSPeter Grehaninterrupts. 94b13e60daSPeter Grehan.It Fl s Ar slot,emulation Ns Op , Ns Ar conf 95b13e60daSPeter GrehanConfigure a virtual PCI slot and function. 96b13e60daSPeter Grehan.Pp 97b13e60daSPeter Grehan.Nm bhyve 98b13e60daSPeter Grehanprovides PCI bus emulation and virtual devices that can be attached to 99b13e60daSPeter Grehanslots on the bus. 100b13e60daSPeter GrehanThere are 32 available slots, with the option of providing up to 8 functions 101b13e60daSPeter Grehanper slot. 102b13e60daSPeter Grehan.Bl -tag -width 10n 103b13e60daSPeter Grehan.It Ar slot 10406db1b4aSNeel Natu.Ar pcislot[:function] 10506db1b4aSNeel Natu 10606db1b4aSNeel Natu.Ar bus:pcislot:function 107b13e60daSPeter Grehan.Pp 108b13e60daSPeter GrehanThe 109b13e60daSPeter Grehan.Ar pcislot 11006db1b4aSNeel Natuvalue is 0 to 31. The optional function value is 0 to 7. The optional 11106db1b4aSNeel Natu.Ar bus 11206db1b4aSNeel Natuvalue is 0 to 255. 113b13e60daSPeter GrehanIf not specified, the function value defaults to 0. 11406db1b4aSNeel NatuIf not specified, the bus value defaults to 0. 115b13e60daSPeter Grehan.It Ar emulation 116b13e60daSPeter Grehan.Bl -tag -width 10n 117b13e60daSPeter Grehan.It Li hostbridge | Li amd_hostbridge 118b13e60daSPeter Grehan.Pp 119b13e60daSPeter GrehanProvide a simple host bridge. 120b13e60daSPeter GrehanThis is usually configured at slot 0, and is required by most guest 121b13e60daSPeter Grehanoperating systems. 122b13e60daSPeter GrehanThe 123b13e60daSPeter Grehan.Li amd_hostbridge 124b13e60daSPeter Grehanemulation is identical but uses a PCI vendor ID of 125b13e60daSPeter Grehan.Li AMD . 126b13e60daSPeter Grehan.It Li passthru 127b13e60daSPeter GrehanPCI pass-through device. 128b13e60daSPeter Grehan.It Li virtio-net 129b13e60daSPeter GrehanVirtio network interface. 130a8be8e5eSRemko Lodder.It Li virtio-blk 131b13e60daSPeter GrehanVirtio block storage interface. 132*9d0c4e17SPeter Grehan.It Li virtio-rnd 133*9d0c4e17SPeter GrehanVirtio RNG interface. 134b13e60daSPeter Grehan.It Li ahci-cd 135b13e60daSPeter GrehanAHCI controller attached to an ATAPI CD/DVD. 136b13e60daSPeter Grehan.It Li ahci-hd 137b13e60daSPeter GrehanAHCI controller attached to a SATA hard-drive. 138b13e60daSPeter Grehan.It Li uart 139b13e60daSPeter GrehanPCI 16550 serial device. 140b13e60daSPeter Grehan.It Li lpc 14106db1b4aSNeel NatuLPC PCI-ISA bridge with COM1 and COM2 16550 serial ports. The LPC bridge 14206db1b4aSNeel Natuemulation can only be configured on bus 0. 143b13e60daSPeter Grehan.El 144b13e60daSPeter Grehan.It Op Ar conf 145b13e60daSPeter GrehanThis optional parameter describes the backend for device emulations. 146b13e60daSPeter GrehanIf 147b13e60daSPeter Grehan.Ar conf 148b13e60daSPeter Grehanis not specified, the device emulation has no backend and can be 149b13e60daSPeter Grehanconsidered unconnected. 150b13e60daSPeter Grehan.Pp 151b13e60daSPeter GrehanNetwork devices: 152b13e60daSPeter Grehan.Bl -tag -width 10n 153b13e60daSPeter Grehan.It Ar tapN Ns Op , Ns Ar mac=xx:xx:xx:xx:xx:xx 154b13e60daSPeter Grehan.It Ar vmnetN Ns Op , Ns Ar mac=xx:xx:xx:xx:xx:xx 155b13e60daSPeter Grehan.Pp 156b13e60daSPeter GrehanIf 157b13e60daSPeter Grehan.Ar mac 158b13e60daSPeter Grehanis not specified, the MAC address is derived from a fixed OUI and the 159b13e60daSPeter Grehanremaining bytes from an MD5 hash of the slot and function numbers and 160b13e60daSPeter Grehanthe device name. 161b13e60daSPeter Grehan.Pp 162b13e60daSPeter GrehanThe MAC address is an ASCII string in 163b13e60daSPeter Grehan.Xr ethers 5 164b13e60daSPeter Grehanformat. 165b13e60daSPeter Grehan.El 166b13e60daSPeter Grehan.Pp 167b13e60daSPeter GrehanBlock storage devices: 168b13e60daSPeter Grehan.Bl -tag -width 10n 169b13e60daSPeter Grehan.It Pa /filename Ns Oo , Ns Li nocache Oc Ns Oo , Ns Li direct Oc Ns Oo , Ns Li ro Oc 170b13e60daSPeter Grehan.It Pa /dev/xxx Ns Oo , Ns Ar nocache Oc Ns Oo , Ns Ar direct Oc Ns Oo , Ns Ar ro Oc 171b13e60daSPeter Grehan.Bl -tag -width 8n 172b13e60daSPeter Grehan.It Li nocache 173b13e60daSPeter GrehanOpen the file with 174b13e60daSPeter Grehan.Dv O_DIRECT . 175b13e60daSPeter Grehan.It Li direct 176b13e60daSPeter GrehanOpen the file using 177b13e60daSPeter Grehan.Dv O_SYNC . 178b13e60daSPeter Grehan.It Li ro 179b13e60daSPeter GrehanForce the file to be opened read-only. 180b13e60daSPeter Grehan.El 181b13e60daSPeter Grehan.Pp 182b13e60daSPeter GrehanThe 183b13e60daSPeter Grehan.Li nocache , 184b13e60daSPeter Grehan.Li direct , 185b13e60daSPeter Grehanand 186b13e60daSPeter Grehan.Li ro 187b13e60daSPeter Grehanoptions are not available for virtio block devices. 188b13e60daSPeter Grehan.El 189b13e60daSPeter Grehan.Pp 190b13e60daSPeter GrehanTTY devices: 191b13e60daSPeter Grehan.Bl -tag -width 10n 192b13e60daSPeter Grehan.It Li stdio 193b13e60daSPeter GrehanConnect the serial port to the standard input and output of 194b13e60daSPeter Grehanthe bhyve process. 195b13e60daSPeter Grehan.It Pa /dev/xxx 196b13e60daSPeter GrehanUse the host TTY device for serial port I/O. 197b13e60daSPeter Grehan.El 198b13e60daSPeter Grehan.Pp 199b13e60daSPeter GrehanPass-through devices: 200b13e60daSPeter Grehan.Bl -tag -width 10n 201b13e60daSPeter Grehan.It Ns Ar slot Ns / Ns Ar bus Ns / Ns Ar function 202b13e60daSPeter GrehanConnect to a PCI device on the host at the selector described by 203b13e60daSPeter Grehan.Ar slot , 204b13e60daSPeter Grehan.Ar bus , 205b13e60daSPeter Grehanand 206b13e60daSPeter Grehan.Ar function 207b13e60daSPeter Grehannumbers. 208b13e60daSPeter Grehan.El 209b13e60daSPeter Grehan.Pp 210b13e60daSPeter GrehanThe host device must have been reserved at boot-time using the 211b13e60daSPeter Grehan.Va pptdev 212b13e60daSPeter Grehanloader variable as described in 213b13e60daSPeter Grehan.Xr vmm 4 . 214b13e60daSPeter Grehan.El 215b13e60daSPeter Grehan.It Fl l Ar lpcdev Ns Op , Ns Ar conf 216b13e60daSPeter GrehanAllow devices behind the LPC PCI-ISA bridge to be configured. 217b13e60daSPeter GrehanThe only supported devices are the TTY-class devices, 218b13e60daSPeter Grehan.Li com1 219b13e60daSPeter Grehanand 220b13e60daSPeter Grehan.Li com2 . 221b13e60daSPeter Grehan.It Fl m Ar size Ns Op Ar K|k|M|m|G|g|T|t 222b13e60daSPeter GrehanGuest physical memory size in bytes. 223b13e60daSPeter GrehanThis must be the same size that was given to 224b13e60daSPeter Grehan.Xr bhyveload 8 . 225b13e60daSPeter Grehan.Pp 226b13e60daSPeter GrehanThe size argument may be suffixed with one of K, M, G or T (either upper 227b13e60daSPeter Grehanor lower case) to indicate a multiple of kilobytes, megabytes, gigabytes, 228b13e60daSPeter Grehanor terabytes. 229b13e60daSPeter GrehanIf no suffix is given, the value is assumed to be in megabytes. 230b13e60daSPeter Grehan.It Fl e 231b13e60daSPeter GrehanForce 232b13e60daSPeter Grehan.Nm 233b13e60daSPeter Grehanto exit when a guest issues an access to an I/O port that is not emulated. 234b13e60daSPeter GrehanThis is intended for debug purposes. 235851d84f1SNeel Natu.It Fl w 236851d84f1SNeel NatuIgnore accesses to unimplemented Model Specific Registers (MSRs). This is intended for debug purposes. 2379777ca20SNeel Natu.It Fl x 2389777ca20SNeel NatuThe guest's local APIC is configured in x2APIC mode. 239b13e60daSPeter Grehan.It Fl h 240b13e60daSPeter GrehanPrint help message and exit. 241b13e60daSPeter Grehan.It Ar vmname 242b13e60daSPeter GrehanAlphanumeric name of the guest. 243b13e60daSPeter GrehanThis should be the same as that created by 244b13e60daSPeter Grehan.Xr bhyveload 8 . 245b13e60daSPeter Grehan.El 246b13e60daSPeter Grehan.Sh EXAMPLES 247b13e60daSPeter GrehanThe guest operating system must have been loaded with 248b13e60daSPeter Grehan.Xr bhyveload 4 249b13e60daSPeter Grehanor a similar boot loader before 250b13e60daSPeter Grehan.Xr bhyve 4 251b13e60daSPeter Grehancan be run. 252b13e60daSPeter Grehan.Pp 253b13e60daSPeter GrehanTo run a virtual machine with 1GB of memory, two virtual CPUs, a virtio 254b13e60daSPeter Grehanblock device backed by the 255b13e60daSPeter Grehan.Pa /my/image 256b13e60daSPeter Grehanfilesystem image, and a serial port for the console: 257b13e60daSPeter Grehan.Bd -literal -offset indent 258b13e60daSPeter Grehanbhyve -c 2 -s 0,hostbridge -s 1,lpc -s 2,virtio-blk,/my/image \\ 259b13e60daSPeter Grehan -l com1,stdio -A -H -P -m 1G vm1 260b13e60daSPeter Grehan.Ed 261b13e60daSPeter Grehan.Pp 262b13e60daSPeter GrehanRun a 24GB single-CPU virtual machine with three network ports, one of which 263b13e60daSPeter Grehanhas a MAC address specified: 264b13e60daSPeter Grehan.Bd -literal -offset indent 265b13e60daSPeter Grehanbhyve -s 0,hostbridge -s 1,lpc -s 2:0,virtio-net,tap0 \\ 266b13e60daSPeter Grehan -s 2:1,virtio-net,tap1 \\ 267b13e60daSPeter Grehan -s 2:2,virtio-net,tap2,mac=00:be:fa:76:45:00 \\ 268b13e60daSPeter Grehan -s 3,virtio-blk,/my/image -l com1,stdio \\ 269b13e60daSPeter Grehan -A -H -P -m 24G bigvm 270b13e60daSPeter Grehan.Ed 271b13e60daSPeter Grehan.Pp 272b13e60daSPeter GrehanRun an 8GB quad-CPU virtual machine with 8 AHCI SATA disks, an AHCI ATAPI 273b13e60daSPeter GrehanCD-ROM, a single virtio network port, an AMD hostbridge, and the console 274b13e60daSPeter Grehanport connected to an 275b13e60daSPeter Grehan.Xr nmdm 4 276b13e60daSPeter Grehannull-model device. 277b13e60daSPeter Grehan.Bd -literal -offset indent 278b13e60daSPeter Grehanbhyve -c 4 \e\ 279b13e60daSPeter Grehan -s 0,amd_hostbridge -s 1,lpc \\ 280b13e60daSPeter Grehan -s 1:0,ahci-hd,/images/disk.1 \\ 281b13e60daSPeter Grehan -s 1:1,ahci-hd,/images/disk.2 \\ 282b13e60daSPeter Grehan -s 1:2,ahci-hd,/images/disk.3 \\ 283b13e60daSPeter Grehan -s 1:3,ahci-hd,/images/disk.4 \\ 284b13e60daSPeter Grehan -s 1:4,ahci-hd,/images/disk.5 \\ 285b13e60daSPeter Grehan -s 1:5,ahci-hd,/images/disk.6 \\ 286b13e60daSPeter Grehan -s 1:6,ahci-hd,/images/disk.7 \\ 287b13e60daSPeter Grehan -s 1:7,ahci-hd,/images/disk.8 \\ 288b13e60daSPeter Grehan -s 2,ahci-cd,/images.install.iso \\ 289b13e60daSPeter Grehan -s 3,virtio-net,tap0 \\ 290b13e60daSPeter Grehan -l com1,/dev/nmdm0A \\ 291b13e60daSPeter Grehan -A -H -P -m 8G 292b13e60daSPeter Grehan.Ed 293b13e60daSPeter Grehan.Sh SEE ALSO 294b13e60daSPeter Grehan.Xr bhyve 4 , 295b13e60daSPeter Grehan.Xr nmdm 4 , 296b13e60daSPeter Grehan.Xr vmm 4 , 297b13e60daSPeter Grehan.Xr ethers 5 , 29805f7cd8bSJoel Dahl.Xr bhyvectl 8 , 29905f7cd8bSJoel Dahl.Xr bhyveload 8 300b13e60daSPeter Grehan.Sh HISTORY 301b13e60daSPeter Grehan.Nm 302b13e60daSPeter Grehanfirst appeared in 303b13e60daSPeter Grehan.Fx 10.0 . 304b13e60daSPeter Grehan.Sh AUTHORS 305b13e60daSPeter Grehan.An Neel Natu Aq neel@freebsd.org 306b13e60daSPeter Grehan.An Peter Grehan Aq grehan@freebsd.org 307