1.\" Copyright (c) 2013 Peter Grehan 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND 14.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE 17.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23.\" SUCH DAMAGE. 24.\" 25.Dd October 9, 2023 26.Dt BHYVE 8 27.Os 28.Sh NAME 29.Nm bhyve 30.Nd "run a guest operating system inside a virtual machine" 31.Sh SYNOPSIS 32.Nm 33.Op Fl AaCDeHhPSuWwxY 34.Oo 35.Sm off 36.Fl c\~ 37.Oo 38.Op Cm cpus= 39.Ar numcpus 40.Oc 41.Op Cm ,sockets= Ar n 42.Op Cm ,cores= Ar n 43.Op Cm ,threads= Ar n 44.Oc 45.Sm on 46.Oo Fl f 47.Sm off 48.Ar name Cm \&, 49.Oo 50.Cm string No | Cm file 51.Oc 52.Cm \&= Ar data 53.Sm on 54.Oc 55.Oo 56.Sm off 57.Fl G\~ 58.Oo Ar w Oc 59.Oo Ar bind_address Cm \&: Oc 60.Ar port 61.Sm on 62.Oc 63.Op Fl k Ar config_file 64.Op Fl K Ar layout 65.Oo Fl l 66.Sm off 67.Ar lpcdev Op Cm \&, Ar conf 68.Sm on 69.Oc 70.Sm off 71.Oo Fl m\~ 72.Ar memsize 73.Oo 74.Cm K | Cm k | Cm M | Cm m | Cm G | Cm g | Cm T | Cm t 75.Oc 76.Sm on 77.Oc 78.Op Fl o Ar var Ns Cm = Ns Ar value 79.Op Fl p Ar vcpu Ns Cm \&: Ns Ar hostcpu 80.Op Fl r Ar file 81.Sm off 82.Oo Fl s\~ 83.Ar slot Cm \&, Ar emulation Op Cm \&, Ar conf 84.Sm on 85.Oc 86.Op Fl U Ar uuid 87.Ar vmname 88.Nm 89.Fl l Cm help 90.Nm 91.Fl s Cm help 92.Sh DESCRIPTION 93.Nm 94is a hypervisor that runs guest operating systems inside a 95virtual machine. 96.Pp 97Parameters such as the number of virtual CPUs, amount of guest memory, and 98I/O connectivity can be specified with command-line parameters. 99.Pp 100If not using a boot ROM, the guest operating system must be loaded with 101.Xr bhyveload 8 102or a similar boot loader before running 103.Nm , 104otherwise, it is enough to run 105.Nm 106with a boot ROM of choice. 107.Pp 108.Nm 109runs until the guest operating system reboots or an unhandled hypervisor 110exit is detected. 111.Sh OPTIONS 112.Bl -tag -width 10n 113.It Fl A 114Generate ACPI tables. 115bhyve always generates ACPI tables so this option is obsolete. 116.It Fl a 117The guest's local APIC is configured in xAPIC mode. 118The xAPIC mode is the default setting so this option is redundant. 119It will be deprecated in a future version. 120.It Fl C 121Include guest memory in core file. 122.It Fl c Op Ar setting ... 123Number of guest virtual CPUs 124and/or the CPU topology. 125The default value for each of 126.Ar numcpus , 127.Ar sockets , 128.Ar cores , 129and 130.Ar threads 131is 1. 132The current maximum number of guest virtual CPUs is 16. 133If 134.Ar numcpus 135is not specified then it will be calculated from the other arguments. 136The topology must be consistent in that the 137.Ar numcpus 138must equal the product of 139.Ar sockets , 140.Ar cores , 141and 142.Ar threads . 143If a 144.Ar setting 145is specified more than once the last one has precedence. 146.It Fl D 147Destroy the VM on guest initiated power-off. 148.It Fl e 149Force 150.Nm 151to exit when a guest issues an access to an I/O port that is not emulated. 152This is intended for debug purposes. 153.It Fl f Ar name Ns Cm \&, Ns Oo Cm string Ns No | Ns Cm file Ns Oc Ns Cm \&= Ns Ar data 154Add a fw_cfg file 155.Ar name 156to the fw_cfg interface. 157If a 158.Cm string 159is specified, the fw_cfg file contains the string as data. 160If a 161.Cm file 162is specified, bhyve reads the file and adds the file content as fw_cfg data. 163.It Fl G Xo 164.Sm off 165.Oo Ar w Oc 166.Oo Ar bind_address Cm \&: Oc 167.Ar port 168.Sm on 169.Xc 170Start a debug server that uses the GDB protocol to export guest state to a 171debugger. 172An IPv4 TCP socket will be bound to the supplied 173.Ar bind_address 174and 175.Ar port 176to listen for debugger connections. 177Only a single debugger may be attached to the debug server at a time. 178If the option begins with 179.Sq w , 180.Nm 181will pause execution at the first instruction waiting for a debugger to attach. 182.It Fl H 183Yield the virtual CPU thread when a HLT instruction is detected. 184If this option is not specified, virtual CPUs will use 100% of a host CPU. 185.It Fl h 186Print help message and exit. 187.It Fl k Ar config_file 188Set configuration variables from a simple, key-value config file. 189Each line of the config file is expected to consist of a config variable 190name, an equals sign 191.It Fl K Ar layout 192Specify the keyboard layout. 193The value that can be specified sets the file name in 194.Ar /usr/share/bhyve/kbdlayout . 195This specification only works when loaded with UEFI mode for VNC. 196When using a VNC client that supports QEMU Extended Key Event Message (e.g. 197TigerVNC), this option isn't needed. 198When using a VNC client that doesn't support QEMU Extended Key Event Message 199(e.g. tightVNC), the layout defaults to the US keyboard unless specified 200otherwise. 201.Pq Sq = , 202and a value. 203No spaces are permitted between the variable name, equals sign, or 204value. 205Blank lines and lines starting with 206.Sq # 207are ignored. 208See 209.Xr bhyve_config 5 210for more details. 211.It Fl l Cm help 212Print a list of supported LPC devices. 213.It Fl l Ar lpcdev Ns Op Cm \&, Ns Ar conf 214Allow devices behind the LPC PCI-ISA bridge to be configured. 215The only supported devices are the TTY-class devices 216.Cm com1 , com2 , com3 , 217and 218.Cm com4 , 219the TPM module 220.Cm tpm , 221the boot ROM device 222.Cm bootrom , 223the 224.Cm fwcfg 225type and the debug/test device 226.Cm pc-testdev . 227.Pp 228The possible values for the 229.Ar conf 230argument are listed in the 231.Fl s 232flag description. 233.It Xo 234.Fl m Ar memsize Ns Oo 235.Sm off 236.Cm K | k | M | m | G | g | T | t 237.Sm on 238.Oc 239.Xc 240Set the guest physical memory size 241This must be the same size that was given to 242.Xr bhyveload 8 . 243.Pp 244The size argument may be suffixed with one of 245.Cm K , M , G 246or 247.Cm T 248(either upper or lower case) 249to indicate a multiple of kilobytes, megabytes, gigabytes, or terabytes. 250If no suffix is given, the value is assumed to be in megabytes. 251.Pp 252The default is 256M. 253.It Fl o Ar var Ns Cm = Ns Ar value 254Set the configuration variable 255.Ar var 256to 257.Ar value . 258.It Fl P 259Force the guest virtual CPU to exit when a PAUSE instruction is detected. 260.It Fl p Ar vcpu Ns Cm \& : Ns Ar hostcpu 261Pin guest's virtual CPU 262.Em vcpu 263to 264.Em hostcpu . 265Host CPUs and guest virtual CPUs are numbered starting from 0. 266A 267.Fl p 268option is required for every guest vCPU to be pinned. 269To map a 4 vCPU guest to host CPUs 12-15: 270.Bd -literal 271-p 0:12 -p 1:13 -p 2:14 -p 3:15 272.Ed 273.It Fl r Ar file 274Resume a guest from a snapshot. 275The guest memory contents are restored from 276.Ar file , 277and the guest device and vCPU state are restored from the file 278.Dq Ar file Ns .kern . 279.Pp 280Note that the current snapshot file format requires that the 281configuration of devices in the new VM match the VM from which the 282snapshot was taken by specifying the same 283.Fl s 284and 285.Fl l 286options. 287The count of vCPUs and memory configuration are read from the snapshot. 288.It Fl S 289Wire guest memory. 290.It Fl s Cm help 291Print a list of supported PCI devices. 292.It Fl s Ar slot Ns Cm \&, Ns Ar emulation Ns Op Cm \&, Ns Ar conf 293Configure a virtual PCI slot and function. 294.Pp 295.Nm 296provides PCI bus emulation and virtual devices that can be attached to 297slots on the bus. 298There are 32 available slots, with the option of providing up to 8 functions 299per slot. 300.Pp 301The 302.Ar slot 303can be specified in one of the following formats: 304.Pp 305.Bl -bullet -compact 306.It 307.Ar pcislot 308.It 309.Sm off 310.Ar pcislot Cm \&: Ar function 311.Sm on 312.It 313.Sm off 314.Ar bus Cm \&: Ar pcislot Cm \&: Ar function 315.Sm on 316.El 317.Pp 318The 319.Ar pcislot 320value is 0 to 31. 321The optional 322.Ar function 323value is 0 to 7. 324The optional 325.Ar bus 326value is 0 to 255. 327If not specified, the 328.Ar function 329value defaults to 0. 330If not specified, the 331.Ar bus 332value defaults to 0. 333.Pp 334The 335.Ar emulation 336argument 337can be one of the following: 338.Bl -tag -width "amd_hostbridge" 339.It Cm hostbridge 340A simple host bridge. 341This is usually configured at slot 0, and is required by most guest 342operating systems. 343.It Cm amd_hostbridge 344Emulation identical to 345.Cm hostbridge 346using a PCI vendor ID of AMD. 347.It Cm passthru 348PCI pass-through device. 349.It Cm virtio-net 350Virtio network interface. 351.It Cm virtio-blk 352Virtio block storage interface. 353.It Cm virtio-scsi 354Virtio SCSI interface. 355.It Cm virtio-9p 356Virtio 9p (VirtFS) interface. 357.It Cm virtio-rnd 358Virtio RNG interface. 359.It Cm virtio-console 360Virtio console interface, which exposes multiple ports 361to the guest in the form of simple char devices for simple IO 362between the guest and host userspaces. 363.It Cm virtio-input 364Virtio input interface. 365.It Cm ahci 366AHCI controller attached to arbitrary devices. 367.It Cm ahci-cd 368AHCI controller attached to an ATAPI CD/DVD. 369.It Cm ahci-hd 370AHCI controller attached to a SATA hard drive. 371.It Cm e1000 372Intel e82545 network interface. 373.It Cm uart 374PCI 16550 serial device. 375.It Cm lpc 376LPC PCI-ISA bridge with COM1, COM2, COM3, and COM4 16550 serial ports, 377a boot ROM, and, 378optionally, a fwcfg type and the debug/test device. 379The LPC bridge emulation can only be configured on bus 0. 380.It Cm fbuf 381Raw framebuffer device attached to VNC server. 382.It Cm xhci 383eXtensible Host Controller Interface (xHCI) USB controller. 384.It Cm nvme 385NVM Express (NVMe) controller. 386.It Cm hda 387High Definition Audio Controller. 388.El 389.Pp 390The optional parameter 391.Ar conf 392describes the backend for device emulations. 393If 394.Ar conf 395is not specified, the device emulation has no backend and can be 396considered unconnected. 397.Pp 398Network device backends: 399.Sm off 400.Bl -bullet 401.It 402.Xo 403.Cm tap Ar N 404.Op Cm \&,mac= Ar xx:xx:xx:xx:xx:xx 405.Op Cm \&,mtu= Ar N 406.Xc 407.It 408.Xo 409.Cm vmnet Ar N 410.Op Cm \&,mac= Ar xx:xx:xx:xx:xx:xx 411.Op Cm \&,mtu= Ar N 412.Xc 413.It 414.Xo 415.Cm netgraph,path= Ar ADDRESS Cm \&,peerhook= Ar HOOK 416.Op Cm \&,socket= Ar NAME 417.Op Cm \&,hook= Ar HOOK 418.Op Cm \&,mac= Ar xx:xx:xx:xx:xx:xx 419.Op Cm \&,mtu= Ar N 420.Xc 421.El 422.Sm on 423.Pp 424If 425.Cm mac 426is not specified, the MAC address is derived from a fixed OUI and the 427remaining bytes from an MD5 hash of the slot and function numbers and 428the device name. 429.Pp 430The MAC address is an ASCII string in 431.Xr ethers 5 432format. 433.Pp 434With 435.Cm virtio-net 436devices, the 437.Cm mtu 438parameter can be specified to inform the guest about the largest MTU 439that should be allowed, expressed in bytes. 440.Pp 441With 442.Cm netgraph 443backend, the 444.Cm path 445and 446.Cm peerhook 447parameters must be specified to set the destination node and corresponding hook. 448The optional parameters 449.Cm socket 450and 451.Cm hook 452may be used to set the 453.Xr ng_socket 4 454node name and source hook. 455The 456.Ar ADDRESS , 457.Ar HOOK , 458and 459.Ar NAME 460must comply with 461.Xr netgraph 4 462addressing rules. 463.Pp 464Block storage device backends: 465.Sm off 466.Bl -bullet 467.It 468.Ar /filename Op Cm \&, Ar block-device-options 469.It 470.Ar /dev/xxx Op Cm \&, Ar block-device-options 471.El 472.Sm on 473.Pp 474The 475.Ar block-device-options 476are: 477.Bl -tag -width 10n 478.It Cm nocache 479Open the file with 480.Dv O_DIRECT . 481.It Cm direct 482Open the file using 483.Dv O_SYNC . 484.It Cm ro 485Force the file to be opened read-only. 486.It Cm sectorsize= Ns Ar logical Ns Oo Cm \&/ Ns Ar physical Oc 487Specify the logical and physical sector sizes of the emulated disk. 488The physical sector size is optional and is equal to the logical sector size 489if not explicitly specified. 490.It Cm nodelete 491Disable emulation of guest trim requests via 492.Dv DIOCGDELETE 493requests. 494.It Li bootindex= Ns Ar index 495Add the device to the bootorder at 496.Ar index . 497A fwcfg file is used to specify the bootorder. 498The guest firmware may ignore or doesn't support this fwcfg file. 499In that case, this feature doesn't work as expected. 500.El 501.Pp 502SCSI device backends: 503.Sm off 504.Bl -bullet 505.It 506.Pa /dev/cam/ctl Oo Ar pp Cm \&. Ar vp Oc Oo Cm \&, Ar scsi-device-options Oc 507.El 508.Sm on 509.Pp 510The 511.Ar scsi-device-options 512are: 513.Bl -tag -width 10n 514.It Cm iid= Ns Ar IID 515Initiator ID to use when sending requests to specified CTL port. 516The default value is 0. 517.It Li bootindex= Ns Ar index 518Add the device to the bootorder at 519.Ar index . 520A fwcfg file is used to specify the bootorder. 521The guest firmware may ignore or doesn't support this fwcfg file. 522In that case, this feature doesn't work as expected. 523.El 524.Pp 5259P device backends: 526.Sm off 527.Bl -bullet 528.It 529.Ar sharename Cm = Ar /path/to/share Op Cm \&, Ar 9p-device-options 530.El 531.Sm on 532.Pp 533The 534.Ar 9p-device-options 535are: 536.Bl -tag -width 10n 537.It Cm ro 538Expose the share in read-only mode. 539.El 540.Pp 541TTY device backends: 542.Bl -tag -width 10n 543.It Cm stdio 544Connect the serial port to the standard input and output of 545the 546.Nm 547process. 548.It Ar /dev/xxx 549Use the host TTY device for serial port I/O. 550.El 551.Pp 552TPM device backends: 553.Bl -tag -width 10n 554.It Ar type Ns \&, Ns Ar path Ns Op Cm \&, Ns Ar tpm-device-options 555Emulate a TPM device. 556.El 557.Pp 558The 559.Ar tpm-device-options 560are: 561.Bl -tag -width 10n 562.It Cm version= Ns Ar version 563Version of the TPM device according to the TCG specification. 564Defaults to 565.Cm 2.0 566.El 567.El 568.Pp 569Boot ROM device backends: 570.Bl -tag -width 10n 571.It Ar romfile Ns Op Cm \&, Ns Ar varfile 572Map 573.Ar romfile 574in the guest address space reserved for boot firmware. 575If 576.Ar varfile 577is provided, that file is also mapped in the boot firmware guest 578address space, and any modifications the guest makes will be saved 579to that file. 580.El 581.Pp 582Fwcfg types: 583.Bl -tag -width 10n 584.It Ar fwcfg 585The fwcfg interface is used to pass information such as the CPU count 586or ACPI tables to the guest firmware. 587Supported values are 588.Ql bhyve 589and 590.Ql qemu . 591Due to backward compatibility reasons, 592.Ql bhyve 593is the default option. 594When 595.Ql bhyve 596is used, bhyve's fwctl interface is used. 597It currently reports only the CPU count to the guest firmware. 598The 599.Ql qemu 600option uses QEMU's fwcfg interface. 601This interface is widely used and allows user-defined information to 602be passed to the guest. 603It is used for passing the CPU count, ACPI tables, a boot order and 604many other things to the guest. 605Some operating systems such as Fedora CoreOS can be configured by 606qemu's fwcfg interface as well. 607.El 608.Pp 609Pass-through device backends: 610.Sm off 611.Bl -bullet 612.It 613.Cm ppt Ar N Oo , Ar passthru-device-options Oc 614.It 615.Ns Ar bus Cm \&/ Ar slot Cm \&/ Ar function 616.Op , Ar passthru-device-options 617.It 618.Cm pci Ar bus Cm : Ar slot Cm : Ns Ar function 619.Op , Ar passthru-device-options 620.El 621.Sm on 622.Pp 623Connect to a PCI device on the host either named ppt 624.Ns Ar N 625or at the selector described by 626.Ar slot , 627.Ar bus , 628and 629.Ar function 630numbers. 631.Pp 632The 633.Ar passthru-device-options 634are: 635.Bl -tag -width 10n 636.It Cm rom= Ns Ar romfile 637Add 638.Ar romfile 639as option ROM to the PCI device. 640The ROM will be loaded by firmware and should be capable of 641initializing the device. 642.It Li bootindex= Ns Ar index 643Add the device to the bootorder at 644.Ar index . 645A fwcfg file is used to specify the bootorder. 646The guest firmware may ignore or doesn't support this fwcfg file. 647In that case, this feature doesn't work as expected. 648.El 649.Pp 650Guest memory must be wired using the 651.Fl S 652option when a pass-through device is configured. 653.Pp 654The host device must have been reserved at boot-time using the 655.Va pptdevs 656loader variable as described in 657.Xr vmm 4 . 658.Pp 659TPM devices: 660.Bl -tag -width 10n 661.It Ns Ar type 662Specifies the type of the TPM device. 663.Pp 664Supported types: 665.Bl -tag -width 10n 666.It Cm passthru 667.El 668.It Cm version= Ns Ar version 669The 670.Ar version 671of the emulated TPM device according to the TCG specification. 672.Pp 673Supported versions: 674.Bl -tag -width 10n 675.It Cm 2.0 676.El 677.El 678.Pp 679Virtio console device backends: 680.Bl -bullet 681.Sm off 682.It 683.Cm port1= Ns Ar /path/to/port1.sock Ns Op Cm ,port Ns Ar N Cm \&= Ns Ar /path/to/port2.sock No \~ Ar ... 684.Sm on 685.El 686.Pp 687A maximum of 16 ports per device can be created. 688Every port is named and corresponds to a Unix domain socket created by 689.Nm . 690.Nm 691accepts at most one connection per port at a time. 692.Pp 693Limitations: 694.Bl -bullet 695.It 696Due to lack of destructors in 697.Nm , 698sockets on the filesystem must be cleaned up manually after 699.Nm 700exits. 701.It 702There is no way to use the 703.Dq console port 704feature, nor the console port 705resize at present. 706.It 707Emergency write is advertised, but no-op at present. 708.El 709.Pp 710Virtio input device backends: 711.Bl -tag -width 10n 712.It Ar /dev/input/eventX 713Send input events of 714.Ar /dev/input/eventX 715to guest by VirtIO Input Interface. 716.El 717.Pp 718Framebuffer devices backends: 719.Bl -bullet 720.Sm off 721.It 722.Op Cm rfb= Ar ip-and-port 723.Op Cm ,w= Ar width 724.Op Cm ,h= Ar height 725.Op Cm ,vga= Ar vgaconf 726.Op Cm ,wait 727.Op Cm ,password= Ar password 728.Sm on 729.El 730.Pp 731Configuration options are defined as follows: 732.Bl -tag -width 10n 733.It Cm rfb= Ns Ar ip-and-port Pq or Cm tcp= Ns Ar ip-and-port 734An IP address and a port VNC should listen on. 735There are two formats: 736.Pp 737.Bl -bullet -compact 738.It 739.Sm off 740.Op Ar IPv4 Cm \&: 741.Ar port 742.Sm on 743.It 744.Sm off 745.Cm \&[ Ar IPv6%zone Cm \&] Cm \&: Ar port 746.Sm on 747.El 748.Pp 749The default is to listen on localhost IPv4 address and default VNC port 5900. 750An IPv6 address must be enclosed in square brackets and may contain an 751optional zone identifier. 752.It Cm w= Ns Ar width No and Cm h= Ns Ar height 753A display resolution, width and height, respectively. 754If not specified, a default resolution of 1024x768 pixels will be used. 755Minimal supported resolution is 640x480 pixels, 756and maximum is 1920x1200 pixels. 757.It Cm vga= Ns Ar vgaconf 758Possible values for this option are 759.Cm io 760(default), 761.Cm on 762, and 763.Cm off . 764PCI graphics cards have a dual personality in that they are 765standard PCI devices with BAR addressing, but may also 766implicitly decode legacy VGA I/O space 767.Pq Ad 0x3c0-3df 768and memory space 769.Pq 64KB at Ad 0xA0000 . 770The default 771.Cm io 772option should be used for guests that attempt to issue BIOS calls which result 773in I/O port queries, and fail to boot if I/O decode is disabled. 774.Pp 775The 776.Cm on 777option should be used along with the CSM BIOS capability in UEFI 778to boot traditional BIOS guests that require the legacy VGA I/O and 779memory regions to be available. 780.Pp 781The 782.Cm off 783option should be used for the UEFI guests that assume that 784VGA adapter is present if they detect the I/O ports. 785An example of such a guest is 786.Ox 787in UEFI mode. 788.Pp 789Please refer to the 790.Nm 791.Fx 792wiki page 793.Pq Lk https://wiki.freebsd.org/bhyve 794for configuration notes of particular guests. 795.It Cm wait 796Instruct 797.Nm 798to only boot upon the initiation of a VNC connection, simplifying the 799installation of operating systems that require immediate keyboard input. 800This can be removed for post-installation use. 801.It Cm password= Ns Ar password 802This type of authentication is known to be cryptographically weak and is not 803intended for use on untrusted networks. 804Many implementations will want to use stronger security, such as running 805the session over an encrypted channel provided by IPsec or SSH. 806.El 807.Pp 808xHCI USB device backends: 809.Bl -tag -width 10n 810.It Cm tablet 811A USB tablet device which provides precise cursor synchronization 812when using VNC. 813.El 814.Pp 815NVMe device backends: 816.Bl -bullet 817.Sm off 818.It 819.Ar devpath 820.Op Cm ,maxq= Ar # 821.Op Cm ,qsz= Ar # 822.Op Cm ,ioslots= Ar # 823.Op Cm ,sectsz= Ar # 824.Op Cm ,ser= Ar # 825.Op Cm ,eui64= Ar # 826.Op Cm ,dsm= Ar opt 827.Sm on 828.El 829.Pp 830Configuration options are defined as follows: 831.Bl -tag -width 10n 832.It Ar devpath 833Accepted device paths are: 834.Ar /dev/blockdev 835or 836.Ar /path/to/image 837or 838.Cm ram= Ns Ar size_in_MiB . 839.It Cm maxq 840Max number of queues. 841.It Cm qsz 842Max elements in each queue. 843.It Cm ioslots 844Max number of concurrent I/O requests. 845.It Cm sectsz 846Sector size (defaults to blockif sector size). 847.It Cm ser 848Serial number with maximum 20 characters. 849.It Cm eui64 850IEEE Extended Unique Identifier (8 byte value). 851.It Cm dsm 852DataSet Management support. 853Supported values are: 854.Cm auto , enable , 855and 856.Cm disable . 857.El 858.Pp 859AHCI device backends: 860.Bl -bullet 861.It 862.Sm off 863.Op Oo Cm hd\&: | cd\&: Oc Ar path 864.Op Cm ,nmrr= Ar nmrr 865.Op Cm ,ser= Ar # 866.Op Cm ,rev= Ar # 867.Op Cm ,model= Ar # 868.Sm on 869.El 870.Pp 871Configuration options are defined as follows: 872.Bl -tag -width 10n 873.It Cm nmrr 874Nominal Media Rotation Rate, known as RPM. 875Value 1 will indicate device as Solid State Disk. 876Default value is 0, not report. 877.It Cm ser 878Serial Number with maximum 20 characters. 879.It Cm rev 880Revision Number with maximum 8 characters. 881.It Cm model 882Model Number with maximum 40 characters. 883.El 884.Pp 885HD Audio device backends: 886.Bl -bullet 887.It 888.Sm off 889.Op Cm play= Ar playback 890.Op Cm ,rec= Ar recording 891.Sm on 892.El 893.Pp 894Configuration options are defined as follows: 895.Bl -tag -width 10n 896.It Cm play 897Playback device, typically 898.Ar /dev/dsp0 . 899.It Cm rec 900Recording device, typically 901.Ar /dev/dsp0 . 902.El 903.It Fl U Ar uuid 904Set the universally unique identifier 905.Pq UUID 906in the guest's System Management BIOS System Information structure. 907By default a UUID is generated from the host's hostname and 908.Ar vmname . 909.It Fl u 910RTC keeps UTC time. 911.It Fl W 912Force virtio PCI device emulations to use MSI interrupts instead of MSI-X 913interrupts. 914.It Fl w 915Ignore accesses to unimplemented Model Specific Registers (MSRs). 916This is intended for debug purposes. 917.It Fl x 918The guest's local APIC is configured in x2APIC mode. 919.It Fl Y 920Disable MPtable generation. 921.It Ar vmname 922Alphanumeric name of the guest. 923This should be the same as that created by 924.Xr bhyveload 8 . 925.El 926.Sh CONFIGURATION VARIABLES 927.Nm 928uses an internal tree of configuration variables to describe global and 929per-device settings. 930When 931.Nm 932starts, 933it parses command line options (including config files) in the order given 934on the command line. 935Each command line option sets one or more configuration variables. 936For example, 937the 938.Fl s 939option creates a new tree node for a PCI device and sets one or more variables 940under that node including the device model and device model-specific variables. 941Variables may be set multiple times during this parsing stage with the final 942value overriding previous values. 943.Pp 944Once all of the command line options have been processed, 945the configuration values are frozen. 946.Nm 947then uses the value of configuration values to initialize device models 948and global settings. 949.Pp 950More details on configuration variables can be found in 951.Xr bhyve_config 5 . 952.Sh DEBUG SERVER 953The current debug server provides limited support for debuggers. 954.Ss Registers 955Each virtual CPU is exposed to the debugger as a thread. 956.Pp 957General purpose registers can be queried for each virtual CPU, but other 958registers such as floating-point and system registers cannot be queried. 959.Ss Memory 960Memory (including memory mapped I/O regions) can be read and written 961by the debugger. 962Memory operations use virtual addresses that are resolved to physical 963addresses via the current virtual CPU's active address translation. 964.Ss Control 965The running guest can be interrupted by the debugger at any time 966.Pq for example, by pressing Ctrl-C in the debugger . 967.Pp 968Single stepping is only supported on Intel CPUs supporting the MTRAP VM exit. 969.Pp 970Breakpoints are supported on Intel CPUs that support single stepping. 971Note that continuing from a breakpoint while interrupts are enabled in the 972guest may not work as expected due to timer interrupts firing while single 973stepping over the breakpoint. 974.Sh SIGNAL HANDLING 975.Nm 976deals with the following signals: 977.Pp 978.Bl -tag -width SIGTERM -compact 979.It SIGTERM 980Trigger ACPI poweroff for a VM 981.El 982.Sh EXIT STATUS 983Exit status indicates how the VM was terminated: 984.Pp 985.Bl -tag -width indent -compact 986.It 0 987rebooted 988.It 1 989powered off 990.It 2 991halted 992.It 3 993triple fault 994.It 4 995exited due to an error 996.El 997.Sh EXAMPLES 998If not using a boot ROM, the guest operating system must have been loaded with 999.Xr bhyveload 8 1000or a similar boot loader before 1001.Xr bhyve 4 1002can be run. 1003Otherwise, the boot loader is not needed. 1004.Pp 1005To run a virtual machine with 1GB of memory, two virtual CPUs, a virtio 1006block device backed by the 1007.Pa /my/image 1008filesystem image, and a serial port for the console: 1009.Bd -literal -offset indent 1010bhyve -c 2 -s 0,hostbridge -s 1,lpc -s 2,virtio-blk,/my/image \\ 1011 -l com1,stdio -A -H -P -m 1G vm1 1012.Ed 1013.Pp 1014Run a 24GB single-CPU virtual machine with three network ports, one of which 1015has a MAC address specified: 1016.Bd -literal -offset indent 1017bhyve -s 0,hostbridge -s 1,lpc -s 2:0,virtio-net,tap0 \\ 1018 -s 2:1,virtio-net,tap1 \\ 1019 -s 2:2,virtio-net,tap2,mac=00:be:fa:76:45:00 \\ 1020 -s 3,virtio-blk,/my/image -l com1,stdio \\ 1021 -A -H -P -m 24G bigvm 1022.Ed 1023.Pp 1024Run an 8GB quad-CPU virtual machine with 8 AHCI SATA disks, an AHCI ATAPI 1025CD-ROM, a single virtio network port, an AMD hostbridge, and the console 1026port connected to an 1027.Xr nmdm 4 1028null-modem device. 1029.Bd -literal -offset indent 1030bhyve -c 4 \\ 1031 -s 0,amd_hostbridge -s 1,lpc \\ 1032 -s 1:0,ahci,hd:/images/disk.1,hd:/images/disk.2,\\ 1033hd:/images/disk.3,hd:/images/disk.4,\\ 1034hd:/images/disk.5,hd:/images/disk.6,\\ 1035hd:/images/disk.7,hd:/images/disk.8,\\ 1036cd:/images/install.iso \\ 1037 -s 3,virtio-net,tap0 \\ 1038 -l com1,/dev/nmdm0A \\ 1039 -A -H -P -m 8G 1040.Ed 1041.Pp 1042Run a UEFI virtual machine with a display resolution of 800 by 600 pixels 1043that can be accessed via VNC at: 0.0.0.0:5900. 1044.Bd -literal -offset indent 1045bhyve -c 2 -m 4G -w -H \\ 1046 -s 0,hostbridge \\ 1047 -s 3,ahci-cd,/path/to/uefi-OS-install.iso \\ 1048 -s 4,ahci-hd,disk.img \\ 1049 -s 5,virtio-net,tap0 \\ 1050 -s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait \\ 1051 -s 30,xhci,tablet \\ 1052 -s 31,lpc -l com1,stdio \\ 1053 -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \\ 1054 uefivm 1055.Ed 1056.Pp 1057Run a UEFI virtual machine with a VNC display that is bound to all IPv6 1058addresses on port 5900. 1059.Bd -literal -offset indent 1060bhyve -c 2 -m 4G -w -H \\ 1061 -s 0,hostbridge \\ 1062 -s 4,ahci-hd,disk.img \\ 1063 -s 5,virtio-net,tap0 \\ 1064 -s 29,fbuf,tcp=[::]:5900,w=800,h=600 \\ 1065 -s 30,xhci,tablet \\ 1066 -s 31,lpc -l com1,stdio \\ 1067 -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \\ 1068 uefivm 1069.Ed 1070.Pp 1071Run a UEFI virtual machine with a VARS file to save EFI variables. 1072Note that 1073.Nm 1074will write guest modifications to the given VARS file. 1075Be sure to create a per-guest copy of the template VARS file from 1076.Pa /usr . 1077.Bd -literal -offset indent 1078bhyve -c 2 -m 4g -w -H \\ 1079 -s 0,hostbridge \\ 1080 -s 31,lpc -l com1,stdio \\ 1081 -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd,BHYVE_UEFI_VARS.fd 1082 uefivm 1083.Ed 1084.Sh SEE ALSO 1085.Xr bhyve 4 , 1086.Xr netgraph 4 , 1087.Xr ng_socket 4 , 1088.Xr nmdm 4 , 1089.Xr vmm 4 , 1090.Xr bhyve_config 5 , 1091.Xr ethers 5 , 1092.Xr bhyvectl 8 , 1093.Xr bhyveload 8 1094.Pp 1095.Rs 1096.%A Intel 1097.%B 64 and IA-32 Architectures Software Developer’s Manual 1098.%V Volume 3 1099.Re 1100.Sh HISTORY 1101.Nm 1102first appeared in 1103.Fx 10.0 . 1104.Sh AUTHORS 1105.An Neel Natu Aq Mt neel@freebsd.org 1106.An Peter Grehan Aq Mt grehan@freebsd.org 1107