bhyve: Initialize more registers in vcpu_reset()

- Clear CR2, EFER, and R8-15 to zero.
- Reset DR6 and DR7 to their documented reset values.
- Reset interrupt shadow state.
- Document the reason CR0

bhyve: Initialize more registers in vcpu_reset()

- Clear CR2, EFER, and R8-15 to zero.
- Reset DR6 and DR7 to their documented reset values.
- Reset interrupt shadow state.
- Document the reason CR0 is reset to a value that doesn't match its
documented value.

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D35622
Sponsored by: Beckhoff Automation GmbH & Co. KG

show more ...

libvmmapi: Add vm_close()

Currently there is no way to safely free a vm structure without
leaking the fd. vm_destroy() closes the fd but also destroys the VM
whereas in some cases a VM needs to be

libvmmapi: Add vm_close()

Currently there is no way to safely free a vm structure without
leaking the fd. vm_destroy() closes the fd but also destroys the VM
whereas in some cases a VM needs to be opened (vm_open) and then
closed (vm_close).

Reviewed by: jhb
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D35073

show more ...

libvmm: constify vm_get_name()

Allows callers of vm_get_name() to retrieve the vm name without having
to allocate a buffer.

While in the vicinity, do minor cleanup in vm_snapshot_basic_metadata().

libvmm: constify vm_get_name()

Allows callers of vm_get_name() to retrieve the vm name without having
to allocate a buffer.

While in the vicinity, do minor cleanup in vm_snapshot_basic_metadata().

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D34290

show more ...

Extend the VMM stats interface to support a dynamic count of statistics.

- Add a starting index to 'struct vmstats' and change the
VM_STATS ioctl to fetch the 64 stats starting at that index.
A

Extend the VMM stats interface to support a dynamic count of statistics.

- Add a starting index to 'struct vmstats' and change the
VM_STATS ioctl to fetch the 64 stats starting at that index.
A compat shim for <= 13 continues to fetch only the first 64
stats.

- Extend vm_get_stats() in libvmmapi to use a loop and a static
thread local buffer which grows to hold the stats needed.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D27463

show more ...

Remove an always-true check.

This fixes a -Wtype-limits error from GCC 9.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D31936

libvmmapi: Fix warnings and stop overridding WARNS

- Avoid shadowing the global optarg.
- Sprinkle __unused.
- Cast nitems() to int.
- Fix sign in vm_copy_setup().

Reviewed by: grehan
MFC after: 1

libvmmapi: Fix warnings and stop overridding WARNS

- Avoid shadowing the global optarg.
- Sprinkle __unused.
- Cast nitems() to int.
- Fix sign in vm_copy_setup().

Reviewed by: grehan
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31306

show more ...

libvmm: explicitly save and restore errno in vm_open()

In commit 6bb140e3ca895a14, vm_destroy() was replaced with free() to
preserve errno. However, it's possible that free() may change the errno
as

libvmm: explicitly save and restore errno in vm_open()

In commit 6bb140e3ca895a14, vm_destroy() was replaced with free() to
preserve errno. However, it's possible that free() may change the errno
as well. Keep the free() call, but explicitly save and restore errno.

Noted by: jhb
Fixes: 6bb140e3ca895a14

show more ...

bhyve: support relocating fbuf and passthru data BARs

We want to allow the UEFI firmware to enumerate and assign
addresses to PCI devices so we can boot from NVMe[1]. Address
assignment of PCI BARs

bhyve: support relocating fbuf and passthru data BARs

We want to allow the UEFI firmware to enumerate and assign
addresses to PCI devices so we can boot from NVMe[1]. Address
assignment of PCI BARs is properly handled by the PCI emulation
code in general, but a few specific cases need additional support.
fbuf and passthru map additional objects into the guest physical
address space and so need to handle address updates. Here we add a
callback to emulated PCI devices to inform them of a BAR
configuration change. fbuf and passthru then watch for these BAR
changes and relocate the frame buffer memory segment and passthru
device mmio area respectively.

We also add new VM_MUNMAP_MEMSEG and VM_UNMAP_PPTDEV_MMIO ioctls
to vmm(4) to facilitate the unmapping needed for addres updates.

[1]: https://github.com/freebsd/uefi-edk2/pull/9/

Originally by: scottph
MFC After: 1 week
Sponsored by: Intel Corporation
Reviewed by: grehan
Approved by: philip (mentor)
Differential Revision: https://reviews.freebsd.org/D24066

show more ...

bhyvectl: print a better error message when vm_open() fails

Use errno to print a more descriptive error message when vm_open() fails

libvmm: preserve errno when vm_device_open() fails

vm_destroy()

bhyvectl: print a better error message when vm_open() fails

Use errno to print a more descriptive error message when vm_open() fails

libvmm: preserve errno when vm_device_open() fails

vm_destroy() squashes errno by making a dive into sysctlbyname() - we
can safely skip vm_destroy() here since it's not doing any critical
clean up at this point. Replace vm_destroy() with a free() call.

PR: 250671
MFC after: 3 days
Submitted by: marko@apache.org
Reviewed by: grehan
Differential Revision: https://reviews.freebsd.org/D29109

show more ...

Honor the disabled setting for MSI-X interrupts for passthrough devices.

Add a new ioctl to disable all MSI-X interrupts for a PCI passthrough
device and invoke it if a write to the MSI-X capability

Honor the disabled setting for MSI-X interrupts for passthrough devices.

Add a new ioctl to disable all MSI-X interrupts for a PCI passthrough
device and invoke it if a write to the MSI-X capability registers
disables MSI-X. This avoids leaving MSI-X interrupts enabled on the
host if a guest device driver has disabled them (e.g. as part of
detaching a guest device driver).

This was found by Chelsio QA when testing that a Linux guest could
switch from MSI-X to MSI interrupts when using the cxgb4vf driver.

While here, explicitly fail requests to enable MSI on a passthrough
device if MSI-X is enabled and vice versa.

Reported by: Sony Arpita Das @ Chelsio
Reviewed by: grehan, markj
MFC after: 2 weeks
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D27212

show more ...

vmm(4), bhyve(8): Expose kernel-emulated special devices to userspace

Expose the special kernel LAPIC, IOAPIC, and HPET devices to userspace
for use in, e.g., fallback instruction emulation (when us

vmm(4), bhyve(8): Expose kernel-emulated special devices to userspace

Expose the special kernel LAPIC, IOAPIC, and HPET devices to userspace
for use in, e.g., fallback instruction emulation (when userspace has a
newer instruction decode/emulation layer than the kernel vmm(4)).

Plumb the ioctl through libvmmapi and register the memory ranges in
bhyve(8).

Reviewed by: grehan
Differential Revision: https://reviews.freebsd.org/D24525

show more ...

Initial support for bhyve save and restore.

Save and restore (also known as suspend and resume) permits a snapshot
to be taken of a guest's state that can later be resumed. In the
current implement

Initial support for bhyve save and restore.

Save and restore (also known as suspend and resume) permits a snapshot
to be taken of a guest's state that can later be resumed. In the
current implementation, bhyve(8) creates a UNIX domain socket that is
used by bhyvectl(8) to send a request to save a snapshot (and
optionally exit after the snapshot has been taken). A snapshot
currently consists of two files: the first holds a copy of guest RAM,
and the second file holds other guest state such as vCPU register
values and device model state.

To resume a guest, bhyve(8) must be started with a matching pair of
command line arguments to instantiate the same set of device models as
well as a pointer to the saved snapshot.

While the current implementation is useful for several uses cases, it
has a few limitations. The file format for saving the guest state is
tied to the ABI of internal bhyve structures and is not
self-describing (in that it does not communicate the set of device
models present in the system). In addition, the state saved for some
device models closely matches the internal data structures which might
prove a challenge for compatibility of snapshot files across a range
of bhyve versions. The file format also does not currently support
versioning of individual chunks of state. As a result, the current
file format is not a fixed binary format and future revisions to save
and restore will break binary compatiblity of snapshot files. The
goal is to move to a more flexible format that adds versioning,
etc. and at that point to commit to providing a reasonable level of
compatibility. As a result, the current implementation is not enabled
by default. It can be enabled via the WITH_BHYVE_SNAPSHOT=yes option
for userland builds, and the kernel option BHYVE_SHAPSHOT.

Submitted by: Mihai Tiganus, Flavius Anton, Darius Mihai
Submitted by: Elena Mihailescu, Mihai Carabas, Sergiu Weisz
Relnotes: yes
Sponsored by: University Politehnica of Bucharest
Sponsored by: Matthew Grooms (student scholarships)
Sponsored by: iXsystems
Differential Revision: https://reviews.freebsd.org/D19495

show more ...

Map negative types passed to vm_capability_type2name to NULL.

Submitted by: vangyzen

Add description string for VM_CAP_BPT_EXIT.

While here, replace the array of mapping structures with an array of
string pointers where the index is the capability value.

Submitted by: Rob Fairbanks

Add description string for VM_CAP_BPT_EXIT.

While here, replace the array of mapping structures with an array of
string pointers where the index is the capability value.

Submitted by: Rob Fairbanks <rob.fx907@gmail.com>
Reviewed by: rgrimes
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D24289

show more ...

Forgotten to remove the previous if statement in commit r355838.

MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D19400

Attempt to load vmm(4) module before creating a guest using vm_create()
wrapper in libvmmapi.

Submitted by: Rob Fairbanks <rob.fx907_gmail.com>
Reviewed by: jhb
MFC after: 1 month
Differential Revis

Attempt to load vmm(4) module before creating a guest using vm_create()
wrapper in libvmmapi.

Submitted by: Rob Fairbanks <rob.fx907_gmail.com>
Reviewed by: jhb
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D19400

show more ...

Merge ^/head r338392 through r338594.

bhyve: Use MAP_GUARD when mapping guest memory ranges.

Instead of relying on PROT_NONE mappings with MAP_ANON, use MAP_GUARD
to reserve address space around guest memory ranges including the
guard r

bhyve: Use MAP_GUARD when mapping guest memory ranges.

Instead of relying on PROT_NONE mappings with MAP_ANON, use MAP_GUARD
to reserve address space around guest memory ranges including the
guard ranges of address space around mappings.

Submitted by: Shawn Webb
Reviewed by: araujo
Approved by: re (rgrimes)
MFC after: 1 month
Sponsored by: HardendBSD and G2, Inc
Differential Revision: https://reviews.freebsd.org/D16822

show more ...

Fix style(9) space vs tab.

Reviewed by: jhb
MFC after: 3 weeks.
Sponsored by: iXsystems Inc.
Differential Revision: https://reviews.freebsd.org/D15774

Add the ability to control the CPU topology of created VMs
from userland without the need to use sysctls, it allows the old
sysctls to continue to function, but deprecates them at
FreeBSD_version 120

Add the ability to control the CPU topology of created VMs
from userland without the need to use sysctls, it allows the old
sysctls to continue to function, but deprecates them at
FreeBSD_version 1200060 (Relnotes for deprecate).

The command line of bhyve is maintained in a backwards compatible way.
The API of libvmmapi is maintained in a backwards compatible way.
The sysctl's are maintained in a backwards compatible way.

Added command option looks like:
bhyve -c [[cpus=]n][,sockets=n][,cores=n][,threads=n][,maxcpus=n]
The optional parts can be specified in any order, but only a single
integer invokes the backwards compatible parse. [,maxcpus=n] is
hidden by #ifdef until kernel support is added, though the api
is put in place.

bhyvectl --get-cpu-topology option added.

Reviewed by: grehan (maintainer, earlier version),
Reviewed by: bcr (manpages)
Approved by: bde (mentor), phk (mentor)
Tested by: Oleg Ginzburg <olevole@olevole.ru> (cbsd)
MFC after: 1 week
Relnotes: Y
Differential Revision: https://reviews.freebsd.org/D9930

show more ...

Add a way to temporarily suspend and resume virtual CPUs.

This is used as part of implementing run control in bhyve's debug
server. The hypervisor now maintains a set of "debugged" CPUs.
Attempting

Add a way to temporarily suspend and resume virtual CPUs.

This is used as part of implementing run control in bhyve's debug
server. The hypervisor now maintains a set of "debugged" CPUs.
Attempting to run a debugged CPU will fail to execute any guest
instructions and will instead report a VM_EXITCODE_DEBUG exit to
the userland hypervisor. Virtual CPUs are placed into the debugged
state via vm_suspend_cpu() (implemented via a new VM_SUSPEND_CPU ioctl).
Virtual CPUs can be resumed via vm_resume_cpu() (VM_RESUME_CPU ioctl).

The debug server suspends virtual CPUs when it wishes them to stop
executing in the guest (for example, when a debugger attaches to the
server). The debug server can choose to resume only a subset of CPUs
(for example, when single stepping) or it can choose to resume all
CPUs. The debug server must explicitly mark a CPU as resumed via
vm_resume_cpu() before the virtual CPU will successfully execute any
guest instructions.

Reviewed by: avg, grehan
Tested on: Intel (jhb), AMD (avg)
Differential Revision: https://reviews.freebsd.org/D14466

show more ...

Add a new variant of the GLA2GPA ioctl for use by the debug server.

Unlike the existing GLA2GPA ioctl, GLA2GPA_NOFAULT does not modify
the guest. In particular, it does not inject any faults or mod

Add a new variant of the GLA2GPA ioctl for use by the debug server.

Unlike the existing GLA2GPA ioctl, GLA2GPA_NOFAULT does not modify
the guest. In particular, it does not inject any faults or modify
PTEs in the guest when performing an address space translation.

This is used by bhyve's debug server to read and write memory for
the remote debugger.

Reviewed by: grehan
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D14075

show more ...

Add two new ioctls to bhyve for batch register fetch/store operations.

These are a convenience for bhyve's debug server to use a single
ioctl for 'g' and 'G' rather than a loop of individual get/set

Add two new ioctls to bhyve for batch register fetch/store operations.

These are a convenience for bhyve's debug server to use a single
ioctl for 'g' and 'G' rather than a loop of individual get/set
ioctl requests.

Reviewed by: grehan
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D14074

show more ...

lib: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
pr

lib: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

show more ...

Merge ^/head r313644 through r313895.