vmm: Fix error handling in vmm_handler()

In commit a97f683fe3c4 I didn't add code to remove the vmmctl device
when vmm.ko is unloaded, so it would persist and prevent vmm.ko from
being re-loaded.

E

vmm: Fix error handling in vmm_handler()

In commit a97f683fe3c4 I didn't add code to remove the vmmctl device
when vmm.ko is unloaded, so it would persist and prevent vmm.ko from
being re-loaded.

Extend vmmdev_cleanup() to destroy the vmmctl cdev. Also call
vmmdev_cleanup() if vmm_init() fails.

Reviewed by: corvink, andrew
Fixes: a97f683fe3c4 ("vmm: Add a device file interface for creating and destroying VMs")
Differential Revision: https://reviews.freebsd.org/D48269

show more ...

amd64/vmm: Rename vm_get_vmspace() to vm_vmspace()

For consistency with other vm accessors.

No functional change intended.

Reviewed by: corvink
Differential Revision: https://reviews.freebsd.org/D

amd64/vmm: Rename vm_get_vmspace() to vm_vmspace()

For consistency with other vm accessors.

No functional change intended.

Reviewed by: corvink
Differential Revision: https://reviews.freebsd.org/D48268

show more ...

amd64/vmm: Remove vmm_mem_init()

It is a no-op and doesn't exist on other platforms. As part of some
work to deduplicate vmm code, just remove it.

No functional change intended.

Reviewed by: corv

amd64/vmm: Remove vmm_mem_init()

It is a no-op and doesn't exist on other platforms. As part of some
work to deduplicate vmm code, just remove it.

No functional change intended.

Reviewed by: corvink
Differential Revision: https://reviews.freebsd.org/D48267

show more ...

vmm: Correctly suspend and resume the vmm driver.

Previously, VMXON would be executed on a resume, contrary to proper
initalization. The contents of MSR_IA32_FEATURE_CONTROL may be lost on
suspensio

vmm: Correctly suspend and resume the vmm driver.

Previously, VMXON would be executed on a resume, contrary to proper
initalization. The contents of MSR_IA32_FEATURE_CONTROL may be lost on
suspension, therefore must be restored. Likewise, the VMX Enable bit may be
cleared upon suspend, requiring it to be re-set.

Concretely disable VMX on suspend, and re-enable it on resume.

Note: any IOMMU context will remain lost for any enabled vmm devices.

Signed-off-by: Joshua Rogers <Joshua@Joshua.Hu>
Reviewed by: jhb,imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1419

show more ...

vmm: Postpone vmm module initialization to after SI_SUB_DEVFS

vmmops_modinit() needs to create a device file, and this must happen
after SI_SUB_DEVFS. On non-EARLY_AP_STARTUP platforms (i.e., !x86)

vmm: Postpone vmm module initialization to after SI_SUB_DEVFS

vmmops_modinit() needs to create a device file, and this must happen
after SI_SUB_DEVFS. On non-EARLY_AP_STARTUP platforms (i.e., !x86) this
happens already by accident, but we shouldn't rely on it.

On riscv, remove the current SI_SUB_SMP ordering since that was copied
from arm64 and isn't needed. In particular, riscv's vmmops_modinit()
does not call smp_rendezvous().

Reported by: Oleksandr Kryvulia <shuriku@shurik.kiev.ua>
Fixes: a97f683fe3c4 ("vmm: Add a device file interface for creating and destroying VMs")

show more ...

vmm: Add a device file interface for creating and destroying VMs

This supersedes the sysctl interface, which has the limitations of being
root-only and not supporting automatic resource destruction,

vmm: Add a device file interface for creating and destroying VMs

This supersedes the sysctl interface, which has the limitations of being
root-only and not supporting automatic resource destruction, i.e., we
cannot easily destroy VMs automatically when bhyve terminates.

For now, two ioctls are implemented VMMCTL_VM_CREATE and
VMMCTL_VM_DESTROY. Eventually I would like to support tying a VM's
lifetime to that of the descriptor, so that it is automatically
destroyed when the descriptor is closed. However, this will require
some work in bhyve: when the guest wants to reboot, bhyve exits with a
status that indicates that it is to be restarted. This is incompatible
with the idea of tying a VM's lifetime to that of a descriptor, since we
want to avoid creating and destroying a VM across each reboot (as this
involves freeing all of the guest memory, among other things). One
possible design would be to decompose bhyve into two processes, a parent
which handles reboots, and a child which runs in capability mode and
handles guest execution.

In any case, this gets us closer to addressing the shortcomings
mentioned above.

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D47028

show more ...

vmm: fix vcpu atomic load

Load vcpu with acquire semantics as we are making a critical code
section between creating vcpu and using it.

Tested on risc-v only.

Pointed out by: markj
Reviewed by: jh

vmm: fix vcpu atomic load

Load vcpu with acquire semantics as we are making a critical code
section between creating vcpu and using it.

Tested on risc-v only.

Pointed out by: markj
Reviewed by: jhb, markj
Differential Revision: https://reviews.freebsd.org/D47306

show more ...

vmm: avoid potential KASSERT kernel panic in vm_handle_db

If the guest VM emits the exit code VM_EXITCODE_DB the kernel will
execute the function named vm_handle_db.

If the value of rsp is not page

vmm: avoid potential KASSERT kernel panic in vm_handle_db

If the guest VM emits the exit code VM_EXITCODE_DB the kernel will
execute the function named vm_handle_db.

If the value of rsp is not page aligned and if rsp+sizeof(uint64_t)
spans across two pages, the function vm_copy_setup will need two structs
vm_copyinfo to prepare the copy operation.

For instance is rsp value is 0xFFC, two vm_copyinfo objects are needed:

* address=0xFFC, len=4
* address=0x1000, len=4

The vulnerability was addressed by commit 51fda658baa ("vmm: Properly
handle writes spanning across two pages in vm_handle_db"). Still,
replace the KASSERT with an error return as a more defensive approach.

Reported by: Synacktiv
Reviewed by markj, emaste
Security: HYP-09
Sponsored by: The Alpha-Omega Project
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D46133

show more ...

vmm: Properly handle writes spanning across two pages in vm_handle_db

The vm_handle_db function is responsible for writing correct status
register values into memory when a guest VM is being single-

vmm: Properly handle writes spanning across two pages in vm_handle_db

The vm_handle_db function is responsible for writing correct status
register values into memory when a guest VM is being single-stepped
using the RFLAGS.TF mechanism. However, it currently does not properly
handle an edge case where the resulting write spans across two pages.
This commit fixes this by making vm_handle_db use two vm_copy_info
structs.

Security: HYP-09
Reviewed by: markj

show more ...

Remove stray whitespaces from sys/amd64/

Signed-off-by: Joshua Rogers <Joshua@Joshua.Hu>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1418

vmm: Merge vmm_dev.c

This file contains the vmm device file implementation. Most of this
code is not machine-dependent and so shouldn't be duplicated this way.
Move most of it into a generic dev/vm

vmm: Merge vmm_dev.c

This file contains the vmm device file implementation. Most of this
code is not machine-dependent and so shouldn't be duplicated this way.
Move most of it into a generic dev/vmm/vmm_dev.c. This will make it
easier to introduce a cdev-based interface for VM creation, which in
turn makes it possible to implement support for running bhyve as an
unprivileged user.

Machine-dependent ioctls continue to be handled in machine-dependent
code. To make the split a bit easier to handle, introduce a pair of
tables which define MI and MD ioctls. Each table entry can set flags
which determine which locks need to be held in order to execute the
handler. vmmdev_ioctl() now looks up the ioctl in one of the tables,
acquires locks and either handles the ioctl directly or calls
vmmdev_machdep_ioctl() to handle it.

No functional change intended. There is a lot of churn in this change
but the underlying logic in the ioctl handlers is the same. For now,
vmm_dev.h is still mostly separate, even though some parts could be
merged in principle. This would involve changing include paths for
userspace, though.

Reviewed by: corvink, jhb
Differential Revision: https://reviews.freebsd.org/D46431

show more ...

vmm: Move duplicated stats code into a generic file

There is a small difference between the arm64 and amd64 implementations:
the latter makes use of a "scope" to exclude AMD-specific stats on Intel

vmm: Move duplicated stats code into a generic file

There is a small difference between the arm64 and amd64 implementations:
the latter makes use of a "scope" to exclude AMD-specific stats on Intel
systems and vice-versa. Replace this with a more generic predicate
callback which can be used for the same purpose.

No functional change intended.

Reviewed by: corvink, jhb
Differential Revision: https://reviews.freebsd.org/D46430

show more ...

vmm: Move vmm_ktr.h to a common directory

No functional change intended.

Reviewed by: corvink, jhb, emaste
Differential Revision: https://reviews.freebsd.org/D46429

vm_iommu_map()/unmap(): stop transiently wiring already wired pages

Namely, switch from vm_fault_quick_hold() to pmap_extract() KPI to
translate gpa to hpa. Assert that the looked up hpa belongs to

vm_iommu_map()/unmap(): stop transiently wiring already wired pages

Namely, switch from vm_fault_quick_hold() to pmap_extract() KPI to
translate gpa to hpa. Assert that the looked up hpa belongs to the wired
page, as it should be for the VM which is configured for pass-throu
(this is theoretically a restriction that could be removed on newer
DMARs).

Noted by: alc
Reviewed by: alc, jhb, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D43140

show more ...

vmm_iommu_modify(): split vm_iommu_map()/unmap() into separate functions

Reviewed by: alc, jhb, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.fr

vmm_iommu_modify(): split vm_iommu_map()/unmap() into separate functions

Reviewed by: alc, jhb, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D43140

show more ...

vmm: implement single-stepping for AMD CPUs

This patch implements single-stepping for AMD CPUs using the RFLAGS.TF
single-stepping mechanism. The GDB stub requests single-stepping
using the VM_CAP_

vmm: implement single-stepping for AMD CPUs

This patch implements single-stepping for AMD CPUs using the RFLAGS.TF
single-stepping mechanism. The GDB stub requests single-stepping
using the VM_CAP_RFLAGS_TF capability. Setting this capability will
set the RFLAGS.TF bit on the selected vCPU, activate DB exception
intercepts, and activate POPF/PUSH instruction intercepts. The
resulting DB exception is then caught by the IDT_DB vmexit handler and
bounced to userland where it is processed by the GDB stub. This patch
also makes sure that the value of the TF bit is correctly updated and
that it is not erroneously propagated into memory. Stepping over PUSHF
will cause the vm_handle_db function to correct the pushed RFLAGS
value and stepping over POPF will update the shadowed TF bit copy.

Reviewed by: jhb
Sponsored by: Google, Inc. (GSoC 2022)
Differential Revision: https://reviews.freebsd.org/D42296

show more ...

x86: Rename {stop,start}_emulating to fpu_{enable,disable}

While here, centralize the macros in <x86/fpu.h>.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D42135

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

vmm: Avoid embedding cpuset_t ioctl ABIs

Commit 0bda8d3e9f7a ("vmm: permit some IPIs to be handled by userspace")
embedded cpuset_t into the vmm(4) ioctl ABI. This was a mistake since
we otherwise

vmm: Avoid embedding cpuset_t ioctl ABIs

Commit 0bda8d3e9f7a ("vmm: permit some IPIs to be handled by userspace")
embedded cpuset_t into the vmm(4) ioctl ABI. This was a mistake since
we otherwise have some leeway to change the cpuset_t for the whole
system, but we want to keep the vmm ioctl ABI stable.

Rework IPI reporting to avoid this problem. Along the way, make VM_RUN
a bit more efficient:
- Split vmexit metadata out of the main VM_RUN structure. This data is
only written by the kernel.
- Have userspace pass a cpuset_t pointer and cpusetsize in the VM_RUN
structure, as is done for cpuset syscalls.
- Have the destination CPU mask for VM_EXITCODE_IPIs live outside the
vmexit info structure, and make VM_RUN copy it out separately. Zero
out any extra bytes in the CPU mask, like cpuset syscalls do.
- Modify the vmexit handler prototype to take a full VM_RUN structure.

PR: 271330
Reviewed by: corvink, jhb (previous versions)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40113

show more ...

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

bhyve: fix restore of kernel structs

vmx_snapshot() and svm_snapshot() do not save any data and error occurs at
resume:

Restoring kernel structs...
vm_restore_kern_struct: Kernel struct size was 0

bhyve: fix restore of kernel structs

vmx_snapshot() and svm_snapshot() do not save any data and error occurs at
resume:

Restoring kernel structs...
vm_restore_kern_struct: Kernel struct size was 0 for: vmx
Failed to restore kernel structs.

Reviewed by: corvink, markj
Fixes: 39ec056e6dbd89e26ee21d2928dbd37335de0ebc ("vmm: Rework snapshotting of CPU-specific per-vCPU data.")
MFC after: 2 weeks
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D38476

show more ...

vmm: fix restore of TSC offset

After suspend/resume Ubuntu 20.04 and 22.04 installer can hang if
tsc-early clocksource has a big skew.

Reviewed by: corvink, jhb
Fixes: a7db532e3a6f83067b342f569b

vmm: fix restore of TSC offset

After suspend/resume Ubuntu 20.04 and 22.04 installer can hang if
tsc-early clocksource has a big skew.

Reviewed by: corvink, jhb
Fixes: a7db532e3a6f83067b342f569b56076d011f8a1e ("vmm: Simplify saving of absolute TSC values in snapshots.")
MFC after: 2 weeks
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D38474

show more ...

vmm: Remove an unneeded initialization of "retu"

vm_handle_ipi() unconditionally initializes "retu". No functional
change intended.

Reviewed by: jhb, corvink
MFC after: 1 week
Sponsored by: The Fr

vmm: Remove an unneeded initialization of "retu"

vm_handle_ipi() unconditionally initializes "retu". No functional
change intended.

Reviewed by: jhb, corvink
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D38446

show more ...

vmm: avoid spurious rendezvous

A vcpu only checks if a rendezvous is in progress or not to decide if it
should handle a rendezvous. This could lead to spurios rendezvous where
a vcpu tries a handle

vmm: avoid spurious rendezvous

A vcpu only checks if a rendezvous is in progress or not to decide if it
should handle a rendezvous. This could lead to spurios rendezvous where
a vcpu tries a handle a rendezvous it isn't part of. This situation is
properly handled by vm_handle_rendezvous but it could potentially
degrade the performance. Avoid that by an early check if the vcpu is
part of the rendezvous or not.

At the moment, rendezvous are only used to spin up application
processors and to send ioapic interrupts. Spinning up application
processors is done in the guest boot phase by sending INIT SIPI
sequences to single vcpus. This is known to cause spurious rendezvous
and only occurs in the boot phase. Sending ioapic interrupts is rare
because modern guest will use msi and the rendezvous is always send to
all vcpus.

Reviewed by: jhb
MFC after: 1 week
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D37390

show more ...