vmm: Postpone vmm module initialization to after SI_SUB_DEVFS

vmmops_modinit() needs to create a device file, and this must happen
after SI_SUB_DEVFS. On non-EARLY_AP_STARTUP platforms (i.e., !x86)

vmm: Postpone vmm module initialization to after SI_SUB_DEVFS

vmmops_modinit() needs to create a device file, and this must happen
after SI_SUB_DEVFS. On non-EARLY_AP_STARTUP platforms (i.e., !x86) this
happens already by accident, but we shouldn't rely on it.

On riscv, remove the current SI_SUB_SMP ordering since that was copied
from arm64 and isn't needed. In particular, riscv's vmmops_modinit()
does not call smp_rendezvous().

Reported by: Oleksandr Kryvulia <shuriku@shurik.kiev.ua>
Fixes: a97f683fe3c4 ("vmm: Add a device file interface for creating and destroying VMs")

show more ...

vmm: Add a device file interface for creating and destroying VMs

This supersedes the sysctl interface, which has the limitations of being
root-only and not supporting automatic resource destruction,

vmm: Add a device file interface for creating and destroying VMs

This supersedes the sysctl interface, which has the limitations of being
root-only and not supporting automatic resource destruction, i.e., we
cannot easily destroy VMs automatically when bhyve terminates.

For now, two ioctls are implemented VMMCTL_VM_CREATE and
VMMCTL_VM_DESTROY. Eventually I would like to support tying a VM's
lifetime to that of the descriptor, so that it is automatically
destroyed when the descriptor is closed. However, this will require
some work in bhyve: when the guest wants to reboot, bhyve exits with a
status that indicates that it is to be restarted. This is incompatible
with the idea of tying a VM's lifetime to that of a descriptor, since we
want to avoid creating and destroying a VM across each reboot (as this
involves freeing all of the guest memory, among other things). One
possible design would be to decompose bhyve into two processes, a parent
which handles reboots, and a child which runs in capability mode and
handles guest execution.

In any case, this gets us closer to addressing the shortcomings
mentioned above.

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D47028

show more ...

vmm: fix vcpu atomic load

Load vcpu with acquire semantics as we are making a critical code
section between creating vcpu and using it.

Tested on risc-v only.

Pointed out by: markj
Reviewed by: jh

vmm: fix vcpu atomic load

Load vcpu with acquire semantics as we are making a critical code
section between creating vcpu and using it.

Tested on risc-v only.

Pointed out by: markj
Reviewed by: jhb, markj
Differential Revision: https://reviews.freebsd.org/D47306

show more ...

vmm: Merge vmm_dev.c

This file contains the vmm device file implementation. Most of this
code is not machine-dependent and so shouldn't be duplicated this way.
Move most of it into a generic dev/vm

vmm: Merge vmm_dev.c

This file contains the vmm device file implementation. Most of this
code is not machine-dependent and so shouldn't be duplicated this way.
Move most of it into a generic dev/vmm/vmm_dev.c. This will make it
easier to introduce a cdev-based interface for VM creation, which in
turn makes it possible to implement support for running bhyve as an
unprivileged user.

Machine-dependent ioctls continue to be handled in machine-dependent
code. To make the split a bit easier to handle, introduce a pair of
tables which define MI and MD ioctls. Each table entry can set flags
which determine which locks need to be held in order to execute the
handler. vmmdev_ioctl() now looks up the ioctl in one of the tables,
acquires locks and either handles the ioctl directly or calls
vmmdev_machdep_ioctl() to handle it.

No functional change intended. There is a lot of churn in this change
but the underlying logic in the ioctl handlers is the same. For now,
vmm_dev.h is still mostly separate, even though some parts could be
merged in principle. This would involve changing include paths for
userspace, though.

Reviewed by: corvink, jhb
Differential Revision: https://reviews.freebsd.org/D46431

show more ...

vmm: Move duplicated stats code into a generic file

There is a small difference between the arm64 and amd64 implementations:
the latter makes use of a "scope" to exclude AMD-specific stats on Intel

vmm: Move duplicated stats code into a generic file

There is a small difference between the arm64 and amd64 implementations:
the latter makes use of a "scope" to exclude AMD-specific stats on Intel
systems and vice-versa. Replace this with a more generic predicate
callback which can be used for the same purpose.

No functional change intended.

Reviewed by: corvink, jhb
Differential Revision: https://reviews.freebsd.org/D46430

show more ...

vmm: Move vmm_ktr.h to a common directory

No functional change intended.

Reviewed by: corvink, jhb, emaste
Differential Revision: https://reviews.freebsd.org/D46429

arm64/vmm: Implement vm_disable_vcpu_creation()

No functional change intended.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D45556

arm64/vmm: Handle VM_EXITCODE_SUSPENDED

This is required for bhyve reboot to work. In particular, unless we
suspend vcpu threads here, vm_reinit() will fail with EBUSY.

The implementation is copie

arm64/vmm: Handle VM_EXITCODE_SUSPENDED

This is required for bhyve reboot to work. In particular, unless we
suspend vcpu threads here, vm_reinit() will fail with EBUSY.

The implementation is copied from amd64; in the not-too-distant future
the amd64 and arm64 copies of vmm.c and vmm_dev.c will be merged, so
for now it's useful to minimize diffs between amd64 and arm64.

Reviewed by: corvink, andrew
MFC after: 2 weeks
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D44934

show more ...

Import the kernel parts of bhyve/arm64

To support virtual machines on arm64 add the vmm code. This is based on
earlier work by Mihai Carabas and Alexandru Elisei at University
Politehnica of Buchare

Import the kernel parts of bhyve/arm64

To support virtual machines on arm64 add the vmm code. This is based on
earlier work by Mihai Carabas and Alexandru Elisei at University
Politehnica of Bucharest, with further work by myself and Mark Johnston.

All AArch64 CPUs should work, however only the GICv3 interrupt
controller is supported. There is initial support to allow the GICv2
to be supported in the future. Only pure Armv8.0 virtualisation is
supported, the Virtualization Host Extensions are not currently used.

With a separate userspace patch and U-Boot port FreeBSD guests are able
to boot to multiuser mode, and the hypervisor can be tested with the
kvm unit tests. Linux partially boots, but hangs before entering
userspace. Other operating systems are untested.

Sponsored by: Arm Ltd
Sponsored by: Innovate UK
Sponsored by: The FreeBSD Foundation
Sponsored by: University Politehnica of Bucharest
Differential Revision: https://reviews.freebsd.org/D37428

show more ...