bhyve: Use vm_get_highmem_base() instead of hard-coding the value

This reduces the coupling between libvmmapi (which creates the highmem
segment) and bhyve, in preparation for the arm64 port.

No fu

bhyve: Use vm_get_highmem_base() instead of hard-coding the value

This reduces the coupling between libvmmapi (which creates the highmem
segment) and bhyve, in preparation for the arm64 port.

No functional change intended.

Reviewed by: corvink, jhb
MFC after: 2 weeks
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D40992

show more ...

bhyve: increase SNAPSHOT_BUFFER_SIZE

After commit fb51ddb20d57, suspending a vm fails with the error:
vm_snapshot_buf: buffer too small

Sponsored by: vStack
Reviewed by: markj, rew
Fixes:

bhyve: increase SNAPSHOT_BUFFER_SIZE

After commit fb51ddb20d57, suspending a vm fails with the error:
vm_snapshot_buf: buffer too small

Sponsored by: vStack
Reviewed by: markj, rew
Fixes: fb51ddb20d57 ("bhyve: increase fbuf display resolution limit")
Differential Revision: https://reviews.freebsd.org/D43218

show more ...

usr.sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

usr.sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

bhyve: Remove init_snapshot() and initialize static vars

vCPU threads are starting before init_snapshot() is called. That can lead
to corruption of vcpu_lock userspace mutex (snapshot.c) and then VM

bhyve: Remove init_snapshot() and initialize static vars

vCPU threads are starting before init_snapshot() is called. That can lead
to corruption of vcpu_lock userspace mutex (snapshot.c) and then VM hangs
in acquiring that mutex.

init_snapshot() initializes only static variables (mutex, cv) and that
code can be optimized and removed.

Fixes: 9a9a248964696 ("bhyve: init checkput before caph_enter")
Reviewed by: markj
MFC after: 1 week
Sponsored by: vStack

show more ...

bhyve: Replace many fprintf(stderr, ...) calls with EPRINTLN

EPRINTLN handles newlines appropriately when stdout/stderr have been
reused as the backend for a serial port.

For bhyverun.c itself, the

bhyve: Replace many fprintf(stderr, ...) calls with EPRINTLN

EPRINTLN handles newlines appropriately when stdout/stderr have been
reused as the backend for a serial port.

For bhyverun.c itself, the rule this attempts to follow is to use
regular fprintf/perror/warn/err prior to init_pci() (which is when
serial ports are configured) and to switch to EPRINTLN afterwards.

Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D42182

show more ...

bhyve: Remove some unneeded includes from snapshot.c

MFC after: 1 week
Sponsored by: Innovate UK

bhyve: Move more amd64-specific code under amd64/

mptable and the e820 are both rather amd64-specific and can be moved
easily.

In the case of e820, move the registration with qemu_fwcfg into e820.c

bhyve: Move more amd64-specific code under amd64/

mptable and the e820 are both rather amd64-specific and can be moved
easily.

In the case of e820, move the registration with qemu_fwcfg into e820.c,
as it simplifies bhyverun.c a bit and I can't see any downsides.

No functional change intended.

Reviewed by: corvink, jhb, emaste
MFC after: 1 week
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D40552

show more ...

bhyve: Move some more amd64-specific drivers to their own subdir

No functional change intended.

Reviewed by: corvink, jhb, emaste
MFC after: 1 week
Sponsored by: Innovate UK
Differential Revision:

bhyve: Move some more amd64-specific drivers to their own subdir

No functional change intended.

Reviewed by: corvink, jhb, emaste
MFC after: 1 week
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D40551

show more ...

bhyve: Start moving machine-dependent code into subdirectories

In preparation for an arm64 port, make an easy change which puts some
machine-dependent code in its own directory.

Going forward, code

bhyve: Start moving machine-dependent code into subdirectories

In preparation for an arm64 port, make an easy change which puts some
machine-dependent code in its own directory.

Going forward, code which is only used on one platform should live in a
MD directory. We should strive to layer modules in such a way as to
avoid polluting shared code with lots of ifdefs. For some existing
files this will take some effort.

task_switch.c and fwctl.c are an easy place to start: the former is very
x86-specific, and the latter provides an I/O port interface which can't
be used on anything other than x86. (fwcfg as implemented has the same
problem, but QEMU also supports a MMIO fwcfg interface.) So I propose
that we start by simply making those files conditional.

Reviewed by: corvink, jhb
MFC after: 1 week
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D40501

show more ...

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Revert "bhyve: add command line parameter and parsing for migration"

Unfortunately, this feature didn't receive much feedback in the past.
However, after committing this, some people came up and com

Revert "bhyve: add command line parameter and parsing for migration"

Unfortunately, this feature didn't receive much feedback in the past.
However, after committing this, some people came up and complain that
this feature requires some more discussion before upstreaming it.
Additionally, it wasn't a good idea to start this new feature by adding
a new command line parameter as it fixes the user interface.

This reverts commit c9fdd4f3cc18c03683de85318ba8d318f96b58c4.

show more ...

bhyve: Remove some unneeded includes of segments.h

They are not needed and are specific to x86. No functional change
intended.

MFC after: 1 week
Sponsored by: Innovate UK

bhyve: add command line parameter and parsing for migration

This covers warm and live migration.

Reviewed by: corvink
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D34717

bhyve: use pci_next() to save/restore pci devices

Current snapshot implementation doesn't support multiple devices with
similar type. For example, two virtio-blk or two CD-ROM-s, etc.

So the follow

bhyve: use pci_next() to save/restore pci devices

Current snapshot implementation doesn't support multiple devices with
similar type. For example, two virtio-blk or two CD-ROM-s, etc.

So the following configuration cannot be restored.

bhyve \
-s 3,virtio-blk,disk.img \
-s 4,virtio-blk,disk2.img

In some cases it is restored silently, but doesn't work. In some cases
it fails during restore stage.

This commit fixes that issue.

Reviewed by: corvink, rew
MFC after: 1 week
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D40109

show more ...

bhyve: rename 'user_dev' with 'devices'

Bhyve don't use 'user' specifier for emulated devices. And
using 'user' adds duality.

Reviewed by: corvink, rew
MFC after: 1 week
Sponsored by: vStack
Dif

bhyve: rename 'user_dev' with 'devices'

Bhyve don't use 'user' specifier for emulated devices. And
using 'user' adds duality.

Reviewed by: corvink, rew
MFC after: 1 week
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D40106

show more ...

bhyve: simplify restore of kernel structs

Both devices and kernel struct can use the same 'lookup_dev'
function instead of having duplicated code.

Reviewed by: corvink, rew
MFC after: 1 week
Spon

bhyve: simplify restore of kernel structs

Both devices and kernel struct can use the same 'lookup_dev'
function instead of having duplicated code.

Reviewed by: corvink, rew
MFC after: 1 week
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D40105

show more ...

bhyve: rename 'structs' snapshot key with 'kern_structs'

Using key 'structs' is ambiguous. This section contains data
related to the kernel. It should use a more informative naming.

Reviewed by: c

bhyve: rename 'structs' snapshot key with 'kern_structs'

Using key 'structs' is ambiguous. This section contains data
related to the kernel. It should use a more informative naming.

Reviewed by: corvink, rew
MFC after: 1 week
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D40104

show more ...

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

bhyve: use directory file descriptor for checkpoint

This is required to enable capsicum for the snapshot code.

Reviewed by: corvink
Sponsored by: vStack
Differential Revision: https://reviews.fre

bhyve: use directory file descriptor for checkpoint

This is required to enable capsicum for the snapshot code.

Reviewed by: corvink
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D38858

show more ...

bhyve: Remove vmctx member from struct vm_snapshot_meta.

This is a userland-only pointer that isn't relevant to the kernel and
doesn't belong in the ioctl structure shared between userland and the
k

bhyve: Remove vmctx member from struct vm_snapshot_meta.

This is a userland-only pointer that isn't relevant to the kernel and
doesn't belong in the ioctl structure shared between userland and the
kernel. For the kernel, the old structure for the ioctl is still
supported under COMPAT_FREEBSD13.

This changes vm_snapshot_req() in libvmmapi to accept an explicit
vmctx argument.

It also changes vm_snapshot_guest2host_addr to take an explicit vmctx
argument. As part of this change, move the declaration for this
function and its wrapper macro from vmm_snapshot.h to snapshot.h as it
is a userland-only API.

Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D38125

show more ...

libvmmapi: Add a struct vcpu and use it in most APIs.

This replaces the 'struct vm, int vcpuid' tuple passed to most API
calls and is similar to the changes recently made in vmm(4) in the
kernel.

s

libvmmapi: Add a struct vcpu and use it in most APIs.

This replaces the 'struct vm, int vcpuid' tuple passed to most API
calls and is similar to the changes recently made in vmm(4) in the
kernel.

struct vcpu is an opaque type managed by libvmmapi. For now it stores
a pointer to the VM context and an integer id.

As an immediate effect this removes the divergence between the kernel
and userland for the instruction emulation code introduced by the
recent vmm(4) changes.

Since this is a major change to the vmmapi API, bump VMMAPI_VERSION to
0x200 (2.0) and the shared library major version.

While here (and since the major version is bumped), remove unused
vcpu argument from vm_setup_pptdev_msi*().

Add new functions vm_suspend_all_cpus() and vm_resume_all_cpus() for
use by the debug server. The underyling ioctl (which uses a vcpuid of
-1) remains unchanged, but the userlevel API now uses separate
functions for global CPU suspend/resume.

Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D38124

show more ...

bhyve: add cap limits for ipc socket

Reviewed by: corvink, markj
MFC after: 1 week
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D38856

bhyve: fix restore of kernel structs

vmx_snapshot() and svm_snapshot() do not save any data and error occurs at
resume:

Restoring kernel structs...
vm_restore_kern_struct: Kernel struct size was 0

bhyve: fix restore of kernel structs

vmx_snapshot() and svm_snapshot() do not save any data and error occurs at
resume:

Restoring kernel structs...
vm_restore_kern_struct: Kernel struct size was 0 for: vmx
Failed to restore kernel structs.

Reviewed by: corvink, markj
Fixes: 39ec056e6dbd89e26ee21d2928dbd37335de0ebc ("vmm: Rework snapshotting of CPU-specific per-vCPU data.")
MFC after: 2 weeks
Sponsored by: vStack
Differential Revision: https://reviews.freebsd.org/D38476

show more ...

bhyve: Remove vmctx argument from PCI device model methods.

Most of these arguments were unused. Device models which do need
access to the vmctx in one of these methods can obtain it from the
pi_vm

bhyve: Remove vmctx argument from PCI device model methods.

Most of these arguments were unused. Device models which do need
access to the vmctx in one of these methods can obtain it from the
pi_vmctx member of the pci_devinst argument instead.

Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D38096

show more ...

bhyve: Drop volatile qualifiers from snapshot code

They accomplish nothing since the qualifier is casted away in calls to
memcpy() and copyin()/copyout(). No functional change intended.

MFC after:

bhyve: Drop volatile qualifiers from snapshot code

They accomplish nothing since the qualifier is casted away in calls to
memcpy() and copyin()/copyout(). No functional change intended.

MFC after: 2 weeks
Reviewed by: corvink, jhb
Differential Revision: https://reviews.freebsd.org/D37292

show more ...