bhyve: Remove the unused vcpu argument from all of the I/O port handlers.

Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37653

bhyve: Remove unused vcpu argument from PCI read/write methods.

Reviewed by: corvink, markj
Differential Revision: https://reviews.freebsd.org/D37652

bhyve: Address some signed/unsigned comparison warnings

MFC after: 1 week

bhyve: Fix some warnings in the snapshot code

- Qualify unexported symbols with "static".
- Drop some unnecessary and incorrect casts.
- Avoid arithmetic on void pointers.
- Avoid signed/unsigned co

bhyve: Fix some warnings in the snapshot code

- Qualify unexported symbols with "static".
- Drop some unnecessary and incorrect casts.
- Avoid arithmetic on void pointers.
- Avoid signed/unsigned comparisons in loops which use nitems() as a
bound.

No functional change intended.

MFC after: 1 week

show more ...

bhyve: Make pci_bars local to pci_emul.c

MFC after: 1 week

bhyve: Annotate unused function parameters

MFC after: 1 week

bhyve: Mark variables and functions as static where appropriate

Mark them const as well when it makes sense to do so. No functional
change intended.

MFC after: 1 week
Sponsored by: The FreeBSD Fou

bhyve: Mark variables and functions as static where appropriate

Mark them const as well when it makes sense to do so. No functional
change intended.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

show more ...

bhyve: Use "void" instead of empty parameter lists

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

bhyve: avoid uninitialized variable

Reviewed by: markj
Signed-off-by: Corvin Köhne <c.koehne@beckhoff.com>
Reported-by: Andy Fiddaman <andy@omniosce.org>
Differential Revision: https://reviews.freeb

bhyve: avoid uninitialized variable

Reviewed by: markj
Signed-off-by: Corvin Köhne <c.koehne@beckhoff.com>
Reported-by: Andy Fiddaman <andy@omniosce.org>
Differential Revision: https://reviews.freebsd.org/D34688

show more ...

bhyve: avoid overflow of BAR index

At the moment, writes to BAR registers that aren't 4 byte aligned are
ignored. So, there's no overflow yet. Nevertheless, if this behaviour
changes in the future,

bhyve: avoid overflow of BAR index

At the moment, writes to BAR registers that aren't 4 byte aligned are
ignored. So, there's no overflow yet. Nevertheless, if this behaviour
changes in the future, it could unintentionally, introduce a buffer
overflow. Additionally, some compiler or tools will detect this
potential overflow and complain about it.

Reviewed by: markj
Signed-off-by: Corvin Köhne <c.koehne@beckhoff.com>
Reported-by: Andy Fiddaman <andy@omniosce.org>
Differential Revision: https://reviews.freebsd.org/D34689

show more ...

bhyve: add ROM emulation

Some PCI devices especially GPUs require a ROM to work properly.
The ROM is executed by boot firmware to initialize the device.
To add a ROM to a device use the new ROM opti

bhyve: add ROM emulation

Some PCI devices especially GPUs require a ROM to work properly.
The ROM is executed by boot firmware to initialize the device.
To add a ROM to a device use the new ROM option for passthru device
(e.g. -s passthru,0/2/0,rom=<path>/<to>/<rom>).

It's necessary that the ROM is executed by the boot firmware.
It won't be executed by any OS.
Additionally, the boot firmware should be configured to execute the
ROM file.
For that reason, it's only possible to use a ROM when using
OVMF with enabled bus enumeration.

Differential Revision: https://reviews.freebsd.org/D33129
Sponsored by: Beckhoff Automation GmbH & Co. KG
MFC after: 1 month

show more ...

bhyve: add more slop to 64 bit BARs

Bhyve allocates small 64 bit BARs below 4 GB and generates ACPI tables
based on this allocation. If the guest decides to relocate those BARs
above 4 GB, it could

bhyve: add more slop to 64 bit BARs

Bhyve allocates small 64 bit BARs below 4 GB and generates ACPI tables
based on this allocation. If the guest decides to relocate those BARs
above 4 GB, it could lead to mismatching ACPI tables. Especially
when using OVMF with enabled bus enumeration it could cause
issues. OVMF relocates all 64 bit BARs above 4 GB. The guest OS
may be unable to recover from this situation and disables some PCI
devices because their BARs are located outside of the MMIO space
reported by ACPI. Avoid this situation by giving the guest more
space for relocating BARs.

Let's be paranoid. The available space for BARs below 4 GB is 512 MB
large. Use a slop of 512 MB. It'll allow the guest to relocate all
BARs below 4 GB to an address above 4 GB. We could run into issues
when we exceeding the memlimit above 4 GB. However, this space has
a size of 32 GB. Even when using many PCI device with large BARs
like framebuffer or when using multiple PCI busses, it's very
unlikely that we run out of space due to the large slop.
Additionally, this situation will occur on startup and not at runtime
which is much better.

Reviewed by: markj
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33118

show more ...

bhyve: enumerate BARs by size

E.g. Framebuffers can require large space and BARs need to be aligned
by their size. If BARs aren't allocated by size, it'll cause much
fragmentation of the MMIO space.

bhyve: enumerate BARs by size

E.g. Framebuffers can require large space and BARs need to be aligned
by their size. If BARs aren't allocated by size, it'll cause much
fragmentation of the MMIO space. Reduce fragmentation by ordering
the BAR allocation on their size to reduce the risk of
OUT_OF_MMIO_SPACE issues.

Reviewed by: markj
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D28278

show more ...

bhyve: clean up trailing whitespaces

Clean up trailing whitespaces. No functional changes.

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D33681

bhyve: Fix compile

We need err.h

Fixes: 5cf21e48ccf11 ("bhyve: use a fixed 32 bit BAR base address")
Sponsored by: Bechoff Automation GmbH & Co. KG

bhyve: use a fixed 32 bit BAR base address

OVMF always uses 0xC0000000 as base address for 32 bit PCI MMIO space.
For that reason, we should use that address too.

Reviewed by: markj
Differential Re

bhyve: use a fixed 32 bit BAR base address

OVMF always uses 0xC0000000 as base address for 32 bit PCI MMIO space.
For that reason, we should use that address too.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D31051
Sponsored by: Beckhoff Automation GmbH & Co. KG

show more ...

bhyve: move 64 bit BAR location to match OVMF assumptions

OVMF will fail, if large 64 bit BARs are used. GCD-Map doesn't cover
64 bit addresses of BARs.
OVMF assumes that 64 bit addresses of BARS ar

bhyve: move 64 bit BAR location to match OVMF assumptions

OVMF will fail, if large 64 bit BARs are used. GCD-Map doesn't cover
64 bit addresses of BARs.
OVMF assumes that 64 bit addresses of BARS are located on next 32 GB
boundary behind Top of High RAM.

This patch moves 64 bit BARs on next 32 GB boundary behind Top of High
RAM to match OVMF assumptions.

Differential Revision: https://reviews.freebsd.org/D27970
Sponsored by: Beckhoff Automation GmbH & Co. KG

show more ...

bhyve: use physical lobits for BARs of passthru devices

Tell the guest whether a BAR uses prefetched memory or not for
passthru devices by using the same lobits as the physical device.

Reviewed by:

bhyve: use physical lobits for BARs of passthru devices

Tell the guest whether a BAR uses prefetched memory or not for
passthru devices by using the same lobits as the physical device.

Reviewed by: grehan
Sponsored by: Beckhoff Autmation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D32685

show more ...

bhyve: Fix the WITH_BHYVE_SNAPSHOT build

Note, this breaks compatibility with snapshots generated by older builds
of bhyve(8).

Fixes: 7fa233534736 ("bhyve: Map the MSI-X table unconditionally for p

bhyve: Fix the WITH_BHYVE_SNAPSHOT build

Note, this breaks compatibility with snapshots generated by older builds
of bhyve(8).

Fixes: 7fa233534736 ("bhyve: Map the MSI-X table unconditionally for passthrough")
Reported by: Greg V <greg@unrelenting.technology>
Reviewed by: grehan, bz
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32523

show more ...

bhyve: ignore low bits of CFGADR

Bhyve could emulate wrong PCI registers.
In the best case, the guest reads wrong registers and the device driver would
report some errors.
In the worst case, the gue

bhyve: ignore low bits of CFGADR

Bhyve could emulate wrong PCI registers.
In the best case, the guest reads wrong registers and the device driver would
report some errors.
In the worst case, the guest writes to wrong PCI registers and could brick
hardware when using PCI passthrough.

According to Intels specification, low bits of CFGADR should be
ignored. Some OS like linux may rely on it. Otherwise, bhyve could
emulate a wrong PCI register.

E.g.
If linux would like to read 2 bytes from offset 0x02, following would
happen.
linux:
outl 0x80000002 at CFGADR
inw at CFGDAT + 2
bhyve:
cfgoff = 0x80000002 & 0xFF = 0x02
coff = cfgoff + (port - CFGDAT) = 0x02 + 0x02 = 0x04
Bhyve would emulate the register at offset 0x04 not 0x02.

Reviewed By: #bhyve, grehan
Differential Revision: https://reviews.freebsd.org/D31819
Sponsored by: Beckhoff Automation GmbH & Co. KG

show more ...

bhyve: support relocating fbuf and passthru data BARs

We want to allow the UEFI firmware to enumerate and assign
addresses to PCI devices so we can boot from NVMe[1]. Address
assignment of PCI BARs

bhyve: support relocating fbuf and passthru data BARs

We want to allow the UEFI firmware to enumerate and assign
addresses to PCI devices so we can boot from NVMe[1]. Address
assignment of PCI BARs is properly handled by the PCI emulation
code in general, but a few specific cases need additional support.
fbuf and passthru map additional objects into the guest physical
address space and so need to handle address updates. Here we add a
callback to emulated PCI devices to inform them of a BAR
configuration change. fbuf and passthru then watch for these BAR
changes and relocate the frame buffer memory segment and passthru
device mmio area respectively.

We also add new VM_MUNMAP_MEMSEG and VM_UNMAP_PPTDEV_MMIO ioctls
to vmm(4) to facilitate the unmapping needed for addres updates.

[1]: https://github.com/freebsd/uefi-edk2/pull/9/

Originally by: scottph
MFC After: 1 week
Sponsored by: Intel Corporation
Reviewed by: grehan
Approved by: philip (mentor)
Differential Revision: https://reviews.freebsd.org/D24066

show more ...

Refactor configuration management in bhyve.

Replace the existing ad-hoc configuration via various global variables
with a small database of key-value pairs. The database supports
heirarchical keys

Refactor configuration management in bhyve.

Replace the existing ad-hoc configuration via various global variables
with a small database of key-value pairs. The database supports
heirarchical keys using a MIB-like syntax to name the path to a given
key. Values are always stored as strings. The API used to manage
configuation values does include wrappers to handling boolean values.
Other values use non-string types require parsing by consumers.

The configuration values are stored in a tree using nvlists. Leaf
nodes hold string values. Configuration values are permitted to
reference other configuration values using '%(name)'. This permits
constructing template configurations.

All existing command line arguments now set configuration values. For
devices, the "-s" option parses its option argument to generate a list
of key-value pairs for the given device.

A new '-o' command line option permits setting an individual
configuration variable. The key name is always given as a full path
of dot-separated components.

A new '-k' command line option parses a simple configuration file.
This configuration file holds a flat list of 'key=value' lines where
the 'key' is the full path of a configuration variable. Lines
starting with a '#' are comments.

In general, bhyve starts by parsing command line options in sequence
and applying those settings to configuration values. Once this is
complete, bhyve then begins initializing its state based on the
configuration values. This means that subsequent configuration
options or files may override or supplement previously given settings.

A special 'config.dump' configuration value can be set to true to help
debug configuration issues. When this value is set, bhyve will print
out the configuration variables as a flat list of 'key=value' lines.

Most command line argments map to a single configuration variable,
e.g. '-w' sets the 'x86.strictmsr' value to false. A few command
line arguments have less obvious effects:

- Multiple '-p' options append their values (as a comma-seperated
list) to "vcpu.N.cpuset" values (where N is a decimal vcpu number).

- For '-s' options, a pci.<bus>.<slot>.<function> node is created.
The first argument to '-s' (the device type) is used as the value of
a "device" variable. Additional comma-separated arguments are then
parsed into 'key=value' pairs and used to set additional variables
under the device node. A PCI device emulation driver can provide
its own hook to override the parsing of the additonal '-s' arguments
after the device type.

After the configuration phase as completed, the init_pci hook
then walks the "pci.<bus>.<slot>.<func>" nodes. It uses the
"device" value to find the device model to use. The device
model's init routine is passed a reference to its nvlist node
in the configuration tree which it can query for specific
variables.

The result is that a lot of the string parsing is removed from
the device models and centralized. In addition, adding a new
variable just requires teaching the model to look for the new
variable.

- For '-l' options, a similar model is used where the string is
parsed into values that are later read during initialization.
One key note here is that the serial ports use the commonly
used lowercase names from existing documentation and examples
(e.g. "lpc.com1") instead of the uppercase names previously
used internally in bhyve.

Reviewed by: grehan
MFC after: 3 months
Differential Revision: https://reviews.freebsd.org/D26035

show more ...

bhyve: remove a hack to map all 8G BARs 1:1

Suggested and reviewed by: grehan
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D27186

bhyve: increase allowed size for 64bit BAR allocation below 4G from 32 to 128 MB.

Reviewed by: grehan
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.f

bhyve: increase allowed size for 64bit BAR allocation below 4G from 32 to 128 MB.

Reviewed by: grehan
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D27095

show more ...

bhyve: avoid allocating BARs above the end of supported physical addresses.

Read CPUID leaf 0x8000008 to determine max supported phys address and
create BAR region right below it, reserving 1/4 of t

bhyve: avoid allocating BARs above the end of supported physical addresses.

Read CPUID leaf 0x8000008 to determine max supported phys address and
create BAR region right below it, reserving 1/4 of the supported guest
physical address space to the 64bit BARs mappings.

PR: 250802 (although the issue from PR is not fixed by the change)
Noted and reviewed by: grehan
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D27095

show more ...