tests/ci: Add CIENV variable

The default behavior of of pre-commit script is to suppress all the logs
as it is assumed that developers will run this on their terminal for the
tests. But there are al

tests/ci: Add CIENV variable

The default behavior of of pre-commit script is to suppress all the logs
as it is assumed that developers will run this on their terminal for the
tests. But there are also other use cases like running these in
different CI tools like Cirrus, GH Actions etc where these build logs
are important in the web interface itself.

To fix this issue introduce a new variable named CIENV which defaults to
local if empty or undefined. If the CIENV is local in those cases the
logs are suppressed while for anything else it is not. This variable is
also important for some other WIP projects of CI I am working on like
running the CI tests on remote cloud AWS, Azure, GCP etc.

Approved by: lwhsu
Differential Revision: https://reviews.freebsd.org/D51178
Event: Oslo hackathon 202508

show more ...

tests/ci: Add KYUA_TEST_FILTERS to allow user to select specific tests

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D51306

tests/ci: Use QEMU blockdev declaration for all platforms

This patch uses the QEMU manpage-recommended '-blockdev' option instead
of '-drive' to declare block devices. This also makes the mandatory

tests/ci: Use QEMU blockdev declaration for all platforms

This patch uses the QEMU manpage-recommended '-blockdev' option instead
of '-drive' to declare block devices. This also makes the mandatory
'-device' declarations constant across all platforms, which fixes the
missing QEMU_DEVICES tarfs device declarations on all Makefile.<arch>
files.

Signed-off-by: Siva Mahadevan <me@svmhdvn.name>
Reviewed by: imp, lwhsu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D51479

show more ...

tests/ci: Fix wrong chflags target path in 'beforeclean' target

Signed-off-by: Siva Mahadevan <me@svmhdvn.name>

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.o

tests/ci: Fix wrong chflags target path in 'beforeclean' target

Signed-off-by: Siva Mahadevan <me@svmhdvn.name>

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D51304

show more ...

tests/ci: Fix race condition with ci-extractmeta

This solves an issue where 'ci-extractmeta' runs in parallel with
'ci-runtest', which would extract an empty test result into the
final test reports

tests/ci: Fix race condition with ci-extractmeta

This solves an issue where 'ci-extractmeta' runs in parallel with
'ci-runtest', which would extract an empty test result into the
final test reports output dir. This also cleans up the intermediate
temporary files created along the way.

Signed-off-by: Siva Mahadevan <me@svmhdvn.name>

Reviewed by: emaste, lwhsu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D51363

show more ...

tests/ci: Indent Makefile properly

Approved by: imp, lwhsu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D51445

tests/ci: Fix missing qemu devices

Fix missing QEMU_DEVICES which in turn is the META_TAR device

Approved by: imp, lwhsu
Sponsored by: The FreeBSD Foundation
Pull Request: https://reviews.freebsd.o

tests/ci: Fix missing qemu devices

Fix missing QEMU_DEVICES which in turn is the META_TAR device

Approved by: imp, lwhsu
Sponsored by: The FreeBSD Foundation
Pull Request: https://reviews.freebsd.org/D51445

show more ...

tests/ci: fix race condition in bhyveload boot

Currently, there is a race condition where the
Makefile sh invocation of 'bhyveload -c stdio'
exits before receiving any user input in the bootloader
(

tests/ci: fix race condition in bhyveload boot

Currently, there is a race condition where the
Makefile sh invocation of 'bhyveload -c stdio'
exits before receiving any user input in the bootloader
(e.g. in the form of a user typically pressing <enter>),
causing the 'expect' script to wrongly spawn the 'bhyve'
invocation while still in the bootloader.

This patch wraps bhyveload in expect(1) to ensure that
stdio is correctly consumed before the process exits.

Signed-off-by: Siva Mahadevan <me@svmhdvn.name>

Sponsored by: The FreeBSD Foundation
Pull Request: https://github.com/freebsd/freebsd-src/pull/1757

show more ...

tests/ci: Use suitable variable for qemu-user-static existence check

And simplify a qemu-system existence check.

Signed-off-by: Siva Mahadevan <me@svmhdvn.name>

Reviewed by: imp
Sponsored by: The

tests/ci: Use suitable variable for qemu-user-static existence check

And simplify a qemu-system existence check.

Signed-off-by: Siva Mahadevan <me@svmhdvn.name>

Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Pull Request: https://github.com/freebsd/freebsd-src/pull/1751

show more ...

Fix typos in the CI configuration

Fixes: a5f03413aade8f69cf954162151da043d34dd1f5
Fixes: c08f5ad160bf7c844677e1960b1a882d492006f8
Reviewed by: asomers, imp, zlei
Pull Request: https://github.com/f

Fix typos in the CI configuration

Fixes: a5f03413aade8f69cf954162151da043d34dd1f5
Fixes: c08f5ad160bf7c844677e1960b1a882d492006f8
Reviewed by: asomers, imp, zlei
Pull Request: https://github.com/freebsd/freebsd-src/pull/1736

show more ...

Refactor pre-commit CI

Currently our test mechanism is not fit for building without
cross-toolchain as we are not allowing EXTRA_MAKE_FLAGS. Also we are not
allowing to build and test other KERNCONF

Refactor pre-commit CI

Currently our test mechanism is not fit for building without
cross-toolchain as we are not allowing EXTRA_MAKE_FLAGS. Also we are not
allowing to build and test other KERNCONF. So allow these variables to
be overridden in command line.

Approved by: lwhsu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D50220

show more ...

tests/ci: Switch vmm detection

Previously, the CI Makefile used `kldload -n vmm` to determine whether
the host supported bhyve virtualization. However, due to changes in how
QEMU/KVM exposes CPU fea

tests/ci: Switch vmm detection

Previously, the CI Makefile used `kldload -n vmm` to determine whether
the host supported bhyve virtualization. However, due to changes in how
QEMU/KVM exposes CPU features to guests, this check no longer reliably
indicates the presence of usable VMX/SVM support.

In particular, modern QEMU configurations expose the vmx CPUID bit by
default, allowing the vmm module to load even when nested virtualization
is disabled. This causes the old check to incorrectly succeed inside
virtualized environments where bhyve cannot function.

This patch replaces the check with a sysctl-based approach that inspects
kern.vm_guest. If the system identifies itself as running inside a guest
(!= "none"), we assume nested virtualization is unavailable and fallback
to QEMU.

This change ensures CI behaves correctly on platforms where the vmm
module loads but is not usable.

Approved by: emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D50115

show more ...

CI: Add full test support

The patch adds support for running full tests in the local CI
Environment. New features added:

- New target `ci-full` which runs the full test. This is also the
default

CI: Add full test support

The patch adds support for running full tests in the local CI
Environment. New features added:

- New target `ci-full` which runs the full test. This is also the
default now
- Renamed the previous target `ci-smokeit` to `ci-smoke`
- Unlike previous if the available memory is more than 16G a default of
8G will be used
- Removed some unnecessary debug messages
- Added `dummybuf` kernel module to the list of modules to be loaded on
the VM

The features that can be tested:
`make TARGET=<TARGET> TARGET_ARCH=<TARGET_ARCH> CITYPE=full ci`
`make TARGET=<TARGET> TARGET_ARCH=<TARGET_ARCH> ci` is also the same as
above as CITYPE full is the default now

Approved by: lwhsu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D48015

show more ...

ci: Redirect output for builds.

This target is far too noisy to be at all useful. Save the output ala
make universe in _. files. Also report where to find errors.

Sponsored by: Netflix

Add preliminary in-tree CI infrastructure for developers

The goal of this project is to integrate the relevant scripts from the
FreeBSD-CI project (https://github.com/freebsd/freebsd-ci) into the sr

Add preliminary in-tree CI infrastructure for developers

The goal of this project is to integrate the relevant scripts from the
FreeBSD-CI project (https://github.com/freebsd/freebsd-ci) into the src
repository. This allows developers to run the test suite similar to how
it is executed on ci.freebsd.org, and eventually, have it directly used
by our CI system. This effort is also part of the workflow improvement
project, aiming to incorporate pre-merge testing.

Current Features:
* Does smoke tests using either bhyve(amd64 only) or qemu(Non x86_64 or
when defined USE_QEMU=1). Currently defined CITYPE=smoke. Once we have
added full tests we can also utilize something like CITYPE=full
* Most of the resources are dynamically allocated based on available
resources in the host
* If CPU supports POPCNT or vmm can be loaded then bhyve is used for
amd64 otherwise automatically installs and uses qemu@nox11
* When required third party applications or packages for booting non-x86
images are automatically installed

Current Limitation:
* Does not support full tests like the one in our Jenkins
* At this moment this is also not suitable to be used in our Jenkins
platform as the jobs are divided in multiple smaller tasks and
artifacts are moved here and there which are not exactly the scenario
for individual developers.

Future Works:
* Add full tests like the one in ci.freebsd.org
* Add different tests or options to disable some tests
* Add test profiles full
* Possibly add test through Cloud Providers like AWS/GCP/Azure or Cirrus
or Github Actions
* Update documentation

Test Plan:
cd /usr/src/tests/ci
make ci
make TARGET=amd64 TARGET_ARCH=amd64 ci
make TARGET=amd64 TARGET_ARCH=amd64 USE_QEMU=1 ci
make TARGET=arm64 TARGET_ARCH=aarch64 ci
make TARGET=powerpc TARGET_ARCH=powerpc64 ci
make TARGET=powerpc TARGET_ARCH=powerpc64le ci
make TARGET=riscv TARGET_ARCH=riscv64 ci

Reviewed by: lwhsu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43786

show more ...