kyua: Pass unprivileged user config prop to ATF using all known names

Kyua and ATF speak different naming styles. In this case, the
unprivileged user property can be named with underscore on the Kyu

kyua: Pass unprivileged user config prop to ATF using all known names

Kyua and ATF speak different naming styles. In this case, the
unprivileged user property can be named with underscore on the Kyua
side, and with a hyphen on the ATF side. Sometimes it is not obvious
which style should be used in which situation. For instance, a test case
may require this configuration property being set using require.config.
Also, a test case may want to read the property using something like
atf_tc_get_config_var(). Which names should be used in these cases?
From the perspective of the original code, it is expected to be this:
require.config unprivileged-user
atf_tc_get_config_var(tc, "unprivileged-user")

But, as long as Kyua is the main interface, its users expect to work
with kyua.conf(5), which says that it must be named as unprivileged_user
(with underscore). As a result, test authors tend to do this instead:
require.config unprivileged_user
atf_tc_get_config_var(tc, "unprivileged_user")

Kyua already has hacks to understand both unprivileged_user and
unprivileged-user coming from require.config. And this patch covers the
missing second part -- make Kyua pass both names back to ATF as two
identical configuration properties named different ways.

Reviewed by: ngie, asomers
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D49039

show more ...

kyua: Do not count skipped as passed in test cmd

It changes output of 'kyua test' CLI command only. Hence, other outputs
like junit are kept intact for CI and other use cases. It's meant to
improve

kyua: Do not count skipped as passed in test cmd

It changes output of 'kyua test' CLI command only. Hence, other outputs
like junit are kept intact for CI and other use cases. It's meant to
improve UX of attended use cases.

The issue is that the following can be tricky to interpret:

222/222 passed (0 failed)

It can be read as all tests are passed, but it might be a summary line
of all tests skipped due to some requirement is not met.

It's reworked to easily distinguish such cases:

222/222 passed (0 broken, 0 failed, 0 skipped)
0/222 passed (0 broken, 0 failed, 222 skipped)

The overall formula is:

<actually passed>/<total> (<details about not actually passed ones>)

Suggested by: kp
Reviewed by: ngie, markj
Approved by: markj (mentor)
Differential Revision: https://reviews.freebsd.org/D46653

show more ...

contrib/kyua: Merge vendor import

* commit '5f174897f67783925f4ec69122673f9bad6ee6fe':
vendor/kyua: Update to snapshot 84c8ec8
Vendor import of freebsd/kyua@a0d44bb356e0c816
Approved by: mark

contrib/kyua: Merge vendor import

* commit '5f174897f67783925f4ec69122673f9bad6ee6fe':
vendor/kyua: Update to snapshot 84c8ec8
Vendor import of freebsd/kyua@a0d44bb356e0c816
Approved by: markj
Differential Revision: https://reviews.freebsd.org/D43475

show more ...

Import the kyua test framework.

Having kyua in the base system will simplify automated testing in CI and
eliminates bootstrapping issues on new platforms.

The build of kyua is controlled by WITH(OU

Import the kyua test framework.

Having kyua in the base system will simplify automated testing in CI and
eliminates bootstrapping issues on new platforms.

The build of kyua is controlled by WITH(OUT)_TESTS_SUPPORT.

Reviewed by: emaste
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24103

show more ...