Kyua is a testing framework for infrastructure software, originally
designed to equip BSD-based operating systems with a test suite. This
means that Kyua is lightweight and simple, and that Kyua integrates well
with various build systems and continuous integration frameworks.
Kyua features an expressive test suite definition language, a safe
runtime engine for test suites and a powerful report generation engine.
Kyua is for both developers and users, from the developer applying a
simple fix to a library to the system administrator deploying a new
release on a production machine.
Kyua is able to execute test programs written with a plethora of testing
libraries and languages. The library of choice is ATF, for which Kyua
was originally designed, but simple, framework-less test programs and
TAP-compliant test programs can also be executed through Kyua.
===> The following configuration options are available for kyua-0.14.1,3:
DOCS=on: Build and/or install documentation
EXAMPLES=on: Build and/or install examples
TEST=on: Build and/or run tests
===> Use 'make config' to modify these settings
A tests group has been added and the tests user should be a member
of it by default rather than nobody. You should update your password
database to match (change the group from 65534 to 977 after updating).
To do it, please run the following command:
pw usermod -n tests -g tests
devel/kyua: update to 0.14.1
This version update brings in some minor fixes around building from
source, etc. Most of the changes are not apparent to FreeBSD as they
relate to FreeBSD-specific features.
More information can be found in the
[release notes](https://github.com/freebsd/kyua/releases/tag/kyua-0.14.1).
Approved by: bofh (maintainer)
Reviewed by: bofh
Differential Revision: https://reviews.freebsd.org/D49574
devel/kyua: revert update to 0.14.1
I accidentally jumped the gun pushing this commit. There was an
outstanding comment that needed to be resolved.
This reverts commit 2922224d0e299e924f2f929a0001f60e99fb371e.
Differential Revision: https://reviews.freebsd.org/D49574
devel/kyua: update to 0.14.1
This version update brings in some minor fixes around building from
source, etc. Most of the changes are not apparent to FreeBSD as they
relate to FreeBSD-specific features.
More information can be found in the
[release notes](https://github.com/freebsd/kyua/releases/tag/kyua-0.14.1).
Approved by: bofh (maintainer)
Reviewed by: bofh
Differential Revision: https://reviews.freebsd.org/D49574
devel/kyua: Keep IGNORE off until the alignment with other ports
Reviewed by: bofh
Approved by: bofh
Differential Revision: https://reviews.freebsd.org/D47824
devel/kyua: Deprecate in favor of the base version
- Kyua is part of the base in all supported versions
- Kyua's evolution happens in the base, let's avoid duplicating work
- Set EXPIRATION_DATE to 2026-04-30 (expected 13 branch EOL)
Reviewed by: bofh
Approved by: bofh
Differential Revision: https://reviews.freebsd.org/D47473
devel/kyua: Update to latest snapshot
- Moved man to share/man
- Adopt based on last commit. Somehow despite mentioning about adoption
I missed to change the MAINTAINER.
Approved by: jmmv (private mail)
devel/kyua: Switch to FreeBSD upstream
- As per the recent discussion switch to the FreeBSD upstream
- For now resetting MAINTAINER to myself with ci@ hat on but anyone from
ci@ feel free to commit
- Pet port{clippy|fmt}
PR: 276071
Approved by: jmmv (private mail)
*/*: Fix broken UCL
Most of this is a missing ] or }. In one case, it was leading
whitespace before a EOM tag.
PR: 259533
Approved by: portmgr (blanket)
devel/kyua: fix build with clang 16
Since clang 16 (and gcc 11) the default C++ standard is now gnu++17.
Because devel/kyua's Makefile does not explicitly set its C++ standard,
this leads to several errors:
In file included from cli/cmd_about.cpp:29:
In file included from ./cli/cmd_about.hpp:35:
./cli/common.hpp:71:14: error: no template named 'auto_ptr' in namespace 'std'
typedef std::auto_ptr< cli_command > cli_command_ptr;
~~~~~^
In file included from cli/cmd_db_exec.cpp:29:
In file included from ./cli/cmd_db_exec.hpp:37:
./cli/common.hpp:71:14: error: no template named 'auto_ptr' in namespace 'std'
typedef std::auto_ptr< cli_command > cli_command_ptr;
(Only the first 15 lines of the commit message are shown above )
Remove WWW entries moved into port Makefiles
Commit b7f05445c00f has added WWW entries to port Makefiles based on
WWW: lines in pkg-descr files.
This commit removes the WWW: lines of moved-over URLs from these
pkg-descr files.
Approved by: portmgr (tcberner)
Add WWW entries to port Makefiles
It has been common practice to have one or more URLs at the end of the
ports' pkg-descr files, one per line and prefixed with "WWW:". These
URLs should point at a project website or other relevant resources.
Access to these URLs required processing of the pkg-descr files, and
they have often become stale over time. If more than one such URL was
present in a pkg-descr file, only the first one was tarnsfered into
the port INDEX, but for many ports only the last line did contain the
port specific URL to further information.
There have been several proposals to make a project URL available as
a macro in the ports' Makefiles, over time.
(Only the first 15 lines of the commit message are shown above )
*/*: Remove redundant '-[0-9]*' from CONFLICTS
The conflict checks compare the patterns first against the package
names without version (as reported by "pkg query "%n"), then - if
there was no match - agsinst the full package names including the
version (as reported by "pkg query "%n-%v").
Many CONFLICTS definitions used patterns like "bash-[0-9]*" to filter
for the bash package in any version. But that pattern is functionally
identical with just "bash".
Approved by: portmgr (blanket)
devel/kyua: Pet portlint
Also while here replace MAKE_FLAGS with MAKE_ARGS and since the
submission was almost there also pet portclippy.
PR: 231843
Submitted by: takefu@airport.fm
Approved by: jmmv (maintainer timeout, >2 years)
Create and use a tests group for the tests user.
Nobody but nobody should be a member of the nobody group.
PR: 245107
Approved by: maintainer timeout
Sponsored by: DARPA
WITH_CCACHE_BUILD: Don't leak ccache in as a runtime-dependency.
fmake and bmake's := feature does not evaluate a variable immediately
if it does not yet exist (bmake's manpage notes this). In this case
BUILD_DEPENDS is empty but later gets ccache added to it. So when
RUN_DEPENDS is finally evaluated it actually gets ccache in it.
This line was not useful anyhow since there were not BUILD_DEPENDS to add
in; LIB_DEPENDS is its own unique thing that works as a BUILD and RUN
dependency but not directly related to the other variables.
Sponsored by: Dell EMC Isilon
Fix utils/datetime_test:timestamp__subtraction
Don't expect a negative time delta to throw an exception--instead, expect
1 microsecond to be returned per the behavior change made in r436312.
Approved by: brooks (as part of a larger diff), (maintainer timeout)
Differential Revision: D10316
PR: 217973
Reported by: Jenkins
Reviewed by: asomers
Sponsored by: Dell EMC Isilon
devel/kyua: remove $FreeBSD$ RCS keyword from patch added in r436312
No functional change.
Approved by: brooks (as part of a larger diff), (maintainer timeout)
Differential Revision: D10316
Requested by: mat
Sponsored by: Dell EMC Isilon
Hack around wall clock time going backwards.
As reported in https://github.com/jmmv/kyua/issues/155, the wall clock
time can go backwards resulting in an apparent negative delta. As
a workaround, convert such deltas to 1us. This allows tests to run
successfully in MIPS64 qemu.
Approved by: jmmv (maintainer)
Sponsored by: DARPA, AFRL
Update devel/kyua to 0.13 from 0.11:
**Released on August 26th, 2016.**
* Fixed execution of test cases as an unprivileged user, at least under
NetBSD 7.0. Kyua-level failures were probably a regression introduced
in Kyua 0.12, but the underlying may have existed for much longer:
test cases might have previously failed for mysterious reasons when
running under an unprivileged user.
* Issue #134: Fixed metadata test broken on 32-bit platforms.
* Issue #139: Added per-test case start/end timestamps to all reports.
* Issue #156: Fixed crashes due to the invalid handling of cleanup
(Only the first 15 lines of the commit message are shown above )
Revert devel/kyua due to random segfaults during longer runs.
See: https://github.com/jmmv/kyua/issues/156
Approved by: jmmv (maintainer), bdrewery (mentor)
Update devel/kyua to 0.12.
**Released on November 22nd, 2015.**
This is a huge release and marks a major milestone for Kyua as it finally
implements a long-standing feature request: the ability to execute test
cases in parallel. This is a big deal because test cases are rarely
CPU-bound: running them in parallel yields much faster execution times for
large test suites, allowing faster iteration of changes during development.
As an example: the FreeBSD test suite as of this date contains 3285 test
cases. With sequential execution, a full test suite run takes around 12
minutes to complete, whereas on a 4-core machine with a high level of
parallelism it takes a little over 1 minute.
(Only the first 15 lines of the commit message are shown above )
Update devel/kyua to 0.11.
Released on October 23rd, 2014.
* Added support to print the details of all test cases (metadata and
their output) to 'report'. This is via a new '--verbose' flag which
replaces the previous '--show-context'.
* Added support to specify the amount of physical disk space required
by a test case. This is in the form of a new "required_disk_space"
metadata property, which can also be provided by ATF test cases as
"require.diskspace".
* Assimilated the contents of all the kyua-*-tester(1) and
kyua-*-interface(7) manual pages into more relevant places. In
(Only the first 15 lines of the commit message are shown above )
Fix TAP parsing bugs.
Pull up an upstream fix to properly deal with the TAP output of several
FreeBSD tests. Do this before a new Kyua release so that I and others can
properly test the changes before Kyua 0.11 is cut.
The FreeBSD test suite no longer reports failures caused by bad TAP parsing.
Bump PORTREVISION to 1.
Differential Revision: D832
Approved by: bapt (ports)
Update kyua to 0.10:
This change merges the kyua-{atf-compat,cli,testers} packages
into a single devel/kyua package, just as upstream has done, and brings
Kyua to version 0.10.
The list of changes since kyua-cli 0.9 and kyua-testers 0.3:
Experimental version released on August 14th, 2014.
* Merged 'kyua-cli' and 'kyua-testers' into a single 'kyua' package.
* Dropped the 'kyua-atf-compat' package.
* Issue 100: Do not try to drop privileges to unprivileged_user when we
(Only the first 15 lines of the commit message are shown above )
Turn devel/kyua into a meta-port and add devel/kyua-cli.
The existing devel/kyua package should have been named devel/kyua-cli to
match the upstream structure. Do the rename now and make devel/kyua be
a meta-port that depends on all kyua packages.
Because there is no version number to represent Kyua as a whole (that's
a problem I have to resolve upstream somehow), bump the epoch of
devel/kyua and use today's date as the version.
No entry should be necessary in MOVED because upgrading to the new
meta-port should do the right thing.
Approved by: bdrewery
Add staging support to kyua* and lutok.
While doing this, consolidate file handling into pkg-plist (instead of
the previous split between the Makefile and the pkg-plist) for simplicity.
Reviewed by: asomers (ex-maintainer)
Approved by: bdrewery (ports)
Change maintainer of devel/{lutok,kyua-testers,kyua,kyua-atf-compat} from
Alan Somers to Julio Merino, who is also the upstream maintainer.
Approved by: bdrewery
Add a tests user to run unprivileged tests as; use it in devel/kyua.
This new user carries a generic 'tests' name because, even though it is
only used by devel/kyua at this point, it could conceivably be used by
other tools.
Bump PORTREVISION of devel/kyua to 1.
PR: ports/182891
Approved by: asomers (maintainer), bdrewery (ports), rpaulo (mentor)
Update to version 0.7. This version depends on the newly updated lutok-0.3.
PR: 183089
Submitted by: Julio Merino <julio@meroh.net>
Reviewed by: asomers
Approved by: bdrewery
New port: devel/kyua:
Kyua (pronounced Q.A.) is a testing framework for both developers and
users. Kyua is different from most other testing frameworks in that it
puts the end user experience before anything else. There are multiple
reasons for users to run the tests themselves, and Kyua ensures that
they can do so in the most convenient way.
At the moment, Kyua is focused on implementing a solid foundation and a
powerful command-line tool to run tests implemented with the Automated
Testing Framework (ATF). Later on, Kyua will also provide a set of
language bindings (C, C++ and shell, at the least) to ease the
implementation of test cases in a variety of programming languages.
In effect, Kyua is intended to be a replacement for ATF.
WWW: https://code.google.com/p/kyua/
PR: ports/177641
Submitted by: asomers
Reviewed by: Garrett Cooper <yaneurabeya@gmail.com>
