In all these cases we don't really care about running the test file,
while building and basic execution it is relevant.
Also they don't support TAP at all.
Meson supports tap protocol results parsing, allowing us to track better
the tests that are running (and the ones that are actually skipped) without
manually parsing the test output.
However this also implies that using the verbose mode for a test doesn't
show its output by default (unless there are failures).
Given that it can be computed using an error-prone strings comparisons it
is better to provide a variable everywhere, so that we don't have the
risk of comparing values that are always false.
We have tests that are failing in some environments, but it's
difficult to handle them because:
- for some environments we just allow all the tests to fail: DANGEROUS
- when we don't allow failures we have flacky tests: A CI pain
So, to avoid this and ensure that:
- New failing tests are tracked in all platforms
- gitlab integration on tests reports is working
- coverage is reported also for failing tests
Add support for `can_fail` keyword on tests that would mark the test as
part of the `failing` test suite.
Not adding the suite directly when defining the tests as this is
definitely simpler and allows to define conditions more clearly (see next
commits).
Now, add a default test setup that does not run the failing and flaky tests
by default (not to bother distributors with testing well-known issues) and
eventually run all the tests in CI:
- Non-flaky tests cannot fail in all platforms
- Failing and Flaky tests can fail
In both cases we save the test reports so that gitlab integration is
preserved.
In principle we could script this so that each max-version.c is compiled
26 times, once per possible MAX_VERSION, but I haven't implemented
that here: just pinning to the oldest possible version is sufficient to
reproduce #2796.
These aren't included in the installed-tests, since they don't really
do anything at runtime (the important thing is that they compile
without warnings).
Reproduces: #2796
Signed-off-by: Simon McVittie <smcv@collabora.com>
Otherwise, the build will fail when the toolchain is static-only, even
with -Ddefault_library=static. I talked to a Meson developer in their
IRC channel, who told me that the correct fix was to ensure that
shared_library is only used if default_library != static.
We've various macros definitions that are depending using C++ features
that may not work in all the standard versions, so recompile the cxx
tests that we have in all the ones we want to support.
We don't need a cpp toolchain for building glib so lets just
automatically disable tests requiring one when not available.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
