Commit Graph

9 Commits

Author SHA1 Message Date
Marco Trevisan (Treviño)
3c56d661d8 meson: Use test setup environment instead of repeating it everywhere 2022-10-31 14:08:31 +01:00
Marco Trevisan (Treviño)
62dca6c1cf meson, ci: Support tests that can fail under certain conditions
We have tests that are failing in some environments, but it's
difficult to handle them because:
 - for some environments we just allow all the tests to fail: DANGEROUS
 - when we don't allow failures we have flacky tests: A CI pain

So, to avoid this and ensure that:
 - New failing tests are tracked in all platforms
 - gitlab integration on tests reports is working
 - coverage is reported also for failing tests

Add support for `can_fail` keyword on tests that would mark the test as
part of the `failing` test suite.
Not adding the suite directly when defining the tests as this is
definitely simpler and allows to define conditions more clearly (see next
commits).

Now, add a default test setup that does not run the failing and flaky tests
by default (not to bother distributors with testing well-known issues) and
eventually run all the tests in CI:
 - Non-flaky tests cannot fail in all platforms
 - Failing and Flaky tests can fail

In both cases we save the test reports so that gitlab integration is
preserved.
2022-10-31 14:08:29 +01:00
Philip Withnall
4bc284fca6 Merge branch 'wip/smcv/deprecated-prop-followup' into 'main'
Run tests with G_ENABLE_DIAGNOSTIC=1

See merge request GNOME/glib!2889
2022-10-15 21:31:29 +00:00
Simon McVittie
88e160dfe4 tests: Move common test environment variables to top level
Signed-off-by: Simon McVittie <smcv@collabora.com>
2022-09-21 11:19:28 +01:00
Xavier Claessens
c00df192ee meson: Set install_tag on installed tests files
This could be done automatically by Meson, this commit can be reverted
when we have that Meson PR in our CI:
https://github.com/mesonbuild/meson/pull/10829
2022-09-20 11:30:02 -04:00
Simon McVittie
72868c026d gobject/tests/performance: Only run a quick version as installed-tests
ginsttest-runner defaults to timing out each test after 5 minutes,
but gobject/tests/performance/performance.c defaults to running each
of 18 tests for 15 seconds. The result is close enough to 5 minutes
that the setup overhead is enough to make it time out.

We're only running these tests to prove that they still work, not to
get meaningful performance numbers, so cut them down to 1 second per
test-case (the result of which is that performance.c takes about a
minute).

Signed-off-by: Simon McVittie <smcv@collabora.com>
2022-07-24 17:34:55 +01:00
Simon McVittie
0714bcb7f4 gobject/tests/performance: Use the other installed-tests template
These are not GTest tests, and don't output TAP.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2022-07-24 17:15:47 +01:00
Philip Withnall
6f435e40cd tests: Run GObject performance tests under meson test
Pass arguments to them so that they take minimal time. This will not
produce useful performance profiling results, but will smoketest that
the tests still run, don’t crash, and therefore probably aren’t
bitrotting too badly.

This is useful because a fair amount of work has gone into these
performance tests, and they’re useful every few years to analyse and
compare GObject performance. We don’t want them to bitrot between uses.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2022-06-21 12:57:33 +01:00
Philip Withnall
d463de05b1 tests: Move GObject performance tests to gobject/tests/performance/
This doesn’t change the tests’ behaviour, but moves them to a slightly
more logical location.

They are still not installed or run by default.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>

Helps: #1434
2022-06-08 11:30:31 +01:00