mirror of
https://gitlab.gnome.org/GNOME/glib.git
synced 2025-08-01 15:03:39 +02:00
meson, ci: Support tests that can fail under certain conditions
We have tests that are failing in some environments, but it's difficult to handle them because: - for some environments we just allow all the tests to fail: DANGEROUS - when we don't allow failures we have flacky tests: A CI pain So, to avoid this and ensure that: - New failing tests are tracked in all platforms - gitlab integration on tests reports is working - coverage is reported also for failing tests Add support for `can_fail` keyword on tests that would mark the test as part of the `failing` test suite. Not adding the suite directly when defining the tests as this is definitely simpler and allows to define conditions more clearly (see next commits). Now, add a default test setup that does not run the failing and flaky tests by default (not to bother distributors with testing well-known issues) and eventually run all the tests in CI: - Non-flaky tests cannot fail in all platforms - Failing and Flaky tests can fail In both cases we save the test reports so that gitlab integration is preserved.
This commit is contained in:
@@ -149,9 +149,9 @@ test_extra_programs = {
|
||||
'gsubprocess-testprog' : {},
|
||||
}
|
||||
|
||||
python_tests = [
|
||||
'codegen.py',
|
||||
]
|
||||
python_tests = {
|
||||
'codegen.py' : {},
|
||||
}
|
||||
|
||||
test_env = environment(common_test_env)
|
||||
test_env.set('G_TEST_SRCDIR', meson.current_source_dir())
|
||||
@@ -907,6 +907,10 @@ foreach test_name, extra_args : gio_tests
|
||||
local_test_env.append(var, value)
|
||||
endforeach
|
||||
|
||||
if extra_args.get('can_fail', false)
|
||||
suite += 'failing'
|
||||
endif
|
||||
|
||||
test(test_name, exe,
|
||||
env : local_test_env,
|
||||
timeout : timeout,
|
||||
@@ -930,13 +934,19 @@ foreach program_name, extra_args : test_extra_programs
|
||||
)
|
||||
endforeach
|
||||
|
||||
foreach test_name : python_tests
|
||||
foreach test_name, extra_args : python_tests
|
||||
suite = ['gio', 'no-valgrind']
|
||||
|
||||
if extra_args.get('can_fail', false)
|
||||
suite += 'failing'
|
||||
endif
|
||||
|
||||
test(
|
||||
test_name,
|
||||
python,
|
||||
args: ['-B', files(test_name)],
|
||||
env: test_env,
|
||||
suite: ['gio', 'no-valgrind'],
|
||||
suite: suite,
|
||||
)
|
||||
|
||||
if installed_tests_enabled
|
||||
|
Reference in New Issue
Block a user