glib/.gitlab-ci/run-tests.sh

#!/bin/bash

set -ex

meson test -v \
        -C _build \
        --timeout-multiplier "${MESON_TEST_TIMEOUT_MULTIPLIER}" \
        "$@"

# Run only the flaky tests, so we can log the failures but without hard failing
meson test -v \
        -C _build \
        --timeout-multiplier "${MESON_TEST_TIMEOUT_MULTIPLIER}" \
        "$@" --setup=unstable_tests --suite=failing --suite=flaky || true
ci: Generate a cover report for the test suite GitLab can show the results of a CI pipeline if the pipeline generates a report using the JUnit XML format. Since Meson provides a machine parseable output for `meson test`, we can take that and turn it into XML soup. 2019-04-05 21:53:11 +02:00			`#!/bin/bash`

ci: Show commands run from the test scripts It allows better debugging in case of failures. 2022-10-20 03:07:52 +02:00			`set -ex`
ci: Fix failing tests not failing CI job 2022-10-26 16:53:36 +02:00
ci: Use verbose output in meson by default Now that we're using TAP parsing, this will show subtest failures in details but without showing any logging error, that we'd still need to parse from actual logs. 2023-01-20 15:41:17 +01:00			`meson test -v \`
ci: Generate a cover report for the test suite GitLab can show the results of a CI pipeline if the pipeline generates a report using the JUnit XML format. Since Meson provides a machine parseable output for `meson test`, we can take that and turn it into XML soup. 2019-04-05 21:53:11 +02:00			`-C _build \`
ci: Fix various shellcheck warnings in the shell scripts I haven’t tested any of them. This is entirely mechanical. I used shellcheck 0.7.0 with default options. Signed-off-by: Philip Withnall <withnall@endlessm.com> 2020-02-25 12:46:11 +01:00			`--timeout-multiplier "${MESON_TEST_TIMEOUT_MULTIPLIER}" \`
ci: Add valgrind memcheck support on Fedora Add a separate CI job which runs memcheck on the unit tests. This is done as a separate job from the main build, since we don’t want it to interact with code coverage at all. Currently, failure of this job is ignored. Issue #333 will eventually fix that. Signed-off-by: Philip Withnall <withnall@endlessm.com> Fixes: #487 2018-06-26 18:23:09 +02:00			`"$@"`
meson, ci: Support tests that can fail under certain conditions We have tests that are failing in some environments, but it's difficult to handle them because: - for some environments we just allow all the tests to fail: DANGEROUS - when we don't allow failures we have flacky tests: A CI pain So, to avoid this and ensure that: - New failing tests are tracked in all platforms - gitlab integration on tests reports is working - coverage is reported also for failing tests Add support for `can_fail` keyword on tests that would mark the test as part of the `failing` test suite. Not adding the suite directly when defining the tests as this is definitely simpler and allows to define conditions more clearly (see next commits). Now, add a default test setup that does not run the failing and flaky tests by default (not to bother distributors with testing well-known issues) and eventually run all the tests in CI: - Non-flaky tests cannot fail in all platforms - Failing and Flaky tests can fail In both cases we save the test reports so that gitlab integration is preserved. 2022-10-19 20:08:15 +02:00
			`# Run only the flaky tests, so we can log the failures but without hard failing`
ci: Use verbose output in meson by default Now that we're using TAP parsing, this will show subtest failures in details but without showing any logging error, that we'd still need to parse from actual logs. 2023-01-20 15:41:17 +01:00			`meson test -v \`
meson, ci: Support tests that can fail under certain conditions We have tests that are failing in some environments, but it's difficult to handle them because: - for some environments we just allow all the tests to fail: DANGEROUS - when we don't allow failures we have flacky tests: A CI pain So, to avoid this and ensure that: - New failing tests are tracked in all platforms - gitlab integration on tests reports is working - coverage is reported also for failing tests Add support for `can_fail` keyword on tests that would mark the test as part of the `failing` test suite. Not adding the suite directly when defining the tests as this is definitely simpler and allows to define conditions more clearly (see next commits). Now, add a default test setup that does not run the failing and flaky tests by default (not to bother distributors with testing well-known issues) and eventually run all the tests in CI: - Non-flaky tests cannot fail in all platforms - Failing and Flaky tests can fail In both cases we save the test reports so that gitlab integration is preserved. 2022-10-19 20:08:15 +02:00			`-C _build \`
			`--timeout-multiplier "${MESON_TEST_TIMEOUT_MULTIPLIER}" \`
			`"$@" --setup=unstable_tests --suite=failing --suite=flaky \|\| true`