This effectively renders those tests useless (since realistically nobody
runs tests locally), but it’s better than every other CI run failing for
unrelated reasons. The idea is that the ‘flaky’ tag can be temporarily
applied to a test while a problem is being investigated or fixed, and
then removed later.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Since we are unable to promote our FreeBSD runner to a shared runner,
we can only enable it in GNOME group. This should avoid problems when
submitting merge requests from forks.
While we can’t add markers to the macro implementations to cause lcov to
ignore them automatically, we can change our lcov configuration to
ignore all calls to them.
See https://github.com/linux-test-project/lcov/issues/44.
This causes all the un-takeable branches and un-reachable assertions to
be ignored by our code coverage, which bumps our statistics:
• Lines: 74.9% → 74.8%
• Functions: 82.3% → 82.3%
• Branches: 53.3% → 64.2%
The rationale is that nobody should be testing programmer error
handling, as g_return_*if_fail() are used to guard against — so it’s not
reasonable to count missed branches like that in code coverage
statistics.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This partially reverts commit bfc362cb68.
The FreeBSD CI runner still seems to be a bit flakey, and appears to be
offline entirely at the moment.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Reviewed-by: nobody
It fails because dist-job (correctly) doesn’t build with the code
coverage CFLAGS enabled.
Leave coverage to be generated on master and development branches.
See this pipeline for an example of when it fails:
https://gitlab.gnome.org/GNOME/glib/pipelines/26349
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Similar issue was fixed with commit f929d148, but it's happening again.
Define G_MESSAGES_DEBUG=all when running CI to ensure we won't regress
anymore.
See comment in !151. Using the "--initial" option of lcov we collect
the coverage of all compiled files and merge them later into the final
report. This way we can see which files are built but never executed
by the test suite.
Because the --initial switch also collects files in the ccache directory
we have to point it to the build directory instead, which in turn breaks
--no-external. Instead of using --no-external in the collection step,
filter out any files not in the source tree in the final coverage job
through a path filter.
This requires meson >= 0.47.0 otherwise building the doc fails:
https://github.com/mesonbuild/meson/issues/3379
While at it, no need to to pass --prefix --libdir to meson, other CIs
don't have them.
- Split the download part into a separate script to so docker keeps that
step in cache and avoid redownloading it.
- With API level >= 28 libiconv is not needed anymore because it's part
of Android's libc.
- Generate standalone toolchains to reduce the docker image size. It's
also easier because it doesn't need to pass sysroot args.
- Use clang compiler because gcc is deprecated in this Android NDK and
will be removed in the next release.
Job names in gitlab pipeline view gets truncated to "fedora-meson-..."
for all jobs which is not really useful. All our CIs are using Meson,
and the host distro is not relevant when doing cross builds.
We should be testing latest NDK release but keep using API level 21 to
ensure GLib does not start using newer APIs. We could also later add a
runner for latest API level 28 which includes iconv API in Android's
libc so we don't need GNU libiconv anymore.
The default `when: on_success` means no artifacts are exported when the
tests fail, which is precisely when we want to see the artifacts.
Always export them, and rely on GitLab to garbage collect them as
appropriate (typically after 30 days, I think).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This means that compilation and test failures will leave logs on the CI
machines which we can use to debug the failures.
Always export the artifacts. The default `when: on_success` is
appropriate for now, but won’t be in future when the tests actual report
their success value correctly.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This adds to the CI a cross build for Android NDK r16 API 21 (because
thats what GStreamer currently use) for arm64.
GNU iconv must be built manually into our docker image because Android
NDK doesn't seems to ship it. The latest NDK r17 API 28 has iconv.h but
iconv_open() symbol isn't found by the linker. Looks like broken NDK.
libffi also needs to be built manually because the meson subproject
doesn't support building for Android platform. It needs a recent RC
release because latest stable release is 4 years old and fails to
build.
Fixes#1385.
GitLab can then use this to annotate each pipeline with its code
coverage statistics. It can only use one figure, so we choose lines
(rather than function or branch coverage) since it’s the most intuitive
figure.
This parses the ‘lines’ line from output like:
Overall coverage rate:
lines......: 76.7% (108959 of 142122 lines)
functions..: 80.7% (10294 of 12763 functions)
branches...: 51.3% (50226 of 97953 branches)
Signed-off-by: Philip Withnall <withnall@endlessm.com>
It looks like the coverage generation makes the tests a bit slower and
some are now hitting timeouts. Flaky tests are always more annoying than
slow ones, and we don't know how much resources we get under CI,
so increase the timeout.
https://bugzilla.gnome.org/show_bug.cgi?id=795636
Use lcov for both Fedora and MSYS2 to create coverage reports and add a second
ci stage which merges the coverage and creates a html report using genhtml.
In the final stage, which is only run on master, the result is published on
gitlab pages.
https://bugzilla.gnome.org/show_bug.cgi?id=795636
This builds glib using meson/ninja/ccache with mingw-w64 on a Windows
machine.
The CI scripts expect a gitlab runner to exist with the "win32" tag
which uses the default "cmd" shell by default.
Before running the tests pacman is invoked to update the system
(potentially including bash etc, thus the extra step)
Then a login shell is started with CHERE_INVOKING to not change the
cwd and finally the test script is executed.
https://bugzilla.gnome.org/show_bug.cgi?id=793729
We're mostly interested into building and testing everything that gets
pushed to the repository — including merge requests.
When pushing tags, though, we should assume we're spinning a release, so
let's run the dist target, and store the tarball, and the generated
documentation while we're at it, as artifacts on GitLab.
The Dockerfile for the image used for the build is included in tree, and
published on Docker Hub. Using a custom image allows us to avoid the
costly "download and install build dependencies" phase, as well as
controlling the environment a little bit better.
https://bugzilla.gnome.org/show_bug.cgi?id=793635
