Now that we're using TAP parsing, this will show subtest failures in
details but without showing any logging error, that we'd still need to parse
from actual logs.
This reverts commit 91f14cd058.
The freedesktop SDK, which is used by gnome-build-meta, only has Meson
0.63. Bumping GLib’s Meson dependency to 0.64 means that, at the moment,
GLib is not buildable in gnome-build-meta and hence can’t be tested in
nightly pipelines against other projects, etc.
That’s bad for testing GLib.
It’s arguably bad that we’re restricted to using an older version of
Meson than shipped by Debian Testing, but that’s a separate discussion
to be had.
Revert the Meson 0.64 dependency until the freedesktop SDK ships Meson ≥
0.64. This also means reverting the simplifications to use of
`gnome.mkenum_simple()`.
See https://gitlab.gnome.org/GNOME/glib/-/merge_requests/3077#note_1601064
We have tests that are failing in some environments, but it's
difficult to handle them because:
- for some environments we just allow all the tests to fail: DANGEROUS
- when we don't allow failures we have flacky tests: A CI pain
So, to avoid this and ensure that:
- New failing tests are tracked in all platforms
- gitlab integration on tests reports is working
- coverage is reported also for failing tests
Add support for `can_fail` keyword on tests that would mark the test as
part of the `failing` test suite.
Not adding the suite directly when defining the tests as this is
definitely simpler and allows to define conditions more clearly (see next
commits).
Now, add a default test setup that does not run the failing and flaky tests
by default (not to bother distributors with testing well-known issues) and
eventually run all the tests in CI:
- Non-flaky tests cannot fail in all platforms
- Failing and Flaky tests can fail
In both cases we save the test reports so that gitlab integration is
preserved.
This is in preparation for porting `GRegex` to libpcre2, which is
happening in !2529. It’s a big port, though, and specially rebuilding
the CI images to add libpcre2 for it is a pain.
Add libpcre2, and then !2529 can drop the old libpcre dependencies when
the port lands.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Helps: #1085
Move the lcovrc file to the root of the project, so that it’s picked up
by Meson when running `ninja coverage` locally.
See https://github.com/mesonbuild/meson/issues/4628
This won’t affect the code coverage run on the CI, since that explicitly
used the lcovrc file already.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
This is what’s available in the new Debian Stable, so we can expect it
to be available pretty much everywhere.
Subsequent commits will clean up old workarounds.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
It was disabled in !875 because lcov didn't support the new coverage
format produced by gcc9+. The latest lcov release in MSYS2 supports
it again, so re-enable everything.
lcov now writes native Windows paths to its output so adjust the path
fixup script to handle those.
pip in MSYS2 seems to install scripts into $USERPROFILE instead of $HOME
which means the MSYS2 meson, which is newer, wins. Make sure $USERPROFILE
is in PATH as well.
I haven’t tested any of them. This is entirely mechanical. I used
shellcheck 0.7.0 with default options.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
lconv 1.13 only supports gcc <=7 and lcov 1.14 supports <=8.
msys2 was just updated to gcc9, so this wont help with coverage support, but
it's a start I guess.
This effectively renders those tests useless (since realistically nobody
runs tests locally), but it’s better than every other CI run failing for
unrelated reasons. The idea is that the ‘flaky’ tag can be temporarily
applied to a test while a problem is being investigated or fixed, and
then removed later.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
While we can’t add markers to the macro implementations to cause lcov to
ignore them automatically, we can change our lcov configuration to
ignore all calls to them.
See https://github.com/linux-test-project/lcov/issues/44.
This causes all the un-takeable branches and un-reachable assertions to
be ignored by our code coverage, which bumps our statistics:
• Lines: 74.9% → 74.8%
• Functions: 82.3% → 82.3%
• Branches: 53.3% → 64.2%
The rationale is that nobody should be testing programmer error
handling, as g_return_*if_fail() are used to guard against — so it’s not
reasonable to count missed branches like that in code coverage
statistics.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
The gresource code uses libelf if available but that also depends on mmap but isn't
guarded with HAVE_MMAP. This can make the build fail under MSYS2 where a mingw version
of libelf exists but there is no mmap.
Instead of guarting the libelf code with HAVE_LIBELF add a new macro named USE_LIBELF
which is only defined if libelf and mmap support are available.
Also install the mingw libelf version for CI so we catch similar errors in the future.
See comment in !151. Using the "--initial" option of lcov we collect
the coverage of all compiled files and merge them later into the final
report. This way we can see which files are built but never executed
by the test suite.
Because the --initial switch also collects files in the ccache directory
we have to point it to the build directory instead, which in turn breaks
--no-external. Instead of using --no-external in the collection step,
filter out any files not in the source tree in the final coverage job
through a path filter.
Use lcov for both Fedora and MSYS2 to create coverage reports and add a second
ci stage which merges the coverage and creates a html report using genhtml.
In the final stage, which is only run on master, the result is published on
gitlab pages.
https://bugzilla.gnome.org/show_bug.cgi?id=795636
Fix various warnings regarding unused variables, duplicated
branches etc by adjusting the ifdeffery and some missing casts.
gnulib triggers -Wduplicated-branches in one of the copied files,
disable as that just makes updating the code harder.
The warning indicating missing features are made none fatal through
pragmas. They still show but don't abort the build.
https://bugzilla.gnome.org/show_bug.cgi?id=793729
This builds glib using meson/ninja/ccache with mingw-w64 on a Windows
machine.
The CI scripts expect a gitlab runner to exist with the "win32" tag
which uses the default "cmd" shell by default.
Before running the tests pacman is invoked to update the system
(potentially including bash etc, thus the extra step)
Then a login shell is started with CHERE_INVOKING to not change the
cwd and finally the test script is executed.
https://bugzilla.gnome.org/show_bug.cgi?id=793729
