We have tests that are failing in some environments, but it's
difficult to handle them because:
- for some environments we just allow all the tests to fail: DANGEROUS
- when we don't allow failures we have flacky tests: A CI pain
So, to avoid this and ensure that:
- New failing tests are tracked in all platforms
- gitlab integration on tests reports is working
- coverage is reported also for failing tests
Add support for `can_fail` keyword on tests that would mark the test as
part of the `failing` test suite.
Not adding the suite directly when defining the tests as this is
definitely simpler and allows to define conditions more clearly (see next
commits).
Now, add a default test setup that does not run the failing and flaky tests
by default (not to bother distributors with testing well-known issues) and
eventually run all the tests in CI:
- Non-flaky tests cannot fail in all platforms
- Failing and Flaky tests can fail
In both cases we save the test reports so that gitlab integration is
preserved.
We were regressing on Python style too often. Since Python code style is
a lot easier to enforce than C code style, split it (and the shellcheck
checks) out from `style-check-diff` into a new CI job which is allowed
to fail the pipeline.
Only trigger it when .sh or .py files have changed, which should reduce
resource consumption.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
This disables the following warning, which was causing CI failures on
macOS when building the libpcre2 subproject:
```
../subprojects/pcre2-10.40/src/pcre2_error.c:66:3: error: string literal of length 4380 exceeds maximum length 4095 that ISO C99 compilers are required to support [-Werror,-Woverlength-strings]
```
We don’t want to explicitly rely on using overlength strings in GLib,
which is why this change is a `CFLAGS` in the CI configuration, rather
than setting a project-level argument in `meson.build`.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Since the macOS CI jobs are run on a machine which isn’t using a
pre-made container image, we can’t ship a cached version of the
subproject, so it has to be pulled as a git submodule.
GitLab doesn’t do that by default unless you set
`GIT_SUBMODULE_STRATEGY` to something other than `none`.
See https://docs.gitlab.com/ee/ci/git_submodules.html
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
This is in preparation for porting `GRegex` to libpcre2, which is
happening in !2529. It’s a big port, though, and specially rebuilding
the CI images to add libpcre2 for it is a pain.
Add libpcre2, and then !2529 can drop the old libpcre dependencies when
the port lands.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Helps: #1085
This means we can specify the standard options for testing GLib under
valgrind consistently, so that developers can use `meson test
--setup=valgrind` to run them.
Port the existing valgrind CI to use them (this will not change its
functional behaviour).
Suggested by Marco Trevisan at
https://gitlab.gnome.org/GNOME/glib/-/merge_requests/2717#note_1478891.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
iconv is complicated to look up. That complexity now resides in
Meson, since 0.60.0, via a `dependency('iconv')` lookup, so use that
instead.
No effort is made to support the old option for which type of iconv to
use. It was a false choice, because if only one was available, then
that's the only one you can use, and if both are available, the external
iconv shadows the builtin one and renders the builtin one unusable,
so there is still only one you can use.
This meant that when configuring glib with -Diconv=libc on systems that
had an external iconv, the configure check would detect a valid libc
iconv, try to use it, and then fail during the build because iconv.h
belongs to the external iconv and generates machine code using the
external iconv ABI, but fails to link to the iconv `find_library()`.
Meson handles this transparently.
Rather than carrying the copylib around inside GLib, which is a pain to
synchronise and affects our code coverage statistics.
This requires updating the CI images to cache the new subproject,
including updating the `cache-subprojects.sh` script to pull in git
submodules.
It also requires adding `gioenumtypes_dep` to be added to the
dependencies list of `libgio`, since it needs to be build before GVDB as
it’s pulled in by the GIO headers which GVDB includes.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Helps: #2603
They are [currently
failing](https://gitlab.gnome.org/GNOME/glib/-/jobs/2032874) with the
error:
```
1/273 glib:glib / array-test FAIL 0.19s killed by signal 11 SIGSEGV
05:04:16 G_DEBUG=gc-friendly G_TEST_BUILDDIR=/builds/GNOME/glib/_build/glib/tests MALLOC_CHECK_=2 MALLOC_PERTURB_=133 G_TEST_SRCDIR=/builds/GNOME/glib/glib/tests valgrind --tool=memcheck --error-exitcode=1 --track-origins=yes --leak-check=full --leak-resolution=high --num-callers=50 --show-leak-kinds=definite,possible --show-error-list=yes --suppressions=/builds/GNOME/glib/tools/glib.supp /builds/GNOME/glib/_build/glib/tests/array-test
----------------------------------- output -----------------------------------
stderr:
valgrind: m_libcfile.c:66 (vgPlain_safe_fd): Assertion 'newfd >= VG_(fd_hard_limit)' failed.
------------------------------------------------------------------------------
```
I’m not really sure what that means, but `show-execution-environment.sh`
says the FD soft limit is set to 524288 on the CI machine. That seems
high; on my machine it’s only 1024 (and the valgrind tests pass). So
let’s try 1024.
The valgrind CI has been failing since we most recently upgraded the CI
image to a new version of Fedora.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
The runner machine is offline and there is no ETA on when it will be
back, so disable the CI job which uses it for now so that pipelines
can proceed.
See https://gitlab.gnome.org/Infrastructure/GitLab/-/issues/558
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Meson used to try and guess at the Python path. While this worked fine
for GLib before, it probably didn’t work 100% for other projects, so
Meson have made it an explicit option.
Set that option with the Python path used on the Windows CI machines.
This fixes a Meson warning with Meson >0.60.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Move the lcovrc file to the root of the project, so that it’s picked up
by Meson when running `ninja coverage` locally.
See https://github.com/mesonbuild/meson/issues/4628
This won’t affect the code coverage run on the CI, since that explicitly
used the lcovrc file already.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
They are functionally quite similar, and combining them saves around 1
minute of CI runner time per pipeline, which is a nice saving for very
little downside.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
The default is 30 days, but we don’t need them around that long. This
should free up some disk space on the GitLab/CI runner systems.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
libgamin was last released in 2007 and is dead
[upstream](https://gitlab.gnome.org/Archive/gamin). Distributions may
still ship it (although Fedora no longer does), but we want people to
use inotify on Linux since it’s actively supported.
BSDs use kqueue. Windows uses win32filemonitor.
FAM might still be used on some commercial Unix distributions, but there
are no contributors from those distributions, and certainly no CI for
them to prevent regressions.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Fixes: #2614
Don’t take the opportunity to add support for Android API 31 or update
the version of the Android NDK we’re using to r23b, though, as I
couldn’t quickly get that to work and ran out of time.
Bumping the Fedora version will at least reduce our CI repository disk
usage through sharing the base image.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Helps: #2605
This is the oldest still-supported version of Fedora.
It no longer ships gamin.
This should fix the installed-tests, which rely on version 0.19 of
python-dbusmock. Fedora 33 only had 0.18.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Test failures were previously ignored on macOS because there are 12
tests which consistently fail (and have not yet been fixed, because
there are no regularly active macOS maintainers for GLib; you could help
here!).
However, this means that new test failures can’t be spotted.
So, explicitly mark those 12 tests as `should_fail` on macOS, and then
make other test failures cause failure of the CI run.
We can track the process of fixing those 12 tests on #1392 and #1251.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Helps: #1392
Rather than running them on each commit on `main`. This saves resources.
Given that Android API 28 and FreeBSD 13 jobs continue to be run on each
commit on `main`, this seems like an acceptable tradeoff. It’s very
unlikely that a regression will happen which affects the older systems
and *not* the newer systems. If it does, it will be caught within a week
by the scheduled job.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
This clarifies the intent of the `branches@GNOME/glib` selector. It
introduces no functional changes.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
As with a previous commit, `.only-default` does things which are
orthogonal to what `.build-linux` does, so it’s clearer and more
extensible for CI jobs to specify both in their top-level `extends`
statements, rather than relying on `.build-linux` to pull
`.only-default` in.
This introduces no functional changes.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
This makes the name a bit more descriptive, and makes it match the rest
of the naming scheme.
This introduces no functional changes.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
This name is more specific to what the template actually does.
This introduces no functional changes.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
`.build` and `.only-schedules` are orthogonal, and I want to use
`.only-schedules` together with `.cross-template` in future, which would
require creating more cross-product templates.
Avoid that by splitting `extends: .build-only-schedules` into
```
extends:
- .build
- .only-schedules
```
Multiple extends are supported by GitLab: https://docs.gitlab.com/ee/ci/yaml/#extends
This introduces no functional changes.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
They were needed for `GMemoryMonitor` support. That was first included
in the gobject-introspection 1.63.2 release and the xdg-desktop-portal
1.5.4 release. The Fedora 33 image we’re using for CI has versions
1.66.1 and 1.8.1.
This should speed up the `installed-tests` CI jobs a bit.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
There are no compiler warnings when building on macOS CI at the moment,
so let’s keep it that way by turning any future ones into errors.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
This is an attempt to fix a persistent error on the macOS CI:
```
Objective-C compiler for the host machine: cc (clang 10.0.0)
Objective-C linker for the host machine: APPLE ld 409.12
WARNING: No include directory found parsing "cc -xobjc -E -v -" output
meson.build:761:2: ERROR: Fatal warnings enabled, aborting
```
For example, seen on https://gitlab.gnome.org/GNOME/glib/-/jobs/1618966.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
This is what’s available in the new Debian Stable, so we can expect it
to be available pretty much everywhere.
Subsequent commits will clean up old workarounds.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Work around a bug in Meson versions < 0.50.0 where
`--wrap-mode=nodownload` would disable the use of fallback subprojects,
even if they’d already been downloaded, with the message:
```
Cross dependency libpcre found: NO (tried pkgconfig and cmake)
Cross dependency libpcre found: NO (tried pkgconfig)
Not looking for a fallback subproject for the dependency libpcre because:
Use of fallbackdependencies is disabled.
meson.build:1998:2: ERROR: Dependency "libpcre" not found, tried pkgconfig
```
This workaround can be dropped in the near future when we bump our Meson
dependency to something less ancient. We have a self-imposed requirement
to be buildable using what’s packaged in Debian Stable, which is
currently Meson 0.49.2 — but the new Debian Stable release is coming
soon, with an updated Meson.
Using `--force-fallback-for=libpcre` would have been a more constrained
workaround, but unfortunately that argument didn’t exist in Meson 0.49.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Helps: #962
This should maintain equivalent functionality, apart from that now you
have to pass `--force-fallback-for libpcre` to `meson configure` in
order to use the subproject; rather than specifying
`-Dinternal_pcre=true` to use the internal copy.
This also fixes#642, as the wrapdb copy of libpcre is version 8.37.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Helps: #962Fixes: #642
It’s a more inclusive name, has the same tab-completion prefix, and is
the default choice for new repositories created locally by git, and on
GitHub and GitLab.
https://sfconservancy.org/news/2020/jun/23/gitbranchname/
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Fixes: #2348
Rather than specifying the default branch explicitly, allow the origin
to choose it. Specify the `--single-branch` option explicitly to make it
clear the CI only needs one branch; this is however already implied by
the `--depth` option so is not strictly necessary.
This will help avoid breakage if gobject-introspection changes its
default branch name in future.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Helps: #2348
Run `systemd-machine-id-setup` when creating the image, so that
`/etc/machine-id` is created with a valid ID. Since systemd isn’t
started when running the CI image with podman/Docker, it’s not created
otherwise. This causes some tests to fail.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
So the tests can access `/var/lib/dbus/machine-id`. This is not a
behaviour change relative to older behaviour on CI.
In future, it might make more sense to revert this commit and change the
CI scripts so they symlink
`/home/user/glib-installed/var/lib/dbus/machine-id` to the system
machine ID; or ensure that `/etc/machine-id` exists on all the CI
machines. That’s too complicated to do right now though.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
It doesn’t seem to be possible to use `only:changes` from a branch
pipeline extending `.only-default` and have it correctly trigger when
`*.sh` or `*.py` files are modified.
We possibly need to convert all our pipelines to be merge-request-only,
but that’s an avenue I’ve been down before and I couldn’t get it to
work. Using
[`workflow:rules`](https://docs.gitlab.com/ee/ci/yaml/README.html#workflowrules)
might fix that, but I don’t have time to rework the entire CI to use
that now.
So in the meantime, move the SH and Py checks into the existing
style-check job so we’re not always spinning up two additional container
instances on every merge request.
See !1743.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Add some basic DAG attributes so that some jobs in the pipeline can
run out of stage order to speed things up a little.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
These jobs (`scan-build` and `valgrind`) take ages to run and rarely
indicate failures, so move them to a weekly schedule, rather than
running them on each branch/MR.
The current schedule is once weekly:
https://gitlab.gnome.org/GNOME/glib/-/pipeline_schedules.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This reduces the bandwidth for downloading xdg-desktop-portal from 2.7MB
to 300KB, on each CI run.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Add a set of new URI parsing and generating functions, including a new
parsed-URI type GUri. Move all the code from gurifuncs.c into guri.c,
reimplementing some of those functions (and
g_string_append_uri_encoded()) in terms of the new code.
Fixes:
https://gitlab.gnome.org/GNOME/glib/issues/110
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
gitlab will drop cmd.exe support with GitLab 13 so I took the opportunity to
add new runners with Windows 2016 and powershell as default.
These runners are tagged with win32-ps instead of win32. The old runners
will be switched off in the coming weeks.
The main difference is that all commands and env expansions use powershell
and Windows 2016 instead of 2012r2.
In cases where performance are critical it can be useful to disable
checks and asserts. GStreamer has those options too, using the same name
and setting them yielding means we can set those options on the main
project (e.g. gst-build) and glib will inherit the same value when built
as subproject.
This will help to debug CI issues that are related to us running in
a container that might have unusual capabilities, mount points,
filesystems etc., such as (probably) #2027, #2028, #2029.
Signed-off-by: Simon McVittie <smcv@collabora.com>
This should speed up CI runs significantly, as the installed tests are
currently running in series.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Newer versions of xdg-desktop-portal need a newer PipeWire, so use the
latest release that doesn't contain this change to avoid having
something else to build.
CI scripts are done using a shell with 'set -e' enabled, but using
'&&' means that the line won't "fail". Run the different commands
sequentially instead.
Spotted by Simon McVittie <smcv@collabora.com>
Closes: #2043
This is required to be able to build the doc. The debian docker is still
pinned to 0.49.2 which ensure we can build with both versions of meson.
Meson 0.52.0 warns about adding -Wall flag manually, we can remove that
because warning_level=1 (the default) option already implies it.
This has the side effect of always rebuilding the doc at each build when
gtk_doc option is enabled (not by default). Most importantly, this will
enable doc check on our CI.
Using the same approach as we have for code style checks (the
`style-check-diff` CI job), check the diff for any banned keywords like
‘TODO’, and also check the commit messages.
The keyword ‘TODO’ is often used by developers to indicate a part of a
commit which needs further work, and hence which shouldn’t yet be merged.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1551
This is a partial revert of commit 595e12b5fb for macOS only, since we
can’t run a VM image on that CI runner, and hence can’t easily
pre-populate it with cached dependencies.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
The CI should not waste resources in downloading subprojects for each
task. It should also not rely on external hosts to be available.
Windows case will be handled in MR #402 by migrating to docker.
We had one before, but the runner machine was too flaky to be useful.
Re-add it now that the Foundation have sorted out a more reliable
machine. (Thanks!)
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1416
The valgrind analysis stage inherits from existing artefacts, so make
sure to reset the permissions on the glib build directory to avoid a new
build failing.
$ meson ${MESON_COMMON_OPTIONS} --werror -Dsystemtap=true -Ddtrace=true -Dfam=true -Dinstalled_tests=true _build
<snip>
PermissionError: [Errno 13] Permission denied: '/builds/GNOME/glib/_build/meson-logs/meson-log.txt'
This doesn’t change how they run, but does split the code out a bit and
mean we can interleave it with comments. Should make it a little less
vile.
Suggested by Emmanuele Bassi; see !1252.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This makes things slightly more extensible in future, but introduces no
functional differences right now.
See https://docs.gitlab.com/ce/ci/yaml/README.html#extends.
Suggested by Jordan Petridis.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Since we added `only: [merge_requests]` for the `style-check-diff` job,
that started running detached pipelines only for merge requests and only
containing that job, and not running the other jobs for merge requests
(only for branches).
That wasn’t the intention.
Follow the guide on
https://docs.gitlab.com/ee/ci/merge_request_pipelines/index.html#excluding-certain-jobs
to ensure that all jobs (including `style-check-diff`) are run for merge
requests.
This means we can no longer unconditionally use
`${CI_MERGE_REQUEST_TARGET_BRANCH_{NAME,SHA}}`, since they are only
defined for jobs which are running against a merge request rather than a
branch. Instead, use some `git rev-list` magic from
https://stackoverflow.com/a/4991675/2931197 to find the newest common
ancestor commit between the detached head that CI is running on, and the
known or likely target branch. Do the style check against the diff
between the newest common ancestor commit and the detached head.
(Note that `${CI_MERGE_REQUEST_TARGET_BRANCH_SHA}` was never actually
defined for any of our pipelines, since it’s only available for CI
pipelines running on merged branches, which is a GitLab Premium
feature. Oops, my bad.)
In order to find the newest common ancestor commit, we need to pull the
upstream remote, since the CI pipeline might be running on a fork of the
main repository where various branches (particularly `master`) are out
of date.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
We can use a template to factor these out and make things a bit more
maintainable. This should introduce no functional changes.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This shouldn’t affect them (since it just adds the `clang-format-7`
package), but it’s good to keep everything on the same version.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
See: #1552
Add a separate CI job which runs memcheck on the unit tests. This is
done as a separate job from the main build, since we don’t want it to
interact with code coverage at all.
Currently, failure of this job is ignored. Issue #333 will eventually
fix that.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #487
The build artifacts from earlier jobs in the pipeline all use the
`_build` directory. When they are copied in to the scan-build job, they
are probably marked as read-only. This means that the `meson scan-build`
run can’t write to `_build/meson-logs/meson-log.txt` and fails.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Use Meson’s support for running scan-build (Clang’s static analyser)
against the build, so that we get static analysis of each pipeline. Add
it in a new pipeline stage, after code coverage, so that we don’t waste
resources on it unless the unit tests pass — a static analysis build
takes perhaps 10× as long as a normal GCC build.
https://mesonbuild.com/howtox.html#use-clang-static-analyzer
Currently, the static analysis results are uploaded as artifacts, but
the job will always succeed (regardless of whether there are any bugs
found in the analysis).
Currently, a large number of reports are outputted by the analyser,
which need to be fixed before we can gate the pipeline on it.
Furthermore, in order to get scan-build to exit with a non-zero status
if any bugs are found, we need to depend on Meson ≥ 0.49.0, which
contains the fix https://github.com/mesonbuild/meson/issues/4334,
allowing us to add the following to .gitlab-ci.yml:
```
variables:
# Exit with a non-zero status if any bugs are found
SCANBUILD: "scan-build --status-bugs"
```
Signed-off-by: Philip Withnall <withnall@endlessm.com>
GitLab can show the results of a CI pipeline if the pipeline generates a
report using the JUnit XML format.
Since Meson provides a machine parseable output for `meson test`, we can
take that and turn it into XML soup.
We want GLib to build correctly with this defined, and for all its tests
to still pass.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1708
This effectively renders those tests useless (since realistically nobody
runs tests locally), but it’s better than every other CI run failing for
unrelated reasons. The idea is that the ‘flaky’ tag can be temporarily
applied to a test while a problem is being investigated or fixed, and
then removed later.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Since we are unable to promote our FreeBSD runner to a shared runner,
we can only enable it in GNOME group. This should avoid problems when
submitting merge requests from forks.
While we can’t add markers to the macro implementations to cause lcov to
ignore them automatically, we can change our lcov configuration to
ignore all calls to them.
See https://github.com/linux-test-project/lcov/issues/44.
This causes all the un-takeable branches and un-reachable assertions to
be ignored by our code coverage, which bumps our statistics:
• Lines: 74.9% → 74.8%
• Functions: 82.3% → 82.3%
• Branches: 53.3% → 64.2%
The rationale is that nobody should be testing programmer error
handling, as g_return_*if_fail() are used to guard against — so it’s not
reasonable to count missed branches like that in code coverage
statistics.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This partially reverts commit bfc362cb68.
The FreeBSD CI runner still seems to be a bit flakey, and appears to be
offline entirely at the moment.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Reviewed-by: nobody
