Add a separate CI job which runs memcheck on the unit tests. This is
done as a separate job from the main build, since we don’t want it to
interact with code coverage at all.
Currently, failure of this job is ignored. Issue #333 will eventually
fix that.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #487
The build artifacts from earlier jobs in the pipeline all use the
`_build` directory. When they are copied in to the scan-build job, they
are probably marked as read-only. This means that the `meson scan-build`
run can’t write to `_build/meson-logs/meson-log.txt` and fails.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Use Meson’s support for running scan-build (Clang’s static analyser)
against the build, so that we get static analysis of each pipeline. Add
it in a new pipeline stage, after code coverage, so that we don’t waste
resources on it unless the unit tests pass — a static analysis build
takes perhaps 10× as long as a normal GCC build.
https://mesonbuild.com/howtox.html#use-clang-static-analyzer
Currently, the static analysis results are uploaded as artifacts, but
the job will always succeed (regardless of whether there are any bugs
found in the analysis).
Currently, a large number of reports are outputted by the analyser,
which need to be fixed before we can gate the pipeline on it.
Furthermore, in order to get scan-build to exit with a non-zero status
if any bugs are found, we need to depend on Meson ≥ 0.49.0, which
contains the fix https://github.com/mesonbuild/meson/issues/4334,
allowing us to add the following to .gitlab-ci.yml:
```
variables:
# Exit with a non-zero status if any bugs are found
SCANBUILD: "scan-build --status-bugs"
```
Signed-off-by: Philip Withnall <withnall@endlessm.com>
GitLab can show the results of a CI pipeline if the pipeline generates a
report using the JUnit XML format.
Since Meson provides a machine parseable output for `meson test`, we can
take that and turn it into XML soup.
We want GLib to build correctly with this defined, and for all its tests
to still pass.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #1708
This effectively renders those tests useless (since realistically nobody
runs tests locally), but it’s better than every other CI run failing for
unrelated reasons. The idea is that the ‘flaky’ tag can be temporarily
applied to a test while a problem is being investigated or fixed, and
then removed later.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Since we are unable to promote our FreeBSD runner to a shared runner,
we can only enable it in GNOME group. This should avoid problems when
submitting merge requests from forks.
While we can’t add markers to the macro implementations to cause lcov to
ignore them automatically, we can change our lcov configuration to
ignore all calls to them.
See https://github.com/linux-test-project/lcov/issues/44.
This causes all the un-takeable branches and un-reachable assertions to
be ignored by our code coverage, which bumps our statistics:
• Lines: 74.9% → 74.8%
• Functions: 82.3% → 82.3%
• Branches: 53.3% → 64.2%
The rationale is that nobody should be testing programmer error
handling, as g_return_*if_fail() are used to guard against — so it’s not
reasonable to count missed branches like that in code coverage
statistics.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This partially reverts commit bfc362cb68.
The FreeBSD CI runner still seems to be a bit flakey, and appears to be
offline entirely at the moment.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Reviewed-by: nobody
It fails because dist-job (correctly) doesn’t build with the code
coverage CFLAGS enabled.
Leave coverage to be generated on master and development branches.
See this pipeline for an example of when it fails:
https://gitlab.gnome.org/GNOME/glib/pipelines/26349
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Similar issue was fixed with commit f929d148, but it's happening again.
Define G_MESSAGES_DEBUG=all when running CI to ensure we won't regress
anymore.
See comment in !151. Using the "--initial" option of lcov we collect
the coverage of all compiled files and merge them later into the final
report. This way we can see which files are built but never executed
by the test suite.
Because the --initial switch also collects files in the ccache directory
we have to point it to the build directory instead, which in turn breaks
--no-external. Instead of using --no-external in the collection step,
filter out any files not in the source tree in the final coverage job
through a path filter.
This requires meson >= 0.47.0 otherwise building the doc fails:
https://github.com/mesonbuild/meson/issues/3379
While at it, no need to to pass --prefix --libdir to meson, other CIs
don't have them.
- Split the download part into a separate script to so docker keeps that
step in cache and avoid redownloading it.
- With API level >= 28 libiconv is not needed anymore because it's part
of Android's libc.
- Generate standalone toolchains to reduce the docker image size. It's
also easier because it doesn't need to pass sysroot args.
- Use clang compiler because gcc is deprecated in this Android NDK and
will be removed in the next release.
Job names in gitlab pipeline view gets truncated to "fedora-meson-..."
for all jobs which is not really useful. All our CIs are using Meson,
and the host distro is not relevant when doing cross builds.
We should be testing latest NDK release but keep using API level 21 to
ensure GLib does not start using newer APIs. We could also later add a
runner for latest API level 28 which includes iconv API in Android's
libc so we don't need GNU libiconv anymore.
The default `when: on_success` means no artifacts are exported when the
tests fail, which is precisely when we want to see the artifacts.
Always export them, and rely on GitLab to garbage collect them as
appropriate (typically after 30 days, I think).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
