Fedora 29 is no longer available, and I need to rebuild these images so
libpcre is pre-downloaded as a subproject.
Update them to Fedora 31, which is what we use in `fedora.Dockerfile`.
This should allow some blobs to be shared in the container repository.
This isn’t the latest Fedora release, but I don’t want to go through the
hassle of updating all the CI images to F33 right now in addition to
updating the subproject caches.
`python-unversioned-command` is needed because the Android NDK calls
`python` without a version number. F29 must have installed this already.
The Android NDK setup appears to be OK with Python 3.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Helps: #962
It’s a more inclusive name, has the same tab-completion prefix, and is
the default choice for new repositories created locally by git, and on
GitHub and GitLab.
https://sfconservancy.org/news/2020/jun/23/gitbranchname/
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Fixes: #2348
Update several links to allow the remote to use its configured default
branch name, rather than specifying `master` as the default branch name.
This will help avoid breakage if any of these projects rename their
default branch in the future.
Fix a few of the links where they were hitting redirects or had moved.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Helps: #2348
Run `systemd-machine-id-setup` when creating the image, so that
`/etc/machine-id` is created with a valid ID. Since systemd isn’t
started when running the CI image with podman/Docker, it’s not created
otherwise. This causes some tests to fail.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Include the size of the `machine-id` file, but not the value itself as
that is sensitive for non-throwaway machines. What’s most useful for
debugging CI problems is knowing whether, and where, the `machine-id` is
set.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
and so checkout only the HEAD of the branch.
This leads to not being able to find an ancestor since there is only one
new commit. I presume that when it worked, it was because of specific
settings of some users.
Now we fetch (with the same depth as the target branch) the source branch
and use it to compare sha1 and find the common ancestor.
Closes#2292
Signed-off-by: Frederic Martinsons <frederic.martinsons@sigfox.com>
The version of `black` on the CI server wanted these changes. Make them
to keep the `style-check-diff` CI job from constantly failing.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
Spotted by shellcheck (warning SC1117): `\e` is not an actual escape
sequence, so it’s interpreted as `\\e`. Best make that explicit.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
This commit is the unmodified results of running
```
black $(git ls-files '*.py')
```
with black version 19.10b0. See #2046.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
It was disabled in !875 because lcov didn't support the new coverage
format produced by gcc9+. The latest lcov release in MSYS2 supports
it again, so re-enable everything.
lcov now writes native Windows paths to its output so adjust the path
fixup script to handle those.
This isn’t used in the build at all, it’s just a copy of the model file
which is uploaded in our Coverity configuration. This should be kept up
to date with changes in the file on the Coverity servers.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Rather than fetching the entire repository history, limit the fetch
depth to commits in the last 4 weeks. If a branch was branched longer
than 4 weeks ago, it seems reasonable to require it to be rebased before
it can be reviewed.
At the current rate of development, this reduces the bandwidth needed
for git pulls in the initial code checks from 73MB to 9MB.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Use the depth= argument from Meson 0.52 to limit the clone depth of
subprojects to 1. This should make the CI images a little smaller, and
reduce the bandwidth required to build them (although that’s not so
important because it only happens once every few months).
Similarly, only clone to a depth of 1 in `cache-subprojects.sh`, which
is run once every few months.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This was mostly machine generated with the following command:
```
codespell \
--builtin clear,rare,usage \
--skip './po/*' --skip './.git/*' --skip './NEWS*' \
--write-changes .
```
using the latest git version of `codespell` as per [these
instructions](https://github.com/codespell-project/codespell#user-content-updating).
Then I manually checked each change using `git add -p`, made a few
manual fixups and dropped a load of incorrect changes.
There are still some outdated or loaded terms used in GLib, mostly to do
with git branch terminology. They will need to be changed later as part
of a wider migration of git terminology.
If I’ve missed anything, please file an issue!
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This should be generated based on the available SDK versions and the
iPhone version you want to target, but that's something we can do
when adding macOS and iOS CI.
The gtk-doc tests are hardcoded by Meson to output as `glib /
gio-doc-check`, `glib / gobject-doc-check`, etc., without an explicit
project name and suite. This causes the following exception in the
report parser:
```
Traceback (most recent call last):
File ".gitlab-ci/meson-junit-report.py", line 50, in <module>
(project_name, suite_name) = full_suite.split(':')
ValueError: not enough values to unpack (expected 2, got 1)
```
Signed-off-by: Philip Withnall <withnall@endlessm.com>
It’s needed to provide the `pip3` executable, which the `Dockerfile`
later uses.
Follow-up to !1464.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
pip in MSYS2 seems to install scripts into $USERPROFILE instead of $HOME
which means the MSYS2 meson, which is newer, wins. Make sure $USERPROFILE
is in PATH as well.
gitlab will drop cmd.exe support with GitLab 13 so I took the opportunity to
add new runners with Windows 2016 and powershell as default.
These runners are tagged with win32-ps instead of win32. The old runners
will be switched off in the coming weeks.
The main difference is that all commands and env expansions use powershell
and Windows 2016 instead of 2012r2.
This will help to debug CI issues that are related to us running in
a container that might have unusual capabilities, mount points,
filesystems etc., such as (probably) #2027, #2028, #2029.
Signed-off-by: Simon McVittie <smcv@collabora.com>
I haven’t tested any of them. This is entirely mechanical. I used
shellcheck 0.7.0 with default options.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Spotted by Daniel Stone: the addition of the `echo` commands in commit
65541f1ad meant that the exit status from `clang-format-diff.py` was
being lost.
The incorrect use of `set +e` rather than `set -e` meant that
intermediate commands could fail without the failure being noticed.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
If podman is used, as is usually the case on a Fedora Workstation
installation, make sure not to use "sudo" as that's not needed.
Also ask podman's backend (buildah) to create Docker compatible images
through an environment variable rather than a command-line argument.
See https://gitlab.gnome.org/GNOME/glib/merge_requests/1255
This is required to be able to build the doc. The debian docker is still
pinned to 0.49.2 which ensure we can build with both versions of meson.
Meson 0.52.0 warns about adding -Wall flag manually, we can remove that
because warning_level=1 (the default) option already implies it.
Using the same approach as we have for code style checks (the
`style-check-diff` CI job), check the diff for any banned keywords like
‘TODO’, and also check the commit messages.
The keyword ‘TODO’ is often used by developers to indicate a part of a
commit which needs further work, and hence which shouldn’t yet be merged.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #1551
This doesn’t change how they run, but does split the code out a bit and
mean we can interleave it with comments. Should make it a little less
vile.
Suggested by Emmanuele Bassi; see !1252.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
The dockerd instance on some (but not all) GitLab CI runners doesn’t
like the OCI output from the version of podman on Fedora 31, which is
causing a lot of spurious CI failures.
If whoever’s running `run-docker.sh` is using podman to emulate docker,
it needs to be told to output in Docker format, not OCI format.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Tools like this should be configurable in a cross or native file. In
particular, if we are cross-compiling (with an executable wrapper like
qemu-arm), the build system ld is not necessarily able to manipulate
host system objects.
Signed-off-by: Simon McVittie <smcv@collabora.com>
When running tests under valgrind, the valgrind summary is printed in
stderr, and the TAP output is printed in stdout. The valgrind summary is
useful to include in the GitLab test report, so append it to the
textual failure information for failed tests.
I can’t find a better XML element in the [JUnit
schema](https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd)
for representing it.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Helps: #487
Add a separate CI job which runs memcheck on the unit tests. This is
done as a separate job from the main build, since we don’t want it to
interact with code coverage at all.
Currently, failure of this job is ignored. Issue #333 will eventually
fix that.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Fixes: #487
lconv 1.13 only supports gcc <=7 and lcov 1.14 supports <=8.
msys2 was just updated to gcc9, so this wont help with coverage support, but
it's a start I guess.
The current Debian stable CI image ships with Python 3.5, so the Meson
log to JUnit report script is failing because it's using an API addition
present in Python 3.6 or later.
Since it's just a cosmetic option for the time stamp, we can get rid of
it.
Fedora 27 was EOL'ed on November 2018.
We move to Fedora 28 because the Android NDK requires Python 2 and
probably other things, and bumping to Fedora 29 is going to be more
painful.
The Fedora image we use contains MinGW bits that ought to go into
their own Docker container. This avoids having a massive Docker image
that gloms everything and is harder to update.
While we're splitting off, we can also update to Fedora 29, as we can
rely on Fedora packagers doing their job and ensuring that the MinGW
cross-compilation toolchain still works.
The Fedora image we use contains Android bits that ought to go into
their own Docker container. This avoids having a massive Docker image
that gloms everything and is harder to update.
We reuse the same Docker image we used for Fedora, to avoid regressing.
GitLab can show the results of a CI pipeline if the pipeline generates a
report using the JUnit XML format.
Since Meson provides a machine parseable output for `meson test`, we can
take that and turn it into XML soup.
This effectively renders those tests useless (since realistically nobody
runs tests locally), but it’s better than every other CI run failing for
unrelated reasons. The idea is that the ‘flaky’ tag can be temporarily
applied to a test while a problem is being investigated or fixed, and
then removed later.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
While we can’t add markers to the macro implementations to cause lcov to
ignore them automatically, we can change our lcov configuration to
ignore all calls to them.
See https://github.com/linux-test-project/lcov/issues/44.
This causes all the un-takeable branches and un-reachable assertions to
be ignored by our code coverage, which bumps our statistics:
• Lines: 74.9% → 74.8%
• Functions: 82.3% → 82.3%
• Branches: 53.3% → 64.2%
The rationale is that nobody should be testing programmer error
handling, as g_return_*if_fail() are used to guard against — so it’s not
reasonable to count missed branches like that in code coverage
statistics.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
The gresource code uses libelf if available but that also depends on mmap but isn't
guarded with HAVE_MMAP. This can make the build fail under MSYS2 where a mingw version
of libelf exists but there is no mmap.
Instead of guarting the libelf code with HAVE_LIBELF add a new macro named USE_LIBELF
which is only defined if libelf and mmap support are available.
Also install the mingw libelf version for CI so we catch similar errors in the future.
See comment in !151. Using the "--initial" option of lcov we collect
the coverage of all compiled files and merge them later into the final
report. This way we can see which files are built but never executed
by the test suite.
Because the --initial switch also collects files in the ccache directory
we have to point it to the build directory instead, which in turn breaks
--no-external. Instead of using --no-external in the collection step,
filter out any files not in the source tree in the final coverage job
through a path filter.
- Split the download part into a separate script to so docker keeps that
step in cache and avoid redownloading it.
- With API level >= 28 libiconv is not needed anymore because it's part
of Android's libc.
- Generate standalone toolchains to reduce the docker image size. It's
also easier because it doesn't need to pass sysroot args.
- Use clang compiler because gcc is deprecated in this Android NDK and
will be removed in the next release.
We should be testing latest NDK release but keep using API level 21 to
ensure GLib does not start using newer APIs. We could also later add a
runner for latest API level 28 which includes iconv API in Android's
libc so we don't need GNU libiconv anymore.
This adds to the CI a cross build for Android NDK r16 API 21 (because
thats what GStreamer currently use) for arm64.
GNU iconv must be built manually into our docker image because Android
NDK doesn't seems to ship it. The latest NDK r17 API 28 has iconv.h but
iconv_open() symbol isn't found by the linker. Looks like broken NDK.
libffi also needs to be built manually because the meson subproject
doesn't support building for Android platform. It needs a recent RC
release because latest stable release is 4 years old and fails to
build.
Fixes#1385.
Use lcov for both Fedora and MSYS2 to create coverage reports and add a second
ci stage which merges the coverage and creates a html report using genhtml.
In the final stage, which is only run on master, the result is published on
gitlab pages.
https://bugzilla.gnome.org/show_bug.cgi?id=795636
Fix various warnings regarding unused variables, duplicated
branches etc by adjusting the ifdeffery and some missing casts.
gnulib triggers -Wduplicated-branches in one of the copied files,
disable as that just makes updating the code harder.
The warning indicating missing features are made none fatal through
pragmas. They still show but don't abort the build.
https://bugzilla.gnome.org/show_bug.cgi?id=793729
This builds glib using meson/ninja/ccache with mingw-w64 on a Windows
machine.
The CI scripts expect a gitlab runner to exist with the "win32" tag
which uses the default "cmd" shell by default.
Before running the tests pacman is invoked to update the system
(potentially including bash etc, thus the extra step)
Then a login shell is started with CHERE_INVOKING to not change the
cwd and finally the test script is executed.
https://bugzilla.gnome.org/show_bug.cgi?id=793729
The base Fedora image we use for our CI does not always have an updated
Meson package to fit our requirements. Let's install Meson using
Python's pip instead.
We're mostly interested into building and testing everything that gets
pushed to the repository — including merge requests.
When pushing tags, though, we should assume we're spinning a release, so
let's run the dist target, and store the tarball, and the generated
documentation while we're at it, as artifacts on GitLab.
The Dockerfile for the image used for the build is included in tree, and
published on Docker Hub. Using a custom image allows us to avoid the
costly "download and install build dependencies" phase, as well as
controlling the environment a little bit better.
https://bugzilla.gnome.org/show_bug.cgi?id=793635
