ci: Limit depth of fetch for initial code checks

Rather than fetching the entire repository history, limit the fetch
depth to commits in the last 4 weeks. If a branch was branched longer
than 4 weeks ago, it seems reasonable to require it to be rebased before
it can be reviewed.

At the current rate of development, this reduces the bandwidth needed
for git pulls in the initial code checks from 73MB to 9MB.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
This commit is contained in:
Philip Withnall 2020-07-25 00:36:33 +01:00
parent 5c09c9e977
commit 1378a40734
2 changed files with 29 additions and 2 deletions

View File

@ -2,11 +2,17 @@
set -e set -e
ancestor_horizon=28 # days (4 weeks)
# We need to add a new remote for the upstream target branch, since this script # We need to add a new remote for the upstream target branch, since this script
# could be running in a personal fork of the repository which has out of date # could be running in a personal fork of the repository which has out of date
# branches. # branches.
#
# Limit the fetch to a certain date horizon to limit the amount of data we get.
# If the branch was forked from origin/master before this horizon, it should
# probably be rebased.
git remote add upstream https://gitlab.gnome.org/GNOME/glib.git git remote add upstream https://gitlab.gnome.org/GNOME/glib.git
git fetch upstream git fetch --shallow-since="$(date --date="${ancestor_horizon} days ago" +%Y-%m-%d)" upstream
# Work out the newest common ancestor between the detached HEAD that this CI job # Work out the newest common ancestor between the detached HEAD that this CI job
# has checked out, and the upstream target branch (which will typically be # has checked out, and the upstream target branch (which will typically be
@ -14,4 +20,11 @@ git fetch upstream
# `${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}` is only defined if were running in # `${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}` is only defined if were running in
# a merge request pipeline; fall back to `${CI_DEFAULT_BRANCH}` otherwise. # a merge request pipeline; fall back to `${CI_DEFAULT_BRANCH}` otherwise.
newest_common_ancestor_sha=$(diff --old-line-format='' --new-line-format='' <(git rev-list --first-parent "upstream/${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-${CI_DEFAULT_BRANCH}}") <(git rev-list --first-parent HEAD) | head -1) newest_common_ancestor_sha=$(diff --old-line-format='' --new-line-format='' <(git rev-list --first-parent "upstream/${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-${CI_DEFAULT_BRANCH}}") <(git rev-list --first-parent HEAD) | head -1)
if [ -z "${newest_common_ancestor_sha}" ]; then
echo "Couldnt find common ancestor with upstream master. This typically"
echo "happens if you branched from master a long time ago. Please update"
echo "your clone, rebase, and re-push your branch."
exit 1
fi
./.gitlab-ci/check-todos.py "${newest_common_ancestor_sha}" ./.gitlab-ci/check-todos.py "${newest_common_ancestor_sha}"

View File

@ -2,21 +2,35 @@
set -e set -e
ancestor_horizon=28 # days (4 weeks)
# Wrap everything in a subshell so we can propagate the exit status. # Wrap everything in a subshell so we can propagate the exit status.
( (
# We need to add a new remote for the upstream target branch, since this script # We need to add a new remote for the upstream target branch, since this script
# could be running in a personal fork of the repository which has out of date # could be running in a personal fork of the repository which has out of date
# branches. # branches.
#
# Limit the fetch to a certain date horizon to limit the amount of data we get.
# If the branch was forked from origin/master before this horizon, it should
# probably be rebased.
git remote add upstream https://gitlab.gnome.org/GNOME/glib.git git remote add upstream https://gitlab.gnome.org/GNOME/glib.git
git fetch upstream git fetch --shallow-since="$(date --date="${ancestor_horizon} days ago" +%Y-%m-%d)" upstream
# Work out the newest common ancestor between the detached HEAD that this CI job # Work out the newest common ancestor between the detached HEAD that this CI job
# has checked out, and the upstream target branch (which will typically be # has checked out, and the upstream target branch (which will typically be
# `upstream/master` or `upstream/glib-2-62`). # `upstream/master` or `upstream/glib-2-62`).
#
# `${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}` is only defined if were running in # `${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}` is only defined if were running in
# a merge request pipeline; fall back to `${CI_DEFAULT_BRANCH}` otherwise. # a merge request pipeline; fall back to `${CI_DEFAULT_BRANCH}` otherwise.
newest_common_ancestor_sha=$(diff --old-line-format='' --new-line-format='' <(git rev-list --first-parent "upstream/${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-${CI_DEFAULT_BRANCH}}") <(git rev-list --first-parent HEAD) | head -1) newest_common_ancestor_sha=$(diff --old-line-format='' --new-line-format='' <(git rev-list --first-parent "upstream/${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-${CI_DEFAULT_BRANCH}}") <(git rev-list --first-parent HEAD) | head -1)
if [ -z "${newest_common_ancestor_sha}" ]; then
echo "Couldnt find common ancestor with upstream master. This typically"
echo "happens if you branched from master a long time ago. Please update"
echo "your clone, rebase, and re-push your branch."
exit 1
fi
git diff -U0 --no-color "${newest_common_ancestor_sha}" | ./clang-format-diff.py -binary "clang-format-7" -p1 git diff -U0 --no-color "${newest_common_ancestor_sha}" | ./clang-format-diff.py -binary "clang-format-7" -p1
) )