2
1
forked from adamm/autogits

197 Commits
0.0.1 ... main

Author SHA256 Message Date
96e1c26600 Fix crash when review.User is nil 2025-04-10 15:29:02 +02:00
9d9964df11 Build in :PR: sub project as wanted for SLFO
Instead of doing it a home project which won't scale.
2025-04-10 13:47:58 +02:00
881fad36a0 Initial support of QA subproject setup
- Nothing handed over to external scripts yet
- Not agreed file format in _obs_staging (YAML!)
- No build monitoring
2025-04-08 17:02:03 +02:00
29906e22d2 Complete project meta
description is not optional

define releasetarget element
2025-04-08 17:02:03 +02:00
52a5cdea94 group-review: fix typo 2025-04-07 14:24:48 +02:00
d3f1b36676 Use "-gitea-url" instead of "-gitea-host" or simiar
This allows to use another schema than https:// to connect to Gitea
2025-04-07 14:20:26 +02:00
5ea5f05b02 common: reviews fix 2025-04-07 09:47:07 +02:00
5877081280 whitespace 2025-04-06 17:32:16 +02:00
c4ce974ddf group-review: fixes 2025-04-05 23:45:40 +02:00
65c718e73b group-review: move config 2025-04-04 18:07:57 +02:00
a8e6c175c0 remove obsolete per-executable go.mod 2025-04-04 13:56:52 +02:00
044416cd2a Merge branch 'main' of c3:gitea_test/autogits 2025-04-04 13:55:54 +02:00
009cc88d54 Merge remote-tracking branch 'gitea/main' 2025-04-04 13:06:28 +02:00
da1f4f4fa0 fix mocks 2025-04-04 13:05:51 +02:00
cfad21e1a3 Set review state only after the end of the build
Instead using normal comments to inform users of the build project
or in case the used source of the pull request has changed
and the build project has been updated.
2025-04-04 10:09:00 +02:00
5eb54d40e0 Define "unknown" build state 2025-04-04 10:09:00 +02:00
80ff036acb group-review: rerequest reviwes missing group review
If user is member of group but doesn't review correctly, request
their review again.
2025-04-04 00:17:55 +02:00
Jan Zerebecki
2ed4f0d05f Build all modules and in obs directly from this repo
Build each go module in a subpackage.
2025-04-03 22:40:04 +02:00
Jan Zerebecki
23ed9b830d Merge all go.mod into a top level one 2025-04-03 22:38:31 +02:00
Jan Zerebecki
4604aaeeba Rename bots-common to common
to make it match the name it is imported as
2025-04-03 22:38:28 +02:00
Jan Zerebecki
2dfe973c51 Generate group-review/go.sum
it wasn't commited before
2025-04-02 18:47:05 +02:00
b7625cd4c4 Fix cloning for src.suse.de instance 2025-04-02 14:15:06 +02:00
12e7a071d9 whitespace only changes 2025-04-02 11:39:52 +02:00
6409741a12 Initial support for SSH based authentification
Moved all HTTP codes to the ObsRequest method.

Make use of the authorization cookie, but only store it in memory.
Should be fine for a constant running bot to do the authorization once
on startup.
2025-04-02 11:14:02 +02:00
78eb9f11e5 Extend regexp to match orgs and projects include - and _ chars 2025-04-01 09:40:39 +02:00
c28f28e852 Merge branch 'main' of c3:gitea_test/autogits 2025-03-28 16:34:10 +01:00
72270c57ed fixes to importer 2025-03-28 16:33:59 +01:00
5d6dc75400 fix maintainership writer 2025-03-28 16:09:36 +01:00
20b02d903c pr: move config to project 2025-03-26 23:20:26 +01:00
58dc4927c2 Flexible OBS api and www endpoints
Allow the endoinds to be configurable
2025-03-25 12:44:17 +01:00
ce48cbee72 Add ability to set build location 2025-03-25 11:52:15 +01:00
3bd179bee1 wip 2025-03-24 00:23:11 +01:00
940e5be2c1 Migrate to prjgit based config
config now only has reference to org or prjgits and the rest
is defined in the "workflow.config" in the prjgit itself. This
allows the config to be updated in the project.
2025-03-23 16:33:06 +01:00
4a4113aad7 wip 2025-03-21 16:39:50 +01:00
3ee939db1d logging 2025-03-18 17:14:19 +01:00
00f4e11f02 fixes when scmsync for packages already there 2025-03-18 16:55:30 +01:00
635bdd0f50 Migrate PR related to to common area 2025-03-18 13:08:49 +01:00
82f7a186a9 Only cherry-pick non-merges 2025-03-17 17:58:51 +01:00
030fa43404 Bug fixes in importer 2025-03-17 17:34:11 +01:00
2ad9f6c179 push changes if link changed 2025-03-16 20:29:42 +01:00
80952913c9 prjgit based config 2025-03-16 20:29:15 +01:00
1ce38c9de2 wip links 2025-03-13 18:44:38 +01:00
bbb721c6fa wip 2025-03-13 08:46:21 +01:00
a50d238715 wip 2025-03-12 23:13:13 +01:00
463a6e3236 wip 2025-03-12 19:13:22 +01:00
91ecf88a38 fixes for updates 2025-03-12 14:37:38 +01:00
4f9a99d232 link 2025-03-12 00:28:55 +01:00
02d3a2e159 fixes, etc. 2025-03-11 19:46:40 +01:00
03370871c4 do not clone if already created 2025-03-11 16:04:01 +01:00
1f4e1ac35e update build project if needed when build pending 2025-03-10 13:44:59 +01:00
debbee17eb wip 2025-03-07 17:40:59 +01:00
c63a56bc4e consistent usage parameters 2025-03-07 16:12:48 +01:00
568346ce3d wip 2025-03-04 22:38:17 +01:00
a010618764 do not demand credentials to print help 2025-03-04 18:37:08 +01:00
a80e04065f list maintainers correctly 2025-03-04 18:36:41 +01:00
72c2967d1f fixes 2025-03-03 15:03:43 +01:00
1cacb914b4 maintainer set finished 2025-03-01 00:26:08 +01:00
517ecbb68a wip 2025-02-28 17:36:35 +01:00
e1313105d1 remove rpm and re-org user queries 2025-02-26 18:10:42 +01:00
5b84f9d5ce maintianer info 2025-02-24 18:55:37 +01:00
7c254047a8 devel-importer: no OBS req header print in debug mode 2025-02-24 13:50:24 +01:00
fffdce2c58 better recovery in unexpected situations 2025-02-24 13:11:54 +01:00
4014747712 always skip LFS 2025-02-24 13:11:38 +01:00
b49df4845a add maintainership fetching 2025-02-24 00:12:28 +01:00
9bac2e924c devel-importer update 2025-02-21 17:18:37 +01:00
ee6d704e1e refactor 2025-02-20 19:25:36 +01:00
bfeaed40d5 remove legacy 2025-02-20 19:17:23 +01:00
4dd864c7b6 move maintainership to common/ 2025-02-20 12:20:14 +01:00
205741dde1 typos 2025-02-19 11:46:25 +01:00
a5acc1e34e yolo 2025-02-19 10:51:49 +01:00
fc2dbab782 wip 2025-02-19 00:06:54 +01:00
9236fa3ff1 check stale reviews 2025-02-18 18:14:17 +01:00
334fe5553e tests 2025-02-18 17:52:36 +01:00
9418b33c6c fix 2025-02-18 17:42:55 +01:00
7a8c84d1a6 check status 2025-02-18 12:40:32 +01:00
367d606870 wip 2025-02-17 17:49:02 +01:00
682397975f wip 2025-02-16 22:40:10 +01:00
b4a1c5dc01 docs 2025-02-15 13:30:58 +01:00
1c38c2105b wip 2025-02-14 17:13:51 +01:00
072d7b4825 workflow-direct: ignore non-sha1 repos 2025-02-13 16:42:59 +01:00
9ecda0c58b obs-staging-bot: log polling cycles 2025-02-11 16:26:29 +01:00
8c2cc51a3c obs-staging-bot: closed requests should no longer need review 2025-02-11 16:22:00 +01:00
2f38e559d1 fix obs staging bot 2025-02-10 15:16:48 +01:00
61d9359ce3 remove init() 2025-02-10 14:11:14 +01:00
d46ca05346 Fix against new git interfaces 2025-02-10 13:50:25 +01:00
a84d55c858 rename interfaces 2025-02-06 19:18:09 +01:00
2cd7307291 remove debug code 2025-02-06 18:56:02 +01:00
efde2fad69 reviewers fix in tests 2025-02-06 17:17:06 +01:00
e537e5d00c wip 2025-02-05 18:30:08 +01:00
adffc67ca0 typos 2025-02-05 14:44:38 +01:00
f0b184f4c3 move to reviewers 2025-02-05 14:44:22 +01:00
656a3bacdf add reviewer parsing 2025-02-05 14:43:38 +01:00
c0c467d72b merge 2025-02-04 17:44:49 +01:00
dbee0e8bd3 submodules 2025-02-04 14:24:38 +01:00
c7723fce2d wip 2025-02-03 18:15:01 +01:00
12a641c86f wip 2025-02-02 21:07:51 +01:00
73e817d408 wip 2025-01-31 17:39:46 +01:00
6aa53bdf25 git.status with rename support 2025-01-30 12:45:56 +01:00
d5dbb37e18 git.status 2025-01-29 17:29:09 +01:00
5108019db0 wip 2025-01-28 10:52:54 +01:00
6fc0607823 wip 2025-01-27 17:43:50 +01:00
c1df08dc59 {wip} 2025-01-21 17:20:00 +01:00
92747f0913 {wip} 2025-01-21 17:19:18 +01:00
f77e35731c workflow-direct: fix building 2025-01-21 16:24:50 +01:00
b9e70132ae workflow-pr: print errors from check handlers 2025-01-17 14:46:53 +01:00
245181ad28 workflow-pr: don't recreate branches on every check
Check if branch exists and if it matches the PR already created.
If yes, then nothing needs to be updated.
2025-01-16 16:36:53 +01:00
fbaeddfcd8 add support for maintainership directories 2025-01-15 00:46:03 +01:00
e63a450c5d add review checks 2025-01-11 21:37:59 +01:00
8ab35475fc review stuff 2025-01-03 00:46:40 +01:00
69776dc5dc Add ReviewSet.ConsistentSet() check 2025-01-02 14:44:31 +01:00
cfe15a0551 Add ReviewSet.ConsistentSet() check 2025-01-02 13:46:59 +01:00
888582a74a wip 2024-12-18 17:30:00 +01:00
72d5f64f90 wip 2024-12-17 23:33:43 +01:00
fe2a577b3b wip 2024-12-17 18:19:04 +01:00
ac6fb96534 wip 2024-12-16 18:12:54 +01:00
f6bd0c10c0 . 2024-12-16 15:50:33 +01:00
50aab4c662 wip 2024-12-16 08:15:49 +01:00
8c6180a8cf . 2024-12-15 13:00:20 +01:00
044241c71e . 2024-12-13 15:28:38 +01:00
e057cdf0d3 workflow-pr: review processing 2024-12-12 19:16:32 +01:00
7ccbd1deb2 wip 2024-12-11 14:41:51 +01:00
68ba45ca9c wip 2024-12-11 01:12:59 +01:00
a7d81d6013 wip 2024-12-10 19:03:54 +01:00
5f00b10f35 wip 2024-12-09 18:20:56 +01:00
7433ac1d3a wip 2024-12-09 00:39:55 +01:00
db766bacc3 commit: PR desc parser and writer 2024-12-07 14:35:34 +01:00
77751ecc46 workflow-pr: wip 2024-12-05 18:38:35 +01:00
a025328fef wip 2024-12-04 08:55:40 +01:00
0c866e8f89 worflow-pr: wip 2024-12-02 10:26:51 +01:00
2d12899da5 workflow-pr: renamed files 2024-12-01 11:36:26 +01:00
f4462190c9 workflow-pr: maintainership API change 2024-11-29 17:33:01 +01:00
7342dc42e9 workflow-pr: refactor 2024-11-29 12:49:11 +01:00
60c0a118c9 workflow-pr: maintainernship function signature change 2024-11-28 18:16:14 +01:00
cf101ef3f0 workflow-pr: maintainership doc update
Sync with https://github.com/openSUSE/openSUSE-git/issues/55

Update README.md with syntax
2024-11-28 17:25:32 +01:00
0331346025 workflow-pr: maintainership parsing 2024-11-28 17:10:26 +01:00
21f7da2257 workflow-pr: maintainership code 2024-11-28 00:15:37 +01:00
2916ec8da5 workflow-pr: tests 2024-11-27 17:50:55 +01:00
2bc9830a7a workflow-pr: tests 2024-11-27 16:13:37 +01:00
f281986c8f workflow-pr: tests 2024-11-26 17:21:17 +01:00
e56f444960 workflow-pr: more unit tests 2024-11-25 17:02:48 +01:00
b96c4d26ca workflow-pr: small refactor 2024-11-14 18:02:11 +01:00
2949e23b11 workflow-pr: test fixes 2024-11-13 16:21:51 +01:00
1d7d0a7b43 workflow-pr: more tests 2024-11-13 16:18:27 +01:00
e8e51e21ca more tests 2024-11-12 14:26:36 +01:00
dc96392b40 pr: more unit tests 2024-11-11 15:52:34 +01:00
c757b50c65 wip 2024-11-10 23:19:23 +01:00
0a7978569e refactor 2024-11-08 16:36:05 +01:00
463e3e198b refactor 2024-11-08 15:05:09 +01:00
8bedcc5195 wip 2024-11-07 18:25:35 +01:00
0d9451e92c {wip} - unit tests
`git submodule status` will display current state, which will be
overwritten by checkout submodule. Solution is to `submodule deinit`
before looking at submodule status.
2024-11-04 15:13:22 +01:00
a230c2aa52 {wip} tests 2024-11-03 22:21:57 +01:00
0f6cb392d6 wip tests 2024-10-30 16:55:51 +01:00
48a889b353 obs-staging-bot: cleanup modules 2024-10-29 15:37:57 +01:00
a672bb85fb wip 2024-10-29 15:36:20 +01:00
6ecc4ecb3a wip 2024-10-01 17:21:28 +02:00
881cba862f common: refactor IsReviewed() 2024-10-01 12:18:37 +02:00
77bdf7649a common: parse all PR for associated PrjGit PR
we may have more than 1 page of PR to parse
2024-10-01 12:03:50 +02:00
a0a79dcf4d pr: add reviewers to PR workflow 2024-09-30 16:19:40 +02:00
3d7336a3a0 doc formatting 2024-09-30 15:37:43 +02:00
bbdd9eb0be docs 2024-09-30 15:36:54 +02:00
c48ff699f4 docs 2024-09-30 15:35:46 +02:00
27014958be docs 2024-09-30 15:28:25 +02:00
5027e98c04 pr: processing checks 2024-09-30 15:09:45 +02:00
e2498afc4d pr: branch forwarding on check 2024-09-30 10:25:08 +02:00
7234811edc pr: wip 2024-09-29 23:11:51 +02:00
4692cfbe6f workflow-pr: app name 2024-09-29 15:32:32 +02:00
464e807747 pr: WIP 2024-09-27 17:58:09 +02:00
76f2ae8aec common: find pull requests by source and target 2024-09-27 17:55:49 +02:00
a0b65ea8f4 common: print stacktrace when recovering from panic 2024-09-27 17:54:31 +02:00
5de077610c direct: use CloneURL instead of SSH
Make sure that we use public CloneURL instead of SSH for submodule
OBS doesn't fetch submodules with SSH schema
2024-09-25 16:33:07 +02:00
d7bbe5695c devel-importer: more fixes 2024-09-19 19:00:56 +02:00
86df1921e0 devel-importer: handle history rewrite
Imports can have history rewritten because email addresses
can change in OBS and are not recorded as in git commits. This
can be handled via comparing Tree objects and rebasing
new changes ontop.
2024-09-18 17:17:24 +02:00
9de8cf698f . 2024-09-18 13:52:50 +02:00
c955811373 devel-importer: handle cases where remotes or factory branch not main 2024-09-18 13:51:52 +02:00
530318a35b add missing file 2024-09-18 12:23:32 +02:00
798f96e364 devel-importer: adapt for scmsync packages 2024-09-18 11:47:42 +02:00
11bf2aafcd remove temp dir created if using another 2024-09-17 15:16:04 +02:00
3a2c590158 Rename variables 2024-09-17 15:06:29 +02:00
a47d217ab3 [devel-importer] configurable import location 2024-09-17 10:56:31 +02:00
6b40bf7bbf devel-importer: migrate logging to log module
also remove webhook stuff, since we use RabbitMQ now
2024-09-16 16:05:46 +02:00
d36c0c407f devel-importer: use common.GitExec() 2024-09-16 13:10:25 +02:00
e71e6f04e8 rename things 2024-09-13 14:58:10 +02:00
7e663964ee Compiles, ship it!
ported pr-review to rabbitmq
2024-09-13 13:41:40 +02:00
7940a8cc86 refactor 2024-09-12 17:57:09 +02:00
edab8aa9dd fix branch handling in repo check 2024-09-12 17:40:51 +02:00
a552f751f0 refactor 2024-09-12 16:40:43 +02:00
b7ec9a9ffb Handle case when branch not exist
Handle default branch name in push and branch create handlers
Don't panic in this case in case the project has multiple configs
2024-09-12 16:25:22 +02:00
b0b39726b8 Do not remove git work directory in debug mode 2024-09-12 14:34:05 +02:00
06228c58f3 remove duplicate RabbitMQ listen topics 2024-09-12 14:30:19 +02:00
d828467d25 fixes 2024-09-12 14:15:59 +02:00
dd316e20b7 direct: change to src as app, instead of gitea 2024-09-11 19:51:49 +02:00
937664dfba enable debug-only rabbitmq webhook hander 2024-09-11 18:56:48 +02:00
4c8eae5e7c simplify error handling 2024-09-11 18:50:49 +02:00
c61d648294 auth token check 2024-09-11 17:51:56 +02:00
630803246c move After= to correct section 2024-09-11 16:32:26 +02:00
69b9e41129 spec file changes 2024-09-11 16:15:38 +02:00
f8ad932e33 spec file fix 2024-09-11 14:08:36 +02:00
1123 changed files with 8588 additions and 3532 deletions

21
.gitattributes vendored Normal file
View File

@@ -0,0 +1,21 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.bsp filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.gem filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.jar filter=lfs diff=lfs merge=lfs -text
*.lz filter=lfs diff=lfs merge=lfs -text
*.lzma filter=lfs diff=lfs merge=lfs -text
*.oxt filter=lfs diff=lfs merge=lfs -text
*.pdf filter=lfs diff=lfs merge=lfs -text
*.png filter=lfs diff=lfs merge=lfs -text
*.rpm filter=lfs diff=lfs merge=lfs -text
*.tbz filter=lfs diff=lfs merge=lfs -text
*.tbz2 filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.ttf filter=lfs diff=lfs merge=lfs -text
*.txz filter=lfs diff=lfs merge=lfs -text
*.whl filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*.changes merge=merge-changes

5
.gitignore vendored Normal file
View File

@@ -0,0 +1,5 @@
mock
node_modules
*.obscpio
autogits-tmp.tar.zst
*.osc

View File

@@ -3,20 +3,18 @@ AutoGits
The bots that drive Git Workflow for package management
* devel-importer -- helper to import an OBS devel project into a Gitea organization
* devel-importer -- helper to import an OBS devel project into a Gitea organization
* gitea-events-rabbitmq-publisher -- takes all events from a Gitea organization (webhook) and publishes it on a RabbitMQ instance
* maintainer-and-policy-bot -- review bot to make sure maintainer signed off on reviews, along with necessary other entities
* obs-staging-bot -- build bot for a PR
* obs-staging-bot -- build bot for a PR
* obs-status-service -- report build status of an OBS project as an SVG
* pr-review -- keeps PR to _ObsPrj consistent with a PR to a package update
* prjgit-updater -- update _ObsPrj based on direct pushes and repo creations/removals from organization
* staging-utils -- review tooling for PR
* workflow-pr -- keeps PR to _ObsPrj consistent with a PR to a package update
* workflow-direct -- update _ObsPrj based on direct pushes and repo creations/removals from organization
* staging-utils -- review tooling for PR
- list PR
- merge PR
- split PR
- diff PR
- accept/reject PR
* random -- random utils and tools
Bugs
----

15
_service Normal file
View File

@@ -0,0 +1,15 @@
<services>
<!-- workaround, go_modules needs a tar and obs_scm doesn't take file://. -->
<service name="roast" mode="manual">
<param name="target">.</param>
<param name="reproducible">true</param>
<param name="outfile">autogits-tmp.tar.zst</param>
<param name="exclude">autogits-tmp.tar.zst</param>
</service>
<service name="go_modules" mode="manual">
<param name="basename">./</param>
<param name="compression">zst</param>
<param name="vendorname">vendor</param>
</service>
</services>

10
autogits.changes Normal file
View File

@@ -0,0 +1,10 @@
-------------------------------------------------------------------
Wed Sep 11 16:00:58 UTC 2024 - Adam Majer <adam.majer@suse.de>
- enable Authorization bearer token checks
-------------------------------------------------------------------
Wed Sep 11 14:10:18 UTC 2024 - Adam Majer <adam.majer@suse.de>
- rabbitmq publisher

View File

@@ -17,12 +17,11 @@
Name: autogits
Version: 0.0.1
Version: 0
Release: 0
Summary: GitWorkflow utilities
License: GPL-2.0-or-later
URL: https://src.opensuse.org/adamm/autogits/
Source: https://src.opensuse.org/adamm/autogits/0.0.1.tar.gz
URL: https://src.opensuse.org/adamm/autogits
Source1: vendor.tar.zst
BuildRequires: golang-packaging
BuildRequires: systemd-rpm-macros
@@ -33,6 +32,7 @@ BuildRequires: zstd
Git Workflow tooling and utilities enabling automated handing of OBS projects
as git repositories
%package -n gitea-events-rabbitmq-publisher
Summary: Publishes Gitea webhook data via RabbitMQ
@@ -41,18 +41,113 @@ Listens on an HTTP socket and publishes Gitea events on a RabbitMQ instance
with a topic
<scope>.src.$organization.$webhook_type.[$webhook_action_type]
%package -n doc
Summary: Common documentation files
%description -n doc
Common documentation files
%package -n devel-importer
Summary: Imports devel projects from obs to git
%description -n devel-importer
Command-line tool to import devel projects from obs to git
%package -n group-review
Summary: Reviews of groups defined in ProjectGit
%description -n group-review
Is used to handle reviews associated with groups defined in the
ProjectGit.
%package -n obs-staging-bot
Summary: Build a PR against a ProjectGit, if review is requested
%description -n obs-staging-bot
Build a PR against a ProjectGit, if review is requested.
%package -n obs-status-service
Summary: Reports build status of OBS service as an easily to produce SVG
%description -n obs-status-service
Reports build status of OBS service as an easily to produce SVG
%package -n workflow-direct
Summary: Keep ProjectGit in sync for a devel project
%description -n workflow-direct
Keep ProjectGit in sync with packages in the organization of a devel project
%package -n workflow-pr
Summary: Keeps ProjectGit PR in-sync with a PackageGit PR
%description -n workflow-pr
Keeps ProjectGit PR in-sync with a PackageGit PR
%prep
%autosetup -p1
cp -r /home/abuild/rpmbuild/SOURCES/* ./
tar x --zstd -f %{SOURCE1}
%build
go build \
-C gitea-events-rabbitmq-publisher \
-mod=vendor \
-buildmode=pie
go build \
-C devel-importer \
-mod=vendor \
-buildmode=pie
go build \
-C group-review \
-mod=vendor \
-buildmode=pie
go build \
-C obs-staging-bot \
-mod=vendor \
-buildmode=pie
go build \
-C obs-status-service \
-mod=vendor \
-buildmode=pie
#go build \
# -C workflow-direct \
# -mod=vendor \
# -buildmode=pie
#go build \
# -C workflow-pr \
# -mod=vendor \
# -buildmode=pie
%install
install -D -m0755 gitea-events-rabbitmq-publisher/gitea-events-rabbitmq-publisher %{buildroot}%{_bindir}
install -D -m0755 systemd/gitea-events-rabbitmq-publisher.service %{buildroot}%{_unitdir}
install -D -m0755 gitea-events-rabbitmq-publisher/gitea-events-rabbitmq-publisher %{buildroot}%{_bindir}/gitea-events-rabbitmq-publisher
install -D -m0644 systemd/gitea-events-rabbitmq-publisher.service %{buildroot}%{_unitdir}/gitea-events-rabbitmq-publisher.service
install -D -m0755 devel-importer/devel-importer %{buildroot}%{_bindir}/devel-importer
install -D -m0755 group-review/group-review %{buildroot}%{_bindir}/group-review
install -D -m0755 obs-staging-bot/obs-staging-bot %{buildroot}%{_bindir}/obs-staging-bot
install -D -m0755 obs-status-service/obs-status-service %{buildroot}%{_bindir}/obs-status-service
#install -D -m0755 workflow-direct/workflow-direct %{buildroot}%{_bindir}/workflow-direct
#install -D -m0755 workflow-pr/workflow-pr %{buildroot}%{_bindir}/workflow-pr
%pre -n gitea-events-rabbitmq-publisher
%service_add_pre gitea-events-rabbitmq-publisher.service
%post -n gitea-events-rabbitmq-publisher
%service_add_post gitea-events-rabbitmq-publisher.service
%preun -n gitea-events-rabbitmq-publisher
%service_del_preun gitea-events-rabbitmq-publisher.service
%postun -n gitea-events-rabbitmq-publisher
%service_del_postun gitea-events-rabbitmq-publisher.service
%files -n gitea-events-rabbitmq-publisher
%license COPYING
@@ -60,5 +155,38 @@ install -D -m0755 systemd/gitea-events-rabbitmq-publisher.service %{buildroot}%{
%{_bindir}/gitea-events-rabbitmq-publisher
%{_unitdir}/gitea-events-rabbitmq-publisher.service
%changelog
%files -n doc
%license COPYING
%doc doc/README.md
%doc doc/workflows.md
%files -n devel-importer
%license COPYING
%doc devel-importer/README.md
%{_bindir}/devel-importer
%files -n group-review
%license COPYING
%doc group-review/README.md
%{_bindir}/group-review
%files -n obs-staging-bot
%license COPYING
%doc obs-staging-bot/README.md
%{_bindir}/obs-staging-bot
%files -n obs-status-service
%license COPYING
%doc obs-status-service/README.md
%{_bindir}/obs-status-service
%files -n workflow-direct
%license COPYING
%doc workflow-direct/README.md
#%{_bindir}/workflow-direct
%files -n workflow-pr
%license COPYING
%doc workflow-pr/README.md
#%{_bindir}/workflow-pr

View File

@@ -1,74 +0,0 @@
package common
/*
* This file is part of Autogits.
*
* Copyright © 2024 SUSE LLC
*
* Autogits is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation, either version 2 of the License, or (at your option) any later
* version.
*
* Autogits is distributed in the hope that it will be useful, but WITHOUT ANY
* WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
* PARTICULAR PURPOSE. See the GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along with
* Foobar. If not, see <https://www.gnu.org/licenses/>.
*/
import (
"encoding/json"
"fmt"
"io"
"os"
"slices"
"strings"
)
type AutogitConfig struct {
Workflows []string // [pr, direct, test]
Organization string
GitProjectName string // Organization/GitProjectName.git is PrjGit
Branch string // branch name of PkgGit that aligns with PrjGit submodules
}
func ReadWorkflowConfigs(reader io.Reader) ([]*AutogitConfig, error) {
data, err := io.ReadAll(reader)
if err != nil {
return nil, fmt.Errorf("Error reading config file. err: %w", err)
}
var config []*AutogitConfig
if err = json.Unmarshal(data, &config); err != nil {
return nil, fmt.Errorf("Error parsing config file. err: %w", err)
}
availableWorkflows := []string{"pr", "direct", "test"}
for _, workflow := range config {
for _, w := range workflow.Workflows {
if !slices.Contains(availableWorkflows, w) {
return nil, fmt.Errorf(
"Invalid Workflow '%s'. Only available workflows are: %s",
w, strings.Join(availableWorkflows, " "),
)
}
}
if len(workflow.GitProjectName) == 0 {
workflow.GitProjectName = DefaultGitPrj
}
}
return config, nil
}
func ReadWorkflowConfigsFile(filename string) ([]*AutogitConfig, error) {
file, err := os.Open(filename)
if err != nil {
return nil, fmt.Errorf("Cannot open config file for reading. err: %w", err)
}
defer file.Close()
return ReadWorkflowConfigs(file)
}

View File

@@ -1,79 +0,0 @@
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 h1:DklsrG3dyBCFEj5IhUbnKptjxatkF07cF2ak3yi77so=
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.1 h1:pKouT5E8xu9zeFC39JXRDukb6JFQPXM5p5I91188VAQ=
github.com/go-logr/logr v1.4.1/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-openapi/analysis v0.23.0 h1:aGday7OWupfMs+LbmLZG4k0MYXIANxcuBTYUC03zFCU=
github.com/go-openapi/analysis v0.23.0/go.mod h1:9mz9ZWaSlV8TvjQHLl2mUW2PbZtemkE8yA5v22ohupo=
github.com/go-openapi/errors v0.22.0 h1:c4xY/OLxUBSTiepAg3j/MHuAv5mJhnf53LLMWFB+u/w=
github.com/go-openapi/errors v0.22.0/go.mod h1:J3DmZScxCDufmIMsdOuDHxJbdOGC0xtUynjIx092vXE=
github.com/go-openapi/jsonpointer v0.21.0 h1:YgdVicSA9vH5RiHs9TZW5oyafXZFc6+2Vc1rr/O9oNQ=
github.com/go-openapi/jsonpointer v0.21.0/go.mod h1:IUyH9l/+uyhIYQ/PXVA41Rexl+kOkAPDdXEYns6fzUY=
github.com/go-openapi/jsonreference v0.21.0 h1:Rs+Y7hSXT83Jacb7kFyjn4ijOuVGSvOdF2+tg1TRrwQ=
github.com/go-openapi/jsonreference v0.21.0/go.mod h1:LmZmgsrTkVg9LG4EaHeY8cBDslNPMo06cago5JNLkm4=
github.com/go-openapi/loads v0.22.0 h1:ECPGd4jX1U6NApCGG1We+uEozOAvXvJSF4nnwHZ8Aco=
github.com/go-openapi/loads v0.22.0/go.mod h1:yLsaTCS92mnSAZX5WWoxszLj0u+Ojl+Zs5Stn1oF+rs=
github.com/go-openapi/runtime v0.28.0 h1:gpPPmWSNGo214l6n8hzdXYhPuJcGtziTOgUpvsFWGIQ=
github.com/go-openapi/runtime v0.28.0/go.mod h1:QN7OzcS+XuYmkQLw05akXk0jRH/eZ3kb18+1KwW9gyc=
github.com/go-openapi/spec v0.21.0 h1:LTVzPc3p/RzRnkQqLRndbAzjY0d0BCL72A6j3CdL9ZY=
github.com/go-openapi/spec v0.21.0/go.mod h1:78u6VdPw81XU44qEWGhtr982gJ5BWg2c0I5XwVMotYk=
github.com/go-openapi/strfmt v0.23.0 h1:nlUS6BCqcnAk0pyhi9Y+kdDVZdZMHfEKQiS4HaMgO/c=
github.com/go-openapi/strfmt v0.23.0/go.mod h1:NrtIpfKtWIygRkKVsxh7XQMDQW5HKQl6S5ik2elW+K4=
github.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+GrE=
github.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ=
github.com/go-openapi/validate v0.24.0 h1:LdfDKwNbpB6Vn40xhTdNZAnfLECL81w+VX3BumrGD58=
github.com/go-openapi/validate v0.24.0/go.mod h1:iyeX1sEufmv3nPbBdX3ieNviWnOZaJ1+zquzJEf2BAQ=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/oklog/ulid v1.3.1 h1:EGfNDEx6MqHz8B3uNV6QAib1UR2Lm97sHi3ocA6ESJ4=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
github.com/opentracing/opentracing-go v1.2.0 h1:uEJPy/1a5RIPAJ0Ov+OIO8OxWu77jEv+1B0VhjKrZUs=
github.com/opentracing/opentracing-go v1.2.0/go.mod h1:GxEUsuufX4nBwe+T+Wl9TAgYrxe9dPLANfrWvHYVTgc=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rabbitmq/amqp091-go v1.10.0 h1:STpn5XsHlHGcecLmMFCtg7mqq0RnD+zFr4uzukfVhBw=
github.com/rabbitmq/amqp091-go v1.10.0/go.mod h1:Hy4jKW5kQART1u+JkDTF9YYOQUHXqMuhrgxOEeS7G4o=
github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=
github.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
go.mongodb.org/mongo-driver v1.14.0 h1:P98w8egYRjYe3XDjxhYJagTokP/H6HzlsnojRgZRd80=
go.mongodb.org/mongo-driver v1.14.0/go.mod h1:Vzb0Mk/pa7e6cWw85R4F/endUC3u0U9jGcNU603k65c=
go.opentelemetry.io/otel v1.24.0 h1:0LAOdjNmQeSTzGBzduGe/rU4tZhMwL5rWgtp9Ku5Jfo=
go.opentelemetry.io/otel v1.24.0/go.mod h1:W7b9Ozg4nkF5tWI5zsXkaKKDjdVjpD4oAt9Qi/MArHo=
go.opentelemetry.io/otel/metric v1.24.0 h1:6EhoGWWK28x1fbpA4tYTOWBkPefTDQnb8WSGXlc88kI=
go.opentelemetry.io/otel/metric v1.24.0/go.mod h1:VYhLe1rFfxuTXLgj4CBiyz+9WYBA8pNGJgDcSFRKBco=
go.opentelemetry.io/otel/sdk v1.24.0 h1:YMPPDNymmQN3ZgczicBY3B6sf9n62Dlj9pWD3ucgoDw=
go.opentelemetry.io/otel/sdk v1.24.0/go.mod h1:KVrIYw6tEubO9E96HQpcmpTKDVn9gdv35HoYiQWGDFg=
go.opentelemetry.io/otel/trace v1.24.0 h1:CsKnnL4dUAr/0llH9FKuc698G04IrpWV0MQA/Y1YELI=
go.opentelemetry.io/otel/trace v1.24.0/go.mod h1:HPc3Xr/cOApsBI154IU0OI0HJexz+aw5uPdbs3UCjNU=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.20.0 h1:Od9JTbYCk261bKm4M/mw7AklTlFYIa0bIp9BgSm1S8Y=
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -7,7 +7,8 @@ gitea-generated/client/gitea_api_client.go:: api.json
[ -d gitea-generated ] || mkdir gitea-generated
podman run --rm -v $$(pwd):/api ghcr.io/go-swagger/go-swagger generate client -f /api/api.json -t /api/gitea-generated
api: gitea-generated/client/gitea_api_client.go
api: gitea-generated/client/gitea_api_client.go mock_gitea_utils.go
go generate
build: api
go build

View File

@@ -0,0 +1,118 @@
package common
import (
"bufio"
"errors"
"fmt"
"io"
"regexp"
"slices"
"strconv"
"strings"
)
const PrPattern = "PR: %s/%s#%d"
type BasicPR struct {
Org, Repo string
Num int64
}
var validOrgAndRepoRx *regexp.Regexp = regexp.MustCompile("^[A-Za-z0-9_-]+$")
func parsePrLine(line string) (BasicPR, error) {
var ret BasicPR
trimmedLine := strings.TrimSpace(line)
// min size > 9 -> must fit all parameters in th PrPattern with at least one item per parameter
if len(trimmedLine) < 9 || trimmedLine[0:4] != "PR: " {
return ret, errors.New("Line too short")
}
trimmedLine = trimmedLine[4:]
org := strings.SplitN(trimmedLine, "/", 2)
ret.Org = org[0]
if len(org) != 2 {
return ret, errors.New("missing / separator")
}
repo := strings.SplitN(org[1], "#", 2)
ret.Repo = repo[0]
if len(repo) != 2 {
return ret, errors.New("Missing # separator")
}
// Gitea requires that each org and repo be [A-Za-z0-9_-]+
var err error
if ret.Num, err = strconv.ParseInt(repo[1], 10, 64); err != nil {
return ret, errors.New("Invalid number")
}
if !validOrgAndRepoRx.MatchString(repo[0]) || !validOrgAndRepoRx.MatchString(org[0]) {
return ret, errors.New("Invalid repo or org character set")
}
return ret, nil
}
func ExtractDescriptionAndPRs(data *bufio.Scanner) (string, []BasicPR) {
prs := make([]BasicPR, 0, 1)
var desc strings.Builder
for data.Scan() {
line := data.Text()
pr, err := parsePrLine(line)
if err != nil {
desc.WriteString(line)
desc.WriteByte('\n')
} else {
prs = append(prs, pr)
}
}
return strings.TrimSpace(desc.String()), prs
}
func prToLine(writer io.Writer, pr BasicPR) {
writer.Write([]byte("\n"))
fmt.Fprintf(writer, PrPattern, pr.Org, pr.Repo, pr.Num)
}
// returns:
// <0 for a<b
// >0 for a>b
// =0 when equal
func compareBasicPRs(a BasicPR, b BasicPR) int {
if c := strings.Compare(a.Org, b.Org); c != 0 {
return c
}
if c := strings.Compare(a.Repo, b.Repo); c != 0 {
return c
}
if a.Num > b.Num {
return 1
}
if a.Num < b.Num {
return -1
}
return 0
}
func AppendPRsToDescription(desc string, prs []BasicPR) string {
var out strings.Builder
out.WriteString(strings.TrimSpace(desc))
out.WriteString("\n")
slices.SortFunc(prs, compareBasicPRs)
prs = slices.Compact(prs)
for _, pr := range prs {
prToLine(&out, pr)
}
return out.String()
}

View File

@@ -0,0 +1,149 @@
package common_test
import (
"bufio"
"slices"
"strings"
"testing"
"src.opensuse.org/autogits/common"
)
func newStringScanner(s string) *bufio.Scanner {
return bufio.NewScanner(strings.NewReader(s))
}
func TestAssociatedPRScanner(t *testing.T) {
testTable := []struct {
name string
input string
prs []common.BasicPR
desc string
}{
{
"No PRs",
"",
[]common.BasicPR{},
"",
},
{
"Single PRs",
"Some header of the issue\n\nFollowed by some description\n\nPR: test/foo#4\n",
[]common.BasicPR{{Org: "test", Repo: "foo", Num: 4}},
"Some header of the issue\n\nFollowed by some description",
},
{
"Multiple PRs",
"Some header of the issue\n\nFollowed by some description\nPR: test/foo#4\n\nPR: test/goo#5\n",
[]common.BasicPR{
{Org: "test", Repo: "foo", Num: 4},
{Org: "test", Repo: "goo", Num: 5},
},
"Some header of the issue\n\nFollowed by some description",
},
{
"Multiple PRs with whitespace",
"Some header of the issue\n\n\tPR: test/goo#5\n\n Followed by some description\n \t PR: test/foo#4\n",
[]common.BasicPR{
{Org: "test", Repo: "foo", Num: 4},
{Org: "test", Repo: "goo", Num: 5},
},
"Some header of the issue\n\n\n Followed by some description",
},
{
"Multiple PRs with missing names and other special cases to ignore",
"Some header of the issue\n\n\n\t PR: foobar#5 \n\t PR: rd/goo5 \n\t PR: test/#5 \n" +
"\t PR: /goo#5 \n\t PR: test/goo# \n\t PR: test / goo # 10 \n\tPR: test/gool# 10 \n" +
"\t PR: test/goo#5 \n\t\n Followed by some description\n\t PR: test/foo#4 \n\t\n\n",
[]common.BasicPR{
{
Org: "test",
Repo: "foo",
Num: 4,
},
{
Org: "test",
Repo: "goo",
Num: 5,
},
},
"Some header of the issue\n\n\n\t PR: foobar#5 \n\t PR: rd/goo5 \n\t PR: test/#5 \n" +
"\t PR: /goo#5 \n\t PR: test/goo# \n\t PR: test / goo # 10 \n\tPR: test/gool# 10 \n" +
"\t\n Followed by some description",
},
}
for _, test := range testTable {
t.Run(test.name, func(t *testing.T) {
desc, prs := common.ExtractDescriptionAndPRs(newStringScanner(test.input))
if len(prs) != len(test.prs) {
t.Error("Unexpected length:", len(prs), "expected:", len(test.prs))
return
}
for _, p := range test.prs {
if !slices.Contains(prs, p) {
t.Error("missing expected PR", p)
}
}
if desc != test.desc {
t.Error("Desc output", len(desc), "!=", len(test.desc), ":", desc)
}
})
}
}
func TestAppendingPRsToDescription(t *testing.T) {
testTable := []struct {
name string
desc string
PRs []common.BasicPR
output string
}{
{
"Append single PR to end of description",
"something",
[]common.BasicPR{
{Org: "a", Repo: "b", Num: 100},
},
"something\n\nPR: a/b#100",
},
{
"Append multiple PR to end of description",
"something",
[]common.BasicPR{
{Org: "a1", Repo: "b", Num: 100},
{Org: "a1", Repo: "c", Num: 100},
{Org: "a1", Repo: "c", Num: 101},
{Org: "b", Repo: "b", Num: 100},
{Org: "c", Repo: "b", Num: 100},
},
"something\n\nPR: a1/b#100\nPR: a1/c#100\nPR: a1/c#101\nPR: b/b#100\nPR: c/b#100",
},
{
"Append multiple sorted PR to end of description and remove dups",
"something",
[]common.BasicPR{
{Org: "a1", Repo: "c", Num: 101},
{Org: "a1", Repo: "c", Num: 100},
{Org: "c", Repo: "b", Num: 100},
{Org: "b", Repo: "b", Num: 100},
{Org: "a1", Repo: "c", Num: 101},
{Org: "a1", Repo: "c", Num: 101},
{Org: "a1", Repo: "b", Num: 100},
},
"something\n\nPR: a1/b#100\nPR: a1/c#100\nPR: a1/c#101\nPR: b/b#100\nPR: c/b#100",
},
}
for _, test := range testTable {
t.Run(test.name, func(t *testing.T) {
d := common.AppendPRsToDescription(test.desc, test.PRs)
if d != test.output {
t.Error(len(d), "vs", len(test.output))
t.Error("unpected output", d)
}
})
}
}

161
common/config.go Normal file
View File

@@ -0,0 +1,161 @@
package common
/*
* This file is part of Autogits.
*
* Copyright © 2024 SUSE LLC
*
* Autogits is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation, either version 2 of the License, or (at your option) any later
* version.
*
* Autogits is distributed in the hope that it will be useful, but WITHOUT ANY
* WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
* PARTICULAR PURPOSE. See the GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along with
* Foobar. If not, see <https://www.gnu.org/licenses/>.
*/
import (
"encoding/json"
"errors"
"fmt"
"io"
"log"
"os"
"strings"
)
//go:generate mockgen -source=config.go -destination=mock/config.go -typed
type ConfigFile struct {
GitProjectName []string
}
type ReviewGroup struct {
Name string
Reviewers []string
}
type AutogitConfig struct {
Workflows []string // [pr, direct, test]
Organization string
GitProjectName string // Organization/GitProjectName.git is PrjGit
Branch string // branch name of PkgGit that aligns with PrjGit submodules
Reviewers []string // only used by `pr` workflow
ReviewGroups []ReviewGroup
}
type AutogitConfigs []*AutogitConfig
func ReadConfig(reader io.Reader) (*ConfigFile, error) {
data, err := io.ReadAll(reader)
if err != nil {
return nil, fmt.Errorf("Error reading config data: %w", err)
}
config := ConfigFile{}
if err := json.Unmarshal(data, &config.GitProjectName); err != nil {
return nil, fmt.Errorf("Error parsing Git Project paths: %w", err)
}
return &config, nil
}
func ReadConfigFile(filename string) (*ConfigFile, error) {
file, err := os.Open(filename)
if err != nil {
return nil, fmt.Errorf("Cannot open config file for reading. err: %w", err)
}
defer file.Close()
return ReadConfig(file)
}
type GiteaFileContentAndRepoFetcher interface {
GiteaFileContentReader
GiteaRepoFetcher
}
func ReadWorkflowConfig(gitea GiteaFileContentAndRepoFetcher, git_project string) (*AutogitConfig, error) {
hash := strings.Split(git_project, "#")
branch := ""
if len(hash) > 1 {
branch = hash[1]
}
a := strings.Split(hash[0], "/")
prjGitRepo := DefaultGitPrj
switch len(a) {
case 1:
case 2:
prjGitRepo = a[1]
default:
return nil, fmt.Errorf("Missing org/repo in projectgit: %s", git_project)
}
data, _, err := gitea.GetRepositoryFileContent(a[0], prjGitRepo, branch, "workflow.config")
if err != nil {
return nil, fmt.Errorf("Error fetching 'workflow.config' for %s/%s#%s: %w", a[0], prjGitRepo, branch, err)
}
var config AutogitConfig
if err := json.Unmarshal(data, &config); err != nil {
return nil, fmt.Errorf("Error parsing workflow config file: %s: %w", string(data), err)
}
if len(config.Organization) < 1 {
config.Organization = a[0]
}
config.GitProjectName = a[0] + "/" + prjGitRepo
if len(branch) == 0 {
if r, err := gitea.GetRepository(a[0], prjGitRepo); err == nil {
branch = r.DefaultBranch
} else {
return nil, fmt.Errorf("Failed to read workflow config in %s: %w", git_project, err)
}
}
config.GitProjectName = config.GitProjectName + "#" + branch
return &config, nil
}
func ResolveWorkflowConfigs(gitea GiteaFileContentAndRepoFetcher, config *ConfigFile) (AutogitConfigs, error) {
configs := make([]*AutogitConfig, 0, len(config.GitProjectName))
for _, git_project := range config.GitProjectName {
c, err := ReadWorkflowConfig(gitea, git_project)
if err != nil {
// can't sync, so ignore for now
log.Println(err)
} else {
configs = append(configs, c)
}
}
return configs, nil
}
func (configs AutogitConfigs) GetPrjGitConfig(org, repo, branch string) *AutogitConfig {
prjgit := org + "/" + repo + "#" + branch
for _, c := range configs {
if c.GitProjectName == prjgit {
return c
}
if c.Organization == org && c.Branch == branch {
return c
}
}
return nil
}
func (config *AutogitConfig) GetReviewGroupMembers(reviewer string) ([]string, error) {
for _, g := range config.ReviewGroups {
if g.Name == reviewer {
return g.Reviewers, nil
}
}
return nil, errors.New("User " + reviewer + " not found as group reviewer for " + config.GitProjectName)
}

49
common/config_test.go Normal file
View File

@@ -0,0 +1,49 @@
package common_test
import (
"testing"
"go.uber.org/mock/gomock"
mock_common "src.opensuse.org/autogits/common/mock"
"src.opensuse.org/autogits/common"
"src.opensuse.org/autogits/common/gitea-generated/models"
)
func TestConfigWorkflowParser(t *testing.T) {
tests := []struct {
name string
config_json string
repo models.Repository
}{
{
name: "Regular workflow file",
config_json: `{
"Workflows": ["direct", "pr"],
"Organization": "testing",
"ReviewGroups": [
{
"Name": "gnuman1",
"Reviewers": ["adamm"]
}
]
}`,
repo: models.Repository{
DefaultBranch: "master",
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
ctl := gomock.NewController(t)
gitea := mock_common.NewMockGiteaFileContentAndRepoFetcher(ctl)
gitea.EXPECT().GetRepositoryFileContent("foo", "bar", "", "workflow.config").Return([]byte(test.config_json), "abc", nil)
gitea.EXPECT().GetRepository("foo", "bar").Return(&test.repo, nil)
_, err := common.ReadWorkflowConfig(gitea, "foo/bar")
if err != nil {
t.Fatal(err)
}
})
}
}

View File

@@ -19,11 +19,14 @@ package common
*/
const (
GiteaTokenEnv = "GITEA_TOKEN"
ObsUserEnv = "OBS_USER"
ObsPasswordEnv = "OBS_PASSWORD"
GiteaTokenEnv = "GITEA_TOKEN"
ObsUserEnv = "OBS_USER"
ObsPasswordEnv = "OBS_PASSWORD"
ObsSshkeyEnv = "OBS_SSHKEY"
ObsSshkeyFileEnv = "OBS_SSHKEYFILE"
DefaultGitPrj = "_ObsPrj"
PrjLinksFile = "links.json"
GiteaRequestHeader = "X-Gitea-Event-Type"
Bot_BuildReview = "autogits_obs_staging_bot"

View File

@@ -1,4 +1,4 @@
package common
package common_test
/*
* This file is part of Autogits.

View File

@@ -19,18 +19,48 @@ package common
*/
import (
"bufio"
"bytes"
"errors"
"fmt"
"io"
"log"
"os"
"os/exec"
"path"
"path/filepath"
"strings"
"sync"
)
type GitHandler struct {
//go:generate mockgen -source=git_utils.go -destination=mock/git_utils.go -typed
type GitSubmoduleLister interface {
GitSubmoduleList(gitPath, commitId string) (submoduleList map[string]string, err error)
GitSubmoduleCommitId(cwd, packageName, commitId string) (subCommitId string, valid bool)
}
type GitStatusLister interface {
GitStatus(cwd string) ([]GitStatusData, error)
}
type Git interface {
GitParseCommits(cwd string, commitIDs []string) (parsedCommits []GitCommit, err error)
GitCatFile(cwd, commitId, filename string) (data []byte, err error)
GetPath() string
GitBranchHead(gitDir, branchName string) (string, error)
io.Closer
GitSubmoduleLister
GitStatusLister
GitExecWithOutputOrPanic(cwd string, params ...string) string
GitExecOrPanic(cwd string, params ...string)
GitExec(cwd string, params ...string) error
GitExecWithOutput(cwd string, params ...string) (string, error)
}
type GitHandlerImpl struct {
DebugLogger bool
GitPath string
@@ -38,20 +68,36 @@ type GitHandler struct {
GitEmail string
}
func CreateGitHandler(git_author, email, name string) (*GitHandler, error) {
var err error
func (s *GitHandlerImpl) GetPath() string {
return s.GitPath
}
git := new(GitHandler)
git.GitCommiter = git_author
git.GitPath, err = os.MkdirTemp("", name)
type GitHandlerGenerator interface {
CreateGitHandler(git_author, email, prjName string) (Git, error)
ReadExistingPath(git_author, email, gitPath string) (Git, error)
}
type GitHandlerGeneratorImpl struct{}
func (s *GitHandlerGeneratorImpl) CreateGitHandler(git_author, email, prj_name string) (Git, error) {
gitPath, err := os.MkdirTemp("", prj_name)
if err != nil {
return nil, fmt.Errorf("Cannot create temp dir: %w", err)
}
if err = os.Chmod(git.GitPath, 0700); err != nil {
if err = os.Chmod(gitPath, 0700); err != nil {
return nil, fmt.Errorf("Cannot fix permissions of temp dir: %w", err)
}
return s.ReadExistingPath(git_author, email, gitPath)
}
func (*GitHandlerGeneratorImpl) ReadExistingPath(git_author, email, gitPath string) (Git, error) {
git := &GitHandlerImpl{
GitCommiter: git_author,
GitPath: gitPath,
}
return git, nil
}
@@ -93,97 +139,16 @@ func (refs *GitReferences) addReference(id, branch string) {
refs.refs = append(refs.refs, GitReference{Branch: branch, Id: id})
}
func processRefs(gitDir string) ([]GitReference, error) {
packedRefsPath := path.Join(gitDir, "packed-refs")
stat, err := os.Stat(packedRefsPath)
func (e *GitHandlerImpl) GitBranchHead(gitDir, branchName string) (string, error) {
id, err := e.GitExecWithOutput(gitDir, "rev-list", "-1", branchName)
if err != nil {
return nil, err
return "", fmt.Errorf("Can't find default remote branch: %s", branchName)
}
if stat.Size() > 10000 || stat.IsDir() {
return nil, fmt.Errorf("Funny business with 'packed-refs' in '%s'", gitDir)
}
data, err := os.ReadFile(packedRefsPath)
if err != nil {
return nil, err
}
var references GitReferences
for _, line := range strings.Split(string(data), "\n") {
if len(line) < 1 || line[0] == '#' {
continue
}
splitLine := strings.Split(line, " ")
if len(splitLine) != 2 {
return nil, fmt.Errorf("Unexpected packaged-refs entry '%#v' in '%s'", splitLine, packedRefsPath)
}
id, ref := splitLine[0], splitLine[1]
const remoteRefPrefix = "refs/remotes/origin/"
if ref[0:len(remoteRefPrefix)] != remoteRefPrefix {
continue
}
references.addReference(id, ref[len(remoteRefPrefix):])
}
return references.refs, nil
return strings.TrimSpace(id), nil
}
func findGitDir(p string) (string, error) {
gitFile := path.Join(p, ".git")
stat, err := os.Stat(gitFile)
if err != nil {
return "", err
}
if stat.IsDir() {
return path.Join(p, ".git"), nil
}
data, err := os.ReadFile(gitFile)
if err != nil {
return "", err
}
for _, line := range strings.Split(string(data), "\n") {
refs := strings.Split(line, ":")
if len(refs) != 2 {
return "", fmt.Errorf("Unknown format of .git file: '%s'\n", line)
}
if refs[0] != "gitdir" {
return "", fmt.Errorf("Unknown header of .git file: '%s'\n", refs[0])
}
return path.Join(p, strings.TrimSpace(refs[1])), nil
}
return "", fmt.Errorf("Can't find git subdirectory in '%s'", p)
}
func (e *GitHandler) GitBranchHead(gitDir, branchName string) (string, error) {
path, err := findGitDir(path.Join(e.GitPath, gitDir))
if err != nil {
return "", fmt.Errorf("Error identifying gitdir in `%s`: %w", gitDir, err)
}
refs, err := processRefs(path)
if err != nil {
return "", fmt.Errorf("Error finding branches (%s): %w\n", branchName, err)
}
for _, ref := range refs {
if ref.Branch == branchName {
return ref.Id, nil
}
}
return "", fmt.Errorf("Can't find default remote branch: %s", branchName)
}
func (e *GitHandler) Close() error {
func (e *GitHandlerImpl) Close() error {
if err := os.RemoveAll(e.GitPath); err != nil {
return err
}
@@ -207,7 +172,28 @@ func (h writeFunc) Close() error {
return err
}
func (e *GitHandler) GitExec(cwd string, params ...string) error {
func (e *GitHandlerImpl) GitExecWithOutputOrPanic(cwd string, params ...string) string {
out, err := e.GitExecWithOutput(cwd, params...)
if err != nil {
log.Panicln("git command failed:", params, "@", cwd, "err:", err)
}
return out
}
func (e *GitHandlerImpl) GitExecOrPanic(cwd string, params ...string) {
if err := e.GitExec(cwd, params...); err != nil {
log.Panicln("git command failed:", params, "@", cwd, "err:", err)
}
}
func (e *GitHandlerImpl) GitExec(cwd string, params ...string) error {
_, err := e.GitExecWithOutput(cwd, params...)
return err
}
var ExtraGitParams []string
func (e *GitHandlerImpl) GitExecWithOutput(cwd string, params ...string) (string, error) {
cmd := exec.Command("/usr/bin/git", params...)
cmd.Env = []string{
"GIT_CEILING_DIRECTORIES=" + e.GitPath,
@@ -218,6 +204,9 @@ func (e *GitHandler) GitExec(cwd string, params ...string) error {
"GIT_LFS_SKIP_SMUDGE=1",
"GIT_SSH_COMMAND=/usr/bin/ssh -o StrictHostKeyChecking=yes",
}
if len(ExtraGitParams) > 0 {
cmd.Env = append(cmd.Env, ExtraGitParams...)
}
cmd.Dir = filepath.Join(e.GitPath, cwd)
cmd.Stdin = nil
@@ -232,10 +221,10 @@ func (e *GitHandler) GitExec(cwd string, params ...string) error {
if e.DebugLogger {
log.Printf(" *** error: %v\n", err)
}
return fmt.Errorf("error executing: git %#v \n%s\n err: %w", cmd.Args, out, err)
return "", fmt.Errorf("error executing: git %#v \n%s\n err: %w", cmd.Args, out, err)
}
return nil
return string(out), nil
}
type ChanIO struct {
@@ -273,18 +262,18 @@ func (c *ChanIO) Read(data []byte) (idx int, err error) {
return
}
type gitMsg struct {
type GitMsg struct {
hash string
itemType string
size int
}
type commit struct {
type GitCommit struct {
Tree string
Msg string
}
type tree_entry struct {
type GitTreeEntry struct {
name string
mode int
hash string
@@ -292,23 +281,23 @@ type tree_entry struct {
size int
}
type tree struct {
items []tree_entry
type GitTree struct {
items []GitTreeEntry
}
func (t *tree_entry) isSubmodule() bool {
func (t *GitTreeEntry) isSubmodule() bool {
return (t.mode & 0170000) == 0160000
}
func (t *tree_entry) isTree() bool {
func (t *GitTreeEntry) isTree() bool {
return (t.mode & 0170000) == 0040000
}
func (t *tree_entry) isBlob() bool {
func (t *GitTreeEntry) isBlob() bool {
return !t.isTree() && !t.isSubmodule()
}
func parseGitMsg(data <-chan byte) (gitMsg, error) {
func parseGitMsg(data <-chan byte) (GitMsg, error) {
var id []byte = make([]byte, 64)
var msgType []byte = make([]byte, 16)
var size int
@@ -319,7 +308,7 @@ func parseGitMsg(data <-chan byte) (gitMsg, error) {
id[pos] = c
pos++
} else {
return gitMsg{}, fmt.Errorf("Invalid character during object hash parse '%c' at %d", c, pos)
return GitMsg{}, fmt.Errorf("Invalid character during object hash parse '%c' at %d", c, pos)
}
}
id = id[:pos]
@@ -331,7 +320,7 @@ func parseGitMsg(data <-chan byte) (gitMsg, error) {
msgType[pos] = c
pos++
} else {
return gitMsg{}, fmt.Errorf("Invalid character during object type parse '%c' at %d", c, pos)
return GitMsg{}, fmt.Errorf("Invalid character during object type parse '%c' at %d", c, pos)
}
}
msgType = msgType[:pos]
@@ -341,26 +330,26 @@ func parseGitMsg(data <-chan byte) (gitMsg, error) {
break
case "missing":
if c != '\x00' {
return gitMsg{}, fmt.Errorf("Missing format weird")
return GitMsg{}, fmt.Errorf("Missing format weird")
}
return gitMsg{
return GitMsg{
hash: string(id[:]),
itemType: "missing",
size: 0,
}, fmt.Errorf("Object not found: '%s'", string(id))
default:
return gitMsg{}, fmt.Errorf("Invalid object type: '%s'", string(msgType))
return GitMsg{}, fmt.Errorf("Invalid object type: '%s'", string(msgType))
}
for c = <-data; c != '\000'; c = <-data {
if c >= '0' && c <= '9' {
size = size*10 + (int(c) - '0')
} else {
return gitMsg{}, fmt.Errorf("Invalid character during object size parse: '%c'", c)
return GitMsg{}, fmt.Errorf("Invalid character during object size parse: '%c'", c)
}
}
return gitMsg{
return GitMsg{
hash: string(id[:]),
itemType: string(msgType),
size: size,
@@ -400,20 +389,20 @@ func parseGitCommitMsg(data <-chan byte, l int) (string, error) {
return string(msg), nil
}
func parseGitCommit(data <-chan byte) (commit, error) {
func parseGitCommit(data <-chan byte) (GitCommit, error) {
hdr, err := parseGitMsg(data)
if err != nil {
return commit{}, err
return GitCommit{}, err
} else if hdr.itemType != "commit" {
return commit{}, fmt.Errorf("expected commit but parsed %s", hdr.itemType)
return GitCommit{}, fmt.Errorf("expected commit but parsed %s", hdr.itemType)
}
var c commit
var c GitCommit
l := hdr.size
for {
hdr, err := parseGitCommitHdr(data)
if err != nil {
return commit{}, nil
return GitCommit{}, nil
}
if len(hdr[0])+len(hdr[1]) == 0 { // hdr end marker
@@ -433,8 +422,8 @@ func parseGitCommit(data <-chan byte) (commit, error) {
return c, err
}
func parseTreeEntry(data <-chan byte, hashLen int) (tree_entry, error) {
var e tree_entry
func parseTreeEntry(data <-chan byte, hashLen int) (GitTreeEntry, error) {
var e GitTreeEntry
for c := <-data; c != ' '; c = <-data {
e.mode = e.mode*8 + int(c-'0')
@@ -463,20 +452,20 @@ func parseTreeEntry(data <-chan byte, hashLen int) (tree_entry, error) {
return e, nil
}
func parseGitTree(data <-chan byte) (tree, error) {
func parseGitTree(data <-chan byte) (GitTree, error) {
hdr, err := parseGitMsg(data)
if err != nil {
return tree{}, err
return GitTree{}, err
}
// max capacity to length of hash
t := tree{items: make([]tree_entry, 0, hdr.size/len(hdr.hash))}
t := GitTree{items: make([]GitTreeEntry, 0, hdr.size/len(hdr.hash))}
parsedLen := 0
for parsedLen < hdr.size {
entry, err := parseTreeEntry(data, len(hdr.hash)/2)
if err != nil {
return tree{}, nil
return GitTree{}, nil
}
t.items = append(t.items, entry)
@@ -513,8 +502,56 @@ func parseGitBlob(data <-chan byte) ([]byte, error) {
return d, nil
}
func (e *GitHandlerImpl) GitParseCommits(cwd string, commitIDs []string) (parsedCommits []GitCommit, err error) {
var done sync.Mutex
done.Lock()
data_in, data_out := ChanIO{make(chan byte, 256)}, ChanIO{make(chan byte, 70)}
parsedCommits = make([]GitCommit, 0, len(commitIDs))
go func() {
defer done.Unlock()
defer close(data_out.ch)
for _, id := range commitIDs {
data_out.Write([]byte(id))
data_out.ch <- '\x00'
c, e := parseGitCommit(data_in.ch)
if e != nil {
err = fmt.Errorf("Error parsing git commit: %w", e)
return
}
parsedCommits = append(parsedCommits, c)
}
}()
cmd := exec.Command("/usr/bin/git", "cat-file", "--batch", "-Z")
cmd.Env = []string{
"GIT_CEILING_DIRECTORIES=" + e.GitPath,
"GIT_LFS_SKIP_SMUDGE=1",
"GIT_CONFIG_GLOBAL=/dev/null",
}
cmd.Dir = filepath.Join(e.GitPath, cwd)
cmd.Stdout = &data_in
cmd.Stdin = &data_out
cmd.Stderr = writeFunc(func(data []byte) (int, error) {
if e.DebugLogger {
log.Println(string(data))
}
return len(data), nil
})
if e.DebugLogger {
log.Printf("command run: %v\n", cmd.Args)
}
err = cmd.Run()
done.Lock()
return
}
// TODO: support sub-trees
func (e *GitHandler) GitCatFile(cwd, commitId, filename string) (data []byte, err error) {
func (e *GitHandlerImpl) GitCatFile(cwd, commitId, filename string) (data []byte, err error) {
var done sync.Mutex
done.Lock()
@@ -557,67 +594,7 @@ func (e *GitHandler) GitCatFile(cwd, commitId, filename string) (data []byte, er
cmd := exec.Command("/usr/bin/git", "cat-file", "--batch", "-Z")
cmd.Env = []string{
"GIT_CEILING_DIRECTORIES=" + e.GitPath,
"GIT_CONFIG_GLOBAL=/dev/null",
}
cmd.Dir = filepath.Join(e.GitPath, cwd)
cmd.Stdout = &data_in
cmd.Stdin = &data_out
cmd.Stderr = writeFunc(func(data []byte) (int, error) {
if e.DebugLogger {
log.Printf(string(data))
}
return len(data), nil
})
if e.DebugLogger {
log.Printf("command run: %v\n", cmd.Args)
}
err = cmd.Run()
done.Lock()
return
}
// return (filename) -> (hash) map for all submodules
// TODO: recursive? map different orgs, not just assume '.' for path
func (e *GitHandler) GitSubmoduleList(cwd, commitId string) (submoduleList map[string]string, err error) {
var done sync.Mutex
submoduleList = make(map[string]string)
done.Lock()
data_in, data_out := ChanIO{make(chan byte, 256)}, ChanIO{make(chan byte, 70)}
go func() {
defer done.Unlock()
defer close(data_out.ch)
data_out.Write([]byte(commitId))
data_out.ch <- '\x00'
var c commit
c, err = parseGitCommit(data_in.ch)
if err != nil {
err = fmt.Errorf("Error parsing git commit. Err: %w", err)
return
}
data_out.Write([]byte(c.Tree))
data_out.ch <- '\x00'
var tree tree
tree, err = parseGitTree(data_in.ch)
if err != nil {
err = fmt.Errorf("Error parsing git tree: %w", err)
return
}
for _, te := range tree.items {
if te.isSubmodule() {
submoduleList[te.name] = te.hash
}
}
}()
cmd := exec.Command("/usr/bin/git", "cat-file", "--batch", "-Z")
cmd.Env = []string{
"GIT_CEILING_DIRECTORIES=" + e.GitPath,
"GIT_LFS_SKIP_SMUDGE=1",
"GIT_CONFIG_GLOBAL=/dev/null",
}
cmd.Dir = filepath.Join(e.GitPath, cwd)
@@ -635,17 +612,72 @@ func (e *GitHandler) GitSubmoduleList(cwd, commitId string) (submoduleList map[s
err = cmd.Run()
done.Lock()
return submoduleList, err
return
}
func (e *GitHandler) GitSubmoduleCommitId(cwd, packageName, commitId string) (subCommitId string, valid bool) {
defer func() {
if recover() != nil {
commitId = ""
valid = false
// return (filename) -> (hash) map for all submodules
// TODO: recursive? map different orgs, not just assume '.' for path
func (e *GitHandlerImpl) GitSubmoduleList(gitPath, commitId string) (submoduleList map[string]string, err error) {
var done sync.Mutex
submoduleList = make(map[string]string)
done.Lock()
data_in, data_out := ChanIO{make(chan byte, 256)}, ChanIO{make(chan byte, 70)}
go func() {
defer done.Unlock()
defer close(data_out.ch)
data_out.Write([]byte(commitId))
data_out.ch <- '\x00'
var c GitCommit
c, err = parseGitCommit(data_in.ch)
if err != nil {
err = fmt.Errorf("Error parsing git commit. Err: %w", err)
return
}
data_out.Write([]byte(c.Tree))
data_out.ch <- '\x00'
var tree GitTree
tree, err = parseGitTree(data_in.ch)
if err != nil {
err = fmt.Errorf("Error parsing git tree: %w", err)
return
}
for _, te := range tree.items {
if te.isSubmodule() {
submoduleList[te.name] = te.hash
}
}
}()
cmd := exec.Command("/usr/bin/git", "cat-file", "--batch", "-Z")
cmd.Env = []string{
"GIT_CEILING_DIRECTORIES=" + e.GitPath,
"GIT_LFS_SKIP_SMUDGE=1",
"GIT_CONFIG_GLOBAL=/dev/null",
}
cmd.Dir = filepath.Join(e.GitPath, gitPath)
cmd.Stdout = &data_in
cmd.Stdin = &data_out
cmd.Stderr = writeFunc(func(data []byte) (int, error) {
if e.DebugLogger {
log.Println(string(data))
}
return len(data), nil
})
if e.DebugLogger {
log.Printf("command run: %v\n", cmd.Args)
}
err = cmd.Run()
done.Lock()
return submoduleList, err
}
func (e *GitHandlerImpl) GitSubmoduleCommitId(cwd, packageName, commitId string) (subCommitId string, valid bool) {
data_in, data_out := ChanIO{make(chan byte, 256)}, ChanIO{make(chan byte, 70)}
var wg sync.WaitGroup
@@ -656,6 +688,14 @@ func (e *GitHandler) GitSubmoduleCommitId(cwd, packageName, commitId string) (su
}
go func() {
defer func() {
if recover() != nil {
subCommitId = "wrong"
commitId = "ok"
valid = false
}
}()
defer wg.Done()
defer close(data_out.ch)
@@ -684,6 +724,7 @@ func (e *GitHandler) GitSubmoduleCommitId(cwd, packageName, commitId string) (su
cmd := exec.Command("/usr/bin/git", "cat-file", "--batch", "-Z")
cmd.Env = []string{
"GIT_CEILING_DIRECTORIES=" + e.GitPath,
"GIT_LFS_SKIP_SMUDGE=1",
"GIT_CONFIG_GLOBAL=/dev/null",
}
cmd.Dir = filepath.Join(e.GitPath, cwd)
@@ -703,3 +744,182 @@ func (e *GitHandler) GitSubmoduleCommitId(cwd, packageName, commitId string) (su
wg.Wait()
return subCommitId, len(subCommitId) == len(commitId)
}
const (
GitStatus_Untracked = 0
GitStatus_Modified = 1
GitStatus_Ignored = 2
GitStatus_Unmerged = 3 // States[0..3] -- Stage1, Stage2, Stage3 of merge objects
GitStatus_Renamed = 4 // orig name in States[0]
)
type GitStatusData struct {
Path string
Status int
States [3]string
}
func parseGitStatusHexString(data io.ByteReader) (string, error) {
str := make([]byte, 0, 32)
for {
c, err := data.ReadByte()
if err != nil {
return "", err
}
switch {
case c == 0 || c == ' ':
return string(str), nil
case c >= 'a' && c <= 'f':
case c >= 'A' && c <= 'F':
case c >= '0' && c <= '9':
default:
return "", errors.New("Invalid character in hex string:" + string(c))
}
str = append(str, c)
}
}
func parseGitStatusString(data io.ByteReader) (string, error) {
str := make([]byte, 0, 100)
for {
c, err := data.ReadByte()
if err != nil {
return "", errors.New("Unexpected EOF. Expected NUL string term")
}
if c == 0 {
return string(str), nil
}
str = append(str, c)
}
}
func skipGitStatusEntry(data io.ByteReader, skipSpaceLen int) error {
for skipSpaceLen > 0 {
c, err := data.ReadByte()
if err != nil {
return err
}
if c == ' ' {
skipSpaceLen--
}
}
return nil
}
func parseSingleStatusEntry(data io.ByteReader) (*GitStatusData, error) {
ret := GitStatusData{}
statusType, err := data.ReadByte()
if err != nil {
return nil, nil
}
switch statusType {
case '1':
var err error
if err = skipGitStatusEntry(data, 8); err != nil {
return nil, err
}
ret.Status = GitStatus_Modified
ret.Path, err = parseGitStatusString(data)
if err != nil {
return nil, err
}
case '2':
var err error
if err = skipGitStatusEntry(data, 9); err != nil {
return nil, err
}
ret.Status = GitStatus_Renamed
ret.Path, err = parseGitStatusString(data)
if err != nil {
return nil, err
}
ret.States[0], err = parseGitStatusString(data)
if err != nil {
return nil, err
}
case '?':
var err error
if err = skipGitStatusEntry(data, 1); err != nil {
return nil, err
}
ret.Status = GitStatus_Untracked
ret.Path, err = parseGitStatusString(data)
if err != nil {
return nil, err
}
case '!':
var err error
if err = skipGitStatusEntry(data, 1); err != nil {
return nil, err
}
ret.Status = GitStatus_Ignored
ret.Path, err = parseGitStatusString(data)
if err != nil {
return nil, err
}
case 'u':
var err error
if err = skipGitStatusEntry(data, 7); err != nil {
return nil, err
}
if ret.States[0], err = parseGitStatusHexString(data); err != nil {
return nil, err
}
if ret.States[1], err = parseGitStatusHexString(data); err != nil {
return nil, err
}
if ret.States[2], err = parseGitStatusHexString(data); err != nil {
return nil, err
}
ret.Status = GitStatus_Unmerged
ret.Path, err = parseGitStatusString(data)
if err != nil {
return nil, err
}
default:
return nil, errors.New("Invalid status type" + string(statusType))
}
return &ret, nil
}
func parseGitStatusData(data io.ByteReader) ([]GitStatusData, error) {
ret := make([]GitStatusData, 0, 10)
for {
data, err := parseSingleStatusEntry(data)
if err != nil {
return nil, err
} else if data == nil {
break
}
ret = append(ret, *data)
}
return ret, nil
}
func (e *GitHandlerImpl) GitStatus(cwd string) (ret []GitStatusData, err error) {
if e.DebugLogger {
log.Println("getting git-status()")
}
cmd := exec.Command("/usr/bin/git", "status", "--porcelain=2", "-z")
cmd.Env = []string{
"GIT_CEILING_DIRECTORIES=" + e.GitPath,
"GIT_LFS_SKIP_SMUDGE=1",
"GIT_CONFIG_GLOBAL=/dev/null",
}
cmd.Dir = filepath.Join(e.GitPath, cwd)
cmd.Stderr = writeFunc(func(data []byte) (int, error) {
log.Println(string(data))
return len(data), nil
})
if e.DebugLogger {
log.Printf("command run: %v\n", cmd.Args)
}
out, err := cmd.Output()
if err != nil {
log.Printf("Error running command %v, err: %v", cmd.Args, err)
}
return parseGitStatusData(bufio.NewReader(bytes.NewReader(out)))
}

View File

@@ -19,9 +19,12 @@ package common
*/
import (
"bufio"
"bytes"
"os"
"os/exec"
"path"
"slices"
"strings"
"testing"
)
@@ -259,7 +262,7 @@ func TestCommitTreeParsingOfHead(t *testing.T) {
t.Run("reads HEAD and parses the tree", func(t *testing.T) {
const nodejs21 = "c678c57007d496a98bec668ae38f2c26a695f94af78012f15d044ccf066ccb41"
h := GitHandler{
h := GitHandlerImpl{
GitPath: gitDir,
}
id, ok := h.GitSubmoduleCommitId("", "nodejs21", commitId)
@@ -272,7 +275,7 @@ func TestCommitTreeParsingOfHead(t *testing.T) {
})
t.Run("reads README.md", func(t *testing.T) {
h := GitHandler{
h := GitHandlerImpl{
GitPath: gitDir,
}
data, err := h.GitCatFile("", commitId, "README.md")
@@ -285,7 +288,7 @@ func TestCommitTreeParsingOfHead(t *testing.T) {
})
t.Run("read HEAD", func(t *testing.T) {
h := GitHandler{
h := GitHandlerImpl{
GitPath: gitDir,
}
@@ -302,3 +305,110 @@ func TestCommitTreeParsingOfHead(t *testing.T) {
t.Run("try to parse unknown item", func(t *testing.T) {
})
}
func TestGitStatusParse(t *testing.T) {
testData := []struct {
name string
data []byte
res []GitStatusData
}{
{
name: "Single modified line",
data: []byte("1 .M N... 100644 100644 100644 dbe4b3d5a0a2e385f78fd41d726baa20e9190f7b5a2e78cbd4885586832f39e7 dbe4b3d5a0a2e385f78fd41d726baa20e9190f7b5a2e78cbd4885586832f39e7 bots-common/git_utils.go\x00"),
res: []GitStatusData{
{
Path: "bots-common/git_utils.go",
Status: GitStatus_Modified,
},
},
},
{
name: "Untracked entries",
data: []byte("1 .M N... 100644 100644 100644 dbe4b3d5a0a2e385f78fd41d726baa20e9190f7b5a2e78cbd4885586832f39e7 dbe4b3d5a0a2e385f78fd41d726baa20e9190f7b5a2e78cbd4885586832f39e7 bots-common/git_utils.go\x00? bots-common/c.out\x00? doc/Makefile\x00"),
res: []GitStatusData{
{
Path: "bots-common/git_utils.go",
Status: GitStatus_Modified,
},
{
Path: "bots-common/c.out",
Status: GitStatus_Untracked,
},
{
Path: "doc/Makefile",
Status: GitStatus_Untracked,
},
},
},
{
name: "Untracked entries",
data: []byte("1 .M N... 100644 100644 100644 dbe4b3d5a0a2e385f78fd41d726baa20e9190f7b5a2e78cbd4885586832f39e7 dbe4b3d5a0a2e385f78fd41d726baa20e9190f7b5a2e78cbd4885586832f39e7 bots-common/git_utils.go\x00? bots-common/c.out\x00! doc/Makefile\x00"),
res: []GitStatusData{
{
Path: "bots-common/git_utils.go",
Status: GitStatus_Modified,
},
{
Path: "bots-common/c.out",
Status: GitStatus_Untracked,
},
{
Path: "doc/Makefile",
Status: GitStatus_Ignored,
},
},
},
{
name: "Nothing",
},
{
name: "Unmerged .gitmodules during a merge",
data: []byte("1 A. S... 000000 160000 160000 0000000000000000000000000000000000000000000000000000000000000000 ed07665aea0522096c88a7555f1fa9009ed0e0bac92de4613c3479516dd3d147 pkgB2\x00u UU N... 100644 100644 100644 100644 587ec403f01113f2629da538f6e14b84781f70ac59c41aeedd978ea8b1253a76 d23eb05d9ca92883ab9f4d28f3ec90c05f667f3a5c8c8e291bd65e03bac9ae3c 087b1d5f22dbf0aa4a879fff27fff03568b334c90daa5f2653f4a7961e24ea33 .gitmodules\x00"),
res: []GitStatusData{
{
Path: "pkgB2",
Status: GitStatus_Modified,
},
{
Path: ".gitmodules",
Status: GitStatus_Unmerged,
States: [3]string{"587ec403f01113f2629da538f6e14b84781f70ac59c41aeedd978ea8b1253a76", "d23eb05d9ca92883ab9f4d28f3ec90c05f667f3a5c8c8e291bd65e03bac9ae3c", "087b1d5f22dbf0aa4a879fff27fff03568b334c90daa5f2653f4a7961e24ea33"},
},
},
},
{
name: "Renamed file",
data: []byte("1 M. N... 100644 100644 100644 d23eb05d9ca92883ab9f4d28f3ec90c05f667f3a5c8c8e291bd65e03bac9ae3c 896cd09f36d39e782d66ae32dd5614d4f4d83fc689f132aab2dfc019a9f5b6f3 .gitmodules\x002 R. S... 160000 160000 160000 3befe051a34612530acfa84c736d2454278453ec0f78ec028f25d2980f8c3559 3befe051a34612530acfa84c736d2454278453ec0f78ec028f25d2980f8c3559 R100 pkgQ\x00pkgC\x00"),
res: []GitStatusData{
{
Path: "pkgQ",
Status: GitStatus_Renamed,
States: [3]string{"pkgC"},
},
{
Path: ".gitmodules",
Status: GitStatus_Modified,
},
},
},
}
for _, test := range testData {
t.Run(test.name, func(t *testing.T) {
r, err := parseGitStatusData(bufio.NewReader(bytes.NewReader(test.data)))
if err != nil {
t.Fatal(err)
}
if len(r) != len(test.res) {
t.Fatal("len(r):", len(r), "is not expected", len(test.res))
}
for _, expected := range test.res {
if !slices.Contains(r, expected) {
t.Fatal("result", r, "doesn't contains expected", expected)
}
}
})
}
}

Some files were not shown because too many files have changed in this diff Show More