4536 Commits

Author SHA1 Message Date
Stephan Kulow
d5947f5c19 Refactored rabbit-openqa to be based on PubSubConsumer
The base class listens to openSUSE pubsub messages (and can be used
standalone) and rabbit-openqa just overwrites the keys and the on_message
handler
2018-10-12 11:06:54 +02:00
Stephan Kulow
ae7f35c7e2 Adopt pika example on async 2018-10-11 15:10:41 +02:00
Stephan Kulow
61e1049895
Merge pull request #1721 from coolo/catch_unmirrored_repos
pkglistgen: Catch unmirrored repos
2018-10-11 11:38:48 +02:00
37ac71d0bf
Merge pull request #1694 from dirkmueller/master
[ARM 15.1] Include JeOS images in ToTest
2018-10-11 16:51:38 +08:00
Stephan Kulow
dc3201d238 Move unlink_list to pkglistgen class too
It's used in update_repos
2018-10-11 07:43:36 +02:00
Stephan Kulow
e70c071ce9 Move update_repos into Pkglistgen class
We need to track state within the tool, not in the command line
interface
2018-10-11 07:43:36 +02:00
Stephan Kulow
b8e451f8d1 Don't make missing repos (per arch) a fatal error
It's possible we link against repos that do not exist for all archs
2018-10-11 07:43:35 +02:00
5d542a82af
Merge pull request #1724 from Vogtinator/armkubic
Add openSUSE-Tumbleweed-Kubic-dvd5-dvd-aarch64 to openSUSE:Factory:ARM
2018-10-11 07:22:40 +02:00
Stephan Kulow
ceaef3ea88 Moved the ISO replace_string into OBS attributes 2018-10-10 20:59:16 +02:00
Stephan Kulow
a8d7b769c1 Find a sed like syntax for Staging ISOs
This can be moved to remote config
2018-10-10 20:07:21 +02:00
Stephan Kulow
3933e4e302 Package rabbit-openqa
I don't really like the name, but lack alternative ideas
2018-10-10 20:07:21 +02:00
Stephan Kulow
b38a1edc34 Make the names of the openQA checks unique
Prefer 'minimal_x' unless there are 2 (the code doesn't
support more than 2)
2018-10-10 20:07:21 +02:00
Stephan Kulow
b886e28db3 No need to if loop - we do the same for all of openqa 2018-10-10 20:07:21 +02:00
Stephan Kulow
6f5978cab6 Link to failed test modules
SLE unfortunately does not fail atm, so I had to implement leap :)
2018-10-10 20:07:21 +02:00
Stephan Kulow
ecdc60065a Fetch all openQA jobs for the ISO every time
This is ugly on first look, but has several advantages:
- we can more easily support a cold start
- as such we don't need to have a persistant queue and
  can directly bind the routing keys we want
- we do the same on all openqa events, simplifying the code
- we can cope support short names for the checks

The last is the most significant benefit (not yet implemented though).
We can name the openqa jobs RAID1 and gnome and only have to append
the machine name (or other settings) if they conflict
2018-10-10 20:07:21 +02:00
Stephan Kulow
dab6850133 Gather initial buildid of repositories 2018-10-10 20:07:21 +02:00
Stephan Kulow
d1c3e212d7 Add helper bot to listen to rabbit bus and feed OBS with openQA Status
There is an API gap on OBS side atm, so this won't complete:
https://github.com/openSUSE/open-build-service/issues/6035
2018-10-10 20:07:21 +02:00
fa6f388dd4 Add openSUSE-Tumbleweed-Kubic-dvd5-dvd-aarch64 to openSUSE:Factory:ARM 2018-10-10 15:40:58 +02:00
Stephan Kulow
e90ac73952
Merge pull request #1723 from coolo/fix_unsorted
pkglistgen: Fix copy&paste error on uploading results
2018-10-10 11:00:06 +02:00
Stephan Kulow
59d9454134 pkglistgen: Fix copy&paste error on uploading results 2018-10-10 10:46:45 +02:00
Stephan Kulow
97f1eda281
Merge pull request #1722 from coolo/filter_arch_3
Fix overlap calculation: modules only have filtered archs
2018-10-10 07:09:33 +02:00
Stephan Kulow
fb4d63ec3d Fix overlap calculation: modules only have filtered archs 2018-10-10 06:45:14 +02:00
Stephan Kulow
97e58e1732
Merge pull request #1719 from coolo/dont_spam_submitters
Do no longer CC submitters in request comments
2018-10-04 11:29:32 +02:00
Stephan Kulow
caee9053c4
Merge pull request #1718 from coolo/filter_arch_2
pkglistgen: Make sure we don't calculate non-existant architectures i…
2018-10-04 09:58:21 +02:00
Stephan Kulow
e6bf5ccb7b Do no longer CC submitters in request comments
Submitters complain more and more about the spam they're getting from
staging projects - and we rather leave that weapon for actual feedback.

One especially annoying case is if a package is added to one staging project
and then later moved to another, the submitter will receive notifications
of all kind of bots and actions for both staging projects.
2018-10-04 09:52:49 +02:00
Stephan Kulow
a96e864c06 pkglistgen: Make sure we don't calculate non-existant architectures in stagings
Otherwise the package list is not staging specific
2018-10-04 07:00:56 +02:00
Stephan Kulow
5ba0b87577
Merge pull request #1717 from coolo/filter_arch
pkglistgen: only PUT new data if it changed
2018-10-04 06:59:26 +02:00
Stephan Kulow
c4fa49aef2 Refactor conditional_PUT as core function
Introduce source_file_ensure in osclib.core following review in #1717
2018-10-03 07:36:42 +02:00
Stephan Kulow
2d4148f3f2 pkglistgen: only PUT new data if it changed
Otherwise we collect quickly a million revisions with no gain
2018-10-02 06:48:58 +02:00
Ludwig Nussel
f1c8f63900 leaper: block on pending factory submissions also for new packages 2018-10-01 09:57:15 +02:00
Stephan Kulow
39a3f83180
Merge pull request #1711 from coolo/jberry-suse-osclib-core-repository-state
Don't trust _result for repository state
2018-09-29 07:24:06 +02:00
Stephan Kulow
6004636b21
Merge pull request #1714 from jberry-suse/repo-checker-build-failure-check
repo_checker: re-instate staging build failure check after generic rework.
2018-09-29 07:22:51 +02:00
Jimmy Berry
a0de4c5b94 repo_checker: re-instate staging build failure check after generic rework. 2018-09-28 17:27:04 -05:00
Stephan Kulow
f3961e4857 pkglistgen: Only check repository check per arch
(also indent 2 lines due to calling autopep)
2018-09-28 08:05:15 +02:00
Stephan Kulow
4539190af7 Optimize repository_state
The _result view checks way more things than we need, so just iterate
through the architectures of a repository and check the binaryversions
per arch. And combine the sha1sums to one sha1 per repository
2018-09-28 08:05:15 +02:00
Jimmy Berry
2deda1a5f1
Merge pull request #1710 from jberry-suse/osclib-core-repository-state
osclib/core: repository_state(): rework to utilize binarylist view.
2018-09-27 08:24:24 -05:00
Stephan Kulow
ed9afeacc2
Merge pull request #1709 from coolo/fix_repo_pkglistgen
Move creation of the Solver object out of loop
2018-09-27 07:17:33 +02:00
Stephan Kulow
5a1307b98b Move creation of the Solver object out of loop
After time for sle_module_basesystem: before 16.5s, after 9.7s
2018-09-27 07:07:05 +02:00
Jimmy Berry
ea94191d21
Merge pull request #1688 from jberry-suse/kubernetes
[WIP] dist/kubernetes: provide basic ksonnet setup for heroes cluster including check-source and repo-checker modules
2018-09-26 17:27:55 -05:00
Jimmy Berry
73edfb8121 osclib/core: repository_state(): rework to utilize binarylist view.
For whatever reason the state hash only includes the overall publish states
and not the binaries within the repository. As such, rebuilding state
before a  rebuild and after may be the same.
2018-09-26 17:11:45 -05:00
Stephan Kulow
015677fa45
Merge pull request #1704 from coolo/use_filtered_unsorted
Don't take unsorted.group as reference for SP1 product changes
2018-09-26 10:02:43 +02:00
Stephan Kulow
1527c616dc
Merge pull request #1705 from lnussel/changelog_checker
check_tags_in_requests: factory property is actually an array
2018-09-24 17:35:26 +02:00
Ludwig Nussel
31004b5724 check_tags_in_requests: factory property is actually an array 2018-09-24 16:01:44 +02:00
Stephan Kulow
ea0e6d4aaa Don't take unsorted.group as reference for SP1 product changes
The release managers are actually supposed to maintain the filter list
in unneeded.yml, but that only applies to unsorted.yml not to the unsorted.group
(which is basically unfiltered to be published into OBS's SLE-15)
2018-09-24 15:33:10 +02:00
Stephan Kulow
e409a673ae
Merge pull request #1702 from jberry-suse/pkglistgen-do-update-state-check
pkglistgen: compare repository state to determine when to update solv.
2018-09-21 14:31:35 +02:00
Jimmy Berry
a074572c45 pkglistgen: compare repository state to determine when to update solv.
Since both pkglistgen and repo_checker share the same rpm metadata cache
either may update it first and thus bs_mirrorfull will only contain output
for one of the tools. As such the check in pkglistgen could cause the solv
file to remain stale. Instead write a secondary file containing the
repository state hash and check to see if the file already exists.
2018-09-20 16:38:05 -05:00
Jimmy Berry
35ea0e17fa
Merge pull request #1701 from jberry-suse/metrics-access-timer-old-systemd
systemd/metrics-access.timer: utilize compatible calendar format.
2018-09-20 09:06:42 -05:00
Jimmy Berry
9341b740df systemd/metrics-access.timer: utilize compatible calendar format.
systemd 228 does not support this format and fails to load timer.
2018-09-19 22:22:53 -05:00
Jimmy Berry
38e2d3fa1c
Merge pull request #1699 from jberry-suse/metrics-access-aggregate-empty-keys
metrics/access/aggregate: correct empty seen key handling.
2018-09-18 16:45:12 -05:00
Jimmy Berry
d3afb6806b
Merge pull request #1698 from jberry-suse/core-pseudometa-ensure-fix
osclib/core: project_pseudometa_file_ensure(): correct comparison logic.
2018-09-18 16:26:03 -05:00