72 Commits

Author SHA1 Message Date
Stephan Kulow
bceaa53e13 pkglisten: Generate a comment for staging diffs
This is the next step towards having multiple submissions to SLE stagings
2022-03-17 11:16:06 +01:00
Stephan Kulow
0bbf3b2bdf
Merge pull request #2751 from coolo/change_reference
pkglistgen: If the groups contain a summary-staging.txt evaluate it
2022-03-15 13:44:45 +01:00
Stephan Kulow
1f85a4aef1 pkglistgen: If the groups contain a summary-staging.txt evaluate it 2022-03-15 13:42:16 +01:00
Stephan Kulow
54e1f9de6c pkglistgen: Remove very noisy debug output 2022-03-14 08:52:48 +01:00
Stephan Kulow
c8721fcb13 Fix filtering of unneeded.yml
The regexps were supposed to match package names, but re.match does not
what I thought it does - it only matches the start of the line not the
end
2022-03-11 08:08:55 +01:00
Stephan Kulow
b93adbd9c1 pkglistgen: Fix exit values - True is actually bad 2022-03-05 15:58:20 +01:00
Stephan Kulow
e3f0a13b96 pkglistgen: Don't exit 1 if we skip weakremovers due to restarted builds
Since 30daff5613803667be7422056431116e84c07d59 we check if the repos are
still what we expected them to be when we started and stop calculating
weakremovers.inc if the build didn't stay still for the full runtime of
the pkglistgen. But the exception makes it impossible to spot real
failures - so make this a regular exit
2022-03-04 13:40:29 +01:00
Stephan Kulow
7a63f0c3bd Treat all group*.yml like groups.yml
Fixes #2655
2022-03-03 16:26:03 +01:00
Stephan Kulow
33da10611b pkglistgen: Also support zstd for weakremovers.inc 2022-02-25 09:43:34 +01:00
Stephan Kulow
aaab4af930 Target F401 - Unused imports 2022-02-18 18:29:27 +01:00
Stephan Kulow
f86579642b Target E501 - Line too long
https://www.flake8rules.com/rules/E501.html - but 79 characters is a
joke
2022-02-18 18:17:27 +01:00
Stephan Kulow
f38bd3a7ed Target E128 - Continuation line under-indented for visual indent
https://www.flake8rules.com/rules/E128.html
2022-02-18 17:35:33 +01:00
Stephan Kulow
27c364b0c9 Target E251 - Unexpected spaces around keyword
https://www.flake8rules.com/rules/E251.html
2022-02-18 17:28:13 +01:00
Stephan Kulow
529d1dafae Target E302 - add two comment lines before class 2022-02-18 17:15:48 +01:00
Stephan Kulow
579145b52c Target F841 - Remove unused variables 2022-02-18 17:01:38 +01:00
Stephan Kulow
ae734ba555 Fix indent of statements E127
https://www.flake8rules.com/rules/E127.html
2022-02-18 14:23:59 +01:00
Stephan Kulow
2de74cfc52 Replace variable name 'l' (x is fine)
https://www.flake8rules.com/rules/E741.html
2022-02-18 14:21:59 +01:00
Stephan Kulow
cd161f34cd
Merge pull request #2668 from dirkmueller/use_newest_package
add option to use newest package for resolving dependencies
2021-11-05 07:56:18 +01:00
Dirk Mueller
f968110f9d add option to use newest package for resolving dependencies
In some cases we'd like to chose from all available repositories
for selecting the right package. in appliance builds and there
is an OBS-UseUnorderedRepos flag for enabling this behavior. Similarly
product-builds have a flag "use_newest_version" that can be set
to true. pkglistgen needs to know about these variant as well
to properly resolve package dependencies.
2021-11-04 20:47:23 +01:00
Stephan Kulow
0d7770bce0 pkglistgen: Skip repos without state
This happens for repos we only inherit for the project config but that
do not build for said architecture
2021-10-27 09:16:25 +02:00
Stephan Kulow
775f9a42a2 pkglistgen: Keep caching repo solv files but copy them away
Not caching them kills botmaster's performance. So we need to keep
that but try to avoid races differently by copying the files from
cachedir to local dir.
2021-10-26 18:48:10 +02:00
Stephan Kulow
30daff5613 pkglistgen: Skip relying on the global cache directory
The weakremovers.inc is still flickering and the only explanation left
is that other pipelines break the solv file during the (quite long)
runtime of the pkglistgen pipeline.

So stop relying on the solv file in the global cache directory - the
repo itself is not as dangerous, but the solv should live in the working
directory while it's running
2021-10-26 08:27:01 +02:00
Stephan Kulow
241f7a966a pkglistgen: Do not generate weakremovers if in release package mode
Technically the weakremovers.inc is part of the release package, but
it can only be calculated in the context of the full product build
2021-10-24 09:05:33 +02:00
Stephan Kulow
de699c125e pkglistgen: Accept i686 as well 2021-09-21 13:54:10 +02:00
Stephan Kulow
ad7e5f960e pkglistgen: replace if not a in with if a not in 2021-09-21 13:53:22 +02:00
Stephan Kulow
35eeb2af2b pkglistgen: Fix architecture handling of weakremovers
15.3 introduced something unexpected: aarch64, ppc64le and s390x are part
of the repos - and all architectures that aren't native to openSUSE:Factory
are considered to bear dropped packages, e.g. *all of them*

On top of that the setdefault using the same hash caused packages to come
and go for repos, creating unstable output
2021-09-21 12:29:12 +02:00
Stephan Kulow
8da0139b93 pkglistgen: Force outdir for the checkouts
Otherwise the osc config for checkout_no_colon will interfer where the
files end up and then we can't open them reliable.

Fixes #2559
2021-05-10 12:21:55 +02:00
Richard Brown
b0817c1741 Revert pkglistgen: ignore bootstrap_copy repo 2021-04-08 12:29:02 +02:00
Ludwig Nussel
e15419a753 pkglistgen: don't die on unset pkglistgen-ignore_repos 2021-04-06 15:18:26 +02:00
Stephan Kulow
6b65b00adf
Merge pull request #2539 from lnussel/master
pkglistgen: ignore bootstrap_copy repo
2021-04-06 07:55:08 +02:00
Stephan Kulow
35ff3f9e34 pkglistgen: After solving check all packages listed
If the packages are already part of the modules that are excluded
(due to dependencies), output an ERROR (which is not fatal atm)

Fixes #2541
2021-04-01 15:48:37 +02:00
Ludwig Nussel
1cf026aef5 pkglistgen: ignore bootstrap_copy repo
Never use the bootstrap_copy repo for package solving. Staging content
might be very different so don't confuse the solver.
2021-03-29 11:44:19 +02:00
Fabian Vogt
440b0f7def Fix repo state error handling in pkglistgen
repository_arch_state doesn't raise HTTPError for 404 any more, but that's what
pkglistgen uses to skip that repo/arch. Handle None as replacement instead.
2021-02-15 14:43:37 +01:00
Stephan Kulow
ad9119d4b2 Calculate the i586 DVD as i686 package set
mozjs is now i686 exclusive and GNOME and KDE depends on it
2021-01-22 11:24:39 +01:00
Stephan Kulow
75c001c96d
Merge pull request #2520 from coolo/fix_2517
pkglistgen: Check the return code of Popen calls
2021-01-19 16:21:22 +01:00
Stephan Kulow
279c95eaf9 pkglistgen: Check the return code of Popen calls
Not updating the product in case of failure is preferred over creating
bogus products

Fixes #2517
2021-01-19 11:41:21 +01:00
Stephan Kulow
2bb1d8a1ce pkglistgen: Allow to set require_all for some groups to disable ignore
For Factory we want the DVD to catch all (and only require some), but the
smaller products kubic and microos should have a require all policy
2021-01-15 15:52:54 +01:00
3e70de6f4a Check for 'weakremovers.inc' in in group_dir before attempting to move it
Signed-off-by: Egbert Eich <eich@suse.com>
2020-05-02 11:43:19 +02:00
32ad0adc2d Fix all E22x errors
This cleans up white spaces around various kinds of operators to be
consistent.
2019-11-27 11:08:06 +01:00
Stephan Kulow
657d2c87d9 Fix conflicts/obsoletes while picking dependencies
We ignored conflicts/obsoletes during pool preparation as it got
into the way of finding supplements (due to conflicting packages in
the list). But this way, conflicts were invisible also during dependency
search, leading to wrong packages picked for the package lists in
general.

So add a flag to prepare_pool to explain if we want conflicts or not
2019-11-05 09:39:10 +01:00
7b8d295abd pkglistgen: fix wrong parsing way on pkglistgen-locale
Since the data is from project attribute, pkglistgen should not parsing a string.
This change also renamed pkglistgen-local to pkglistgen-locale.
2019-10-15 19:17:57 +08:00
Stephan Kulow
778e82b044 Fix some python2/python3 confusion with pkglistgen (for sle) 2019-07-05 09:47:57 +02:00
Stephan Kulow
29bd141145 Sort architectures to fix #2021 2019-05-16 07:30:21 +02:00
Stephan Kulow
1b20c92d41 pkglistgen: Output strings on the console 2019-05-15 08:49:30 +02:00
Stephan Kulow
c043a2d537 Even more fixes for pkglistgen on python3 2019-05-13 14:09:57 +02:00
Stephan Kulow
df8de99945 Even more python3 fixes for pkglistgen 2019-05-13 08:57:34 +02:00
Stephan Kulow
d0eef97d27 Fix problems with pkglistgen under python3 2019-05-12 21:22:07 +02:00
Stephan Kulow
fec762e5a4 Avoid warning on using .warn (python3) 2019-05-11 14:25:02 +02:00
Stephan Kulow
57f15a4609 Avoid gocd warning on 9 minutes without print
the weak remover code is a little slow, but it's also a lot of data
2019-03-20 15:04:23 +01:00
Stephan Kulow
dff9bddfc4 Replace weakremovers.inc in 000release-packages
But skip it during --only-release-packages as we have to assume the
binaries aren't final
2019-01-21 16:55:19 +01:00