138 Commits

Author SHA1 Message Date
Stephan Kulow
c4862cd75a pkglistgen: Add --staging option for easier scoping
To run on gitlab worker, it's easier to just use the project name
as argument, as this is also what we pass to other jobs
2018-12-03 14:46:21 +01:00
Stephan Kulow
1f0e92c643
Merge pull request #1799 from nilxam/correct_repos_handling
pkglistgen: for nonefree repo handling should update to tool's repos rather than commandline's repos
2018-11-20 15:14:58 +01:00
9ad6881a46 pkglistgen: nonefree repo handling should update to tool's repos rather than commandline's repos 2018-11-20 21:55:39 +08:00
Stephan Kulow
40513d9503 Import url functions properly for python3
Make it possible to run pkglistgen under python3
2018-11-17 10:16:14 +01:00
Stephan Kulow
a47ffacfc1 pkglistgen: Support a recommends flag in the OUTPUT 2018-11-12 11:50:57 +01:00
Jimmy Berry
4c8ade1a93 flake8: utilize raw strings to avoid invalid escape character warnings. 2018-10-24 16:34:16 -05:00
e6e2f5903b pkglistgen: fix AttributeError since update_repos() moved to PkgListGen class 2018-10-21 19:21:19 +08:00
Stephan Kulow
dc3201d238 Move unlink_list to pkglistgen class too
It's used in update_repos
2018-10-11 07:43:36 +02:00
Stephan Kulow
e70c071ce9 Move update_repos into Pkglistgen class
We need to track state within the tool, not in the command line
interface
2018-10-11 07:43:36 +02:00
Stephan Kulow
b8e451f8d1 Don't make missing repos (per arch) a fatal error
It's possible we link against repos that do not exist for all archs
2018-10-11 07:43:35 +02:00
Stephan Kulow
59d9454134 pkglistgen: Fix copy&paste error on uploading results 2018-10-10 10:46:45 +02:00
Stephan Kulow
fb4d63ec3d Fix overlap calculation: modules only have filtered archs 2018-10-10 06:45:14 +02:00
Stephan Kulow
a96e864c06 pkglistgen: Make sure we don't calculate non-existant architectures in stagings
Otherwise the package list is not staging specific
2018-10-04 07:00:56 +02:00
Stephan Kulow
c4fa49aef2 Refactor conditional_PUT as core function
Introduce source_file_ensure in osclib.core following review in #1717
2018-10-03 07:36:42 +02:00
Stephan Kulow
2d4148f3f2 pkglistgen: only PUT new data if it changed
Otherwise we collect quickly a million revisions with no gain
2018-10-02 06:48:58 +02:00
Stephan Kulow
f3961e4857 pkglistgen: Only check repository check per arch
(also indent 2 lines due to calling autopep)
2018-09-28 08:05:15 +02:00
Stephan Kulow
5a1307b98b Move creation of the Solver object out of loop
After time for sle_module_basesystem: before 16.5s, after 9.7s
2018-09-27 07:07:05 +02:00
Stephan Kulow
ea0e6d4aaa Don't take unsorted.group as reference for SP1 product changes
The release managers are actually supposed to maintain the filter list
in unneeded.yml, but that only applies to unsorted.yml not to the unsorted.group
(which is basically unfiltered to be published into OBS's SLE-15)
2018-09-24 15:33:10 +02:00
Jimmy Berry
a074572c45 pkglistgen: compare repository state to determine when to update solv.
Since both pkglistgen and repo_checker share the same rpm metadata cache
either may update it first and thus bs_mirrorfull will only contain output
for one of the tools. As such the check in pkglistgen could cause the solv
file to remain stale. Instead write a secondary file containing the
repository state hash and check to see if the file already exists.
2018-09-20 16:38:05 -05:00
Jimmy Berry
c47bb4c902 Utilize CacheManager for all existing caches. 2018-09-04 15:19:02 -05:00
Jimmy Berry
6069245350 Remove SUSE copyright, warranty, and license headers.
Distinct copyrights were left as I do not wish to track down commit
history to ensure it properly documents the copyright holders. Also left
non-GPLv2 licenses and left bs_copy untouched as a mirror from OBS.

Already have a mix of with and without headers and even OBS does not place
on majority of files. If SUSE lawyers have an issue it will come up in
legal review for Factory.
2018-08-23 19:18:06 -05:00
Jimmy Berry
36a5a6c2e4 pkglistgen: utilize osclib.core.repository_path_expand(). 2018-08-21 20:49:31 -05:00
Jimmy Berry
760ddf39e6 osclib/conf: decouple from StagingAPI and always fetch remote config.
As the remote config is no longer optional for SLE and is utilized by
openSUSE to the point were it is dangerous not to load the remote config
it should be required. Currently only certain users call apply_remote()
while this will make it built-in during construction and thus makes the
usage consistent and no longer require StagingAPI.
2018-08-16 21:53:40 -05:00
8129bef009 pkglistgen: fix calling expand_repos() in CommandLineInterface 2018-08-03 16:21:16 +08:00
ec019c42ce pkglistgen: respect main_repo setup for other variant
Respect main_repo setup for other variant eg. ARM. Also removed ports
from SCOPES.
2018-08-03 16:21:16 +08:00
Stephan Kulow
af1252ef3e pkglistgen: Support passing multiple scopes
For SLE I need -s target -s staging, but it only did the later - as
I didn't figure the concept of commandline :)
2018-08-01 15:46:50 +02:00
Stephan Kulow
fb114a75d4 pkglistgen: Commit a summary.yml into 000product-summary if existant 2018-07-19 21:55:52 +02:00
Stephan Kulow
57fe0871dc pkglist: Review the proper package (000release-packages) 2018-07-19 21:55:52 +02:00
Stephan Kulow
9e23eab573 pkglistgen: allow to pass staging:a as scope 2018-07-19 21:55:50 +02:00
Stephan Kulow
715d94c01a pkglistgen: remove default-support-status option
The OUTPUT groups need to specify the status now or it will default to
unsupported (which should still be fine for leap/factory)
2018-07-19 21:55:14 +02:00
Stephan Kulow
9951b454fb
Merge pull request #1620 from coolo/fix_repo_checker
repo_checker: Fix the review loop
2018-07-19 21:54:39 +02:00
Stephan Kulow
4f275a5af8 Move expanded_repos into staging api
and make pkglistgen and repo_checker use it. We will have more
cases in the future of tools requiring it :)
2018-07-19 09:14:30 +02:00
4f58794f8c Remove debug code accidentally part of #1615
Apparently accidentally did a commit --amend --all on last commit,
revert.
2018-07-17 10:03:00 +02:00
d113e01bbb Add Leap 15.1 ARM support 2018-07-16 15:51:47 +02:00
Stephan Kulow
29d6152fa4 pkglistgen: Don't crash on empty groups (unsorted is one) 2018-07-03 15:35:44 +02:00
Stephan Kulow
91a8b5dfa6 pkglistgen: Add --stop-after-solve option to ease debugging 2018-07-03 15:35:44 +02:00
Stephan Kulow
c1177170d5 pkglistgen: Add --no-checkout option to accelerate reruns 2018-07-03 15:35:44 +02:00
Stephan Kulow
121cec768b pkglistgen: Calculate the repos list instead of hardcoding 2018-07-03 15:35:44 +02:00
Stephan Kulow
3efedd4096 pkglistgen: drop ring 2 2018-07-03 14:10:10 +02:00
Stephan Kulow
03ea996ba4 pkglistgen: Don't warn about locked packages not found
We sometimes have to lock packages that only exist on one arch - and
that makes debug output unreadable. So be careful with locked packages
instead :)
2018-06-19 07:21:52 +02:00
Stephan Kulow
f8ff45896a pkglistgen: Copy away inc files too 2018-06-19 07:21:52 +02:00
Stephan Kulow
e452271e5e pkglistgen: Output more infos about overlap 2018-06-19 07:21:52 +02:00
Stephan Kulow
a68be43d04 Generate an unsorted.group if configured
We need this to have the remains of SLE15 in OBS to build against
2018-06-15 17:13:10 +02:00
Ludwig Nussel
a3f320c8be
Merge pull request #1540 from lnussel/pkglistgen
pkglistgen: log unsorted locale packages
2018-06-11 09:56:29 +02:00
086bd38980 Skip nonfree for ports scope
We don't actually have any nonfree items for ports, this is a x86_64
specific thing. so skip it for ports - makes no sense only eats time.
2018-05-15 11:58:39 +02:00
Ludwig Nussel
5311708270 pkglistgen: log unsorted locale packages
For some yet to be determined reason, the namespaceprovider does not
pick up package that just provide eg. locale(ja). So find and and log
them (boo#1092055)
2018-05-14 15:21:46 +02:00
Jimmy Berry
547703d92a pkglistgen: do_update_and_solve(): print exception and continue for all projects. 2018-04-12 00:08:43 -05:00
Ludwig Nussel
5d0c024ffe
Merge pull request #1453 from coolo/add_sle_droplist
Add create_sle_weakremovers command
2018-03-23 11:21:43 +01:00
Jimmy Berry
9a26912ab5
Merge pull request #1442 from jberry-suse/pkglistgen-update-hash
pkglistgen: do_dump_solv(): include hash in update solv to avoid download.
2018-03-12 15:55:44 -05:00
Stephan Kulow
78b107e6e5 Add create_sle_weakremovers command
It's similiar but not the same as the droplist for openSUSE, so I created
a fork of the openSUSE command

To be called with the target project as first argument and then a
list of old projects

e.g. create_sle_weakremovers SUSE:SLE-15:GA SUSE:SLE-12-SP3:GA \
SUSE:SLE-12-SP3:Update SUSE:SLE-12-SP2:GA SUSE:SLE-12-SP2:Update ...
2018-03-12 08:20:42 +01:00