The summary dictionary used to create summary-{scope}.txt was
previously returned by solve_project() via write_all_groups().
After c46dd3e304d8e66e5f8f6a51bfa3ebda8cba8936, solve_project()
doesn't return anything anymore, as write_all_groups() is being
invoked just after solve_project().
Thus, the "summary" dictionary would always be None.
Since the summary dict creation is generic enough, move it
to its own function, and call it afterwards after project
solving.
This commit also drops the return statement both on write_all_groups()
and write_productcompose() (where it would always return an empty
dictionary), and moves the write_productcompose() call after
solve_project() as well to match the change with write_all_groups().
Signed-off-by: Eugenio Paolantonio <eugenio.paolantonio@suse.com>
Previously pkglistgen.py assumed that a project either used
product-builder or the new product-composer, by exiting unrecoverably
when a *.productcompose.in file was found but the relative
000productcompose directory was absent.
This is not the case for SL Micro as the shared SLFO codebase is still
an hybrid OBS/git project and hence its stagings are not git based and
can't use product composer.
This patch was tested for the SUSE:ALP:Source:Standard:1.0:Staging:A and
SUSE:ALP:Products:Marble:6.0 projects (together with
https://github.com/openSUSE/openSUSE-release-tools/pull/3078). Instead
of raising an exception when the 000productcompose directory is not
found it makes a log stating that product-composer will be skipped.
release rpm files currently do not have .changes files, but these are
needed to extract SOURCE_DATE_EPOCH for reproducible builds.
Using the time stamp of the last commit to any of the input, which is
the 000package-groups and 000update-repos source package, as the time
for the changes entry would be reproducible, however there is resistance
to that.
The other date that is related is the time of the last
source change for any input to the whole distribution, however AFAIK
this date is not cheap enough to compute for OBS-VCS. This hopefully
changes once we move to git.
However to not delay making progress with reproducible builds in
Factory, I propose this change which makes the output of this script
non-reproducible, by using the current wall-clock time. But only when
this script produced a change to be commited. The release binary rpms
will still be reproducible. We can revisit making the generated source
reproducible, later.
See https://reproducible-builds.org/ for more general information on
this topic.
- Create custom-cache-tag for pkglistgen to enable separate cache dir
name in case of parallel running
- Add custom-cache-tag to SLSA services
- Check for systemd service instead of process to ensure that SLSA
services will not run in parallel
- Change pkglisgen timer to avoid overlap with relpkggen timer
This has multiple benefits:
* Easier to interface with the python code that calls it
* Uses the osc python interface for connection and authentication handling
* Allows to drop the bundled OBS code in the bs_copy directory
We will check if there are approved pkglistgen comments and if so, apply
them to the target. For this we need to parse the generated markdown, so
refactor both in a common class.
There diff is applied, but not yet commited in this commit
As we deprecated 'all' we can just as well rely on only one project
given (which is also what we do in gocd) to ease error reporting.
The old method of running all projects in a loop stemed from the
old pkglistgen not having a scheduler so we relied on serial execution.
Since 30daff5613803667be7422056431116e84c07d59 we check if the repos are
still what we expected them to be when we started and stop calculating
weakremovers.inc if the build didn't stay still for the full runtime of
the pkglistgen. But the exception makes it impossible to spot real
failures - so make this a regular exit
In some cases we'd like to chose from all available repositories
for selecting the right package. in appliance builds and there
is an OBS-UseUnorderedRepos flag for enabling this behavior. Similarly
product-builds have a flag "use_newest_version" that can be set
to true. pkglistgen needs to know about these variant as well
to properly resolve package dependencies.
Not caching them kills botmaster's performance. So we need to keep
that but try to avoid races differently by copying the files from
cachedir to local dir.
The weakremovers.inc is still flickering and the only explanation left
is that other pipelines break the solv file during the (quite long)
runtime of the pkglistgen pipeline.
So stop relying on the solv file in the global cache directory - the
repo itself is not as dangerous, but the solv should live in the working
directory while it's running
