# Contents This document describes in detail the contents of this repository. It is still a *work in progress*, but do not hesitate to report if something is missing. ## Overview The repository contains a set of tools to aid in the process of building, testing and releasing (open)SUSE based distributions. The [Tools](#tools) section enumerates and describes all these tools, including links to documentation, source code and some information about where they are used. Apart from these tools, the repository includes: * Some documentation in the [docs](docs) directory. * A Python module called [osclib](osclib) which includes code that is shared by several tools. They are available in the `osclib` package. * A Docker-based [tests suite](tests). The Docker manifests and the Docker Compose files are located in the [dist](dist) directory. * [GoCD](https://www.gocd.org) configuration files in [gocd](gocd). GoCD is an open source CI/CD server that is used to deploy the bots on OBS. * Several [systemd](systemd) units: the Metrics instance makes use of them. ## Tools Most of these tools are available as packages for several distributions. Check the [spec file in this repository](dist/package/openSUSE-release-tools.spec) or the [devel project](https://build.opensuse.org/package/show/openSUSE:Tools/openSUSE-release-tools) for further information. For the time being, we have classified them into three different groups: *command line tools*, *OBS bots* and *osc plugins*. Bear in mind that the information in the following list might be wrong and incomplete. ### Command Line Tools Usually, the executables are renamed as `osrt-NAME` (e.g., `osrt-announcer`). #### announcer Generates email diffs summaries to announce product releases. * Sources: [factory-package-news/announcer.py](factory-package-news/announcer.py) * Documentation: [factory-package-news/README.asciidoc](factory-package-news/README.asciidoc) * Package: openSUSE-release-tools-announcer * Usage: [gocd](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+announcer) #### pkglistgen Generates and updates OBS products for openSUSE and SLE. It generates package lists based on `000package-groups` and puts them in `000product` (resulting kiwi files) and `000release-packages` (release package spec files). * Sources: [pkglistgen.py](pkglistgen.py) * Documentation: [docs/pkglistgen.md](docs/pkglistgen.md) * Package: openSUSE-release-tools-pkglistgen * Usage: [gocd](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+pkglistgen) #### container_cleaner Clean old containers from a given project like [openSUSE:Containers:Tumbleweed](https://build.opensuse.org/project/show/openSUSE:Containers:Tumbleweed). Only those containers providing binaries to the latest five versions for each architecture are kept. * Sources: [container_cleaner.py](container_cleaner.py) * Documentation: -- * Package: openSUSE-release-tools * Usage: [gocd](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+container_cleaner) #### metrics Generates insightful metrics from relevant OBS and annotation data, based on InfluxDB and Grafana. See . * Sources: [metrics.py](metrics.py) * Documentation: [docs/metrics.md](./docs/metrics.md) * Package: openSUSE-release-tools-metrics * Usage: ? #### metrics-access Ingests `download.opensuse.org` Apache access logs and generates metrics. It is composed of a PHP script and a set of [systemd units](systemd). * Sources: [metrics/access/aggregate.php](metrics/access/aggregate.php) * Documentation: [docs/metrics.md](./docs/metrics.md) * Package: openSUSE-release-tools-metrics-access * Usage: ? #### totest-manager Releases distribution snapshots to openQA and publishes if the result is positive. * Sources: [totest-manager.py](totest-manager.py) and [ttm](ttm) * Documentation: [ttm/README.md](ttm/README.md) * Package: openSUSE-release-tools * Usage: [gocd](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+totest-manager) #### bugowner Manages bugowner information * Sources: [bugowner.py](bugowner.py) * Documentation: -- * Package: openSUSE-release-tools * Usage: ? #### bs_mirrorfull Mirrors repositories from the build service to a local directory. * Souces: [bs_mirrorfull](bs_mirrorfull) * Documentation: -- * Package: openSUSE-release-tools * Usage: Used by other tools like `pkglistgen` or `repocheck` #### biarchtool Manages biarch packages * Source: [biarchtool.py](biarchtool.py) * Documentation: -- * Package: openSUSE-release-tools * Usage: ? #### build-fail-reminder Sends e-mails about packages failing to build for a long time. * Sources: [build-fail-reminder.py](build-fail-reminder.py) * Documentation: -- * Package: openSUSE-release-tools * Usage: [gocd](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+build-fail-reminder) #### checknewer Checks if all packages in a repository are newer than all other repositories. * Sources: [checknewer.py](checknewer.py) * Documentation: -- * Package: openSUSE-release-tools * Usage: ? #### deptool Assists in debugging dependencies * Sources: [deptool.py](deptool.py) * Documentation: -- * Package: openSUSE-release-tools * Usage: ? #### requestfinder Allows to retrieve requests from OBS with quite elaborated queries. * Sources: [requestfinder.py](requestfinder.py) * Documentation: -- * Package: openSUSE-release-tools * Usage: ? #### create_staging Scripts and templates to create staging projects. * Sources: [staging_templates](staging_templates) * Documentation: -- * Package: -- * Usage: ? ### Bots #### check_maintenance_incidents Handles maintenance incident requests * Sources: [check_maintenance_incidents.py](check_maintenance_incidents.py) * Documentation: [docs/maintbot.asciidoc](docs/maintbot.asciidoc) * Package: openSUSE-release-tools-maintenance * Usage: obsolete (by origin-manager) #### origin-manager Keeps track of from what project a package originates, submit updates, review requests to detect origin changes, and enforce origin specific policies like adding appropriate reviews * Sources: [origin-manager.py](origin-manager.py) and [web](web) * Documentation: [docs/origin-manager.md](docs/origin-manager.md) * Package: openSUSE-release-tools-origin-manager * Usage: [gocd](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+origin-manager) #### staging-bot Assists in management of staging projects. * Sources: [devel-project.py][devel-project], [staging-report.py](staging-report.py), [suppkg_rebuild.py](suppkg_rebuild.py). * Documentation: -- * Package: openSUSE-release-tools-staging-bot * Usage: gocd ([staging-report.py](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+staging-report) [suppkg_rebuild.py](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+suppkg_rebuild), etc.) #### legal-auto Makes automatic legal reviews based on the legaldb API * Sources: [legal-auto.py](legal-auto.py) * Documentation: -- * Package: openSUSE-release-tools * Usage: [gocd](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+legal-auto) #### check_tags_in_requests Checks that a submit request has correct tags specified. * Sources: [check_tags_in_requests.py](check_tags_in_requests.py) * Documentation: -- * Package: openSUSE-release-tools * Usage: [gocd](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+check_tags_in_requests) #### abichecker Checks ABI compatibility in OBS requests. * Sources: [abichecker](abichecker) * Documentation: -- * Package: openSUSE-release-tools-abichecker * Usage: gocd? #### check_source_in_factory Checks if the sources of a submission are either in Factory or a request for Factory with the same sources exist. Not used as a standalone bot anymore, but called internally from check_tags_in_requests. * Sources: [check_source_in_factory.py](check_source_in_factory.py) * Documentation: [docs/factory-source.asciidoc](docs/factory-source.asciidoc) * Package: openSUSE-release-tools * Usage: used from other bots (check_tags_in_requests) #### openqa-maintenance OpenQA stuff, not sure about the details. * Sources: [openqa-maintenance.py](openqa-maintenance.py) and [oqamaint](oqamaint) * Documentation: -- * Package: openSUSE-release-tools * Usage: [gocd](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+openqa-maintenance) #### repo-checker Inspects built RPMs from staging projects. * Sources: [project-installcheck.py](project-installcheck.py), [staging-installcheck.py](staging-installcheck.py), [maintenance-installcheck.py](maintenance-installcheck.py), [findfileconflicts](findfileconflicts), [write_repo_susetags_file.pl](write_repo_susetags_file.pl) * Documentation: -- * Package: openSUSE-release-tools-repo-checker * Usage: gocd ([project-installcheck.py](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+project-installcheck), [staging-installcheck](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+staging-installcheck) and [maintenance-installcheck.py](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+maintenance-installcheck) ### OSC Plugins #### osc-check_source.py Checks for usual mistakes and problems in the source packages submitted by users. Used also as review bot that assigns reviews (?). * Sources: [check_source.py](check_source.py) and [check_source.pl](check_source.pl) * Documentation: [docs/check_source.asciidoc](docs/check_source.asciidoc) * Package: openSUSE-release-tools-check-source * Usage: [gocd](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+check_source) #### osc-origin.py Tools for working with origin information. * Sources: [osc-origin.py](osc-origin.py) * Documentation: [docs/origin-manager.md](docs/origin-manager.md) * Package: openSUSE-release-tools-origin-manager * Usage: [gocd](https://github.com/openSUSE/openSUSE-release-tools/search?q=path%3A%2Fgocd+osc-origin) #### osc-cycle.py Helps with OBS build cycles visualization. See the [openSUSE:Factory/standard example](https://build.opensuse.org/project/repository_state/openSUSE:Factory/standard). * Sources: [osc-cycle.py](osc-cycle.py) * Documentation: -- * Package: -- * Usage: used to debug problems. See https://github.com/openSUSE/openSUSE-release-tools/pull/992 as an example. #### compare_pkglist.py Compares packages status between two projects. It determines which project has the newer version of a package, shows the diff, etc. Additionally, it is able to create a submit request from SOURCE to TARGET in case packages are different. * Sources: [compare_pkglist.py](compare_pkglist.py) * Documentation: -- * Package: openSUSE-release-tools * Usage: ??? #### staging Manages staging projects. * Sources: [osc-staging.py](osc-staging.py) * Documentation: [docs/staging.asciidoc](docs/staging.asciidoc) and [docs/testing.asciidoc](docs/testing.asciidoc) * Package: osc-plugin-staging * Usage: staging projects management #### fcc_submitter.py The FactoryCandidates projects are used to determine whether a new package in Factory does build in the Leap version under development (see [openSUSE:Leap:15.2:FactoryCandidates](https://build.opensuse.org/project/show/openSUSE:Leap:15.2:FactoryCandidates) as example). This tool helps to manage this project by creating/updating project links for new packagers and creating SR from FactoryCandidates to the Leap project on successful builds. * Sources: [fcc_submitter.py](fcc_submitter.py) * Documentation: -- * Package: openSUSE-release-tools * Usage: manually #### issue-diff.py Compares packages from a project against factory for differences in referenced issues and presents changes to allow whitelisting before creating Bugzilla entries. * Sources: [issue-diff.py](issue-diff.py) * Documentation: -- * Package: openSUSE-release-tools * Usage: ???