git-workflow-documentation/architecture/hld/workflows.md

Workflows

This document outlines the workflows for managing products using git and the Open Build Service (OBS). While the current bot configuration allows for significant flexibility, this document describes the standard, recommended workflows.

Official Product Management

The primary mechanism for product management is the Pull Request (PR) workflow, managed by the workflow-pr bot. For more details on the bot, see the workflow-pr documentation.

A key distinction is made between the "development" and "maintenance" phases of a product lifecycle, primarily due to technical constraints in the maintenance phase.

Development Phase

During the development phase, changes are managed through a coordinated PR process between package repositories and the main project repository.

The workflow-pr bot is configured to monitor package repositories within a specific Gitea organization (e.g., pool) for changes on the product branch. The key steps of the workflow are:

1. The workflow-pr bot monitors package repositories for new pull requests targeting the product branch.
2. When a new PR is created in the package repository, the bot automatically creates a corresponding PR in the main project repository (unless NoProjectGitPR is set to true).
3. Reviewers, as specified in the workflow.config file, are assigned to both the project PR and the package PR. You can specify different reviewers for the project, the package, or both.
4. The bot also assigns package maintainers as reviewers on the package PR, based on the _maintainership.json file. Maintainers are only added if the PR was not opened by one of them.
5. Once a package maintainer approves the PR, the other maintainers are removed from the list of reviewers. Note: An option is planned to opt out of this behavior and always require all maintainer reviews.
6. Once all approvals are gathered, the project maintainer (defined in _maintainership.json for an empty package name) must comment "merge ok" on the project PR. This action triggers the bot to merge both the project and package PRs.

The following diagram illustrates the development phase workflow:

Maintenance Phase

The maintenance phase workflow is similar to the development phase, with one major difference: project PRs are not created automatically.

• The NoProjectGitPR option is set to true, so the workflow-pr bot does not create project PRs. These must be created manually by the maintenance team.
• Packagers must open PRs with the "Allow edits from maintainers" option enabled. This allows the workflow-pr bot to synchronize new commits from the package PR with the corresponding submodule in the main project.

Group review bot

To facilitate reviews from entire teams, especially for official products requiring formal sign-off, a group-review bot is used. This approach is necessary to overcome a limitation in Gitea's handling of team reviews; when a team is assigned as a reviewer, an approval from a team member is recorded under their individual username, not as an explicit team approval.

The group-review bot solves this by having an authorized member of a pre-configured group trigger a group approval via a comment, such as @group-review-bot: approve. The bot then posts a clear, uniform comment like "reviewed by on behalf of ", providing an unambiguous audit trail.

This solution requires a dedicated IDP account for each bot instance, so its use is reserved for situations where this level of formal, auditable team sign-off is essential, such as for certification or compliance reasons. For more details, see the group-review documentation.

Staging Projects

A key component of the review process is the use of staging projects, which are managed by the OBS Staging Bot. Typically, one of the reviewers configured in the workflow.config file is this bot.

The primary role of the staging bot is to transform a project PR into a dedicated staging project in OBS. This allows all package changes contained within that single PR to be built and tested together in an isolated environment.

The staging bot offers two main operational modes:

1. Package-Only Build: By default, the bot creates a staging project that builds only the specific packages referenced in the project PR.
2. Full Product Build: Optionally, the bot can be configured to create a complete product build that includes the newly built packages. This full build can then be handed over to the Quality Engineering (QE) team for comprehensive testing.

The bot's source code and further documentation can be found at https://src.opensuse.org/git-workflow/autogits/src/branch/main/obs-staging-bot.

Workflows

Individual Contribution Workflow

Packagers who need to modify one or more packages and test them in a dedicated OBS project can follow this workflow.

1. Fork the package: To begin, use the osc fork command to create a copy of the package in your personal OBS project:

   osc fork <OBS_PROJECT_NAME> <PACKAGE_NAME>
   

   This command automatically creates a subproject (e.g., home:<user>:branches:<PROJECT>) containing the package. The package is configured with scmsync to a forked package repository in your personal Gitea organization (e.g., src.opensuse.org/<user>/<package>) on the appropriate product branch.

2. Add more packages (optional): You can add additional packages to your project at any time:

   osc fork <OBS_PROJECT_NAME> <PACKAGE_NAME> --target-project home:<user>:branches:<OBS_PROJECT_NAME>
   

   You can also create a dedicated project in OBS from the start if you prefer not to use the default one.

3. Check out the project and make changes: Check out the project sources using osc:

   osc co home:<user>:branches:<OBS_PROJECT_NAME>
   

   Then, you can apply your changes to the package(s).

4. Commit and push changes: For each modified package, commit and push your changes:

   git add <files>
   git commit -m "Your commit message"
   git push origin <product-branch>
   

5. Create a Pull Request: Finally, use the git-obs command to create a pull request:

   git-obs -G ibs pr create --title "Your PR Title" --description "Your PR description"
   

   This will trigger the standard Development Phase workflow for the created PR.

The following diagram illustrates the individual contribution workflow:

Devel Project Workflow

This workflow is designed for developers working on large-scale projects, such as development projects or major product updates, which involve managing a high number of packages simultaneously.

Prerequisites

Before starting, ensure the following requirements are met:

• Permissions: You must have push permissions to the target project repository in Gitea.
• OBS Project: A corresponding OBS project must be set up. The project's metadata must include the scmsync tag pointing to the correct Gitea repository and branch, for example: <scmsync>https://src.opensuse.org/develOrganization/_ObsPrj#devel</scmsync>.

Workflow Steps

1. Clone the Project Repository: Clone the main project repository, which contains all the packages as submodules.

   git clone https://src.opensuse.org/develOrganization/_ObsPrj.git develProject
   cd develProject
   

2. Switch to the Development Branch: Check out the development branch. If the branch doesn't exist locally, create it.

   # Create the branch if it doesn't exist
   git checkout -b devel
   # Or, if it already exists
   git checkout devel
   

3. Initialize Package Submodules: Initialize the package submodules you need to work on.

   # To initialize only specific packages
   git submodule update --init pkgA pkgB pkgC
   
   # Or, to initialize all packages in the project
   git submodule update --init
   

4. Checkout Submodule Branches: Within each submodule, check out the development branch where you will commit your changes.

   git submodule foreach 'git checkout -b devel'
   

5. Make and Commit Changes: Make your changes to the package sources. Once ready, commit and push them from within each submodule.

   Disclaimer: Be cautious when using git add -A. Ensure that your .gitignore and .gitattributes files are configured correctly to avoid pushing unwanted or temporary files to the repository.

   git submodule foreach 'git add -A && git commit -m "upgrade pkgA and its deps to x.y" && git push origin devel'
   

6. Create the Project Pull Request: After pushing all submodule changes, commit the updated submodule references in the main project repository and create the project pull request.

   # Commit the updated submodule pointers
   git add -u
   git commit -m "upgrade pkgA to x.y"
   git push origin devel
   
   # Create the project pull request against the target branch (e.g., factory)
   git-obs -G ibs pr create --self --title "upgrade pkgA to x.y" --description "upgrade pkgA and its deps to x.y" --target-branch factory
   

7. Merge Changes (Post-Review): After the pull request has been reviewed and approved, you may need to merge the changes. The following command provides an example of how to merge the devel branch into another branch like factory across all submodules.

   git submodule foreach 'git checkout factory && git pull origin factory && git merge devel && git push origin factory'
   

8. Create Submodule Pull Requests (Post-Review): After the main project pull request has been reviewed and approved, create individual pull requests for each submodule to merge the changes from your development branch into the target pool branch (e.g., factory).

   git submodule foreach 'git-obs -G ibs pr create --title "upgrade to x.y" --target-owner pool --target-branch factory'
   

The following diagram illustrates the devel project workflow:

Direct Push Workflow

For projects where formal pull request reviews are not required for every synchronization, the workflow-direct bot provides a simpler, push-based workflow. This is often used for development projects where rapid iteration is more important than formal review for every commit.

The workflow-direct bot automates the synchronization of a main project repository (containing submodules) with its constituent package repositories.

Key Bot Actions:

• Push Events: The bot monitors all repositories in the organization defined in workflow.config. When a push is made to a configured branch in any of the submodule repositories, the bot automatically updates the corresponding submodule pointer in the main project repository to the new commit.
• Repository Creation: When a new repository is created within the organization, the bot automatically adds it as a new submodule to the main project, but only if the new repository contains the configured branch.
• Repository Deletion: When a repository is deleted, the bot automatically removes the corresponding submodule from the main project.

This workflow ensures that the main project repository is always in sync with the latest changes in the package repositories, completely bypassing the pull request process.

For more details, see the workflow-direct documentation.

Release Manager's Workflow

The release management process utilizes a set of labels to track the status of project PRs. The workflow-pr bot can be configured to automatically apply the "Backlog" label.

The primary labels are:

• staging/Backlog: This label is automatically applied by the workflow-pr bot to newly created project PRs (referencing a single package). It signifies that the PR is ready for consideration by the release team.
• staging/In Progress: Release team uses this label for project PRs that group multiple individual PRs. When "forwarded" project PRs are consolidated into a larger staging PR, the staging/Backlog label is removed and replaced with this one to indicate that it is being actively tested.
• staging/On Hold: This label is applied to pull requests that cannot be processed by the release team at the present time. This pauses the PR and indicates that it is awaiting further action or clarification before it can proceed.

Release managers often need to group multiple project PRs to test a set of changes together. The git-obs staging command provides functionality to:

• Group multiple PRs into a single staging PR.
• Remove a PR from an existing staging PR.
• Search for project PRs that have already been fully reviewed.

To further aid in this process, a staging dashboard has been created to help monitor the review and build status of these different staging projects. It is currently deployed in the internal build service and can be accessed at https://packages.suse.de.