git-workflow/giwtf
SHA256
16
0
Fork 1
Code Pull Requests Activity
Files
main
giwtf/test-plan.md
Andrii Nikitin 5e2f88a57b Init version
2026-02-11 12:22:17 +01:00

9.8 KiB
Raw Permalink Blame History

Test Plan: workflow-pr Bot

1. Introduction

This document outlines the test plan for the workflow-pr bot. The bot is responsible for synchronizing pull requests between ProjectGit and PackageGit repositories, managing reviews, and handling merges. This test plan aims to ensure the bot's functionality, reliability, and performance.

2. Scope

In Scope

  • Pull Request synchronization (creation, update, closing).
  • Reviewer management (adding, re-adding, mandatory vs. advisory).
  • Merge management, including ManualMergeOnly and ManualMergeProject flags.
  • Configuration parsing (workflow.config).
  • Label management (staging/Auto, review/Pending, review/Done).
  • Maintainership and permissions handling.

Out of Scope

  • Package deletion requests (planned feature).
  • Underlying infrastructure (Gitea, RabbitMQ, OBS).
  • Performance and load testing.
  • Closing a PackageGit PR (currently disabled).

3. Test Objectives

  • Verify that pull requests are correctly synchronized between ProjectGit and PackageGit.
  • Ensure that reviewers are correctly added to pull requests based on the configuration.
  • Validate that pull requests are merged only when all conditions are met.
  • Confirm that the bot correctly handles various configurations in workflow.config.
  • Verify that labels are correctly applied to pull requests.
  • Ensure that maintainership and permissions are correctly enforced.

4. Test Strategy

The testing will be conducted in a dedicated test environment that mimics the production environment. The strategy will involve a combination of:

  • Component Testing: Testing individual components of the bot in isolation using unit tests written in Go.
  • Integration Testing: Testing the bot's interaction with Gitea, RabbitMQ, and a mock OBS server using pytest.
  • End-to-End Testing: Testing the complete workflow from creating a pull request to merging it using pytest.

Test Automation

  • Unit Tests: Go's built-in testing framework will be used to write unit tests for individual functions and methods.
  • Integration and End-to-End Tests: pytest will be used to write integration and end-to-end tests that use the Gitea API to create pull requests and verify the bot's behavior.

Success Metrics

  • Test Coverage: The goal is to achieve at least 80% test coverage for the bot's codebase.
  • Bug Detection Rate: The number of bugs found during the testing phase.
  • Test Pass Rate: The percentage of test cases that pass without any issues.

5. Test Cases

Test Case ID Description Steps to Reproduce Expected Results Priority
TC-SYNC-001 Create ProjectGit PR from PackageGit PR 1. Create a new PR in a PackageGit repository. 1. A new PR is created in the corresponding ProjectGit repository with the title "Forwarded PRs: <package_name>".
2. The ProjectGit PR description contains a link to the PackageGit PR (e.g., PR: org/package_repo!pr_number).
3. The package submodule in the ProjectGit PR points to the PackageGit PR's commit.		 High
TC-SYNC-002 Update ProjectGit PR from PackageGit PR 1. Push a new commit to an existing PackageGit PR. 1. The corresponding ProjectGit PR's head branch is updated with the new commit. High
TC-SYNC-003 WIP Flag Synchronization 1. Mark a PackageGit PR as "Work In Progress".
2. Remove the WIP flag from the PackageGit PR.		 1. The corresponding ProjectGit PR is also marked as "Work In Progress".
2. The WIP flag on the ProjectGit PR is removed.		 Medium
TC-SYNC-004 WIP Flag (multiple referenced package PRs) 1. Create a ProjectGit PR that references multiple PackageGit PRs.
2. Mark one of the PackageGit PRs as "Work In Progress".
3. Remove the "Work In Progress" flag from all PackageGit PRs.		 1. The ProjectGit PR is marked as "Work In Progress".
2. The "Work In Progress" flag is removed from the ProjectGit PR only after it has been removed from all associated PackageGit PRs.		 Medium
TC-SYNC-005 NoProjectGitPR = true, edits disabled 1. Set NoProjectGitPR = true in workflow.config.
2. Create a PackageGit PR without "Allow edits from maintainers" enabled.
3. Push a new commit to the PackageGit PR.		 1. No ProjectGit PR is created.
2. The bot adds a warning comment to the PackageGit PR explaining that it cannot update the PR.		 High
TC-SYNC-006 NoProjectGitPR = true, edits enabled 1. Set NoProjectGitPR = true in workflow.config.
2. Create a PackageGit PR with "Allow edits from maintainers" enabled.
3. Push a new commit to the PackageGit PR.		 1. No ProjectGit PR is created.
2. The submodule commit on the project PR is updated with the new commit from the PackageGit PR.		 High
TC-COMMENT-001 Detect duplicate comments 1. Create a PackageGit PR.
2. Wait for the workflow-pr bot to act on the PR.
3. Edit the body of the PR to trigger the bot a second time.		 1. The bot should not post a duplicate comment. High
TC-REVIEW-001 Add mandatory reviewers 1. Create a new PackageGit PR. 1. All mandatory reviewers are added to both the PackageGit and ProjectGit PRs. High
TC-REVIEW-002 Add advisory reviewers 1. Create a new PackageGit PR with advisory reviewers defined in the configuration. 1. Advisory reviewers are added to the PR, but their approval is not required for merging. Medium
TC-REVIEW-003 Re-add reviewers 1. Push a new commit to a PackageGit PR after it has been approved. 1. The original reviewers are re-added to the PR. Medium
TC-REVIEW-004 Package PR created by a maintainer 1. Create a PackageGit PR from the account of a package maintainer. 1. No review is requested from other package maintainers. High
TC-REVIEW-005 Package PR created by an external user (approve) 1. Create a PackageGit PR from the account of a user who is not a package maintainer.
2. One of the package maintainers approves the PR.		 1. All package maintainers are added as reviewers.
2. Once one maintainer approves the PR, the other maintainers are removed as reviewers.		 High
TC-REVIEW-006 Package PR created by an external user (reject) 1. Create a PackageGit PR from the account of a user who is not a package maintainer.
2. One of the package maintainers rejects the PR.		 1. All package maintainers are added as reviewers.
2. Once one maintainer rejects the PR, the other maintainers are removed as reviewers.		 High
TC-REVIEW-007 Package PR created by a maintainer with ReviewRequired=true 1. Set ReviewRequired = true in workflow.config.
2. Create a PackageGit PR from the account of a package maintainer.		 1. A review is requested from other package maintainers if available. High
TC-MERGE-001 Automatic Merge 1. Create a PackageGit PR.
2. Ensure all mandatory reviews are completed on both project and package PRs.		 1. The PR is automatically merged. High
TC-MERGE-002 ManualMergeOnly with Package Maintainer 1. Create a PackageGit PR with ManualMergeOnly set to true.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the package PR from the account of a package maintainer for that package.		 1. The PR is merged. High
TC-MERGE-003 ManualMergeOnly with unauthorized user 1. Create a PackageGit PR with ManualMergeOnly set to true.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the package PR from the account of a user who is not a maintainer for that package.		 1. The PR is not merged. High
TC-MERGE-004 ManualMergeOnly with multiple packages 1. Create a ProjectGit PR that references multiple PackageGit PRs with ManualMergeOnly set to true.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on each package PR from the account of a package maintainer.		 1. The PR is merged only after "merge ok" is commented on all associated PackageGit PRs. High
TC-MERGE-005 ManualMergeOnly with Project Maintainer 1. Create a PackageGit PR with ManualMergeOnly set to true.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the package PR from the account of a project maintainer.		 1. The PR is merged. High
TC-MERGE-006 ManualMergeProject with Project Maintainer 1. Create a PackageGit PR with ManualMergeProject set to true.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the project PR from the account of a project maintainer.		 1. The PR is merged. High
TC-MERGE-007 ManualMergeProject with unauthorized user 1. Create a PackageGit PR with ManualMergeProject set to true.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the project PR from the account of a package maintainer.		 1. The PR is not merged. High
TC-CONFIG-001 Invalid Configuration 1. Provide an invalid workflow.config file. 1. The bot reports an error and does not process any PRs. High
TC-LABEL-001 Apply staging/Auto label 1. Create a new PackageGit PR. 1. The staging/Auto label is applied to the ProjectGit PR. High
TC-LABEL-002 Apply review/Pending label 1. Create a new PackageGit PR. 1. The review/Pending label is applied to the ProjectGit PR when there are pending reviews. Medium
TC-LABEL-003 Apply review/Done label 1. Ensure all mandatory reviews for a PR are completed. 1. The review/Done label is applied to the ProjectGit PR when all mandatory reviews are completed. Medium
View Git Blame Copy Permalink