diff --git a/integration/test-plan.md b/integration/test-plan.md index e86e84e..c9ae045 100644 --- a/integration/test-plan.md +++ b/integration/test-plan.md @@ -53,31 +53,37 @@ The testing will be conducted in a dedicated test environment that mimics the pr ## 5. Test Cases -| Test Case ID | Description | Steps to Reproduce | Expected Results | Priority | -| :--- | :--- | :--- | :--- | :--- | -| **TC-SYNC-001** | **Create ProjectGit PR from PackageGit PR** | 1. Create a new PR in a PackageGit repository. | 1. A new PR is created in the corresponding ProjectGit repository with the title "Forwarded PRs: ".
2. The ProjectGit PR description contains a link to the PackageGit PR (e.g., `PR: org/package_repo!pr_number`).
3. The package submodule in the ProjectGit PR points to the PackageGit PR's commit. | High | -| **TC-SYNC-002** | **Update ProjectGit PR from PackageGit PR** | 1. Push a new commit to an existing PackageGit PR. | 1. The corresponding ProjectGit PR's head branch is updated with the new commit. | High | -| **TC-SYNC-003** | **WIP Flag Synchronization** | 1. Mark a PackageGit PR as "Work In Progress".
2. Remove the WIP flag from the PackageGit PR. | 1. The corresponding ProjectGit PR is also marked as "Work In Progress".
2. The WIP flag on the ProjectGit PR is removed. | Medium | -| **TC-SYNC-004** | **WIP Flag (multiple referenced package PRs)** | 1. Create a ProjectGit PR that references multiple PackageGit PRs.
2. Mark one of the PackageGit PRs as "Work In Progress".
3. Remove the "Work In Progress" flag from all PackageGit PRs. | 1. The ProjectGit PR is marked as "Work In Progress".
2. The "Work In Progress" flag is removed from the ProjectGit PR only after it has been removed from all associated PackageGit PRs. | Medium | -| **TC-SYNC-005** | **NoProjectGitPR = true, edits disabled** | 1. Set `NoProjectGitPR = true` in `workflow.config`.
2. Create a PackageGit PR without "Allow edits from maintainers" enabled.
3. Push a new commit to the PackageGit PR. | 1. No ProjectGit PR is created.
2. The bot adds a warning comment to the PackageGit PR explaining that it cannot update the PR. | High | -| **TC-SYNC-006** | **NoProjectGitPR = true, edits enabled** | 1. Set `NoProjectGitPR = true` in `workflow.config`.
2. Create a PackageGit PR with "Allow edits from maintainers" enabled.
3. Push a new commit to the PackageGit PR. | 1. No ProjectGit PR is created.
2. The submodule commit on the project PR is updated with the new commit from the PackageGit PR. | High | -| **TC-COMMENT-001** | **Detect duplicate comments** | 1. Create a PackageGit PR.
2. Wait for the `workflow-pr` bot to act on the PR.
3. Edit the body of the PR to trigger the bot a second time. | 1. The bot should not post a duplicate comment. | High | -| **TC-REVIEW-001** | **Add mandatory reviewers** | 1. Create a new PackageGit PR. | 1. All mandatory reviewers are added to both the PackageGit and ProjectGit PRs. | High | -| **TC-REVIEW-002** | **Add advisory reviewers** | 1. Create a new PackageGit PR with advisory reviewers defined in the configuration. | 1. Advisory reviewers are added to the PR, but their approval is not required for merging. | Medium | -| **TC-REVIEW-003** | **Re-add reviewers** | 1. Push a new commit to a PackageGit PR after it has been approved. | 1. The original reviewers are re-added to the PR. | Medium | -| **TC-REVIEW-004** | **Package PR created by a maintainer** | 1. Create a PackageGit PR from the account of a package maintainer. | 1. No review is requested from other package maintainers. | High | -| **TC-REVIEW-005** | **Package PR created by an external user (approve)** | 1. Create a PackageGit PR from the account of a user who is not a package maintainer.
2. One of the package maintainers approves the PR. | 1. All package maintainers are added as reviewers.
2. Once one maintainer approves the PR, the other maintainers are removed as reviewers. | High | -| **TC-REVIEW-006** | **Package PR created by an external user (reject)** | 1. Create a PackageGit PR from the account of a user who is not a package maintainer.
2. One of the package maintainers rejects the PR. | 1. All package maintainers are added as reviewers.
2. Once one maintainer rejects the PR, the other maintainers are removed as reviewers. | High | -| **TC-REVIEW-007** | **Package PR created by a maintainer with ReviewRequired=true** | 1. Set `ReviewRequired = true` in `workflow.config`.
2. Create a PackageGit PR from the account of a package maintainer. | 1. A review is requested from other package maintainers if available. | High | -| **TC-MERGE-001** | **Automatic Merge** | 1. Create a PackageGit PR.
2. Ensure all mandatory reviews are completed on both project and package PRs. | 1. The PR is automatically merged. | High | -| **TC-MERGE-002** | **ManualMergeOnly with Package Maintainer** | 1. Create a PackageGit PR with `ManualMergeOnly` set to `true`.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the package PR from the account of a package maintainer for that package. | 1. The PR is merged. | High | -| **TC-MERGE-003** | **ManualMergeOnly with unauthorized user** | 1. Create a PackageGit PR with `ManualMergeOnly` set to `true`.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the package PR from the account of a user who is not a maintainer for that package. | 1. The PR is not merged. | High | -| **TC-MERGE-004** | **ManualMergeOnly with multiple packages** | 1. Create a ProjectGit PR that references multiple PackageGit PRs with `ManualMergeOnly` set to `true`.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on each package PR from the account of a package maintainer. | 1. The PR is merged only after "merge ok" is commented on all associated PackageGit PRs. | High | -| **TC-MERGE-005** | **ManualMergeOnly with Project Maintainer** | 1. Create a PackageGit PR with `ManualMergeOnly` set to `true`.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the package PR from the account of a project maintainer. | 1. The PR is merged. | High | -| **TC-MERGE-006** | **ManualMergeProject with Project Maintainer** | 1. Create a PackageGit PR with `ManualMergeProject` set to `true`.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the project PR from the account of a project maintainer. | 1. The PR is merged. | High | -| **TC-MERGE-007** | **ManualMergeProject with unauthorized user** | 1. Create a PackageGit PR with `ManualMergeProject` set to `true`.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the project PR from the account of a package maintainer. | 1. The PR is not merged. | High | -| **TC-CONFIG-001** | **Invalid Configuration** | 1. Provide an invalid `workflow.config` file. | 1. The bot reports an error and does not process any PRs. | High | -| **TC-LABEL-001** | **Apply `staging/Auto` label** | 1. Create a new PackageGit PR. | 1. The `staging/Auto` label is applied to the ProjectGit PR. | High | -| **TC-LABEL-002** | **Apply `review/Pending` label** | 1. Create a new PackageGit PR. | 1. The `review/Pending` label is applied to the ProjectGit PR when there are pending reviews. | Medium | -| **TC-LABEL-003** | **Apply `review/Done` label** | 1. Ensure all mandatory reviews for a PR are completed. | 1. The `review/Done` label is applied to the ProjectGit PR when all mandatory reviews are completed. | Medium | +| Test Case ID | Status | Description | Steps to Reproduce | Expected Results | Priority | +| :--- | :--- | :--- | :--- | :--- | :--- | +| **TC-SYNC-001** | P | **Create ProjectGit PR from PackageGit PR** | 1. Create a new PR in a PackageGit repository. | 1. A new PR is created in the corresponding ProjectGit repository with the title "Forwarded PRs: ".
2. The ProjectGit PR description contains a link to the PackageGit PR (e.g., `PR: org/package_repo!pr_number`).
3. The package submodule in the ProjectGit PR points to the PackageGit PR's commit. | High | +| **TC-SYNC-002** | P | **Update ProjectGit PR from PackageGit PR** | 1. Push a new commit to an existing PackageGit PR. | 1. The corresponding ProjectGit PR's head branch is updated with the new commit. | High | +| **TC-SYNC-003** | P | **WIP Flag Synchronization** | 1. Mark a PackageGit PR as "Work In Progress".
2. Remove the WIP flag from the PackageGit PR. | 1. The corresponding ProjectGit PR is also marked as "Work In Progress".
2. The WIP flag on the ProjectGit PR is removed. | Medium | +| **TC-SYNC-004** | - | **WIP Flag (multiple referenced package PRs)** | 1. Create a ProjectGit PR that references multiple PackageGit PRs.
2. Mark one of the PackageGit PRs as "Work In Progress".
3. Remove the "Work In Progress" flag from all PackageGit PRs. | 1. The ProjectGit PR is marked as "Work In Progress".
2. The "Work In Progress" flag is removed from the ProjectGit PR only after it has been removed from all associated PackageGit PRs. | Medium | +| **TC-SYNC-005** | x | **NoProjectGitPR = true, edits disabled** | 1. Set `NoProjectGitPR = true` in `workflow.config`.
2. Create a PackageGit PR without "Allow edits from maintainers" enabled.
3. Push a new commit to the PackageGit PR. | 1. No ProjectGit PR is created.
2. The bot adds a warning comment to the PackageGit PR explaining that it cannot update the PR. | High | +| **TC-SYNC-006** | x | **NoProjectGitPR = true, edits enabled** | 1. Set `NoProjectGitPR = true` in `workflow.config`.
2. Create a PackageGit PR with "Allow edits from maintainers" enabled.
3. Push a new commit to the PackageGit PR. | 1. No ProjectGit PR is created.
2. The submodule commit on the project PR is updated with the new commit from the PackageGit PR. | High | +| **TC-COMMENT-001** | - | **Detect duplicate comments** | 1. Create a PackageGit PR.
2. Wait for the `workflow-pr` bot to act on the PR.
3. Edit the body of the PR to trigger the bot a second time. | 1. The bot should not post a duplicate comment. | High | +| **TC-REVIEW-001** | P | **Add mandatory reviewers** | 1. Create a new PackageGit PR. | 1. All mandatory reviewers are added to both the PackageGit and ProjectGit PRs. | High | +| **TC-REVIEW-002** | - | **Add advisory reviewers** | 1. Create a new PackageGit PR with advisory reviewers defined in the configuration. | 1. Advisory reviewers are added to the PR, but their approval is not required for merging. | Medium | +| **TC-REVIEW-003** | - | **Re-add reviewers** | 1. Push a new commit to a PackageGit PR after it has been approved. | 1. The original reviewers are re-added to the PR. | Medium | +| **TC-REVIEW-004** | x | **Package PR created by a maintainer** | 1. Create a PackageGit PR from the account of a package maintainer. | 1. No review is requested from other package maintainers. | High | +| **TC-REVIEW-005** | P | **Package PR created by an external user (approve)** | 1. Create a PackageGit PR from the account of a user who is not a package maintainer.
2. One of the package maintainers approves the PR. | 1. All package maintainers are added as reviewers.
2. Once one maintainer approves the PR, the other maintainers are removed as reviewers. | High | +| **TC-REVIEW-006** | P | **Package PR created by an external user (reject)** | 1. Create a PackageGit PR from the account of a user who is not a package maintainer.
2. One of the package maintainers rejects the PR. | 1. All package maintainers are added as reviewers.
2. Once one maintainer rejects the PR, the other maintainers are removed as reviewers. | High | +| **TC-REVIEW-007** | P | **Package PR created by a maintainer with ReviewRequired=true** | 1. Set `ReviewRequired = true` in `workflow.config`.
2. Create a PackageGit PR from the account of a package maintainer. | 1. A review is requested from other package maintainers if available. | High | +| **TC-MERGE-001** | x | **Automatic Merge** | 1. Create a PackageGit PR.
2. Ensure all mandatory reviews are completed on both project and package PRs. | 1. The PR is automatically merged. | High | +| **TC-MERGE-002** | - | **ManualMergeOnly with Package Maintainer** | 1. Create a PackageGit PR with `ManualMergeOnly` set to `true`.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the package PR from the account of a package maintainer for that package. | 1. The PR is merged. | High | +| **TC-MERGE-003** | - | **ManualMergeOnly with unauthorized user** | 1. Create a PackageGit PR with `ManualMergeOnly` set to `true`.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the package PR from the account of a user who is not a maintainer for that package. | 1. The PR is not merged. | High | +| **TC-MERGE-004** | - | **ManualMergeOnly with multiple packages** | 1. Create a ProjectGit PR that references multiple PackageGit PRs with `ManualMergeOnly` set to `true`.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on each package PR from the account of a package maintainer. | 1. The PR is merged only after "merge ok" is commented on all associated PackageGit PRs. | High | +| **TC-MERGE-005** | - | **ManualMergeOnly with Project Maintainer** | 1. Create a PackageGit PR with `ManualMergeOnly` set to `true`.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the package PR from the account of a project maintainer. | 1. The PR is merged. | High | +| **TC-MERGE-006** | - | **ManualMergeProject with Project Maintainer** | 1. Create a PackageGit PR with `ManualMergeProject` set to `true`.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the project PR from the account of a project maintainer. | 1. The PR is merged. | High | +| **TC-MERGE-007** | - | **ManualMergeProject with unauthorized user** | 1. Create a PackageGit PR with `ManualMergeProject` set to `true`.
2. Ensure all mandatory reviews are completed on both project and package PRs.
3. Comment "merge ok" on the project PR from the account of a package maintainer. | 1. The PR is not merged. | High | +| **TC-CONFIG-001** | - | **Invalid Configuration** | 1. Provide an invalid `workflow.config` file. | 1. The bot reports an error and does not process any PRs. | High | +| **TC-LABEL-001** | P | **Apply `staging/Auto` label** | 1. Create a new PackageGit PR. | 1. The `staging/Auto` label is applied to the ProjectGit PR. | High | +| **TC-LABEL-002** | x | **Apply `review/Pending` label** | 1. Create a new PackageGit PR. | 1. The `review/Pending` label is applied to the ProjectGit PR when there are pending reviews. | Medium | +| **TC-LABEL-003** | - | **Apply `review/Done` label** | 1. Ensure all mandatory reviews for a PR are completed. | 1. The `review/Done` label is applied to the ProjectGit PR when all mandatory reviews are completed. | Medium | + +#### Legend: +* P = implemented and passing; +* x = likely implemented, but investigation is needed; +* X = implemented and likely to pass, but someteimes may fail, but troubleshooting is needed; +* - = test is not implemented diff --git a/integration/tests/conftest.py b/integration/tests/conftest.py index 13f88db..c5e882b 100644 --- a/integration/tests/conftest.py +++ b/integration/tests/conftest.py @@ -53,6 +53,17 @@ BRANCH_CONFIG_CUSTOM = { "ManualMergeProject": True, "NoProjectGitPR": True } + }, + "label-test": { + "workflow.config": { + "ManualMergeProject": True, + "Reviewers": ["*usera"], + "ReviewRequired": True, + "Labels": { + "StagingAuto": "staging/Backlog", + "ReviewPending": "review/Pending" + } + } } } @@ -65,7 +76,7 @@ def setup_users_from_config(client: GiteaAPIClient, wf: dict, mt: dict): # Extract from workflow.config Reviewers reviewers = wf.get("Reviewers", []) for r in reviewers: - username = r.lstrip("+-") + username = r.lstrip("+-*") if username and username not in ["autogits_obs_staging_bot", "workflow-pr"]: all_users.add(username) @@ -147,6 +158,10 @@ def gitea_env(): client.update_repo_settings("pool", "pkgA") client.update_repo_settings("pool", "pkgB") + # Create labels + client.create_label("products", "SLFO", "staging/Backlog", color="#0000ff") + client.create_label("products", "SLFO", "review/Pending", color="#ffff00") + # Submodules in SLFO client.add_submodules("products", "SLFO") @@ -217,6 +232,10 @@ def review_required_env(gitea_env): def no_project_git_pr_env(gitea_env): return gitea_env, "products/SLFO", "dev" +@pytest.fixture(scope="session") +def label_env(gitea_env): + return gitea_env, "products/SLFO", "label-test" + @pytest.fixture(scope="session") def ownerA_client(gitea_env): return GiteaAPIClient(base_url=gitea_env.base_url, token=gitea_env.headers["Authorization"].split(" ")[1], sudo="ownerA") @@ -229,6 +248,10 @@ def ownerB_client(gitea_env): def ownerBB_client(gitea_env): return GiteaAPIClient(base_url=gitea_env.base_url, token=gitea_env.headers["Authorization"].split(" ")[1], sudo="ownerBB") +@pytest.fixture(scope="session") +def staging_bot_client(gitea_env): + return GiteaAPIClient(base_url=gitea_env.base_url, token=gitea_env.headers["Authorization"].split(" ")[1], sudo="autogits_obs_staging_bot") + @pytest.fixture(scope="session") def test_user_client(gitea_env): username = f"test-user-{int(time.time())}" diff --git a/integration/tests/lib/common_test_utils.py b/integration/tests/lib/common_test_utils.py index d0e31ef..c5960c6 100644 --- a/integration/tests/lib/common_test_utils.py +++ b/integration/tests/lib/common_test_utils.py @@ -226,6 +226,21 @@ index 0000000..{pkg_b_sha} self._request("PATCH", f"repos/{org_name}/{repo_name}", json=repo_data) print(f"Repository settings for '{org_name}/{repo_name}' updated.") + def create_label(self, owner: str, repo: str, name: str, color: str = "#abcdef"): + print(f"--- Creating label '{name}' in {owner}/{repo} ---") + url = f"repos/{owner}/{repo}/labels" + data = { + "name": name, + "color": color + } + try: + self._request("POST", url, json=data) + print(f"Label '{name}' created.") + except requests.exceptions.HTTPError as e: + if e.response.status_code == 422: # Already exists + print(f"Label '{name}' already exists.") + else: + raise def create_file(self, owner: str, repo: str, file_path: str, content: str, branch: str = "main", message: str = "Add file"): file_info = self.get_file_info(owner, repo, file_path, branch=branch) diff --git a/integration/tests/workflow_pr_label_test.py b/integration/tests/workflow_pr_label_test.py new file mode 100644 index 0000000..9274291 --- /dev/null +++ b/integration/tests/workflow_pr_label_test.py @@ -0,0 +1,97 @@ +import pytest +import re +import time +from pathlib import Path +from tests.lib.common_test_utils import ( + GiteaAPIClient, +) + +# ============================================================================= +# TEST CASES +# ============================================================================= + +@pytest.mark.t001 +@pytest.mark.xfail(reason="review pending label is not applied") +def test_001_project_pr_labels(label_env, staging_bot_client): + """ + Test scenario: + 1. Setup custom workflow.config with Labels: { "StagingAuto": "staging/Backlog", "ReviewPending": "review/Pending" }. + 2. Create a package PR in 'label-test' branch. + 3. Make sure the workflow-pr service created related project PR in 'label-test' branch. + 4. Wait for the project PR to have the label "staging/Backlog". + 5. Post approval from autogits_obs_staging_bot. + 6. Check that the project PR gets the label "review/Pending". + """ + gitea_env, test_full_repo_name, branch_name = label_env + + # 1. Create a package PR + diff = """diff --git a/label_test_fixture.txt b/label_test_fixture.txt +new file mode 100644 +index 0000000..e69de29 +""" + print(f"--- Creating package PR in pool/pkgA on branch {branch_name} ---") + package_pr = gitea_env.create_gitea_pr("pool/pkgA", diff, "Test Labels Fixture", False, base_branch=branch_name) + package_pr_number = package_pr["number"] + print(f"Created package PR pool/pkgA#{package_pr_number}") + + # 2. Make sure the workflow-pr service created related project PR + project_pr_number = None + print(f"Polling pool/pkgA PR #{package_pr_number} timeline for forwarded PR event...") + for _ in range(40): + time.sleep(1) + timeline_events = gitea_env.get_timeline_events("pool/pkgA", package_pr_number) + for event in timeline_events: + if event.get("type") == "pull_ref": + if not (ref_issue := event.get("ref_issue")): + continue + url_to_check = ref_issue.get("html_url", "") + match = re.search(r"products/SLFO/pulls/(\d+)", url_to_check) + if match: + project_pr_number = int(match.group(1)) + break + if project_pr_number: + break + + assert project_pr_number is not None, "Workflow bot did not create a project PR." + print(f"Found project PR: products/SLFO#{project_pr_number}") + + # 3. Wait for the project PR to have the label "staging/Backlog" + print(f"Checking for 'staging/Backlog' label on project PR products/SLFO#{project_pr_number}...") + + backlog_label_found = False + expected_backlog_label = "staging/Backlog" + + for _ in range(20): + project_pr_details = gitea_env.get_pr_details("products/SLFO", project_pr_number) + labels = project_pr_details.get("labels", []) + label_names = [l["name"] for l in labels] + if expected_backlog_label in label_names: + backlog_label_found = True + break + time.sleep(1) + + assert backlog_label_found, f"Project PR products/SLFO#{project_pr_number} does not have the expected label '{expected_backlog_label}'." + print(f"Project PR products/SLFO#{project_pr_number} has the expected label '{expected_backlog_label}'.") + + # 4. Post approval from autogits_obs_staging_bot + print(f"--- Posting approval from autogits_obs_staging_bot on project PR products/SLFO#{project_pr_number} ---") + staging_bot_client.create_review("products/SLFO", project_pr_number, event="APPROVED", body="Staging OK") + + # 5. Check that the project PR has the label "review/Pending" + print(f"Checking for 'review/Pending' label on project PR products/SLFO#{project_pr_number}...") + + pending_label_found = False + expected_pending_label = "review/Pending" + + for _ in range(20): + project_pr_details = gitea_env.get_pr_details("products/SLFO", project_pr_number) + labels = project_pr_details.get("labels", []) + label_names = [l["name"] for l in labels] + print(f"Current labels: {label_names}") + if expected_pending_label in label_names: + pending_label_found = True + break + time.sleep(1) + + assert pending_label_found, f"Project PR products/SLFO#{project_pr_number} does not have the expected label '{expected_pending_label}'." + print(f"Project PR products/SLFO#{project_pr_number} has the expected label '{expected_pending_label}'.") diff --git a/integration/workflow-pr/workflow-pr.json b/integration/workflow-pr/workflow-pr.json index e59a145..b8340a8 100644 --- a/integration/workflow-pr/workflow-pr.json +++ b/integration/workflow-pr/workflow-pr.json @@ -3,5 +3,6 @@ "products/SLFO#dev", "products/SLFO#merge", "products/SLFO#maintainer-merge", - "products/SLFO#review-required" + "products/SLFO#review-required", + "products/SLFO#label-test" ]