git-workflow/autogits
SHA256
21
2
Fork 15
Code Issues 44 Pull Requests 16 Actions Projects 2 Activity

Add integration test for project PR labels #134

Merged
andriinikitin merged 2 commits from andriinikitin/autogits:t_add_label_test into main 2026-02-23 11:45:50 +01:00
Conversation 0 Commits 2 Files Changed 5 +171 -29
5 changed files with 171 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
integration/test-plan.md
View File

@@ -53,31 +53,37 @@ The testing will be conducted in a dedicated test environment that mimics the pr
## 5. Test Cases
| Test Case ID | Description | Steps to Reproduce | Expected Results | Priority |
| :--- | :--- | :--- | :--- | :--- |
| **TC-SYNC-001** | **Create ProjectGit PR from PackageGit PR** | 1. Create a new PR in a PackageGit repository. | 1. A new PR is created in the corresponding ProjectGit repository with the title "Forwarded PRs: <package_name>".<br>2. The ProjectGit PR description contains a link to the PackageGit PR (e.g., `PR: org/package_repo!pr_number`).<br>3. The package submodule in the ProjectGit PR points to the PackageGit PR's commit. | High |
| **TC-SYNC-002** | **Update ProjectGit PR from PackageGit PR** | 1. Push a new commit to an existing PackageGit PR. | 1. The corresponding ProjectGit PR's head branch is updated with the new commit. | High |
| **TC-SYNC-003** | **WIP Flag Synchronization** | 1. Mark a PackageGit PR as "Work In Progress".<br>2. Remove the WIP flag from the PackageGit PR. | 1. The corresponding ProjectGit PR is also marked as "Work In Progress".<br>2. The WIP flag on the ProjectGit PR is removed. | Medium |
| **TC-SYNC-004** | **WIP Flag (multiple referenced package PRs)** | 1. Create a ProjectGit PR that references multiple PackageGit PRs.<br>2. Mark one of the PackageGit PRs as "Work In Progress".<br>3. Remove the "Work In Progress" flag from all PackageGit PRs. | 1. The ProjectGit PR is marked as "Work In Progress".<br>2. The "Work In Progress" flag is removed from the ProjectGit PR only after it has been removed from all associated PackageGit PRs. | Medium |
| **TC-SYNC-005** | **NoProjectGitPR = true, edits disabled** | 1. Set `NoProjectGitPR = true` in `workflow.config`.<br>2. Create a PackageGit PR without "Allow edits from maintainers" enabled. <br>3. Push a new commit to the PackageGit PR. | 1. No ProjectGit PR is created.<br>2. The bot adds a warning comment to the PackageGit PR explaining that it cannot update the PR. | High |
| **TC-SYNC-006** | **NoProjectGitPR = true, edits enabled** | 1. Set `NoProjectGitPR = true` in `workflow.config`.<br>2. Create a PackageGit PR with "Allow edits from maintainers" enabled.<br>3. Push a new commit to the PackageGit PR. | 1. No ProjectGit PR is created.<br>2. The submodule commit on the project PR is updated with the new commit from the PackageGit PR. | High |
| **TC-COMMENT-001** | **Detect duplicate comments** | 1. Create a PackageGit PR.<br>2. Wait for the `workflow-pr` bot to act on the PR.<br>3. Edit the body of the PR to trigger the bot a second time. | 1. The bot should not post a duplicate comment. | High |
| **TC-REVIEW-001** | **Add mandatory reviewers** | 1. Create a new PackageGit PR. | 1. All mandatory reviewers are added to both the PackageGit and ProjectGit PRs. | High |
| **TC-REVIEW-002** | **Add advisory reviewers** | 1. Create a new PackageGit PR with advisory reviewers defined in the configuration. | 1. Advisory reviewers are added to the PR, but their approval is not required for merging. | Medium |
| **TC-REVIEW-003** | **Re-add reviewers** | 1. Push a new commit to a PackageGit PR after it has been approved. | 1. The original reviewers are re-added to the PR. | Medium |
| **TC-REVIEW-004** | **Package PR created by a maintainer** | 1. Create a PackageGit PR from the account of a package maintainer. | 1. No review is requested from other package maintainers. | High |
| **TC-REVIEW-005** | **Package PR created by an external user (approve)** | 1. Create a PackageGit PR from the account of a user who is not a package maintainer.<br>2. One of the package maintainers approves the PR. | 1. All package maintainers are added as reviewers.<br>2. Once one maintainer approves the PR, the other maintainers are removed as reviewers. | High |
| **TC-REVIEW-006** | **Package PR created by an external user (reject)** | 1. Create a PackageGit PR from the account of a user who is not a package maintainer.<br>2. One of the package maintainers rejects the PR. | 1. All package maintainers are added as reviewers.<br>2. Once one maintainer rejects the PR, the other maintainers are removed as reviewers. | High |
| **TC-REVIEW-007** | **Package PR created by a maintainer with ReviewRequired=true** | 1. Set `ReviewRequired = true` in `workflow.config`.<br>2. Create a PackageGit PR from the account of a package maintainer. | 1. A review is requested from other package maintainers if available. | High |
| **TC-MERGE-001** | **Automatic Merge** | 1. Create a PackageGit PR.<br>2. Ensure all mandatory reviews are completed on both project and package PRs. | 1. The PR is automatically merged. | High |
| **TC-MERGE-002** | **ManualMergeOnly with Package Maintainer** | 1. Create a PackageGit PR with `ManualMergeOnly` set to `true`.<br>2. Ensure all mandatory reviews are completed on both project and package PRs.<br>3. Comment "merge ok" on the package PR from the account of a package maintainer for that package. | 1. The PR is merged. | High |
| **TC-MERGE-003** | **ManualMergeOnly with unauthorized user** | 1. Create a PackageGit PR with `ManualMergeOnly` set to `true`.<br>2. Ensure all mandatory reviews are completed on both project and package PRs.<br>3. Comment "merge ok" on the package PR from the account of a user who is not a maintainer for that package. | 1. The PR is not merged. | High |
| **TC-MERGE-004** | **ManualMergeOnly with multiple packages** | 1. Create a ProjectGit PR that references multiple PackageGit PRs with `ManualMergeOnly` set to `true`.<br>2. Ensure all mandatory reviews are completed on both project and package PRs.<br>3. Comment "merge ok" on each package PR from the account of a package maintainer. | 1. The PR is merged only after "merge ok" is commented on all associated PackageGit PRs. | High |
| **TC-MERGE-005** | **ManualMergeOnly with Project Maintainer** | 1. Create a PackageGit PR with `ManualMergeOnly` set to `true`.<br>2. Ensure all mandatory reviews are completed on both project and package PRs.<br>3. Comment "merge ok" on the package PR from the account of a project maintainer. | 1. The PR is merged. | High |
| **TC-MERGE-006** | **ManualMergeProject with Project Maintainer** | 1. Create a PackageGit PR with `ManualMergeProject` set to `true`.<br>2. Ensure all mandatory reviews are completed on both project and package PRs.<br>3. Comment "merge ok" on the project PR from the account of a project maintainer. | 1. The PR is merged. | High |
| **TC-MERGE-007** | **ManualMergeProject with unauthorized user** | 1. Create a PackageGit PR with `ManualMergeProject` set to `true`.<br>2. Ensure all mandatory reviews are completed on both project and package PRs.<br>3. Comment "merge ok" on the project PR from the account of a package maintainer. | 1. The PR is not merged. | High |
| **TC-CONFIG-001** | **Invalid Configuration** | 1. Provide an invalid `workflow.config` file. | 1. The bot reports an error and does not process any PRs. | High |
| **TC-LABEL-001** | **Apply `staging/Auto` label** | 1. Create a new PackageGit PR. | 1. The `staging/Auto` label is applied to the ProjectGit PR. | High |
| **TC-LABEL-002** | **Apply `review/Pending` label** | 1. Create a new PackageGit PR. | 1. The `review/Pending` label is applied to the ProjectGit PR when there are pending reviews. | Medium |
| **TC-LABEL-003** | **Apply `review/Done` label** | 1. Ensure all mandatory reviews for a PR are completed. | 1. The `review/Done` label is applied to the ProjectGit PR when all mandatory reviews are completed. | Medium |
| Test Case ID | Status | Description | Steps to Reproduce | Expected Results | Priority |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **TC-SYNC-001** | P | **Create ProjectGit PR from PackageGit PR** | 1. Create a new PR in a PackageGit repository. | 1. A new PR is created in the corresponding ProjectGit repository with the title "Forwarded PRs: <package_name>".<br>2. The ProjectGit PR description contains a link to the PackageGit PR (e.g., `PR: org/package_repo!pr_number`).<br>3. The package submodule in the ProjectGit PR points to the PackageGit PR's commit. | High |
| **TC-SYNC-002** | P | **Update ProjectGit PR from PackageGit PR** | 1. Push a new commit to an existing PackageGit PR. | 1. The corresponding ProjectGit PR's head branch is updated with the new commit. | High |
| **TC-SYNC-003** | P | **WIP Flag Synchronization** | 1. Mark a PackageGit PR as "Work In Progress".<br>2. Remove the WIP flag from the PackageGit PR. | 1. The corresponding ProjectGit PR is also marked as "Work In Progress".<br>2. The WIP flag on the ProjectGit PR is removed. | Medium |
| **TC-SYNC-004** | - | **WIP Flag (multiple referenced package PRs)** | 1. Create a ProjectGit PR that references multiple PackageGit PRs.<br>2. Mark one of the PackageGit PRs as "Work In Progress".<br>3. Remove the "Work In Progress" flag from all PackageGit PRs. | 1. The ProjectGit PR is marked as "Work In Progress".<br>2. The "Work In Progress" flag is removed from the ProjectGit PR only after it has been removed from all associated PackageGit PRs. | Medium |
| **TC-SYNC-005** | x | **NoProjectGitPR = true, edits disabled** | 1. Set `NoProjectGitPR = true` in `workflow.config`.<br>2. Create a PackageGit PR without "Allow edits from maintainers" enabled. <br>3. Push a new commit to the PackageGit PR. | 1. No ProjectGit PR is created.<br>2. The bot adds a warning comment to the PackageGit PR explaining that it cannot update the PR. | High |
| **TC-SYNC-006** | x | **NoProjectGitPR = true, edits enabled** | 1. Set `NoProjectGitPR = true` in `workflow.config`.<br>2. Create a PackageGit PR with "Allow edits from maintainers" enabled.<br>3. Push a new commit to the PackageGit PR. | 1. No ProjectGit PR is created.<br>2. The submodule commit on the project PR is updated with the new commit from the PackageGit PR. | High |
| **TC-COMMENT-001** | - | **Detect duplicate comments** | 1. Create a PackageGit PR.<br>2. Wait for the `workflow-pr` bot to act on the PR.<br>3. Edit the body of the PR to trigger the bot a second time. | 1. The bot should not post a duplicate comment. | High |
| **TC-REVIEW-001** | P | **Add mandatory reviewers** | 1. Create a new PackageGit PR. | 1. All mandatory reviewers are added to both the PackageGit and ProjectGit PRs. | High |
| **TC-REVIEW-002** | - | **Add advisory reviewers** | 1. Create a new PackageGit PR with advisory reviewers defined in the configuration. | 1. Advisory reviewers are added to the PR, but their approval is not required for merging. | Medium |
| **TC-REVIEW-003** | - | **Re-add reviewers** | 1. Push a new commit to a PackageGit PR after it has been approved. | 1. The original reviewers are re-added to the PR. | Medium |
| **TC-REVIEW-004** | x | **Package PR created by a maintainer** | 1. Create a PackageGit PR from the account of a package maintainer. | 1. No review is requested from other package maintainers. | High |
| **TC-REVIEW-005** | P | **Package PR created by an external user (approve)** | 1. Create a PackageGit PR from the account of a user who is not a package maintainer.<br>2. One of the package maintainers approves the PR. | 1. All package maintainers are added as reviewers.<br>2. Once one maintainer approves the PR, the other maintainers are removed as reviewers. | High |
| **TC-REVIEW-006** | P | **Package PR created by an external user (reject)** | 1. Create a PackageGit PR from the account of a user who is not a package maintainer.<br>2. One of the package maintainers rejects the PR. | 1. All package maintainers are added as reviewers.<br>2. Once one maintainer rejects the PR, the other maintainers are removed as reviewers. | High |
| **TC-REVIEW-007** | P | **Package PR created by a maintainer with ReviewRequired=true** | 1. Set `ReviewRequired = true` in `workflow.config`.<br>2. Create a PackageGit PR from the account of a package maintainer. | 1. A review is requested from other package maintainers if available. | High |
| **TC-MERGE-001** | x | **Automatic Merge** | 1. Create a PackageGit PR.<br>2. Ensure all mandatory reviews are completed on both project and package PRs. | 1. The PR is automatically merged. | High |
| **TC-MERGE-002** | - | **ManualMergeOnly with Package Maintainer** | 1. Create a PackageGit PR with `ManualMergeOnly` set to `true`.<br>2. Ensure all mandatory reviews are completed on both project and package PRs.<br>3. Comment "merge ok" on the package PR from the account of a package maintainer for that package. | 1. The PR is merged. | High |
| **TC-MERGE-003** | - | **ManualMergeOnly with unauthorized user** | 1. Create a PackageGit PR with `ManualMergeOnly` set to `true`.<br>2. Ensure all mandatory reviews are completed on both project and package PRs.<br>3. Comment "merge ok" on the package PR from the account of a user who is not a maintainer for that package. | 1. The PR is not merged. | High |
| **TC-MERGE-004** | - | **ManualMergeOnly with multiple packages** | 1. Create a ProjectGit PR that references multiple PackageGit PRs with `ManualMergeOnly` set to `true`.<br>2. Ensure all mandatory reviews are completed on both project and package PRs.<br>3. Comment "merge ok" on each package PR from the account of a package maintainer. | 1. The PR is merged only after "merge ok" is commented on all associated PackageGit PRs. | High |
| **TC-MERGE-005** | - | **ManualMergeOnly with Project Maintainer** | 1. Create a PackageGit PR with `ManualMergeOnly` set to `true`.<br>2. Ensure all mandatory reviews are completed on both project and package PRs.<br>3. Comment "merge ok" on the package PR from the account of a project maintainer. | 1. The PR is merged. | High |
| **TC-MERGE-006** | - | **ManualMergeProject with Project Maintainer** | 1. Create a PackageGit PR with `ManualMergeProject` set to `true`.<br>2. Ensure all mandatory reviews are completed on both project and package PRs.<br>3. Comment "merge ok" on the project PR from the account of a project maintainer. | 1. The PR is merged. | High |
| **TC-MERGE-007** | - | **ManualMergeProject with unauthorized user** | 1. Create a PackageGit PR with `ManualMergeProject` set to `true`.<br>2. Ensure all mandatory reviews are completed on both project and package PRs.<br>3. Comment "merge ok" on the project PR from the account of a package maintainer. | 1. The PR is not merged. | High |
| **TC-CONFIG-001** | - | **Invalid Configuration** | 1. Provide an invalid `workflow.config` file. | 1. The bot reports an error and does not process any PRs. | High |
| **TC-LABEL-001** | P | **Apply `staging/Auto` label** | 1. Create a new PackageGit PR. | 1. The `staging/Auto` label is applied to the ProjectGit PR. | High |
| **TC-LABEL-002** | x | **Apply `review/Pending` label** | 1. Create a new PackageGit PR. | 1. The `review/Pending` label is applied to the ProjectGit PR when there are pending reviews. | Medium |
| **TC-LABEL-003** | - | **Apply `review/Done` label** | 1. Ensure all mandatory reviews for a PR are completed. | 1. The `review/Done` label is applied to the ProjectGit PR when all mandatory reviews are completed. | Medium |
#### Legend:
* P = implemented and passing;
* x = likely implemented, but investigation is needed;
* X = implemented and likely to pass, but someteimes may fail, but troubleshooting is needed;
* - = test is not implemented

25
integration/tests/conftest.py
View File

@@ -53,6 +53,17 @@ BRANCH_CONFIG_CUSTOM = {
"ManualMergeProject": True,
"NoProjectGitPR": True
}
},
"label-test": {
"workflow.config": {
"ManualMergeProject": True,
"Reviewers": ["*usera"],
"ReviewRequired": True,
"Labels": {
"StagingAuto": "staging/Backlog",
"ReviewPending": "review/Pending"
}
}
}
}
@@ -65,7 +76,7 @@ def setup_users_from_config(client: GiteaAPIClient, wf: dict, mt: dict):
# Extract from workflow.config Reviewers
reviewers = wf.get("Reviewers", [])
for r in reviewers:
username = r.lstrip("+-")
username = r.lstrip("+-*")
if username and username not in ["autogits_obs_staging_bot", "workflow-pr"]:
all_users.add(username)
@@ -147,6 +158,10 @@ def gitea_env():
client.update_repo_settings("pool", "pkgA")
client.update_repo_settings("pool", "pkgB")
# Create labels
client.create_label("products", "SLFO", "staging/Backlog", color="#0000ff")
client.create_label("products", "SLFO", "review/Pending", color="#ffff00")
# Submodules in SLFO
client.add_submodules("products", "SLFO")
@@ -217,6 +232,10 @@ def review_required_env(gitea_env):
def no_project_git_pr_env(gitea_env):
return gitea_env, "products/SLFO", "dev"
@pytest.fixture(scope="session")
def label_env(gitea_env):
return gitea_env, "products/SLFO", "label-test"
@pytest.fixture(scope="session")
def ownerA_client(gitea_env):
return GiteaAPIClient(base_url=gitea_env.base_url, token=gitea_env.headers["Authorization"].split(" ")[1], sudo="ownerA")
@@ -229,6 +248,10 @@ def ownerB_client(gitea_env):
def ownerBB_client(gitea_env):
return GiteaAPIClient(base_url=gitea_env.base_url, token=gitea_env.headers["Authorization"].split(" ")[1], sudo="ownerBB")
@pytest.fixture(scope="session")
def staging_bot_client(gitea_env):
return GiteaAPIClient(base_url=gitea_env.base_url, token=gitea_env.headers["Authorization"].split(" ")[1], sudo="autogits_obs_staging_bot")
@pytest.fixture(scope="session")
def test_user_client(gitea_env):
username = f"test-user-{int(time.time())}"

15
integration/tests/lib/common_test_utils.py
View File

@@ -226,6 +226,21 @@ index 0000000..{pkg_b_sha}
self._request("PATCH", f"repos/{org_name}/{repo_name}", json=repo_data)
print(f"Repository settings for '{org_name}/{repo_name}' updated.")
def create_label(self, owner: str, repo: str, name: str, color: str = "#abcdef"):
print(f"--- Creating label '{name}' in {owner}/{repo} ---")
url = f"repos/{owner}/{repo}/labels"
data = {
"name": name,
"color": color
}
try:
self._request("POST", url, json=data)
print(f"Label '{name}' created.")
except requests.exceptions.HTTPError as e:
if e.response.status_code == 422: # Already exists
print(f"Label '{name}' already exists.")
else:
raise
def create_file(self, owner: str, repo: str, file_path: str, content: str, branch: str = "main", message: str = "Add file"):
file_info = self.get_file_info(owner, repo, file_path, branch=branch)

97
integration/tests/workflow_pr_label_test.py Normal file
View File

@@ -0,0 +1,97 @@
import pytest
import re
import time
from pathlib import Path
from tests.lib.common_test_utils import (
GiteaAPIClient,
)
# =============================================================================
# TEST CASES
# =============================================================================
@pytest.mark.t001
@pytest.mark.xfail(reason="review pending label is not applied")
def test_001_project_pr_labels(label_env, staging_bot_client):
"""
Test scenario:
1. Setup custom workflow.config with Labels: { "StagingAuto": "staging/Backlog", "ReviewPending": "review/Pending" }.
2. Create a package PR in 'label-test' branch.
3. Make sure the workflow-pr service created related project PR in 'label-test' branch.
4. Wait for the project PR to have the label "staging/Backlog".
5. Post approval from autogits_obs_staging_bot.
6. Check that the project PR gets the label "review/Pending".
"""
gitea_env, test_full_repo_name, branch_name = label_env
# 1. Create a package PR
diff = """diff --git a/label_test_fixture.txt b/label_test_fixture.txt
new file mode 100644
index 0000000..e69de29
"""
print(f"--- Creating package PR in pool/pkgA on branch {branch_name} ---")
package_pr = gitea_env.create_gitea_pr("pool/pkgA", diff, "Test Labels Fixture", False, base_branch=branch_name)
package_pr_number = package_pr["number"]
print(f"Created package PR pool/pkgA#{package_pr_number}")
# 2. Make sure the workflow-pr service created related project PR
project_pr_number = None
print(f"Polling pool/pkgA PR #{package_pr_number} timeline for forwarded PR event...")
for _ in range(40):
time.sleep(1)
timeline_events = gitea_env.get_timeline_events("pool/pkgA", package_pr_number)
for event in timeline_events:
if event.get("type") == "pull_ref":
if not (ref_issue := event.get("ref_issue")):
continue
url_to_check = ref_issue.get("html_url", "")
match = re.search(r"products/SLFO/pulls/(\d+)", url_to_check)
if match:
project_pr_number = int(match.group(1))
break
if project_pr_number:
break
assert project_pr_number is not None, "Workflow bot did not create a project PR."
print(f"Found project PR: products/SLFO#{project_pr_number}")
# 3. Wait for the project PR to have the label "staging/Backlog"
print(f"Checking for 'staging/Backlog' label on project PR products/SLFO#{project_pr_number}...")
backlog_label_found = False
expected_backlog_label = "staging/Backlog"
for _ in range(20):
project_pr_details = gitea_env.get_pr_details("products/SLFO", project_pr_number)
labels = project_pr_details.get("labels", [])
label_names = [l["name"] for l in labels]
if expected_backlog_label in label_names:
backlog_label_found = True
break
time.sleep(1)
assert backlog_label_found, f"Project PR products/SLFO#{project_pr_number} does not have the expected label '{expected_backlog_label}'."
print(f"Project PR products/SLFO#{project_pr_number} has the expected label '{expected_backlog_label}'.")
# 4. Post approval from autogits_obs_staging_bot
print(f"--- Posting approval from autogits_obs_staging_bot on project PR products/SLFO#{project_pr_number} ---")
staging_bot_client.create_review("products/SLFO", project_pr_number, event="APPROVED", body="Staging OK")
# 5. Check that the project PR has the label "review/Pending"
print(f"Checking for 'review/Pending' label on project PR products/SLFO#{project_pr_number}...")
pending_label_found = False
expected_pending_label = "review/Pending"
for _ in range(20):
project_pr_details = gitea_env.get_pr_details("products/SLFO", project_pr_number)
labels = project_pr_details.get("labels", [])
label_names = [l["name"] for l in labels]
print(f"Current labels: {label_names}")
if expected_pending_label in label_names:
pending_label_found = True
break
time.sleep(1)
assert pending_label_found, f"Project PR products/SLFO#{project_pr_number} does not have the expected label '{expected_pending_label}'."
print(f"Project PR products/SLFO#{project_pr_number} has the expected label '{expected_pending_label}'.")

3
integration/workflow-pr/workflow-pr.json
View File

@@ -3,5 +3,6 @@
"products/SLFO#dev",
"products/SLFO#merge",
"products/SLFO#maintainer-merge",
"products/SLFO#review-required"
"products/SLFO#review-required",
"products/SLFO#label-test"
]