git-workflow/autogits
SHA256
20
1
Fork 12
Code Issues 47 Pull Requests 12 Actions 1 Projects 2 Activity

BR is project-centric, not package-centric #96

New Issue
Open
opened 2025-11-05 18:12:11 +01:00 by michals · 10 comments
michals commented 2025-11-05 18:12:11 +01:00
Copy Link

BR URL like https://br.opensuse.org/status/Java:packages/jameica refers to OBS project.

For this to be 'package-centric' and URL that provides all build results for a git repository and git repository branch should also be provided.

BR URL like https://br.opensuse.org/status/Java:packages/jameica refers to OBS project. For this to be 'package-centric' and URL that provides all build results for a git repository and git repository branch should also be provided.
adamm commented 2025-11-06 09:05:48 +01:00
Owner
Copy Link

This is "OBS" centric... If we want package-centric we'd probably need output like registry.opensuse.org to accommodate all the possibilities. It's definitely possible to do.

This is "OBS" centric... If we want package-centric we'd probably need output like registry.opensuse.org to accommodate all the possibilities. It's definitely *possible* to do.
michals commented 2025-11-06 09:42:23 +01:00
Author
Copy Link

Since you cache the results you can include the scmsync URL, and index by scmsync components as well (repository, repository+branch)

Since you cache the results you can include the scmsync URL, and index by scmsync components as well (repository, repository+branch)
adamm commented 2025-11-06 09:45:18 +01:00
Owner
Copy Link

REDIS data knows nothing about SCMSYNC or project META here. It only knows repository data. It's being fed directly from OBS scheduler, not going through the frontend database.

REDIS data knows nothing about SCMSYNC or project META here. It only knows repository data. It's being fed directly from OBS scheduler, not going through the frontend database.
michals commented 2025-11-06 09:49:58 +01:00
Author
Copy Link

So you would have to expand the data collected there or do an active query to get the URL.

Not trivial but needed if we ever want build results included in a gitea template.

You cannot expect the template to do the work.

So you would have to expand the data collected there or do an active query to get the URL. Not trivial but needed if we ever want build results included in a gitea template. You cannot expect the template to do the work.
michals commented 2025-11-06 09:53:00 +01:00
Author
Copy Link

Also what exactly is repository data?

The repository has packages, and those have VCS links in them.

Also what exactly is repository data? The repository has packages, and those have VCS links in them.
adamm commented 2025-11-06 10:57:41 +01:00
Owner
Copy Link

Just fetch the json output. That's all the data that we have. There's additional data which host it's building on or scheduled on, if it's building or scheduled.

The "dirty" flag in json/xml is not actually in the data, just default. The "updated" timestamp is at time of fetch, not from redis either.

     "Project": "Java:packages",     --- we have
      "Repository": "SLE_15",        --- we have
      "Arch": "x86_64",              --- we have
      "Code": "",
      "Dirty": true,
      "ScmSync": "",
      "ScmInfo": "",
      "Status": [
         {
            "Package": "SweetHome3D",   -- we have
            "Code": "succeeded",        -- we have
            "Details": "",              -- we have
            "LastUpdate": "2025-11-06T09:53:29.432460611Z"
         },
         {
            "Package": "adaptx",
            "Code": "succeeded",
            "Details": "",
            "LastUpdate": "2025-11-06T09:53:29.432460611Z"
         },

So we have about 6 pieces of data. Michael also added "last build status prior to scheduled/blocked" but that's not fetched yet, but should be available.

Just fetch the json output. That's all the data that we have. There's additional data which host it's building on or scheduled on, if it's building or scheduled. The "dirty" flag in json/xml is not actually in the data, just default. The "updated" timestamp is at time of fetch, not from redis either. ``` "Project": "Java:packages", --- we have "Repository": "SLE_15", --- we have "Arch": "x86_64", --- we have "Code": "", "Dirty": true, "ScmSync": "", "ScmInfo": "", "Status": [ { "Package": "SweetHome3D", -- we have "Code": "succeeded", -- we have "Details": "", -- we have "LastUpdate": "2025-11-06T09:53:29.432460611Z" }, { "Package": "adaptx", "Code": "succeeded", "Details": "", "LastUpdate": "2025-11-06T09:53:29.432460611Z" }, ``` So we have about 6 pieces of data. Michael also added "last build status prior to scheduled/blocked" but that's not fetched yet, but should be available.
adamm commented 2025-11-06 11:00:21 +01:00
Owner
Copy Link

prior to using redis, I wanted to fetch this data via the _results endpoint, but it was terribly slow for projects like Factory. _results is mostly meant to be used for single package results, and even then it's some overhead.

Redis was added to OBS like 10 years ago in hope of being more efficient way of getting build results for the UIs. This service is first user of this data.

prior to using redis, I wanted to fetch this data via the _results endpoint, but it was terribly slow for projects like Factory. _results is mostly meant to be used for single package results, and even then it's some overhead. Redis was added to OBS like 10 years ago in hope of being more efficient way of getting build results for the UIs. This service is first user of this data.
michals commented 2025-11-06 11:04:48 +01:00
Author
Copy Link

So you have scmsync but it's empty?

What is that?

So you have scmsync but it's empty? What is that?
adamm commented 2025-11-06 11:13:43 +01:00
Owner
Copy Link

We do not have it. The underlying data structure was to parse XML from OBS _results API. When that took 10+ seconds to just fetch data from openSUSE:Factory, we needed different solution. Adrian then proposed REDIS that no one was using and so that's where we are today.

With REDIS server for state information, we get 250,000 repositories in 3 seconds from OBS. And it takes 10 seconds to fetch complete package list for ALL projects in entire OBS. Performance wise to using OBS _results API, we are some orders of magnitude ahead here. Even just fetching projects list using the osc ls / is slow in comparison.

We *do not* have it. The underlying data structure was to parse XML from OBS _results API. When that took 10+ seconds to just fetch data from openSUSE:Factory, we needed different solution. Adrian then proposed REDIS that no one was using and so that's where we are today. With REDIS server for state information, we get 250,000 repositories in 3 seconds from OBS. And it takes 10 seconds to fetch complete package list for ALL projects in entire OBS. Performance wise to using OBS _results API, we are some orders of magnitude ahead here. Even just fetching projects list using the `osc ls /` is slow in comparison.
michals commented 2025-11-06 11:29:25 +01:00
Author
Copy Link

Can it be added to REDIS?

Can it be added to REDIS?
michals changed title from BR is repository-centric, not package-centric to BR is project-centric, not package-centric 2025-11-06 17:55:32 +01:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
M0ses adamm andriinikitin atartamo bigironman dancermak darix dirkmueller dmach eroca jdsn jzerebecki mstrigl oertel smithfarm
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: git-workflow/autogits#96
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?