Stephan Kulow
ed4b7367eb
Reset branch if the devel branch is based on Factory
...
This happens in packages that change their devel project over time. Then
the commit in the devel project no longer has the parent in the devel branch
but is based on factory
2022-11-03 15:12:07 +01:00
Stephan Kulow
f5b3e42165
Add a test case that switches devel project in its life time
2022-11-03 15:06:12 +01:00
8aed76e52a
change cached file naming pattern
2022-11-03 14:22:19 +01:00
639096b548
optimize cached file locations and add option for cache directory
2022-11-03 14:12:32 +01:00
7678967ae0
implement file caching
...
to prevent having to download files multiple times
2022-11-03 14:05:11 +01:00
Stephan Kulow
1c54a74ecd
Download the full revision
2022-11-02 20:55:09 +01:00
Stephan Kulow
c2294d6200
Add a default LFS .gitattributes for now
...
Otherwise some packages will break to import
2022-11-02 18:27:17 +01:00
Stephan Kulow
ba7436f10c
Keep a reference to the database in DBRevision
...
To avoid passing the db to all actions
2022-11-02 18:27:09 +01:00
Stephan Kulow
172242891d
Fix up some code after aplanas' continued review
2022-11-02 15:22:24 +01:00
Stephan Kulow
05cf792b26
Add the file_md5 to the download function so it can cache and verify
2022-11-02 13:35:45 +01:00
Stephan Kulow
a1ead29734
Extend documention and use some more pythonier loops
2022-11-02 13:29:18 +01:00
Stephan Kulow
05a5e6aea7
Don't refresh packages we're already looking at
2022-11-02 10:52:53 +01:00
Stephan Kulow
68ded48be1
Don't crash on user with missing realname either
2022-11-02 10:52:37 +01:00
Stephan Kulow
fce8aac001
Import creator users as well
2022-11-02 08:59:25 +01:00
Stephan Kulow
bbf1bc2fda
Fetch source projects of requests
...
We do not care for current devel project, but for the projects we saw
requests from
2022-11-02 08:50:54 +01:00
Stephan Kulow
c4654dd896
Split GitExporter out of Importer class
2022-11-02 07:59:25 +01:00
Stephan Kulow
9de0d6e6c5
Rename Exporter to TestExporter to make it more obvious
2022-11-02 07:39:04 +01:00
Stephan Kulow
4ff9b9771a
Split out Flat generator to be able to test it
2022-11-02 07:20:53 +01:00
Stephan Kulow
c94d13d74e
Don't crash on packages without merges ever (very few packages)
2022-11-01 19:30:41 +01:00
Stephan Kulow
ab8120ca53
Don't crash on last_node
2022-11-01 19:02:29 +01:00
Stephan Kulow
b2cadb8c01
Don't crash on packages that didn't get updates in devel
2022-11-01 18:44:59 +01:00
Stephan Kulow
578fb2a30a
Change tree pruning algorithm
...
The first merge we see in Factory determines if we keep the devel
commits in the factory chain or cut that branch.
2022-11-01 13:52:15 +01:00
Stephan Kulow
e6a401d8ac
Remove old history handling
2022-11-01 11:37:30 +01:00
Stephan Kulow
9ed8abad2b
Make database usage the default
...
Some cleanup of no longer used functions
2022-11-01 11:23:40 +01:00
Stephan Kulow
9554fea7e1
Reuse the repository directory by storing a state yaml
...
Not using the database for that so that removing the repository directory will
automatically recreate it
2022-11-01 11:22:58 +01:00
Stephan Kulow
2168c898a2
Add users not known to the FAKE_ACCOUNTS
...
This is technically incorrect but we need to handle them all the same anyway
2022-11-01 09:12:42 +01:00
Stephan Kulow
cfab0a522b
Download .spec and .changes file only for now
2022-10-31 16:31:10 +01:00
Stephan Kulow
bfdade8ecf
Create a flat list of commits to do from the tree
...
Walk the node tree and record the parents, then reverse the tree so we
can have the exact order in which to create git commits
2022-10-31 12:28:12 +01:00
Stephan Kulow
8a3db6c183
Collect the requests for revisions in one DB call
...
For smaller packages this doesn't matter as much, but for large ones
the N+1 queries can sum up badly
2022-10-31 08:03:01 +01:00
Stephan Kulow
822329279d
Switch to walker for tree print
2022-10-31 07:03:19 +01:00
Stephan Kulow
e636f3ed9d
Print the calculated tree for now
2022-10-27 13:34:04 +02:00
Stephan Kulow
7a9e9ce334
Add actual tree and a test case for zsh
2022-10-27 13:16:50 +02:00
Stephan Kulow
65000cd586
Fix even more issues with the linked_revs considered cache
2022-10-27 10:16:20 +02:00
Stephan Kulow
38e4996280
Fix the fake revisions after introduction of cache
...
The way I stored it in the linked_revs was the wrong way
2022-10-27 07:35:08 +02:00
Stephan Kulow
9bd6643e8a
Fix files_hash calculcation and fake revisions
2022-10-26 22:18:16 +02:00
Stephan Kulow
1c846e963c
Import files of revisions more reliable and faster
2022-10-26 21:47:39 +02:00
Stephan Kulow
4dc7e972a2
Make use of db.cursor() in with blocks
2022-10-26 15:49:14 +02:00
Stephan Kulow
217bbcadb5
Run the formatter from make all
2022-10-26 14:57:08 +02:00
Stephan Kulow
1346223822
Reshuffle schema versions after rebase
2022-10-26 14:56:38 +02:00
Stephan Kulow
9dbf3b9094
Import all projects before checking files - otherwise we end with 0s
2022-10-26 14:56:38 +02:00
Stephan Kulow
d961c13614
Ignore commits after the merge
2022-10-26 14:56:38 +02:00
Stephan Kulow
2784171f75
Create fake revisions for every commit in the base of a linked package
...
This is a rather complex operation, but whenever a package changes in
Factory, the inherited package gets a shadow revision consisting of the
3-way merge. If this happens due to a request being accepted, this is
actually in most cases also commited by 'buildservice-autocommit',
so we're making sure this is always happening (and are actually
duplicating revisions in cases that we filter out later as empty
commits).
To differenciate the fake revisions from the real revisions, I add
a fraction part
2022-10-26 14:56:38 +02:00
Stephan Kulow
d17e60a608
Add another table to store linked_revs
...
We need to create fake revisions when packages were touched that are
linked themselves
2022-10-26 14:20:28 +02:00
Stephan Kulow
089b215f4c
Build a revision tree
2022-10-26 14:16:22 +02:00
Stephan Kulow
bc74b6950b
Implement a fileshash function for revision
2022-10-26 14:15:42 +02:00
Stephan Kulow
5bd790e915
Import the fixtures into the test database for queries
2022-10-26 14:15:41 +02:00
Stephan Kulow
ce3863e1b5
Add the capability to export the package data as test fixture
2022-10-26 14:13:24 +02:00
dbeb534108
cleanup context managers
2022-10-26 13:46:02 +02:00
1656d73340
fixed formatting and other suggestions
2022-10-26 11:58:01 +02:00
f14c609db2
Add batch import
2022-10-26 09:30:02 +02:00
bab8309d32
Parse users without email
2022-10-25 10:21:28 +02:00
10cd7e4e38
execute user import
2022-10-24 18:12:44 +02:00
5ed847de8b
added user table and import function
2022-10-21 15:16:34 +02:00
Stephan Kulow
eae411941b
Don't crash on delete requests
2022-10-20 08:57:14 +02:00
Stephan Kulow
988b10e4b7
Add also an index to the requests table
2022-10-20 07:22:48 +02:00
Stephan Kulow
0a3816be67
Create index for request_number
2022-10-20 07:00:26 +02:00
Stephan Kulow
fa9ec8618e
Load requests into the DB - not sure yet what for :)
2022-10-19 16:35:33 +02:00
Stephan Kulow
771595ba85
Catch invalid XML in links
2022-10-18 21:36:58 +02:00
Stephan Kulow
8dcae5df50
Remove debug output
2022-10-18 19:31:41 +02:00
Stephan Kulow
89c0335812
Import file list of revisions
2022-10-18 19:29:25 +02:00
Stephan Kulow
d747a3a761
Calculate the linked revision
2022-10-18 15:40:11 +02:00
Stephan Kulow
fb130bbf6f
Import packages links point to
2022-10-18 14:57:23 +02:00
Stephan Kulow
e8094cbc68
Import the link information into an extra table
2022-10-18 13:48:27 +02:00
Stephan Kulow
95412bc834
Prepare link collection
2022-10-18 13:13:52 +02:00
Stephan Kulow
25b45c5073
Split Revision into OBS and DB
2022-10-18 12:17:43 +02:00
Stephan Kulow
c534fb028e
Add a small test case
2022-10-17 20:52:05 +02:00
Stephan Kulow
8563334b07
Add experimental import into DB
2022-10-17 20:39:03 +02:00
Stephan Kulow
a82562b794
Move modules into lib subdir
2022-10-17 19:56:41 +02:00
