Export the built tree as git repo #9

Merged
Ghost merged 24 commits from add_export into main 2022-11-02 14:17:24 +01:00

24 Commits

Author SHA1 Message Date
Stephan Kulow
05cf792b26 Add the file_md5 to the download function so it can cache and verify 2022-11-02 13:35:45 +01:00
Stephan Kulow
a1ead29734 Extend documention and use some more pythonier loops 2022-11-02 13:29:18 +01:00
Stephan Kulow
05a5e6aea7 Don't refresh packages we're already looking at 2022-11-02 10:52:53 +01:00
Stephan Kulow
68ded48be1 Don't crash on user with missing realname either 2022-11-02 10:52:37 +01:00
Stephan Kulow
fce8aac001 Import creator users as well 2022-11-02 08:59:25 +01:00
Stephan Kulow
bbf1bc2fda Fetch source projects of requests
We do not care for current devel project, but for the projects we saw
requests from
2022-11-02 08:50:54 +01:00
Stephan Kulow
8a863498db Move default repos into a subdirectory that is git ignored 2022-11-02 08:01:22 +01:00
Stephan Kulow
c4654dd896 Split GitExporter out of Importer class 2022-11-02 07:59:25 +01:00
Stephan Kulow
9de0d6e6c5 Rename Exporter to TestExporter to make it more obvious 2022-11-02 07:39:04 +01:00
Stephan Kulow
a8dfd80fdd Reduce zsh fixture data 2 years to ease test runtime
The interesting parts are before 2012 anyway
2022-11-02 07:24:24 +01:00
Stephan Kulow
4ff9b9771a Split out Flat generator to be able to test it 2022-11-02 07:20:53 +01:00
Stephan Kulow
c94d13d74e Don't crash on packages without merges ever (very few packages) 2022-11-01 19:30:41 +01:00
Stephan Kulow
ab8120ca53 Don't crash on last_node 2022-11-01 19:02:29 +01:00
Stephan Kulow
b2cadb8c01 Don't crash on packages that didn't get updates in devel 2022-11-01 18:44:59 +01:00
Stephan Kulow
578fb2a30a Change tree pruning algorithm
The first merge we see in Factory determines if we keep the devel
commits in the factory chain or cut that branch.
2022-11-01 13:52:15 +01:00
Stephan Kulow
9aaba03bcc Add llvm13 as test case as I'm missing rev 3 in the export 2022-11-01 11:45:20 +01:00
Stephan Kulow
e6a401d8ac Remove old history handling 2022-11-01 11:37:30 +01:00
Stephan Kulow
9ed8abad2b Make database usage the default
Some cleanup of no longer used functions
2022-11-01 11:23:40 +01:00
Stephan Kulow
9554fea7e1 Reuse the repository directory by storing a state yaml
Not using the database for that so that removing the repository directory will
automatically recreate it
2022-11-01 11:22:58 +01:00
Stephan Kulow
2168c898a2 Add users not known to the FAKE_ACCOUNTS
This is technically incorrect but we need to handle them all the same anyway
2022-11-01 09:12:42 +01:00
Stephan Kulow
cfab0a522b Download .spec and .changes file only for now 2022-10-31 16:31:10 +01:00
Stephan Kulow
bfdade8ecf Create a flat list of commits to do from the tree
Walk the node tree and record the parents, then reverse the tree so we
can have the exact order in which to create git commits
2022-10-31 12:28:12 +01:00
Stephan Kulow
8a3db6c183 Collect the requests for revisions in one DB call
For smaller packages this doesn't matter as much, but for large ones
the N+1 queries can sum up badly
2022-10-31 08:03:01 +01:00
Stephan Kulow
822329279d Switch to walker for tree print 2022-10-31 07:03:19 +01:00