|
|
6dd3cf3eba
|
Merge pull request 'implement file caching' (#11) from file-cache into main
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/11
|
2022-11-03 14:24:22 +01:00 |
|
|
|
8aed76e52a
|
change cached file naming pattern
|
2022-11-03 14:22:19 +01:00 |
|
|
|
639096b548
|
optimize cached file locations and add option for cache directory
|
2022-11-03 14:12:32 +01:00 |
|
|
|
7678967ae0
|
implement file caching
to prevent having to download files multiple times
|
2022-11-03 14:05:11 +01:00 |
|
coolo
|
74f5cd901e
|
Merge pull request 'Keep a reference to the database in DBRevision' (#12) from add_export into main
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/12
|
2022-11-03 08:16:30 +01:00 |
|
Stephan Kulow
|
1c54a74ecd
|
Download the full revision
|
2022-11-02 20:55:09 +01:00 |
|
Stephan Kulow
|
c2294d6200
|
Add a default LFS .gitattributes for now
Otherwise some packages will break to import
|
2022-11-02 18:27:17 +01:00 |
|
Stephan Kulow
|
ba7436f10c
|
Keep a reference to the database in DBRevision
To avoid passing the db to all actions
|
2022-11-02 18:27:09 +01:00 |
|
coolo
|
75f9f56a57
|
Merge pull request 'Fix up some code after aplanas' continued review' (#10) from add_export into main
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/10
|
2022-11-02 18:05:04 +01:00 |
|
Stephan Kulow
|
172242891d
|
Fix up some code after aplanas' continued review
|
2022-11-02 15:22:24 +01:00 |
|
coolo
|
6ac0f90ec7
|
Merge pull request 'Export the built tree as git repo' (#9) from add_export into main
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/9
|
2022-11-02 14:17:24 +01:00 |
|
Stephan Kulow
|
05cf792b26
|
Add the file_md5 to the download function so it can cache and verify
|
2022-11-02 13:35:45 +01:00 |
|
Stephan Kulow
|
a1ead29734
|
Extend documention and use some more pythonier loops
|
2022-11-02 13:29:18 +01:00 |
|
Stephan Kulow
|
05a5e6aea7
|
Don't refresh packages we're already looking at
|
2022-11-02 10:52:53 +01:00 |
|
Stephan Kulow
|
68ded48be1
|
Don't crash on user with missing realname either
|
2022-11-02 10:52:37 +01:00 |
|
Stephan Kulow
|
fce8aac001
|
Import creator users as well
|
2022-11-02 08:59:25 +01:00 |
|
Stephan Kulow
|
bbf1bc2fda
|
Fetch source projects of requests
We do not care for current devel project, but for the projects we saw
requests from
|
2022-11-02 08:50:54 +01:00 |
|
Stephan Kulow
|
8a863498db
|
Move default repos into a subdirectory that is git ignored
|
2022-11-02 08:01:22 +01:00 |
|
Stephan Kulow
|
c4654dd896
|
Split GitExporter out of Importer class
|
2022-11-02 07:59:25 +01:00 |
|
Stephan Kulow
|
9de0d6e6c5
|
Rename Exporter to TestExporter to make it more obvious
|
2022-11-02 07:39:04 +01:00 |
|
Stephan Kulow
|
a8dfd80fdd
|
Reduce zsh fixture data 2 years to ease test runtime
The interesting parts are before 2012 anyway
|
2022-11-02 07:24:24 +01:00 |
|
Stephan Kulow
|
4ff9b9771a
|
Split out Flat generator to be able to test it
|
2022-11-02 07:20:53 +01:00 |
|
Stephan Kulow
|
c94d13d74e
|
Don't crash on packages without merges ever (very few packages)
|
2022-11-01 19:30:41 +01:00 |
|
Stephan Kulow
|
ab8120ca53
|
Don't crash on last_node
|
2022-11-01 19:02:29 +01:00 |
|
Stephan Kulow
|
b2cadb8c01
|
Don't crash on packages that didn't get updates in devel
|
2022-11-01 18:44:59 +01:00 |
|
Stephan Kulow
|
578fb2a30a
|
Change tree pruning algorithm
The first merge we see in Factory determines if we keep the devel
commits in the factory chain or cut that branch.
|
2022-11-01 13:52:15 +01:00 |
|
Stephan Kulow
|
9aaba03bcc
|
Add llvm13 as test case as I'm missing rev 3 in the export
|
2022-11-01 11:45:20 +01:00 |
|
Stephan Kulow
|
e6a401d8ac
|
Remove old history handling
|
2022-11-01 11:37:30 +01:00 |
|
Stephan Kulow
|
9ed8abad2b
|
Make database usage the default
Some cleanup of no longer used functions
|
2022-11-01 11:23:40 +01:00 |
|
Stephan Kulow
|
9554fea7e1
|
Reuse the repository directory by storing a state yaml
Not using the database for that so that removing the repository directory will
automatically recreate it
|
2022-11-01 11:22:58 +01:00 |
|
Stephan Kulow
|
2168c898a2
|
Add users not known to the FAKE_ACCOUNTS
This is technically incorrect but we need to handle them all the same anyway
|
2022-11-01 09:12:42 +01:00 |
|
Stephan Kulow
|
cfab0a522b
|
Download .spec and .changes file only for now
|
2022-10-31 16:31:10 +01:00 |
|
Stephan Kulow
|
bfdade8ecf
|
Create a flat list of commits to do from the tree
Walk the node tree and record the parents, then reverse the tree so we
can have the exact order in which to create git commits
|
2022-10-31 12:28:12 +01:00 |
|
Stephan Kulow
|
8a3db6c183
|
Collect the requests for revisions in one DB call
For smaller packages this doesn't matter as much, but for large ones
the N+1 queries can sum up badly
|
2022-10-31 08:03:01 +01:00 |
|
Stephan Kulow
|
822329279d
|
Switch to walker for tree print
|
2022-10-31 07:03:19 +01:00 |
|
|
|
6f83425538
|
Merge pull request 'Add a test case for a simpler and newer package' (#8) from add_second_test into main
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/8
|
2022-10-27 15:19:47 +02:00 |
|
Stephan Kulow
|
83544e45d7
|
Add a test case for a simpler and newer package
|
2022-10-27 15:15:41 +02:00 |
|
coolo
|
62511d15bc
|
Merge pull request 'Add actual tree and a test case for zsh' (#7) from add_tree into main
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/7
|
2022-10-27 14:04:52 +02:00 |
|
Stephan Kulow
|
e636f3ed9d
|
Print the calculated tree for now
|
2022-10-27 13:34:04 +02:00 |
|
Stephan Kulow
|
7a9e9ce334
|
Add actual tree and a test case for zsh
|
2022-10-27 13:16:50 +02:00 |
|
Stephan Kulow
|
65000cd586
|
Fix even more issues with the linked_revs considered cache
|
2022-10-27 10:16:20 +02:00 |
|
coolo
|
996b90dd7a
|
Merge pull request 'Fix calculation of the tree data' (#5) from fix_fake into main
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/5
|
2022-10-27 10:01:33 +02:00 |
|
Stephan Kulow
|
38e4996280
|
Fix the fake revisions after introduction of cache
The way I stored it in the linked_revs was the wrong way
|
2022-10-27 07:35:08 +02:00 |
|
Stephan Kulow
|
9bd6643e8a
|
Fix files_hash calculcation and fake revisions
|
2022-10-26 22:18:16 +02:00 |
|
Stephan Kulow
|
1c846e963c
|
Import files of revisions more reliable and faster
|
2022-10-26 21:47:39 +02:00 |
|
coolo
|
22b70d8bb7
|
Merge pull request 'Build a Tree of commits (at least prep it)' (#4) from tree_builder into main
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/4
|
2022-10-26 19:33:13 +02:00 |
|
Stephan Kulow
|
4dc7e972a2
|
Make use of db.cursor() in with blocks
|
2022-10-26 15:49:14 +02:00 |
|
Stephan Kulow
|
217bbcadb5
|
Run the formatter from make all
|
2022-10-26 14:57:08 +02:00 |
|
Stephan Kulow
|
1346223822
|
Reshuffle schema versions after rebase
|
2022-10-26 14:56:38 +02:00 |
|
Stephan Kulow
|
9dbf3b9094
|
Import all projects before checking files - otherwise we end with 0s
|
2022-10-26 14:56:38 +02:00 |
|
