importers/git-importer
3
0
Fork 2
Code Issues Pull Requests Packages Projects Releases Wiki Activity
55 Commits 4 Branches 0 Tags 1.8 MiB
8a3db6c183
Commit Graph

55 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Stephan Kulow
8a3db6c183 Collect the requests for revisions in one DB call 
For smaller packages this doesn't matter as much, but for large ones
the N+1 queries can sum up badly
 2022-10-31 08:03:01 +01:00
Stephan Kulow
822329279d Switch to walker for tree print 2022-10-31 07:03:19 +01:00
Alberto Planas
6f83425538 Merge pull request 'Add a test case for a simpler and newer package' (#8) from add_second_test into main 
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/8
 2022-10-27 15:19:47 +02:00
Stephan Kulow
83544e45d7 Add a test case for a simpler and newer package 2022-10-27 15:15:41 +02:00
coolo
62511d15bc Merge pull request 'Add actual tree and a test case for zsh' (#7) from add_tree into main 
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/7
 2022-10-27 14:04:52 +02:00
Stephan Kulow
e636f3ed9d Print the calculated tree for now 2022-10-27 13:34:04 +02:00
Stephan Kulow
7a9e9ce334 Add actual tree and a test case for zsh 2022-10-27 13:16:50 +02:00
Stephan Kulow
65000cd586 Fix even more issues with the linked_revs considered cache 2022-10-27 10:16:20 +02:00
coolo
996b90dd7a Merge pull request 'Fix calculation of the tree data' (#5) from fix_fake into main 
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/5
 2022-10-27 10:01:33 +02:00
Stephan Kulow
38e4996280 Fix the fake revisions after introduction of cache 
The way I stored it in the linked_revs was the wrong way
 2022-10-27 07:35:08 +02:00
Stephan Kulow
9bd6643e8a Fix files_hash calculcation and fake revisions 2022-10-26 22:18:16 +02:00
Stephan Kulow
1c846e963c Import files of revisions more reliable and faster 2022-10-26 21:47:39 +02:00
coolo
22b70d8bb7 Merge pull request 'Build a Tree of commits (at least prep it)' (#4) from tree_builder into main 
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/4
 2022-10-26 19:33:13 +02:00
Stephan Kulow
4dc7e972a2 Make use of db.cursor() in with blocks 2022-10-26 15:49:14 +02:00
Stephan Kulow
217bbcadb5 Run the formatter from make all 2022-10-26 14:57:08 +02:00
Stephan Kulow
1346223822 Reshuffle schema versions after rebase 2022-10-26 14:56:38 +02:00
Stephan Kulow
9dbf3b9094 Import all projects before checking files - otherwise we end with 0s 2022-10-26 14:56:38 +02:00
Stephan Kulow
d961c13614 Ignore commits after the merge 2022-10-26 14:56:38 +02:00
Stephan Kulow
2784171f75 Create fake revisions for every commit in the base of a linked package 
This is a rather complex operation, but whenever a package changes in
Factory, the inherited package gets a shadow revision consisting of the
3-way merge. If this happens due to a request being accepted, this is
actually in most cases also commited by 'buildservice-autocommit',
so we're making sure this is always happening (and are actually
duplicating revisions in cases that we filter out later as empty
commits).

To differenciate the fake revisions from the real revisions, I add
a fraction part
 2022-10-26 14:56:38 +02:00
Stephan Kulow
d17e60a608 Add another table to store linked_revs 
We need to create fake revisions when packages were touched that are
linked themselves
 2022-10-26 14:20:28 +02:00
Stephan Kulow
089b215f4c Build a revision tree 2022-10-26 14:16:22 +02:00
Stephan Kulow
bc74b6950b Implement a fileshash function for revision 2022-10-26 14:15:42 +02:00
Stephan Kulow
5bd790e915 Import the fixtures into the test database for queries 2022-10-26 14:15:41 +02:00
Stephan Kulow
ce3863e1b5 Add the capability to export the package data as test fixture 2022-10-26 14:13:24 +02:00
nkrapp
87d9fcc131 Merge pull request 'add a user database' (#3) from user-db into main 
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/3
 2022-10-26 14:05:47 +02:00
nkrapp
dbeb534108
cleanup context managers 2022-10-26 13:46:02 +02:00
nkrapp
1656d73340
fixed formatting and other suggestions 2022-10-26 11:58:01 +02:00
nkrapp
f14c609db2
Add batch import 2022-10-26 09:30:02 +02:00
nkrapp
bab8309d32
Parse users without email 2022-10-25 10:21:28 +02:00
nkrapp
10cd7e4e38
execute user import 2022-10-24 18:12:44 +02:00
nkrapp
5ed847de8b
added user table and import function 2022-10-21 15:16:34 +02:00
Stephan Kulow
eae411941b Don't crash on delete requests 2022-10-20 08:57:14 +02:00
Stephan Kulow
988b10e4b7 Add also an index to the requests table 2022-10-20 07:22:48 +02:00
Stephan Kulow
0a3816be67 Create index for request_number 2022-10-20 07:00:26 +02:00
Stephan Kulow
fa9ec8618e Load requests into the DB - not sure yet what for :) 2022-10-19 16:35:33 +02:00
Stephan Kulow
771595ba85 Catch invalid XML in links 2022-10-18 21:36:58 +02:00
Stephan Kulow
8dcae5df50 Remove debug output 2022-10-18 19:31:41 +02:00
Stephan Kulow
89c0335812 Import file list of revisions 2022-10-18 19:29:25 +02:00
Stephan Kulow
1a750f2171 Testing to fetch it back 2022-10-18 17:58:38 +02:00
Stephan Kulow
d747a3a761 Calculate the linked revision 2022-10-18 15:40:11 +02:00
Stephan Kulow
fb130bbf6f Import packages links point to 2022-10-18 14:57:23 +02:00
Stephan Kulow
e8094cbc68 Import the link information into an extra table 2022-10-18 13:48:27 +02:00
Stephan Kulow
95412bc834 Prepare link collection 2022-10-18 13:13:52 +02:00
Stephan Kulow
25b45c5073 Split Revision into OBS and DB 2022-10-18 12:17:43 +02:00
Stephan Kulow
c534fb028e Add a small test case 2022-10-17 20:52:05 +02:00
Stephan Kulow
365696213b Put all the formatters into a Makefile (just a shell script wrapper) 2022-10-17 20:40:19 +02:00
Stephan Kulow
8563334b07 Add experimental import into DB 2022-10-17 20:39:03 +02:00
Stephan Kulow
a82562b794 Move modules into lib subdir 2022-10-17 19:56:41 +02:00
Stephan Kulow
88322fffae Split the database config into 2 sections 2022-10-17 19:50:25 +02:00
Stephan Kulow
74fd937732 Add basic postgresql setup 2022-10-17 16:31:48 +02:00