Dirk Müller
022ae5ab58
remember failed tasks in a separate directory
2024-06-10 17:04:43 +02:00
Dirk Müller
2ff8ed76d0
Reconnect to the AMQP bus when the connection breaks down
2024-06-10 17:04:25 +02:00
Dirk Müller
5f228dc046
enable robust push
2024-05-17 21:47:35 +02:00
Dirk Müller
4e07d8272e
don't loop over failed packages
2024-05-17 21:47:15 +02:00
Dirk Müller
2a3475ab6e
Create with sha256 enabled
2024-05-17 20:39:55 +02:00
Dirk Müller
574bc9aa10
Avoid guessing in switch
2024-05-17 20:07:16 +02:00
Dirk Müller
0414b33206
Fix testing for origin
...
The previous code path was untested and not working
2024-05-17 20:06:25 +02:00
Dirk Müller
b9670821a9
Only init the repository if it doesn't exist already
...
harmless, but avoids a scary warning
2024-05-17 20:05:54 +02:00
Dirk Müller
073550825c
Fixups to improve the conversion process
2024-05-17 14:41:42 +02:00
Dirk Müller
5a353c98d3
Add tasks
2024-05-17 11:46:18 +02:00
Dirk Müller
1fc466d15b
Add monitor for commits
2024-05-17 11:40:19 +02:00
Dirk Müller
39fde7744a
Code cleanup
2024-05-16 15:47:45 +02:00
Dirk Müller
f5ffc83a69
Remove double quoting of url parameters
...
makeurl quotes by itself, so this was messing it up
2024-05-16 11:49:14 +02:00
Dirk Müller
d0ccf83684
Revert "Try to fetch the element as deleted if initial access failed"
...
The OBS api has been fixed to provide an automatic fallback via
https://github.com/openSUSE/open-build-service/pull/15655
This reverts commit c9e07e536f
.
2024-05-16 11:49:14 +02:00
Dirk Müller
b0ffb01c59
cleanups
2024-05-16 11:49:14 +02:00
Dirk Müller
28d5c6e606
Switch to psycopg rather than psycopg2
...
It's a bit more modern and uses dedicated c bindings
2024-05-16 11:49:14 +02:00
Dirk Mueller
1e22c2895a
Merge pull request 'Switch to sha-256 git repo and use git tools again' ( #23 ) from adamm/git-importer:main into main
...
Reviewed-on: importers/git-importer#23
2024-05-16 11:48:36 +02:00
Adam Majer
5da7861c2a
Switch to sha-256 git repo and use git tools again
2024-04-09 11:40:26 +02:00
Dirk Müller
c9e07e536f
Try to fetch the element as deleted if initial access failed
...
The reference to the object might be already deleted by when the
request is failing. plus setting deleted=0 is rejected by the API.
So try with deleted=1 if and only if the previous access failed.
2023-12-07 18:30:36 +01:00
Dirk Müller
dc0f33354e
Failing to LFS register should abort the import
2023-12-07 18:29:56 +01:00
Dirk Müller
56cbe0a125
Avoid multi-threading races on import
...
There seems to be races when using db cursors from multiple threads. as
found by import issues after switching to a newer computer that has
performance and energy efficient cores.
As this is not particularly performance critical, convert to single
threaded use which makes it work again
2023-11-28 23:36:44 +01:00
Dirk Müller
4353f015c8
Switch to localhost:9999 which is provided via a ssh tunnel
...
The port is no longer directly exposed, so we need to ssh tunnel it
2023-11-22 14:39:55 +01:00
Dirk Müller
9cbe0899bc
Remove unused import
2023-06-19 13:19:52 +02:00
Dirk Müller
9e80a64fe0
Change hostname references from gitea.opensuse.org to src.opensuse.org
2023-06-19 10:59:56 +02:00
Dirk Müller
12001b1640
Commit local changes
2023-04-18 22:31:38 +02:00
Stephan Kulow
3797ea178a
Merge pull request 'Add a list of packages no longer existing' ( #22 ) from add_gone into main
...
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/22
2023-02-09 10:23:35 +01:00
Stephan Kulow
999dcabcfa
Add a list of packages no longer existing
...
I made this a file and not a DB that is automatically maintained as I think
for now adding an entry in there should be done manually - OBS being OBS
packages might look have gone for a brief moment and reappar the day after.
2022-12-02 11:00:31 +01:00
9962673eff
Merge pull request 'Add force push for the devel branch' ( #21 ) from add_force into main
...
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/21
2022-12-02 09:35:40 +01:00
Stephan Kulow
7b20c03256
Add force push for the devel branch
...
As devel branches can change in case of factory reverts we need to force
push. Factory branch shouldn't be affected, so not force pushing there
2022-12-02 09:12:11 +01:00
Stephan Kulow
4692d47120
Make the refresh a debug output, not info
2022-11-16 09:05:36 +01:00
coolo
d311d54f26
Merge pull request 'Also treat some more mimetypes as text' ( #20 ) from add_further_mimetypes into main
...
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/20
2022-11-15 07:28:05 +01:00
Stephan Kulow
dddc54ab1c
Remove ProcessPool from exporting
...
It's ignoring exceptions and makes debugging way too hard to justify
what's happening
2022-11-11 16:33:44 +01:00
Stephan Kulow
4d1ca8d882
Also treat some more mimetypes as text
2022-11-11 16:22:18 +01:00
Stephan Kulow
7861a7e9b0
Fix LFS register (it needs json not data)
...
Refactored the LFS Oid handling in its class of its own and
add a way to recheck all LFS handles (or re-register)
2022-11-09 08:32:18 +01:00
coolo
f5b29886ae
Merge pull request 'No longer rely on external service for LFS tracking' ( #18 ) from add_lfs into main
...
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/18
2022-11-08 11:00:34 +01:00
coolo
d1a8a3288d
Merge pull request 'Push to the remote when the repo changed' ( #19 ) from push_it_baby into main
...
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/19
2022-11-08 09:48:56 +01:00
Stephan Kulow
9f6c8f62e7
Push to the remote when the repo changed
2022-11-08 09:32:03 +01:00
Stephan Kulow
3e1fbaa1c3
Migrate the ProxySHA256 data into postgresql DB
...
The calculation of the sha256 and the mimetype is local due to that
2022-11-07 21:50:31 +01:00
Stephan Kulow
e1b32999f0
Fix confusion about User constructor
2022-11-07 16:04:44 +01:00
coolo
86490b51dd
Merge pull request 'Fix the maintenance of .gitattributes file' ( #17 ) from fix_lfs_attributes into main
...
Reviewed-on: https://gitea.opensuse.org/importers/git-importer/pulls/17
2022-11-07 13:26:36 +01:00
Stephan Kulow
da5de04171
Add packages to consider
2022-11-07 07:29:20 +01:00
Stephan Kulow
be8fb2ab94
Fix fake revision creation
2022-11-06 12:27:36 +01:00
Stephan Kulow
9e895e34b6
Adding a gitea remote when creating the git repo
2022-11-06 12:18:16 +01:00
Stephan Kulow
5e495dbd95
Fancy up the git commit message
2022-11-06 11:46:04 +01:00
Stephan Kulow
5ae02a413d
Store API URL in the revision table
...
Will be important once we get into SLE
2022-11-06 10:57:32 +01:00
Stephan Kulow
f1457e8f8e
Move git commit message creation into class
2022-11-06 10:16:42 +01:00
Stephan Kulow
9114c2fff8
Change debug output for downloading files
2022-11-06 10:16:42 +01:00
Stephan Kulow
834cf61634
Use proper user info in commits
2022-11-06 09:53:52 +01:00
Stephan Kulow
a294c0f670
Readd the skipping of _staging_workflow file
...
A repository with 150k commits is just very hard to work with -
especially if 99% of them are worthless
2022-11-06 08:29:17 +01:00
Stephan Kulow
7bc4d6c8b1
Make downloading a little more careful for races
...
As we're downloading packages in parallel, it could happen that we
copy a file that isn't fully copied yet
2022-11-06 08:24:11 +01:00