Additional changes:
* rename getbinaries --debug to --debuginfo to avoid conflicts
* switch conf['verbose'] from int to bool
The changes were needed because the new argparser behaves differently
and commands such as `osc ls` ran in verbose mode by default.
If `sshkey` config option is set, then osc prefers it over password auth.
If `sshkey` config option is not set and the server supports both basic
and signature auth, basic auth is used and ssh key is NOT auto-detected.
Users who want to use ssh auth with ssh key auto-detection can now leave
the `pass` config option empty to trigger ssh key auto-detection.
The ssh-key autodetection picks the first key that matches:
- key loaded to ssh-agent (`ssh-add -l`) that has a public key in ~/.ssh
- ~/.ssh/{id_ed25519,id_rsa}
It is also recommended to use Obfuscated or Plaintext credentials manager.
Please be aware that storing passwords using these credentials managers
is unsafe, because they're stored in plain text on disk.
Example:
[<apiurl>]
user=<username>
pass=
# ssh key is auto-detected because `pass` is empty
sshkey=
credentials_mgr_class=osc.credentials.ObfuscatedConfigFileCredentialsManager
When using ssh keys from gpg, there are no private key files on
disk. The public keys are available from "ssh-add -L". Conveniently,
users store the public keys in some ".pub" file under ~/.ssh
(see e.g. https://serverfault.com/questions/906871/force-the-use-of-a-gpg-key-as-an-ssh-key-for-a-given-server;
this is also necessary to use IdentityFile= in ssh itself).
Thus public key files can't be ignored any more in list_ssh_dir_keys().
"ssh-keygen -Y sign" works nicely with a public key file if the agent
has access to the private key.
* init command is working inside of a git repository
* downloadassets command fetches references assets from build description
* checkout is cloning from git
This changes the code back to retrying up to 5 times for old
python version 2.6.6-2.7.9. The complete backport of the basic auth
changes clutters up the code way to much for such a little gain.
(This basically reverts commit 326abe0c8b)
Only ask for a password if it is really needed for authentication.
The new lazy password approach is much smarter than the old callable
hack. That's why we deprecate returning a callable from
AbstractCredentialsManager.get_password. The current compatibility code
for a callable will be removed in the near future.
Minor nitpick: actually it would have been "cleaner" to introduce a new
subclass like an AbstractLazyPasswordCredentialsManager that encapsulates
the lazy password behavior. Currently, if, for instance, a credentials
manager is always non-lazy it would just override get_password but still
inherits the abstract (and unused) _get_password method.
In many cases the session cookie is already available, so there
is no need to ask for a password. To make this work with the
python authentication implementation, we add a small proxy object
for the password and only ask the credential manager if the
stringify method is called.
This approach also makes it possible to offer a non-password based
authorization type if the server allows multiple authentication
methods.
This allows a format to be specified in the sccache uri, specifically
the file uri so that a per-package cache can be created. This way
an osc build locally doesn't ruin your cache moving between different
packages.
When a user creates a symlink pointing from ~/.config/osc/oscrc
to a different location, don't overwrite the symlink but follow
it when writing configuration on disk.
os.sysconf is not available on all platforms (like Windows) but it
is used to retrieve the number of online processors. If missing,
assume one processor (building on such a platform will most likely
not work, though).
Fixes: #948 ("Windows compatibility") (at least it improves the
Windows support a bit)
The order is now:
- ~/.osc_cookiejar, if it exists
- $XDG_STATE_HOME/osc/cookiejar if XDG_STATE_HOME neither null nor empty
- ~/.local/state/osc/cookiejar
A password can be deleted via "osc config -d <apiurl> pass". Actually,
if we really want to support password deletion, we should introduce
a --delete-password option because the "pass" config option can be
considered as an implementation detail, which we should not expose
to our users.
The password store can be changed (without entering the password
again) via "osc config <apiurl> --select-password-store". This
command deletes the password from the current password store and
stores it in the selected password store.
Previously, the --select-password-store option had no meaningful
semantics. In order to use it, one always had to provide a password
and explicitly pass "pass" as the config option (the same could be
achieved by using --change-password). Hence, in a strict sense,
this change breaks the UI.
Sccache is an alternate build caching system to ccache/icecream. It
supports C, C++ and Rust. It can optionally have distributed or remote
caches via redis, s3 object stores, memcached, azure storage or
google cloud storage.
This can help to significantly improve the performance of Rust rebuilds.
For example, Kanidm changes from 400s to 122s on a rebuild, and rust-lang
rebuilds improve from 7200s to 4770s. With some changes to the rust
packages especially this will be possible to speed up over version
changes as well.
See also: obs-build PR https://github.com/openSUSE/obs-build/pull/680
The old code only supports a file whose size is less then or equal
to INT_MAX (due to a reasonable(!) limit in M2Crypto). The actual
issue is in core.http_request which mmap(...)s the file, wraps it
into a memoryview/buffer and then passes the memoryview/buffer to
urlopen. Eventually, the whole memoryview/buffer is read into memory
(see m2_PyObject_GetBufferInt). If the file is too large (> INT_MAX),
m2_PyObject_GetBufferInt raises a ValueError (which is perfectly
fine!).
Reading a whole file into memory is completely insane. In order to
avoid this, we now simply pass a file-like object to urlopen (more
precisely, the file-like object is associated with the Request
instance that is passed to urlopen). The advantange is that the
file-like object is processed in chunks of 8192 bytes (see
http.client.HTTPConnection) (that is, only 8192 bytes are read into
memory (instead of the whole file)).
There are two pitfalls when passing a file-like object to urlopen:
* By default, a chunked Transfer-Encoding is applied. It seems that
some servers (like api.o.o) do not like this (PUTing a file with
a chunked Transfer-Encoding to api.o.o results in status 400). In
order to avoid a chunked Transfer-Encoding, we explicitly set a
Content-Length header (we also do this in the non-file case (just
for the sake of completeness)).
* If the request fails with status 401, it is retried with an
appropriate Authorization header. When retrying the request, the
file's offset has to be repositioned to the beginning of the file
(otherwise, a 0-length body is sent which most likely does not
match the Content-Length header).
Note: core.http_request's "data" and "file" parameters are now mutually
exclusive because specifying both makes no sense (only one of them
is considered) and it simplifies the implementation a bit.
Fixes: #202 ("osc user authentification seems to be broken with last
commit")
Fixes: #304 ("osc ci - cannot handle more than 2 GB file uploads")
