If an error occurs while sending a file to the API (e.g. 401), the io handle
is reused and the current postion is at the end of the file. This results in an
empty value.
This patch takes care that the current file postion gets resetted
if a retransmission is required.
Additional changes:
* rename getbinaries --debug to --debuginfo to avoid conflicts
* switch conf['verbose'] from int to bool
The changes were needed because the new argparser behaves differently
and commands such as `osc ls` ran in verbose mode by default.
Cmdln.py is unmaintained for years and uses deprecated optparse.
Let's replace it with a simpler custom code. Also remove code
that generates man page, we'll replace it with a 3rd party tool.
If `sshkey` config option is set, then osc prefers it over password auth.
If `sshkey` config option is not set and the server supports both basic
and signature auth, basic auth is used and ssh key is NOT auto-detected.
Users who want to use ssh auth with ssh key auto-detection can now leave
the `pass` config option empty to trigger ssh key auto-detection.
The ssh-key autodetection picks the first key that matches:
- key loaded to ssh-agent (`ssh-add -l`) that has a public key in ~/.ssh
- ~/.ssh/{id_ed25519,id_rsa}
It is also recommended to use Obfuscated or Plaintext credentials manager.
Please be aware that storing passwords using these credentials managers
is unsafe, because they're stored in plain text on disk.
Example:
[<apiurl>]
user=<username>
pass=
# ssh key is auto-detected because `pass` is empty
sshkey=
credentials_mgr_class=osc.credentials.ObfuscatedConfigFileCredentialsManager
When using ssh keys from gpg, there are no private key files on
disk. The public keys are available from "ssh-add -L". Conveniently,
users store the public keys in some ".pub" file under ~/.ssh
(see e.g. https://serverfault.com/questions/906871/force-the-use-of-a-gpg-key-as-an-ssh-key-for-a-given-server;
this is also necessary to use IdentityFile= in ssh itself).
Thus public key files can't be ignored any more in list_ssh_dir_keys().
"ssh-keygen -Y sign" works nicely with a public key file if the agent
has access to the private key.
Emojis were broken due to this commit.
The revert causes a change in behavior:
"\n" no longer gets resolved to a newline.
It is suggested to do the following instead:
$ osc <command> ... -m "first line
second line"
This reverts commit 16fda3115c.
* src/noarch rpm packaages needs to be stored in scheduler architecture to avoid
conflicts of the multiple versions
* avoid removal of every downloaded file
There seem to be a bug in how GitHub generates archives.
"Format:" and "$" characters get removed from the version string,
setting it to:
version = "%(describe:tags=true)"
Fixes the following error:
```
% osc
Traceback (most recent call last):
File "/usr/bin/osc", line 45, in <module>
r = babysitter.run(osccli)
File "/usr/lib/python3.10/site-packages/osc/babysitter.py", line 67, in run
return prg.main(argv)
File "/usr/lib/python3.10/site-packages/osc/cmdln.py", line 341, in main
self.postoptparse()
File "/usr/lib/python3.10/site-packages/osc/commandline.py", line 137, in postoptparse
if self._get_canonical_cmd_name(self.args[0]) == "help":
IndexError: list index out of range
```
* init command is working inside of a git repository
* downloadassets command fetches references assets from build description
* checkout is cloning from git
This changes the code back to retrying up to 5 times for old
python version 2.6.6-2.7.9. The complete backport of the basic auth
changes clutters up the code way to much for such a little gain.
(This basically reverts commit 326abe0c8b)
It is possible that the self._pwfunc() call returns a callable. For
instance, if the keyutils.osc.OscKernelKeyringBackend is configured
in the oscrc. Hence, check in credentials._LazyPassword.__str__
if the returned password is a callable and, if so, call it. Moreover,
a deprecation warning is printed. Eventually, this compat code will
be removed again.
This is a follow-up commit for commit
784d330f20 ("Only prompt for a password
if the server asks for it") (actually, it is a regression that was
not caught during the review...).
Only ask for a password if it is really needed for authentication.
The new lazy password approach is much smarter than the old callable
hack. That's why we deprecate returning a callable from
AbstractCredentialsManager.get_password. The current compatibility code
for a callable will be removed in the near future.
Minor nitpick: actually it would have been "cleaner" to introduce a new
subclass like an AbstractLazyPasswordCredentialsManager that encapsulates
the lazy password behavior. Currently, if, for instance, a credentials
manager is always non-lazy it would just override get_password but still
inherits the abstract (and unused) _get_password method.
In many cases the session cookie is already available, so there
is no need to ask for a password. To make this work with the
python authentication implementation, we add a small proxy object
for the password and only ask the credential manager if the
stringify method is called.
This approach also makes it possible to offer a non-password based
authorization type if the server allows multiple authentication
methods.
This applies when downloading multiple packages, typically the whole repo.
When downloading a single package, everything works as usual
and the subdir is not created.
This allows a format to be specified in the sccache uri, specifically
the file uri so that a per-package cache can be created. This way
an osc build locally doesn't ruin your cache moving between different
packages.
They have identical names for all downloaded packages
and get overwritten by the last downloaded file.
Unless we dowload them into subdirs or prefix them with package name,
it makes no sense to download them.
The ':' character is used as a separator in Open Build Service
and constantly appears in directory names after running osc commands.
Windows do not support ':' as a valid character on file system.
This breaks not only osc but also basic commands such
as 'git clone' on a project that contains colons in paths.
That's why we decided to make osc unsupported on Windows.
Do not fail with a traceback if the config file parsing fails.
Hence, catch the configparser.Error exception and print its
"message" attribute to the user.
Fixes: #985 ("Type error on any osc call")
Since a recent backend change, a bdep has a hdrmd5 by default. That
is, osc always downloads these bdeps from the API (unless they are
cached) instead of a mirror. This is not intended.
Using a mirror is no problem because the hdrmd5s are verified in
the build module.
Note: If this causes a problem, one could also use "osc build
--download-api-only" to mimic the old behavior.
If the oscrc is a symlink, follow the symlink when writing the
configuration file. The old code replaced the symlink with a
regular file (see #390 ("symlinked $HOME/.oscrc gets replaced
with a ordinary file")).
Implementation note: if the directory, which contains the resolved
config file, has a <config file>.new file, the file is overwritten.
When a user creates a symlink pointing from ~/.config/osc/oscrc
to a different location, don't overwrite the symlink but follow
it when writing configuration on disk.
Without escaping the % character, the download URL could be subject
to string formatting (depending on the subsequent characters). For
instance, if the url attribute's value of a buildinfo's path element
contains the substring "c_c%2B%2B", the "%2B" is interpreted as a
format string (see issue #965), which is wrong ("B" is not a valid
format character at all). In order to avoid this, escape all
% characters in the download urls.
Note: escaping the % characters in the download url itself is OK
because we only intend to "format" the path.
Note: we do not escape the % characters for urls from the config
file (implicit assumption: the user already correctly escaped the
urls (whether this assumption is sensible or not is debatable, of
course)).
Fixes: #965 ("unsupported format character 'B' (0x42) at index 66")
The old code passes try_again=False to the recursive postoptparse
call when calling it from one of the exception handlers. This is
wrong because it can result in an incomplete conf.config dict (for
instance, if two apiurl sections have no user and no password and
no credentials_mgr_class option - see #761 ("Traceback config with
two backends and no username")).
Hence, Osc.postoptparse should only return if the conf.get_config
call succeeds. For this, unconditionally call Osc.postoptparse from
within the exception handlers. Note: this could potentially (although
quite unlikely) result in an endless recursion but in each recursive
call "user" interaction is required (that is, the user could simply
press CTRL+c) - so this should not be a problem.
Implementation note: this change breaks the API. Rationale: the
semantics of Osc.postoptparse changed. Hence, "pretending" to
honor the try_again parameter could result in unexpected behavior
(from the API consumer's POV). Hence, a traceback might be more
sensible.
Fixes: #761 ("Traceback config with two backends and no username")
Support a zst compressed control.tar in debquery.DebQuery. A zst compressed
control tar is used, for instance, in Ubuntu 21.10.
Note: this requires the 3rd-party python-zstandard module.
Current OBS is delivering hdrmd5 in buildinfo. It turns out
that osc has already code for validating cached files, but it
invalidates all local files atm with python 3.x
Current OBS is delivering hdrmd5 in buildinfo. It turns out
that osc has already code for validating cached files, but it
invalidates all local files atm with python 3.x
Do not create an MR for the entire project if "osc mr" is invoked in
a package wc (only create an MR for the specific package instead).
Strictly speaking, the "breaks" the existing UI - but this rather seems
to be a "fix" than a "break";)
Some of our repositories have specific download urls. osc is ignoring
this so far and just tries to use the generic downloadurl
This code prefers definitions for individual path elements if they exist.
We could IMHO remove the old code, since old OBS instances would still
work via the api download fallback.
Real life examples for repo specific configs are on openSUSE all
repositories outside of the /repositories/ directory. eg.
<path project="openSUSE:Tumbleweed" repository="dod" url="http://download.opensuse.org/tumbleweed/repo/oss/"/>
Co-Author: Marcus Hüwe <suse-tux@gmx.de>
The old code does not print any information about the host, for
which the access failed, in case of an URLError. In order to fix
this, add information about the host (and port) to the URLError
instance in core.http_request and use this information in the
babysitter to print out a more detailed error message (which includes
the host (and port)).
For now, we simply add a "private" "_osc_host_port" attribute to
the URLError instance (this way we avoid potential name clashes (due
to the "_osc" prefix) and could come up with a different/more clever
way in the future (due to its privateness)).
Fixes: #954 ("Better diagnostic for domain name issues")
os.sysconf is not available on all platforms (like Windows) but it
is used to retrieve the number of online processors. If missing,
assume one processor (building on such a platform will most likely
not work, though).
Fixes: #948 ("Windows compatibility") (at least it improves the
Windows support a bit)
A workflow token can be created via "osc token --create --operation
workflow --scm-token <SCM_TOKEN>".
Triggering a workflow token via osc is probably unlikely - that's
why it is not yet implemented (it would also make the UI a bit
awkward because one has to specify a concrete http header).
Fixes: #943 ("implement osc token --operation=workflow")
The use of makeurl makes the code more readable/maintainable (IMHO)
and it also does proper percentage encoding of the query string (not
that the osc codebase cares much about it, though:/).
Newer rexml Ruby gem used on OBS server side uses stricter XPath parsing.
This change fixes incorrect XPath that was accepted by older rexml,
but not accepted by newer one.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
