If no "dest" argument is specified when calling CpioRead.copyin or
CpioRead.copyin_file, a TypeError occurs in CpioRead._copyin_file
because os.getcwd(), which returns a str, is used as dest and, hence,
the subsequent os.path.join(...) fails (because it tries to join a
str and a bytes).
In order to avoid this, encode the result of os.getcwd().
Note that the existing
archive.copyin_file(hdr.filename,
os.path.dirname(tmpfile),
os.path.basename(tmpfile))
was OK because CpioRead._copyin_file os.path.join()s "dest" and
"new_fn", which are both str. It is just changed to stress that
CpioRead is a bytes-only API.
Fixes: #865 ("Traceback in osc/util/cpio.py line 128: TypeError:
Can't mix strings and bytes in path components")
Currently, when trying to initialize a non existent (server-side)
project via "osc init <prj>", osc errors out (after creating the wc)
because it fails to retrieve the package list. However, there is no
need to retrieve the package list in the "osc init <prj>" case. Hence,
skip the package list retrieval. As a result, osc does not error out.
For the background, see the discussion in #858 ("osc fails to check
out an empty project as project") [1].
[1] https://github.com/openSUSE/osc/issues/858#issuecomment-722330024
If meta=True is passed to checkRevision, the meta parameter is used
as a revision in the show_upstream_rev call. Instead, it should be
bound to show_upstream_rev's meta parameter.
Currently, if the --offline option is passed to "osc build ...", a
preinstallimage is not used (even if it exists). Instead, a
preinstallimage should be used (if it exists) even if the --offline
option is specified.
This is faster in best case since the binary search does not need
to be executed on the server.
It also finds package names where no binary with that name exists.
(as for some multibuild cases)
Replace usage with better explanation. It was missing that it requires a
prefixed hostarch. Also workername is instead called workerid in the
API.
Usage help was before: osc workerinfo WORKER
Add actual example.
See also the fix for this in OBS API docs:
https://github.com/openSUSE/open-build-service/pull/10024
In the API a new request action release was implemented. This changes
enables the user to create a release request for non-maintenance projects
and to review / view the release requests
Without this patch, running an individual service that has parameters
defined in the _service file fails:
$ osc service run obs_scm
Please specify valid --scm=... options
Aborting: service call failed: /usr/lib/obs/service/obs_scm --outdir [snipped]
This is because although the service is defined in the _service file and
the "scm" parameter is set in it, the service wasn't being found in the
data structure and so the service executable wasn't being called with
the parameters supplied in the _service file. This patch corrects the
issue with the services data structure so that the service data isn't
overridden if it is defined in the _service file.
A side effect of this correction for services defined in the _service
file is that instead of overriding the service mode with '', the mode is
taken from the _service file. When using the "run" command, this would
mean that the call mode of None may not be in agreement with the service
mode defined in the file, e.g. "manual", and so the "run" command would
no longer cause it to run when it would before. We can take this
opportunity to define this as the correct behavior - the "run" command
now only runs services with "trylocal", "localonly", or no mode set -
and also ensure that other call mode commands result in sensible
behavior when called with a service name, for instance "osc service
manualrun download_files" will run only services with mode="manual" and
name="download"files" instead of all services with mode="manual".
Additionally, services that aren't defined in the _service file can be
called with a call mode command and will use that call mode rather than
None.
Add a "manual" service mode. A service with mode "manual" is not executed
by default (that is, via "osc service run"). As of now, "manual" is in
some sense just an alias for "disabled". However, this might change in the
future [1]. Also, "localrun" now executes services with mode "serveronly".
Moreover, the documentation of "disabledrun" is updated ("disabledrun"
never executed services with mode "serveronly"). Additionally, "localrun"
and "disabledrun" are marked as "[n]ot for common usage anymore" in the
service command's description.
The rationale for these changes is (partly) described in [1]. The main
motivation is to add some clarity (in contrast to the "disabled" mode,
it is probably easier to get/guess the semantics of the "manual" mode).
[1] https://github.com/openSUSE/osc/pull/826
The "disabledrun" service commands is marked as deprecated but has no
explicit replacement. It is still a useful command for updating packages
manually or through a CI system without being forced to run all defined
services with the "runall" command. This change adds a new command
"manualrun" and a new mode "manual" which behave the same as the
deprecated "disabledrun" command and "disabled" mode but have clearer
meaning. "manualrun" does not attempt backwards-compatible behavior with
the "disabledrun" mode for "disabled" services because "disabled" mode
may eventually be removed or change meaning. The "localrun" command is
enhanced to consider the "serveronly" mode. Since "disabledrun" never
executed services with mode "serveronly", its docs are updated
accordingly.
Improve error message in do_service a bit. The old "Too few arguments."
was misleading (for instance, if a non-remote command was not executed
in a package wc).
Note: with the new logic we could also get rid of the
"raise oscerr.WrongArgs('Local directory is no package')" statement.
Add a --status-filter option to "osc results" that can be used to
show, for instance, only the repos where a package failed to build. As
a short circuit, a -f/--failed option is added, too.
Add a --brief option to "osc prjresults" and "osc results" that can be
used to get a more compact representation of the results. In case of
"osc results", --brief is ignored if the results for a package are
requested.
in do_results:
* add --brief option on prj level:
[packagename] [repo] [arch] [buildstatus]
* filter by --status-filter <long status name>
works on prj and pkg level
in do_prjresults:
* --brief
* assume len(state)>1 as long state
core.py
* filter packages by build status
* long status handling in get_prj_results
* brief output generation in get_prj_results
Interprete unicode escape characters in a "--message" option. In some
cases, this breaks the existing UI (but that's OK because it can be
fixed by properly escaping the escape character(s)).
Note: if we are going to do more advanced stuff in the future, we should
move the logic into a separate function.
In commit 276d6e2439 ("Do not use the
chardet module in util.helper.decode_it") util.helper.decode_it was
changed to always decode the passed object if it has a decode method.
Since a python2 str has a decode method, the new code tries to utf-8
decode the passed str. As a result, a unicode object is returned (if
the decoding worked). Since a unicode object is not an instance of
type str, all subsequent isinstance(decoded_obj, str) checks evaluate
to False, which break some codepaths.
In order to fix this, restore the old python2 behavior (that is, if
the passed object is a str, it is not decode it). This change does not
affect the python3 codepaths.
Fixes: #814 ("osc log | fails")
Improve "osc rdiff --issues-only ..." output: now, it shows the added,
deleted and changed issues. Also, add a new "osc rdiff --xml ..." option,
which only works in combination with the "--issues-only" option: it prints
the raw xml.
Note: server_diff_noex has no option for the "full" parameter. Hence,
with the addition of the "xml=False" parameter, the signatures of the
server_diff_noex and server_diff functions are going to differ "forever".
That's OK (IMHO) because it is probably more sane to simply specify the
additional args via the kwargs syntax.
Remove dead code from the fetch module. Actually, it should have
been removed in commit 95ec7dee7b
('- fixed#590606 ("osc/fetch.py does not support authenticated URLs")').
In general, decode_it is used to get a str from an arbitrary bytes
instance. For this, decode_it used the chardet module (if present)
to detect the underlying encoding (if the bytes instance corresponds
to a "supported" encoding). The drawback of this detection is that
it can take quite some time in case of a large bytes instance, which
represents no "supported" encoding (see #669 and #746).
Instead of doing a potentially "time consuming" detection, either
assume an utf-8 encoding or a latin-1 encoding. Rationale: it is just
not worth the effort to detect a _potential_ encoding because we have
no clue what the _correct_ encoding is. For instance, consider the
following bytes instance:
b'This character group is not supported: [abc\xc3\xbf]'
It represents a valid utf-8 and latin-1 encoding. What is the "correct"
one? We don't know... Even if you interpret the bytes instance as a
human you cannot give a definite answer (implicit assumption: there is
no additional context available).
That is, if we cannot give a definite answer in case of two potential
encodings, there is no point in bringing even more potential encodings
into play. Hence, do not use the chardet module.
Note: the rationale for trying utf-8 first is that utf-8 is pretty
much in vogue these days and, hence, the chances are "high" that we
guess the "correct" encoding.
Fixes: #669 ("check in huge shell archives is insanely slow")
Fixes: #746 ("Very slow local buildlog parsing")
Importing `cElementTree` has been deprecated since Python 3.3 -
importing `ElementTree` automatically uses the fastest
implementation available - and is finally removed in Python 3.9.
Importing cElementTree directly (not as part of xml) is an even
older relic, it's for Ye Time Before ElementTree Was Added To
Python and it was instead an external module...which was before
Python 2.5.
We still need to work with Python 2.7 for now, so we use a try/
except to handle both 2.7 and 3.9 cases. Also, let's not repeat
this import 12 times in one file for some reason.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This checks if the filename of a downloaded file has
been modified (for example by a MITM attack) to contain
slashes. This could mean that the file is compromised
and that the attacker tries to overwrite system files.
