The ':' character is used as a separator in Open Build Service
and constantly appears in directory names after running osc commands.
Windows do not support ':' as a valid character on file system.
This breaks not only osc but also basic commands such
as 'git clone' on a project that contains colons in paths.
That's why we decided to make osc unsupported on Windows.
The old code does not print any information about the host, for
which the access failed, in case of an URLError. In order to fix
this, add information about the host (and port) to the URLError
instance in core.http_request and use this information in the
babysitter to print out a more detailed error message (which includes
the host (and port)).
For now, we simply add a "private" "_osc_host_port" attribute to
the URLError instance (this way we avoid potential name clashes (due
to the "_osc" prefix) and could come up with a different/more clever
way in the future (due to its privateness)).
Fixes: #954 ("Better diagnostic for domain name issues")
Offer a force ("f") choice if, for instance, "osc meta prj foobar -e"
fails due to a HTTPError in metafile.edit. If the force choice is
selected, a new url is constructed by invoking the metafile._URLFactory
instance with a "force='1'" argument (this adds a "force=1" to the
original url's query string (*)) and the corresponding file is PUTed
to the new url. If this PUT fails again and now the "y" choice is
selected, the file is PUTed to the original url (*).
(*): Stricly speaking, from metafile.edit's POV, the concrete url
depends on the passed in metafile._URLFactory instance, though.
Note: the metafile._URLFactory class and its is_force_supported method
is a gross hack. That's why this class is marked as private (that is,
we can remove it at any point in time again without breaking the
API/3rd party applications). An alternative to the metafile._URLFactory
approach would be manual URL parsing and manual URL construction
(adding "force=1" to the query string)... but this is also pretty
awkward (if done properly).
Fixes: #916 ("for osc meta edit change y/n to y/n/f")
Fixes: #942 ("Offer -f when prjmeta change leads to repo_dependency")
The old code uses a variable .rXYZ suffix (where XYZ is the revision
of the package wc during the merge operation). Now, if Package.mergefile
is invoked during an update, XYZ represents the "old" revision. That
is, if a merge conflict happens, then a subsequent "osc resolved <file>"
will not unlink the <file>.rXYZ file (because
Package.clear_from_conflictlist only takes the current rev into account).
In order to fix this, use a fixed ".new" suffix. This way,
Package.clear_from_conflictlist can properly unlink the corresponding
*.new file. This naming scheme for the "upfilename" is in line with
"osc pull" and "osc repairlink".
Note: if a working copy was updated with an "old" osc version (without
this commit) and a "new" osc version (with this commit) is used to run
"osc resolved <file>", then the <file>.rXYZ file is _NOT_ removed (it
is not worth the effort to add compat code for this).
Always send the sha256sums of all tracked files in case of a
frozen package wc. For instance, this is needed if the package is
a plain link (no branch) because in this case the backend might
request a sha256sum for a tracked but unmodified file (this can
happen because the backend cannot expand the link).
The new behavior is in line with a pulled/linkrepair package wc.
Fixes: #924 ("Transmitting file data There is no sha256 sum for
file")
When trying to commit a non-existent package via Project.commit it
is treated as an external package (because a non-existent package
has no "state" inside the project). That is, Project.commitExtPackage
is called, which fails with a FileNotFoundError in case of a
non-existent package (and the traceback is printed to the user).
In order to fix this, treat a non-existent package as broken package.
That is, simply print an info message and do not error out with a
traceback (note: the commit is _not_ aborted).
Fixes: #920 ("osc commit should fail gracefully in case of
nonexistent filename")
The old code only supports a file whose size is less then or equal
to INT_MAX (due to a reasonable(!) limit in M2Crypto). The actual
issue is in core.http_request which mmap(...)s the file, wraps it
into a memoryview/buffer and then passes the memoryview/buffer to
urlopen. Eventually, the whole memoryview/buffer is read into memory
(see m2_PyObject_GetBufferInt). If the file is too large (> INT_MAX),
m2_PyObject_GetBufferInt raises a ValueError (which is perfectly
fine!).
Reading a whole file into memory is completely insane. In order to
avoid this, we now simply pass a file-like object to urlopen (more
precisely, the file-like object is associated with the Request
instance that is passed to urlopen). The advantange is that the
file-like object is processed in chunks of 8192 bytes (see
http.client.HTTPConnection) (that is, only 8192 bytes are read into
memory (instead of the whole file)).
There are two pitfalls when passing a file-like object to urlopen:
* By default, a chunked Transfer-Encoding is applied. It seems that
some servers (like api.o.o) do not like this (PUTing a file with
a chunked Transfer-Encoding to api.o.o results in status 400). In
order to avoid a chunked Transfer-Encoding, we explicitly set a
Content-Length header (we also do this in the non-file case (just
for the sake of completeness)).
* If the request fails with status 401, it is retried with an
appropriate Authorization header. When retrying the request, the
file's offset has to be repositioned to the beginning of the file
(otherwise, a 0-length body is sent which most likely does not
match the Content-Length header).
Note: core.http_request's "data" and "file" parameters are now mutually
exclusive because specifying both makes no sense (only one of them
is considered) and it simplifies the implementation a bit.
Fixes: #202 ("osc user authentification seems to be broken with last
commit")
Fixes: #304 ("osc ci - cannot handle more than 2 GB file uploads")
Element.getchildren is deprecated and not available on python39
anymore. Instead, iterate over the element itself (which iterates
over the element's children).
Fixes: #903 ("AttributeError: 'xml.etree.ElementTree.Element' object
has no attribute 'getchildren'")
When creating a new request via the core.Request.create method, there is
no need to escape the data that is assigned to the "description" attribute
of a core.Request instance. Internally, core.Request.create ensures that
the data, which is POSTed to the api, is correctly escaped (the escaping
is implicitly done by ET (see core.Request.to_str)). Manually escaping the
description results in a double escaping (the escaped description is
escaped by ET again) - this is not the desired behavior.
Analogously, there is no need to escape the data that is passed to the
message parameter of the core.create_submit_request function because
core.create_submit_request takes care of escaping it.
Fixes: #869 ("Silly encoding of htmlencodable entities")
So far, core.get_prj_results only supports a substring based name
filtering mechanism. Now, a regex based name filtering mechanism
is used. That is, if the regex matches a package name, the package
is not filtered out.
This is an API incompatible change:
- 3rd party code which looks like this is going to break:
class Foo(str):
...
def __eq__(self, other):
return ...
... = core.get_prj_results(..., name_filter=Foo())
(My gut feeling says there are no such callers. However, if this
really breaks any serious code, we can fix it in a follow-up commit)
- "osc prjresults openSUSE:Leap:15.2:Update --name-filter zypper." will
now also show "zypper-docker" etc. because the dot (".") matches any
character (except a newline). Previously, only packages that contained
the str "zypper" followed by a dot (".") were shown. The old behavior
can be restored, if the dot is escaped: "osc prjresults
openSUSE:Leap:15.2:Update --name-filter 'zypper\.'".
Of course, this affects all other special characters, too.
Additionally, if an illegal regex is passed to the --name-filter option,
an exception is raised. The previous code did not fail.
This can break existing workflows and scripts. We could avoid this by
introducing a --name-filter-regex option but this would clutter the
UI (IMHO).
A regex based name filtering feature was requested by darix.
If meta=True is passed to checkRevision, the meta parameter is used
as a revision in the show_upstream_rev call. Instead, it should be
bound to show_upstream_rev's meta parameter.
Some services expect "old" service files (that is, files from a
previous service run) to be present in an ".old" dir. Hence, osc
should support that.
Instead of removing all files from a previous service run, move them
to the ".old" dir, run the services, and, finally, remove the ".old"
dir.
Unfortunately, the location of the ".old" dir is hardcoded in the
specific services. That is, we have to be careful if an ".old" dir
exists (in this case, we error out).
Based on [1].
[1] https://github.com/openSUSE/osc/pull/846
This is faster in best case since the binary search does not need
to be executed on the server.
It also finds package names where no binary with that name exists.
(as for some multibuild cases)
