1
0
mirror of https://github.com/openSUSE/osc.git synced 2024-09-22 02:06:18 +02:00
Commit Graph

3735 Commits

Author SHA1 Message Date
Marcus Huewe
e807abbf7c Enable python3.8 testing in travis
Based on PR#750 [1].

[1] https://github.com/openSUSE/osc/pull/750
2020-06-04 12:05:07 +02:00
Marcus Huewe
3d21f2498b Merge branch 'tests_python3_fixes' of https://github.com/marcus-h/osc
Make the testsuite work with python3.8
2020-06-04 12:00:14 +02:00
Marcus Huewe
613daa7a09 Merge branch 'no-celementtree' of https://github.com/AdamWill/osc
Use xml.etree.ElementTree if xml.etree.cElementTree is not available
anymore (python3.9).
2020-06-03 22:01:04 +02:00
Marcus Huewe
443e3ef90b tests: Ignore the ordering of attributes in XML documents
Old xml.etree.cElementTree versions (python2) reorder the attributes
while recent xml.etree.cElementTree versions (python3) keep the
document order.

Example:

python3:
>>> ET.tostring(ET.fromstring('<foo y="foo" x="bar"/>'))
b'<foo y="foo" x="bar" />'
>>>

python2:
>>> ET.tostring(ET.fromstring('<foo y="foo" x="bar"/>'))
'<foo x="bar" y="foo" />'
>>>

So far, the testsuite compared two serialized XML documents via a simple
string comparison. For instance via,
self.assertEqual(actual_serialized_xml, expected_serialized_xml) where
the expected_serialized_xml is, for instance, a hardcoded str. Obviously,
this would only work for python2 or python3.
In order to support both python versions, we first parse both XML
documents and then compare the corresponding trees (this is OK because
we do not care about comments etc.).

A related issue is the way how the testsuite compares data that is "send"
to the API. So far, this was a plain bytes comparison. Again, this won't
work in case of XML documents (see above). Moreover, we have currently
no notion to "indicate" that the transmitted data is an XML document.
As a workaround, we keep the plain bytes comparison and in case it fails,
we try an xml comparison (see above) as a last resort. Strictly speaking,
this is "wrong" (there might be cases (in the future) where we want to
ensure that the transmitted XML data is bit identical to a fixture file)
but a reasonable comprise for now.

Fixes: #751 ("[python3.8] Testsuite fails")
2020-06-03 21:50:50 +02:00
Marcus Huewe
27ef627f1d Get rid of bytes usage in tests.common
Simply use the .encode() method (encode using the utf-8 encoding).
2020-06-03 21:50:50 +02:00
Marcus Huewe
c0343207ea Make tests.common.MyHTTPHandler.__mock_PUT bytes aware
For now, we assume that if the "exp" keyword argument is specified,
then it is a str. In this case, we simply encode it (using the utf-8
encoding).
Also, simplify the code a bit (get rid of the if-statement that is
always executed).
2020-06-03 21:50:10 +02:00
Adam Williamson
13a13a87c4 Fix ElementTree imports for Python 3.9
Importing `cElementTree` has been deprecated since Python 3.3 -
importing `ElementTree` automatically uses the fastest
implementation available - and is finally removed in Python 3.9.
Importing cElementTree directly (not as part of xml) is an even
older relic, it's for Ye Time Before ElementTree Was Added To
Python and it was instead an external module...which was before
Python 2.5.

We still need to work with Python 2.7 for now, so we use a try/
except to handle both 2.7 and 3.9 cases. Also, let's not repeat
this import 12 times in one file for some reason.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-06-02 15:13:10 -07:00
lethliel
e99b2942ef increase version number in core.py 2020-05-29 20:35:00 +02:00
lethliel
0e70579113 release 0.169.1 2020-05-29 20:28:42 +02:00
Marcus Huewe
1d26cb3079 Merge branch 'ccache' of https://github.com/sjamgade/osc
Add "--ccache" option to "osc getbinaries".
2020-05-27 16:09:06 +02:00
lethliel
22ac336f8b open 0.170.0 development 2020-05-27 14:50:37 +02:00
lethliel
42b4992ecb release 0.169.0 2020-05-27 14:49:10 +02:00
Marco Strigl
fdcd606122
Merge pull request #797 from lethliel/fix_1122675
fix security issue (bsc#1122675) no / in filename
2020-05-27 11:23:49 +02:00
lethliel
a79c54418b fix security issue (bsc#1122675) no / in filename
This checks if the filename of a downloaded file has
been modified (for example by a MITM attack) to contain
slashes. This could mean that the file is compromised
and that the attacker tries to overwrite system files.
2020-05-27 11:17:40 +02:00
Sumit Jamgade
5557a06e5b add and implement ccache option for getbinaries
If this option is used, the api is asked to list ccache archive if
present, osc will then download the archive.
2020-05-25 09:03:33 +02:00
Marco Strigl
3967133022
Merge pull request #793 from sjamgade/ccache_path
add option pkg-ccache to be passed to build-cmd
2020-05-22 15:55:48 +02:00
Marco Strigl
3e10473e1a
Merge pull request #794 from Vogtinator/kiwifix
Fix kiwi build with --prefer-pkgs
2020-05-20 13:40:16 +02:00
Fabian Vogt
f3d07eab3f Fix kiwi build with --prefer-pkgs
The buildinfo contains packages from the local dir, so it needs the list of
local packages to work correctly.
2020-05-20 11:34:27 +02:00
Sumit Jamgade
5dd853e302 add option pkg-ccache to be passed to build-cmd
this option is a /path/to/_ccache.tar. The patch just forwards that
option to the build script.
2020-05-19 16:15:49 +02:00
Marcus Huewe
6b5b0fd55d Merge branch 'fix_help_depends_on' of https://github.com/lethliel/osc
Split dependson and whatdependson into two commands. This way, both
commands are part of "osc --help".
2020-05-19 12:56:55 +02:00
lethliel
798e3e4f63 split dependson/whatdependon in two commands
split the code of do_dependson into two separate commands (just for
the osc help overview)

They are doing the opposite of each other.

Duplicate code was moved to _dependson()

do_whatdependson and do_dependson just call _dependson with an option
reverse set to None or 1.
2020-05-19 12:50:15 +02:00
Marcus Huewe
9b2cbfead0 Merge branch 'fix_osc_api_without_arguments' of https://github.com/lethliel/osc
Fix handling of incorrect CLI arguments (python3).
2020-05-18 20:05:25 +02:00
lethliel
69b1233316 add regex for python3 missing arguments err
add new regex and check for missing arguments.
The error message in python3 differs from the one in python2.

python3:
do_api() missing 1 required positional argument: 'url'

python2:
do_api() takes exactly 4 arguments (3 given)

To be compatible with python2 two checks are needed.
2020-05-18 19:46:22 +02:00
Marco Strigl
2b26419e6d
Merge pull request #791 from adrianschroeter/shell_cmd
complete --shell-cmd support
2020-05-12 13:19:14 +02:00
12e0b67117 complete --shell-cmd support parts to get it working 2020-05-12 13:16:29 +02:00
Marcus Huewe
1d056c01ea Merge branch 'fix_git_detectoin' of https://github.com/adrianschroeter/osc
Fix heuristic to detect a git url. It does not have to necessarily end
with ".git" and can start with git:// or git@.
2020-05-05 12:22:34 +02:00
5d6351b1bd Support git@ (private github) or git:// URLs correctly 2020-05-05 10:54:43 +02:00
Marco Strigl
9aa3777a60
Merge pull request #788 from lethliel/documentation
improve documenation
2020-04-30 09:00:36 +02:00
lethliel
26b2fea6d2 improve documenation
removed tutorial. Added missing osc.util.helper

added more modules:

* OscConfigParser
* build
* conf
* credentials
2020-04-29 20:13:48 +02:00
Marcus Huewe
582ac00483 Merge branch 'quote_arch_on_bl' of https://github.com/lethliel/osc
Callers of core.print_buildlog should properly quote the project, package,
repo, and arch parameters. Actually, doing this at the caller's level
is pretty insane but this way, we do not break the existing API (of
core.print_buildlog).
In the future, we should move all the quoting logic into core.makeurl
(even if this breaks the API).
2020-04-28 15:59:29 +02:00
lethliel
9a3fda8471 quote prj, repository, arch and package
quote the arguments before creating the apiurl.
This prevents weird behaviour when any has an '#'
at the end.
2020-04-28 14:47:40 +02:00
Marco Strigl
e98728d394
Merge pull request #657 from lethliel/better_errmsg_oscci_editor_not_present
Better errmsg oscci editor not presentosc ci improve error message on fail
2020-04-20 21:02:29 +02:00
Marcus Huewe
c156815d5e Merge branch '776-ccache' of https://github.com/Firstyear/osc
Add ccache option to the oscrc (default: disabled). If enabled, the
--ccache option is always passed to the build script (if invoked via
osc.build.main).
2020-04-15 13:29:33 +02:00
William Brown
bdb0c8033c Add ccache argument for oscrc 2020-04-14 14:50:24 +10:00
Marcus Huewe
01d235d998 Merge branch 'travis' of https://github.com/crazyscientist/osc
Improve deployment via travis. Unfortunately, style + semantics changes
are in a single commit... but let's not be too picky. See also the
discussion in  [1].

[1] https://github.com/openSUSE/osc/pull/739
2020-04-10 20:04:17 +02:00
Andreas Hasenkopf
f817db1053 Automatic package publication via Travis CI
* `.travis.yml`
  - Reformatted for easier reading
  - Used `before_*` statements instead of script chains
  - Publish only source packages
* `setup.py`
  - Reformatted for easier reading
  - Use README contents for `long_description` to have a nice description on PyPI
  - Added classifiers
  - Added explicit package dependencies
* fixes #658
* fixes #708
2020-04-06 09:32:01 +02:00
Marcus Huewe
07d422063b Merge branch 'fix_HTTPError_reason' of https://github.com/lethliel/osc
Fix str formatting in build.get_preinstall_image.
2020-04-01 13:29:25 +02:00
lethliel
81acc39c9f e.reason is a string not a integer decimal 2020-04-01 12:48:12 +02:00
Marcus Huewe
894f1c281a Merge branch 'packagequery_bytes_fixes' of https://github.com/marcus-h/osc
bytes fixes/cleanups in the packagequery and repodata modules.
2020-03-25 19:39:11 +01:00
lethliel
b99c4486e8 update news and open new development milestone 2020-03-23 13:50:01 +01:00
Marco Strigl
81d221242b
Merge pull request #767 from Vogtinator/completion
Completion usability improvements
2020-03-23 13:36:31 +01:00
Marco Strigl
93a5a4d353
Merge pull request #736 from lethliel/improve_bash_completion
improve speed of bash completion
2020-03-23 13:36:14 +01:00
Marcus Huewe
55aef1a014 Convert repodata.RepoDataQueryResult to a bytes API
The repodata.RepoDataQueryResult is supposed to be a bytes API and
that's what our users (see build module) expect.
Note that the repodata.RepoDataQueryResult.path method still returns
a str. That's what the rpmquery.RpmQuery, debquery.DebQuery, and
archquery.ArchQuery classes also do (if the "path" was initially
passed as a str).

Fixes: #760 ("osc build fails when called with --prefer-pkgs where the
       passed directory is a repodata repository or a subdirectory of one")
2020-03-15 18:30:28 +01:00
Marcus Huewe
cd51f47a77 Return bytes in packagequery.PackageQueryResult.evr() instead of a str
The packagequery.PackageQueryResult class is supposed to provide a
bytes API. Hence, packagequery.PackageQueryResult.evr() should return
bytes instead of a str. Also, adjust the single caller in the build
module.
2020-03-15 18:30:00 +01:00
Fabian Vogt
ee44164bf1 Discard stderr of the completion script
Otherwise it pollutes the terminal with warnings and errors such as
"interrupted!", which makes the terminal line unusable.
2020-03-13 15:36:08 +01:00
Fabian Vogt
e11b49e2f1 Fetch list of all projects lazily
Currently osc <TAB><TAB> takes a long time to complete, which is incredibly
annoying. Only run "osc ls" if it is necessary for proper completion.
2020-03-13 15:32:29 +01:00
lethliel
3914b8c5e8 release 0.168.2 2020-03-13 07:44:34 +01:00
Marcus Huewe
33bbc57b5f Fix the previously introduced escaping via the html module
This is a follow-up commit for commit
6dbf103e10 ("Use html.escape instead
removed cgi.escape"), which breaks the python2 backward compatibility
(since the "html" module is not available by default) and also breaks
the code in general (due to missing html imports).

The fix is based on the proposed fix in [1].

Fixes: boo#1166537 ("osc rq accept - forwarding request causes backtrace")

[1] https://github.com/openSUSE/osc/pull/764
2020-03-12 23:00:47 +01:00
9b01f59fee
Merge pull request #763 from mimi1vx/patch-3
Use html.escape instead removed cgi.escape
2020-03-11 17:10:42 +01:00
6dbf103e10
Use html.escape instead removed cgi.escape
Fixes:

`Traceback (most recent call last):
  File "/usr/bin/osc", line 41, in <module>
    r = babysitter.run(osccli)
  File "/usr/lib/python3.8/site-packages/osc/babysitter.py", line 64, in run
    return prg.main(argv)
  File "/usr/lib/python3.8/site-packages/osc/cmdln.py", line 344, in main
    return self.cmd(args)
  File "/usr/lib/python3.8/site-packages/osc/cmdln.py", line 367, in cmd
    retval = self.onecmd(argv)
  File "/usr/lib/python3.8/site-packages/osc/cmdln.py", line 501, in onecmd
    return self._dispatch_cmd(handler, argv)
  File "/usr/lib/python3.8/site-packages/osc/cmdln.py", line 1232, in _dispatch_cmd
    return handler(argv[0], opts, *args)
  File "/usr/lib/python3.8/site-packages/osc/commandline.py", line 1458, in do_submitrequest
    result = create_submit_request(apiurl,
  File "/usr/lib/python3.8/site-packages/osc/core.py", line 4244, in create_submit_request
    cgi.escape(message))
AttributeError: module 'cgi' has no attribute 'escape'
`
`cgi.escape` was deprecated in python 3.2
2020-03-11 16:58:04 +01:00