1
0
mirror of https://github.com/openSUSE/osc.git synced 2024-12-31 20:26:13 +01:00
Commit Graph

2817 Commits

Author SHA1 Message Date
4cc87b2c57 fix DISTURL checkout for package containers using a multibuild flavor 2018-06-27 10:25:45 +02:00
Sean Kelly
b42e6f5e04 Support larger uids and gids 2018-06-25 16:55:47 -04:00
Marcus Huewe
b730f880cf Disable ssl session resumption
The old code could potentially yield to a use-after-free situation,
which results in UB. For this, consider the following scenario, where
osc performs several HTTPS requests (assumption: the server supports
ssl session resumption):

- HTTPS Request 1:
  * a new SSL *s connection is established, which also creates a new
    SSL_SESSION *ss => ss->references == 1
  * once the handshake is done, the ss is put into the session cache
    (see ssl_update_cache) => ss->references == 2
  - osc saves the session ss in a class variable
  - s is SSL_free()d, which calls SSL_SESSION_free => ss->references == 1

- HTTPS Request 2:
  * setup a new SSL *s connection that reuses the saved session ss
    => ss->references == 2
  * once the handshake is done, ssl_update_cache is called, which is a
    NOP, because s->hit == 1 (that is, the session was resumed)
  * osc saves the session ss in a class variable
  * s is SSL_free()d, which calls SSL_SESSION_free => ss->references == 1

...

> 2 hours later (see tls1_default_timeout)

...

- HTTPS Request 256:
  * setup a new SSL *s connection that reuses the saved session ss
    => ss->references == 2
  * once the handshake is done, ssl_update_cache is called, but is
    _no_ NOP anymore
  * ssl_update_cache flushes the session cache (this is done every
    255/256 (depending on the way we count) connections) => ss is
    SSL_SESSION_free()d => ss->references == 1
  * osc saves the session ss in a class variable
  * s is SSL_free()d, which calls SSL_SESSION_free:
    since ss->references == 1, ss is eventually free()d

- HTTPS Request 257:
  * setup a new SSL *s connection that reuses the saved session ss

Since ss does not exist anymore, the remaining program execution is UB.

(Note: SSL_free(...) is _NOT_ called, if M2Crypto 0.29 is used.
M2Crypto 0.30 calls SSL_free(...) again.)

Due to a bug in OpenSSL_1_1_0h (see openssl commit 8e405776858) the
scenario from above can be triggered with exactly 2 HTTPS requests (the
SSL_SESSION is not cached, because we configured SSL_VERIFY_PEER, but
no sid_ctx was set). This is fixed in openssl commit c4fa1f7fc01.

In order to reliably reuse a session, we probably need to listen to the
session cache changes. Such callbacks could be registered via
SSL_CTX_sess_set_new_cb and/or SSL_CTX_sess_set_remove_cb, but both
functions are not provided by M2Crypto. Another idea is to directly utilize
the session cache, but this also has to be implemented in M2Crypto first.
Yet another approach is to retrieve the session via SSL_get1_session, which
increases the session's refcnt, but this also needs to be implemented in
M2Crypto first (if we choose to use this approach, we also have to make
sure that we eventually free the session manually...).

Fixes: #398 ("SIGSEGV on \"osc commit\"")
2018-05-08 14:32:33 +02:00
jonsger
6bd8572cf7
Fix typo in "osc revert" help 2018-05-03 16:16:38 +02:00
Marcus Huewe
d6a04e5327 Merge branch 'master' of https://github.com/adrianschroeter/osc
Always pass the "--statistics" option to the build script (only affects
a vm build). It is not implemented as an option, because we already have
so many of them... (see also https://github.com/openSUSE/osc/pull/412).
2018-05-03 11:02:57 +02:00
Marcus Huewe
88274c92ed Merge branch 'fix_newline_in_verbose' of https://github.com/lethliel/osc
Print each unresolvable on a separate line.
2018-05-03 09:57:02 +02:00
0c3edd3fa0 always run build using --statistics. I could have made this
yet another option, but
* only very old build scripts don't know it, we should just require a recent one
* build script is ignoring it for chroot case

so why bother with another option?
2018-04-27 15:01:09 +02:00
bfcfddb0da fix some commands when used with / syntax between arguments 2018-04-27 14:41:08 +02:00
628f7e2cf5 - support "addcontainers" command 2018-04-27 09:16:13 +02:00
Marcus Huewe
d756f56af2 Show diff when creating a new comment (interactive_review)
Note that the diff is only shown, if it was requested before (that is
the "i" command was issued). The new behavior is consistent with the
other commands like "a", "d" etc.
2018-04-26 17:01:49 +02:00
lethliel
b7ada2cb5b fix missing newline in osc r -v
This adds a newline after unresolvable:. So the
messages are now well printed.
2018-04-26 15:38:25 +02:00
Marcus Huewe
cbd10644a2 Merge branch 'fix_IncompleteRead' of https://github.com/lethliel/osc
Retry 3 times in print_buildlog in case of an incomplete read. (Instead
of 3, the "http_retries" config option might be more appropriate.)
2018-04-25 11:02:52 +02:00
lethliel
1caa825e89 fixed IncompleteRead(0) error on long builds.
catch IncompleteRead() and retry until retry_counter is
reached. Then re-raise IncompleteRead()
2018-04-24 10:44:46 +02:00
lethliel
eb2647fd4f fix order of options valdiation. Now -P and -p is
honored if given by user.
2018-04-18 16:04:06 +02:00
fba9af7eee add new options to diff command:
--unexpand for local diffs only (bsc#1089025)
--meta for diffing meta files
2018-04-11 08:01:45 +02:00
Marcus Huewe
5341f3aae5 Merge branch 'fix_dod_get_binaries' of https://github.com/lethliel/osc
Support dod binaries in core.get_binarylist.
2018-03-14 14:06:13 +01:00
lethliel
9b42897413 fix getbinaries with DOD binaries
getbinaries of dod binaries do not have a size or mtime.
This will break. So just set to None and print unkown instead.
2018-03-14 13:45:38 +01:00
Marcus Huewe
a25209ac31 Merge branch 'improve_created_binaries_msg' of https://github.com/lethliel/osc
Improve message that is printed during "osc getbinaries...".
2018-03-13 15:16:12 +01:00
lethliel
e042c585c0 change output when creating binaries directory
was:
Creating binaries

is now:
Creating directory "binaries"
2018-03-13 15:00:46 +01:00
Marcus Huewe
caaefb0bf5 Fix and simplify util.safewriter.SafeWriter
Storing the error encoding in an "encoding" attribute "breaks" the
python3 "input" function: In essence, builtin_input_impl does a
getattr(sys.stdout, 'encoding'), which returns our error encoding
instead of the "real" stdout encoding. In order to avoid this, we
store the error encoding in an "_encoding" attribute.

Making SafeWriter a new-style class simplifies the code a lot.
2018-03-09 16:55:56 +01:00
305501f92c Revert "handle SSL connection closing ourself as advised in boo#1068470 bye"
This reverts commit 0ddb598743.

was incomplete and seems to be the wrong approach
2018-02-26 15:54:20 +01:00
Marcus Huewe
5fb71aa6d4 Merge branch 'fix_385' of https://github.com/lethliel/osc
Fix for python26, since it does not support an epath/xpath like
.//entry[@hash].

Fixes: #385 ("osc 0.162.x does not work anymore on SLE11")
2018-02-23 13:29:11 +01:00
Marco Strigl
e017a8e237 do not use complex path separator fixes #385
This is a fix for issue #385. osc commit breaks due to
the use of sfilelist.findall('.//entry[@hash]')

I now will iterate through the sfilelist and use

for entry in sfilelist.findall('entry'):
    if entry.get('hash'):
        ... execute hash code ...

This is a little bit slower, but should not break
on SLE11 anymore
2018-02-23 13:24:48 +01:00
0ddb598743 handle SSL connection closing ourself as advised in boo#1068470 bye
M2Crypto upstream maintainer.

Should fix hanging connections
2018-02-23 07:33:27 +01:00
Björn Geuken
030c2a41f8 Fix typo in example for osc search 2018-02-18 21:27:22 +01:00
Marcus Huewe
8948a26ae4 Fix --multibuild-package option handling in do_sendsysrq
Concatenating a str and a list does not work.
2018-02-09 15:35:52 +01:00
7fc251d81e fix sendsysrq command description 2018-02-09 10:08:13 +01:00
80352cbfc7 add sendsysrq command 2018-02-09 08:50:21 +01:00
Marcus Huewe
9c4f0d5d30 Ignore "--alternative-project <prj>" if it is equal to the wc's project
Without this change, using "--alternative-project <prj>" in combination
with "--multibuild-package <flavor>" yields to unexpected results (from
a user's point of view). Note that this may break existing (artificial)
workflows (e.g., using --alternative-project to ignore the package
meta's debuginfo flag), but these workflows should be rare and there
are options to achieve the same.

Fixes: #376 ("osc build -M something does not work with
--alternative-project")
2018-02-07 14:35:35 +01:00
Andreas Schwab
50222810f0 osc chroot: also mount devpts
This is needed for running expect, for example.
2018-01-28 16:53:52 +01:00
5e3fe8ba08 give a hint which package to install to get CA certificates 2018-01-26 09:46:52 +01:00
Marcus Huewe
d0213c63a9 All values in the conf.DEFAULTS dict should be strings
Non-strings cause errors during config value interpolation.

Fixes: #372 ("Commit causes crash: The 'build-jobs' config option
should be an integer")
2018-01-24 14:00:02 +01:00
Marcus Huewe
1a92c8b995 The 'build-jobs' config option should be an integer
This fixes the "config['build-jobs'] > 1" check in the build module.
2018-01-23 12:28:55 +01:00
Marcus Huewe
8cb0246c10 Merge branch 'fix_369_crash_at_req_issues' of https://github.com/lethliel/osc
Only include a tag if it "has" text in get_request_issues. Note
that the code in get_formatted_issues always assumes the presence
of the "label" tag.

Fixes: #369 ("crash trying to view diff of a request")
2018-01-22 14:46:21 +01:00
Marco Strigl
5b3d8633fa only assign issue detail if text for tag exists 2018-01-22 11:00:40 +01:00
Marco Strigl
2405134cbf fixes not working osc maintainer <prj>
The retrun at this point breaks the call, because in most
cases <prj> is not a binary. And the code always checks for the
binary first and then returns if no binary with the name <prj>
is found.
2018-01-11 11:57:22 +01:00
Marcus Huewe
0b241a9586 Consider "recommends" and "supplements" in build.create_deps
Fixes: #363 ("osc build -p ../rpms/tw doesnt send recommends to
the server which makes client side build behave differently to server
side build").
2017-12-26 23:28:14 +01:00
Marcus Huewe
ebb2f2ee0d Add support for querying weak dependencies
The following abstract methods are added to the PackageQueryResult
class: recommends(), suggests(), supplements(), and enhances().
Note that not all package/metadata formats have a notion of these
weak dependencies.

              rpm   rpmmd   deb   arch
recommends     x      x      x
suggests       x      x      x     x
supplements    x      x
enhances       x      x      x

(where "x" represents "supported"). In case of an unsupported weak
dependency, the implementation returns an empty list.

We need the weak dependency support in order to fix #363 ("osc build
-p ../rpms/tw doesnt send recommends to the server which makes client
side build behave differently to server side build").
2017-12-26 23:14:47 +01:00
ChrisWi
26f00c6749 change WARNING message for better understanding 2017-12-14 12:11:25 +01:00
lethliel
43be8f0e9c add meaningful error if no attribute name 2017-12-08 13:20:24 +01:00
Marcus Huewe
b1d6d8347b Fix "osc buildinfo --prefer-pkgs <dir>..." (finally!)
This is a follow-up commit for commit ce9a6d0 ("Fixed
\"osc buildinfo --prefer-pkgs...\"").
2017-12-06 18:04:30 +01:00
lethliel
aa01b1c52a reopen 0.163 development 2017-12-06 10:22:15 +01:00
lethliel
067fd4cd2d release 0.162.1 2017-12-05 14:29:03 +01:00
Marcus Huewe
9116d8ff97 Support expansion/unexpansion of a link when updating to certain rev
There is no good reason why "--revision <rev>" and "--expand-link" or
"--revision <rev>" and "--unexpand-link" should be mutually exclusive
during an "osc up" of a package wc.
Introduce the new "--linkrev <rev>" option to specify a rev of the link
target that is used during link expansion.
2017-12-03 16:45:36 +01:00
Marcus Huewe
f698103977 Send sha256 hashes for tracked files if the wc is pulled/linkrepair
In case of a pulled/linkrepair wc, it is possible that the backend
requests a hash for a tracked file, which is neither added, restored,
nor modified. For instance, this can happen if a new file was added
to the link target. Hence, for a pulled/linkrepair wc always send
the sha256 hashes of the tracked files.
2017-12-03 15:02:22 +01:00
e818989d21 0.163 development open 2017-11-28 15:13:26 +01:00
2b0f015de0 0.162.0 2017-11-28 15:12:29 +01:00
lethliel
f0325eb0b5 added code to get the sha256 hashes of files
This is needed for a new validation of the source server.

The source server will 'ask' for the sha256 sum of files which are new or
modified and osc calculates the sha256 sums for those files and sends them
back to the server.

The server checks the sha256 sums and if dies if something is wrong.
2017-11-27 16:18:19 +01:00
Marcus Huewe
03c25eb8f9 Merge branch 'return_external_2.6_compat' of https://github.com/lethliel/osc
Workaround for python26 due to missing subprocess.check_output.

Fixes: #355 ("subprocess.check_output() not in Python 2.6")
2017-11-21 14:25:55 +01:00
lethliel
7581856d9b added compat code for python 2.6
if subprocess has no method check_output us Popen instead
2017-11-21 14:06:31 +01:00
Marcus Huewe
5791d1bb5c Merge branch 'mount_sysfs' of https://github.com/lethliel/osc
Mount sysfs during "osc chroot". The current implementation
of "osc chroot" is a major pain for plain "su" users, because the
root password has to be entered several times - we should fix this.

Fixes: #354 ("Mount sysfs in chroot")
2017-11-21 12:34:15 +01:00
lethliel
00b6a9fe27 in chroot also mount /sys in addition to proc 2017-11-21 10:52:40 +01:00
Marcus Huewe
0f9ab38948 Merge branch 'fix_#351' of https://github.com/lethliel/osc
Fix logic for finding disabled repos in do_repositories.

Fixes: #351 ("Not all options for osc getbinaries is listed")
2017-11-09 11:56:36 +01:00
marco
12b17cfc5d consider arch when checking the disabled repos
At the moment just repo.name is considered. So if
the repo is disabled for s390 all other repo / arch
combination are not shown in the repo list.

To be able to change this r is now a list of dicts
containing the name and arch of the disabled repo.

None for repo if a complete arch gets disabled
None for arch if a complete repo gets disabled
2017-11-08 14:02:20 +01:00
Marcus Huewe
cb376a1a34 Merge branch 'config_replace_issue_313' of https://github.com/lethliel/osc
Store a newly created config file in $XDG_CONFIG_HOME/osc/. For backward
compatibility, ~/.oscrc is used, if present.

Fixes: #313 ("oscrc should be stored in $XDG_CONFIG_HOME on linux")
2017-11-08 12:23:07 +01:00
marco
6bc2d3f939 use XDG_CONFIG_HOME/osc/oscrc as default config
write oscrc to the default location for user-specific configuration.
If XDG_CONFIG_HOME is not set use ~/.config/osc/oscrc which is basically the same.

If there is already a ~/.oscrc use this one (for compat reasons). Existing user
installations should not get affected by this commit.

The order is the following:

Given config with -c
config defined in OSC_CONFIG
existing ~/.oscrc
default XDG_CONFIG_HOME/osc/oscrc
2017-11-08 11:17:11 +01:00
690dbf42ab Small documentation fix to clarify attribute usage. 2017-11-08 10:55:38 +01:00
Marcus Huewe
9d4390eed9 Close file before returning from core.dgst
The old f.close() call was never reached.
2017-11-07 17:52:34 +01:00
Marcus Huewe
3c1bb1cf0a Merge branch 'xz-support' of https://github.com/suihkulokki/osc
Support an xz compressed control.tar file. In case of a control.tar.xz and
a missing lzma module, an exception is thrown at runtime (for now, in order
to avoid a hard depedency to the lzma module, which is no standard module).
2017-10-30 22:24:00 +01:00
Riku Voipio
9166a7f075 debquery: support control.tar.xz
Similar to recent fixes in libsolv and obs-build. Since tarfile
on python2 doesn't do lzma, decompress the file into memory and
feed it as a fake file via StringIO to tarfile
2017-10-30 14:10:54 +02:00
marco
b52edb6513 Open 0.162 development 2017-10-26 14:47:40 +02:00
marco
fbcda8ae6e prepare 0.161.1 release 2017-10-26 14:21:33 +02:00
Marcus Huewe
a884b58313 Fix python 2.6 SyntaxError
Multiple context expressions are only supported since version 2.7.
It was introduced in commit f6f879d ("Fix potential shell injection
when running rpm2cpio").
2017-10-26 14:16:58 +02:00
marco
b7d3ae992c Open 0.162 development 2017-10-26 13:51:51 +02:00
marco
72f9f89e85 prepare 0.161 release 2017-10-26 13:33:06 +02:00
Marcus Huewe
06d0693f64 Merge branch 'show_issues_in_diff' of https://github.com/lethliel/osc
When showing a diff in the interactive review mode, also show the issues
that are associated with the request.
2017-10-20 17:39:27 +02:00
marco
895f168b91 show issues (bugs) in interactive review diff 2017-10-20 12:59:47 +02:00
marco
3a5bd607f8 fixes broken osc results on project level 2017-10-18 15:17:56 +02:00
Marcus Huewe
760a3e5558 Merge branch 'show_overview_in_diff' of https://github.com/lethliel/osc
Show request details, when displaying a diff in the interactive
review mode.
2017-10-11 13:30:48 +02:00
marco
4f518b5e59 prepend request summary to diff in interactive mode 2017-10-11 08:55:55 +02:00
Marcus Huewe
0eecdaf830 Add missing comment
Follow-up commit for f6f879d.
2017-10-10 16:36:58 +02:00
Marcus Huewe
f6f879dac5 Fix potential shell injection when running rpm2cpio
Actually, there is nothing that can be injected, except the "-h"
option. However, in case rpm2cpio evolves, we are on the safe side.
Also, document the potential shell injection in the cpio call
(the comment was accidentally removed in commit dbdc712) (the
current osc code is not affected, because we never pass filenames
via *files to core.unpack_srcrpm).
2017-10-10 16:24:42 +02:00
Marcus Huewe
a5c7611aee Support unusual filenames in "osc add <directory>"
This way, we can also support directories/files that contain
a newline "\n" etc.
2017-10-10 16:18:45 +02:00
Marcus Huewe
c3ba1fbf63 Eventually fix potential shell injections for find
It seems that the "find" binary has no way to indicate an
end of options for its arguments. Hence, we use os.walk to mimic
"find"'s behavior, which is also the cleaner solution.

Fixes: #340 ("osc add of directories does not quote the argument")
2017-10-10 16:18:39 +02:00
Marcus Huewe
d66ccb2a7d Cleanup babysitter a bit
In case of an exception, return status code 1 by default.
2017-10-06 13:25:25 +02:00
Marcus Huewe
6acc90d438 Exit with status 1 in case of a ServiceRuntimeError
Fixes: #344 ("'osc ci' exit code doesn't take source service results
       into account")
2017-10-06 13:19:15 +02:00
Marcus Huewe
760d4d65d7 Supersede existing requests if --yes is passed to "osc sr"
This basically reverts commit b2b59ca, because the old code performed
a "no" instead of a "yes" (see also the discussion in
https://github.com/openSUSE/osc/pull/269).

Fixes: #343 ("'osc sr --yes ...' doesn't supersede existing requests
       as promised")
2017-10-06 12:32:52 +02:00
Marcus Huewe
aea395a7e7 Document 'R' (replaced) file state
Fixes: #336 ("Missing explanation for status 'R'")
2017-10-03 22:05:41 +02:00
Marcus Huewe
f52cf7745a Take --repo/--arch options in "osc prjresults --xml..." into account
Fixes: #341 ("prjresults doesn't support repo/arch options")
2017-09-28 15:55:11 +02:00
Marcus Huewe
8a81a68028 Cleanup url construction in core.show_results_meta
The old code was flawed, because, for instance,
core.show_results_meta(apiurl, project, arch=['x86_64']) resulted
in a wrong http request: GET <apiurl>/build/<project>/_result&arch=x86_64
(note the "&" instead of the correct "?"). The drawback of the new
implementation is that we have to do the proper quoting manually.
2017-09-28 15:52:25 +02:00
Marcus Huewe
2d327df4e7 Allow multiple --repo and --arch options in "osc prjresults"
This harmonizes "osc prjresults --help" with osc's actual behavior.
Also, core.get_prj_results expects lists (or None) instead of strings
for the corresponding repo/arch parameters.
2017-09-28 15:29:48 +02:00
Marcus Huewe
dbdc712018 Really fix potential shell injections
This is a follow-up commit for commit c9c0f8a. Using core.run_external
with shell=True is too error-prone.

Fixes: #340 ("osc add of directories does not quote the argument")
2017-09-28 14:48:30 +02:00
Marcus Huewe
63c2aa3630 Do not run diff3 in the shell
This fixes a potential shell injection.

See also: #340 ("osc add of directories does not quote the argument")
2017-09-28 12:03:53 +02:00
Marcus Huewe
c9c0f8a731 Fix potential shell injections
Also, document a potential shell injection in core.unpack_srcrpm
(via the "files" parameter), which cannot be exploited, because
"files" is not used by the current osc code.

Fixes: #340 ("osc add of directories does not quote the argument")
2017-09-28 00:09:42 +02:00
Marcus Huewe
08504a1509 Merge branch 'lintlog_in_interactive_review' of https://github.com/lethliel/osc
Add support for showing rpmlint logs during interactive review.
2017-09-26 23:47:38 +02:00
marco
b59d40c57c show lintlog (li) in interactive mode 2017-09-26 20:20:02 +02:00
Marcus Huewe
3389db2c8f Merge branch 'repotype-pacsuffix-hint' of https://github.com/Conan-Kudo/osc
Use the buildconfig's binarytype to determine the correct pacsuffix
(TODO: cleanup Buildinfo pacsuffix code a bit).
2017-09-26 14:06:29 +02:00
Neal Gompa
ef9b308d50 Add pacsuffix hint using BinaryType from build config 2017-09-26 07:57:59 -04:00
Marcus Huewe
c2922e8a67 Open 0.161 development 2017-09-26 12:45:48 +02:00
73e8e3e97b prepare 0.160.0 release
mainly needed for new container support
2017-09-22 09:06:18 +02:00
Marcus Huewe
b07ba53ad0 Allow slash separated arguments in "osc rpmlintlog" 2017-09-01 10:51:28 +02:00
Marcus Huewe
38af00d509 Merge branch 'rpmlint' of https://github.com/lethliel/osc
Add new "osc rpmlint" command to retrieve the rpmlint.log file (if
present).
2017-09-01 10:49:16 +02:00
marco
99309db315 show rpmlint.log
Provide function to show the content of rpmlint.log.
The core function get_rpmlint_log will later be used in the interactive
request mode to provide the rpmlint logs to the reviewer.

I add the osc rpmlint | rpmlintlog | lint command also.
2017-09-01 10:27:05 +02:00
Marcus Huewe
c2a4d14c42 Merge branch 'patch-2' of https://github.com/jnweiger/osc
Fix UnboundLocalError in do_submitrequest.
2017-09-01 09:45:35 +02:00
Jürgen Weigert
2e6c838d66 submitreq entire project fix
`osc submitreq TARGETPROJ`
bails out with unbound variable t. This is true. the variable should be target_project.
2017-08-31 14:58:22 +02:00
3a41ae64f0
osc tr: show the time when the event happened 2017-08-31 13:57:21 +02:00
Marcus Huewe
e0bce5652a Merge branch 'show_comments_in_reviews' of https://github.com/lethliel/osc
Show the comments of a request during interactive review.
2017-08-25 13:23:17 +02:00
marco
7804a8c577 show user comments in interactive mode 2017-08-25 11:51:55 +02:00
Marcus Huewe
28368ecd4a Document possible action types for "osc rq --type <TYPE>"
The "group" action type is omitted on purpose, because it is not/was
never supported.
2017-08-24 12:14:25 +02:00
Marcus Huewe
1c21c67659 Do not ignore --type parameter in "osc review list --type..."
There is no reason why we should ignore the --type parameter in this
case.
2017-08-24 12:01:56 +02:00