- core.get_binarylist(): add "verbose" switch that returns instances of class
File, instead of filenames
- improve core.get_binary_file():
* clean up tempfiles,
* set mtime,
* set file mask
Function signature is changed, because I'm not aware of any user than the example
script I once published on http://en.opensuse.org/Build_Service/osc_plugins
- do_addremove():
* if "do_package_tracking" is enabled scan the project dir for new/disappeared packages
* added --recursive option (If CWD is a project dir then scan all package dirs as well)
* minor pathname fix
- fixed argument handling in some methods: don't remove items from a list while looping over it
now 'osc submitrequest create' checks if you already submitted requests for this package, if so it'll ask you if the previous submissions should be revoked
OscConfigParser() behaves like a normal ConfigParser() object. The
only differences is that it preserves the order+format of configuration entries
and that it stores comments.
In order to keep the order and the format it makes use of the ConfigLineOrder()
class.
- removed .netrc cruft from the conf module
- other config cleanups
- fix streaming support in osc buildlog. do not download all of the
buildlog into memory before printing it (it could be very big), just
print as the log comes in
* extra-pkgs and urllist settings can be separated by a comma and/or whitespace:
'extra-pkgs: foo, bar xyz'
* 'True', 'False' etc. are accepted values for boolean options
one, when a source link cannot be applied [bnc#409373]
- linkinfo elements can have an "error" attribute which indicates link
application failure, which means that updating the working copy is not
possible.
- Before this fix, osc could silently remove locally modified sources!
Default is "extra-pkgs = vim gdb strace" -- and it can be overridden with -x ''
on the commandline, or set to empty in the config. [bnc#345142]
- make tilde expansion work on the packagecachedir setting
otherwise packages are stored incorrectly in the package cache
and that might cause problems later. Also refuse to build an
architecture locally that is not supported by the host architecture
circumstances -- when there is a working copy, and the package is a source
link, osc will figure out what's needed.
- don't stop submitrequests on packages that have a devel project defined, if
the submit comes from that project.
- osc checkout: checkout of source links is now done in expanded form per
default. --unexpand-link can be used to get the raw link file (and patches).
- edit_meta(): raise an exception instead of printing an "useless" error message.
- make "osc meta prjconf <project> -e" work again (probably r3702 caused the problem)
https://forgesvn1.novell.com/svn/opensuse/branches/buildservice/osc-exception-handling
........
r3774 | poeml | 2008-04-22 17:39:45 +0200 (Tue, 22 Apr 2008) | 3 lines
A first draft on implement systematic exception handling:
Add errors.py with some exceptions defined, and babysitter.py to handle them
........
r3775 | poeml | 2008-04-22 22:34:12 +0200 (Tue, 22 Apr 2008) | 9 lines
- new global options:
--debugger jump into the debugger before executing anything
--post-mortem jump into the debugger in case of errors
-t, --traceback print call trace in case of errors
- traceback and post_mortem can also be set in .oscrc.
- catch more errors (HTTPError).
- make config accessible from outside of the Osc instance, by making it a class
attribute
........
r3779 | poeml | 2008-04-23 00:55:49 +0200 (Wed, 23 Apr 2008) | 5 lines
- new global option:
-d, --debug print info useful for debugging
- catch some more errors (HTTPError), with OscHTTPError which isn't very
advanced yet.
........
r3781 | Marcus_H | 2008-04-23 01:02:00 +0200 (Wed, 23 Apr 2008) | 2 lines
- added OscConfigError class (just for testing).
- small change in the signature of the OscBaseError constructor (actually we should get rid of the 'args' tuple because it will be deprecated sooner or later
........
r3782 | Marcus_H | 2008-04-23 02:02:13 +0200 (Wed, 23 Apr 2008) | 2 lines
- access the prg.conf.config dict in a save way - this way we avoid AttributeErrors if the prg.conf.config dict doesn't exist
- in case of an configparser error we have to use the prg.options object directly (this was removed in r3781 by accident)
........
r3787 | poeml | 2008-04-23 09:23:56 +0200 (Wed, 23 Apr 2008) | 4 lines
- remove local exception handling from do_req
- for HTTPError, print details (headers and response) when in debug mode
- catch AttributeError
........
r3789 | poeml | 2008-04-23 16:23:14 +0200 (Wed, 23 Apr 2008) | 4 lines
- errors: add two new classes named OscWrongOptionsError and OscWrongArgsError
- commandline: raise instances of the new errors in a number of places
- commandline: add get_cmd_help() to Osc instance which returns the formatted help of a subcommand
........
r3790 | Marcus_H | 2008-04-23 16:48:28 +0200 (Wed, 23 Apr 2008) | 4 lines
- added 2 new exception classes: OscNoConfigfileError and OscIOError
- added new method write_config() to the conf.py module: This method writes osc's configuration file
- minor cleanups in the conf module
........
r3791 | poeml | 2008-04-23 17:11:07 +0200 (Wed, 23 Apr 2008) | 3 lines
small compatibility fix for r3790: try-except-finally isn't supported in
python-2.4.2, thus do the same as try-except inside a try-finally.
........
r3792 | poeml | 2008-04-23 17:37:53 +0200 (Wed, 23 Apr 2008) | 2 lines
fix up the remaining places regarding handling of errors related to commandline parsing
........
r3793 | poeml | 2008-04-23 17:40:34 +0200 (Wed, 23 Apr 2008) | 3 lines
raise a NoWorkingCopyError in osc.core.store_read_project() in case of an
IOError
........
r3798 | Marcus_H | 2008-04-23 23:55:24 +0200 (Wed, 23 Apr 2008) | 1 line
ported -r3797 from trunk
........
r3802 | Marcus_H | 2008-04-24 11:00:55 +0200 (Thu, 24 Apr 2008) | 1 line
ported -r3801 from trunk
........
r3805 | poeml | 2008-04-24 12:52:30 +0200 (Thu, 24 Apr 2008) | 2 lines
raise OscHTTPError in show_pattern_meta(), replacing local error handling
........
r3817 | poeml | 2008-04-24 20:21:32 +0200 (Thu, 24 Apr 2008) | 9 lines
- remove errors.OscHTTPError again.
it seems simpler to use urllib2.HTTPError instead (and just add a specific
error text message where appropriate, and re-raise)
- for 404s, check out _which_ part was not found
it is very ugly, but may be considered Good for pragmatic reasons
- removed local exception handling and workaround for returned 500's from
delete_package() and delete_project(), thereby getting rid of 4 possible exit
points.
........
r3818 | Marcus_H | 2008-04-24 22:36:17 +0200 (Thu, 24 Apr 2008) | 1 line
- this check is superfluous because every HTTPError instance has a code attribute
........
r3819 | poeml | 2008-04-25 00:39:39 +0200 (Fri, 25 Apr 2008) | 1 line
remove a forgotten debug line from core.delete_project()
........
r3820 | poeml | 2008-04-25 10:07:58 +0200 (Fri, 25 Apr 2008) | 2 lines
- ditch local error handling from wipebinaries(), rebuild(), and abortbuild()
........
r3821 | poeml | 2008-04-25 10:56:38 +0200 (Fri, 25 Apr 2008) | 2 lines
It is never needed to import the exception module.
........
r3822 | poeml | 2008-04-25 11:13:39 +0200 (Fri, 25 Apr 2008) | 4 lines
- when going into the debugger with --post-mortem, always print a traceback before
(thus implying -t)
- do not jump into the debugger if not on a TTY, or working in interactive mode
........
r3827 | poeml | 2008-04-25 13:07:46 +0200 (Fri, 25 Apr 2008) | 9 lines
- add errors.OscWorkingCopyOutdated, which takes a tuple with three args:
path to working copy, current rev, expected rev
- add handler for urllib2.URLError errors to the babysitter
- simplify the OscWrongArgsError and OscWrongOptionsError handlers, by removing
the extra line "Sorry, wrong ..." that was printed before the messages given
when the error was raised.
- remove one more errors.OscHTTPError which was still there, and raise
urllib2.HTTPError instead (show_package_meta())
........
r3829 | poeml | 2008-04-25 14:19:10 +0200 (Fri, 25 Apr 2008) | 11 lines
- comment some methods in osc.core which are used by nearly all do_* methods in
osc.commandline
- improve "is not a package/project dir" error messages, by printing the
absolute path tried, instead of '.' for the cwd
- make core.store_read_package() raise a proper NoWorkingCopyError instead of
terminating
- give attribution to things in babysitter.py copied from mercurial
- prefix HTTPError exceptions with 'Server returned an error:'
- remove obsolete local error handling from do_prjresults(), do_importsrcpkg(),
do_repos()
........
r3830 | poeml | 2008-04-25 14:29:35 +0200 (Fri, 25 Apr 2008) | 1 line
catch IOError exceptions in the babysitter
........
r3837 | poeml | 2008-04-25 17:27:36 +0200 (Fri, 25 Apr 2008) | 5 lines
- do_remotebuildlog: raise errors for wrong arguments, remove exits
- raise AttributeError in make_meta_url() instead of exiting
- delete unused method core.delete_server_files()
- replace exit call inside make_meta_url() with an AttributeError
........
r3838 | poeml | 2008-04-25 17:49:18 +0200 (Fri, 25 Apr 2008) | 1 line
simplify the check in do_checkout if a project exists, by using show_project_meta() instead of meta_exists
........
r3839 | poeml | 2008-04-25 18:31:26 +0200 (Fri, 25 Apr 2008) | 6 lines
- commandline do_checkout(): change the order of the two checks, first do the
(cheaper) check for existing directory
- core.core checkout_package(): simplify the check in if the package
exists, by using show_package_meta() instead of meta_exists
Let it throw an exception, instead of using sys.exit().
........
r3841 | Marcus_H | 2008-04-27 15:48:06 +0200 (Sun, 27 Apr 2008) | 5 lines
- added 2 new exception classes: PackageError() and PackageExistsError. The PackageError() class is meant to be the base class for all subsequent package exceptions.
- get rid of 2 sys.exit(1) calls
- make the update() method of the Project() class safer: in any case we have to write the _packages file otherwise the package tracking will be screwed up.
- minor fix in delPackage(): use getTransActPath() when printing out the filename
........
r3842 | Marcus_H | 2008-04-27 16:52:55 +0200 (Sun, 27 Apr 2008) | 3 lines
- make the commit() method safe: in any case we have to write the _packages file otherwise the package tracking will be screwed up.
- removed another sys.exit(1): raise an exception if a package is missing when doing a commit. For now we use the PackageExistsError() exception but this might change in the future (updated description of PackageExistsError() according to this change)
........
r3848 | poeml | 2008-04-28 12:46:45 +0200 (Mon, 28 Apr 2008) | 3 lines
rename several error classes, dropping the "Osc" prefix, and "Error" suffix in
cases where they don't really make sense.
........
r3849 | poeml | 2008-04-28 12:57:32 +0200 (Mon, 28 Apr 2008) | 3 lines
- rename osc.errors module to osc.oscerr, to make it easier to import it from
other programs and have a crystal clear namespace
........
r3850 | poeml | 2008-04-28 13:26:12 +0200 (Mon, 28 Apr 2008) | 2 lines
split PackageExists exception class into PackageExists and PackageMissing
........
the BUILD_DIST environment variable will be ignored by osc
- the following environment variables can still be used:
# OSC_SU_WRAPPER overrides the setting of su-wrapper.
# OSC_BUILD_ROOT overrides the setting of build-root.
# OSC_PACKAGECACHEDIR overrides the setting of packagecachedir.
(now they're documented in the helptext)
* make_dir() and checkout_package() got a new optionally 'prj_dir' parameter. This parameter specifies in which dir we want to create new project/package dirs.
Add support for handling linked packages in expanded form. They can be checked
out, updated (expanding or unexpanding them), and built locally.
Missing: commit support.
Newly introduced options are:
- osc checkout: --expand-link
- osc update: --expand-link and --unexpand-link
Other changes:
- osc core: add show_upstream_xsrcmd5() method which returns the xsrcmd5 (if a
linkinfo element exists only, of course)
- made updating of expanded sources work with package tracking, too.
- add isexpanded() method to Linkinfo class
- add isexpandedlink() method to Package class
- read lsrcmd5 and srcmd5 from the linkinfo element, which are present in
expanded links.
osc info:
- make it show info about expanded and non-expanded links
a list. The query can now also be given as a dictionary, and in that case it
will be automatically urlencoded. The behaviour for a list is unchanged for
the reason of backward compatibility.
* createPackageDir(): create and initialize a new package dir in
the given project.
* get_apiurl_usr(): returns the username for a certain apiurl
* get_configParser(): returns an ConfigParser() object which can be
used for parsing the ~/.oscrc file
- fixed username issues when creating a new package (the problem was
that the username for the default host was used and not the one for
that specific apiurl)
- some small fixes in the do_importsrcpkg() method
- when requesting a submit, save the source package's revision id (looking up
what it currently is)
- give the user a way to override it, to submit an older revision
- when using show --diff, take the actual old revision into account. Thus, the
diff is against the source revision of the time of request creation.
- rename 'refuse' -> 'decline', which is more polite.
- show: make the diff optional
- add RequestState class
- handle state history
- implement 'decline' action
meta_exists, make_meta_url, checkRevision
if no apiurl parameter is specified the global value (conf.config['apiurl'])
will be used. This should fix bug #361764
- some small apiurl fixes in the make_diff method (the package_tracking branch already has this fix)
* --local-package: use this option if you're trying to build a
package which doesn't exist on the server
* --alternative-project <project>: use this option to specify an
alternative project if the current project doesn't exist on the
server, e.g.: osc build [OPTS] --alternative-project openSUSE:10.3 standard i586 BUILD_DESCR
- some other minor cleanups in build.py
- get_repos_of_project() is now a generator function
(you don't have to checkout the package to view the buildlog)
- renamed "get_log()" to "get_buildlog" to avoid any confusion with
"get_commitlog()"
- added new method "print_buildlog()" which prints out the buildlog on the
standard output
* some new things like showing newly added files
* moved the whole code into a new methode (make_diff())
* behave more like svn when doing a diff against a certain revision
- changed the storedir attribut of the Package() class to an absolute
filename - normally this shouldn't have any impact on existing methods, functionality etc.
get_config(override_conffile = None,
override_http_debug = None,
override_apisrv = None)
Thus, it should be possible to use the osc module with one simple
conf.get_config() call. It is no longer required to set up the api url in the
config dict, and call conf.init_basicauth().
* by default the new files will be comitted
(if it is a new package or if there are no
source files on the server)
* added --disable-commit option switch to
import the package only locally
* added --delete-old-files option switch to
delete old files from the server.
- some changes in the Package class:
* extended existing commit() method to do
all the commit work (before it was done
in the do_commit() method).
* moved stuff from the constructor into a
new method (update_datastructs()).
* now a package can be imported from a
src.rpm
* thanks to Peter for his is_rpm() and is_srcrpm()
methods
- split out some methods into smaller ones (so they
can be reused for other tasks)
- added some new methods for unpacking a src.rpm
- changed read_meta_from_spec() method to be more
flexible (it is possible to specify the tags and
sections).
- rework editmeta class, because it was programmed to pull content from the
server first
- add a catch-all for the depracated commands which refers to the new meta command
- if PUT on metadata fails with a 500, and http_debug is True, print out the
body of the server reply
- rename update_pacmeta() to update_local_pacmeta() for clarity
- rename update_filesmeta() to update_local_filesmeta() for clarity
* when searching a package/project it
is also possible to search for the search term
in the <title /> and <description /> elements of
a package/project.
* show only exact matches
* etc.
- please tell me if the usability is too "complex"
or if the output of the results is too weird.
edituser. TODO:
- write usage info
- implement -c|--create
- implement -F|--file
- check whether I broke usage of metadata copying/creation from withing
linkpac or copypac
- re-implement createpac and friends as aliasses to the new meta command -- I
just commented them out for now
- fix metadata change detection, which no longer relies on the timestamp of the
temporary file
- add close() in dgst(), which seemed to miss (I can't see a reason why it
should not be there0
- commit: new option -F/--file to read commit message from file
- commit: print committed revision number
- Package.commit(): return the new revision number
- implementing a provisionary log command for the commit log
- commit: implemented -m option for the commit message
- commit files together, if (and only if) do_commits = 1 is set in the
[general] section of .oscrc
- extended "osc co prj pac" to checkout a specific revision of pac
- extended "osc up" to update to a specific revision
- extended "osc diff" to diff the working copy against a
specific revision on the server. NOTE: comparing two
server-side revisions (osc diff -r 11:12) is currently
not supported!
called with change_is_required=False.
- linkpac: correctly set the destination project in the template used for the
new package, although the api didn't seem to notice the mistake so far.
related to points that were raised during employment in osc:
# v0.8.3
- Fix a bug where errors with passing an incorrect number of args to
functions in do_foo() implementations would be masked.
# v1.0.0
- [backward incompat] `Cmdln.main()` no longer takes an `optparser`
argument. Top-level option parsing has been changed so that top-level
options for a `Cmdln` subclass can more naturally be defined and
handled on the class definition. Changes:
- `Cmdln.main()` calls `self.get_optparser` to get an option handler.
Subclasses should overload this method for custom top-level options.
- After option parsing, but before sub-command handling, the
`self.postoptparse()` hook is called.
- Add a `version` attribute on `Cmdln` subclasses. If set, the default
top-level option parser will have a `--version` attribute.
- [backward incompat] Simplify the StopProcessing/opts.stop handling for
option handling in subcommands. The "opts" argument to "do_*"
sub-command functions will no longer have a "stop" value.
StopProcessing is now called StopOptionProcessing. This shouldn't
affect simple usage of cmdln.py.
- adjust osc.commandline for these changes.
- implement --extra-pkgs option
- pass the list of extra packages to the backend, as "add=pkg" query parameters
- use osc.core.get_buildinfo(), instead of os.system('osc buildinfo ...')
- implement adding query parameters to constructed URLs in a more generic way
- save api server url to .osc/_apiurl. This requires changing makeurl() and all
calls to it.
- implement 'info' subcommand, essential for debugging these changes.
- use new api routes in all places
- buildhistory works again
- copypac: implement package copy from one buildservice instance to another
(--to-apiurl option)
- the results subcommand handles <working copy> arguments now
which was still using sys.argv
- build: use own api server as upstream URL in urllist. So far,
api.opensuse.org was hardcoded in the download URL [#265211].
a URL now, so the variable "scheme" which was needed in addition before
becomes obsolete. For backward compatibility, a hostname (and scheme
variable) are still accepted. Likewise, the auth sections in the config take
a URL now, and a hostname:port to keep old config working.
Furthermore, apisrv can be overridden by -A on the commandline. HTTP or HTTPS
scheme is determined from the URL. Credentials must be configured in .oscrc.
Bump version to 0.95.
New features:
- implement "rebuild all failed packages", via --failed option in rebuildpac
subcommand (new api route)
- status -v shows all files, including unmodified ones
- suppress the legend in prjresults by default (show with -l)
- add global options to override config
- can use arbitrary api server via global -A option
- -H enables HTTP traffic debugging
- --version
Bugfixes:
- fix typo in delete_project() (the line building up the URL got lost)
- fix the commit subcommand's arguments. This works correctly now:
osc ci ../test/onlyinwc `pwd` fstab ../test/f2
- fix buildinfo subcommand, if no specfile is posted. Broke with the recent URL
handling rewrite, but didn't seem to bother because the build subcommand
always sends the specfile.
- try to fix buildhistory route, but it might be gone actually (need to pursue)
- add --clean/--noinit to osc build help output
- adding http_GET/POST/PUT/DELETE() functions, which dispatch to
http_request(), and use them everywhere
- removing othermethods.py
- keeping urlopen(), in case it is used from externally, but have it print out
a "depracated" message
- finally, implementing a global HTTP debug mode
* I moved edit_meta into an own class representing metadata. that
metadata is retrieved in the constructor, can be accessed as an
elementree and can be synced back to the filesystem.
I'm using that myself to update metadata and I changed the use of
edit_meta to use that class.
to create a user if it doesn't exist yet. A new command 'usermeta' replaces
'id' respectively 'userid'.
- add preliminary support for doing uploads with a subsequent "commit" request
(not switched on now, since I want to do more testing on this, and there is
no way to pass on a commit message yet anyway.)
- add runtime check for build.rpm version, so the rpm package dependencies is
no longer required
- fix typo introduced last time in the editmeta, deletepac, deleteprj commands
- rewrite the PUT and DELETE request methods to
- transparently handle HTTPS
- handle path parameters (for commits)
- send user agent
- rename get_user_id() function to get_user_meta()
- change copyright notice from Peter Poeml to Peter Poeml / Novell Inc.
- fix testsuite
- add CREDITS file
- move all configuration code into module osc/conf.py
- now, the API server can be configured in .oscrc with
apisrv = ...
- also, 'scheme' (http/https) is no longer a module variable but can be
configured
- all config is in DEFAULT
- ignore vim swap files
- use urllib() convenience wrapper in some functions that used
urllib2.urlopen() and had their own error handling. Instead, it seems to make
sense -- in the future-- to create our own errors and propagate them up, in
cases where the error handling of urlopen() is too generic
- rename get_slash_source() to meta_get_project_list() for consistency
- show local time in get_buildhistory(), not UTC
- rewrite help text of 'rebuildpac' command
- allow to run commandline.py from the commandline (without the wrapper)
- don't send a space when doing a POST request without body
- fix testsuite, and add test for relative directories with 'status'
- add is_package_dir() method
- add pathjoin() method, similar to os.path.join but it removes a leading ./
- add 'deleteprj' command and delete_project() method. Note: the API server
doesn't seem to support it yet.
- add 'prjresults' command to display aggregated build status over the entire
project
- 'rebuildpac': accept additional repo and arch argument. Note: the syntax has
changed.
- 'log': print usage info if called with missing arguments
- extend urlopen() so it takes data for optional POST
- extend get_buildinfo() to take optional specfile argument
- osc.build.main: send specfile to server
- buildinfo(): accept specfile as third argument, and document it
'buildhistory', and make it work. It still gives out raw xml, which might be
changed
- add some help text to the linkpac documentation
- add '.git' to exclude_stuff
- extend edit_meta() with two arguments: template=new_package_templ, change_is_required=True
- increase BLOCKSIZE in othermethods.putfile() (1024 seemed a little small)
- extend othermethods.putfile() to serve a string buffer instead of a real file
- for filelists, don't read _meta file any longer (depracated). Read directly
from /source/<project>/
(now also in DELETE requests, where no quoting was done before at all)
- 'id': properly quote the query part of the URL
- fix up tests.py
- add to TODO
- remove requirement on pyxml package
- use list comprehensions in some places
- fix 'rebuildpac' command to not show raw xml
- fix updatepacmetafromspec to cope with subpackages during parsing
- add 'rebuildpac' command, which triggers a rebuild for all
repositories/architectures of the package
- fix merge on 'update', if osc is called from another directory
- escape '+' signs in filenames in GET requests as well (adds to [#153725,
181593])
- don't fail on 'resolve' when the working copy is in a newer rev already
requests [#153725, 181593]
- before committing, make sure that the working copy is up to date (added
show_rev() function)
- add 'commit' as subcommand alias for 'ci/checkin'
- use os.path.abspath() in Project and Package classes
- checkin: update local copy in store (fixes diff using obsolete old version)
- use shutil.copy2 instead of home-grown copy_file
- make put_source_file() and delete_source_file() methods of the Package class;
rename delfile method to delete_localfile
- make all subcommands functions -- so they can be used from outside (they
still print directly their output; they could become iterators later)
- deprecate 'init' subcommand
- 0.5
status:
- use new file metadata which provides checksum, size and mtime
- don't list unmodified files
- fix some conditions where status was too stupid so far
update:
- much faster now, since only new or changed files are downloaded
- print revision number
checkout:
- mark project directory as such (to be used later)
diff:
- faster, since 'status' is faster
- work against local copy
checkin:
- update package metadata from specfile
- fix argument handling
add:
- ignore .osc and other files
other changes:
- sanitize argument handling, so most commands can work with
arbitrary files/directories as arguments (work is always done per package)
- add history command (doesn't seem to be working yet)
- on first usage, ask for username password on first usage,
and store them in .oscrc (.netrc can still be used)
- add setup.py
- add osc-wrapper.py as a wrapper script, so the module can be used when
installed in the system as well as in the uninstalled source directory
