The two slowest staging API calls are for information that rarely changes.
By caching the result the commands typically execute over twice as fast.
Going further can see improvements of an order of magnitude or more by
caching almost all the GET requests.
In contrast to osclib/memoize.py this cache operates at the HTTP request
level. This has several advantages:
- Caches the expensive part (ie the HTTP request). There are a number of
functions in osc.core and elsewhere that make the same API request, but
process the result differently which would require multiple API calls
using memoize.
- Handles cases were a loader function uses class attributes as input and
output and thus no relevant method parameters or return. An important
example is StagingAPI._generate_ring_packages().
- Storage is project aware which allows caches to be deleted when a project
is known to have changed.
- Due to project awareness, can utilize OBS /statistics/latest_updated API
call to determine which projects need to be expired.
The cache file structure is as follows:
- hostname(apiurl)
- project
- sha1(url)
- sha1(url)
See Cache.PATTERNS for changing the time to live (ttl) or add patterns to
be cached.
In the process of porting many improvements and bug fixes were added. Usage
follows the standard ReviewBot format and allows for new execution styles,
but the following examples replicate previous usage.
# Review all requests assigned to factory-auto.
./check_source.py --group factory-auto review
# Review request 13370.
./check_source.py --group factory-auto id 13370
# --project translates to project command.
# No longer automatically includes $project:NonFree.
./check_source.py --group factory-auto project openSUSE:Factory
New options available are:
--ignore-devel ignore devel projects for target package
--devel-whitelist=FILE file containing whitelisted projects (one per line)
--review-team=GROUP review team group added to requests with > 8 diff
--repo-checker=USER repo checker user added after accepted review
Note that --ignore-devel used to be provided as env[IGNORE_DEVEL_PROJECTS].
Some highlights about what was cleaned up:
- cryptic variable names replaced
- replaced custom osc queries with osc.core calls where applicable
- removed the need to load package information for all packages within
target project which cuts runtime in half for single review
- removed extraneous code that performed extra steps for no reason
os.path.dirname(os.path.realpath(__file__.replace('.pyc', '.py')))
os.path.dirname(os.path.realpath(__file__))
and _checker_prepare_dir() which needlessly chdir() twice.
- one logic branch failed to cleanup checkout directory
- new flags provide additional flexibility
