The two slowest staging API calls are for information that rarely changes.
By caching the result the commands typically execute over twice as fast.
Going further can see improvements of an order of magnitude or more by
caching almost all the GET requests.
In contrast to osclib/memoize.py this cache operates at the HTTP request
level. This has several advantages:
- Caches the expensive part (ie the HTTP request). There are a number of
functions in osc.core and elsewhere that make the same API request, but
process the result differently which would require multiple API calls
using memoize.
- Handles cases were a loader function uses class attributes as input and
output and thus no relevant method parameters or return. An important
example is StagingAPI._generate_ring_packages().
- Storage is project aware which allows caches to be deleted when a project
is known to have changed.
- Due to project awareness, can utilize OBS /statistics/latest_updated API
call to determine which projects need to be expired.
The cache file structure is as follows:
- hostname(apiurl)
- project
- sha1(url)
- sha1(url)
See Cache.PATTERNS for changing the time to live (ttl) or add patterns to
be cached.
Move OBS simulation to separate class, should be eventually moved to separate
file as well. And converted test_dispatch_open_requests to use new and shiny
testing infrastructure.
To make testing more reliable and nice and generally better, we need to address
few issues. First of all, currently it is hard to figure out what was test
trying to access if accessing it fails. Second issue was that we mixed both
tests and OBS simulation logic. In perfect world, we should write some state
machine simulating OBS in some limited way and checking that scripts behaviour
corresponds to it.
This commit attempts to solve it at one example - review handling test.
So far we have a mess everywhere, but we are mainly using/working on
osc-staging.py file, so letting test suite use it by symlinking it for now.
General idea is once we cleanup everything, we get rid of all the mess around
and move this file to the correct location. Also moving some directories around
to clean up stuff even more.
