Necessary for writing involved tests since the assumptions made by the
cache are violated and thus cause problems.
To keep the tests passing the oscrc intended to be used must be set as
environment variable so when osclib.conf.Config is use the config is not
lost. This would be made simpler without the needless extra config added
by StagingWorkflow.
It seems osc.core and similar generate both quoted and unquoted URLs that
are cached separately. Additionally, urlopen() handles quoted paths
differently from os.path.*() methods which can create issues for quoted
projects.
The two slowest staging API calls are for information that rarely changes.
By caching the result the commands typically execute over twice as fast.
Going further can see improvements of an order of magnitude or more by
caching almost all the GET requests.
In contrast to osclib/memoize.py this cache operates at the HTTP request
level. This has several advantages:
- Caches the expensive part (ie the HTTP request). There are a number of
functions in osc.core and elsewhere that make the same API request, but
process the result differently which would require multiple API calls
using memoize.
- Handles cases were a loader function uses class attributes as input and
output and thus no relevant method parameters or return. An important
example is StagingAPI._generate_ring_packages().
- Storage is project aware which allows caches to be deleted when a project
is known to have changed.
- Due to project awareness, can utilize OBS /statistics/latest_updated API
call to determine which projects need to be expired.
The cache file structure is as follows:
- hostname(apiurl)
- project
- sha1(url)
- sha1(url)
See Cache.PATTERNS for changing the time to live (ttl) or add patterns to
be cached.
