To support clustered registry, upload UUIDs must be recognizable by
registries that did not issue the UUID. By creating an HMAC verifiable
upload state token, registries can validate upload requests that other
instances authorized. The tokenProvider interface could also use a redis
store or other system for token handling in the future.
During client implementation, it was found that requiring the size argument
made client implementation more complex. The original benefit of the size
argument was to provide an additional check alongside of tarsum to validate
incoming data. For the purposes of the registry, it has been determined that
tarsum should be enough to validate incoming content.
At this time, the size check is optional but we may consider removing it
completely.
Routes and errors are now all referenced from a single v2 package. This
packages exports are acceptable for use in the server side as well as
integration into docker core.
This commit locks down the set of http error codes that will be part of the
inital V2 specification, proposed in docker/docker#9015. The naming order has
been slightly changed and there are few tweaks to ensure all conditions are
captured but this will be set the docker core will be impleemnted against.
To support this, the errors have been moved into an api/errors package. A new
type, ErrorDescriptor, has been defined to centralize the code, message and
definitions used with each type. The information therein can be used to
generate documentation and response code mappings (yet to come...).
In addition to the refactoring that came along with this change, several tests
have been added to ensure serialization round trips are reliable. This allows
better support for using these error types on the client and server side. This
is coupled with some tweaks in the client code to fix issues with error
reporting.
Other fixes in the client include moving client-specific errors out of the base
package and ensuring that we have correct parameters for finishing uploads.
Push, pull and delete of manifest files in the registry have been implemented
on top of the storage services. Basic workflows, including reporting of missing
manifests are tested, including various proposed response codes. Common testing
functionality has been collected into shared methods. A test suite may be
emerging but it might better to capture more edge cases (such as resumable
upload, range requests, etc.) before we commit to a full approach.
To support clearer test cases and simpler handler methods, an application aware
urlBuilder has been added. We may want to export the functionality for use in
the client, which could allow us to abstract away from gorilla/mux.
A few error codes have been added to fill in error conditions missing from the
proposal. Some use cases have identified some problems with the approach to
error reporting that requires more work to reconcile. To resolve this, the
mapping of Go errors into error types needs to pulled out of the handlers and
into the application. We also need to move to type-based errors, with rich
information, rather than value-based errors. ErrorHandlers will probably
replace the http.Handlers to make this work correctly.
Unrelated to the above, the "length" parameter has been migrated to "size" for
completing layer uploads. This change should have gone out before but these
diffs ending up being coupled with the parameter name change due to updates to
the layer unit tests.
The http API has its first set of endpoints to implement the core aspects of
fetching and uploading layers. Uploads can be started and completed in a single
chunk and the content can be fetched via tarsum. Most proposed error conditions
should be represented but edge cases likely remain.
In this version, note that the layers are still called layers, even though the
routes are pointing to blobs. This will change with backend refactoring over
the next few weeks.
The unit tests are a bit of a shamble but these need to be carefully written
along with the core specification process. As the the client-server interaction
solidifies, we can port this into a verification suite for registry providers.
To bring the implementation inline with the specification, the names and
structure of the API routes have been updated.
The overloaded term "image" has been replaced with the term "manifest", which
may also be known as "image manifest". The desire for the layer storage to be
more of a general blob storage is reflected in moving from "layer" api prefixes
to "blob". The "tarsum" path parameter has been replaced by a more general
"digest" parameter and is no longer required to start uploads. Another set of
changes will come along to support this change at the storage service layer.
This changeset defines the application structure to be used for the http side
of the new registry. The main components are the App and Context structs. The
App context is instance global and manages global configuration and resources.
Context contains request-specific resources that may be created as a by-product
of an in-flight request.
To latently construct per-request handlers and leverage gorilla/mux, a dispatch
structure has been propped up next to the main handler flow. Without this, a
router and all handlers need to be constructed on every request. By
constructing handlers on each request, we ensure thread isolation and can
carefully control the security context of in-flight requests. There are unit
tests covering this functionality.
