Commit Graph

401 Commits

Author SHA1 Message Date
Stephen Day
2b74c46795 Merge pull request #791 from stevvooe/add-double-domain-image-name
Add image name tests around hostnames
2015-07-30 14:35:27 -07:00
Stephen Day
ad1b7d8f35 Merge pull request #740 from stevvooe/disable-storage-redirects
Allow disabling of storage driver redirects
2015-07-30 14:28:31 -07:00
Stephen J Day
bb5466245c Add image name tests around hostnames
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-30 14:26:21 -07:00
Stephen Day
9038e48c3b Merge pull request #609 from RichardScothern/single-log-line
Log a single line when a response completes.
2015-07-29 17:52:13 -07:00
Richard Scothern
54f7e5d54b When a request completes ensure only one log line is printed which
includes the http response.

When debugging non-successful registry requests this will place
the error details and http status fields in the same log line
giving easier visibility to what error occured in the request.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-07-29 16:52:47 -07:00
Aaron Lehmann
70d9a9d543 Automatically generate a HTTP secret if none is provided
Log a warning if the registry generates its own secret.

Update configuration doc, and remove the default secret from the
development config file.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-29 14:28:56 -07:00
Aaron Lehmann
73702a8707 Fix for api_test.go
This passed in the #744 before merge, but apparently the test changed
since the PR was created in ways that led to a new failures.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-28 10:59:11 -07:00
Derek McGowan
0355c3026c Merge pull request #744 from aaronlehmann/manifest-put-response-code
Manifest PUT should return 201 Created
2015-07-28 10:42:54 -07:00
Stephen J Day
9f9a7f230b Allow disabling of starage driver redirects
Storage drivers can implement a method called URLFor which can return a direct
url for a given path. The functionality allows the registry to direct clients
to download content directly from the backend storage. This is commonly used
with s3 and cloudfront. Under certain conditions, such as when the registry is
not local to the backend, these redirects can hurt performance and waste
incoming bandwidth on pulls. This feature addition allows one to disable this
feature, if required.

Signed-off-by: Stephen J Day <stephen.day@docker.com>

Conflicts:
	configuration/configuration.go
	registry/handlers/app.go
	registry/storage/catalog_test.go
	registry/storage/manifeststore_test.go
	registry/storage/registry.go
2015-07-24 16:59:35 -07:00
Aaron Lehmann
be404d7557 Make the registry client more tolerant about HTTP status codes
Generally, all 2xx and 3xx codes should be treated as success.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-24 16:14:04 -07:00
Stephen Day
060465882b Merge pull request #736 from stevvooe/authorization-interface-cleanup
Authorization interface cleanup
2015-07-24 15:39:49 -07:00
Stephen Day
b49d77a42f Merge pull request #739 from stevvooe/etags-must-be-quoted
Etags must be quoted according to http spec
2015-07-24 15:08:27 -07:00
Stephen J Day
338e645f20 Etags must be quoted according to http spec
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-24 13:07:38 -07:00
Stephen Day
9e9e9a2859 Merge pull request #677 from RichardScothern/soft-delete-remove-links
Manifest and layer soft deletion
2015-07-24 13:02:36 -07:00
Aaron Lehmann
cf32056218 Manifest PUT should return 201 Created
Change handler, update descriptors table, regenerate API spec, and
update test.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-24 12:58:16 -07:00
Richard
9c1dd69439 Manifest and layer soft deletion.
Implement the delete API by implementing soft delete for layers
and blobs by removing link files and updating the blob descriptor
cache.  Deletion is configurable - if it is disabled API calls
will return an unsupported error.

We invalidate the blob descriptor cache by changing the linkedBlobStore's
blobStatter to a blobDescriptorService and naming it blobAccessController.

Delete() is added throughout the relevant API to support this functionality.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-07-24 09:57:20 -07:00
Stephen J Day
0b89cdfcd4 Do not replace logger when adding hooks
Because the logger was incorrectly replaced while adding hooks, log output did
not include the version and instance ids. The main issue was the the
logrus.Entry was replaced with the logger, which included no context. Replacing
the logger on the context is not necessary when configuring hooks since we are
configuring the contexts logger directly.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-23 20:51:11 -07:00
Stephen J Day
d31f9fd5b1 auth.AccessController interface now uses distribution/context
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-23 19:48:47 -07:00
Stephen J Day
a0fdfb9d4d Simplify auth.Challenge interface to SetHeaders
This removes the erroneous http.Handler interface in favor a simple SetHeaders
method that only operattes on the response. Several unnecessary uses of pointer
types were also fixed up.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-23 19:47:57 -07:00
Stephen Day
984037f7fc Merge pull request #729 from stevvooe/pagination-spec-cleanup
Clean up pagination specification
2015-07-23 15:46:32 -07:00
Doug Davis
307f34eeef Remove dead code
thanks to @tiborvass for noticing

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-07-23 07:09:48 -07:00
Stephen Day
34e5b1849c Merge pull request #493 from nevermosby/storage-support-openstack-swift
Storage support openstack swift
2015-07-22 23:53:36 -07:00
Stephen J Day
88831b0523 Clean up pagination specification
Some missing descriptions and error code for tags pagination was cleaned up to
ensure clarity. Specifically, we ensure the request variations are named and
the proper error codes are included.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-22 20:00:28 -07:00
Stephen Day
76f29c2630 Merge pull request #653 from pdevine/catalog-api
Catalog for V2 API Implementation
2015-07-22 18:54:48 -07:00
Aaron Lehmann
4703e9980c Merge pull request #724 from aaronlehmann/repository-name-regexp-tests
Add additional test coverage for the regexp contained in RepositoryNameRegexp
2015-07-22 18:41:44 -07:00
Aaron Lehmann
0d27f70d0c Unify the testcases for the two tests in names_test.go
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-22 18:33:15 -07:00
Patrick Devine
14749fdce4 Add Registry to client bindings for Repositories
The way Repositories() was initially called was somewhat different than
other parts of the client bindings because there was no way to instantiate a
Namespace.  This change implements a NewRegistry() function which changes
it so that Repositories() can be called the way one would expect.

It doesn't implement any of the other functions of Namespaces.

Signed-off-by: Patrick Devine <patrick.devine@docker.com>
2015-07-22 17:48:15 -07:00
Doug Davis
f5283cc5a2 Make Error.Error() return the post-arg-substitution Message
Missed this during the removal of the args property

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-07-22 12:44:59 -07:00
Stephen Day
e04ea232b5 Merge pull request #709 from hopkings2008/master
Close reader after the test is finished.
2015-07-22 12:22:22 -07:00
Patrick Devine
b7e26bac74 Create Repositories method
This change removes the Catalog Service and replaces it with a more
simplistic Repositories() method for obtaining a catalog of all
repositories.  The Repositories method takes a pre-allocated slice
and fills it up to the size of the slice and returns the amount
filled.  The catalog is returned lexicographically and will start
being filled from the last entry passed to Repositories().  If there
are no more entries to fill, io.EOF will be returned.

Signed-off-by: Patrick Devine <patrick.devine@docker.com>

Conflicts:
	registry/client/repository.go
	registry/handlers/api_test.go
2015-07-21 21:45:14 -07:00
Patrick Devine
74563efe98 Catalog for V2 API Implementation
This change adds a basic catalog endpoint to the API, which returns a list,
or partial list, of all of the repositories contained in the registry.  Calls
to this endpoint are somewhat expensive, as every call requires walking a
large part of the registry.

Instead, to maintain a list of repositories, you would first call the catalog
endpoint to get an initial list, and then use the events API to maintain
any future repositories.

Signed-off-by: Patrick Devine <patrick.devine@docker.com>
2015-07-21 21:45:14 -07:00
Stephen J Day
006214d902 Paginate catalog and tag results with Link header
Move the specification to use a Link header, rather than a "next" entry in the
json results. This prevents requiring clients from parsing the request body to
issue the next request. It also ensures that the returned response body does
not change in between requests.

The ordering of the specification has been slightly tweaked, as well. Listing
image tags has been moved after the catalog specification. Tag pagination now
heavily references catalog pagination.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-21 21:45:14 -07:00
Stephen J Day
aebe850f73 Catalog V2 API specification proposal
This contains a proposal for a catalog API, provided access to the internal
contents of a registry instance. The API endpoint is prefixed with an
underscore, which is illegal in images names, to prevent collisions with
repositories names. To avoid issues with large result sets, a paginated version
of the API is proposed. We make an addition to the tags API to support
pagination to ensure the specification is conistent.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-21 21:45:14 -07:00
Aaron Lehmann
9423b38d5f Add additional test coverage for the regexp contained in RepositoryNameRegexp
This was inspired by problems found with new regexps proposed in PR #690

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-21 17:10:36 -07:00
davidli
d1b4bae078 Remove IPC support from test file
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:10 +02:00
Sylvain Baubeau
cca15a76f1 Catch either missing containers or objects
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:10 +02:00
Sylvain Baubeau
b4cf6c053b Do not use Swift server side copy for manifests to handle >5G files
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:10 +02:00
Sylvain Baubeau
326c3a9c49 Inline Swift errors handling
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:10 +02:00
Sylvain Baubeau
15d567671b Retrieve all the objects using pagination
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
49582a6188 Do not create objects for directories
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
ed08d8d4e0 Refactor segment path concatenation code
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
91e17e10cd Use http.StatusRequestedRangeNotSatisfiable instead of error code
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
f4ca709d70 Use file instead of filepath as it may cause troubles on Windows
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
78d722e708 Protect against deletion of objects with the same prefix
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
61e3bce794 Show distribution version in User-Agent
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
f256797cac Do not use suite style testing for Swift specific tests
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
53e80944f1 Rename environment variables to run Swift testsuite
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
d0dd36125c Check file has been opened before closing it
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
1a80ec340b Rename DriverParameters structure to Parameters
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
2846913552 Change folder mime type to application/vnc.swift.directory
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
33028c11a7 Add code documentation
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
f21cbcebc6 Do not read segment if no padding is necessary
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
3242378e1b Improve 404 errors handling
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
57cef57e1b Use only one Swift container for both files and manifests
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
a40502ec02 Increase default chunk size to 20M
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
davidli
af99dbd6bf Add support for Openstack Identity v3 API
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
3fb42a1502 Use 'prefix' parameter instead of 'path' when listing files
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
1c01ca4b39 Create full folder hierarchy instead of just the top level folder
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
c4d845cb6c Remove one level of indentation in swift path handling code
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
4497d6c973 Use mitchellh/mapstructure library to parse Swift parameters
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
2632fd9dc8 Move Dynamic Large Object handling to dedicated methods
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
d8d50fb0e0 Simplify code that handles non existing manifests
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
5e650a820a Handle error during copy of original content
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:09 +02:00
nevermosby
7372c13f29 Update the import path for swift driver test
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
davidli
6dc1596be4 Use gofmt to format the code of swift driver.
Signed-off-by: Li Wenquan <wenquan.li@hp.com>
2015-07-21 23:55:09 +02:00
Sylvain Baubeau
ea7c53df08 Add Openstack Swift storage driver
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>
2015-07-21 23:55:08 +02:00
Stephen Day
5e9d0702a1 Merge pull request #713 from stevvooe/descriptor-size-field
Use "Size" field to describe blobs over "Length"
2015-07-17 22:06:59 -07:00
Stephen J Day
26b7fe4a91 Use "Size" field to describe blobs over "Length"
After consideration, we've changed the main descriptor field name to for number
of bytes to "size" to match convention. While this may be a subjective
argument, commonly we refer to files by their "size" rather than their
"length". This will match other conventions, like `(FileInfo).Size()` and
methods on `io.SizeReaderAt`. Under more broad analysis, this argument doesn't
necessarily hold up. If anything, "size" is shorter than "length".

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-17 17:07:11 -07:00
yuzou
0fc7babf76 Close reader after the test is finished.
Signed-off-by: yuzou <zouyu7@huawei.com>
2015-07-17 14:55:31 +08:00
Stephen Day
fed58bd2d3 Merge pull request #705 from stevvooe/export-servejson-errors
Export ServeJSON for serving error codes
2015-07-16 14:35:27 -07:00
Brian Bland
c25f4af00f Reduces log level of auth error lines from error->warn
An error level log is already produced within app.authorized() if an
actual unexpected error occurs during authorization, so this warning
level log remains for auditability purposes, but should not be
considered an error condition.

Addresses #704

Signed-off-by: Brian Bland <brian.bland@docker.com>
2015-07-16 12:43:33 -07:00
Stephen J Day
86ffe515df Export ServeJSON for serving error codes
This changeset provides a common http handler for serving errcodes. This should
unify http responses across webservices in the face of errors.

Several type assertions have been added, as well, to ensure the error interface
is implemented.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-16 12:14:14 -07:00
Richard
f331da2daa Allow Manifest Service to be configured with function arguments
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-07-15 12:25:16 -07:00
Stephen Day
2b88cb9413 Merge pull request #651 from duglin/ErrArgs
Add ability to pass in substitution args into an Error
2015-07-15 11:22:59 -07:00
Doug Davis
94e2e9f4a0 Add ability to pass in substitution args into an Error
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-07-15 11:02:10 -07:00
Derek McGowan
0cda2f61e8 Merge pull request #699 from RichardScothern/client-manifest-etags-clean
Allow conditional fetching of manifests with the registry client.
2015-07-14 17:29:59 -07:00
Richard Scothern
48a2010ac3 Allow conditional fetching of manifests with the registry client.
Add a functional argument to pass a digest to (ManifestService).GetByTag().
If the digest matches an empty manifest and nil error are returned.

See 1bc740b0d5 for server implementation.

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-07-14 16:52:52 -07:00
Richard Scothern
7e6fda610d Fix build when using build tag 'noresumabledigest'
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-07-14 11:14:09 -07:00
Stephen J Day
b54cc5ed47 Allow single character repository names
The main goal of this changeset is to allow repository name components to
consist of a single character. The number of components allowed and the slash
separation requirements have also been clarified.

To go along with this simplification, errant constants and unneeded error types
have been removed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-07-10 12:10:42 -06:00
Derek McGowan
8fc782ae09 Fix typo in Version doc
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-07-08 11:02:47 -07:00
Derek McGowan
3531b22b46 Add challenge manager interface
Challenger manager interface is used to handle getting authorization challenges from an endpoint as well as extracting challenges from responses.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-07-01 15:00:25 -07:00
Derek McGowan
c8fac94617 Separate version and challenge parsing from ping
Replace ping logic with individual functions to extract API version and authorization challenges. The response from a ping operation can be passed into these function. If an error occurs in parsing, the version or challenge will not be used. Sending the ping request is the responsibility of the caller.
APIVersion has been converted from a string to a structure type. A parse function was added to convert from string to the structure type.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-07-01 15:00:25 -07:00
Derek McGowan
5a7dab4670 Refactor client auth
Move client auth into a separate package.
Separate ping from the authorizer and export Challenges type.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-07-01 15:00:25 -07:00
Stephen Day
c56e28826e Merge pull request #670 from stevvooe/remove-ipc
Remove half-baked Storage Driver IPC support
2015-06-30 19:21:54 -07:00
Jon Poler
f09051fe54 Remove timeout for http.Client in registry/client/repository.go.
Timeouts should not be a discrete period of time, because they end
up being arbitrary and may be difficult to gauge correctly against
very large Docker layers. Rather, timeouts should be set at the
transport level using the SetDeadline attribute on a net.Conn
object.

Signed-off-by: Jon Poler <jonathan.poler@apcera.com>
2015-06-30 17:35:24 -07:00
Stephen Day
c1423aa2aa Merge pull request #675 from RichardScothern/fix-ctx
Pass correct context into tracer
2015-06-30 11:18:51 -07:00
Richard Scothern
3629aac84f Pass correct context into tracer
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-06-30 10:28:14 -07:00
Stephen J Day
d3d4423ff7 Remove half-baked Storage Driver IPC support
This removes documentation and code related to IPC based storage driver
plugins. The existence of this functionality was an original feature goal but
is now not maintained and actively confusing incoming contributions. We will
likely explore some driver plugin mechanism in the future but we don't need
this laying around in the meantime.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-29 16:52:12 -07:00
Richard Scothern
0edb91185b Prevent the ErrUnsupportedMethod error from being returned up the stack.
It eventually causes the go http library to do a double WriteHeader()
which is an error

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-06-29 16:44:09 -07:00
Stephen Day
b6faf3fa9f Merge pull request #645 from RichardScothern/manifest-cache-headers
Cache headers for manifests.
2015-06-25 13:20:39 -07:00
Richard Scothern
1bc740b0d5 Add Etag header for manifests.
Return 304 (Not Modified) if retrieved with If-None-Match header

Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
2015-06-19 10:44:21 -07:00
Doug Davis
7bb98181ab Add 'message' back to BlobTest sample json
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-06-18 18:24:54 -07:00
Doug Davis
cb45ec56ff Add back in the "errors" wrapper in the Errors serialization
See: https://github.com/docker/distribution/pull/548/files#r32794064

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-06-18 18:00:26 -07:00
Stephen Day
d2ca423500 Merge pull request #638 from duglin/MoveHTTPRC
Move challenge http status code logic
2015-06-18 12:03:29 -07:00
Doug Davis
c4eb195cc1 Move challenge http status code logic
See: 3ea67df373/registry/handlers/app.go (L498)

Per the comment on line 498, this moves the logic of setting the http
status code into the serveJSON func, leaving the auth.Challenge.ServeHTTP()
func to just set the auth challenge header.

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-06-17 18:23:55 -07:00
Stephen J Day
b65a4a8713 Ensure that rados is disabled without build tag
This ensures that rados is not required when building the registry. This was
slightly tricky in that when the flags were applied, the rados package was
completely missing. This led to a problem where rados was basically unlistable
and untestable as a package. This was fixed by simply adding a doc.go file that
is included whether rados is built or not.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-16 20:00:02 -07:00
Stephen Day
3ea67df373 Merge pull request #623 from ahmetalpbalkan/azure-vendor
storage/driver/azure: Update vendored Azure SDK
2015-06-16 17:41:38 -07:00
Ahmet Alp Balkan
daa22cacba storage/driver/azure: Update vendored Azure SDK
This change refreshes the updated version of Azure SDK
for Go that has the latest changes.

I manually vendored the new SDK (github.com/Azure/azure-sdk-for-go)
and I removed `management/` `core/` packages manually simply because
they're not used here and they have a fork of `net/http` and `crypto/tls`
for a particular reason. It was introducing a 44k SLOC change otherwise...

This also undoes the `include_azure` flag (actually Steven removed the
driver from imports but forgot to add the build flag apparently, so the
flag wasn't really including azure. 😄 ). This also must be obsolete
now.

Fixes #620, #175.

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2015-06-16 17:13:44 -07:00
Olivier Gambier
d678e3dc57 Merge pull request #608 from dmcgowan/http-basic-auth
Implementation of a basic authentication scheme using standard .htpasswd
2015-06-15 17:40:58 -07:00
Stephen Day
e57e731821 Merge pull request #548 from duglin/MoveErrors
Move ErrorCode logic to new errcode package
2015-06-15 14:33:28 -07:00
Doug Davis
441f7cac87 Round 4
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-06-11 21:33:35 -07:00
Stephen J Day
d9d55bcbab Minor formatting fixes related to htpasswd auth
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-11 17:06:35 -07:00
Olivier Gambier
6e0cfc17dc Saner default data location
Signed-off-by: Olivier Gambier <olivier@docker.com>
2015-06-10 20:54:24 -07:00
Stephen J Day
0f654c25ac Rename the basic access controller to htpasswd
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-10 19:41:35 -07:00
Stephen J Day
ffd3662982 Harden basic auth implementation
After consideration, the basic authentication implementation has been
simplified to only support bcrypt entries in an htpasswd file. This greatly
increases the security of the implementation by reducing the possibility of
timing attacks and other problems trying to detect the password hash type.

Also, the htpasswd file is only parsed at startup, ensuring that the file can
be edited and not effect ongoing requests. Newly added passwords take effect on
restart. Subsequently, password hash entries are now stored in a map.

Test cases have been modified accordingly.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-10 19:38:56 -07:00
Stephen J Day
ffe56ebe41 Refactor Basic Authentication package
This change refactors the basic authentication implementation to better follow
Go coding standards. Many types are no longer exported. The parser is now a
separate function from the authentication code. The standard functions
(*http.Request).BasicAuth/SetBasicAuth are now used where appropriate.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-10 19:06:38 -07:00
Dave Trombley
abd142855a Unexported function to comply with golint
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:38 -07:00
Dave Trombley
e4c3ab4377 Removed dashes from comments, unexported htpasswd struct
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:38 -07:00
Dave Trombley
04f6a4811d Fixed golint, gofmt warning advice.
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:38 -07:00
Dave Trombley
c50dfb7dae Added support for bcrypt, plaintext; extension points for other htpasswd hash methods.
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:38 -07:00
Dave Trombley
c4849bb99a Aligned formatting with gofmt
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:38 -07:00
Dave Trombley
0ecaa7f40a Fixed WWW-Authenticate: header, added example config and import into main, fixed golint warnings
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:38 -07:00
BadZen
8a204f59e7 Implementation of a basic authentication scheme using standard .htpasswd files
Signed-off-by: BadZen <dave.trombley@gmail.com>
Signed-off-by: Dave Trombley <dave.trombley@gmail.com>
2015-06-10 19:06:37 -07:00
Stephen Day
f63313de1f Merge pull request #599 from stevvooe/clarify-deletion-by-digest-constraint
Clarify digest in API specification
2015-06-08 19:04:53 -07:00
Stephen J Day
c54adb667f Clarify that manifests can only be deleted by digest
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-04 18:59:08 -07:00
Stephen J Day
4bff4910b5 Add more repository name validation test cases
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-06-04 16:12:35 -07:00
Doug Davis
38393b63b7 Round 3 - Add Register function
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-06-02 10:01:21 -07:00
Vincent Giersch
7b3f571555 doc: coherence between requests and parameters + typo
In the request parameters lists `tag` was used instead of
`reference` present in the HTTP requests paths

Signed-off-by: Vincent Giersch <vincent.giersch@ovh.net>
2015-06-02 14:36:55 +00:00
Stephen Day
0d40913b9a Merge pull request #394 from xiekeyang/feature-panic-hook
Feature: Add Hook for Web Application Panic
2015-06-01 13:23:32 -07:00
Vincent Giersch
3c9a2a9bac Fix rados build, remove uuid dependency
Signed-off-by: Vincent Giersch <vincent.giersch@ovh.net>
2015-06-01 09:57:40 +00:00
xiekeyang
47aa47e3f6 Feature: Web Panic Reporting via hooks
This PR is for issue of "email after registry webapp panic" #41, improving my
previous design (closed).
It use self setting up hooks, to catch panic in web application.
And, send email in hooks handle directly, to no use new http server and
handler.

Signed-off-by: xiekeyang <keyangxie@126.com>
2015-05-31 14:21:22 +00:00
Philip Misiowiec
1f31f595d9 fixed typos
Signed-off-by: Phil Misiowiec <phil.misiowiec@atlashealth.com>
2015-05-30 18:22:41 -07:00
Stephen Day
318af0b1ce Merge pull request #443 from gierschv/driver-rados
Storage Driver: Ceph Object Storage (RADOS)
2015-05-29 18:10:49 -07:00
Doug Davis
8a0827f799 Round 2
Make Errors a []Error

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-05-26 17:18:32 -07:00
Doug Davis
0a6a6f5b81 Move ErrorCode logic to new errcode package
Make HTTP status codes match the ErrorCode by looking it up in the Descriptors

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-05-26 13:18:54 -07:00
Stephen Day
a3006e8e45 Merge pull request #556 from stevvooe/remove-uuid-dependency
Replace uuid dependency with internal library
2015-05-26 10:39:35 -07:00
Stephen J Day
36e34a55ad Replace uuid dependency with internal library
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-22 18:59:41 -07:00
Stephen Day
8ce4dcaef8 Merge pull request #546 from stevvooe/resumable-digest-refactor
Remove digest package's dependency on external sha implementation
2015-05-22 18:15:37 -07:00
Stephen J Day
44da954565 Refactor specification of supported digests
To make the definition of supported digests more clear, we have refactored the
digest package to have a special Algorithm type. This represents the digest's
prefix and we associated various supported hash implementations through
function calls.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-22 18:08:23 -07:00
Stephen J Day
ff52d7f53a Better error message when failing to get AWS auth
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-22 16:45:45 -07:00
Stephen J Day
eee6cad2cf Remove digest package's dependency on external sha implementation
The change relies on a refactor of the upstream resumable sha256/sha512 package
that opts to register implementations with the standard library. This allows
the resumable support to be detected where it matters, avoiding unnecessary and
complex code. It also ensures that consumers of the digest package don't need
to depend on the forked sha implementations.

We also get an optimization with this change. If the size of data written to a
digester is the same as the file size, we check to see if the digest has been
verified. This works if the blob is written and committed in a single request.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-20 23:44:08 -07:00
Stephen J Day
87ff947038 Decouple redis dependency from blob descriptor cache
Ensure that clients can use the blob descriptor cache provider without needing
the redis package.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-05-20 18:58:10 -07:00
Tibor Vass
cb99c076db client: fix a typo preventing compilation
Signed-off-by: Tibor Vass <teabee89@gmail.com>
2015-05-20 21:06:13 -04:00
Stephen Day
ea54a5560a Merge pull request #387 from dmcgowan/registry-client-update
Add client implementation of distribution interface
2015-05-20 15:55:20 -07:00
Derek McGowan
68c1ceac95 Remove error message shortening
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-20 14:55:59 -07:00
Derek McGowan
49369ffe9a Only do auth checks for endpoints starting with v2
Changes behavior so ping doesn't happen if /v2/ is anywhere in a request path, but instead only at the beginning. This fixes attempts to ping on redirected URLs.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-20 13:35:23 -07:00
Derek McGowan
c7f7747368 Update transport package to sever distribution dependency
The transport package no longer requires importing distribution for the ReadSeekCloser, instead declares its own.
Added comments on the Authenication handler in session.
Added todo on http seek reader to highlight its lack of belonging to the client transport.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-20 10:09:37 -07:00
Derek McGowan
572ff64d21 Add unauthorized error check
Add check for unauthorized error code and explicitly set the error code if the content could not be parsed.
Updated repository test for unauthorized tests and nit feedback.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-20 10:05:44 -07:00
Derek McGowan
13894e8736 Break down type dependencies
Each type no longer requires holding a reference to repository.
Added implementation for signatures get.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-19 19:56:27 -07:00
Derek McGowan
a3276fcc5b Feedback update
Update comments and TODOs
Fix switch style
Updated parse http response to take in reader
Add Cancel implementation
Update blobstore variable name

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-19 19:18:30 -07:00
Vincent Giersch
394eea0231 Storage Driver: Ceph Object Storage (RADOS)
This driver implements the storagedriver.StorageDriver interface and
uses Ceph Object Storage as storage backend.

Since RADOS is an object storage and no hierarchy notion, the
following convention is used to keep the filesystem notions stored in
this backend:

* All the objects data are stored with opaque UUID names prefixed
  (e.g. "blob:d3d232ff-ab3a-4046-9ab7-930228d4c164).
* All the hierarchy information are stored in rados omaps, where the
  omap object identifier is the virtual directory name, the keys in
  a specific are the relative filenames and the values the blob
  object identifier (or empty value for a sub directory).

  e.g. For the following hierarchy:

     /directory1
     /directory1/object1
     /directory1/object2
     /directory1/directory2/object3

  The omap "/directory1" will contains the following key / values:
    - "object1" "blob:d3d232ff-ab3a-4046-9ab7-930228d4c164"
    - "object2" "blob:db2e359d-4af0-4bfb-ba1d-d2fd029866a0"
    - "directory2" ""

  The omap "/directory1/directory2" will contains:
    - "object3" "blob:9ae2371c-81fc-4945-80ac-8bf7f566a5d9"

* The MOVE is implemented by changing the reference to a specific
  blob in its parent virtual directory omap.

This driver stripes rados objects to a fixed size (e.g. 4M). The idea
is to keep small objects (as done by RBD on the top of RADOS) that
will be easily synchronized accross OSDs. The information of the
original object (i.e total size of the chunks) is stored as a Xattr
in the first chunk object.

Signed-off-by: Vincent Giersch <vincent.giersch@ovh.net>
2015-05-20 01:44:34 +00:00
Jordan Liggitt
32ed2d7df1 Fix typo: respository->repository
Signed-off-by: Jordan Liggitt <jliggitt@redhat.com>
2015-05-19 13:25:39 -04:00
Derek McGowan
006ddd8283 Lint and documentation fixes
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-15 17:56:28 -07:00
Derek McGowan
2874454224 Create client transport package
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-15 17:56:28 -07:00
Derek McGowan
94e375c5d1 Remove unused and duplicate error types
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-15 17:56:28 -07:00
Derek McGowan
98836d6267 Update to track refactor updates
Added use of cache blob statter

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-15 17:56:28 -07:00
Derek McGowan
fdf7c8ff15 Open cache interface
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-15 17:56:28 -07:00
Derek McGowan
70074b2286 Rename layer files to blob
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-15 17:56:28 -07:00
Derek McGowan
9d64e461be Update to use blob interfaces
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-15 17:56:28 -07:00
Derek McGowan
fddeb1c8d5 Add missing defer on Tags
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-05-15 17:56:28 -07:00