From 2fb5c97a97b0017fe5a85e97eda4df413c383f5f Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Fri, 22 May 2015 02:14:55 -0700 Subject: [PATCH 01/17] Deployment rewrote Strong focus on simplification. Entirely removed custom build instructions. Providing sane, one-liner defaults. Verified, easy-to-use TLS instructions. Removed hybrid instructions. Removed authentication with nginx instructions (either wait for native support, or move it to advanced topics). Signed-off-by: Olivier Gambier --- docs/deploying.md | 619 +++++++--------------------------------------- 1 file changed, 95 insertions(+), 524 deletions(-) diff --git a/docs/deploying.md b/docs/deploying.md index 835075c9..35ce4cb3 100644 --- a/docs/deploying.md +++ b/docs/deploying.md @@ -7,572 +7,143 @@ IGNORES--> # Deploying a registry server -This section explains how to deploy a Docker Registry either privately -for your own company or publicly for other users. For example, your company may -require a private registry to support your continuous integration (CI) system as -it builds new releases or test servers. Alternatively, your company may have a -large number of products or services with images you wish to serve in a branded -manner. +You obviously need to [install Docker](https://docs.docker.com/installation/) (remember you need at Docker version 1.6.0 or newer). -Docker's public registry maintains a default `registry` image to assist you in the -deployment process. This registry image is sufficient for running local tests -but is insufficient for production. For production you should configure and -build your own custom registry image from the `docker/distribution` code. +## Getting started in 2 lines ->**Note**: The examples on this page were written and tested using Ubuntu 14.04. ->If you are running Docker in a different OS, you may need to "translate" ->the commands to meet the requirements of your own environment. +Create a folder for your registry data: + + $ mkdir registry-data + +Start your registry: + + $ docker run -d -p 5000:5000 -v `pwd`/registry-data:/tmp/registry-dev --restart=always --name registry registry:2 + +That's it. + +You can now tag an image and push it: + + $ docker tag ubuntu localhost:5000/batman/ubuntu + $ docker push localhost:5000/batman/ubuntu + +Then pull it: + + $ docker pull localhost:5000/batman/ubuntu -## Simple example with the official image +## Making your Registry available -In this section, you create a container running Docker's official registry -image. You push an image to, and then pull the same image from, this registry. -This a good exercise for understanding the basic interactions a client has with -a local registry. +Now that your registry works on localhost, you probably want to make it available as well to other hosts. -1. Install Docker. +Let assume your registry is accessible via the domain name `myregistrydomain.com` (still on port `5000`). -2. Run the `hello-world` image from the Docker public registry. +If you try to `docker pull myregistrydomain.com:5000/batman/ubuntu`, you will see the following error message: - $ docker run hello-world - - The `run` command automatically pulls a `hello-world` image from Docker's - official images. - -3. Start a registry on your localhost. - - $ docker run -p 5000:5000 registry:2.0 - - This starts a registry on your `DOCKER_HOST` running on port `5000`. - -3. List your images. - - $ docker images - REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE - registry 2.0 bbf0b6ffe923 3 days ago 545.1 MB - golang 1.4 121a93c90463 5 days ago 514.9 MB - hello-world latest e45a5af57b00 3 months ago 910 B - - Your list should include a `hello-world` image from the earlier run. - -4. Retag the `hello-world` image for your local repoistory. - - $ docker tag hello-world:latest localhost:5000/hello-mine:latest - - The command labels a `hello-world:latest` using a new tag in the - `[REGISTRYHOST/]NAME[:TAG]` format. The `REGISTRYHOST` is this case is - `localhost`. In a Mac OSX environment, you'd substitute `$(boot2docker - ip):5000` for the `localhost`. - -5. List your new image. - - $ docker images - REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE - registry 2.0 bbf0b6ffe923 3 days ago 545.1 MB - golang 1.4 121a93c90463 5 days ago 514.9 MB - hello-world latest e45a5af57b00 3 months ago 910 B - localhost:5000/hello-mine latest ef5a5gf57b01 3 months ago 910 B - - You should see your new image in your listing. - -6. Push this new image to your local registry. - - $ docker push localhost:5000/hello-mine:latest - The push refers to a repository [localhost:5000/hello-mine] (len: 1) - e45a5af57b00: Image already exists - 31cbccb51277: Image successfully pushed - 511136ea3c5a: Image already exists - Digest: sha256:a1b13bc01783882434593119198938b9b9ef2bd32a0a246f16ac99b01383ef7a - -7. Use the `curl` command and the Docker Registry API v2 to list your - image in the registry: - - $ curl -v -X GET http://localhost:5000/v2/hello-mine/tags/list - * Hostname was NOT found in DNS cache - * Trying 127.0.0.1... - * Connected to localhost (127.0.0.1) port 5000 (#0) - > GET /v2/hello-mine/tags/list HTTP/1.1 - > User-Agent: curl/7.35.0 - > Host: localhost:5000 - > Accept: */* - > - < HTTP/1.1 200 OK - < Content-Type: application/json; charset=utf-8 - < Docker-Distribution-Api-Version: registry/2.0 - < Date: Sun, 12 Apr 2015 01:29:47 GMT - < Content-Length: 40 - < - {"name":"hello-mine","tags":["latest"]} - * Connection #0 to host localhost left intact - - You can also get this information by entering the - `http://localhost:5000/v2/hello-mine/tags/list` address in your browser. - -8. Remove all the unused images from your local environment: - - $ docker rmi -f $(docker images -q -a ) - - This command is for illustrative purposes; removing the image forces any `run` - to pull from a registry rather than a local cache. If you run `docker images` - after this you should not see any instance of `hello-world` or `hello-mine` in - your images list. - - $ docker images - REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE - registry 2.0 bbf0b6ffe923 3 days ago 545.1 MB - golang 1.4 121a93c90463 5 days ago 514.9 MB - -9. Try running `hello-mine`. - - $ docker run hello-mine - Unable to find image 'hello-mine:latest' locally - Pulling repository hello-mine - FATA[0001] Error: image library/hello-mine:latest not found - - The `run` command fails because your new image doesn't exist in the Docker public - registry. - -10. Now, try running the image but specifying the image's registry: - - $ docker run localhost:5000/hello-mine - - If you run `docker images` after this you'll fine a `hello-mine` instance. - -### Making Docker's official registry image production ready - -Docker's official image is for simple tests or debugging. Its configuration is -unsuitable for most production instances. For example, any client with access to -the server's IP can push and pull images to it. See the next section for -information on making this image production ready. - -## Understand production deployment - -The Docker Registry 2.0 only accepts connections from a Docker client that is -version 1.6.0 or newer. This means, for example, if you are deploying in a -environment with 1.5.X clients you'll need to either upgrade them or deploy an -older version of the Docker Registry to support them. For information on how to -do this, see [Configure Nginx with a v1 and v2 -registry](#configure-nginx-with-a-v1-and-v2-registry) on this page. - -When deploying a registry for a production deployment you should consider these -key factors: - - - - - - - - - - - - - - - - - - -
- backend storage - - Where should you store the images? -
- access and/or authentication - - Should users have full or controlled access? This can depend on whether - you are serving images to the public or internally to your company only. -
- debugging - - When problems or issues arise, do you have the means of solving them. Logs - are useful as is reporting to see trends. -
- caching - - Quickly retrieving images can be crucial if you are relying on images for - tests, builds, or other automated systems. -
- -You can configure your registry features to adjust for these factors. You do -this by specifying options on the command line or, more typically, by writing a -registry configuration file. The configuration file is in YAML format. - -Docker's official repository image is preconfigured using the following -configuration file: - -```yaml -version: 0.1 -log: - level: debug - fields: - service: registry - environment: development -storage: - cache: - blobdescriptor: inmemory - filesystem: - rootdirectory: /tmp/registry-dev - maintenance: - uploadpurging: - enabled: false -http: - addr: :5000 - secret: asecretforlocaldevelopment - debug: - addr: localhost:5001 -redis: - addr: localhost:6379 - pool: - maxidle: 16 - maxactive: 64 - idletimeout: 300s - dialtimeout: 10ms - readtimeout: 10ms - writetimeout: 10ms -notifications: - endpoints: - - name: local-8082 - url: http://localhost:5003/callback - headers: - Authorization: [Bearer ] - timeout: 1s - threshold: 10 - backoff: 1s - disabled: true - - name: local-8083 - url: http://localhost:8083/callback - timeout: 1s - threshold: 10 - backoff: 1s - disabled: true +``` +FATA[0000] Error response from daemon: v1 ping attempt failed with error: Get https://nonregistry:5000/v1/_ping: dial tcp: lookup nonregistry: no such host. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry nonregistry:5000` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/nonregistry:5000/ca.crt ``` -This configuration is very basic and you can see it would present some problems -in a production environment. For example, the `http` section details the configuration for -the HTTP server that hosts the registry. The server is not using even the most -minimal transport layer security (TLS). Let's configure that in the next section. +You basically have three different options to comply with docker security requirements here. -## Configure TLS on a registry server +### 1. buy a SSL certificate for your domain -In this section, you configure TLS on the server to enable communication through -the `https` protocol. Enabling TLS on the server is the minimum layer of -security recommended for running a registry behind a corporate firewall. One way -to do this is to build your own registry image. +This is the (highly) recommended solution. -### Download the source and generate certificates +You can buy a certificate for as cheap as 10$ a year (some registrars even offer certificates for free), and this will save you a lot of trouble. -1. [Download the registry -source](https://github.com/docker/distribution/releases/tag/v2.0.0). +Assuming you now have a `domain.crt` and `domain.key` inside a directory named `certs`: - Alternatively, use the `git clone` command if you are more comfortable with that. +``` +# Stop your registry +docker stop registry && docker rm registry -2. Unpack the the downloaded package into a local directory. +# Start your registry with TLS enabled +docker run -d -p 5000:5000 \ + -v `pwd`/registry-data:/tmp/registry-dev \ + -v `pwd`/certs:/certs \ + -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \ + -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \ + --restart=always --name registry \ + registry:2 +``` - The package creates a `distribution` directory. +**Pros:** -3. Change to the root of the new `distribution` directory. + - best solution + - work without further ado (assuming you bought your certificate from a CA that is trusted by your operating system) - $ cd distribution +**Cons:** -4. Make a `certs` subdirectory. - - $ mkdir certs - -5. Use SSL to generate some self-signed certificates. - - $ openssl req \ - -newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key \ - -x509 -days 365 -out certs/domain.crt - - This command prompts you for basic information it needs to create the certificates. - -6. List the contents of the `certs` directory. + - ? - $ ls certs - domain.crt domain.key +### 2. instruct docker to trust your registry as insecure - When you build this container, the `certs` directory and its contents - automatically get copied also. - -### Add TLS to the configuration +This basically tells Docker to entirely disregard security for your registry. -The `distribution` repo includes sample registry configurations in the `cmd` -subdirectory. In this section, you edit one of these configurations to add TLS -support. - -1. Edit the `./cmd/registry/config.yml` file. +1. edit the file `/etc/default/docker` so that there is a line that reads: `DOCKER_OPTS="--insecure-registry myregistrydomain:5000"` (or add that to existing `DOCKER_OPTS`) +2. restart your Docker daemon: on ubuntu, this is usually `service docker stop && service docker start` - $ vi ./cmd/registry/config.yml +**Pros:** -2. Locate the `http` block. + - easy to configure + +**Cons:** + + - very insecure + - you have to configure every docker daemon that wants to access your registry + +### 3. use a self-signed certificate and configure docker to trust it - http: - addr: :5000 - secret: asecretforlocaldevelopment - debug: - addr: localhost:5001 +Alternatively, you can generate your own certificate: -3. Add a `tls` block for the server's self-signed certificates: +``` +mkdir -p certs && openssl req \ + -newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key \ + -x509 -days 365 -out certs/domain.crt +``` - http: - addr: :5000 - secret: asecretforlocaldevelopment - debug: - addr: localhost:5001 - tls: - certificate: /go/src/github.com/docker/distribution/certs/domain.crt - key: /go/src/github.com/docker/distribution/certs/domain.key - - You provide the paths to the certificates in the container. If you want - two-way authentication across the layer, you can add an optional `clientcas` - section. - -4. Save and close the file. +Be sure to use the name `myregistrydomain.com` as a CN. - -### Build and run your registry image +Now go to solution 1 above and stop and restart your registry. -1. Build your registry image. +Then you have to instruct every docker daemon to trust that certificate. This is done by copying the `domain.crt` file to `/etc/docker/certs.d/myregistrydomain.com:5000/ca.crt` - $ docker build -t secure_registry . - -2. Run your new image. +**Pros:** - $ docker run -p 5000:5000 secure_registry:latest - time="2015-04-12T03:06:18.616502588Z" level=info msg="endpoint local-8082 disabled, skipping" environment=development instance.id=bf33c9dc-2564-406b-97c3-6ee69dc20ec6 service=registry - time="2015-04-12T03:06:18.617012948Z" level=info msg="endpoint local-8083 disabled, skipping" environment=development instance.id=bf33c9dc-2564-406b-97c3-6ee69dc20ec6 service=registry - time="2015-04-12T03:06:18.617190113Z" level=info msg="using inmemory blob descriptor cache" environment=development instance.id=bf33c9dc-2564-406b-97c3-6ee69dc20ec6 service=registry - time="2015-04-12T03:06:18.617349067Z" level=info msg="listening on :5000, tls" environment=development instance.id=bf33c9dc-2564-406b-97c3-6ee69dc20ec6 service=registry - time="2015-04-12T03:06:18.628589577Z" level=info msg="debug server listening localhost:5001" - 2015/04/12 03:06:28 http: TLS handshake error from 172.17.42.1:44261: remote error: unknown certificate authority - - Watch the messages at startup. You should see that `tls` is running. - -3. Use `curl` to verify that you can connect over `https`. + - more secure than solution 2 - $ curl -v https://localhost:5000 - * Rebuilt URL to: https://localhost:5000/ - * Hostname was NOT found in DNS cache - * Trying 127.0.0.1... - * Connected to localhost (127.0.0.1) port 5000 (#0) - * successfully set certificate verify locations: - * CAfile: none - CApath: /etc/ssl/certs - * SSLv3, TLS handshake, Client hello (1): - * SSLv3, TLS handshake, Server hello (2): - * SSLv3, TLS handshake, CERT (11): - * SSLv3, TLS alert, Server hello (2): - * SSL certificate problem: self signed certificate - * Closing connection 0 - curl: (60) SSL certificate problem: self signed certificate - More details here: http://curl.haxx.se/docs/sslcerts.html - -## Configure Nginx with a v1 and v2 registry +**Cons:** -This sections describes how to use `docker-compose` to run a combined version -1 and version 2.0 registry behind an `nginx` proxy. The combined registry is -accessed at `localhost:5000`. If a `docker` client has a version less than 1.6, -Nginx will route its requests to the 1.0 registry. Requests from newer clients -will route to the 2.0 registry. + - you have to configure every docker daemon that wants to access your registry -This procedure uses the same `distribution` directory you created in the last -procedure. The directory includes an example `compose` configuration. -### Install Docker Compose -1. Open a new terminal on the host with your `distribution` directory. +## Using Compose -2. Get the `docker-compose` binary. +It's highly recommended to use Docker Compose to facilitate managing your Registry configuration. - $ sudo wget https://github.com/docker/compose/releases/download/1.1.0/docker-compose-`uname -s`-`uname -m` -O /usr/local/bin/docker-compose +Here is a simple `docker-compose.yml` that does setup your registry exactly as above, with TLS enabled. - This command installs the binary in the `/usr/local/bin` directory. - -3. Add executable permissions to the binary. +``` +registry: + restart: always + image: registry:2 + ports: + - 5000:5000 + environment: + REGISTRY_HTTP_TLS_CERTIFICATE: /certs/domain.crt + REGISTRY_HTTP_TLS_KEY: /certs/domain.key + REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data + volumes: + - `pwd`/registry-data:/data + - `pwd`/certs:/certs +``` - $ sudo chmod +x /usr/local/bin/docker-compose - +You can then start your registry with a simple -### Do some housekeeping + $ docker-compose up -d -1. Remove any previous images. - $ docker rmi -f $(docker images -q -a ) - - This step is a house keeping step. It prevents you from mistakenly picking up - an old image as you work through this example. - -2. Edit the `distribution/cmd/registry/config.yml` file and remove the `tls` block. - - If you worked through the previous example, you'll have a `tls` block. - -4. Save any changes and close the file. - -### Configure SSL - -1. Change to the `distribution/contrib/compose/nginx` directory. - - This directory contains configuration files for Nginx and both registries. - -2. Use SSL to generate some self-signed certificates. - - $ openssl req \ - -newkey rsa:4096 -nodes -sha256 -keyout domain.key \ - -x509 -days 365 -out domain.crt - - This command prompts you for basic information it needs to create certificates. - -3. Edit the `Dockerfile`and add the following lines. - - COPY domain.crt /etc/nginx/domain.crt - COPY domain.key /etc/nginx/domain.key - - When you are done, the file looks like the following. - - FROM nginx:1.7 - - COPY nginx.conf /etc/nginx/nginx.conf - COPY registry.conf /etc/nginx/conf.d/registry.conf - COPY docker-registry.conf /etc/nginx/docker-registry.conf - COPY docker-registry-v2.conf /etc/nginx/docker-registry-v2.conf - COPY domain.crt /etc/nginx/domain.crt - COPY domain.key /etc/nginx/domain.key - -4. Save and close the `Dockerfile` file. - -5. Edit the `registry.conf` file and add the following configuration. - - ssl on; - ssl_certificate /etc/nginx/domain.crt; - ssl_certificate_key /etc/nginx/domain.key; - - This is an `nginx` configuration file. - -6. Save and close the `registry.conf` file. - -### Build and run - -1. Go up to the `distribution/contrib/compose` directory - - This directory includes a single `docker-compose.yml` configuration. - - nginx: - build: "nginx" - ports: - - "5000:5000" - links: - - registryv1:registryv1 - - registryv2:registryv2 - registryv1: - image: registry - ports: - - "5000" - registryv2: - build: "../../" - ports: - - "5000" - - This configuration builds a new `nginx` image as specified by the - `nginx/Dockerfile` file. The 1.0 registry comes from Docker's official public - image. Finally, the registry 2.0 image is built from the - `distribution/Dockerfile` you've used previously. - -2. Get a registry 1.0 image. - - $ docker pull registry:0.9.1 - - The Compose configuration looks for this image locally. If you don't do this - step, later steps can fail. - -3. Build `nginx`, the registry 2.0 image, and - - $ docker-compose build - registryv1 uses an image, skipping - Building registryv2... - Step 0 : FROM golang:1.4 - - ... - - Removing intermediate container 9f5f5068c3f3 - Step 4 : COPY docker-registry-v2.conf /etc/nginx/docker-registry-v2.conf - ---> 74acc70fa106 - Removing intermediate container edb84c2b40cb - Successfully built 74acc70fa106 - - The commmand outputs its progress until it completes. - -4. Start your configuration with compose. - - $ docker-compose up - Recreating compose_registryv1_1... - Recreating compose_registryv2_1... - Recreating compose_nginx_1... - Attaching to compose_registryv1_1, compose_registryv2_1, compose_nginx_1 - ... - - -5. In another terminal, display the running configuration. - - $ docker ps - CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES - a81ad2557702 compose_nginx:latest "nginx -g 'daemon of 8 minutes ago Up 8 minutes 80/tcp, 443/tcp, 0.0.0.0:5000->5000/tcp compose_nginx_1 - 0618437450dd compose_registryv2:latest "registry cmd/regist 8 minutes ago Up 8 minutes 0.0.0.0:32777->5000/tcp compose_registryv2_1 - aa82b1ed8e61 registry:latest "docker-registry" 8 minutes ago Up 8 minutes 0.0.0.0:32776->5000/tcp compose_registryv1_1 - -### Explore a bit - -1. Check for TLS on your `nginx` server. - - $ curl -v https://localhost:5000 - * Rebuilt URL to: https://localhost:5000/ - * Hostname was NOT found in DNS cache - * Trying 127.0.0.1... - * Connected to localhost (127.0.0.1) port 5000 (#0) - * successfully set certificate verify locations: - * CAfile: none - CApath: /etc/ssl/certs - * SSLv3, TLS handshake, Client hello (1): - * SSLv3, TLS handshake, Server hello (2): - * SSLv3, TLS handshake, CERT (11): - * SSLv3, TLS alert, Server hello (2): - * SSL certificate problem: self signed certificate - * Closing connection 0 - curl: (60) SSL certificate problem: self signed certificate - More details here: http://curl.haxx.se/docs/sslcerts.html - -2. Tag the `v1` registry image. - - $ docker tag registry:latest localhost:5000/registry_one:latest - -2. Push it to the localhost. - - $ docker push localhost:5000/registry_one:latest - - If you are using the 1.6 Docker client, this pushes the image the `v2 `registry. - -4. Use `curl` to list the image in the registry. - - $ curl -v -X GET http://localhost:32777/v2/registry_one/tags/list - * Hostname was NOT found in DNS cache - * Trying 127.0.0.1... - * Connected to localhost (127.0.0.1) port 32777 (#0) - > GET /v2/registry_one/tags/list HTTP/1.1 - > User-Agent: curl/7.36.0 - > Host: localhost:32777 - > Accept: */* - > - < HTTP/1.1 200 OK - < Content-Type: application/json; charset=utf-8 - < Docker-Distribution-Api-Version: registry/2.0 - < Date: Tue, 14 Apr 2015 22:34:13 GMT - < Content-Length: 39 - < - {"name":"registry1","tags":["latest"]} - * Connection #0 to host localhost left intact - - This example refers to the specific port assigned to the 2.0 registry. You saw - this port earlier, when you used `docker ps` to show your running containers. +## Next +You are now ready to explore [the registry configuration](configuration.md) From 4eb8d907b9c4d6fdf7bc971321da7a11be62d4ec Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Fri, 22 May 2015 02:15:45 -0700 Subject: [PATCH 02/17] Separate introduction Use cases, generalities, image naming overview. Removed most of it from index, and some of it from other random pages. Signed-off-by: Olivier Gambier --- docs/introduction.md | 47 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 docs/introduction.md diff --git a/docs/introduction.md b/docs/introduction.md new file mode 100644 index 00000000..5efec283 --- /dev/null +++ b/docs/introduction.md @@ -0,0 +1,47 @@ + + +# Understanding the Registry + +A registry is a storage and content delivery system, holding named Docker images, available in different tagged versions. For example, the image `distribution/registry`, with tags `2.0` and `latest`. + +Users interact with a registry by using docker push and pull commands. For example, `docker pull myregistry.com/stevvooe/batman:voice`. + +Storage itself is delegated to drivers. The default storage driver is the local posix filesystem, which is suitable for development or small deployments. Additional cloud-based storage driver like S3, Microsoft Azure and Ceph are also supported. People looking into using other storage backends may do so by writing their own driver implementing the [Storage API](storagedrivers.md). + +Since securing access to your hosted images is paramount, the Registry natively supports TLS. You can also enforce basic authentication through a proxy like Nginx. + +The Registry GitHub repository includes reference implementations for additional authentication and authorization methods. Only very large or public deployments are expected to extend the Registry in this way. + +Finally, the Registry includes a robust [notification system](notifications.md), calling webhooks in response to activity, and both extensive logging and reporting. Reporting is mostly useful for large installations that want to collect metrics. Currently, New Relic and Bugsnag are supported. + +## Understanding image naming + +Image names as used in typical docker commands reflect their origin: + + * `docker pull ubuntu` instructs docker to pull an image named `ubuntu` from the official Docker Hub. This is simply a shortcut for the longer `docker pull registry-1.docker.io/library/ubuntu` command + * `docker pull myregistrydomain:port/foo/bar` instructs docker to contact the registry located at `myregistrydomain:port` to find that image + +You can find out more about the various Docker commands dealing with images in the [official Docker engine documentation](https://docs.docker.com/reference/commandline/cli/). + +## Use cases + +Running your own Registry is a great solution to integrate with and complement your CI/CD system. In a typical workflow, a commit to your source revision control system would trigger a build on your CI system, which would then push a new image to your Registry if the build is succesful. A notification from the Registry would then trigger a deployment on a staging environment, or notify other systems that a new image is available. + +It's also an essential component if you want to quickly deploy a new image over a large cluster of machines. + +Finally, it's the best way to distribute images inside an airgap environment. + + +## Requirements + +You absolutely need to be familiar with Docker, specifically with regard to pushing and pulling images. You must understand the difference between the daemon and the cli, and at least grasp basic concepts about networking. + +Also, while just starting a registry is fairly easy, operating it in a production environment requires operational skills, just like any other service. You are expected to be familiar with systems availability and scalability, logging and log processing, systems monitoring, and security 101. Strong understanding of http and overall network communications, plus familiarity with golang are certainly useful as well. + +## Next + +You are now ready to [deploy your registry](deploying.md). From d9dfe543082639077c181b302e7b2edc9999bfb5 Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Fri, 22 May 2015 02:16:25 -0700 Subject: [PATCH 03/17] Separated "help" Help page is separate now. Removed the bulk of it from various pages. Signed-off-by: Olivier Gambier --- docs/help.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 docs/help.md diff --git a/docs/help.md b/docs/help.md new file mode 100644 index 00000000..6223c57d --- /dev/null +++ b/docs/help.md @@ -0,0 +1,19 @@ + + +# Getting help + +If you need help, or just want to chat, you can reach us: + +- on irc: `#docker-distribution` on freenode +- on the [mailing list](https://groups.google.com/a/dockerproject.org/forum/#!forum/distribution) (mail at ) + +If you want to report a bug: + +- be sure to first read about [how to contribute](https://github.com/docker/distribution/blob/master/CONTRIBUTING.md) +- you can then do so on the [GitHub project bugtracker](https://github.com/docker/distribution/issues) + +You can also find out more about the Docker's project [Getting Help resources](https://docs.docker.com/project/get-help). From 431e829fa599dee933fc71a5fb8138de3918105c Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Fri, 22 May 2015 02:17:23 -0700 Subject: [PATCH 04/17] Simplified index Simplified index again, to make access to information more obvious and more direct. Added a TLDR for people in a hurry. Signed-off-by: Olivier Gambier --- docs/index.md | 70 ++++++++++++++++++++++++++++++++------------------- 1 file changed, 44 insertions(+), 26 deletions(-) diff --git a/docs/index.md b/docs/index.md index 01d4e222..34b3a8b6 100644 --- a/docs/index.md +++ b/docs/index.md @@ -6,50 +6,68 @@ IGNORES--> # Docker Registry -The Registry is a stateless, highly scalable server side component that stores and lets you distribute Docker images. +## What it is -Users looking for a ready-to-go solution are encouraged to head-over to the [Docker Hub](https://hub.docker.com), which provides a free-to-use, hosted Registry, plus additional features (organization accounts, automated builds, and more). +The Registry is a stateless, highly scalable server side application that stores and lets you distribute Docker images. +The Registry is open-source, under the permissive [Apache license](http://en.wikipedia.org/wiki/Apache_License). -On the other hand, people interested in finer grained integration, or more control over the content they publish, should run and operate their own Registry. +## Why use it -## About versions +You should use the Registry if you want to: -You need to use a Docker client that is version 1.6.0 or newer for this to work. + * tightly control where your images are being stored + * fully own your images distribution pipeline + * integrate images storage and distribution into your inhouse, on premises development workflow + +## Alternatives + +Users looking for a zero maintenance, ready-to-go solution are encouraged to head-over to the [Docker Hub](https://hub.docker.com), which provides a free-to-use, hosted Registry, plus additional features (organization accounts, automated builds, and more). + +Users looking for a commercially supported version of the Registry should look into [Docker Hub Enterprise](https://docs.docker.com/docker-hub-enterprise/). + +## Requirements + +The Registry is compatible with Docker engine version 1.6.0 or higher. If you really need to work with older Docker versions, you should look into the [old python registry](https://github.com/docker/docker-registry) -## Understanding the Registry +## TL;DR -A registry is a storage and content delivery system, holding named Docker images, available in different tagged versions. For example, the image `distribution/registry`, with tags `2.0` and `latest`. +``` +# Start your registry +docker run -d -p 5000:5000 registry:2 -Users interact with a registry by using docker push and pull commands. For example, `docker pull myregistry.com/stevvooe/batman:voice`. +# Pull (or build) some image +docker pull ubuntu -Storage itself is delegated to drivers. The default storage driver is the local posix filesystem, which is suitable for development or small deployments. Additional cloud-based storage driver like S3, Microsoft Azure and Ceph are also supported. People looking into using other storage backends may do so by writing their own driver implementing the [Storage API](storagedrivers.md). +# Name the image to point to your registry +docker tag ubuntu localhost:5000/myfirstimage -Since securing access to your hosted images is paramount, the Registry natively supports TLS. You can also enforce basic authentication through a proxy like Nginx. +# Push it +docker push localhost:5000/myfirstimage -The Registry GitHub repository includes reference implementations for additional authentication and authorization methods. Only very large or public deployments are expected to extend the Registry in this way. +# Pull it +docker pull localhost:5000/myfirstimage +``` -Finally, the Registry includes a robust [notification system](notifications.md), calling webhooks in response to activity, and both extensive logging and reporting. Reporting is mostly useful for large installations that want to collect metrics. Currently, New Relic and Bugsnag are supported. +Simple as that? Yes. Now, please read the... -## Getting help - -The Registry is an open source project and is under active development. If you need help, would like to contribute, or simply want to talk about the project, we have a number of open channels for communication. - -- To report bugs or file feature requests: please use the [issue tracker on Github](https://github.com/docker/distribution/issues). -- To talk about the project please post a message to the [mailing list](https://groups.google.com/a/dockerproject.org/forum/#!forum/distribution) or join the `#docker-distribution` channel on IRC. -- To contribute code or documentation changes: please submit a [pull request on Github](https://github.com/docker/distribution/pulls). - -For more information and resources, please visit the [Getting Help project page](https://docs.docker.com/project/get-help/). ## Documentation -Basics: - + - [Introduction](introduction.md) - [Deployment](deploying.md) - [Configuration](configuration.md) + - [Getting help](help.md) + - [Contributing](../CONTRIBUTING.md) -Advanced: +Advanced topics: - - [Storage driver model](storagedrivers.md) + - [Authentication](authentication.md) - [Working with notifications](notifications.md) - - [Registry API](spec/api.md) \ No newline at end of file + - [Advanced ops](advanced.md) + +Development resources: + - [Storage driver model](storagedrivers.md) + - [Registry API](spec/api.md) + + From 26d87ed1a5d76720d7541646795f6fa0bc830940 Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Fri, 22 May 2015 02:17:55 -0700 Subject: [PATCH 05/17] No reason to detail here Signed-off-by: Olivier Gambier --- docs/configuration.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/configuration.md b/docs/configuration.md index 47cd0b28..d6ecbee9 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -604,8 +604,7 @@ auth: rootcertbundle: /root/certs/bundle ``` -The `auth` option is **optional** as there are use cases (i.e. a mirror that -only permits pulls) for which authentication may not be desired. There are +The `auth` option is **optional**. There are currently 2 possible auth providers, `silly` and `token`. You can configure only one `auth` provider. From b7b8e64f1dadaa01b9e57ba4bd472b0bf271e5b9 Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Fri, 22 May 2015 02:18:50 -0700 Subject: [PATCH 06/17] Moved instructions up Make it more obvious that environment variables is the way to go. Signed-off-by: Olivier Gambier --- docs/configuration.md | 70 +++++++++++++++++++++---------------------- 1 file changed, 34 insertions(+), 36 deletions(-) diff --git a/docs/configuration.md b/docs/configuration.md index d6ecbee9..5493d313 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -7,9 +7,39 @@ IGNORES--> # Registry Configuration Reference -You configure a registry server using a YAML file. This page explains the -configuration options and the values they can take. You'll also find examples of -middleware and development environment configurations. +The Registry configuration is based on a YAML file, detailed below. While it comes with sane default values out of the box, you are heavily encouraged to review it exhaustively before moving your systems to production. + +## Override configuration options + +In a typical setup where you run your Registry from the official image, you can specify any configuration variable from the environment by passing `-e` arguments to your `docker run` stanza, or from within a Dockerfile using the `ENV` instruction. + +To override a configuration option, create an environment variable named +`REGISTRY_variable` where *`variable`* is the name of the configuration option +and the `_` (underscore) represents indention levels. For example, you can +configure the `rootdirectory` of the `filesystem` storage backend: + +``` +storage: + filesystem: + rootdirectory: /tmp/registry-dev +``` + +To override this value, set an environment variable like this: + +``` +REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/tmp/registry/test +``` + +This variable overrides the `/tmp/registry` value to the `/tmp/registry/test` +directory. + +>**Note**: If an environment variable changes a map value into a string, such +>as replacing the storage driver type with `REGISTRY_STORAGE=filesystem`, then +>all sub-fields will be erased. As such, specifying the storage type in the +>environment will remove all parameters related to the old storage +>configuration. + + ## List of configuration options @@ -42,7 +72,7 @@ log: loglevel: debug # deprecated: use "log" storage: filesystem: - rootdirectory: /tmp/registry + rootdirectory: /tmp/registry-dev azure: accountname: accountname accountkey: base64encodedaccountkey @@ -142,38 +172,6 @@ options marked as **required**. This indicates that you can omit the parent with all its children. However, if the parent is included, you must also include all the children marked **required**. -## Override configuration options - -You can use environment variables to override most configuration parameters. The -exception is the `version` variable which cannot be overridden. You can set -environment variables on the command line using the `-e` flag on `docker run` or -from within a Dockerfile using the `ENV` instruction. - -To override a configuration option, create an environment variable named -`REGISTRY\variable_` where *`variable`* is the name of the configuration option -and the `_` (underscore) represents indention levels. For example, you can -configure the `rootdirectory` of the `filesystem` storage backend: - -``` -storage: - filesystem: - rootdirectory: /tmp/registry -``` - -To override this value, set an environment variable like this: - -``` -REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/tmp/registry/test -``` - -This variable overrides the `/tmp/registry` value to the `/tmp/registry/test` -directory. - ->**Note**: If an environment variable changes a map value into a string, such ->as replacing the storage driver type with `REGISTRY_STORAGE=filesystem`, then ->all sub-fields will be erased. As such, specifying the storage type in the ->environment will remove all parameters related to the old storage ->configuration. ## version From c3b38025038ce269ab44f2226269d3f4f91e3157 Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Fri, 22 May 2015 02:46:23 -0700 Subject: [PATCH 07/17] Small fixes Signed-off-by: Olivier Gambier --- docs/configuration.md | 2 +- docs/deploying.md | 17 ++++++++++++----- 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/docs/configuration.md b/docs/configuration.md index 5493d313..fa57a177 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -30,7 +30,7 @@ To override this value, set an environment variable like this: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/tmp/registry/test ``` -This variable overrides the `/tmp/registry` value to the `/tmp/registry/test` +This variable overrides the `/tmp/registry-dev` value to the `/tmp/registry/test` directory. >**Note**: If an environment variable changes a map value into a string, such diff --git a/docs/deploying.md b/docs/deploying.md index 35ce4cb3..434607fc 100644 --- a/docs/deploying.md +++ b/docs/deploying.md @@ -7,7 +7,7 @@ IGNORES--> # Deploying a registry server -You obviously need to [install Docker](https://docs.docker.com/installation/) (remember you need at Docker version 1.6.0 or newer). +You obviously need to [install Docker](https://docs.docker.com/installation/) (remember you need Docker version 1.6.0 or newer). ## Getting started in 2 lines @@ -17,7 +17,9 @@ Create a folder for your registry data: Start your registry: - $ docker run -d -p 5000:5000 -v `pwd`/registry-data:/tmp/registry-dev --restart=always --name registry registry:2 + $ docker run -d -p 5000:5000 \ + -v `pwd`/registry-data:/tmp/registry-dev \ + --restart=always --name registry registry:2 That's it. @@ -40,7 +42,12 @@ Let assume your registry is accessible via the domain name `myregistrydomain.com If you try to `docker pull myregistrydomain.com:5000/batman/ubuntu`, you will see the following error message: ``` -FATA[0000] Error response from daemon: v1 ping attempt failed with error: Get https://nonregistry:5000/v1/_ping: dial tcp: lookup nonregistry: no such host. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry nonregistry:5000` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/nonregistry:5000/ca.crt +FATA[0000] Error response from daemon: v1 ping attempt failed with error: Get +https://nonregistry:5000/v1/_ping: dial tcp: lookup nonregistry: no such host. If +this private registry supports only HTTP or HTTPS with an unknown CA certificate, +please add `--insecure-registry nonregistry:5000` to the daemon's arguments. In +the case of HTTPS, if you have access to the registry's CA certificate, no need +for the flag; simply place the CA certificate at /etc/docker/certs.d/nonregistry:5000/ca.crt ``` You basically have three different options to comply with docker security requirements here. @@ -135,8 +142,8 @@ registry: REGISTRY_HTTP_TLS_KEY: /certs/domain.key REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data volumes: - - `pwd`/registry-data:/data - - `pwd`/certs:/certs + - /path/registry-data:/data + - /path/certs:/certs ``` You can then start your registry with a simple From 10ba376a99c9b49ec5f62ac82b7408ce4b71f2b4 Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Sun, 31 May 2015 20:08:28 -0700 Subject: [PATCH 08/17] Minor cleanup in the index - adding glossary - removing empty "advanced" - commenting out building and architecture for now - minor text enhancements Signed-off-by: Olivier Gambier --- docs/index.md | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/docs/index.md b/docs/index.md index 34b3a8b6..2a7f08f2 100644 --- a/docs/index.md +++ b/docs/index.md @@ -17,7 +17,7 @@ You should use the Registry if you want to: * tightly control where your images are being stored * fully own your images distribution pipeline - * integrate images storage and distribution into your inhouse, on premises development workflow + * integrate images storage and distribution into your inhouse development workflow ## Alternatives @@ -27,7 +27,7 @@ Users looking for a commercially supported version of the Registry should look i ## Requirements -The Registry is compatible with Docker engine version 1.6.0 or higher. +The Registry is compatible with Docker engine **version 1.6.0 or higher**. If you really need to work with older Docker versions, you should look into the [old python registry](https://github.com/docker/docker-registry) ## TL;DR @@ -36,16 +36,16 @@ If you really need to work with older Docker versions, you should look into the # Start your registry docker run -d -p 5000:5000 registry:2 -# Pull (or build) some image +# Pull (or build) some image from the hub docker pull ubuntu -# Name the image to point to your registry +# Tag the image so that it points to your registry docker tag ubuntu localhost:5000/myfirstimage # Push it docker push localhost:5000/myfirstimage -# Pull it +# Pull it back docker pull localhost:5000/myfirstimage ``` @@ -60,14 +60,19 @@ Simple as that? Yes. Now, please read the... - [Getting help](help.md) - [Contributing](../CONTRIBUTING.md) -Advanced topics: +### Reference and advanced topics + - [Glossary](glossary.md) - [Authentication](authentication.md) - [Working with notifications](notifications.md) - - [Advanced ops](advanced.md) -Development resources: +### Development resources + - [Storage driver model](storagedrivers.md) - [Registry API](spec/api.md) + - + \ No newline at end of file From 56ff32c6834634719e224159a1e61c41218c4af1 Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Sun, 31 May 2015 20:08:48 -0700 Subject: [PATCH 09/17] Revising glossary Signed-off-by: Olivier Gambier --- docs/glossary.md | 81 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 54 insertions(+), 27 deletions(-) diff --git a/docs/glossary.md b/docs/glossary.md index 95c8ec9e..6f07a190 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -1,43 +1,70 @@ + + # Glossary -This page contains distribution related terms. For a complete Docker glossary, -see the [glossary in the full documentation set](http://docs.docker.com/reference/glossary/). +This page contains definitions for distribution related terms.
-
Blob
+

Blob

- The primary unit of registry storage. A string of bytes identified by - content-address, known as a _digest_. +
A blob is any kind of content that is stored by a Registry under a content-addressable identifier (a "digest").
+

+ Layers are a good example of "blobs". +

-
Image
-
An image is a collection of content from which a docker container can be created.
- -
Layer
+

Image

- A tar file representing the partial content of a filesystem. Several - layers can be "stacked" to make up the root filesystem. +
An image is a named set of immutable data from which a Docker container can be created.
+

+ An image is represented by a json file called a manifest, and is conceptually a set of layers. + + Image names indicate the location where they can be pulled from and pushed to, as they usually start with a registry domain name and port. + +

+
+ +

Layer

+
+
A layer is a tar archive bundling partial content from a filesystem.
+

+ Layers from an image are usually extracted in order on top of each other to make up a root filesystem from which containers run out. +

-
Manifest
-
Describes a collection layers that make up an image.
+

Manifest

+
A manifest is the JSON representation of an image.
-
Registry
-
A registry is a service which serves repositories.
- -
Repository
-
- A repository is a collection of docker images, made up of manifests, tags - and layers. The base unit of these components are blobs. +

Namespace

+
A namespace is a collection of repositories with a common name prefix.
+

+ The namespace with an empty prefix is considered the Global Namespace. +

-
Tag
-
Tag provides a common name to an image.
+

Registry

+
A registry is a service that let you store and deliver images.
+
-
Namespace
-
A namespace is a collection of repositories with a common name prefix. The - namespace with an empty common prefix is considered the Global Namespace.
+

Repository

+
+
A repository is a set of data containing all versions of a given image.
+
+ +

Scope

+
A scope is the portion of a namespace onto which a given authorization token is granted.
+ +

Tag

+
A tag is conceptually a "version" of a named image.
+

+ Example: `docker pull myimage:latest` instructs docker to pull the image "myimage" in version "latest". +

+ +
+ -
Scope
-
A common repository name prefix.
From c405f3717a35a50c4fabeb98ecbdd2badcb84b12 Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Sun, 31 May 2015 20:09:15 -0700 Subject: [PATCH 10/17] Minor fixes Signed-off-by: Olivier Gambier --- docs/deploying.md | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/docs/deploying.md b/docs/deploying.md index 434607fc..46d9ce4c 100644 --- a/docs/deploying.md +++ b/docs/deploying.md @@ -1,13 +1,12 @@ - # Deploying a registry server -You obviously need to [install Docker](https://docs.docker.com/installation/) (remember you need Docker version 1.6.0 or newer). +You obviously need to [install Docker](https://docs.docker.com/installation/) (remember you need **Docker version 1.6.0 or newer**). ## Getting started in 2 lines @@ -25,32 +24,29 @@ That's it. You can now tag an image and push it: - $ docker tag ubuntu localhost:5000/batman/ubuntu + $ docker pull ubuntu && docker tag ubuntu localhost:5000/batman/ubuntu $ docker push localhost:5000/batman/ubuntu -Then pull it: +Then pull it back: $ docker pull localhost:5000/batman/ubuntu ## Making your Registry available -Now that your registry works on localhost, you probably want to make it available as well to other hosts. +Now that your registry works on `localhost`, you probably want to make it available as well to other hosts. Let assume your registry is accessible via the domain name `myregistrydomain.com` (still on port `5000`). If you try to `docker pull myregistrydomain.com:5000/batman/ubuntu`, you will see the following error message: ``` -FATA[0000] Error response from daemon: v1 ping attempt failed with error: Get -https://nonregistry:5000/v1/_ping: dial tcp: lookup nonregistry: no such host. If -this private registry supports only HTTP or HTTPS with an unknown CA certificate, -please add `--insecure-registry nonregistry:5000` to the daemon's arguments. In -the case of HTTPS, if you have access to the registry's CA certificate, no need -for the flag; simply place the CA certificate at /etc/docker/certs.d/nonregistry:5000/ca.crt +FATA[0000] Error response from daemon: v1 ping attempt failed with error: Get https://myregistrydomain.com:5000/v1/_ping: tls: oversized record received with length 20527. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry myregistrydomain.com:5000` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/myregistrydomain.com:5000/ca.crt ``` -You basically have three different options to comply with docker security requirements here. +If trying to reach a non `localhost` registry, Docker requires that you secure it using https, or make it explicit that you want to run an insecure registry. + +You basically have three different options to comply with that security requirement here. ### 1. buy a SSL certificate for your domain @@ -87,7 +83,7 @@ docker run -d -p 5000:5000 \ This basically tells Docker to entirely disregard security for your registry. -1. edit the file `/etc/default/docker` so that there is a line that reads: `DOCKER_OPTS="--insecure-registry myregistrydomain:5000"` (or add that to existing `DOCKER_OPTS`) +1. edit the file `/etc/default/docker` so that there is a line that reads: `DOCKER_OPTS="--insecure-registry myregistrydomain.com:5000"` (or add that to existing `DOCKER_OPTS`). Restart docker. 2. restart your Docker daemon: on ubuntu, this is usually `service docker stop && service docker start` **Pros:** @@ -113,7 +109,7 @@ Be sure to use the name `myregistrydomain.com` as a CN. Now go to solution 1 above and stop and restart your registry. -Then you have to instruct every docker daemon to trust that certificate. This is done by copying the `domain.crt` file to `/etc/docker/certs.d/myregistrydomain.com:5000/ca.crt` +Then you have to instruct every docker daemon to trust that certificate. This is done by copying the `domain.crt` file to `/etc/docker/certs.d/myregistrydomain.com:5000/ca.crt` (don't forget to restart docker after doing so). **Pros:** From c1c638d01f05cd292163005a0ff341d602f4ab85 Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Sun, 31 May 2015 20:09:24 -0700 Subject: [PATCH 11/17] Minor keyword add Signed-off-by: Olivier Gambier --- docs/configuration.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/configuration.md b/docs/configuration.md index fa57a177..68fb54c5 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1,7 +1,7 @@ From 74873aed14411163b10463dbbbe226364639f036 Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Sun, 31 May 2015 20:09:44 -0700 Subject: [PATCH 12/17] Removing internal information Removed content has been ported to the wiki Signed-off-by: Olivier Gambier --- CONTRIBUTING.md | 28 ---------------------------- 1 file changed, 28 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 30d792d3..a4ffc028 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -90,31 +90,3 @@ It's mandatory to: Complying to these simple rules will greatly accelerate the review process, and will ensure you have a pleasant experience in contributing code to the Registry. Have a look at a great, succesful contribution: the [Ceph driver PR](https://github.com/docker/distribution/pull/443) - - -## Issue and PR labels - -To keep track of the state of issues and PRs, we've adopted a set of simple labels. The following are currently in use: - -
-
Backlog
-
Issues marked with this label are considered not yet ready for implementation. Either they are untriaged or require futher detail to proceed.
- -
Blocked
-
If an issue requires further clarification or is blocked on an unresolved dependency, this label should be used.
- -
Sprint
-
Issues marked with this label are being worked in the current sprint. All required information should be available and design details have been worked out.
- -
In Progress
-
The issue or PR is being actively worked on by the assignee.
- -
Done
-
Issues marked with this label are complete. This can be considered a psuedo-label, in that if it is closed, it is considered "Done".
-
- -If an issue or PR is not labeled correctly or you believe it is not in the right state, please contact a maintainer to fix the problem. - -## Milestones - -Issues and PRs should be assigned to relevant milestones. If an issue or PR is assigned a milestone, it should be available by that date. Depending on level of effort, items may be shuffled in or out of milestones. Issues or PRs that don't have a milestone are considered unscheduled. Typically, "In Progress" issues should have a milestone. From 96d79eb30e29c05c0bee7fe86e0b29c4fbda6bd5 Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Sun, 31 May 2015 20:09:59 -0700 Subject: [PATCH 13/17] Adding authentication Signed-off-by: Olivier Gambier --- docs/authentication.md | 171 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 171 insertions(+) create mode 100644 docs/authentication.md diff --git a/docs/authentication.md b/docs/authentication.md new file mode 100644 index 00000000..8c6fff25 --- /dev/null +++ b/docs/authentication.md @@ -0,0 +1,171 @@ + + +# Authentication + +While running an unrestricted registry is certainly ok for development, secured local networks, or test setups, you should probably implement access restriction if you plan on making your registry available to a wider audience or through public internet. + +The Registry supports two different authentication methods to get your there: + + * direct authentication, through the use of a proxy + * delegated authentication, redirecting to a trusted token server + +The first method is recommended for most people as the most straight-forward solution. + +The second method requires significantly more investment, and only make sense if you want to fully configure ACLs and more control over the Registry integration into your global authorization and authentication systems. + +## Direct authentication through a proxy + +With this method, you implement basic authentication in a reverse proxy that sits in front of your registry. + +Since the Docker engine uses basic authentication to negociate access to the Registry, securing communication between docker engines and your proxy is absolutely paramount. + +While this model gives you the ability to use whatever authentication backend you want through a secondary authentication mechanism implemented inside your proxy, it also requires that you move TLS termination from the Registry to the proxy itself. + +Below is a simple example of secured basic authentication (using TLS), using nginx as a proxy. + +### Requirements + +You should have followed entirely the basic [deployment guide](deployement.md). + +If you have not, please take the time to do so. + +At this point, it's assumed that: + + * you understand Docker security requirements, and how to configure your docker engines properly + * you have installed Docker Compose + * you have a `domain.crt` and `domain.key` files, for the CN `myregistrydomain.com` (or whatever domain name you want to use) + * these files are located inside the current directory, and there is nothing else in that directory + * it's HIGHLY recommended that you get a certificate from a known CA instead of self-signed certificates + * be sure you have stopped and removed any previously running registry (typically `docker stop registry && docker rm registry`) + + +### Setting things up + +Read again the requirements. + +Ready? + +Run the following: + +``` +mkdir auth +mkdir data + +# This is the main nginx configuration you will use +cat < auth/registry.conf +upstream docker-registry { + server registry:5000; +} + +server { + listen 443 ssl; + server_name myregistrydomain.com; + + # SSL + ssl_certificate /etc/nginx/conf.d/domain.crt; + ssl_certificate_key /etc/nginx/conf.d/domain.key; + + # disable any limits to avoid HTTP 413 for large image uploads + client_max_body_size 0; + + # required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486) + chunked_transfer_encoding on; + + location /v2/ { + # Do not allow connections from docker 1.5 and earlier + # docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents + if ($http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*$" ) { + return 404; + } + + # To add basic authentication to v2 use auth_basic setting plus add_header + auth_basic "registry.localhost"; + auth_basic_user_file /etc/nginx/conf.d/registry.password; + add_header 'Docker-Distribution-Api-Version' 'registry/2.0' always; + + proxy_pass http://docker-registry; + proxy_set_header Host $http_host; # required for docker client's sake + proxy_set_header X-Real-IP $remote_addr; # pass on real client's IP + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_read_timeout 900; + } +} +EOF + +# Now, create a password file for "testuser" and "testpassword" +echo 'testuser:$apr1$8TVQAgWs$jzdkx8IgYR.guL0hzNDXq.' > auth/registry.password + +# Alternatively you could have achieved the same thing with htpasswd +# htpasswd -bc auth/registry.password testuser testpassword + +# Copy over your certificate files +cp domain.crt auth +cp domain.key auth + +# Now create your compose file + +cat < docker-compose.yml +nginx: + image: "nginx:1.9" + ports: + - 5043:443 + links: + - registry:registry + volumes: + - `pwd`/auth/:/etc/nginx/conf.d + +registry: + image: registry:2 + ports: + - 127.0.0.1:5000:5000 + environment: + REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data + volumes: + - `pwd`/data:/data +EOF +``` + +### Starting and stopping + +That's it. You can now: + + * `docker-compose up -d` to start your registry + * `docker login myregistrydomain.com:5043` (using `testuser` and `testpassword`) + * `docker tag ubuntu myregistrydomain.com:5043/toto` + * `docker push myregistrydomain.com:5043/toto` + +### Docker still complains about the certificate? + +That's certainly because you are using a self-signed certificate, despite the warnings. + +If you really insist on using these, you have to trust it at the OS level. + +Usually, on Ubuntu this is done with: +``` +cp auth/domain.crt /usr/local/share/ca-certificates/myregistrydomain.com.crt +update-ca-certificates +``` + +... and on RedHat with: +``` +cp auth/domain.crt /etc/pki/ca-trust/source/anchors/myregistrydomain.com.crt +update-ca-trust +``` + +Now: + + * `service docker stop && docker service start` (or any other way you use to restart docker) + * `docker-compose up -d` to bring your registry up + +## Token-based delegated authentication + +This is **advanced**. + +You will find [background information here](spec/auth/token.md), [configuration information here](configuration.md#auth). + +Beware that you will have to implement your own authentication service for this to work (though there exist third-party open-source implementations). From 8c1784c838e30883cf4bd61f6fc4799e4012d51b Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Sun, 31 May 2015 20:23:43 -0700 Subject: [PATCH 14/17] Keeping in synch - commenting out both the "JSON" and "token" specs, unless someone thinks they should be here - added help, glossary, introduction and authentication documents Signed-off-by: Olivier Gambier --- docs/mkdocs.yml | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index c888ed4e..07bab4ec 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -1,12 +1,16 @@ - ['registry/index.md', 'Reference', 'Docker Registry 2.0'] +- ['registry/introduction.md', 'Reference', '    ▪  Introduction' ] - ['registry/deploying.md', 'Reference', '    ▪  Deploy a registry' ] - ['registry/configuration.md', 'Reference', '    ▪  Configure a registry' ] +- ['registry/authentication.md', 'Reference', '    ▪  Authentication' ] +- ['registry/glossary.md', 'Reference', '    ▪  Glossary' ] +- ['registry/help.md', 'Reference', '    ▪  Getting help' ] - ['registry/storagedrivers.md', 'Reference', '    ▪  Storage driver model' ] - ['registry/notifications.md', 'Reference', '    ▪  Work with notifications' ] - ['registry/spec/api.md', 'Reference', '    ▪  Registry Service API v2' ] -- ['registry/spec/json.md', 'Reference', '    ▪  JSON format' ] -- ['registry/spec/auth/token.md', 'Reference', '    ▪  Authenticate via central service' ] +- ['registry/spec/json.md', '**HIDDEN**'] +- ['registry/spec/auth/token.md', '**HIDDEN**'] - ['registry/storage-drivers/azure.md', '**HIDDEN**' ] - ['registry/storage-drivers/filesystem.md', '**HIDDEN**' ] - ['registry/storage-drivers/inmemory.md', '**HIDDEN**' ] From 81e8657d7b92a6b07fa9df1ac270d837137df77e Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Sun, 31 May 2015 20:32:07 -0700 Subject: [PATCH 15/17] Fix here doc conf generation Signed-off-by: Olivier Gambier --- docs/authentication.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/authentication.md b/docs/authentication.md index 8c6fff25..cd96bb92 100644 --- a/docs/authentication.md +++ b/docs/authentication.md @@ -78,7 +78,7 @@ server { location /v2/ { # Do not allow connections from docker 1.5 and earlier # docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents - if ($http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*$" ) { + if (\$http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*\$" ) { return 404; } @@ -88,10 +88,10 @@ server { add_header 'Docker-Distribution-Api-Version' 'registry/2.0' always; proxy_pass http://docker-registry; - proxy_set_header Host $http_host; # required for docker client's sake - proxy_set_header X-Real-IP $remote_addr; # pass on real client's IP - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; + proxy_set_header Host \$http_host; # required for docker client's sake + proxy_set_header X-Real-IP \$remote_addr; # pass on real client's IP + proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto \$scheme; proxy_read_timeout 900; } } From b8d785c79e6c225c6e8be21b5bee82332543700d Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Mon, 8 Jun 2015 15:54:37 -0700 Subject: [PATCH 16/17] Use bcrypt Signed-off-by: Olivier Gambier --- docs/authentication.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/authentication.md b/docs/authentication.md index cd96bb92..14516263 100644 --- a/docs/authentication.md +++ b/docs/authentication.md @@ -98,10 +98,10 @@ server { EOF # Now, create a password file for "testuser" and "testpassword" -echo 'testuser:$apr1$8TVQAgWs$jzdkx8IgYR.guL0hzNDXq.' > auth/registry.password +echo 'testuser:$2y$05$.nIfPAEgpWCh.rpts/XHX.UOfCRNtvMmYjh6sY/AZBmeg/dQyN62q' > auth/registry.password # Alternatively you could have achieved the same thing with htpasswd -# htpasswd -bc auth/registry.password testuser testpassword +# htpasswd -Bbc auth/registry.password testuser testpassword # Copy over your certificate files cp domain.crt auth From 25bd3fc77708b1be5f0cbc140a42fe83f4a39bef Mon Sep 17 00:00:00 2001 From: Olivier Gambier Date: Mon, 8 Jun 2015 15:54:46 -0700 Subject: [PATCH 17/17] Link to compose Signed-off-by: Olivier Gambier --- docs/deploying.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/deploying.md b/docs/deploying.md index 46d9ce4c..c0f31aac 100644 --- a/docs/deploying.md +++ b/docs/deploying.md @@ -119,11 +119,9 @@ Then you have to instruct every docker daemon to trust that certificate. This is - you have to configure every docker daemon that wants to access your registry - - ## Using Compose -It's highly recommended to use Docker Compose to facilitate managing your Registry configuration. +It's highly recommended to use [Docker Compose](https://docs.docker.com/compose/) to facilitate managing your Registry configuration. Here is a simple `docker-compose.yml` that does setup your registry exactly as above, with TLS enabled.