In my earlier post I recommended including a DigitalOcean Container Registry subscription when setting up a budget Kubernetes cluster, because it’s convenient and avoids circular dependency issues. I would still recommend that option if the Kubernetes cluster is your only internet-facing cloud host. However, I quickly outgrew the 5-repository limit of the basic plan, so I took another look at setting up a self-hosted Docker registry.
As it turns out, it’s not as difficult as I thought. The trickiest part was installing the SSL certificate, since that’s a requirement for the registry. This will be my configuration:
- TLS provided by Let’s Encrypt (see my previous guide)
- HTTP Basic Auth
- Storage backed by DigitalOcean Spaces (but S3 or another S3-compatible host would work)
- Docker Compose to start the service
Firstly, there’s a Docker image for the registry at https://hub.docker.com/_/registry. I used this as a starting point for my docker-compose file below, which I ran on another of my VMs. (Note: don’t attempt to install this on the same Kubernetes cluster, because that would create a circular dependency).
registry: restart: always image: registry:2.7 ports: - 5000:5000 environment: REGISTRY_HTTP_TLS_CERTIFICATE: /certs/server.crt REGISTRY_HTTP_TLS_KEY: /certs/server.key REGISTRY_AUTH: htpasswd REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm REGISTRY_STORAGE_S3_ACCESSKEY: <My access key> REGISTRY_STORAGE_S3_SECRETKEY: <My secret key> SETTINGS_FLAVOR: prod DOCKER_REGISTRY_CONFIG: /conf/config.yml volumes: - /etc/letsencrypt/live/mycert/fullchain.pem:/certs/server.crt - /etc/letsencrypt/live/mycert/privkey.pem:/certs/server.key - /docker_volumes/registry/data:/var/lib/registry - /docker_volumes/registry/auth:/auth - /docker_volumes/registry/conf/config.yml:/etc/docker/registry/config.yml
Note this relies on already having issued a cert named mycert from Let’s Encrypt. You can now start the registry with docker-compose up. Next, edit /docker_volumes/registry/conf/config.yml to configure the storage:
storage: s3: regionendpoint: sgp1.digitaloceanspaces.com region: sgp1 bucket: my-registry-bucket
You will also have to configure a user for the registry using htpasswd (run from /docker_volumes/registry):
docker run \ --entrypoint htpasswd \ httpd:2 -Bbn testuser testpassword > auth/htpasswd
Assuming the server is accessible, you can now use docker login my-server-ip:5000 with the credentials above to login to the registry.
Using the Registry From Kubernetes
To use the new private registry with Kubernetes, you first need to create a secret with the credentials you added above:
kubectl create secret docker-registry regcred \ --docker-server=<your-registry-server> \ --docker-username=<your-name> \ --docker-password=<your-pword> \ --docker-email=<your-email>
Next, you just need to use this secret as an imagePullSecret, such as:
$ kubectl edit sa default # Add this imagePullSecrets: - name: regcred
Kubernetes will now be able to use images with URLs from the new private registry!
After setting up a self-hosted docker registry I can create unlimited images, because I only need to pay for the storage on Spaces. As a bonus, I migrated my images from the managed Container Registry to my new private registry, reducing the monthly cost of my cluster to $45.