There are a variety of use-cases that call for the movement of container images between different environments.

Example 1 - Managing Images Across the SDLC

An OpenShift deployment may be divided into non-production and production clusters. In order to promote images between the clusters it is necessary to pull images from the Image Registry of the non-production cluster, and push them to the production cluster.

Example 2 - Distributing Images to Multiple Geos

OpenShift may be deployed anywhere RHEL can be deployed. This opens up the ability to create hybrid private, public cloud environments across a number of different geographical locations. Customer may wish to leverage public cloud availability in specific geos to provide low-latency resources to their customers living and working in those locations, for example.

Trying it out with OpenShift

In preparing this guide I developed the following examples using OpenShift Enterprise 3.1.1, and the Red Hat Container Development Kit beta 5, with my local Docker instance running on Fedora 23. Where possible I’ve added notes regarding differences you may encounter on other versions or platforms.

There are a few steps needed to get this working:

1. Expose OpenShift’s Docker Registry, to make it available to external systems
2. We may need to allow our local Docker daemon to access insecure registries
3. Create, or identify, a service account with sufficient access rights
4. Prepare local images for pushing to OpenShift

Making OpenShift’s Docker Registry Externally Accessible

Prepare the docker registry, by default, the registry is not externally accessible

$ oc expose service docker-registry -n default

Make sure the route name is resolvable by the external system. For example:

<span>docker-registry-default.apps.example.com</span>

Your local docker registry needs to be configured to accept communication with this registry, by default it will be listening on port 80 and be insecure (you may be required to provide a secured registry in which case I recommend following the OpenShift documentation on Accessing The Registry Directly). To allow Docker to communicate with an insecure registry add the --insecure-registry option to your docker daemon service configuration, and include the port specifier. For example on Fedora, Centos or RHEL, edit /etc/sysconfig/docker on your local machine.

# /etc/sysconfig/docker
# Modify these options if you want to change the way the docker daemon runs
OPTIONS='--add-registry=docker.io --log-driver=journald --insecure-registry docker-registry.192.168.121.113.xip.io:80'
#OPTIONS='--selinux-enabled --log-driver=journald'
DOCKER_CERT_PATH=/etc/docker
...

Once you’ve updated your local docker configuration, you will need to restart the local docker daemon.

<span>$ </span><b>systemctl restart docker</b>

Preparing an OpenShift Project to Accept Pushes

In order to access the Docker Registry remotely it is necessary to provide access credentials. These credentials could be based on your personal user account, but for automation approaches, for example using an external Jenkins service, it’s recommended that a Service Account (SA) is used.

You could use the ‘builder’ SA within a project, but let's create a new SA for this role.

<span>$ </span><b>oc new-project pushed</b>

Creating a new SA in OpenShift Origin, and OpenShift 3.2 has been simplified to the following command:

<span>$ </span><b>oc create serviceaccount pusher</b>

For older versions of OpenShift, use the following method:

$ oc create -f - << API
apiVersion: v1
kind: ServiceAccount
metadata:
  name: pusher
API

To pull images users need to get imagestreams/layers

To push images users need to update imagestreams/layers

The system:image-puller role will just provide pull capability.

The system:image-builder role allows both pull and push capability.

Note: the system:image-builder role can only be given by cluster-admins, for project admins, the edit role will provide sufficient access.

Add the system:image-builder, or edit role to the SA.

For cluster admins

$ oc policy add-role-to-user system:image-builder 
system:serviceaccount:pushed:pusher

For project admins

$ oc policy add-role-to-user edit 
system:serviceaccount:pushed:pusher

We will need the SA’s token

$ oc describe sa pusher
Name: pusher
Namespace: pushed
Labels: <none>
Mountable secrets:  pusher-token-fxg2k
                    pusher-dockercfg-vwddo
Tokens:             pusher-token-98rix
                    pusher-token-fxg2k
Image pull secrets: pusher-dockercfg-vwddo

Select one of the token names, and discover its value

$ oc describe secret pusher-token-98rix
Name: pusher-token-98rix
Namespace: pushed
Labels: <none>
Annotations: kubernetes.io/service-account.name=pusher,kubernetes.io/service-account.uid=0bd3b6e0-f655-11e5-aca3-525400723ce5
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1066 bytes
token: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiw…
ETq0W3G3wHiO1Rdg4nsk7R8k-VJ8Qk-2V7elYVz8LYsNpkd1PGQGE5Jtegnr5GoFDk5wKxzA7GT1zXt2vVg

You will also need an image stream to accept the image to be pushed.

A fully qualified docker image name should conform to the following format:

<span>&lt;registry&gt;/&lt;repo&gt;/&lt;image&gt;:&lt;tag&gt;</span>

When mapping to OpenShift,

<span>&lt;registry&gt;/&lt;project&gt;/&lt;imagestream&gt;:&lt;tag&gt;</span>

Create the image stream

$ oc create -f - <<API
apiVersion: v1
kind: ImageStream
metadata:
  annotations:
    description: Keeps track of changes in the application image
  name: myimage
API

The project is now ready to accept pull and push requests from the “pusher” service account.

Pushing an Image

You will need to login to the remote repository in order to push to it.

<span>$ docker login --username=anything </span><a href="mailto:--email=anyone@anywhere.com"><span>--email=anyone@anywhere.com</span></a><span> --password=&lt;token&gt; &lt;docker registry route&gt;</span>

For example:

$ docker login --username=ed --email=eseymour@redhat.com 
 docker-registry.192.168.121.113.xip.io:80
password: <paste SA token value>

Once logged in, you will need to tag your image.

$ docker images
REPOSITORY                                            TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
<none>                                            latest              445f4be0f46f        6 days ago          392.1 MB
$ docker tag 445f4be0f46f 
docker-registry.192.168.121.113.xip.io:80/pushed/myimage:latest

And once tagged, the image can now be pushed to OpenShift.

<span>$</span><b> docker push docker-registry.192.168.121.113.xip.io:80/pushed/myimage:latest</b>

You may then check the status of the associated imagestream in OpenShift

$ oc describe is myimage
Name: myimage
Created: About an hour ago
Labels: <none>
Description: Keeps track of changes in the application image
Annotations: openshift.io/image.dockerRepositoryCheck=2016-03-30T10:28:30Z
Docker Pull Spec: 172.30.55.172:5000/pushed/myimage
 
Tag Spec Created PullSpec Image
latest <pushed> 17 seconds ago 172.30.55.172:5000/pushed/myimage@sha256:4ed6572d87aa07e9ac044421c30e6e6bd47349c6391ea6f2efdde9a2c2e9fcba

Pulling an Image

Pulling an image to a local Docker instance is simply a task of using the fully qualified image name from the remote OSE repository, for example:

$ docker pull docker-registry.192.168.121.113.xip.io:80/pushed/myimage

As you would expect you can also reference remote images in OpenShift to deploy as applications. In the diagram below the images are pulled directly into the OpenShift Production Cluster, rather than being source from the cluster’s own Image Registry.

Normally we would create an ImageStream to present the image to OpenShift Production Cluster projects, however, in this example we will use a Deployment Configuration to reference the image directly.

apiVersion: v1

kind: DeploymentConfig

metadata:

  name: myapp

spec:

  replicas: 1

  ...

  template:

    ...

    spec:

      containers:

        ...

        image: docker-registry.192.168.121.113.xip.io:80/pushed/myimage:v1.1

        imagePullPolicy: IfNotPresent

        name: myimage

        ...

We still need authorise the OpenShift Production Cluster project so that it can access the remote Image Registry. We do this by created pull secrets for the service account in the destination project.

$ oc secrets new-dockercfg <pull_secret_name> 
    --docker-server=<registry_server> --docker-username=<user_name> 
    --docker-password=<password> --docker-email=<email>

• <pull secret name> - an arbitrary label
• <registry_server> - the remote registry server, for example: docker-registry.192.168.121.113.xip.io:80
• <user_name> - an appropriate username, if accessing an OpenShift Image Registry this can be anything.
• <password> - the associated password, or if accessing an OpenShift Image Registry this is the authorised service account token from the source project.
• <email> - any valid email address.

The secret then needs to be attached to the service account that will deploy container, in OpenShift this is normally the default service account.

<span>$ oc secrets add serviceaccount/default secrets/</span><i><span>&lt;pull_secret_name&gt;</span></i><span> --for=pull</span>

For more information on this see Image Pull Secrets in the OpenShift documentation.

In OpenShift 3.2 it will be possible to use the oc import-image command to automatically create an ImageStream and sync with an external authenticated repository. The ImageStream will be able to use the docker pull secret created above.

<span>$ oc import-image myimage --from=docker-registry.192.168.121.113.xip.io:80 --confirm --insecure-repository=true</span>

If, as in this example, we are using an unsecured registry, it was necessary to add the --insecure-repository option. If you view the resulting ImageStream configuration, you’ll notice the added annotation.

apiVersion: v1

kind: ImageStream

metadata:

  annotations:

    openshift.io/image.insecureRepository: "true"

  name: myimage

...