Jupyter on OpenShift Part 7: Adding the Image to the Catalog

When you are deploying an application from the OpenShift web console you have the choice of deploying an image hosted on an external image registry, or an existing image which was built within the OpenShift cluster using either the Docker or Source build strategies. This is done from Deploy Image after having selected Add to project in the web console.

Deploying an image from the Deploy Image page only works where the image is a final runnable application image. This does not allow you to select an arbitrary image and run it as a Source-to-Image (S2I) builder against a Git repository Url, to create an application using the Source build strategy, and deploy it.

In order to be able to use an arbitrary image as a S2I builder from the web console, you need to tell OpenShift a bit more information about it. This is done by creating an Image Stream definition and adding annotations to it which inform OpenShift that it is a S2I builder. With that information OpenShift is then able to categorise the image and display it as part of the application catalog.

In this blog post we will explore the steps required to do this, by creating an Image Stream definition for the S2I enabled Jupyter Notebook image which has been the subject of this series of blogs posts about running Jupyter Notebooks on OpenShift.

Importing an Exernal Image

In the last post, we built the s2i-minimal-notebook image from source contained in a remote Git repository using the Docker build strategy of OpenShift. The ability to use the Docker build strategy in OpenShift will not always be an option, as use of it may be restricted.

In this situation it is necessary to build the image outside of OpenShift and host it on an external image registry. One way of doing this is to use the ability of the Docker Hub Registry to perform an Automated Build from a Git repository hosted on GitHub.

Using this feature, the Git repository:

has been set up to be automatically built and made available on Docker Hub Registry under the image name:

  • getwarped/s2i-minimal-notebook:latest

This means that to run the S2I builder against a Git repository containing a set of notebooks, you can run:

oc new-app --name sample-notebooks \
  getwarped/s2i-minimal-notebook:latest~https://github.com/getwarped/sample-notebooks.git

This is still using the command line though, and we want to be able to use the S2I builder from the web console.

The first step to achieving that is to create an Image Stream definition. Using the image from the Docker Hub Registry, this can be done by running the command:

oc import-image getwarped/s2i-minimal-notebook:latest --confirm

The --confirm option is required as OpenShift doesn’t as yet know about the image and so it requires you to confirm that it should create an Image Stream definition for you.

The output of running this command will be similar to:

The import completed successfully.

Name:           s2i-minimal-notebook
Namespace:      myproject
Created:        Less than a second ago
Labels:         <none>
Annotations:        openshift.io/image.dockerRepositoryCheck=2017-04-05T00:59:51Z
Docker Pull Spec:   172.30.249.50:5000/myproject/s2i-minimal-notebook
Unique Images:      1
Tags:           1

latest
  tagged from getwarped/s2i-minimal-notebook:latest

  * getwarped/s2i-minimal-notebook@sha256:f0c7f2b7d8aa63576ebddd27e69a881c923409eb7f20b1f83df6e5f727260486
      Less than a second ago

Looking at the actual Image Stream definition created by running:

oc get is/s2i-minimal-notebook -o json

the key information from the definition which was created is:

{
    "kind": "ImageStream",
    "apiVersion": "v1",
    "metadata": {
        "name": "s2i-minimal-notebook"
    },
    "spec": {
        "tags": [
            {
                "name": "latest",
                "from": {
                    "kind": "DockerImage",
                    "name": "getwarped/s2i-minimal-notebook:latest"
                }
            }
        ]
    }
}

Having imported the image and an Image Stream definition created, when running oc new-app, rather than use the name of the image as it is identified by Docker Hub Registry, we can now use just the local image stream name.

oc new-app --name sample-notebooks \
  s2i-minimal-notebook~https://github.com/getwarped/sample-notebooks.git

We still cannot use the image as an S2I builder from the web console.

Image Stream Annotations

The next step to having the S2I builder image appear in the web console application catalog, is to add annotations to the image stream so OpenShift knows it is a S2I builder.

The annotations that are required need to be added as spec.tags.annotations in the Image Stream definition. In this case we only have one tag, but if you had more than one, you would need to add the annotations to all tags for the image versions that you want to be selectable from the web console.

The one annotation which results in OpenShift making the S2I builder image selectable from the web console is called tags, with it needing to include the value builder, yielding:

{
    "kind": "ImageStream",
    "apiVersion": "v1",
    "metadata": {
        "name": "s2i-minimal-notebook"
    },
    "spec": {
        "tags": [
            {
                "name": "latest",
                "annotations": {
                    "tags": "builder"
                },
                "from": {
                    "kind": "DockerImage",
                    "name": "getwarped/s2i-minimal-notebook:latest"
                }
            }
        ]
    }
}

To add the annotation you can either edit the Image Stream definition directly by running:

oc edit is/s2i-minimal-notebook -o json

or you can run the command:

oc patch is/s2i-minimal-notebook --type json \
  --patch '[{"op": "replace", "path": "/spec/tags/0/annotations", "value": { "tags": "builder" }}]'

After adding this annotation you should now be able to find the S2I builder image listed in the web console under the Uncategorized section of the application catalog displayed when using Add to project.

This lets us select the S2I builder image from the web console, but lets see if we can have it be better categorised, add a better name, as well as a description.

Adding a Display Name

At the moment the S2I builder image will display with the name s2i-minimal-notebook. To use an alternative name which isn’t perhaps as cryptic as the name of the Image Stream, you can set the openshift.io/display-name annotation.

This annotation can be added in two places. The first place it would be added is to the metadata for the Image Stream. This is the generic display name. The second place you can add it is as an annotation on the tagged version of an image. This might be used where you want to qualify the display name with additional version information.

{
    "kind": "ImageStream",
    "apiVersion": "v1",
    "metadata": {
        "name": "s2i-minimal-notebook",
        "annotations": {
            "openshift.io/display-name": "Jupyter Notebook"
        }
    },
    "spec": {
        "tags": [
            {
                "name": "latest",
                "annotations": {
                    "openshift.io/display-name": "Jupyter Notebook (latest)",
                    "tags": "builder"
                },
                "from": {
                    "kind": "DockerImage",
                    "name": "getwarped/s2i-minimal-notebook:latest"
                }
            }
        ]
    }
}

Adding to a Category

To have the S2I builder appear under a specific category, instead of being Uncategorized, you can add additional tag values to the tags annotation. That is, in addition to the existing builder value. Any additional tag values should be separated by a comma.

At the same time as specifying any categories to add the S2I builder image, to help with identifying its purpose, you can also specify that one of a set number of icons can be used when the details of the S2I builder image are displayed in the web console. This is defined by the iconClass annotation.

For more details on what categories are available, as well as what icons you can choose from, see the documentation on specifying an image stream for a S2I builder.

For the Jupyter Notebook builder image, we want to have it be associated with the Python category, so for the tags annotation will use the value builder,python and for the iconClass annotation, the value icon-python.

Adding a Description

The display name helps with identifying what the S2I builder image is for, but a description helps out even more. A description can be specified by supplying the description annotation.

If the purpose of the S2I builder image is not obvious, or you want to provide an example to people of how to use it, you can also specify a sample Git repository that a user can use to try it out. This can be supplied in the sampleRepo annotation.

Running the S2I Builder

The final Image Stream definition for the Jupyter Notebook S2I builder image ends up being as follows.

{
    "kind": "ImageStream",
    "apiVersion": "v1",
    "metadata": {
        "name": "s2i-minimal-notebook",
        "annotations": {
            "openshift.io/display-name": "Jupyter Notebook"
        }
    },
    "spec": {
        "tags": [
            {
                "name": "latest",
                "annotations": {
                    "openshift.io/display-name": "Notebook",
                    "description": "Build and deploy a custom Jupyter Notebook image with pre-bundled notebooks, data files and Python packages.",
                    "iconClass": "icon-python",
                    "sampleRepo": "https://github.com/getwarped/sample-notebooks.git",
                    "tags": "builder,python",
                    "version": "latest"
                },
                "from": {
                    "kind": "DockerImage",
                    "name": "getwarped/s2i-minimal-notebook:latest"
                }
            }
        ]
    }
}

When we use the web console and drill down into the Python category in the application catalog we will see:

Clicking on Select for the Jupyter Notebook option, we are then presented with:

Enter in sample-notebooks for the Name and click on Try it to use the sample Git repository.

Before creating the application, first click on Show advanced routing, build and source options. Scroll down to the Deployment Configuration section.

Add a new environment variable JUPYTER_NOTEBOOK_PASSWORD and supply a password as its value.

Now select on Create and the Jupyter Notebook instance will be deployed, including the sample notebooks from the Git repository, with any Python packages they require which were listed in the requirements.txt file in the Git repository, being automatically installed.

Loading the Image Stream

We went through the steps to add the annotations to the Image Stream step by step. The quicker way of managing this is to instead create a file which contains the Image Stream definition. This file can then be loaded using the oc create command.

The Image Stream definition as shown above is provided as part of the Git repository for the Jupyter Notebook image. To load it into a new project, you can run the command:

oc create -f https://raw.githubusercontent.com/getwarped/s2i-minimal-notebook/master/image-streams.json

Jupyter Notebook should now show under the Python category in the application catalog of the web console and you can deploy it.

Custom Application Templates

In this post we used the ability of OpenShift to add annotations to an Image Stream to have the S2I builder image be added to the application catalog. Because this used a generic mechanism, it meant that in order to supply the password for the Jupyter Notebook instance, it was necessary to drill down into the advanced options and add an environment variable.

An alternative to using this approach is to create an application template. Using an application template you can customise what options are presented to a user and which would need to be filled in to deploy the application. Under the covers the template can also control exactly how the underlying resources are set up and configured.

Use of application templates can therefore simplify the process further and make it easier in many cases for a user wanting to deploy an application.

Although this is now the last post in the current series of posts about running Jupyter Notebooks on OpenShift, I will revisit the use of application templates in a future series of posts. In addition to explaining how to create an application template, I will also delve into topics such as setting up readiness and liveness probes, and persistent volumes as part of the resources created from the application template.

All Posts in this Series

The full list of posts in this series of posts on running Jupyter Notebooks on OpenShift were as follows.

Jupyter on OpenShift: Using OpenShift for Data Analytics – Introductory post which introduced the series and provided a quick tour of deploying a Jupyter Notebook on OpenShift.

Jupyter on OpenShift Part 2: Using Jupyter Project Images – Showed how the Jupyter Notebook images from the Jupyter Project can be deployed on OpenShift.

Jupyter on OpenShift Part 3: Creating a S2I Builder Image – Showed how the Jupyter Project images can be Source-to-Image (S2I) enabled, allowing them to be run against a Git repository to bundle notebooks and data files, as well as install required Python packages.

Jupyter on OpenShift Part 4: Adding a Persistent Workspace – Showed how to add a persistent volume and automatically transfer notebooks and data files into it so work is saved.

Jupyter on OpenShift Part 5: Ad-hoc Package Installation – Showed how to deal with ad-hoc package installation and moving the Python virtual environment into the persistent volume.

Jupyter on OpenShift Part 6: Running as an Assigned User ID – Showed how the S2I enabled image can be modified to allow it to run under the default security policy of OpenShift.

Jupyter on OpenShift Part 7: Adding the Image to the Catalog – The current and final post.

Categories
OpenShift Ecosystem, OpenShift Origin, Python
Tags
, ,
  • The links to parts 1-4 and 6 are broken :(

    • spurtell

      Fixed. Thanks!