OpenShift Container Platform 3.6: Streamlined Installation and Configuration of Red Hat Gluster Storage for Containers

(Republished from the original blog post by Erin Boyd, Jose Rivera, and Scott Creeley at redhatstorage.redhat.com.)

Did you ever get a new, exciting toy only to have that excitement squashed by the phrase “Batteries not included”?

With the introduction of Red Hat OpenShift Container Platform 3.6, no longer will customers have to wait or jump extra hurdles to get resilient, persistent storage with their new installations. Now they can more easily deploy Red Hat Gluster Storage ready for use by their containerized applications – this is PaaS with batteries included!

With the release of Red Hat OpenShift Container Platform 3.6, users will have the convenience of using a single tool to use Red Hat Gluster Storage as either Container-Native Storage (CNS) or Container-Ready Storage (CRS) alongside the rest of their OpenShift installation. As part of the OpenShift Advanced Installation users can specify two new storage options: Red Hat Gluster Storage for hosted registry storage or for general application storage. To facilitate evaluation of these, an Openshift Container Platform evaluation subscription now includes Red Hat Gluster Storage evaluation binaries and subscriptions.

The following is an example inventory file that would be used with an OpenShift Container Platform Advanced Installation which deploys two Container Native Storage clusters for both hosted registry storage and general application storage.

[OSEv3:children]
masters
nodes
glusterfs_registry
glusterfs

[OSEv3:vars]
ansible_ssh_user=root
openshift_master_default_subdomain=cloudapps.example.com
openshift_deployment_type=openshift-enterprise
openshift_hosted_registry_storage_kind=glusterfs
openshift_disable_check=disk_availability,memory_availability

[nodes]
master1 node=True storage=True master=True openshift_schedulable=False
node1 node=True storage=True openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True
node2 node=True storage=True openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True
node3 node=True storage=True openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True
node4 node=True storage=True openshift_schedulable=True
node5 node=True storage=True openshift_schedulable=True
node6 node=True storage=True openshift_schedulable=True

[glusterfs_registry]
node1 glusterfs_devices="[ '/dev/xvdc' ]"
node2 glusterfs_devices="[ '/dev/xvdc' ]"
node3 glusterfs_devices="[ '/dev/xvdc' ]"

[glusterfs]
node4 glusterfs_devices="[ '/dev/xvdc' ]"
node5 glusterfs_devices="[ '/dev/xvdc' ]"
node6 glusterfs_devices="[ '/dev/xvdc' ]"

[masters]
master1 node=True storage=True master=True openshift_schedulable=False

Let’s go over the bold portions in detail.

The first section defines the host groups the installation will be using. We’ve defined two new groups: glusterfs_registry and glusterfs. The first specifies a cluster that will host a single volume for use exclusively by a hosted registry. The second specifies a cluster for general application storage and will, by default, come with a Storage Class to enable dynamic provisioning.

[OSEv3:children]
masters
nodes
glusterfs_registry
glusterfs

In the following section, we indicate that we want the hosted registry to use Red Hat Gluster Storage for its storage needs.

[OSEv3:vars]
ansible_ssh_user=root
openshift_master_default_subdomain=cloudapps.example.com
openshift_deployment_type=openshift-enterprise
openshift_hosted_registry_storage_kind=glusterfs
openshift_disable_check=disk_availability,memory_availability

In the [nodes] section, we need to specify all nodes in the OpenShift Container Platform cluster. For our installation, we also need to specify which nodes will run pods for the hosted registry. This is done by specifying "openshift_node_labels="{'region': 'infra'}"" for each such node. It is recommended to have at least three nodes running your hosted registry.

[nodes]
master1 node=True storage=True master=True openshift_schedulable=False
node1 node=True storage=True openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True
node2 node=True storage=True openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True
node3 node=True storage=True openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True
node4 node=True storage=True openshift_schedulable=True
node5 node=True storage=True openshift_schedulable=True
node6 node=True storage=True openshift_schedulable=True

Now we get to our new sections where we specify the nodes that will be used for storage. Container Native Storage and Container Ready Storage require that each cluster have a minimum of three nodes. Multiple clusters can not share a given node. Since we are deploying two clusters, we need to specify six nodes total. It is also required that each node have at least one dedicated, bare storage device (no data or formatting of any kind) for exclusive use by Red Hat Gluster Storage.

Our first new section is [glusterfs_registry]. Here we specify the nodes of the Red Hat Gluster Storage cluster and the storage devices on those nodes that will be used for a hosted registry’s storage. It is not required that these nodes be the same as the ones running the hosted registry.

[glusterfs_registry]
node1 glusterfs_devices="[ '/dev/xvdc' ]"
node2 glusterfs_devices="[ '/dev/xvdc' ]"
node3 glusterfs_devices="[ '/dev/xvdc' ]"

Our second new section, [glusterfs], is used for specifying the Red Hat Gluster Storage cluster and storage devices that will be used for general application storage. These storage devices must also be for exclusive use by Red Hat Gluster Storage. As mentioned, these nodes may not also be part of the cluster used by [glusterfs_registry]. In the case of Container Native Storage, it is not required that these nodes be dedicated exclusively to serving storage; Container Native Storage pods can coexist with other application pods.

[glusterfs]
node4 glusterfs_devices="[ '/dev/xvdc' ]"
node5 glusterfs_devices="[ '/dev/xvdc' ]"
node6 glusterfs_devices="[ '/dev/xvdc' ]"

Once the installer is complete, the user can see the pre-defined Storage Class by executing:

glusterfs-storage   kubernetes.io/glusterfs

This Storage Class can be used for applications by specifying a Persistent Volume Claim to dynamically provision the required storage volume:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: mypvc
 namespace: glusterfs
spec:
 accessModes:
 - ReadWriteOnce
 resources:
   requests:
     storage: 100Gi
 storageClassName: glusterfs-storage

And that’s it, your PaaS solution with built-in storage is ready to go! If you want to tune the installation further there are more options available in the Advanced Installation and a demo video is available here.

Categories
News, OpenShift Container Platform
Tags
, ,