Managing Elasticsearch with Kopf in OpenShift Origin 3.6

Kopf is a popular web administration tool for Elasticsearch v2. It allows administrators to view, diagnose, and modify Elasticsearch clusters, both operationally and for managing the data. It offers three main views of Elasticsearch:

• The cluster allows an admin to see the overall state of nodes in the cluster, and the state of the indexes located on those nodes.
• The node provides views of average load, CPU/memory/disk usage, and uptime, and also its IP address and name.
• Rest provides convenient access to various HTTP REST APIs of Elasticsearch.

Various cluster, node, and index settings are available for display and modification. Besides the ability to view, delete, and create indices, administrators can use the tool to disable shard allocation with the click of a button. This is particularly useful when temporarily shutting down nodes for maintenance as it prevents unnecessary data transfer between nodes.

Here is the Node view:
 

Here is the Cluster view:
 

With the release of OpenShift Origin 3.6, we can show how to use Kopf to manage Elasticsearch using Origin's Aggregated Logging feature. If you're not using the Origin Aggregated Logging stack already, the project's documentation will help you get set up.

Installation

The Aggregated Logging method of installing Kopf is via an httpd-based image. This allows Kopf to be deployed and undeployed as desired without impacting the cluster. To make the deployment as easy as possible, an OpenShift template is provided. The template requires one parameter, KOPF_SECURE_URL, which is the URL under which Kopf will be exposed to web clients.

Securing Kopf

Kopf mounts Elasticsearch admin credentials at runtime, which are necessary for carrying out most of the actions Kopf allows administrators to perform. Therefore access has to be properly secured. To support this, the OpenShift oauth proxy is used as a side-car container along side a Kopf container in the same pod. One of the essential features of pods is that containers within one pod share the same Linux namespace settings. This is leveraged by exposing only the proxy port outside the Pod and all allowed requests are then proxied to Kopf via localhost. Only requests with a valid oauth token are allowed. To obtain a valid token, a user is redirected to the OpenShift login site where they present their credentials. Only users with the cluster-admin role are allowed to access Kopf.

Kopf and oauth proxy run in the same net namespace, thus only the proxy port is exposed:
 

Deployment Instructions

$ oc create -f kopf-template.yaml

$ oc new-app --template=kopf-template -p KOPF_SECURE_URL=kopf.example.com

Note: Enter the URL without the https:// prefix.

Deprovision

When done, deprovision Kopf by scaling its DeploymentConfig to 0 replicas. This way you can easily come back later and scale Kopf back up.

oc scale dc logging-es-kopf --replicas=0

To completely remove all of the Kopf resources, you can take advantage of labels that are associated with every resource:

oc delete all -l logging=logging-es-kopf ; \

oc delete oauthclient kopf-proxy ; \

oc delete cm logging-es-kopf-config logging-es-kopf-run

Troubleshooting Kopf

If you need to learn about what’s going on in Kopf or the associated oauth proxy container, use these commands. Note that Kopf and proxy run within the same pod, so you also need to tell OpenShift, which container you want to get logs from:

oc describe pod

oc logs -c logging-es-kopf

oc logs -c logging-es-kopf-auth-proxy

Kopf is an amazing tool and provides great insight into the Elasticsearch cluster. However, it can do what we want it to do only because is has unconstrained access to the cluster and some of the actions it can do might result in corruption of the data. Remember: The pod mounts the Elasticsearch admin certificates. And as we all know, with great power comes great responsibility.