Automating OpenShift application DNS with external-dns and Red Hat’s Satellite 6 server

The configuration of DNS for applications in OpenShift Container Platform is largely handled by pre-configured wildcard zones that point to the routers in the OCP infrastructure. In the event that an application lands on an external IP for Ingress traffic or load balancer, DNS has to be configured for the endpoints. Endpoints are added or taken away as an application lifecycle changes. This creates some administrative overhead for DNS administration.

Red Hat’s Satellite 6 provides an all-in-one solution for managing your system’s lifecycle. This myriad of software includes the option of installing BIND for DNS services. This article will cover the use of BIND and ExternalDNS to automate the creation and removal of DNS records for external OCP services. ExternalDNS’s GitHub describes itself, “ExternalDNS synchronizes exposed Kubernetes Services and Ingresses with DNS providers.”

Bind uses RFC2136 for dynamic updates in DNS. Satellite 6 BIND has a pre-configured RNDC key for use with updating zone files.  The documentation page provides more information on how to configure RNDC for dynamic updates from start to finish.

 

BIND DNS Configuration

The file named.conf is the first configuration to examine for configuration of dynamic updates in DNS:

# cat /etc/named.conf
include "/etc/rndc.key";

controls {
inet 10.x.y.2 port 953 allow { 10.x.y.151; } keys { "rndc-key"; };
};
options {
include "/etc/named/options.conf";
};

include "/etc/named.rfc1912.zones";

// Public view read by Server Admin
include "/etc/named/zones.conf";

Open the files mentioned in the primary named.conf file.

The rndc key listed below will be the secret, name and algorithm provided to ExternalDNS:

# cat /etc/rndc.key
key "rndc-key" { (1)
algorithm hmac-md5; (2)
secret "T7wgNa7I1zL7oXSdd/4Pgg=="; (3)
};
1 The name of the key to reference
2 Algorithm type of the key
3 TSIG Secret

The options file enables dnssec for updates:

# cat /etc/named/options.conf

..output omitted..
dnssec-enable yes;
dnssec-validation yes;
..output omitted..

Lastly, the permissions to allow updates with said key are allowed in the zones.conf file:

# cat /etc/named/zones.conf

..output omitted..
zone "example.com" {
type master;
file "/var/named/dynamic/db.example.com";
update-policy {
grant rndc-key zonesub ANY;
};
};
..output omitted..

OpenShift Container Platform Configuration

Once the proper configuration of DNSSEC is enabled, the focus turns to ExternalDNS and setting up ingress traffic to external IP addresses on the OCP cluster in question. The OpenShift Container Platform documentation describes how to assign unique External IPs for Ingress traffic. Most of the cloud providers provide a facility to create Load Balancer services on their platform. This feature is only supported in a non-cloud deployment. The addresses assigned to the Ingress traffic needs to be configured in ipfailover.

The following command configures this:

# oc adm policy add-scc-to-user privileged -z ipfailover
# oc adm ipfailover --selector="beta.kubernetes.io/os=linux" \
--interface=ens192 --virtual-ips="10.x.y.249-253" \
--watch-port=80 --replicas=5 --create

On the master nodes in the cluster, the following options should be defined:

# cat /etc/origin/master/master-config.yaml

..output omitted..
networkConfig:
ingressIPNetworkCIDR: 10.x.y.248/29
..output omitted..

The AWS documentation for External-DNS provides a template for a POD deployment file:

# vi manifest.yml
apiVersion: v1
kind: ServiceAccount
metadata:
name: external-dns
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: external-dns
rules:
- apiGroups: [""]
resources: ["services"]
verbs: ["get","watch","list"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get","watch","list"]
- apiGroups: ["extensions"]
resources: ["ingresses"]
verbs: ["get","watch","list"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["list"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: external-dns-viewer
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: external-dns
subjects:
- kind: ServiceAccount
name: external-dns
namespace: default
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: external-dns
spec:
strategy:
type: Recreate
template:
metadata:
labels:
app: external-dns
spec:
serviceAccountName: external-dns
containers:
- name: external-dns
image: registry.opensource.zalan.do/teapot/external-dns:latest
args:
- --source=service
- --provider=rfc2136
- --rfc2136-host=10.x.y.2
- --rfc2136-port=53
- --rfc2136-zone=example.com
- --rfc2136-tsig-secret='T7wgNa7I1zL7oXSdd/4Pgg=='
- --rfc2136-tsig-secret-alg=hmac-md5
- --rfc2136-tsig-keyname=rndc-key
- --rfc2136-tsig-axfr

# oc create -f manifest.yml

The relevant sections from the DNS configuration have been populated above. Next, create an example service that can be added to test DNS resolution and the record creation.

# example.yaml

apiVersion: v1
kind: Service
metadata:
name: hello-openshift
annotations:
external-dns.alpha.kubernetes.io/hostname: hello-openshift.example.com
spec:
type: LoadBalancer
loadBalancerIP: 10.x.y.252
ports:
- port: 8080
name: http
targetPort: 8080
selector:
app: hello-openshift

---

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: hello-openshift
spec:
template:
metadata:
labels:
app: hello-openshift
spec:
containers:
- image: docker.io/openshift/hello-openshift
name: hello-openshift
ports:
- containerPort: 8080
name: http

# oc create -f example.yml

Now, if the DNS server is monitored, the record is created and can be queried:

# watch dig @10.x.y.2 hello-openshift.example.com +short
10.x.y.252
# curl hello-openshift.example.com:8080
Hello OpenShift!

Lastly, clean up the environment and the record is removed from DNS:

# oc delete deployment.apps/hello-openshift
# oc delete service hello-openshift
# watch dig @10.x.y.2 hello-openshift.example.com +short

Alternatively, OCP external IP addresses can be used in this exercise as well. This will allow the use cloud providers and the administration of the external IP address is left to the user. On the master nodes:

On the master nodes in the cluster, the following options should be defined:

# cat /etc/origin/master/master-config.yaml

..output omitted..
networkConfig:
externalIPNetworkCIDRs: 10.x.y.248/29
..output omitted..

The service example will differ slightly:

apiVersion: v1
kind: Service
metadata:
name: hello-openshift
annotations:
external-dns.alpha.kubernetes.io/hostname: hello-openshift.example.com
spec:
externalIPs:
- 10.x.y.252
ports:
- port: 8080
name: http
targetPort: 8080
selector:
app: hello-openshift

This article has highlighted some functionality combining ExternalDNS and Red Hat’s Satellite 6 server. This functionality automates some typical DNS overhead in creating external service records for OpenShift administrators. In this example RFC2136, or dynamic updates in BIND server were utilized. ExternalDNS is an extensible solution that can cover many different DNS providers, see the documentation for more information about its uses.