Routes

This blog post explains how to configure the Openshift router in order to adapt it to specific needs.

Intended audience : Openshift users and operators.

By Jacques Roussel, Cloud Consultant @Objectif Libre

How to customize the Openshift router

While using Openshift, we create lots of routes. These routes, made for web apps, are used to expose internal services outside of the cluster. Behind these routes, there are one or many haproxy pods, with fully automated and managed configuration. But the features are limited. For example you cannot forbid access to some specific path – which can be disappointing: knowing that haproxy is there, you may want to use it fully.

Let’s say we use WordPress for a blog deployed on Openshift: we would probably want to forbid the access to the wp-admin pages from the public access.

This blog post will show you how to do this, and we’ll see how powerful it can be to standardize the practices between projects.

 

Setup your environment

First of all, we need an Openshift. We will use minishift for simplicity.

$ sudo curl -L https://github.com/dhiltgen/docker-machine-kvm/releases/download/v0.7.0/docker-machine-driver-kvm -o /usr/local/bin/docker-machine-driver-kvm
$ sudo chmod +x /usr/local/bin/docker-machine-driver-kvm
$ sudo apt install libvirt-bin qemu-kvm
$ sudo usermod -a -G libvirtd $(whoami)
$ newgrp libvirtd
$ cd /tmp
$ wget https://github.com/minishift/minishift/releases/download/v1.13.1/minishift-1.13.1-linux-amd64.tgz
$ tar -xf minishift-1.13.1-linux-amd64.tgz
$ sudo mv minishift-1.13.1-linux-amd64/minishift /usr/local/bin/minishift
$ minishift version
$ minishift start --memory 4GB
$ eval $(minishift oc-env)

Create a testing project

For our example, we create a project with some httpd pods, one service and one route.

$ oc login -u system:admin https://$(minishift ip):8443
$ oc new-project blog-article
$ oc run --image=registry.access.redhat.com/rhscl/httpd-24-rhel7 --replicas=2 httpd
$ cat <<EOF | oc create -f -
apiVersion: v1
kind: Service
metadata:
 name: httpd
spec:
 ports:
   - name: 8080-tcp
     port: 8080
     protocol: TCP
     targetPort: 8080
 selector:
   run: httpd
EOF
$ oc expose service httpd
$ curl -I httpd-blog-article.$(minishift ip).nip.io/admin
HTTP/1.1 404 Not Found
Date: Wed, 11 Apr 2018 12:38:42 GMT
Server: Apache/2.4.27 (Red Hat) OpenSSL/1.0.1e-fips
Content-Type: text/html; charset=iso-8859-1
Set-Cookie: a3fe51630340065f682306a4f6d5c314=a50715fc19b88e07724c5dc4309782aa; path=/; HttpOnly

Note that the deployment of the images can take one or two minutes.

Downloading and customizing the default OpenShift template

We want to forbid the access to the “/admin” path, using a specific annotation. To achieve this, we have to download the template of the router. Next, we customize it so that if a regexp is matched, haproxy will return a 403 error. Finally, we configure the router to use the new template.

Download the template

oc -n default rsh $(oc -n default get po -l router=router --template="{{ (index .items 0).metadata.name }}") cat haproxy-config.template > haproxy-config.template

Customize the template

To customize the template, we edit the file haproxy-config.template in order to modify its behavior. We add after the line 332 (mode http) these four lines:

  {{- if (ne (index $cfg.Annotations "haproxy.router.openshift.io/forbidden_regexp") "") }}
  acl forbidden_regexp path_reg {{ index $cfg.Annotations "haproxy.router.openshift.io/forbidden_regexp" }}
  http-request deny if forbidden_regexp
  {{- end}}

Explanations:
If the annotation “haproxy.router.openshift.io/forbidden_regexp” is not empty, we insert in the final haproxy configuration file the following lines:

“acl forbidden_regexp path_reg {{ index $cfg.Annotations “haproxy.router.openshift.io/forbidden_regexp” }}” -> here we insert the regexp set in the annotation and define the variable forbidden_regexp if the path is matched when a request arrive.
http-request deny if forbidden_regexp -> These line tells haproxy to reject the request if the variable forbidden_regexp exists.

Use the new template

Finally we must update the router configuration to make it use the new template:

$ oc create configmap customrouter --from-file=haproxy-config.template -n default
$ oc set env dc/router TEMPLATE_FILE=/var/lib/haproxy/conf/custom/haproxy-config.template -n default
$ oc volume dc/router --add --overwrite --name=config-volume --mount-path=/var/lib/haproxy/conf/custom --source='{"configMap": { "name": "customrouter"}}' -n default

That’s it! We can now use our new annotation.

Using our new annotation

$ cat <<EOF |oc create -f -
apiVersion: v1
kind: Route
metadata:
 annotations:
   openshift.io/host.generated: "true"
   haproxy.router.openshift.io/forbidden_regexp: "^/admin$"
 name: httpd2
spec:
 host: httpd2-blog-article.$(minishift ip).nip.io
 port:
   targetPort: 8080-tcp
 to:
   kind: Service
   name: httpd
EOF
$ curl -I httpd-blog-article.$(minishift ip).nip.io/admin
HTTP/1.1 404 Not Found
Date: Wed, 11 Apr 2018 12:38:42 GMT
Server: Apache/2.4.27 (Red Hat) OpenSSL/1.0.1e-fips
Content-Type: text/html; charset=iso-8859-1
Set-Cookie: a3fe51630340065f682306a4f6d5c314=a50715fc19b88e07724c5dc4309782aa; path=/; HttpOnly
$ curl -I httpd2-blog-article.$(minishift ip).nip.io/admin
HTTP/1.0 403 Forbidden
Cache-Control: no-cache
Connection: close
Content-Type: text/html

Conclusion

It’s very simple to customize the OpenShift router to change its behavior.

You still have to be careful: to much customization could be very hard to maintain.

The router has to stay generic, focusing on customizations that can be used by many projects.