I wanted to give a little insight as to the type of security automation that happens in the background of OpenShift. As a provider, it’s always a little scary to talk about what is behind the scenes or isn’t. I have blogged in the past about OpenShift’s use of cgroups, poly-instantiation and SElinux. There are many great web pages that explain what each offers to a multi-tenancy platform so I will dig in on the other non publicized tools. If you are building an OpenShift Origin infrastructure, this would be a good addition to your build out.
In a world of agile development and the ever changing layered build, one must really be careful that security remains at the level that the policy and product demands. At the end of this post, you will have tools that you can implement to help assure security controls stay within your specified policy.
With developers and operations staff that can change a layered build real time, how do you assure that it is still in a safe and secured state since it’s impossible to keep up with every code check in? The biggest thing that comes to mind for me is automation. You will need tools that can check your security policy across all the various instances.
The heart of policy checking in OpenShift is OpenScap (pronounced S-Cap). SCAP stands for Security Content Automation Protocol. There are other SCAP type tools that can be used in this spot but OpenScap was chosen for its open source roots. It is a tool that allows you to define your security policy into a list of automated checks that can be ran at specific times or actions. You will want to define a rule for every line item in your security policy. If your policy contains items that are not already defined, you can create custom checks with scap-workbench-editor or the Script Check Engine (SCE) using any script-able language that you are comfortable with. Creating the custom checks will be covered in a future blog.
Here’s how to get OpenScap working on RHEL6 with the default policy.
Install the RPMs that are needed
$ yum install openscap-utils openscap-content openscap-python openscap openscap-extra-probes
If you want to add your OpenScap logs to syslog – this is an optional step.
$ yum install html2text util-linux-ng
Create a run file that can be called from what ever action/run hook you decide and chmod it executable. The run hook can be a simple crontab entry like: 0 */3 * * * /usr/local/bin/openscap.sh
#!/bin/sh # 01-openscap - A shell script to run openscap via CRON # Run xccdf profile tests and create technical punch list /usr/bin/oscap xccdf eval --profile RHEL6-Default --results /var/log/xccdf-results.xml /usr/share/scap-rhel6-xccdf.xml > /dev/null 2>&1 # Generate xccdf management report with nice HTML view /usr/bin/oscap xccdf generate report /var/log/xccdf-results.xml &> /var/log/report-xccdf.html ## If you added the html2text and util-linux-ng rpm's, you can do the following steps to put the file into your remote syslog server. # If you added html2text rpm - create rsyslog'able text file from html /usr/bin/html2text -o /var/log/openscap_rsyslog.txt /var/log/report-xccdf.html # If you added html2text rpm - make the openscap_rsyslog.txt file a summary only file - no need to store the whole file in syslog /bin/sed -n '/Result for Red Hat GPG Keys are Installed/q;p' /var/log/openscap_rsyslog.txt > /var/log/openscap_summary.txt # If you added the util-linux-ng rpm - push openscap_summary.txt into syslog /usr/bin/logger -t openscap
OpenScap outputs .xml and .html, neither of which look good in a remote system log. If your goal is to have a historical trend of all your nodes, you can convert the report .html output with something like html2text and push it into the remote syslog with a command like logger.
There will be several output files that you might find useful.
- /var/log/xccdf-results.xml is a technical report on how to fix the resulting issues.
- /var/log/report-xccdf.html is a management type .html report with a score graph.
- /var/log/openscap_rsyslog.txt is a text version of the report-xccdf.html (if you installed html2text).
The first time I used OpenScap, I was amazed at the security issues list that was generated and needed to be fixed on the first run. Once you get all of your fixes into the layered build and it runs 100% clean, you can begin to look into how to automate the runs. The frequency is really a risk exercise on how often you want to see any items that have changed.
Our policy takes approximately fifteen minutes to complete on each instance and can be nice'd or throttled to an acceptable level and ran every two to three hours. You can also tie into a git hook or yum install that will trigger a run as well. You can decide on the frequency that best suites your environment and risk level.
If you use monitoring tools, and you should really be using monitoring tools, you can have a check that looks at the value of the “urn:xccdf:scoring:flat” output. If that value is ever lower than expected, the monitoring system can email, sms, etc to alert someone to what it has found so that action can be taken right away.
- Define your security policy into individual checks.
- Do your first test run and fix issues in the layered build until the report is clean.
- Decide how and when you want the checks to be processed.
- Automate the checking process.
- Do the management by leveraging the monitoring system and remote syslog.
OpenScap will assure policy compliance and auditing automation and the platform is still within a certain level of expected security. There are many other tools that will also help but since the audit team will come some day, it's best to have this done early.
If you want security to be a core competency of your PaaS provider – it will be hard to find a provider better than Red Hat. And as a developer, you can focus on what's most important to you. Your applications and your customers.
For more information please look at: