GearD: The Intersection of PaaS, Docker and Project Atomic

When it comes to the technology available to leverage in a Platform as a Service (PaaS), this is an amazing time. PaaS platforms such as OpenShift are usually at the intersection of developers and operations to establish patterns that make both parties efficient. Practically, this means that PaaS platforms are tightly integrated with both the development tooling and the operating system itself. And in the operating system, there is a lot of change underfoot!

The operating system change is occurring for a few reasons. The first dates back to the introduction of Infrastructure as a Service (IaaS) and cloud. IaaS started changing the way we think about workloads and operating system instances. As they became more elastic and developers began designing their applications around that elasticity. They didn’t assume an operating system instance or host would just run forever.

Then Linux containers came along and amplified the expectations around elasticity. You didn’t need to spin up a full operating system instance per application anymore, like you did with virtual machines, but instead you could spin up lightweight containers. You could run lots of them on the same host, they booted faster than VMs, used resources more efficiently and provided good-enough isolation.

The trouble was that containers and the applications built on them were now a lot more elastic than the underlying hosts they ran on. So we started asking ourselves, what if we made the hosts just as minimal, dynamic and elastic as the containers they ran?

Project Atomic

Introducing Project Automic by Red Hat Picture

Enter: Project Atomic. This project redefines the way people think about the operating system. It’s a minimal, lightweight host tuned to run Linux containers. Its updates are applied atomically in one operation and allow you to rollback to older versions if necessary. But it also introduces next-generation process management, or init system, with systemd. The init system in the operating system is the boss – it controls everything running on the machine.

But the traditional init systems didn’t really help your typical PaaS. They started and stopped processes, but all the mechanics like setting up security contexts and Linux control groups (cgroups) for processes had to be done manually. Even monitoring processes to see if they were still running fell on the shoulders of the PaaS platform to implement. Up until this point, the PaaS itself provided the process management for the operating system.

That’s all changed with systemd. It not only starts and stops processes but does lots of the low level integration with other Linux subsystems that the PaaS used to do. SELinux and cgroups are setup out of the gate with API’s to change them easily. It also keeps track of those processes to keep them running and even start them on-demand if needed. All this is handled at the heart of operating system which makes it as efficient as possible. Now we can throw away the workarounds in the PaaS that we’ve been doing for years and depend directly on the operating system.

Docker

On top of that change, Linux containers themselves are evolving. Docker is a Linux Container technology that introduced a well thought-out API for interacting with containers and a layered image format that defined how to introduce content into a container. It is an impressive combination and an open source ecosystem building around both the images and the Docker API.

With Docker, developers now have an easy way to leverage a vast and growing amount of technology runtimes for their applications. A simple ‘docker pull’ and they can be running a Java stack, Ruby stack or Python stack very quickly.

But how to answer some of the still some looming questions like:

  • What is the best way to deploy my application on that container I just pulled down?
  • How do I connect a lot of containers to form an application?
  • How do I solve all the network challenges to make those containers appear as a single application to the outside world?
  • When I created that container, did it setup all the things that my sysadmin would expect?

As powerful as the technologies were, there weren’t established patterns yet on how to solve these questions.

GearD

OpenShift Origin GearD Introduction Picture

This is where GearD comes in. While these new building blocks are exceedingly capable and exciting in their own right, most developers and sysadmins just want to leverage their benefit as quickly as possible. GearD provides a bridge from the developer services needed in a PaaS to the next generation technology in the operating system. It brings together Docker and the capabilities in the Atomic stack, granting developers access to that ecosystem of content and images so they can easily deploy it the way their operations team expects.

To start, GearD will focus on these use cases:

  1. Converting application sources to Docker images
  2. Providing elastic container wiring
  3. Providing a sysadmin friendly setup

Source to Image

Project Atomic Docker to Source Images Picture

The first use case, source to image, is critical to developers. Developers tend to focus on their application and its source code more than they focus on the mechanics of packaging that application into an RPM, image, or other delivery format. To developers, the container image is just the delivery format.

GearD allows developers to simply point at their application source in a repository like GIT and a Docker image to get a newly built Docker image that runs the application. This is process is critical for responding quickly and easily to changes anywhere in the stack. Now you can easily keep your images in sync with changes in dependent images or changes in the application itself. It also enables the developer to focus on application code without worrying about the details of deploying a new or updated application container.

Elastic Container Wiring

Project Atomic Elastic Wiring Picture

The next use case is around elastic container wiring. A typical enterprise application consists of multiple application components, that will typically be deployed across multiple containers. When your application is composed of many elastic containers, you still need to be able to communicate across containers, even if those containers are on separate hosts.

A classic use case is an application server connecting to a database. They are most likely in separate containers and need to either have access to the other container’s IP address or domain name. However both of those provide complications. DNS-based addressing introduces some complexities by requiring a very dynamic DNS system which isn’t as common as one might think, especially at high scale. However, hard coding IP dependencies is problematic as well since containers can move from host to host.

Project Atomic Elastic Wiring Connected Picture

To solve this, GearD introduces a ip-based container linking framework. This allows the container to talk to a specific ipaddress that is remapable by an operator using GearD. In the above use case, this allows your application server to talk the database on an IP address and port that (from its perspective) never changes (e.g. 127.0.0.1:3306). However, that IP address can be re-routed at the GearD location to another location. This allows the developer to always interact with a well known address in the container while the operations team can map those locations to the appropriate address depending on the environment or other factors.

Project Atomic Elastic Wiring Solution Picture

This provides a powerful abstraction to keep containers consistent across environments while letting operations work with an iptables-based approach that lets them leverage a vast array of networking technologies with which they are already familiar. It’s also complementary with other container linking technologies so it doesn’t force you down a restrictive path.

The Developer is King. So is the Sysadmin.

We’ve always believed that development and operations have to work as a team for the best results. This is also what’s driven so much interest in DevOps in general. PaaS is all about developer efficiency, but it’s critical to provide that without causing problems for the sysadmin. If you gain development efficiency but lose operational efficiency, how much have you really gain?

To solve this, GearD tightly integrates with both Docker and operating system technology such as systemd. We know that some operations shops will want to build their tooling and practices around the Docker daemon and API. We want to make sure that works. At the same time, we know that some shops will build their tooling around systemd. We want to make sure that works as well.

GearD Sysadmin Integration

When GearD creates an application, it builds the plumbing to setup the containers via Docker and also create the necessary structures in systemd, such as unit files. This gives visibility to both systems. It also makes sure that logging for containers is setup in a way that is visible through both Docker and the systemd journalctl system. This setup let’s sysadmins take advantage of a vast array of capabilities within systemd without losing any capabilities that Docker brings to the table. It’s the best of both worlds.

How do GearD and OpenShift Interact?

OpenShift Origin GearD Integration Picture

GearD provides the foundation for the next generation of Gears (i.e. Linux containers) in OpenShift. By leveraging Docker, GearD taps into a large eco-system of existing content, bringing a vast new set of runtime choices for developers to use in their applications. And by leveraging systemd, it immediately benefits from the next generation process management. But much of the power of GearD comes in the flexibility of how it can be leveraged.

For OpenShift, GearD can simply operate behind an OpenShift Broker, which provides centralized orchestration, allowing OpenShift to leverage all the capabilities of GearD in a more automated fashion. The OpenShift CLI and API stay the same, the tooling integration stays the same, and the capabilities you know and love from OpenShift are still there, just on a new technical foundation. The architectural versatility of this new foundation will allow more data centers to experience OpenShift than ever before. Operations now has a new choice in how much of a PaaS framework they want to consume.

How do I get involved?

GearD is a component of OpenShift Origin. Naturally we welcome anyone who would like to participate. Learn more about GearD here. If you’d like to get involved, join us on the mailing lists or IRC at OpenShift Origin. Try it out, hack around, and let us know what you think!

Summary

At the end of the day, it’s times like this when a PaaS really shines. There is a tremendous amount of change occurring, which has the potential to make things run better and more efficiently, if you can harness it. OpenShift and GearD do that work for you, letting you take advantage of the technology while keeping your focus on developing your applications.

Categories
OpenShift Enterprise, OpenShift Online, OpenShift Origin, Thought Leadership
Tags
, , , ,
Comments are closed.