Part 1: Organization and Culture
Red Hat customers and employees have come to expect continuous innovation. We strive to introduce new digital services in days—not weeks or months. Our mantra is rapid service delivery.
Moving to OpenShift Container Platform in 2016 laid the foundation for faster cycle times. But transforming service delivery requires more than a platform. Our journey has included three overlapping stages:
- Changing our culture.
- Rethinking the application lifecycle.
- Evolving our infrastructure.
This post describes the changes we’ve made to our organization and culture.
Creating a Rapid Digital Delivery Team
Our IT organization used to be subdivided into separate teams (applications, infrastructure, and so on), each with separate goals. We knew that faster innovation would require a more collaborative culture. We had to shift the culture to become one team with one goal: Delivering digital solutions to run Red Hat’s business. No more silos.
To that end, we’re currently reorganizing our more than 150-member digital solutions delivery team. We’ve completed the first step: Consolidating our application delivery, infrastructure services, and architecture teams. Now they all operate under one umbrella: Digital Solutions Delivery. We’ve also embedded Platform-as-a-Service (PaaS) specialists within the application team. The resulting Rapid Digital Delivery team has the full-stack skills we need to develop new delivery models.
Shifting from Project Teams to Product Teams
With product teams, a full-stack engineer might work on a storage team for one launch, and a systems automation team for another. A developer who usually works on corporate search applications might sometimes be assigned to onboard other types of applications.
Product teams provide two advantages. One advantage is that it helps us standardize our IT offerings. The other advantage is that it exposes team members to more projects. That’s good for the company because it gives us a deeper bench. It’s also good for employees it provides career mobility, supporting Red Hat’s “People First” philosophy.
Acquiring New Skill Sets
Our new delivery model goes beyond PaaS and beyond DevOps. It’s also about automation everywhere, and about working side by side with our business partners to meet their objectives. That means team members need a solid understanding of Red Hat’s business.
As a result, we now focus on different skills when we hire and train. Before, we hired systems administrators and release engineers to validate and deploy applications. Now we recruit full-stack engineers who understand the business. Their job is to integrate applications with enterprise data services and our infrastructure management framework. Developer skills have also changed. They now need to understand how applications behave with ephemeral infrastructure and fully automated CI/CD pipelines.
We’re taking several approaches to building skills:
- Training existing team members by giving them hands-on experience with Red Hat products.
- Looking for full-stack engineering skills when we hire.
- Reducing the need for scarce skills by increasing automation and self-service portals. We’re doing more under the hood, including capacity management.
Business Control Over the Application Lifecycle
When we first adopted the OpenShift Container Platform, a full-stack PaaS engineer had to help application teams containerize their builds to run on OpenShift (as shown in Figure 1). As a result, PaaS engineers became a bottleneck. They spent about 60% of their time helping applications teams plug into our CI pipeline. They could deploy no more than 2-3 applications each quarter.
Figure 1: Before OpenShift, our platform team had full responsibility for application onboarding
OpenShift Container Platform standards make it easier to share onboarding responsibilities with business units so that they don’t have to wait for a PaaS engineer (as shown in Figure 2). Developers themselves plug their code into our CI/CD pipeline. They have the same deployment target, OpenShift, regardless of environment or data center location. Business units like self-service onboarding because their ideas get into production more quickly.
Our IT team continues to validate security at the appropriate checkpoints and to make sure business units follow our change-control and deployment processes. All of these activities are built into the CI/CD pipeline. Business units that want to manage change control and deployment themselves can do so by using the CI/CD pipeline.
Figure 2: Today the Rapid Digital Services team and business units share responsibility for onboarding
Increased Delivery Capacity
The combination of OpenShift, the CI/CD pipeline, and our cultural and organizational changes helped us accelerate service delivery in a big way. No longer are we limited to onboarding 2-3 releases per quarter. Instead, application teams can onboard applications at their own pace, using self-service tools. Self-service also shrinks the portion of time that our PaaS team devotes to onboarding, giving them more time to add CI/CD pipeline capabilities and adopt new OpenShift Container Platform capabilities. We’ll spend even less time onboarding when we add more self-service capabilities for business units (as shown in Figure 3).
Figure 3: By the end of FY18, we expect applications teams to be self-sufficient, thanks to self-service and automation
New application development is also much faster. For example, in mid-2017 we built a prototype to replace our old performance and development tool in just six weeks from start to production deployment.
Check back for the next blog post in this series, describing how we’re rethinking our application lifecycle to support PaaS and rapid delivery. The third post 3 will explain our evolving infrastructure, including multiple clouds, and a new self-service consumption model.