The Most Important DevOps Metric to Measure

Over the past few weeks, I've had the great privilege to partner on a series of roadshows with Gene Kim (@realgenekim) author of "The Phoenix Project", "The DevOps Handbook" and the annual "State of DevOps Report". The events are called "Culture, Containers and accelerating DevOps, the path to Digital Transformation" and they provide us with an opportunity to speak with developers, enterprise architects, IT operations engineers and business executives about how they are implementing technology and culture changes to help them deliver software faster into their markets.

During Gene's presentation, he highlights a series of lessons that he's learned since writing The Phoenix Project. Some of these include:

1. The Business Value of DevOps is higher than expected.
2. DevOps is as Good for Ops, as it is for Devs.
3. The Importance of Measuring Code Deployment Lead Times.
4. The Surprising Implications of Conway's Law. (organizational structure)
5. DevOps is for the Unicorns, and the Horses too.

The lessons are supported by a series of stories, examples and data from businesses that Gene has interacted with over the past 4-5 years, as they navigate their DevOps journey.

What's the Most Important DevOps Metric?

At some point in every event, someone from the audience will ask the question, "If you had to boil it down to a single thing, what is the most important DevOps metric for us to track?" Gene's answer is often the topic #3 (above), focused on measuring code lead times. It comes from his experience studying the Toyota Production System and it's approach to flows-of-work, managing and correcting defects, and empowering employees to make the on-going changes needed to improve production of the end product. In essence, he highlights that today's data centers have become 21st-century bits factories, with the goal of producing high-quality, agile software applications.

For the most part, I'd agree with Gene that this is an important metric to track. But with all due respect to his expertise in this area, I actually believe that there is a better metric to track. And Gene actually calls out this concept in his talk: An Organization's Confidence Level of Shipping Software into Production. 

In my opinion, this concept is more appropriate than any specific metric, because it forces the technology team to think about their actions in business terms. It also allows them to have a conversation with the business leaders in a way that is focused on impact to customers. It directly aligns to every company becoming a software company, and the importance of making software "go to market" becoming a core competency of the business.

It allows the technology team to begin with the end in mind, and work backwards from the goal of safely shipping software into production. Their level of confidence in the end goal will also force them to consider why their current confidence level may not be the highest it could possibly be.

• Are we investing (people, technology, partnerships) for success towards the goal?
• Are we designing our software, our systems and our platforms to handle the emerging needs of the business?
• Are we enabling a culture and set of systems that allow us to learn from our mistakes, and make improvements when needed?

When I think about this concept, I'm encouraged by the level of confidence from John Rzeszotarski - SVP, Director of Continuous Delivery and Feedback - at KeyBank. 

At Red Hat Summit, John talked about the DevOps journey at KeyBank and how they focused on culture, continuous integration pipelines, automation, containers and their deployment platform (Red Hat OpenShift Container Platform). This was a 12-18 month journey, but where they are today is pretty remarkable. He summed up his talk by telling a story about how they recently re-launched services from one of the banks they had acquired. The highlight was that they were able to deploy 10 new updates to the production application, with 0 defects, during the middle of the day. That is a very high level of confidence in shipping software into production, and in the elements that make up their DevOps culture.

The KeyBank story is a great example of making a significant impact to the business, and measuring the technology in terms of business success and agility.

NOTE: Attached are my slides from the Culture, Containers and Accelerating DevOps roadshows.