What is DevOps?

What is DevOps?

These days, it's hard not to encounter the term "DevOps." At first glance, DevOps is a fairly simple idea. After all, isn't the term just a fusion of "Dev" and "Ops"?

At its core, just like the term itself, DevOps unites traditionally distinct functional areas. It's about "bridging the gap" or "eliminating silos between" software development ("Dev") and IT operations ("Ops") teams, often aiming to deliver software more rapidly and with increased stability.

But since its introduction, DevOps has become a catchall buzzword for every trend in software development and IT operations. It’s understandable, given that DevOps is still evolving, encompasses many areas, and is adapted and adopted based on an organization’s specific business objectives, priorities, and existing knowledge base.

What DevOps is NOT

People (A Role, Team, or Re-Org)

Merely experimenting with personnel changes or team structure does not equate to DevOps. Some common attempts at implementing DevOps with a focus on people include:

• Hiring DevOps engineers to tick the “Doing DevOps” box

• Creating a third silo between Dev and Ops, naming it the DevOps team

• “Tearing down the wall” either literally (through team co-location) or figuratively (via organizational changes) to remove silos between software development and IT Operations teams

Organizations that treat DevOps as something achievable through a re-org or hiring initiative only consider one piece of the DevOps puzzle. Team structure and roles are secondary to the underlying collaborative practices at work.

Process (or Lack Thereof)

You can’t talk about DevOps without mentioning process. Many people might perceive DevOps as a complete departure from process or at least a departure from the historically visible by-products of a process-oriented organization, such as extensive documentation and review-heavy “gates” within the SDLC.

For example, two key movements that inspired DevOps—Agile software development and Continuous Delivery—deviate from the increasingly outdated Waterfall software development model. But they do not throw process to the wind, as some might assume.

Instead, they reimagine process using methods like Scrum to facilitate team communication and flow of work and implementing CI/CD automated processes and tooling to continuously integrate code changes and deploy applications to production on-demand. What feels like a departure from process is merely the continual refining of a process to reduce the reliance on manual efforts and time spent creating documentation to enable asynchronous cross-team collaboration instead of doing it face-to-face every day.

DevOps Tools

Google “What is DevOps?” and you’ll also quickly find that most technology companies that deliver automated tooling (Infrastructure as Code, Microservices, Containerization, Cloud services, etc.) in the DevOps space present their own definitions of DevOps with a focus on their contributions to the market. For example:

• Amazon’s take is (not surprisingly) rooted deeply in their cloud-based Infrastructure as a Service offering.

• GitLab’s take is all about driving a common toolset to optimize collaboration between development and operations.

• Microsoft’s take also speaks to many areas (Infrastructure as Code, Microservices, and Monitoring) in which they deliver solutions.

• New Relic’s take is centered on monitoring and measurement.

These examples are useful but underscore our assertion that DevOps still means different things to different people (or organizations). DevOps is interpreted and implemented differently, depending on the organization and its core values and competencies. Tools that enable IT automation are a key aspect of making DevOps work in practice. Still, if the underlying principles and processes aren’t a part of the equation, tools alone won’t bring your organization to the desired high-performing outcomes.

What DevOps IS

DevOps is not any one of the items listed above (People, Process, or Tools), but it’s often confused as one or more of the three cobbled together. They are all essential parts of an organization’s adoption of DevOps, but they won’t do the job independently.

So what is DevOps exactly?

The Wikipedia entry for DevOps is really quite telling:

“DevOps is a set of software development practices that combines software development (Dev) and information technology operations (Ops) to shorten the systems development life cycle while delivering features, fixes, and updates frequently in close alignment with business objectives.”

That’s a pretty good outcome-focused definition, but what’s most interesting about the Wikipedia entry is that the next section quickly admits its sources still need verification and that there is no one standard, agreed-upon definition of the term:

”Academics and practitioners have not developed a unique definition for the term ‘DevOps’.”

A Set of Principles

The closest thing we have to an agreement between academics and practitioners in defining DevOps is chronicled in The DevOps Handbook, written by some of the movement’s thought leaders and pioneers. Their definition is rooted in the theoretical and historical underpinnings of DevOps and the principles upon which it is built. But although the definition is certainly comprehensive, it may not be the most memorable simply because of its length:

“DevOps is the outcome of applying the most trusted principles from the domain of physical manufacturing and leadership to the IT value stream. DevOps relies on bodies of knowledge from Lean, Theory of Constraints, the Toyota Production System, resilience engineering, learning organizations, safety culture, human factors, and many others. Other valuable contexts that DevOps draws from include high-trust management cultures, servant leadership, and organizational change management. The result is world-class reliability, stability, and security at ever lower cost and effort; and accelerated flow and reliability through the technology value stream, including Product Management, Development, QA, IT Operations, and Infosec.”

The key takeaway from this definition is understanding the depth and breadth of the bodies of work that inform DevOps, focusing on applying these trusted principles to achieve desired outcomes within the technology value stream quickly. (We’ll discuss these principles in greater detail in a later article.)

A Culture (of Learning and Continuous Improvement)

When you consider all of the aspects of DevOps mentioned up to this point (People, Process, Tools, and Principles), you’re 80% “there” in understanding what DevOps encompasses.

But the remaining 20% can make or break a DevOps approach within an organization because without it, the important “fail fast, learn, and try again” mantra won’t feel safe or obtainable.

Several keywords within The DevOps Handbook definition above deserve further emphasis and commentary to round out our definition:

• Learning organizations

• Safety culture

• High-trust management cultures

• Servant leadership

In Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations, the authors discuss the need for a generative culture (a term coined by Westrum), which focuses on the mission and optimizing the flow of information across teams, rather than a pathological, command-and-control, power-centered mindset:

“…in organizations with a generative culture, people collaborate more effectively and there is a higher level of trust both across the organization and up and down the hierarchy.”

A generative organizational culture devoted to this type of continuous learning and improvement also provides an important distinction in their approach to DevOps adoption, as Steve Mezak noted:

“It’s reasonable to describe DevOps as a journey, or perhaps an aspiration, rather than defined destination. DevOps, like lean manufacturing, seeks continuous improvement, seeking higher output, greater efficiencies and even continuous deployment. Automated tools that support DevOps continue to evolve.”

When you build the people, processes, and tools on top of foundational DevOps principles and a culture committed to learning and continuous improvement, you’re well on your way to understanding DevOps from a holistic perspective.

Therefore, the definition of DevOps we propose is the following:

“DevOps is a discipline rooted in collaboration and communication, made possible by removing perceived barriers between teams and building trust in a culture of learning and continuous improvement, and drawing from proven technical and management practices that work toward a common goal of shortening software delivery cycles and improving the stability of deployments.”

Why DevOps?

The definition touches on all of the key points of DevOps. Still, without all of its constituent parts—the People, Processes, Tools, Principles, and Culture—you’re just dabbling in DevOps, instead of genuinely embracing its mission.

A Short History of DevOps: What Problems Does DevOps Try to Solve?

It is the culmination of many events, from the Agile and Lean movements to practitioners and thought leaders trying to solve the recurring problem of painful software deployments, all while development teams became increasingly tasked with accelerating innovation through more frequent releases.

The term “DevOps” was coined by Patrick Debois, an IT consultant and one of the authors of The DevOps Handbook, in 2009.

In The Origin of DevOps: What in a Name? Steve Mezak describes this “perfect storm of events” that led to its inception, including a frustrating project where Debois was tasked with straddling the line between application development and IT operations to accomplish certification/readiness testing.

Those influences and experiences, coupled with inspiration from John Allspaw and Paul Hammond’s now-famous Velocity conference talk 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr (which he wasn’t able to attend in person), led Debois to create the DevOps Days conference in Ghent later in 2009.

Thus, “DevOps” was born.

The Lessons of Lean: Optimizing the Value Stream

As mentioned earlier, the Lean and Agile movements were some of the key influencers of DevOps. Although different sources vary on when Lean Manufacturing began, the Lean movement's principles gained traction in the early 1990s, after the publication of The Machine that Changed the World.

Lean grew out of the need to deliver value to customers faster by identifying and removing waste from a manufacturing process, or (put another way) by reducing lead times through optimization of the value stream.

In terms of manufacturing, where the flow of work is visible, The DevOps Handbook describes a value stream as the following:

“In manufacturing operations, the value stream is often easy to see and observe: it starts when a customer order is received and the raw materials are released onto the plant floor.”

Although it can be more difficult to identify the flow of work in the software delivery value stream because it isn’t observable from a bird’s-eye view, many organizations have adopted these same techniques to carry them through a comparable technology value stream, summarized by The DevOps Handbook as “the process required to convert a business hypothesis into a technology-enabled service that delivers value to the customer.”

The challenge of adapting Lean principles to IT Operations is illustrated in The Phoenix Project, where Bill, Director of IT Operations, finds himself one day overlooking his company's manufacturing plant floor. He’s just met with Erik, a potential Board member candidate. After briefly discussing the recurring issues plaguing the IT team, Erik urges Bill to follow him to the plant floor to observe the flow of work, though Bill is puzzled as to why.

Over time, Erik continues to take Bill under his wing, introducing him to The Three Ways—Flow, Feedback, and Continuous Learning and Experimentation—in the context of a manufacturing environment. He then waits, sometimes impatiently, for Bill to connect the dots between the principles and how they can be applied to his own flailing team’s issues.

Whether it’s being applied to processes on the manufacturing floor or from the workstations of IT and Development folk, the principles of lean are much the same (The DevOps Handbook):

“To enable fast and predictable lead times in any value stream, there is usually a relentless focus on creating a smooth and even flow of work, using techniques such as small batch sizes, reducing work in process (WIP), preventing rework to ensure we don’t pass defects to downstream work centers, and constantly optimizing our system toward our global goals.”

The Advent of Agile: Accelerating Software Releases

The advent of agile software development processes in the early 2000s was necessitated by software teams’ inability to continue to deliver software at the speed consumers demanded as PC computing proliferated in the 1990s.

In the TechBeacon article To Agility and Beyond: The History—and Legacy—of Agile Development, author Peter Varhol explains that organizations in the 1990s using traditional waterfall software development methodology had typical development cycles spanning 3 years, with certain industries (e.g., aerospace, government) having even longer cycles. He continues:

“The problem was, businesses moved faster than that, even 25 years ago. Within the space of three years, requirements, systems, and even entire businesses were likely to change. That meant that many projects ended up being cancelled partway through, and many of those that were completed didn’t meet all the business’s current needs, even if the project’s original objectives were met.”

Software development thought leaders understood the need for “lightweight” principles to replace process-heavy waterfall development methodologies in order to release usable software to customers more frequently.

Movements such as extreme programming, rapid application development (RAD), and iterative software development were all precursors to Agile that shared many of the same goals: getting working software into the hands of users more quickly, getting rapid feedback, and continuing to iterate to meet changing business requirements and user needs.

In 2001, the Agile Manifesto was solidified, including 12 principles that focus on customer satisfaction, effective collaboration, and continuous delivery.

The Business Case for DevOps

In our current “software is eating the world” climate where much of the innovation across industries is software-driven, Nicole Forsgren, et al., in Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations assert that how well an organization is able to deliver software to their customers is integral to how well an organization performs in “profitability, productivity, and market share.”

In fact, the Accelerate authors seemed almost surprised by the demonstrable value they surfaced:

“The findings from our research program show clearly that the value of adopting DevOps is even larger than we had initially thought, and the gap between high and low performers continues to grow.”

Their research found that high-performing DevOps teams when compared to low performers, have:

• 46 times more frequent code deployments

• 440 times faster lead times from committing to deploy

• 170 times faster mean time to recover from downtime

• times lower change failure rate (⅕ as likely for a system change to fail)

These four key metrics span the entire DevOps pipeline, revealing that both accelerated development and operations stability are integral measures in improved software delivery performance—and are not at odds with one another.

The Accelerate authors explain further:

“Astonishingly, these results demonstrate that there is no trade-off between improving performance and achieving higher levels of stability and quality. Rather, high performers do better at all of these measures. This is precisely what the Agile and Lean movements predict, but much dogma in our industry still rests on the false assumption that moving faster means trading off against other performance goals, rather than enabling and reinforcing them.”

With the trade-off assumption busted, and the gap between high and low performers in DevOps ever-widening, the case for DevOps adoption has never been stronger.