A Brief DevOps History: The Roots of Infrastructure as Code
DevOps is full of buzzwords, jargon and abbreviations. DevOps itself has only been around for a little more than a decade, so some of these concepts are relatively new. However, some are quite old, and their definitions and uses have changed over time. Today, I’ll explore the history of Infrastructure as Code. It may seem new, but it’s both older and more historically complex than you might realize.
When someone says, “Infrastructure-as-Code,” your mind probably jumps to tools like Chef or Ansible or Terraform, but its origins are much older than that. It has its roots in configuration management, and most configuration management tools use some sort of Infrastructure-as-Code approach. It turns out that we’ve had trouble managing and configuring fleets of machines since we first decided to let them talk to one another, whether they’re virtual machines or containers on a cloud provider, or physical machines in a lab.
If you dig in and go all the way back to the dawn of modern computing, configuration management tools have been around since the 1970s. Unix “make” was released in 1976, allowing a rudimentary form of configuration management, and PXE boot was introduced in 1981, bringing an early form of configuration management to whole machines.
Both tools are still widely used today, but they don’t jump to mind when most people think of configuration management or Infrastructure as Code. You could easily argue that neither of these tools is true configuration management, but the fact remains that we used these tools to automate configuration to the best of our abilities before we had more sophisticated tooling to do it for us.
One of the oldest open source examples of the modern definition of configuration management is CFEngine, initially released in 1993. It was built by Mark Burgess while he was pursuing a postdoctorate in theoretical physics. Mark had the task of maintaining a series of Unix workstations, all running different versions of Unix and having different problems, which required a lot of time-consuming manual scripting and one-on-one user support.
Abstracting away the differences between those platforms behind a domain-specific language (DSL) allowed him to significantly reduce the workload involved in maintaining them, and CFEngine1 was born. This style of configuration management served us well for a long time, and CFEngine is generally regarded as the earliest fully formed origin of today’s tools.
Figure 1
So why does Infrastructure as Code seem so new if configuration management is so old? Did the complexity that made configuration management useful not exist outside of academia and enterprises, or what? No, but like many advancements in technology, a need to handle greater and greater complexity at scale has driven the evolution of Infrastructure as Code.
Prior to the ubiquity of cloud computing, provisioning computing resources meant acquiring new physical infrastructure. Utility computing, or the “pay for what you use” model, wasn’t widely available yet, so scaling to infinity wasn’t nearly as easy as it is today. But the threshold for what constitutes “at scale” changes and grows over time as technologies evolve, and in 2006, AWS released the first version of EC2.
Rapidly, scalability became everyone’s problem. EC2 made it easy for anyone to take up the exact amount of computing resources they needed, but tools for managing that infrastructure hadn’t kept up with the rapidly growing complexity of applications built for this environment.
Manually provisioning and managing instructions for potentially hundreds of different environments is slow and unreliable, so this led to the introduction of a new class of tools for configuring and managing your infrastructure.
In tandem, Puppet and Chef came to light. Puppet was released in 2005. It uses its own declarative domain-specific language to define the resources and state for a system. Puppet accomplished a lot of the same goals that CFEngine did, although in a different language, but drastically reduced the learning curve.
In 2009, we got Chef. Unlike Puppet, Chef was built around a Ruby DSL. While this means that you have access to a language that’s close to full-fledged Ruby, it could be less friendly for folks who come from an IT Ops background rather than a programming background. For a lot of users, the difference between these two tools comes down to whether or not you are more comfortable writing configuration files or something that looks more like a program.
In 2012, Ansible was released. Like CFEngine and Puppet, it uses a declarative, domain-specific language. However, unlike CFEngine, Puppet, and Chef, Ansible is agentless, meaning there is no piece of Ansible installed or running on the machine it controls. Instead, it works via a temporary SSH connection, which is pretty cool.
Figure 2
These tools were each revolutionary in their own ways, and they dominated the market for years. But we tend to think of them mostly as configuration management tools, even though they do some of the same jobs as modern Infrastructure-as-Code tools and can provision infrastructure for us in a code-like way. So where IS the line, and how blurry is it?
Initially, I wanted to say that it’s the difference between managing resources and state for an application rather than for an entire machine, but that isn’t entirely clear. Tools for configuring machines and operating systems rather than applications do exist in classes of their own under the umbrella of “configuration management.”
So instead, my opinion is this: Configuration management exists on its own with these aforementioned tools, and also exists as a part of the greater concept of Infrastructure as Code.
Maybe it’s more accurate to say that Infrastructure as Code is the natural evolution of configuration management. Therefore, how did this inevitability happen?
The growing popularity of container technologies and Docker’s introduction in 2013 caused complexity to increase radically once again, allowing for much more sophisticated solutions and architectures but causing new pain points with respect to scalability.
This is also right around when the concept of developing applications as a collection of microservices rather than monoliths really began to take off, a design paradigm that Docker is well-suited to handle. Suddenly, we need smarter tools built for this type of work.
In 2014, the same year Kubernetes was introduced, Terraform also launched. Like most of its predecessors, it provides broad infrastructure configuration management with a domain-specific language; but unlike its predecessors, it was designed primarily for the management IT operational workflows, which are now more diverse and complicated than ever. Terraform has become wildly popular and continues as a result of the growing community and expansion into the cloud.
We’ve come a very long way so far, and each of these tools has strengths (usually reflected by the time and circumstances in which they were built), but IT Ops teams and site reliability engineers are still left with an outsized responsibility for their infrastructure.
Meanwhile, software engineers are given tools that use DSLs they don’t know, and increasingly face situations that would benefit from the expressiveness of a programming language without a real way to handle that. In fact, outcrops like Pulumi start approaching tools from a programming language capability rather than DSL.
Each step along this tooling timeline has gotten us closer to the DevOps ideal. I’ve skipped over some tools here and there for simplicity’s sake, such as Pulumi, Nix or SaltStack, but suffice it to say that the most detailed timeline of the evolution from make files to full-fledged IaC is much longer than this long blog post.
Infrastructure as Code feels shiny and new, but like most things in computing, it’s actually quite old — we’re just coming at the same problems from different angles to work around the circumstances of the day and build abstraction layers for the ones we’ve already solved.
