What is it?
Infrastructure as code is a way to describe infrastructure in a language that is stored as a text file just like code.
This text will then be read by specific tools (like interpreters or compilers read code) to produce an effect. This effect can be the creation/destruction of a resource or its provisioning. In the rest of this article we will focus on the problems infra-as-code (IaC) solves and the benefits it can bring to both managers and DevOps teams.
What problems does IaC solve?
Before focusing on infrastructure as code, it is important to understand how the context changed in terms of infrastructure management.
A user would open a ticket asking for a given ressource. An Ops would grant this resource and close the ticket.
This workflow worked for a long time because the infrastructure churn was considered low: people rarely changed their infrastructure, and infrastructure itself doesn't change every day.
The installation of software was typically done manually and/or using semi-automated scripts that worked on a specific context.
This workflow raised several issues:
- Scalability issues: The amount of time and issues grow as the number of resources is managed in an ad-hoc way.
- No single source of truth: It makes it hard to understand what production is made of and it is also hard to recreate a copy for a pre-prod.
- No Standard tools: Internal scripts cannot easily be reused in different contexts. It requires additional training for internal tools and their specificities.
This context shifted swiftly in the past years. Now churn is raising as projects rise and fall faster than ever. Standard tools are available and widespread (we will talk about them in a later article). Finally, Ops work is shifting towards a different kind of work where automation handles a larger part of the tasks.
Describing Resources & Configuration as Code
Infra-as-code includes two types of descriptions:
The description of the infrastructure resources hosting an application: virtual machines, baremetal machines, configuration of security groups, volumes, etc.
The description of the configuration of the services deployed and hosted on this underlying infrastructure (or software environment): installed packages, configuration files, libraries on which the application depends, etc.
Why should we use code to track infrastructure?
Classical code workflow applied to cloud infrastructure
The first advantage to use code to describe infrastructure is to use the same kind of tooling that we use to write code.
A user can reuse the same editor used to write code, no additional tools are necessary for this.
In addition to reusing the same editor, we can also reuse the same versionning system to track changes brought to the infrastructure.
Once a versioning system is in place, we can centralize all changes to a single point. For instance, we can use GitHub or GitLab to centralize all the code in a single point. This repository creates a single source of truth. This centralization is useful to track all the changes that were brought to the infrastructure and having commit messages that help to understand why they happened.
Paradigm shift: From imperative to declarative
In mathematics and computer science, idempotence means that an operation has the same effect as it is applied once or several times. For example, the absolute value is idempotent abs(abs(-42)) = abs(-42) = 42.
Idempotency is a result. You can see idempotency as a light switch. If it is already on and you turn it on you don't change anything.
In our case, we want the the actions of our infra-as-code tool to be as idempotent as possible: we therefore prefer, for example, to "ensure the presence of a line in a file / ensure the existence of a virtual machine" rather than "add a line in a file / add a virtual machine"; the idea of being to be able to launch our tool in a loop without the state changing in the absence of changes in the code describing the infrastructure.
This notion will be crucial to understand some of the choices and limitations of the solutions we will offer you.
In short: we do not describe actions; we describe a desired state, then our tool will infer the actions to be performed in order to converge towards this state.
I'm a Dev / Ops. Why should I care?
Simplify collaboration between Dev and Ops
When this pattern of version tracking is enforced, the repository that contains this code becomes a Single Source of truth.
Having this unique point of control on the infrastructure helps a great deal to introspect and discover how the infrastructure is built.
Infra-as-code will become a key area of collaboration for developers and operational staff.
What could be easier than sending a Pull Request rather than a ticket or an e-mail when you want to change something?
What better way to collaborate than on code, which will be executed as written? The idea is to deliver code, whether it's for the application or the infrastructure, and automation does the rest!
This is why this infra-as-code movement often comes from both developers and system administrators: the former see it as a way to take a certain independence while the latter find a way to simplify their daily lives with repetitive and often low added value tasks. But let's remember: the objective of the DevOps movement is to align teams with business objectives by improving collaboration between Developers and Operations. It is therefore crucial that these populations do not each automate "in their own corner", but work together to develop a common infra-as-code as collective property.
Pet vs Cattle
Traditionally, servers can be compared to pets. Often, the team responsible for the internal infrastructure even gives them nicknames. These teams take care of these servers on a daily basis as we would with pets, hence these nicknames. In case those "pet" servers go down, it is very difficult to rebuild them from scratch because the internal configuration is not well tracked. Therefore, those servers are fragile and as time goes on they become a liability.
The modern approach is more oriented towards cattle management. The objective is to manage the infrastructure like a herd: your servers are simply assigned a number, and you manage them as a whole. It is not sad to see them disappear, and if one gets "sick" you simply replace them. This "replacement" in a more concrete context would mean a pure replacement of a virtual machine or the re-provisioning of the physical server concerned.
Assume that each server should be able to be assigned a random and unique identifier. However, this does not exclude having some differentiations! Eventually, these servers will be assigned roles or labels based on their capabilities, such as the presence of GPUs.
Is infrastructure as code platform dependent? In certain regards yes and in other no. For the provisioning/configuration part, it is dependent on the software platform and not the service provider. An Ubuntu machine will work the same way in two different vendors. So the dependency will be on the operating system: location of files, commands used... In many cases, automation tools can manage this and abstract it away for you. If they don't provide an abstraction you can always use conditional execution to run only subset of a given program that target a specific platform.
For the infrastructure part, the answer is mostly no. If your resources are configured for a given provider you cannot copy and paste it directly to another provider.
Some resources will have some attributes that are not exactly the same for another provider. That being said, because the language used to describe the resources is the same, it will be easy to understand exactly what need to be migrated.
I am a manager, Why should I care?
Infrastructure as code does not only yield advantages for Devops! Managers can also benefit from Infrastructure as Code.
Align incentives between Dev and Ops
When Infrastructure as Code is enforced, development and operational teams are using the same source repository. As a result both those teams can review and apply changes to them. This lowers the risk of misunderstanding between teams because they all share the same context and see the same files.
Easier to grasp the infrastructure and contribute to it
As a corrollary, it is easier to grasp the infrastructure if it is in a written format. This allow anybody that got access to it to start reading and understand how the infrastructure and provisioning is happening. It becomes then easier to train many people and by doing so, reduce the risk to depend on a single person to perform a task.
Easier to test a complete deployment using CI/CD
Having a single CI/CD allows you to be able to recreate a copy of the same infrastructure. This is useful for instance when you want to have a single template for a particular purpose. For instance, having a copy of the same infrastructure for each client.
It also helps you to test if a given release of your software project will work on a given architecture. You can have a build that will create a copy of your infra, deploy the code on it and ensure that the code is running normally.
Infrastructure as code is a new way to express how to provision and configure infrastructure using code. It is also a paradigm shift towards a declarative way of describing infrastructure and using tools to make them happen.
In the next articles we will see how we use such tools at Scaleway for our internal needs and how you can also start managing your infrastructure using these tools.