After weeks of hard work, you’re taking a new build live. Then disaster strikes.

Everything worked great in the cloud dev environment with test data, and with real data in staging. But in production, it’s chaos. ‘It worked great in staging!’ exclaims one engineer, bewildered. ‘What’s different now?’ Eventually, you find the culprit: a network config mismatch overlooked during manual setup.

Horror stories like this were all too common before Infrastructure as Code (IaC). Maintenance was labor-intensive and error-prone, requiring much preparation and checking (with backups in case things went south). But in the last 10-15 years, the ‘shift left’ strategy has seen more engineers setting up their own infrastructure. To avoid bottlenecks during periods of high demand, IaC tools have emerged to allow less-experienced engineers to adjust infrastructure under expert DevOps guidance.

There are several IaC tools for a variety of tasks, but I’d like to focus on the current go-to solution: Terraform. I recommend avoiding platform-limited tools like (for example) Cloud Formation for AWS. A generic tool like Terraform offers a range of providers including all major cloud and container services like Kubernetes. This flexibility accelerates development, collaboration, and reliability - without binding you to a provider. Plus, you can work with the strong Terraform community and its open-source registry of provider modules, instead of relying on one company.

Benefits of Terraform's IaC:

• Precisely define infrastructure with declarative code. Providers automatically calculate resource dependencies and create and destroy them in a reliable order - eliminating 'it worked on the other environment' syndrome.
• Use a state file as a single source of truth. Configure the remote storage (e.g. S3) to enable collaboration and prevent conflicts through version control and pull request reviews. You can even test for issues before deploying with tools like tftest. This eliminates knowledge silos, as the config is accessible to everyone (even without direct environment access).
• Create modules as abstractions for multiple connected resources, so infrastructure can be clearly described in terms of structure rather than physical hardware - great for scalability. Compliant, secure modules can be written by an expert - or pulled from the Terraform Registry - and then reused everywhere, enforcing security by design.
• Reduce costs by reducing manual overhead. Clarify what is running across environments, and swiftly destroy unused instances. Tools like InfraCost can even calculate the cost of changes from a pull request before deployment.

It's essential, but the transition to IaC has a learning curve. Make sure to invest in training from day one, including sufficient documentation. With the right approach, switching from manual infrastructure config to automated IaC promises to save you from the horror of late-night deployment woes - and facilitate scalable, agile growth.