I work on Terrateam[0], an open source IaC orchesterator. In our opinion, workspaces in Terraform/Tofu kind of suck. The problem is: multiple environments are never actually the same. Workspaces are built under the premise that the differences are small enough to encode in some conditionals, but that just doesn't scale well.<p>What we recommend people do is use modules to encapsulate the shape of your infrastructure and parameterize it. Then have each environment be a directory which instantiates the module (or modules).<p>This is more robust for a few reasons:<p>1. In most cases, as you scale, differences between environments will grow, with this approach you don't have to make a single root module act like a bunch of root modules via variables and conditionals, instead each environment is its own root module and if you need to do something unique in a particular environment, you can just implement that in the appropriate root module.<p>2. It's easier to see what environments are under management by inspecting the layout of the repository. With workspaces, you need to understand how whatever tooling you are using is executed because that is where the environments will be expressed.<p>Last weekend I also implemented what I call "Terralith" which is a proof-of-concept for how to treat a single root module as multiple environments in a principled way. I wrote a blog about the experience if anyone is interested: <a href="https://pid1.dev/posts/terralith/" rel="nofollow">https://pid1.dev/posts/terralith/</a><p>[0] <a href="https://github.com/terrateamio/terrateam">https://github.com/terrateamio/terrateam</a>
The terraform documentation explicitly advises you NOT to do this (<a href="https://developer.hashicorp.com/terraform/cli/workspaces#when-not-to-use-multiple-workspaces" rel="nofollow">https://developer.hashicorp.com/terraform/cli/workspaces#whe...</a>)<p>> In particular, organizations commonly want to create a strong separation between multiple deployments of the same infrastructure serving different development stages or different internal teams. In this case, the backend for each deployment often has different credentials and access controls. CLI workspaces within a working directory use the same backend, so they are not a suitable isolation mechanism for this scenario.<p>For a practical scenario, you will often need different environments to roll out changes at different times, or to have other slight variance. If you rely solely on variables to be the only difference between environments, you will need a lot of tricky shenanigans to say, create a new dynamodb for a proof of concept only in “dev” but not in prod. Sure, you can use `count = var.env == “dev” ? 1 : 0` but this gets old fast.<p>Much better to make modules for common stuff, and then compose them in your different environments. Depending on the complexity of your needs, keeping good organization & practice around using modules can be a bit challenging, but it will definitely scale through composition.<p>Modules also important to make multiple copies of things within an environment, for example to have a cluster in us-west-2 and a cluster in eu-central-1, both are in production environment. I would assume if I started with workspaces I would rapidly hit a point where I want to use it as a module and then need to re-organize things. If you chose workspaces as soon as you want a second region you need a big refactoring, but if you are using modules, you just instantiate the module a second time in your second region.
Is there any benefit to using workspaces over just introducing some variables and having an 'environment' variable?<p>You can have a directory per environment and a directory of shared resources that are used by all environments.<p>It seems like workspaces add a new construct to be learned and another thing to add to all commands without much benefit. Could we just stick with the simple way of doing this?
The Terraformer tool was the biggest blessing when I had to reverse engineer our AWS stack into .tf modules.<p>Shoutout to the Waze team for creating it!<p><a href="https://github.com/GoogleCloudPlatform/terraformer">https://github.com/GoogleCloudPlatform/terraformer</a><p>We built out a large serverless stack on AWS, and we got a request from higher ups to convert it all into Terraform modules for portability and transparency purposes.<p>The Terraformer tool pulled in the entire stack and spit out the whole thing into tf files in less than 30 seconds.<p>Everyone was super impressed on the team lol.
I generally consider the AWS CLI configuration to be something that's unique to a developer's workstation, and shouldn't be referenced in terraform code (in the form of tying the workspace name to your AWS profile name).<p>This would only work if all developers on a team have synchronised the same AWS CLI config (which to me is like asking people to synchronise dotfiles, not something I'd be willing to do).<p>My go-to architecture for multi-environment tends to be this, as it lends itself relatively well to Git Flow (or GitHub Flow): <a href="https://github.com/antonbabenko/terraform-best-practices/tree/master/examples/medium-terraform">https://github.com/antonbabenko/terraform-best-practices/tre...</a>