Terraform and AWS

At a previous employer I used Cloud Formation Templates (CFT) to create infrastructure for both a batch server and Neo4J database cluster.  This is just a bit of a mild comparison - the overall environment you are in makes as much or more of a difference in how well infrastructure provisioning works than the the base technology itself.

Cloud Formation Templates worked ok but provided some quirks. Additionally, the organization made it difficult in some ways by adding a number of layers of processing around the CFT processing.  You never directly processed any CFT with AWS but instead had to run jobs which used centralized services to process the CFT and other associated provisioning tasks.  

Now I am in an organization where Terraform is promoted but there are no centralized services. Each team pretty much "rolls their own" for any needs. It is odd being at the total opposite side of the spectrum. 

Since I intended this it focus more on Terraform than CFT, let me start with features I like (using version .11).

• The JSON type format is decent - I prefer it over YAML. 
• The distinction between a "resource" for creation and a "data source" for look-ups is useful. 
• There are a decent number of functions available for various needs.
• Modules are useful.
• Work spaces are useful.
• Being able to see a "plan" is nice even if it isn't always able to provide everything you hoped.
• Template files are especially useful when you need to produce some JSON to be consumed by another resource.
• There are a decent number of other resources on the internet that help with ways to handle "odd" things or promote good/best practices.

Some of the items that bug me are:

• The documentation isn't as good as AWS in general.
• Some items are not quite a direct port from the AWS equivalent and when the documentation doesn't clarify things it gets painful. It is even worse when the AWS documentation isn't totally clear.
• Basically the handling of data types could be improved. Having places where I end up with conversions something like List -> string -> List because Terraform is confused about what was passed to a module.
• The amount of duplication to convey parameters from the top-level through a number of modules - it just seems overly wordy.
• Terraform enterprise isn't cheap enough (aka free) - I don't control the purse strings so nothing I can do about that.
• State handling is even more awkward when I am working on branches for the terraform scripts. 
• Various flow control constructs could be better - it looks like Terraform .12 will include a number of enhancements.

I know my "bugs me" list is longish but in general Terraform is getting the job done.  I've been using it for a few months now and I know there are still some features I've not used as of yet.  There is a little bit of a learning curve but if you've done CFT then it isn't a hard transition. We've got some teams using the Unix 'make' utility to provide some additional capabilities that are hard to represent directly in Terraform - I'm inclined to do the same soon.

I know that the tool tends to get promoted as cross platform but since the various entities tend to be near 1-1 mappings from the target cloud provider you won't get reuse out of the infrastructure definitions themselves. You can work with multiple cloud providers concurrently with a single tool though.

Breaking up terraform items into reasonably small modules and putting them into distinct repositories seems like a best practice at this point.  I started towards a monolithic "build it all" type setup (but was using modules) but I think more "islands" of infrastructure which when coordinated will produce a "sum greater than the whole".  Sort of like composition in functional languages compared to imperative language development. 

I think given some effort, terraform can provide some benefits above what CFT provides but you will likely end up using some additional utilities to get the most of it.

As is, I don't think I am spending as much time debugging terraform scripts as I did CFT but that may be more about the overall environment.  It has been an interesting learning opportunity if nothing else.

[edit 2019/04/28] This isn't a terraform issue but I do want to mention that not following the general AWS recommended guidelines for VPC setup related to public/private subnets will more than likely result in a lot of headaches.  That situation wasn't my choice and I am still trying to get it changed.

Thanks for reading..
Scott

Psalm 19:2 (NASB)
Day to day pours forth speech,
            And night to night reveals knowledge.